2024-09-21T06:47:47.024 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T06:47:47.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:47:47.157 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913574' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIb99x46UDtdBnjld2yTgn+eGOZFLY8ZV30UWiX6Jj3/lEKAk4zk/+48E9LZkuESzNhLKXr9GCtC6HEoe3RQH8A= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5x5v7WJ5CnR2FiGxhFxlQX0t8BadNjGS0ELyuImgqons2ZEgLhZqSpEHj+BHPgVNsYqQ28DZrZBhEO+II6GMg= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBERGuXhKFb879rv6REb/wF1om2FMyWSjR8X+zUdgZo7ujlA0nr+UvI3/WIFg4lHY/1taIwyc4cF6wQYgaKLH4+s= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T06:47:47.157 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T06:47:47.158 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T06:47:47.158 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T06:47:47.160 INFO:teuthology.task.internal:Checking packages... 2024-09-21T06:47:47.180 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T06:47:47.180 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T06:47:47.180 INFO:teuthology.packaging:ref: None 2024-09-21T06:47:47.181 INFO:teuthology.packaging:tag: None 2024-09-21T06:47:47.181 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:47:47.181 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:47:47.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:47:47.333 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T06:47:47.335 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T06:47:47.341 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T06:47:47.341 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T06:47:47.349 INFO:teuthology.task.internal:Saving configuration 2024-09-21T06:47:47.372 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T06:47:47.378 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T06:47:47.402 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:41:36.659044', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIb99x46UDtdBnjld2yTgn+eGOZFLY8ZV30UWiX6Jj3/lEKAk4zk/+48E9LZkuESzNhLKXr9GCtC6HEoe3RQH8A='} 2024-09-21T06:47:47.427 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:41:36.660296', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5x5v7WJ5CnR2FiGxhFxlQX0t8BadNjGS0ELyuImgqons2ZEgLhZqSpEHj+BHPgVNsYqQ28DZrZBhEO+II6GMg='} 2024-09-21T06:47:47.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:41:36.657033', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBERGuXhKFb879rv6REb/wF1om2FMyWSjR8X+zUdgZo7ujlA0nr+UvI3/WIFg4lHY/1taIwyc4cF6wQYgaKLH4+s='} 2024-09-21T06:47:47.451 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T06:47:47.458 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T06:47:47.458 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T06:47:47.458 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T06:47:47.458 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T06:47:47.537 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d45f08160>, signals=[15]) 2024-09-21T06:47:47.538 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T06:47:47.544 INFO:teuthology.task.internal:Opening connections... 2024-09-21T06:47:47.544 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T06:47:47.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:47:47.620 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-09-21T06:47:47.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:47:47.693 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-09-21T06:47:47.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:47:47.766 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T06:47:47.773 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-09-21T06:47:47.793 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-09-21T06:47:47.793 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-09-21T06:47:47.851 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-09-21T06:47:47.851 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:47:47.852 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:47:47.853 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-09-21T06:47:47.877 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-09-21T06:47:47.895 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-09-21T06:47:47.895 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-09-21T06:47:47.953 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-09-21T06:47:47.953 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-09-21T06:47:47.953 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:47:47.954 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:47:47.955 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-09-21T06:47:47.977 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-09-21T06:47:47.996 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-09-21T06:47:47.996 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:47:48.054 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:47:48.055 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-09-21T06:47:48.085 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T06:47:48.095 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T06:47:48.100 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T06:47:48.101 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:47:48.103 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:47:48.105 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:47:48.122 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T06:47:48.128 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T06:47:48.128 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:47:48.161 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:47:48.163 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:47:48.267 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T06:47:48.337 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T06:47:48.351 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T06:47:48.351 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-21T06:47:48.351 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-21T06:47:48.351 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:47:48.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:47:48.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T06:47:48.536 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:47:48.536 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T06:47:48.537 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-21T06:47:48.555 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-503.el9.x86_64 2024-09-21T06:47:48.556 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:47:48.556 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T06:47:48.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2024-09-21T06:47:52.186 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-21T06:47:55.752 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 20 MB 00:02 2024-09-21T06:48:00.320 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T06:48:02.054 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-21T06:48:02.866 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T06:48:08.514 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-21T06:48:10.581 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-21T06:48:10.590 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-21T06:48:10.590 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-21T06:48:10.590 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-21T06:48:10.590 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 1 Package 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 388 M 2024-09-21T06:48:10.591 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-21T06:48:47.746 INFO:teuthology.orchestra.run.smithi027.stdout:linux-firmware-20240905-146.el9.noarch.rpm 10 MB/s | 388 MB 00:37 2024-09-21T06:48:47.746 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:48:47.746 INFO:teuthology.orchestra.run.smithi027.stdout:Total 10 MB/s | 388 MB 00:37 2024-09-21T06:48:51.017 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-21T06:48:51.108 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-21T06:48:51.108 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-21T06:48:54.550 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-21T06:48:54.550 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-21T06:49:03.930 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-21T06:49:04.598 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:49:04.681 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:49:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:49:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:49:06.035 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:49:06.035 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:49:06.035 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-21T06:49:06.035 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T06:49:06.035 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:49:06.035 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-21T06:49:06.566 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T06:49:06.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:49:06.725 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T06:49:06.726 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T06:49:06.728 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2024-09-21T06:49:13.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21 06:49:13 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T06:49:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2024-09-21 06:49:13-- 2024-09-21T06:49:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 6.3s 2024-09-21T06:49:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-21T06:49:13.035 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T06:49:13.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T06:49:15.200 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2024-09-21T06:49:15.263 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2024-09-21T06:49:15.327 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2024-09-21T06:49:23.668 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T06:50:04.546 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T06:50:04.546 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T06:50:04.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T06:50:04.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T06:50:04.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T06:50:04.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T06:50:04.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T06:50:04.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T06:50:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T06:50:04.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T06:50:04.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T06:50:04.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T06:50:04.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T06:50:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T06:50:04.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T06:50:04.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T06:50:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T06:50:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T06:50:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T06:50:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T06:50:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T06:50:04.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T06:50:04.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T06:50:04.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T06:50:04.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T06:50:04.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T06:50:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T06:50:04.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T06:50:04.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T06:50:04.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T06:50:04.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T06:50:04.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T06:50:04.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T06:50:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T06:50:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T06:50:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T06:50:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T06:50:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T06:50:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T06:50:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T06:50:04.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T06:50:04.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T06:50:04.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T06:50:04.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T06:50:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T06:50:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T06:50:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T06:50:04.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T06:50:04.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T06:50:04.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T06:50:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T06:50:04.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T06:50:04.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T06:50:04.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T06:50:04.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T06:50:04.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T06:50:04.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T06:50:04.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T06:50:04.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T06:50:04.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T06:50:04.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T06:50:04.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T06:50:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T06:50:04.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T06:50:04.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T06:50:04.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T06:50:04.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T06:50:04.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T06:50:04.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T06:50:04.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T06:50:04.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T06:50:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T06:50:04.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T06:50:04.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T06:50:04.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T06:50:04.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T06:50:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T06:50:04.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T06:50:04.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T06:50:04.707 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T06:50:04.708 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T06:50:04.709 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T06:50:04.710 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T06:50:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T06:50:04.712 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T06:50:04.713 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T06:50:04.714 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T06:50:04.715 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T06:50:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T06:50:04.717 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T06:50:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T06:50:04.719 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T06:50:04.720 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T06:50:04.721 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T06:50:04.722 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T06:50:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T06:50:04.724 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T06:50:04.725 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T06:50:04.726 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T06:50:04.727 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T06:50:04.728 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T06:50:04.729 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T06:50:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T06:50:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T06:50:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T06:50:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T06:50:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T06:50:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T06:50:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T06:50:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T06:50:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T06:50:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T06:50:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T06:50:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T06:50:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T06:50:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T06:50:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T06:50:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T06:50:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T06:50:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T06:50:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T06:50:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T06:50:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T06:50:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T06:50:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T06:50:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T06:50:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T06:50:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T06:50:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T06:50:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T06:50:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T06:50:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T06:50:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T06:50:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T06:50:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T06:50:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:50:04.767 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.767 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:for file in vmlinuz System.map config; do 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout: fi 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:done 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T06:50:04.793 INFO:teuthology.orchestra.run.smithi027.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T06:50:04.795 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 82.el9 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8274670 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T06:50:04.871 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:50:04.872 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T06:50:04.873 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T06:50:04.873 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:04.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T06:50:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-09-21T06:50:06.960 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T06:50:06.976 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-09-21T06:50:06.979 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-21T06:50:07.052 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.E4PvC1N2Vd 2024-09-21T06:50:07.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.E4PvC1N2Vd 2024-09-21T06:50:07.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.E4PvC1N2Vd 2024-09-21T06:50:07.259 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.E4PvC1N2Vd is 6KB 2024-09-21T06:50:07.307 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.E4PvC1N2Vd 2024-09-21T06:50:07.322 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T06:50:07.386 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T06:50:07.386 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T06:50:07.387 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T06:50:07.387 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T06:50:07.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:50:07.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-09-21T06:50:37.543 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T06:50:37.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:50:37.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:50:56.111 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T06:51:05.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:51:05.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:51:08.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T06:51:20.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:51:20.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:51:23.247 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T06:51:38.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:51:38.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:51:41.327 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T06:51:59.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:51:59.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:52:17.779 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T06:52:38.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:52:38.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:52:39.155 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-21T06:52:39.729 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:52:39.730 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T06:52:39.730 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-21T06:52:39.765 INFO:teuthology.orchestra.run.smithi027.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:52:39.765 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:52:39.765 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T06:52:39.765 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T06:52:40.767 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:40.767 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T06:52:40.776 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T06:52:40.776 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:52:40.788 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:52:40.790 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:52:40.851 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T06:52:40.863 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T06:52:40.896 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T06:52:40.896 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:52:40.905 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:52:40.908 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:52:41.010 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T06:52:41.016 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T06:52:41.017 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:52:41.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:52:41.034 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:52:41.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:52:41.051 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:52:41.060 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:52:41.077 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:52:41.119 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:52:41.126 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:52:41.136 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:52:41.147 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:52:41.254 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:52:41.314 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:52:41.318 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T06:52:41.326 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T06:52:41.326 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:52:41.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:52:41.338 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:52:41.448 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T06:52:41.457 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T06:52:41.457 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:52:41.466 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:52:41.469 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:52:41.542 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:52:41.646 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:52:41.782 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:52:41.880 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:52:41.940 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T06:52:41.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:52:41.996 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:52:42.037 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:52:42.093 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:52:42.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:52:42.221 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T06:52:42.221 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:52:42.288 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:52:42.327 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:52:42.381 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:52:42.462 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:52:42.507 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T06:52:42.507 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:52:42.573 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-09-21T06:52:42.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-09-21T06:52:42.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-09-21T06:52:42.620 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:52:42.646 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:52:42.653 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:52:43.128 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T06:52:43.135 INFO:teuthology.task.internal:Starting timer... 2024-09-21T06:52:43.135 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T06:52:43.152 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T06:52:43.160 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T06:52:43.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-09-21T06:52:43.408 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-09-21T06:52:43.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-09-21T06:52:43.459 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-09-21T06:52:43.461 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-09-21T06:52:43.517 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-09-21T06:52:43.519 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T06:52:43.519 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-09-21T06:52:43.646 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-09-21T06:52:43.647 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-09-21T06:52:43.675 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-09-21T06:52:43.676 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-09-21T06:52:43.696 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-09-21T06:52:43.696 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2024-09-21T06:52:43.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:52:43.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:52:43.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:52:43.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:52:43.783 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:52:43.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:52:43.812 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T06:52:43.812 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:52:43.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:52:43.914 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:52:43.949 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T06:52:43.958 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T06:52:43.969 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T06:52:43.970 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi119.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T06:57:40.141 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2024-09-21T06:57:40.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:57:40.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:57:40.250 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-21T06:57:40.371 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T06:57:40.371 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-21T06:57:40.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:57:40.448 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-21T06:57:40.533 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-21T06:57:40.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-21T06:57:40.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:57:40.611 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-09-21T06:57:40.695 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-21T06:57:40.696 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T06:57:40.705 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T06:57:40.706 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:57:40.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:57:40.710 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:57:40.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:57:40.713 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:57:40.713 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:57:40.745 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:57:40.761 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:57:40.767 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:57:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:57:40.782 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:57:40.788 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-09-21T06:57:40.800 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-09-21T06:57:40.807 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:57:40.808 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-09-21T06:57:40.814 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:57:40.819 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-09-21T06:57:40.829 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:57:40.834 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:57:40.849 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:57:40.861 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-09-21T06:57:40.886 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:57:40.886 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-09-21T06:57:40.890 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:57:40.890 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-09-21T06:57:40.890 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.891 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.891 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.891 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.905 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:57:40.909 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:57:40.910 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-21T06:57:40.910 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.910 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.910 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.911 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:40.916 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:57:40.947 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:57:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:57:41.048 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:57:41.049 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-21T06:57:41.049 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:41.049 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:41.049 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:41.049 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:57:41.051 INFO:teuthology.run_tasks:Running task install... 2024-09-21T06:57:41.059 DEBUG:teuthology.task.install:project ceph 2024-09-21T06:57:41.059 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T06:57:41.059 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T06:57:41.059 INFO:teuthology.task.install:Using flavor: default 2024-09-21T06:57:41.064 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T06:57:41.065 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T06:57:41.065 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:57:41.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:57:41.067 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:57:41.067 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:57:41.068 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:57:41.068 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:57:41.210 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:57:41.210 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:57:41.223 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:57:41.223 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:57:41.226 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:57:41.226 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:57:41.317 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:57:41.318 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T06:57:41.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:57:41.331 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:57:41.331 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T06:57:41.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:57:41.338 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:57:41.338 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T06:57:41.338 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:57:41.358 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:57:41.358 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:57:41.364 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:57:41.364 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:57:41.376 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:57:41.376 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:57:41.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:57:41.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:57:41.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:57:41.499 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:57:41.505 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:57:41.554 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:57:41.563 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-09-21T06:57:41.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-09-21T06:57:41.569 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-09-21T06:57:41.571 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-09-21T06:57:41.610 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-09-21T06:57:41.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-09-21T06:57:41.789 INFO:teuthology.orchestra.run.smithi119.stdout:45 files removed 2024-09-21T06:57:41.794 INFO:teuthology.orchestra.run.smithi153.stdout:45 files removed 2024-09-21T06:57:41.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:57:41.821 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:57:41.892 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-09-21T06:57:41.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:57:42.360 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 320 kB/s | 84 kB 00:00 2024-09-21T06:57:42.381 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 291 kB/s | 84 kB 00:00 2024-09-21T06:57:42.628 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 297 kB/s | 84 kB 00:00 2024-09-21T06:57:42.665 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T06:57:42.792 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 62 kB/s | 24 kB 00:00 2024-09-21T06:57:42.941 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T06:57:42.946 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 89 kB/s | 24 kB 00:00 2024-09-21T06:57:43.070 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T06:57:43.230 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T06:57:43.699 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-21T06:57:43.714 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.3 MB 00:00 2024-09-21T06:57:45.500 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-21T06:57:46.595 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.3 MB 00:03 2024-09-21T06:57:51.501 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-21T06:57:52.848 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-21T06:57:52.985 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-21T06:57:53.797 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T06:57:54.053 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-09-21T06:57:57.481 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-21T06:57:58.487 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T06:57:59.082 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-21T06:57:59.153 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T06:57:59.979 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-21T06:57:59.994 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T06:58:00.842 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-21T06:58:01.325 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:58:01.326 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:58:01.340 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:58:01.341 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:58:01.342 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:58:01.343 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:58:01.345 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:58:01.347 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:58:01.348 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:58:01.411 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-21T06:58:01.426 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:58:01.427 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:58:01.428 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:58:01.429 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:58:01.430 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:58:01.431 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:58:01.432 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:58:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:58:01.434 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:58:01.435 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:58:01.436 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:58:01.437 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-09-21T06:58:01.438 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout:Install 149 Packages 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 8 Packages 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 228 M 2024-09-21T06:58:01.439 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-21T06:58:01.954 INFO:teuthology.orchestra.run.smithi119.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-09-21T06:58:02.196 INFO:teuthology.orchestra.run.smithi119.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.5 MB/s | 873 kB 00:00 2024-09-21T06:58:02.255 INFO:teuthology.orchestra.run.smithi119.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.5 MB/s | 148 kB 00:00 2024-09-21T06:58:02.697 INFO:teuthology.orchestra.run.smithi119.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 5.2 MB/s | 2.3 MB 00:00 2024-09-21T06:58:02.947 INFO:teuthology.orchestra.run.smithi119.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.1 MB/s | 1.5 MB 00:00 2024-09-21T06:58:03.314 INFO:teuthology.orchestra.run.smithi119.stdout:(6/157): ceph-common-19.3.0-5120.g18432218.el9. 12 MB/s | 20 MB 00:01 2024-09-21T06:58:03.431 INFO:teuthology.orchestra.run.smithi119.stdout:(7/157): ceph-base-19.3.0-5120.g18432218.el9.x8 3.2 MB/s | 5.5 MB 00:01 2024-09-21T06:58:03.515 INFO:teuthology.orchestra.run.smithi119.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 8.8 MB/s | 5.0 MB 00:00 2024-09-21T06:58:03.565 INFO:teuthology.orchestra.run.smithi119.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 506 kB/s | 25 kB 00:00 2024-09-21T06:58:04.407 INFO:teuthology.orchestra.run.smithi119.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 11 MB/s | 10 MB 00:00 2024-09-21T06:58:04.482 INFO:teuthology.orchestra.run.smithi119.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.9 MB/s | 226 kB 00:00 2024-09-21T06:58:04.532 INFO:teuthology.orchestra.run.smithi119.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 543 kB/s | 27 kB 00:00 2024-09-21T06:58:04.724 INFO:teuthology.orchestra.run.smithi119.stdout:(13/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 13 MB/s | 18 MB 00:01 2024-09-21T06:58:04.741 INFO:teuthology.orchestra.run.smithi119.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 152 kB/s | 32 kB 00:00 2024-09-21T06:58:04.791 INFO:teuthology.orchestra.run.smithi119.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T06:58:04.825 INFO:teuthology.orchestra.run.smithi119.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T06:58:04.850 INFO:teuthology.orchestra.run.smithi119.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T06:58:04.883 INFO:teuthology.orchestra.run.smithi119.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T06:58:04.934 INFO:teuthology.orchestra.run.smithi119.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 902 kB/s | 45 kB 00:00 2024-09-21T06:58:04.992 INFO:teuthology.orchestra.run.smithi119.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T06:58:05.051 INFO:teuthology.orchestra.run.smithi119.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T06:58:05.126 INFO:teuthology.orchestra.run.smithi119.stdout:(22/157): librgw2-19.3.0-5120.g18432218.el9.x86 19 MB/s | 5.3 MB 00:00 2024-09-21T06:58:05.152 INFO:teuthology.orchestra.run.smithi119.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.1 MB/s | 321 kB 00:00 2024-09-21T06:58:05.193 INFO:teuthology.orchestra.run.smithi119.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T06:58:05.219 INFO:teuthology.orchestra.run.smithi119.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T06:58:05.244 INFO:teuthology.orchestra.run.smithi119.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T06:58:05.328 INFO:teuthology.orchestra.run.smithi119.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.0 MB/s | 170 kB 00:00 2024-09-21T06:58:05.395 INFO:teuthology.orchestra.run.smithi119.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 515 kB/s | 34 kB 00:00 2024-09-21T06:58:05.453 INFO:teuthology.orchestra.run.smithi119.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T06:58:05.503 INFO:teuthology.orchestra.run.smithi119.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 11 MB/s | 3.1 MB 00:00 2024-09-21T06:58:06.121 INFO:teuthology.orchestra.run.smithi119.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 21 MB/s | 14 MB 00:00 2024-09-21T06:58:06.197 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T06:58:06.212 INFO:teuthology.orchestra.run.smithi119.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 10 MB/s | 7.4 MB 00:00 2024-09-21T06:58:06.238 INFO:teuthology.orchestra.run.smithi119.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.4 MB/s | 292 kB 00:00 2024-09-21T06:58:06.263 INFO:teuthology.orchestra.run.smithi119.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 983 kB/s | 49 kB 00:00 2024-09-21T06:58:06.288 INFO:teuthology.orchestra.run.smithi119.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T06:58:06.314 INFO:teuthology.orchestra.run.smithi119.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.5 MB/s | 285 kB 00:00 2024-09-21T06:58:06.347 INFO:teuthology.orchestra.run.smithi119.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T06:58:06.372 INFO:teuthology.orchestra.run.smithi119.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 314 kB/s | 18 kB 00:00 2024-09-21T06:58:06.515 INFO:teuthology.orchestra.run.smithi119.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 477 kB/s | 80 kB 00:00 2024-09-21T06:58:06.615 INFO:teuthology.orchestra.run.smithi119.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-21T06:58:06.715 INFO:teuthology.orchestra.run.smithi119.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-21T06:58:06.740 INFO:teuthology.orchestra.run.smithi119.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-21T06:58:06.791 INFO:teuthology.orchestra.run.smithi119.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T06:58:06.825 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T06:58:06.883 INFO:teuthology.orchestra.run.smithi119.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-21T06:58:07.250 INFO:teuthology.orchestra.run.smithi119.stdout:(45/157): ceph-test-19.3.0-5120.g18432218.el9.x 11 MB/s | 40 MB 00:03 2024-09-21T06:58:07.275 INFO:teuthology.orchestra.run.smithi119.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 319 kB/s | 125 kB 00:00 2024-09-21T06:58:07.309 INFO:teuthology.orchestra.run.smithi119.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-09-21T06:58:07.492 INFO:teuthology.orchestra.run.smithi119.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 888 kB/s | 215 kB 00:00 2024-09-21T06:58:07.601 INFO:teuthology.orchestra.run.smithi119.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-21T06:58:07.643 INFO:teuthology.orchestra.run.smithi119.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-21T06:58:07.668 INFO:teuthology.orchestra.run.smithi119.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-21T06:58:07.693 INFO:teuthology.orchestra.run.smithi119.stdout:(52/157): boost-program-options-1.75.0-8.el9.x8 532 kB/s | 107 kB 00:00 2024-09-21T06:58:07.719 INFO:teuthology.orchestra.run.smithi119.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-21T06:58:07.744 INFO:teuthology.orchestra.run.smithi119.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-09-21T06:58:07.769 INFO:teuthology.orchestra.run.smithi119.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-21T06:58:07.794 INFO:teuthology.orchestra.run.smithi119.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T06:58:07.828 INFO:teuthology.orchestra.run.smithi119.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-21T06:58:07.853 INFO:teuthology.orchestra.run.smithi119.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T06:58:07.887 INFO:teuthology.orchestra.run.smithi119.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-09-21T06:58:07.945 INFO:teuthology.orchestra.run.smithi119.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 9.8 MB/s | 581 kB 00:00 2024-09-21T06:58:07.971 INFO:teuthology.orchestra.run.smithi119.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T06:58:07.996 INFO:teuthology.orchestra.run.smithi119.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-21T06:58:08.021 INFO:teuthology.orchestra.run.smithi119.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T06:58:08.055 INFO:teuthology.orchestra.run.smithi119.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-21T06:58:08.089 INFO:teuthology.orchestra.run.smithi119.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-09-21T06:58:08.114 INFO:teuthology.orchestra.run.smithi119.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 654 kB/s | 38 kB 00:00 2024-09-21T06:58:08.316 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:58:08.317 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:58:08.328 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:58:08.329 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:58:08.330 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:58:08.331 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:58:08.332 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:58:08.335 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:58:08.336 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:58:08.399 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:58:08.409 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:58:08.410 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:58:08.411 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:58:08.412 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:58:08.413 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:58:08.414 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:58:08.415 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:58:08.417 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:58:08.418 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:58:08.419 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:58:08.420 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout:Install 149 Packages 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 8 Packages 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:08.421 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 228 M 2024-09-21T06:58:08.422 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-09-21T06:58:08.614 INFO:teuthology.orchestra.run.smithi119.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-21T06:58:08.781 INFO:teuthology.orchestra.run.smithi119.stdout:(68/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-09-21T06:58:08.782 INFO:teuthology.orchestra.run.smithi153.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-21T06:58:08.840 INFO:teuthology.orchestra.run.smithi119.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-09-21T06:58:08.898 INFO:teuthology.orchestra.run.smithi119.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T06:58:09.007 INFO:teuthology.orchestra.run.smithi153.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 873 kB 00:00 2024-09-21T06:58:09.124 INFO:teuthology.orchestra.run.smithi153.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-21T06:58:09.149 INFO:teuthology.orchestra.run.smithi153.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.0 MB/s | 148 kB 00:00 2024-09-21T06:58:09.215 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:58:09.215 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:58:09.227 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:58:09.228 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:58:09.229 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:58:09.230 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:58:09.232 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:58:09.234 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:58:09.235 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:58:09.250 INFO:teuthology.orchestra.run.smithi153.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 18 MB/s | 2.3 MB 00:00 2024-09-21T06:58:09.273 INFO:teuthology.orchestra.run.smithi119.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 1.8 MB/s | 910 kB 00:00 2024-09-21T06:58:09.305 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-21T06:58:09.317 INFO:teuthology.orchestra.run.smithi153.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:58:09.326 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:58:09.327 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:58:09.328 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-21T06:58:09.329 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:58:09.330 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:58:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:58:09.332 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:58:09.333 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:58:09.334 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:58:09.335 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:58:09.336 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:58:09.337 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:58:09.338 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-21T06:58:09.338 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T06:58:09.338 INFO:teuthology.orchestra.run.smithi027.stdout:Install 149 Packages 2024-09-21T06:58:09.338 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 8 Packages 2024-09-21T06:58:09.338 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:58:09.338 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 228 M 2024-09-21T06:58:09.338 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-21T06:58:09.548 INFO:teuthology.orchestra.run.smithi119.stdout:(72/157): postgresql-server-13.16-1.el9.x86_64. 8.9 MB/s | 5.8 MB 00:00 2024-09-21T06:58:09.558 INFO:teuthology.orchestra.run.smithi153.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 16 MB/s | 5.0 MB 00:00 2024-09-21T06:58:09.599 INFO:teuthology.orchestra.run.smithi119.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-21T06:58:09.792 INFO:teuthology.orchestra.run.smithi153.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T06:58:09.826 INFO:teuthology.orchestra.run.smithi153.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 759 kB/s | 25 kB 00:00 2024-09-21T06:58:09.849 INFO:teuthology.orchestra.run.smithi119.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-09-21T06:58:09.908 INFO:teuthology.orchestra.run.smithi119.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T06:58:10.066 INFO:teuthology.orchestra.run.smithi119.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-21T06:58:10.202 INFO:teuthology.orchestra.run.smithi027.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-09-21T06:58:10.267 INFO:teuthology.orchestra.run.smithi119.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 8.9 MB/s | 6.0 MB 00:00 2024-09-21T06:58:10.325 INFO:teuthology.orchestra.run.smithi119.stdout:(78/157): openblas-openmp-0.3.26-2.el9.x86_64.r 2.2 MB/s | 4.9 MB 00:02 2024-09-21T06:58:10.350 INFO:teuthology.orchestra.run.smithi119.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 307 kB/s | 26 kB 00:00 2024-09-21T06:58:10.375 INFO:teuthology.orchestra.run.smithi119.stdout:(80/157): python3-jinja2-2.11.3-6.el9.noarch.rp 803 kB/s | 248 kB 00:00 2024-09-21T06:58:10.401 INFO:teuthology.orchestra.run.smithi119.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-09-21T06:58:10.426 INFO:teuthology.orchestra.run.smithi119.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-21T06:58:10.484 INFO:teuthology.orchestra.run.smithi119.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-21T06:58:10.568 INFO:teuthology.orchestra.run.smithi119.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-21T06:58:10.651 INFO:teuthology.orchestra.run.smithi153.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 9.5 MB/s | 10 MB 00:01 2024-09-21T06:58:10.652 INFO:teuthology.orchestra.run.smithi119.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-09-21T06:58:10.677 INFO:teuthology.orchestra.run.smithi119.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-09-21T06:58:10.702 INFO:teuthology.orchestra.run.smithi119.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-21T06:58:10.709 INFO:teuthology.orchestra.run.smithi153.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T06:58:10.769 INFO:teuthology.orchestra.run.smithi027.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 873 kB 00:00 2024-09-21T06:58:10.811 INFO:teuthology.orchestra.run.smithi119.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-21T06:58:10.836 INFO:teuthology.orchestra.run.smithi027.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T06:58:10.869 INFO:teuthology.orchestra.run.smithi119.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-09-21T06:58:10.893 INFO:teuthology.orchestra.run.smithi153.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 11 MB/s | 18 MB 00:01 2024-09-21T06:58:10.918 INFO:teuthology.orchestra.run.smithi153.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 129 kB/s | 27 kB 00:00 2024-09-21T06:58:10.928 INFO:teuthology.orchestra.run.smithi119.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 877 kB/s | 51 kB 00:00 2024-09-21T06:58:10.943 INFO:teuthology.orchestra.run.smithi153.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T06:58:10.986 INFO:teuthology.orchestra.run.smithi153.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T06:58:11.011 INFO:teuthology.orchestra.run.smithi153.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T06:58:11.020 INFO:teuthology.orchestra.run.smithi119.stdout:(91/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-09-21T06:58:11.045 INFO:teuthology.orchestra.run.smithi119.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 459 kB/s | 54 kB 00:00 2024-09-21T06:58:11.053 INFO:teuthology.orchestra.run.smithi153.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T06:58:11.071 INFO:teuthology.orchestra.run.smithi119.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.0 MB/s | 279 kB 00:00 2024-09-21T06:58:11.086 INFO:teuthology.orchestra.run.smithi153.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T06:58:11.096 INFO:teuthology.orchestra.run.smithi119.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-21T06:58:11.121 INFO:teuthology.orchestra.run.smithi119.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-21T06:58:11.162 INFO:teuthology.orchestra.run.smithi153.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 602 kB/s | 45 kB 00:00 2024-09-21T06:58:11.172 INFO:teuthology.orchestra.run.smithi119.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-09-21T06:58:11.222 INFO:teuthology.orchestra.run.smithi119.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 953 kB/s | 47 kB 00:00 2024-09-21T06:58:11.245 INFO:teuthology.orchestra.run.smithi153.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T06:58:11.255 INFO:teuthology.orchestra.run.smithi119.stdout:(98/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-21T06:58:11.297 INFO:teuthology.orchestra.run.smithi119.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-21T06:58:11.319 INFO:teuthology.orchestra.run.smithi027.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 4.7 MB/s | 2.3 MB 00:00 2024-09-21T06:58:11.371 INFO:teuthology.orchestra.run.smithi153.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.3 MB/s | 161 kB 00:00 2024-09-21T06:58:11.381 INFO:teuthology.orchestra.run.smithi119.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-21T06:58:11.406 INFO:teuthology.orchestra.run.smithi119.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-21T06:58:11.448 INFO:teuthology.orchestra.run.smithi119.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T06:58:11.465 INFO:teuthology.orchestra.run.smithi119.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 760 kB/s | 64 kB 00:00 2024-09-21T06:58:11.523 INFO:teuthology.orchestra.run.smithi027.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 3.5 MB/s | 5.5 MB 00:01 2024-09-21T06:58:11.561 INFO:teuthology.orchestra.run.smithi027.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.3 MB/s | 1.5 MB 00:00 2024-09-21T06:58:11.746 INFO:teuthology.orchestra.run.smithi153.stdout:(22/157): ceph-test-19.3.0-5120.g18432218.el9.x 21 MB/s | 40 MB 00:01 2024-09-21T06:58:11.771 INFO:teuthology.orchestra.run.smithi153.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 802 kB/s | 321 kB 00:00 2024-09-21T06:58:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 6.7 MB/s | 5.3 MB 00:00 2024-09-21T06:58:11.871 INFO:teuthology.orchestra.run.smithi153.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 994 kB/s | 99 kB 00:00 2024-09-21T06:58:11.888 INFO:teuthology.orchestra.run.smithi153.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 86 kB 00:00 2024-09-21T06:58:11.903 INFO:teuthology.orchestra.run.smithi027.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.0 MB 00:00 2024-09-21T06:58:11.914 INFO:teuthology.orchestra.run.smithi153.stdout:(27/157): python3-rbd-19.3.0-5120.g18432218.el9 1.8 MB/s | 307 kB 00:00 2024-09-21T06:58:11.939 INFO:teuthology.orchestra.run.smithi153.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T06:58:11.982 INFO:teuthology.orchestra.run.smithi119.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 42 kB/s | 22 kB 00:00 2024-09-21T06:58:11.989 INFO:teuthology.orchestra.run.smithi153.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 457 kB/s | 34 kB 00:00 2024-09-21T06:58:12.014 INFO:teuthology.orchestra.run.smithi153.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.0 MB/s | 154 kB 00:00 2024-09-21T06:58:12.032 INFO:teuthology.orchestra.run.smithi119.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T06:58:12.086 INFO:teuthology.orchestra.run.smithi119.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 5.6 MB/s | 308 kB 00:00 2024-09-21T06:58:12.108 INFO:teuthology.orchestra.run.smithi119.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 907 kB/s | 19 kB 00:00 2024-09-21T06:58:12.219 INFO:teuthology.orchestra.run.smithi119.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T06:58:12.241 INFO:teuthology.orchestra.run.smithi119.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T06:58:12.267 INFO:teuthology.orchestra.run.smithi119.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T06:58:12.292 INFO:teuthology.orchestra.run.smithi119.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T06:58:12.298 INFO:teuthology.orchestra.run.smithi153.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 7.3 MB/s | 3.1 MB 00:00 2024-09-21T06:58:12.318 INFO:teuthology.orchestra.run.smithi119.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T06:58:12.359 INFO:teuthology.orchestra.run.smithi119.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T06:58:12.390 INFO:teuthology.orchestra.run.smithi153.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T06:58:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T06:58:12.435 INFO:teuthology.orchestra.run.smithi119.stdout:(115/157): protobuf-compiler-3.14.0-13.el9.x86_ 890 kB/s | 863 kB 00:00 2024-09-21T06:58:12.460 INFO:teuthology.orchestra.run.smithi119.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-09-21T06:58:12.465 INFO:teuthology.orchestra.run.smithi153.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 660 kB/s | 49 kB 00:00 2024-09-21T06:58:12.494 INFO:teuthology.orchestra.run.smithi119.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 536 kB/s | 14 kB 00:00 2024-09-21T06:58:12.519 INFO:teuthology.orchestra.run.smithi119.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 7.2 MB/s | 172 kB 00:00 2024-09-21T06:58:12.540 INFO:teuthology.orchestra.run.smithi153.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 219 kB/s | 16 kB 00:00 2024-09-21T06:58:12.544 INFO:teuthology.orchestra.run.smithi119.stdout:(119/157): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-09-21T06:58:12.578 INFO:teuthology.orchestra.run.smithi119.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T06:58:12.595 INFO:teuthology.orchestra.run.smithi027.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 7.6 MB/s | 20 MB 00:02 2024-09-21T06:58:12.603 INFO:teuthology.orchestra.run.smithi119.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T06:58:12.615 INFO:teuthology.orchestra.run.smithi153.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T06:58:12.687 INFO:teuthology.orchestra.run.smithi027.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 276 kB/s | 25 kB 00:00 2024-09-21T06:58:12.762 INFO:teuthology.orchestra.run.smithi119.stdout:(122/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.3 MB/s | 16 MB 00:01 2024-09-21T06:58:12.787 INFO:teuthology.orchestra.run.smithi119.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 1.2 MB/s | 220 kB 00:00 2024-09-21T06:58:12.821 INFO:teuthology.orchestra.run.smithi119.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 410 kB/s | 11 kB 00:00 2024-09-21T06:58:12.846 INFO:teuthology.orchestra.run.smithi119.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 768 kB/s | 18 kB 00:00 2024-09-21T06:58:12.849 INFO:teuthology.orchestra.run.smithi153.stdout:(36/157): ceph-mgr-dashboard-19.3.0-5120.g18432 16 MB/s | 14 MB 00:00 2024-09-21T06:58:12.871 INFO:teuthology.orchestra.run.smithi119.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 922 kB/s | 23 kB 00:00 2024-09-21T06:58:12.883 INFO:teuthology.orchestra.run.smithi153.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 2.9 MB/s | 784 kB 00:00 2024-09-21T06:58:12.896 INFO:teuthology.orchestra.run.smithi119.stdout:(127/157): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-09-21T06:58:12.900 INFO:teuthology.orchestra.run.smithi153.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 367 kB/s | 18 kB 00:00 2024-09-21T06:58:12.922 INFO:teuthology.orchestra.run.smithi119.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T06:58:12.947 INFO:teuthology.orchestra.run.smithi119.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-21T06:58:12.981 INFO:teuthology.orchestra.run.smithi119.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T06:58:12.992 INFO:teuthology.orchestra.run.smithi153.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 7.5 MB/s | 7.4 MB 00:00 2024-09-21T06:58:13.004 INFO:teuthology.orchestra.run.smithi027.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 9.5 MB/s | 10 MB 00:01 2024-09-21T06:58:13.006 INFO:teuthology.orchestra.run.smithi119.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:58:13.031 INFO:teuthology.orchestra.run.smithi119.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:58:13.063 INFO:teuthology.orchestra.run.smithi027.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T06:58:13.065 INFO:teuthology.orchestra.run.smithi119.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-21T06:58:13.090 INFO:teuthology.orchestra.run.smithi119.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-21T06:58:13.115 INFO:teuthology.orchestra.run.smithi119.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-21T06:58:13.141 INFO:teuthology.orchestra.run.smithi119.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T06:58:13.166 INFO:teuthology.orchestra.run.smithi119.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T06:58:13.191 INFO:teuthology.orchestra.run.smithi119.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-09-21T06:58:13.233 INFO:teuthology.orchestra.run.smithi119.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-09-21T06:58:13.258 INFO:teuthology.orchestra.run.smithi119.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 887 kB/s | 59 kB 00:00 2024-09-21T06:58:13.271 INFO:teuthology.orchestra.run.smithi027.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 10 MB/s | 18 MB 00:01 2024-09-21T06:58:13.284 INFO:teuthology.orchestra.run.smithi119.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-21T06:58:13.297 INFO:teuthology.orchestra.run.smithi027.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 116 kB/s | 27 kB 00:00 2024-09-21T06:58:13.309 INFO:teuthology.orchestra.run.smithi119.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-21T06:58:13.322 INFO:teuthology.orchestra.run.smithi027.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 637 kB/s | 32 kB 00:00 2024-09-21T06:58:13.334 INFO:teuthology.orchestra.run.smithi119.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-21T06:58:13.360 INFO:teuthology.orchestra.run.smithi119.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T06:58:13.372 INFO:teuthology.orchestra.run.smithi027.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T06:58:13.375 INFO:teuthology.orchestra.run.smithi153.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 162 kB/s | 80 kB 00:00 2024-09-21T06:58:13.376 INFO:teuthology.orchestra.run.smithi119.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-21T06:58:13.397 INFO:teuthology.orchestra.run.smithi027.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T06:58:13.402 INFO:teuthology.orchestra.run.smithi119.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-21T06:58:13.423 INFO:teuthology.orchestra.run.smithi027.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T06:58:13.435 INFO:teuthology.orchestra.run.smithi119.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T06:58:13.456 INFO:teuthology.orchestra.run.smithi027.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T06:58:13.460 INFO:teuthology.orchestra.run.smithi119.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T06:58:13.511 INFO:teuthology.orchestra.run.smithi119.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T06:58:13.532 INFO:teuthology.orchestra.run.smithi027.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 602 kB/s | 45 kB 00:00 2024-09-21T06:58:13.542 INFO:teuthology.orchestra.run.smithi153.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-21T06:58:13.590 INFO:teuthology.orchestra.run.smithi027.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T06:58:13.665 INFO:teuthology.orchestra.run.smithi027.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.1 MB/s | 161 kB 00:00 2024-09-21T06:58:13.726 INFO:teuthology.orchestra.run.smithi153.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 970 kB/s | 801 kB 00:00 2024-09-21T06:58:13.749 INFO:teuthology.orchestra.run.smithi027.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T06:58:13.751 INFO:teuthology.orchestra.run.smithi153.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 674 kB/s | 253 kB 00:00 2024-09-21T06:58:13.816 INFO:teuthology.orchestra.run.smithi027.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T06:58:13.828 INFO:teuthology.orchestra.run.smithi119.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.6 MB/s | 3.2 MB 00:00 2024-09-21T06:58:13.851 INFO:teuthology.orchestra.run.smithi153.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-09-21T06:58:13.876 INFO:teuthology.orchestra.run.smithi153.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-21T06:58:13.878 INFO:teuthology.orchestra.run.smithi119.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 7.7 MB/s | 3.4 MB 00:00 2024-09-21T06:58:13.925 INFO:teuthology.orchestra.run.smithi027.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.3 MB 00:00 2024-09-21T06:58:13.945 INFO:teuthology.orchestra.run.smithi119.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 5.3 MB/s | 2.3 MB 00:00 2024-09-21T06:58:13.950 INFO:teuthology.orchestra.run.smithi027.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 742 kB/s | 99 kB 00:00 2024-09-21T06:58:13.968 INFO:teuthology.orchestra.run.smithi153.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T06:58:13.970 INFO:teuthology.orchestra.run.smithi119.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 445 kB/s | 41 kB 00:00 2024-09-21T06:58:13.975 INFO:teuthology.orchestra.run.smithi027.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T06:58:13.994 INFO:teuthology.orchestra.run.smithi153.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T06:58:14.004 INFO:teuthology.orchestra.run.smithi119.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-21T06:58:14.034 INFO:teuthology.orchestra.run.smithi027.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.8 MB/s | 170 kB 00:00 2024-09-21T06:58:14.093 INFO:teuthology.orchestra.run.smithi027.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 589 kB/s | 34 kB 00:00 2024-09-21T06:58:14.110 INFO:teuthology.orchestra.run.smithi153.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-09-21T06:58:14.153 INFO:teuthology.orchestra.run.smithi153.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-21T06:58:14.186 INFO:teuthology.orchestra.run.smithi027.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 13 MB/s | 3.1 MB 00:00 2024-09-21T06:58:14.186 INFO:teuthology.orchestra.run.smithi153.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-09-21T06:58:14.210 INFO:teuthology.orchestra.run.smithi027.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.3 MB/s | 154 kB 00:00 2024-09-21T06:58:14.211 INFO:teuthology.orchestra.run.smithi153.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-21T06:58:14.237 INFO:teuthology.orchestra.run.smithi119.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 4.4 MB/s | 1.8 MB 00:00 2024-09-21T06:58:14.245 INFO:teuthology.orchestra.run.smithi153.stdout:(52/157): python3-cryptography-36.0.1-4.el9.x86 1.7 MB/s | 1.2 MB 00:00 2024-09-21T06:58:14.279 INFO:teuthology.orchestra.run.smithi153.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 486 kB/s | 33 kB 00:00 2024-09-21T06:58:14.279 INFO:teuthology.orchestra.run.smithi119.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.7 MB/s | 2.4 MB 00:00 2024-09-21T06:58:14.304 INFO:teuthology.orchestra.run.smithi153.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T06:58:14.313 INFO:teuthology.orchestra.run.smithi119.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 1.4 MB 00:00 2024-09-21T06:58:14.318 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:58:14.318 INFO:teuthology.orchestra.run.smithi119.stdout:Total 18 MB/s | 228 MB 00:12 2024-09-21T06:58:14.329 INFO:teuthology.orchestra.run.smithi153.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T06:58:14.355 INFO:teuthology.orchestra.run.smithi153.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-21T06:58:14.380 INFO:teuthology.orchestra.run.smithi153.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T06:58:14.405 INFO:teuthology.orchestra.run.smithi153.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T06:58:14.430 INFO:teuthology.orchestra.run.smithi153.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-21T06:58:14.464 INFO:teuthology.orchestra.run.smithi153.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-21T06:58:14.497 INFO:teuthology.orchestra.run.smithi153.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-21T06:58:14.523 INFO:teuthology.orchestra.run.smithi153.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-21T06:58:14.548 INFO:teuthology.orchestra.run.smithi153.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T06:58:14.573 INFO:teuthology.orchestra.run.smithi153.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T06:58:14.598 INFO:teuthology.orchestra.run.smithi153.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T06:58:14.623 INFO:teuthology.orchestra.run.smithi153.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-21T06:58:14.674 INFO:teuthology.orchestra.run.smithi153.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.0 MB/s | 3.0 MB 00:00 2024-09-21T06:58:14.760 INFO:teuthology.orchestra.run.smithi027.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-21T06:58:14.783 INFO:teuthology.orchestra.run.smithi153.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-21T06:58:14.824 INFO:teuthology.orchestra.run.smithi153.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-21T06:58:14.892 INFO:teuthology.orchestra.run.smithi153.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-21T06:58:14.925 INFO:teuthology.orchestra.run.smithi153.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-21T06:58:14.950 INFO:teuthology.orchestra.run.smithi153.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T06:58:15.000 INFO:teuthology.orchestra.run.smithi153.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T06:58:15.017 INFO:teuthology.orchestra.run.smithi153.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T06:58:15.126 INFO:teuthology.orchestra.run.smithi153.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-21T06:58:15.151 INFO:teuthology.orchestra.run.smithi153.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 458 kB/s | 61 kB 00:00 2024-09-21T06:58:15.176 INFO:teuthology.orchestra.run.smithi153.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T06:58:15.202 INFO:teuthology.orchestra.run.smithi153.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T06:58:15.219 INFO:teuthology.orchestra.run.smithi153.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-09-21T06:58:15.235 INFO:teuthology.orchestra.run.smithi153.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-09-21T06:58:15.244 INFO:teuthology.orchestra.run.smithi027.stdout:(32/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T06:58:15.261 INFO:teuthology.orchestra.run.smithi153.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-21T06:58:15.278 INFO:teuthology.orchestra.run.smithi153.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 834 kB/s | 35 kB 00:00 2024-09-21T06:58:15.286 INFO:teuthology.orchestra.run.smithi027.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 556 kB/s | 292 kB 00:00 2024-09-21T06:58:15.311 INFO:teuthology.orchestra.run.smithi027.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 741 kB/s | 49 kB 00:00 2024-09-21T06:58:15.336 INFO:teuthology.orchestra.run.smithi027.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T06:58:15.370 INFO:teuthology.orchestra.run.smithi153.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-21T06:58:15.378 INFO:teuthology.orchestra.run.smithi027.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.2 MB/s | 285 kB 00:00 2024-09-21T06:58:15.395 INFO:teuthology.orchestra.run.smithi153.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-21T06:58:15.420 INFO:teuthology.orchestra.run.smithi027.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 446 kB/s | 18 kB 00:00 2024-09-21T06:58:15.421 INFO:teuthology.orchestra.run.smithi153.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T06:58:15.446 INFO:teuthology.orchestra.run.smithi153.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T06:58:15.471 INFO:teuthology.orchestra.run.smithi153.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-21T06:58:15.496 INFO:teuthology.orchestra.run.smithi153.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-21T06:58:15.507 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-21T06:58:15.522 INFO:teuthology.orchestra.run.smithi153.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T06:58:15.547 INFO:teuthology.orchestra.run.smithi153.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T06:58:15.572 INFO:teuthology.orchestra.run.smithi153.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:58:15.587 INFO:teuthology.orchestra.run.smithi027.stdout:(38/157): ceph-mgr-dashboard-19.3.0-5120.g18432 10 MB/s | 14 MB 00:01 2024-09-21T06:58:15.597 INFO:teuthology.orchestra.run.smithi153.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T06:58:15.629 INFO:teuthology.orchestra.run.smithi027.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 2.6 MB/s | 784 kB 00:00 2024-09-21T06:58:15.659 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-21T06:58:15.659 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-21T06:58:15.671 INFO:teuthology.orchestra.run.smithi027.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 319 kB/s | 80 kB 00:00 2024-09-21T06:58:15.672 INFO:teuthology.orchestra.run.smithi153.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T06:58:15.698 INFO:teuthology.orchestra.run.smithi153.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-21T06:58:15.723 INFO:teuthology.orchestra.run.smithi153.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-09-21T06:58:15.748 INFO:teuthology.orchestra.run.smithi153.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T06:58:15.755 INFO:teuthology.orchestra.run.smithi027.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T06:58:15.774 INFO:teuthology.orchestra.run.smithi153.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-21T06:58:15.788 INFO:teuthology.orchestra.run.smithi027.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-21T06:58:15.799 INFO:teuthology.orchestra.run.smithi153.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T06:58:15.813 INFO:teuthology.orchestra.run.smithi027.stdout:(43/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-21T06:58:15.824 INFO:teuthology.orchestra.run.smithi153.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T06:58:15.839 INFO:teuthology.orchestra.run.smithi027.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T06:58:15.849 INFO:teuthology.orchestra.run.smithi153.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T06:58:15.864 INFO:teuthology.orchestra.run.smithi027.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T06:58:15.866 INFO:teuthology.orchestra.run.smithi153.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-21T06:58:15.889 INFO:teuthology.orchestra.run.smithi027.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T06:58:15.900 INFO:teuthology.orchestra.run.smithi153.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T06:58:15.914 INFO:teuthology.orchestra.run.smithi027.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T06:58:15.925 INFO:teuthology.orchestra.run.smithi153.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-09-21T06:58:15.957 INFO:teuthology.orchestra.run.smithi027.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-09-21T06:58:16.064 INFO:teuthology.orchestra.run.smithi027.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-09-21T06:58:16.091 INFO:teuthology.orchestra.run.smithi027.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 800 kB/s | 107 kB 00:00 2024-09-21T06:58:16.132 INFO:teuthology.orchestra.run.smithi027.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-09-21T06:58:16.157 INFO:teuthology.orchestra.run.smithi027.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 234 kB/s | 22 kB 00:00 2024-09-21T06:58:16.183 INFO:teuthology.orchestra.run.smithi027.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-21T06:58:16.184 INFO:teuthology.orchestra.run.smithi153.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-21T06:58:16.200 INFO:teuthology.orchestra.run.smithi027.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-09-21T06:58:16.234 INFO:teuthology.orchestra.run.smithi153.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T06:58:16.259 INFO:teuthology.orchestra.run.smithi153.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T06:58:16.283 INFO:teuthology.orchestra.run.smithi027.stdout:(55/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-21T06:58:16.301 INFO:teuthology.orchestra.run.smithi153.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 699 kB/s | 19 kB 00:00 2024-09-21T06:58:16.309 INFO:teuthology.orchestra.run.smithi027.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-21T06:58:16.334 INFO:teuthology.orchestra.run.smithi027.stdout:(57/157): flexiblas-openblas-openmp-3.0.4-8.el9 120 kB/s | 18 kB 00:00 2024-09-21T06:58:16.359 INFO:teuthology.orchestra.run.smithi027.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-21T06:58:16.385 INFO:teuthology.orchestra.run.smithi027.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-21T06:58:16.410 INFO:teuthology.orchestra.run.smithi027.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-09-21T06:58:16.427 INFO:teuthology.orchestra.run.smithi153.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-21T06:58:16.460 INFO:teuthology.orchestra.run.smithi153.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 821 kB/s | 26 kB 00:00 2024-09-21T06:58:16.485 INFO:teuthology.orchestra.run.smithi153.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T06:58:16.510 INFO:teuthology.orchestra.run.smithi153.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T06:58:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.2 MB/s | 662 kB 00:00 2024-09-21T06:58:16.536 INFO:teuthology.orchestra.run.smithi027.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 195 kB/s | 24 kB 00:00 2024-09-21T06:58:16.536 INFO:teuthology.orchestra.run.smithi153.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T06:58:16.569 INFO:teuthology.orchestra.run.smithi027.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-21T06:58:16.577 INFO:teuthology.orchestra.run.smithi153.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T06:58:16.603 INFO:teuthology.orchestra.run.smithi027.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 532 kB/s | 47 kB 00:00 2024-09-21T06:58:16.603 INFO:teuthology.orchestra.run.smithi153.stdout:(114/157): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-09-21T06:58:16.628 INFO:teuthology.orchestra.run.smithi027.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T06:58:16.637 INFO:teuthology.orchestra.run.smithi153.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-21T06:58:16.653 INFO:teuthology.orchestra.run.smithi027.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T06:58:16.671 INFO:teuthology.orchestra.run.smithi153.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.0 MB/s | 32 kB 00:00 2024-09-21T06:58:16.678 INFO:teuthology.orchestra.run.smithi027.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-21T06:58:16.696 INFO:teuthology.orchestra.run.smithi153.stdout:(117/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 470 kB/s | 43 kB 00:00 2024-09-21T06:58:16.721 INFO:teuthology.orchestra.run.smithi153.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T06:58:16.746 INFO:teuthology.orchestra.run.smithi153.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T06:58:16.772 INFO:teuthology.orchestra.run.smithi153.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T06:58:16.795 INFO:teuthology.orchestra.run.smithi027.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-21T06:58:16.797 INFO:teuthology.orchestra.run.smithi153.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.7 MB/s | 220 kB 00:00 2024-09-21T06:58:16.822 INFO:teuthology.orchestra.run.smithi153.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-21T06:58:16.855 INFO:teuthology.orchestra.run.smithi153.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T06:58:16.870 INFO:teuthology.orchestra.run.smithi027.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-21T06:58:16.872 INFO:teuthology.orchestra.run.smithi153.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 639 kB/s | 11 kB 00:00 2024-09-21T06:58:16.889 INFO:teuthology.orchestra.run.smithi153.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T06:58:16.906 INFO:teuthology.orchestra.run.smithi153.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.3 MB/s | 23 kB 00:00 2024-09-21T06:58:16.923 INFO:teuthology.orchestra.run.smithi153.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-21T06:58:16.929 INFO:teuthology.orchestra.run.smithi027.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-21T06:58:16.940 INFO:teuthology.orchestra.run.smithi153.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.1 MB/s | 20 kB 00:00 2024-09-21T06:58:16.971 INFO:teuthology.orchestra.run.smithi027.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-21T06:58:16.991 INFO:teuthology.orchestra.run.smithi153.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-21T06:58:16.996 INFO:teuthology.orchestra.run.smithi027.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T06:58:17.007 INFO:teuthology.orchestra.run.smithi153.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-21T06:58:17.033 INFO:teuthology.orchestra.run.smithi153.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:58:17.058 INFO:teuthology.orchestra.run.smithi153.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:58:17.063 INFO:teuthology.orchestra.run.smithi027.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-21T06:58:17.089 INFO:teuthology.orchestra.run.smithi027.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 951 kB/s | 87 kB 00:00 2024-09-21T06:58:17.092 INFO:teuthology.orchestra.run.smithi153.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-21T06:58:17.134 INFO:teuthology.orchestra.run.smithi153.stdout:(134/157): protobuf-compiler-3.14.0-13.el9.x86_ 714 kB/s | 863 kB 00:01 2024-09-21T06:58:17.159 INFO:teuthology.orchestra.run.smithi153.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-21T06:58:17.184 INFO:teuthology.orchestra.run.smithi153.stdout:(136/157): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-09-21T06:58:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-21T06:58:17.218 INFO:teuthology.orchestra.run.smithi153.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 1.0 MB/s | 28 kB 00:00 2024-09-21T06:58:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 459 kB/s | 61 kB 00:00 2024-09-21T06:58:17.244 INFO:teuthology.orchestra.run.smithi153.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 597 kB/s | 31 kB 00:00 2024-09-21T06:58:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-21T06:58:17.270 INFO:teuthology.orchestra.run.smithi153.stdout:(139/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 670 kB/s | 90 kB 00:00 2024-09-21T06:58:17.296 INFO:teuthology.orchestra.run.smithi153.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T06:58:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-21T06:58:17.322 INFO:teuthology.orchestra.run.smithi153.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T06:58:17.336 INFO:teuthology.orchestra.run.smithi153.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-21T06:58:17.361 INFO:teuthology.orchestra.run.smithi153.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-21T06:58:17.386 INFO:teuthology.orchestra.run.smithi153.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T06:58:17.407 INFO:teuthology.orchestra.run.smithi027.stdout:(79/157): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-21T06:58:17.411 INFO:teuthology.orchestra.run.smithi153.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-21T06:58:17.432 INFO:teuthology.orchestra.run.smithi027.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 153 kB/s | 26 kB 00:00 2024-09-21T06:58:17.437 INFO:teuthology.orchestra.run.smithi153.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-21T06:58:17.462 INFO:teuthology.orchestra.run.smithi153.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T06:58:17.486 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-21T06:58:17.486 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-21T06:58:17.496 INFO:teuthology.orchestra.run.smithi153.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T06:58:17.532 INFO:teuthology.orchestra.run.smithi027.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 82 kB/s | 19 kB 00:00 2024-09-21T06:58:17.546 INFO:teuthology.orchestra.run.smithi153.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T06:58:17.557 INFO:teuthology.orchestra.run.smithi027.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-21T06:58:17.591 INFO:teuthology.orchestra.run.smithi027.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-09-21T06:58:17.700 INFO:teuthology.orchestra.run.smithi027.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 5.1 MB 00:00 2024-09-21T06:58:17.758 INFO:teuthology.orchestra.run.smithi027.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-09-21T06:58:17.763 INFO:teuthology.orchestra.run.smithi153.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-21T06:58:17.784 INFO:teuthology.orchestra.run.smithi027.stdout:(86/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-09-21T06:58:17.801 INFO:teuthology.orchestra.run.smithi027.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 767 kB/s | 77 kB 00:00 2024-09-21T06:58:17.821 INFO:teuthology.orchestra.run.smithi153.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 9.4 MB/s | 3.4 MB 00:00 2024-09-21T06:58:17.826 INFO:teuthology.orchestra.run.smithi027.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 627 kB/s | 42 kB 00:00 2024-09-21T06:58:17.868 INFO:teuthology.orchestra.run.smithi027.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-21T06:58:17.893 INFO:teuthology.orchestra.run.smithi027.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-21T06:58:17.918 INFO:teuthology.orchestra.run.smithi027.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-21T06:58:17.935 INFO:teuthology.orchestra.run.smithi027.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 760 kB/s | 51 kB 00:00 2024-09-21T06:58:17.952 INFO:teuthology.orchestra.run.smithi027.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 913 kB/s | 54 kB 00:00 2024-09-21T06:58:17.986 INFO:teuthology.orchestra.run.smithi027.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-21T06:58:18.003 INFO:teuthology.orchestra.run.smithi027.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 875 kB/s | 44 kB 00:00 2024-09-21T06:58:18.045 INFO:teuthology.orchestra.run.smithi027.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T06:58:18.061 INFO:teuthology.orchestra.run.smithi027.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-09-21T06:58:18.104 INFO:teuthology.orchestra.run.smithi027.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-09-21T06:58:18.113 INFO:teuthology.orchestra.run.smithi153.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 140 kB/s | 41 kB 00:00 2024-09-21T06:58:18.129 INFO:teuthology.orchestra.run.smithi027.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-21T06:58:18.179 INFO:teuthology.orchestra.run.smithi027.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 772 kB/s | 58 kB 00:00 2024-09-21T06:58:18.204 INFO:teuthology.orchestra.run.smithi027.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-21T06:58:18.397 INFO:teuthology.orchestra.run.smithi027.stdout:(102/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-21T06:58:18.421 INFO:teuthology.orchestra.run.smithi027.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 206 kB/s | 45 kB 00:00 2024-09-21T06:58:18.422 INFO:teuthology.orchestra.run.smithi153.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 807 kB/s | 249 kB 00:00 2024-09-21T06:58:18.447 INFO:teuthology.orchestra.run.smithi027.stdout:(104/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 238 kB/s | 64 kB 00:00 2024-09-21T06:58:18.464 INFO:teuthology.orchestra.run.smithi153.stdout:(154/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-09-21T06:58:18.514 INFO:teuthology.orchestra.run.smithi027.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-21T06:58:18.572 INFO:teuthology.orchestra.run.smithi027.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 5.2 MB/s | 308 kB 00:00 2024-09-21T06:58:18.598 INFO:teuthology.orchestra.run.smithi027.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-21T06:58:18.606 INFO:teuthology.orchestra.run.smithi153.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 2.1 MB/s | 1.8 MB 00:00 2024-09-21T06:58:18.715 INFO:teuthology.orchestra.run.smithi027.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-21T06:58:18.722 INFO:teuthology.orchestra.run.smithi153.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-09-21T06:58:18.740 INFO:teuthology.orchestra.run.smithi027.stdout:(109/157): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-09-21T06:58:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-09-21T06:58:18.798 INFO:teuthology.orchestra.run.smithi027.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-21T06:58:18.824 INFO:teuthology.orchestra.run.smithi027.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2024-09-21T06:58:18.849 INFO:teuthology.orchestra.run.smithi027.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-21T06:58:18.899 INFO:teuthology.orchestra.run.smithi027.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-21T06:58:18.924 INFO:teuthology.orchestra.run.smithi027.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-21T06:58:18.950 INFO:teuthology.orchestra.run.smithi027.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T06:58:18.975 INFO:teuthology.orchestra.run.smithi027.stdout:(117/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-09-21T06:58:19.009 INFO:teuthology.orchestra.run.smithi027.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-21T06:58:19.034 INFO:teuthology.orchestra.run.smithi027.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T06:58:19.068 INFO:teuthology.orchestra.run.smithi027.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T06:58:19.073 INFO:teuthology.orchestra.run.smithi153.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 1.4 MB 00:00 2024-09-21T06:58:19.079 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:58:19.080 INFO:teuthology.orchestra.run.smithi153.stdout:Total 21 MB/s | 228 MB 00:10 2024-09-21T06:58:19.093 INFO:teuthology.orchestra.run.smithi027.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T06:58:19.118 INFO:teuthology.orchestra.run.smithi027.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T06:58:19.143 INFO:teuthology.orchestra.run.smithi027.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T06:58:19.169 INFO:teuthology.orchestra.run.smithi027.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T06:58:19.194 INFO:teuthology.orchestra.run.smithi027.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-21T06:58:19.219 INFO:teuthology.orchestra.run.smithi027.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-21T06:58:19.245 INFO:teuthology.orchestra.run.smithi027.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-21T06:58:19.270 INFO:teuthology.orchestra.run.smithi027.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-21T06:58:19.329 INFO:teuthology.orchestra.run.smithi027.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-21T06:58:19.354 INFO:teuthology.orchestra.run.smithi027.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:58:19.416 INFO:teuthology.orchestra.run.smithi027.stdout:(131/157): protobuf-compiler-3.14.0-13.el9.x86_ 869 kB/s | 863 kB 00:00 2024-09-21T06:58:19.463 INFO:teuthology.orchestra.run.smithi027.stdout:(132/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-09-21T06:58:19.488 INFO:teuthology.orchestra.run.smithi027.stdout:(133/157): python3-more-itertools-8.12.0-2.el9. 589 kB/s | 79 kB 00:00 2024-09-21T06:58:19.513 INFO:teuthology.orchestra.run.smithi027.stdout:(134/157): python3-natsort-7.1.1-5.el9.noarch.r 595 kB/s | 58 kB 00:00 2024-09-21T06:58:19.538 INFO:teuthology.orchestra.run.smithi027.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 340 kB/s | 16 kB 00:00 2024-09-21T06:58:19.564 INFO:teuthology.orchestra.run.smithi027.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T06:58:19.589 INFO:teuthology.orchestra.run.smithi027.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T06:58:19.614 INFO:teuthology.orchestra.run.smithi027.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T06:58:19.640 INFO:teuthology.orchestra.run.smithi027.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T06:58:19.665 INFO:teuthology.orchestra.run.smithi027.stdout:(140/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-21T06:58:19.682 INFO:teuthology.orchestra.run.smithi027.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 855 kB/s | 36 kB 00:00 2024-09-21T06:58:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-21T06:58:19.732 INFO:teuthology.orchestra.run.smithi027.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-21T06:58:19.758 INFO:teuthology.orchestra.run.smithi027.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T06:58:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-21T06:58:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T06:58:19.834 INFO:teuthology.orchestra.run.smithi027.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T06:58:19.867 INFO:teuthology.orchestra.run.smithi027.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T06:58:19.917 INFO:teuthology.orchestra.run.smithi027.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T06:58:20.118 INFO:teuthology.orchestra.run.smithi027.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-21T06:58:20.202 INFO:teuthology.orchestra.run.smithi027.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 9.2 MB/s | 3.4 MB 00:00 2024-09-21T06:58:20.260 INFO:teuthology.orchestra.run.smithi027.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 13 MB/s | 1.8 MB 00:00 2024-09-21T06:58:20.285 INFO:teuthology.orchestra.run.smithi027.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 492 kB/s | 41 kB 00:00 2024-09-21T06:58:20.311 INFO:teuthology.orchestra.run.smithi027.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.8 MB/s | 249 kB 00:00 2024-09-21T06:58:20.319 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-09-21T06:58:20.369 INFO:teuthology.orchestra.run.smithi027.stdout:(155/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.3 MB/s | 3.2 MB 00:00 2024-09-21T06:58:20.428 INFO:teuthology.orchestra.run.smithi027.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-09-21T06:58:20.461 INFO:teuthology.orchestra.run.smithi027.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2024-09-21T06:58:20.467 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:58:20.467 INFO:teuthology.orchestra.run.smithi027.stdout:Total 20 MB/s | 228 MB 00:11 2024-09-21T06:58:20.474 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-09-21T06:58:20.474 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-09-21T06:58:22.305 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-09-21T06:58:22.305 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-09-21T06:58:22.567 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-21T06:58:22.771 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-21T06:58:22.771 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-21T06:58:23.922 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-21T06:58:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:58:24.146 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:58:24.317 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:58:24.350 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:58:24.376 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:58:24.418 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:58:24.749 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:58:24.860 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:58:24.984 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:58:25.011 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:58:25.103 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:58:25.132 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:58:25.161 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:58:25.287 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-21T06:58:25.288 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-21T06:58:25.312 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:58:25.354 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:58:25.383 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:58:25.480 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:58:25.509 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:58:25.554 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:58:25.576 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:58:25.601 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:58:25.769 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:58:25.923 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:58:26.058 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:58:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:26.251 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:26.474 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:26.593 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:58:26.622 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:58:26.657 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:58:26.731 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:58:26.775 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:58:26.806 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:58:26.850 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:58:26.886 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:58:26.913 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:58:26.982 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:58:27.036 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:58:27.066 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:58:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:58:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:58:27.221 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:58:27.264 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:58:27.293 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:58:27.363 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:58:27.407 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:58:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:58:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:58:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:58:28.193 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:58:28.280 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:58:28.309 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:58:28.352 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:58:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:58:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:58:28.597 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:58:28.653 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-09-21T06:58:28.659 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:58:28.751 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:58:28.793 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:58:28.815 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:58:28.897 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:58:28.923 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:58:28.930 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:58:28.966 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:58:29.013 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:58:29.049 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:58:29.052 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:58:29.085 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:58:29.092 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:58:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:58:29.134 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:58:29.153 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:58:29.268 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:58:29.311 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:58:29.352 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:58:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:58:29.537 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:58:29.561 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:58:29.620 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:58:29.747 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:58:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:58:29.771 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:58:29.860 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:58:29.863 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:58:29.885 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:58:29.892 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:58:29.922 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:58:29.962 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:58:30.075 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:58:30.085 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:58:30.125 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:58:30.131 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:58:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:58:30.159 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:58:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:58:30.225 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:58:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:58:30.264 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:58:30.287 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:58:30.293 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:58:30.330 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:58:30.364 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:58:30.364 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:58:30.386 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:58:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:58:30.406 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:58:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:58:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:58:30.570 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:58:30.597 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:58:30.601 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:58:30.702 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:58:30.723 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:58:30.751 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:58:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:58:30.939 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:31.002 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:31.214 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:31.316 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:58:31.348 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:58:31.383 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:58:31.448 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:58:31.463 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:58:31.495 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:58:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:58:31.527 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:58:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:58:31.571 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:58:31.603 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:58:31.655 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:58:31.656 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:58:31.713 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:58:31.725 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:58:31.757 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:58:31.787 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:58:31.817 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:58:31.835 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:58:31.852 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:58:31.885 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:58:31.935 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:58:31.972 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:58:32.016 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:58:32.044 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:58:32.069 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:58:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:58:32.115 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:58:32.160 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:58:32.176 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:58:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-21T06:58:32.234 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:58:32.368 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:58:32.379 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:58:32.420 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:58:32.446 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:58:32.485 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:58:32.529 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:58:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:58:32.628 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:58:32.813 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:58:32.930 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:58:32.992 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:58:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:58:33.075 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:58:33.112 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:58:33.144 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:58:33.153 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:58:33.192 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:58:33.233 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:58:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:58:33.329 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:58:33.387 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:58:33.436 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:58:33.443 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:58:33.475 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:58:33.516 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:58:33.526 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:58:33.600 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:58:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:58:33.712 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:58:33.749 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:58:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:58:33.796 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:58:33.831 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:58:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:58:33.866 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:58:33.874 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:58:33.916 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:58:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:58:34.035 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:58:34.054 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:58:34.093 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:58:34.130 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:58:34.134 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:58:34.137 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:58:34.152 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:58:34.198 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:58:34.228 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:58:34.237 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:58:34.256 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:58:34.270 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:58:34.289 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:58:34.309 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:58:34.420 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:58:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:58:34.569 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:58:34.612 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:58:34.646 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:58:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:58:34.652 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:58:34.742 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:58:34.770 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:58:34.774 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:58:34.800 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:58:34.809 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:58:34.817 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:58:34.844 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:58:34.879 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:58:34.918 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:58:34.922 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:34.950 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:58:34.979 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:58:35.014 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:58:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:35.064 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:58:35.105 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:58:35.140 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:58:35.226 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:58:35.270 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:58:35.305 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:58:35.310 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:58:35.321 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:58:35.338 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:58:35.408 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:58:35.413 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:58:35.431 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:58:35.466 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:58:35.469 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:58:35.499 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:58:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:58:35.630 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:58:35.690 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:58:35.735 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:58:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:58:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:58:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:58:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:58:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:58:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:58:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:58:36.187 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:58:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:58:36.241 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:58:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:58:36.275 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:58:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:58:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:58:36.376 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:58:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:58:36.430 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:58:36.475 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:58:36.487 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:58:36.563 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:58:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:58:36.602 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:58:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:58:36.787 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:58:36.825 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:58:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:58:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:58:37.543 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:58:37.543 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:58:37.657 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:58:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:37.836 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:58:37.871 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:37.875 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:58:37.919 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:58:37.967 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:58:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:58:38.211 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:58:38.270 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:58:38.390 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:58:38.438 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:38.458 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:58:38.472 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:58:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:58:38.566 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:58:38.566 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:58:38.567 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:58:38.567 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:38.599 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:58:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:58:38.649 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:58:38.709 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:58:38.748 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:58:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:58:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:58:38.879 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:58:38.994 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:58:39.028 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:58:39.075 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:58:39.077 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:58:39.121 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:58:39.252 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:58:39.273 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:58:39.302 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:58:39.346 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:58:39.391 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:58:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:58:39.582 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:58:39.709 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:58:39.754 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:58:39.760 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:58:39.802 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:58:39.839 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:58:39.852 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:58:39.873 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:58:39.882 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:58:39.914 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:58:39.917 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:58:39.953 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:58:39.990 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:58:40.025 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:58:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:58:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:58:40.134 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:58:40.240 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:58:40.311 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:58:40.346 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:58:40.416 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:58:40.446 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:58:40.475 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:58:40.517 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:58:40.544 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:58:40.548 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:58:41.505 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:58:41.561 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:58:41.603 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:58:41.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:58:41.787 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:58:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:58:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:58:41.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:58:42.254 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:58:42.286 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:58:42.392 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:58:42.801 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:42.836 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:43.250 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:58:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:58:43.405 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:43.447 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:58:43.541 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:58:43.541 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:58:43.541 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:58:43.541 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:43.574 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:58:45.066 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:58:45.158 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:58:45.195 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:58:45.219 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:58:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:58:45.680 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:58:45.811 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:58:45.853 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:58:45.902 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:58:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:58:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:58:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:58:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:58:49.266 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:49.308 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:50.008 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:58:50.053 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:58:50.191 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:58:50.192 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:58:50.192 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:58:50.192 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:58:50.271 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-09-21T06:58:50.539 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:50.710 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:58:50.726 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:58:50.726 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:50.726 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:58:50.726 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:52.296 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:58:52.323 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:58:52.323 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:52.323 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:58:52.323 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:58:52.323 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:58:52.323 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:52.547 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:58:52.570 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:58:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:58:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:58:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:58:52.570 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:53.039 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:58:53.062 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:58:53.062 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:53.062 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:58:53.062 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:58:53.062 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:58:53.062 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:53.161 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:58:53.209 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:58:53.264 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:58:53.448 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:58:53.498 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:58:53.567 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:58:53.659 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:58:53.796 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:58:53.831 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:58:53.864 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:58:53.900 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:58:54.242 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:58:54.273 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:58:54.315 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:58:54.378 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:58:54.404 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:58:55.542 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-09-21T06:58:55.543 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:55.726 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:58:55.742 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:58:55.742 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:55.742 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:58:55.742 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:57.346 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:58:57.372 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:58:57.372 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:57.372 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:58:57.373 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:58:57.373 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:58:57.373 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:58:57.604 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:58:57.627 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:58:57.627 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:58:57.628 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:57.628 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:58:57.628 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:58:57.628 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:58:57.628 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:58.097 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:58:58.120 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:58:58.121 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:58.121 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:58:58.121 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:58:58.121 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:58:58.121 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:58:58.244 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:58:58.300 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:58:58.356 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:58:58.542 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:58:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:58:58.597 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:58:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:58:58.675 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:58:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:58:58.763 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:58:58.895 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:58:58.902 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:58:58.921 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:58:58.939 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:58:58.944 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:58:58.945 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:58:58.945 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:58:58.945 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:58:58.945 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:58:58.946 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:58:58.974 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:58:58.989 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:58:59.000 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:58:59.007 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:58:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:58:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:58:59.467 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:58:59.497 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:58:59.529 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:58:59.595 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:58:59.620 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:59:00.109 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:59:00.141 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:59:00.164 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:59:00.164 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:00.164 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:59:00.165 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:59:00.165 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:59:00.165 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:59:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:59:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:59:00.231 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:00.231 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:59:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:59:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:59:00.298 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:59:00.298 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:00.298 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:59:00.298 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:59:00.299 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:59:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:59:00.617 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:59:00.641 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:59:00.641 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:00.641 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:59:00.641 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:59:00.641 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:59:00.641 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:59:02.833 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:59:02.884 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-09-21T06:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-09-21T06:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-09-21T06:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-09-21T06:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-09-21T06:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-09-21T06:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-09-21T06:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:03.860 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:59:03.893 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:59:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:59:03.954 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:59:03.954 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:03.955 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:59:03.955 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:04.007 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:59:04.178 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:59:04.211 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:59:04.235 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:59:04.235 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:04.235 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:59:04.235 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:59:04.235 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:59:04.236 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:59:04.287 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:59:04.299 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:59:04.397 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:59:04.443 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:59:04.614 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:59:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:59:04.695 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:59:04.727 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:59:04.754 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:59:04.929 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:59:04.976 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:59:05.006 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:59:05.036 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:59:05.082 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:59:05.405 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:59:05.431 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:59:05.455 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:59:05.456 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:05.456 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:59:05.456 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:59:05.456 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:59:05.456 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:59:05.498 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:59:05.521 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:59:05.522 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:05.522 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:59:05.522 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:59:05.574 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:59:05.593 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:05.597 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:59:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:59:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:59:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:59:05.598 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:59:05.630 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:05.650 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:05.753 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:59:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:59:05.813 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:05.813 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:59:05.813 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:59:05.813 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:59:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:59:05.948 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:59:05.948 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:05.948 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:59:05.949 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:59:05.949 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:59:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:59:06.078 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:59:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:59:06.132 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:06.132 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:59:06.132 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:59:06.132 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:59:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:06.467 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:59:06.542 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:59:06.584 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:59:06.664 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:59:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:59:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:59:06.721 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:59:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:59:06.731 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:06.731 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:59:06.732 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:59:06.732 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:59:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:59:06.747 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:59:06.814 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:59:06.814 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:59:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:59:06.881 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:59:06.906 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:59:06.912 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:06.912 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:59:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:59:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:07.100 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:59:07.184 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:59:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:59:07.265 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:59:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:59:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:59:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:59:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:59:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:59:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:59:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:59:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:59:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:59:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:59:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:59:10.004 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:59:10.045 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:59:10.094 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:59:10.126 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:59:10.153 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:59:10.334 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:59:10.383 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:59:10.413 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:59:10.443 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:59:10.481 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:59:11.062 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:11.103 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:11.124 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:11.954 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:59:12.158 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:59:12.199 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:59:12.281 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:59:12.311 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:59:12.336 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:59:12.362 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:59:12.362 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:59:12.446 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:59:12.446 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:59:12.530 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:59:12.531 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:59:12.555 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:59:12.560 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:12.560 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:12.593 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:59:12.782 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:12.783 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:59:12.875 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:59:12.938 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:59:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:59:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:59:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:59:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:59:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:59:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:59:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:59:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:59:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:59:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:59:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:59:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:59:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:59:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:59:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:59:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:59:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:59:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:59:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:59:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:59:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:59:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:59:13.748 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:59:13.778 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:59:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:59:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:59:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:59:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:59:14.218 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:14.219 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:59:14.219 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:59:14.219 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:59:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:59:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:59:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:59:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:59:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:59:15.632 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:59:15.685 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:59:15.685 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:15.685 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:59:15.686 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:59:15.686 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:59:15.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:59:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:59:15.793 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:15.793 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:59:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:15.858 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:59:15.912 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:59:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:59:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:59:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:59:15.913 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:15.927 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:15.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:59:15.927 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-09-21T06:59:15.928 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:15.928 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:15.928 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:15.928 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:15.928 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.928 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.928 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:15.929 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:15.929 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-21T06:59:15.929 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:59:15.929 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:59:15.929 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:59:15.929 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:59:15.929 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.930 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.930 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.930 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.930 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.930 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.930 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.930 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.931 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.932 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:59:15.933 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:59:15.934 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:59:15.934 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:59:15.934 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.934 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.934 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.934 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:59:15.935 INFO:teuthology.orchestra.run.smithi119.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:59:15.935 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:59:15.936 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:59:15.936 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:59:15.936 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:59:15.936 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.936 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.936 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:59:15.937 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.937 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:59:15.937 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:59:15.937 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:59:15.937 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:59:15.937 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:59:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:59:15.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:59:15.940 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:59:15.941 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:59:15.941 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:59:15.941 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:59:15.941 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:59:15.942 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:59:15.942 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:59:15.942 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:59:15.942 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:59:15.942 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:59:15.942 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:59:15.942 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:59:15.943 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:59:15.944 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:59:15.945 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:59:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:59:15.947 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:59:15.948 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:59:15.948 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T06:59:15.948 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-21T06:59:16.284 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:59:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:59:16.337 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:59:16.337 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:59:16.338 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:59:16.338 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:59:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:16.389 DEBUG:teuthology.parallel:result is None 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:59:18.976 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:59:18.977 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:59:18.978 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:59:18.979 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:59:18.980 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:59:18.981 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:59:18.982 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:59:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:59:18.984 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:59:18.985 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:59:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:59:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:59:18.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:59:18.988 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:59:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:59:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:59:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:59:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:59:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:59:21.422 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:59:21.468 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:59:21.497 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:59:21.529 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:59:21.565 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:59:21.615 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:21.615 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:59:21.615 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-09-21T06:59:21.615 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:21.615 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:21.615 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.616 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.617 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:59:21.618 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:59:21.619 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:21.620 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.621 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:59:21.622 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:59:21.623 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:59:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:59:21.625 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-09-21T06:59:22.045 DEBUG:teuthology.parallel:result is None 2024-09-21T06:59:22.350 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:59:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:59:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:59:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:59:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:59:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:59:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:59:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:59:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:59:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:59:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:59:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:59:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:59:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:24.786 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:59:24.786 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:59:24.895 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:59:24.990 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:25.209 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:25.209 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T06:59:25.209 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:31.357 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:59:31.357 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:59:31.357 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:59:31.357 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:59:31.357 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:59:31.358 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:59:31.359 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:59:31.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:59:31.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:59:31.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:59:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:59:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:59:31.375 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:59:31.376 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:59:31.378 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:59:31.379 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:59:33.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-21T06:59:33.644 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.648 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:59:33.649 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:59:33.653 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:59:33.654 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:59:33.656 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:59:33.656 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:59:33.656 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T06:59:33.656 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-21T06:59:34.270 DEBUG:teuthology.parallel:result is None 2024-09-21T06:59:34.271 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:59:34.411 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:59:34.459 INFO:teuthology.orchestra.run.smithi027.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:59:34.460 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:59:34.460 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:59:34.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:59:34.599 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:59:34.625 INFO:teuthology.orchestra.run.smithi119.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:59:34.625 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:59:34.625 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:59:34.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:59:34.761 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:59:34.787 INFO:teuthology.orchestra.run.smithi153.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:59:34.788 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:59:34.788 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:59:34.790 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T06:59:34.790 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T06:59:34.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:59:34.851 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T06:59:34.852 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:59:34.883 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T06:59:34.884 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:59:34.914 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T06:59:34.915 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T06:59:34.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:59:34.978 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:59:35.077 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T06:59:35.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:59:35.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:59:35.176 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T06:59:35.176 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:59:35.201 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:59:35.267 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T06:59:35.267 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T06:59:35.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:59:35.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:59:35.427 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T06:59:35.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:59:35.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:59:35.517 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T06:59:35.517 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:59:35.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:59:35.607 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T06:59:35.607 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T06:59:35.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:59:35.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:59:35.764 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T06:59:35.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:59:35.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:59:35.854 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T06:59:35.854 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:59:35.878 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:59:35.942 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T06:59:36.043 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-21T06:59:36.043 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:59:36.043 INFO:tasks.cephadm:Cluster fsid is 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T06:59:36.043 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T06:59:36.044 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.119', 'mon.c': '172.21.15.153'} 2024-09-21T06:59:36.044 INFO:tasks.cephadm:First mon is mon.a on smithi027 2024-09-21T06:59:36.044 INFO:tasks.cephadm:First mgr is x 2024-09-21T06:59:36.044 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T06:59:36.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-09-21T06:59:36.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2024-09-21T06:59:36.126 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-09-21T06:59:36.151 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T06:59:36.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:59:36.296 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T06:59:36.435 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:59:36.439 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:59:36.439 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:59:36.439 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:59 /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:37.008 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:37.564 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:59 /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:37.564 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:38.181 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:59 /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:38.181 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:38.224 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:38.244 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:59:38.279 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T06:59:38.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:59:38.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:59:38.287 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:59:39.883 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:59:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:59:40.195 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T07:00:15.201 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:00:15.201 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T07:00:15.201 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T07:00:15.201 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-09-21T07:00:15.201 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T07:00:15.201 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:00:15.201 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:00:28.891 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T07:00:28.892 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T07:00:28.892 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T07:00:28.892 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2024-09-21T07:00:28.892 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T07:00:28.892 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-21T07:00:28.892 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T07:00:33.910 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-21T07:00:33.910 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T07:00:33.910 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T07:00:33.910 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-09-21T07:00:33.910 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T07:00:33.911 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-09-21T07:00:33.911 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-21T07:00:33.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-09-21T07:00:34.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2024-09-21T07:00:34.035 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-09-21T07:00:34.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-09-21T07:00:34.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2024-09-21T07:00:34.152 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-09-21T07:00:34.177 INFO:tasks.cephadm:Writing seed config... 2024-09-21T07:00:34.178 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T07:00:34.178 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-21T07:00:34.178 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T07:00:34.178 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T07:00:34.178 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T07:00:34.178 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T07:00:34.179 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T07:00:34.180 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T07:00:34.180 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T07:00:34.180 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T07:00:34.180 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T07:00:34.180 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:00:34.180 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T07:00:34.217 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0ff1e276-77e7-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 2024-09-21T07:00:34.218 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T07:00:34.262 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T07:00:34.307 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T07:00:34.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:00:34.572 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:00:34.572 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2024-09-21T07:00:34.573 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T07:00:34.573 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2024-09-21T07:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.2.2 2024-09-21T07:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2024-09-21T07:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2024-09-21T07:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T07:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T07:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T07:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T07:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-09-21T07:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-09-21T07:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-09-21T07:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2024-09-21T07:00:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.2.2 2024-09-21T07:00:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T07:00:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2024-09-21T07:00:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2024-09-21T07:00:34.734 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T07:00:34.734 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T07:00:34.747 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T07:00:34.747 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T07:00:34.761 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-09-21T07:00:34.773 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-09-21T07:00:34.773 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-09-21T07:00:34.773 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2024-09-21T07:00:34.773 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:34.774 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 140099924608432 on /run/cephadm/0ff1e276-77e7-11ef-baf5-efdc52797490.lock 2024-09-21T07:00:34.774 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 140099924608432 acquired on /run/cephadm/0ff1e276-77e7-11ef-baf5-efdc52797490.lock 2024-09-21T07:00:34.774 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2024-09-21T07:00:34.775 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2024-09-21T07:00:34.775 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T07:00:34.782 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.27 metric 100 2024-09-21T07:00:34.782 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2024-09-21T07:00:34.788 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T07:00:34.788 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T07:00:34.788 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1749sec hoplimit 64 pref medium 2024-09-21T07:00:34.794 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T07:00:34.794 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T07:00:34.794 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T07:00:34.794 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T07:00:34.794 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T07:00:34.795 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T07:00:34.795 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-09-21T07:00:34.795 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-09-21T07:00:34.795 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T07:00:34.796 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T07:00:34.797 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T07:00:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T07:00:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T07:00:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T07:00:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T07:00:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T07:00:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T07:00:35.371 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T07:00:36.515 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:00:36.516 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:00:36.516 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T07:00:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2024-09-21T07:00:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2024-09-21T07:00:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQAWb+5mWetVABAAZRGEev0KWnD+hRUwchYdtQ== 2024-09-21T07:00:39.692 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQAXb+5ml16QBRAA0yjJwMVJLzQ78LrLUSiRKw== 2024-09-21T07:00:40.731 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQAYb+5mOOD9CRAA4eB+a3MUmnQCqRsPoD54YQ== 2024-09-21T07:00:40.732 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2024-09-21T07:00:41.843 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T07:00:41.843 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:00:41.844 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2024-09-21T07:00:42.875 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2024-09-21T07:00:43.217 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T07:00:43.454 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T07:00:43.720 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490.target. 2024-09-21T07:00:43.720 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490.target. 2024-09-21T07:00:44.055 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a 2024-09-21T07:00:44.055 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T07:00:44.326 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490.target.wants/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@.service. 2024-09-21T07:00:44.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:44 smithi027 podman[30389]: 2024-09-21 07:00:44.868883793 +0000 UTC m=+0.312210249 container start 55c6c41582f5d15ef1be42a844cce5e0de98c89632cd94acc5fbccd3df94c8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T07:00:44.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:44 smithi027 bash[30389]: 55c6c41582f5d15ef1be42a844cce5e0de98c89632cd94acc5fbccd3df94c8cc 2024-09-21T07:00:44.982 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T07:00:44.982 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T07:00:44.998 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T07:00:44.998 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T07:00:44.998 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T07:00:44.998 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T07:00:44.998 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2024-09-21T07:00:44.998 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2024-09-21T07:00:45.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:44 smithi027 systemd[1]: Started Ceph mon.a for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:00:45.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:45 smithi027 ceph-mon[30408]: mkfs 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:45.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:45 smithi027 ceph-mon[30408]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T07:00:46.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T07:00:46.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: monmap epoch 1 2024-09-21T07:00:46.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:46.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: last_changed 2024-09-21T07:00:41.226200+0000 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: min_mon_release 19 (squid) 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: election_strategy: 1 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: fsmap 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: mgrmap e1: no daemons active 2024-09-21T07:00:46.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:46 smithi027 ceph-mon[30408]: from='client.? 172.21.15.27:0/2763496275' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T07:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T07:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.644624s) 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2024-09-21T07:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T07:00:47.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:47 smithi027 ceph-mon[30408]: from='client.? 172.21.15.27:0/2128708523' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T07:00:47.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:47 smithi027 ceph-mon[30408]: from='client.? 172.21.15.27:0/2128708523' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T07:00:47.952 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T07:00:47.953 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T07:00:47.954 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T07:00:47.954 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T07:00:47.954 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T07:00:47.954 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T07:00:47.954 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2024-09-21T07:00:49.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:48 smithi027 ceph-mon[30408]: from='client.? 172.21.15.27:0/3118369285' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:00:49.331 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2024-09-21T07:00:49.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:49 smithi027 systemd[1]: Stopping Ceph mon.a for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:00:49.994 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a[30404]: 2024-09-21T07:00:49.679+0000 7f10f1e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:00:49.995 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a[30404]: 2024-09-21T07:00:49.679+0000 7f10f1e00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T07:00:49.995 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:49 smithi027 podman[30660]: 2024-09-21 07:00:49.721534727 +0000 UTC m=+0.153046740 container died 55c6c41582f5d15ef1be42a844cce5e0de98c89632cd94acc5fbccd3df94c8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:00:50.244 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 podman[30660]: 2024-09-21 07:00:50.242051563 +0000 UTC m=+0.673563576 container cleanup 55c6c41582f5d15ef1be42a844cce5e0de98c89632cd94acc5fbccd3df94c8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T07:00:50.244 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 bash[30660]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a 2024-09-21T07:00:50.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 podman[30672]: 2024-09-21 07:00:50.401405464 +0000 UTC m=+0.667620132 container remove 55c6c41582f5d15ef1be42a844cce5e0de98c89632cd94acc5fbccd3df94c8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T07:00:50.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 systemd[1]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T07:00:50.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 systemd[1]: Stopped Ceph mon.a for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:00:50.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 systemd[1]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a.service: Consumed 1.317s CPU time. 2024-09-21T07:00:50.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 systemd[1]: Starting Ceph mon.a for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:00:51.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 podman[30761]: 2024-09-21 07:00:50.886744495 +0000 UTC m=+0.028255935 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:00:51.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:50 smithi027 podman[30761]: 2024-09-21 07:00:50.985436821 +0000 UTC m=+0.126948250 container create cd5196f7b24b359047ef0e17b8abab361d2b49b8047e84599ebeeb4a6df11e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T07:00:51.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 podman[30761]: 2024-09-21 07:00:51.179785597 +0000 UTC m=+0.321297049 container init cd5196f7b24b359047ef0e17b8abab361d2b49b8047e84599ebeeb4a6df11e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:00:51.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 podman[30761]: 2024-09-21 07:00:51.186303561 +0000 UTC m=+0.327814990 container start cd5196f7b24b359047ef0e17b8abab361d2b49b8047e84599ebeeb4a6df11e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T07:00:51.273 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T07:00:51.564 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 bash[30761]: cd5196f7b24b359047ef0e17b8abab361d2b49b8047e84599ebeeb4a6df11e43 2024-09-21T07:00:51.564 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 systemd[1]: Started Ceph mon.a for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:00:51.564 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T07:00:51.564 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T07:00:51.564 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: pidfile_write: ignore empty --pid-file 2024-09-21T07:00:51.564 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: load: jerasure load: lrc 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: RocksDB version: 7.9.2 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Git sha 0 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: DB SUMMARY 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: DB Session ID: JKQP8R3TA30S3GMMES39 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: CURRENT file: CURRENT 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94864 ; 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.error_if_exists: 0 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.create_if_missing: 0 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.paranoid_checks: 1 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T07:00:51.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.env: 0x55c763a24d60 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.info_log: 0x55c775a9b320 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.statistics: (nil) 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.use_fsync: 0 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_log_file_size: 0 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.allow_fallocate: 1 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T07:00:51.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T07:00:51.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.use_direct_reads: 0 2024-09-21T07:00:51.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T07:00:51.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T07:00:51.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.db_log_dir: 2024-09-21T07:00:51.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.wal_dir: 2024-09-21T07:00:51.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T07:00:51.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T07:00:51.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T07:00:51.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T07:00:51.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T07:00:51.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.write_buffer_manager: 0x55c775a9fd60 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.unordered_write: 0 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T07:00:51.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.row_cache: None 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.wal_filter: None 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.two_write_queues: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.wal_compression: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.atomic_flush: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.log_readahead_size: 0 2024-09-21T07:00:51.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_background_jobs: 2 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_background_compactions: -1 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_subcompactions: 1 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T07:00:51.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_open_files: -1 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_background_flushes: -1 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Compression algorithms supported: 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kZSTD supported: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kXpressCompression supported: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kBZip2Compression supported: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T07:00:51.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kLZ4Compression supported: 1 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kZlibCompression supported: 1 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: kSnappyCompression supported: 1 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.merge_operator: 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_filter: None 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T07:00:51.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c775a9af60) 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T07:00:51.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x55c771a5f610 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2024-09-21T07:00:51.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2024-09-21T07:00:51.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression: NoCompression 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.num_levels: 7 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T07:00:51.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T07:00:51.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T07:00:51.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T07:00:51.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.inplace_update_support: 0 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T07:00:51.581 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.bloom_locality: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.max_successive_merges: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.ttl: 2592000 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.enable_blob_files: false 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.min_blob_size: 0 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T07:00:51.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 42026786-fc55-40f1-b630-0f1a3dada22e 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902051299031, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902051301322, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 89154, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11607, "raw_average_key_size": 47, "raw_value_size": 82554, "raw_average_value_size": 341, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726902051, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "42026786-fc55-40f1-b630-0f1a3dada22e", "db_session_id": "JKQP8R3TA30S3GMMES39", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902051301617, "job": 1, "event": "recovery_finished"} 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T07:00:51.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c775ac0a80 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: DB pointer 0x55c776862000 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T07:00:51.584 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 90.83 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 90.83 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 60.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T07:00:51.585 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 11.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 11.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T07:00:51.586 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x55c771a5f610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???) e1 preinit fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???).mds e1 new map 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???).mds e1 print_map 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: btime 2024-09-21T07:00:45:000882+0000 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T07:00:51.587 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: monmap epoch 1 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: last_changed 2024-09-21T07:00:41.226200+0000 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: min_mon_release 19 (squid) 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: election_strategy: 1 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: fsmap 2024-09-21T07:00:51.588 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T07:00:51.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:51 smithi027 ceph-mon[30779]: mgrmap e1: no daemons active 2024-09-21T07:00:52.585 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T07:00:52.611 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:00:52.611 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2024-09-21T07:00:52.611 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T07:00:52.612 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T07:00:52.961 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.x 2024-09-21T07:00:52.961 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T07:00:53.200 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490.target.wants/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@.service. 2024-09-21T07:00:53.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:52 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1491556377' entity='client.admin' 2024-09-21T07:00:53.546 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:00:53 smithi027 systemd[1]: Starting Ceph mgr.x for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:00:53.833 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:00:53 smithi027 podman[30989]: 2024-09-21 07:00:53.449509927 +0000 UTC m=+0.027920479 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:00:53.833 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:00:53 smithi027 podman[30989]: 2024-09-21 07:00:53.554143849 +0000 UTC m=+0.132554410 container create 56905a59dc48dac07207282cc6b01789f213fe62765f915a7b19112ed0181706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T07:00:53.833 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:00:53 smithi027 podman[30989]: 2024-09-21 07:00:53.748279372 +0000 UTC m=+0.326689924 container init 56905a59dc48dac07207282cc6b01789f213fe62765f915a7b19112ed0181706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:00:53.833 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:00:53 smithi027 podman[30989]: 2024-09-21 07:00:53.754304473 +0000 UTC m=+0.332715034 container start 56905a59dc48dac07207282cc6b01789f213fe62765f915a7b19112ed0181706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:00:53.833 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:00:53 smithi027 bash[30989]: 56905a59dc48dac07207282cc6b01789f213fe62765f915a7b19112ed0181706 2024-09-21T07:00:53.861 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T07:00:53.861 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T07:00:53.876 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T07:00:53.876 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T07:00:53.876 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T07:00:53.876 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T07:00:53.894 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T07:00:53.894 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T07:00:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T07:00:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T07:00:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T07:00:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T07:00:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2024-09-21T07:00:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2024-09-21T07:00:54.091 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:00:53 smithi027 systemd[1]: Started Ceph mgr.x for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:00:54.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:54 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2683695573' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:55.254 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:55.255 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T07:00:55.256 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T07:00:45:000882+0000", 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T07:00:55.257 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T07:00:45.001933+0000", 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:00:55.258 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2024-09-21T07:00:58.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:00:57 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3611836067' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T07:00:58.503 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:58.504 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T07:00:58.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:58.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T07:00:58.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T07:00:58.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T07:00:58.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T07:00:58.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T07:00:58.505 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T07:00:58.508 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:58.508 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T07:00:58.508 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T07:00:58.508 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T07:00:58.508 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T07:00:58.508 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T07:00:58.508 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T07:00:45:000882+0000", 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T07:00:58.509 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T07:00:45.001933+0000", 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:00:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: Activating manager daemon x 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: mgrmap e2: x(active, starting, since 0.00549125s) 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T07:01:00.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: Manager daemon x is now available 2024-09-21T07:01:00.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T07:01:00.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T07:01:00.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' 2024-09-21T07:01:00.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' 2024-09-21T07:01:00.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:00 smithi027 ceph-mon[30779]: from='mgr.14100 172.21.15.27:0/3038492019' entity='mgr.x' 2024-09-21T07:01:01.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:01 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1742285687' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T07:01:01.429 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:01 smithi027 ceph-mon[30779]: mgrmap e3: x(active, since 1.00995s) 2024-09-21T07:01:01.816 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:01:01.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T07:01:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T07:01:01.819 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T07:01:01.819 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T07:01:01.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:01.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T07:01:01.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T07:01:01.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T07:01:01.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T07:00:45:000882+0000", 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T07:01:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T07:00:45.001933+0000", 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T07:01:01.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:01:01.825 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (3/15)... 2024-09-21T07:01:03.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:03 smithi027 ceph-mon[30779]: mgrmap e4: x(active, since 2s) 2024-09-21T07:01:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:04 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4056920341' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T07:01:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:01:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:01:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:01:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T07:01:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T07:01:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T07:00:45:000882+0000", 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T07:01:05.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T07:00:45.001933+0000", 2024-09-21T07:01:05.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T07:01:05.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T07:01:05.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T07:01:05.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:01:05.331 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2024-09-21T07:01:06.387 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:06 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2054544318' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T07:01:06.714 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T07:01:06.715 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T07:01:06.716 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T07:01:06.716 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T07:01:06.716 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T07:01:06.716 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T07:01:06.716 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T07:01:06.716 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2024-09-21T07:01:08.150 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:07 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1194536316' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T07:01:08.150 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:08 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ignoring --setuser ceph since I am not root 2024-09-21T07:01:08.150 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:08 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ignoring --setgroup ceph since I am not root 2024-09-21T07:01:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:08 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1194536316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T07:01:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:08 smithi027 ceph-mon[30779]: mgrmap e5: x(active, since 7s) 2024-09-21T07:01:09.924 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:09 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3610618464' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-09-21T07:01:09.933 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: Active manager daemon x restarted 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: Activating manager daemon x 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: mgrmap e6: x(active, starting, since 0.00466654s) 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T07:01:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T07:01:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T07:01:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:14 smithi027 ceph-mon[30779]: Manager daemon x is now available 2024-09-21T07:01:16.013 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:01:16.013 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T07:01:16.013 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T07:01:16.014 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:01:16.014 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2024-09-21T07:01:16.014 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying orchestrator module is enabled... 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: mgrmap e7: x(active, since 1.00766s) 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: Found migration_current of "None". Setting to last migration. 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:16.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T07:01:16.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:16 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T07:01:17.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-mon[30779]: mgrmap e8: x(active, since 2s) 2024-09-21T07:01:17.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1897758266' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T07:01:18.125 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T07:01:18.126 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2024-09-21T07:01:18.266 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:01:17.909+0000 7f9261e00640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:07:01:17] ENGINE Error in HTTPServer.serve 2024-09-21T07:01:18.266 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: Traceback (most recent call last): 2024-09-21T07:01:18.266 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._connections.run(self.expiration_interval) 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._run(expiration_interval) 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: s = self.context.wrap_socket( 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: return self.sslsocket_class._create( 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:01:18.267 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self.do_handshake() 2024-09-21T07:01:18.268 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:01:18.268 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._sslobj.do_handshake() 2024-09-21T07:01:18.268 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:01:18.268 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:01:18.521 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1897758266' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: mgrmap e9: x(active, since 3s) 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:17] ENGINE Bus STARTING 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:17] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:17] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:17] ENGINE Bus STARTED 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:18 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:17] ENGINE Error in HTTPServer.serve 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T07:01:18.522 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T07:01:18.523 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:01:20.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:19 smithi027 ceph-mon[30779]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:20.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:19 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:20.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:19 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:20.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T07:01:20.946 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2024-09-21T07:01:21.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:21 smithi027 ceph-mon[30779]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-mon[30779]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-mon[30779]: Generating ssh key... 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: Generating public/private rsa key pair. 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: Your identification has been saved in /tmp/tmpq_t0b473/key 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: Your public key has been saved in /tmp/tmpq_t0b473/key.pub 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: The key fingerprint is: 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: SHA256:RVar5t865ODuFZWHbFUMiyDfES0dDyLRe1YxRLXIRiw ceph-0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: The key's randomart image is: 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: +---[RSA 3072]----+ 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | . *+*=*OB| 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | = E=B+B=| 2024-09-21T07:01:22.342 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | o =*Boo| 2024-09-21T07:01:22.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | . .ooo. | 2024-09-21T07:01:22.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | S o .o | 2024-09-21T07:01:22.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | o. .. | 2024-09-21T07:01:22.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | ..+. | 2024-09-21T07:01:22.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | .oo. | 2024-09-21T07:01:22.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: | oo oo. | 2024-09-21T07:01:22.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:22 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: +----[SHA256]-----+ 2024-09-21T07:01:23.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:23 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:23.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:23 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:24.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7ecjv5ww6ZDShW0zcRgr9s8qnzys7uL/z0gJ9Nsm/3EIjNT04OYJtCS29oGsBM/HQxEko6zRqZ9UFF5VenOaBXXlOawGyU7rSq59weD2042XDLTXYNv96jex/Jm8fBcKXt7UZB2dVpgAa3wWHJVsFb/SmsookYaUIEMotH2fGIPSgR/ERMZE23emn8yUeGhdqKnro7tOKBYkiI3RkSyWtoUVfe4C8bNysoRdH2LM0VJPaA9fqEUwIFOt+J7ggf+V+8GZkvL14d9fKVljlNz6wEsNL0l3KxVSHILSCz9sP497w+mNWiXLLqFA3GizFfKoHcn7M6cuZ/O4ZLGBax+XqlpEH0mPlvR6kbvP1PCtJ3/i3Y3iZoLKOiQwcom/gzyqBaIiiuif5+W7Hi4RPrz+aN4RU1DUS+Bv5pOszCJG0j/ik+hYf34h3Wwdthnc5JeDQ7jYYAwEerouCfWY9IOdJuYNDeUNgZcRDzBaZqjqj0Hzi/1MpCEcDB7jM7i7RpbM= ceph-0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:01:24.319 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T07:01:24.319 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T07:01:24.329 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2024-09-21T07:01:24.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:24 smithi027 ceph-mon[30779]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:26.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:26 smithi027 ceph-mon[30779]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:27.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:27 smithi027 ceph-mon[30779]: Deploying cephadm binary to smithi027 2024-09-21T07:01:29.958 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2024-09-21T07:01:29.959 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2024-09-21T07:01:30.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:30 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:30.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:30 smithi027 ceph-mon[30779]: Added host smithi027 2024-09-21T07:01:30.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:30 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:31.929 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T07:01:31.929 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2024-09-21T07:01:32.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:31 smithi027 ceph-mon[30779]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:32.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:31 smithi027 ceph-mon[30779]: Saving service mon spec with placement count:5 2024-09-21T07:01:32.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:31 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:33.602 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T07:01:34.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:33 smithi027 ceph-mon[30779]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:34.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:33 smithi027 ceph-mon[30779]: Saving service mgr spec with placement count:2 2024-09-21T07:01:34.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:33 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:34.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:33 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:34.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:33 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:35.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:35 smithi027 ceph-mon[30779]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:01:35.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:35 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1342042504' entity='client.admin' 2024-09-21T07:01:37.026 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:36 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:37.026 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:36 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1978803607' entity='client.admin' 2024-09-21T07:01:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2024-09-21T07:01:38.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:37 smithi027 ceph-mon[30779]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:01:38.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:37 smithi027 ceph-mon[30779]: from='mgr.14120 172.21.15.27:0/2577317757' entity='mgr.x' 2024-09-21T07:01:38.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:38 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2600476321' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T07:01:39.236 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:39 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ignoring --setuser ceph since I am not root 2024-09-21T07:01:39.237 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:39 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ignoring --setgroup ceph since I am not root 2024-09-21T07:01:40.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:39 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2600476321' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T07:01:40.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:39 smithi027 ceph-mon[30779]: mgrmap e10: x(active, since 24s) 2024-09-21T07:01:41.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:40 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3040066922' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T07:01:41.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:01:41.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T07:01:41.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T07:01:41.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T07:01:41.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T07:01:41.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:01:41.111 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-09-21T07:01:41.111 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 10... 2024-09-21T07:01:45.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: Active manager daemon x restarted 2024-09-21T07:01:45.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: Activating manager daemon x 2024-09-21T07:01:45.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T07:01:45.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: mgrmap e11: x(active, starting, since 0.00488803s) 2024-09-21T07:01:45.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:01:45.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T07:01:45.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T07:01:45.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T07:01:45.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T07:01:45.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:45 smithi027 ceph-mon[30779]: Manager daemon x is now available 2024-09-21T07:01:46.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:46.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T07:01:46.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T07:01:46.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:46 smithi027 ceph-mon[30779]: mgrmap e12: x(active, since 1.00878s) 2024-09-21T07:01:46.977 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T07:01:46.977 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T07:01:46.978 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T07:01:46.978 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T07:01:46.978 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 10 is available 2024-09-21T07:01:46.978 INFO:teuthology.orchestra.run.smithi027.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T07:01:47.237 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:01:47.188+0000 7feed1600640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:07:01:47] ENGINE Error in HTTPServer.serve 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: Traceback (most recent call last): 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._connections.run(self.expiration_interval) 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._run(expiration_interval) 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: s = self.context.wrap_socket( 2024-09-21T07:01:47.238 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:01:47.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: return self.sslsocket_class._create( 2024-09-21T07:01:47.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:01:47.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self.do_handshake() 2024-09-21T07:01:47.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:01:47.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._sslobj.do_handshake() 2024-09-21T07:01:47.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:01:47.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:01:47.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:46] ENGINE Bus STARTING 2024-09-21T07:01:47.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:47 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:47] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T07:01:48.511 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:48 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:47] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T07:01:48.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:48 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:47] ENGINE Bus STARTED 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:48 smithi027 ceph-mon[30779]: [21/Sep/2024:07:01:47] ENGINE Error in HTTPServer.serve 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:01:48.550 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:48 smithi027 ceph-mon[30779]: mgrmap e13: x(active, since 2s) 2024-09-21T07:01:48.551 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:48 smithi027 ceph-mon[30779]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:48.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:48 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:01:49.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUBiakwS9iVvo3Sf2QDdQPrBHzAA0wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTA3MDE0OFoXDTM0\nMDkyMjA3MDE0OFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjI3MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEA0MOsYOTd91DCQaPu3wLuseMLJJ+Xn1gnaKF5\n9DRPJJfmjjS90oaJRUYez/5J8pwv6xtH+WP96QAIhHNbG3hA57M9oDsV4VS2nLzr\nZEjgtn3wxT++4zSfRgvqfTugs8CYZ2wXjhd7Wc+/FgC6MviscNy0F2UnUzvBWDYi\nacIyd7TIC9eFOTbRGYOAuSP0RRyMRMWapBQdbeoU/3Q+IXF5uCjEsO02XkHld0Ww\nN6MERTTOoCcRVUFzvRHjgivuTxn7F7YOREspkUGF7Zhpr2rd6xnviiHBojNQcfu6\n/v9St6tzuB8jEdBmqj4iH0l+HhJz0DgIXAE5NCE5+lAXPWNhtEVdII3IVuk06m6v\nG2MsADI48ItNX60/Fjiceby3UpndUfejsgz1jd2goz6CPaeO7bBgw8OdmmdSq2j3\neypjumzR/kyql6/WvpcV8RmJadvuXL8/TU218tIqa3A4vS8nk9dRyyehwR/4OE0U\ntNjuksbm4q96RYprYZCebA6ga9j64Xdhc+94wdgBL/ckysmqeKQqOJ2RomkfTLmP\nojtQQds+AyWsrd7Ax/FKL5C0DiPIfT3gQYy5vH7FUVoVwALlmNkOYjWTdznECmV+\niaAtac+yjcR9fS4Nedxamh3htDy8TDadptuUoQpKa+L/jmj84fs1Bcx0m3gO9uIp\nEzOTfwUCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDI3ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPGzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAY\nRw4VhOw/TrUYvjqe19NcJu4e/JMDRrTxyk8CzPkWoYHYQOIxMY76qjNXj56Ddej1\n+iIz2aBEU5xIz5EAq/YSiq5m9RlA+9gUNCpSZHecXG4i+sq/tKzRel4dce+jykwJ\nY7hMhj055sEyflbFSMmoHlscycG/xyJ4u9qa6b9hsiwCU3qSWZT1tThYzzn4YXBF\nEi7wBL0dwXc7SggDKv3z0SckG55OMdSOox7VDoE3E2GKF3ldASQbRo1I1gTH4O7O\n9luLkURfA3BZ8ntBi+O+Rq+o1EfpkeQN3EEot/1rKk8jjFr4KZaY9rpD09EjlNTG\n0BMuMMAlKx52pFmzDzeB+sh3UQnSnv/Xkwy+psgdCaq6V0EmXFSkIzrUsTn8NRKs\n45Dec08v6oie57O39MhOr8GrO+Jx9BMSQAkPYdPQICB0uZgoVjvpm9KUx0KNkfBO\n/YplWgKu6xijTcTD+JR52YtrjLJJu+WNiBy+hfPjPKZWjbwrrgkmqFGZNUd8mcOs\nidvpvOEoBIN9s1MsDv+7ONhOJn5+NxsKKTBSK0xp5HWqdB2RaPrqE5N1orqUkAGt\nGX8Z7dEvVv4Mqtzj08pVWbSjk5wW1lPz2zG1oyMGWpPD114TIPdyusy/r4KKnO/7\nKMKEz0whHj5Q0FKUyWfiznvtnHvlDOwlPU9b3d9hxg==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEA0MOsYOTd91DCQaPu3wLuseMLJJ+Xn1gnaKF59DRPJJfmjjS9\n0oaJRUYez/5J8pwv6xtH+WP96QAIhHNbG3hA57M9oDsV4VS2nLzrZEjgtn3wxT++\n4zSfRgvqfTugs8CYZ2wXjhd7Wc+/FgC6MviscNy0F2UnUzvBWDYiacIyd7TIC9eF\nOTbRGYOAuSP0RRyMRMWapBQdbeoU/3Q+IXF5uCjEsO02XkHld0WwN6MERTTOoCcR\nVUFzvRHjgivuTxn7F7YOREspkUGF7Zhpr2rd6xnviiHBojNQcfu6/v9St6tzuB8j\nEdBmqj4iH0l+HhJz0DgIXAE5NCE5+lAXPWNhtEVdII3IVuk06m6vG2MsADI48ItN\nX60/Fjiceby3UpndUfejsgz1jd2goz6CPaeO7bBgw8OdmmdSq2j3eypjumzR/kyq\nl6/WvpcV8RmJadvuXL8/TU218tIqa3A4vS8nk9dRyyehwR/4OE0UtNjuksbm4q96\nRYprYZCebA6ga9j64Xdhc+94wdgBL/ckysmqeKQqOJ2RomkfTLmPojtQQds+AyWs\nrd7Ax/FKL5C0DiPIfT3gQYy5vH7FUVoVwALlmNkOYjWTdznECmV+iaAtac+yjcR9\nfS4Nedxamh3htDy8TDadptuUoQpKa+L/jmj84fs1Bcx0m3gO9uIpEzOTfwUCAwEA\nAQKCAgAMaK0c303ATxkU3MEsmViF01iMtz3Ed3F0OzDsDpXrt2m5ZzInSt7q8rWP\nKG7N0HHauDbpgIONr3ZSJScztUyqWRGfLhPpXmcyXtphDiHpRwgiOV1H1Q9okvm/\nsuUk+hw0/YxS4ltvJbWWgbmPoQ9i8AN2ZolVXbZq32DhR21Hh0PizE87bOI18Yab\nuqLg3scJqXM9MILlTyYrXwX4Z+3Tv/ydBaSXKrSQh+BnFjYBRRfJFjlmB2txQBh1\nWtYh9onhyW0EqzEP1MS4sDgL+L6zJEoEPGhtkz5DYDKnyJZqm3LX7WXgv7HyiZnu\noSgxcmhW5pW8nkf/rL/s/Ii9GeSrPnBcO9sSUibOAtndHBBBmgk98xylFjeDHHbi\nlNvhrIcBGgxLHDGxzXlibGcXXvu7qT8uCIZ88OX/bv2KZgrQN8K8c/50RRcHfau9\n2mZqML1OK1dDfiHP77rdEtav0a8Dm9I7JCPkrn3Ybyg0ZZe1aBWUlsmf20HpxFsR\nk+iFOkQQVwezve7P43AX9CGjiJK3oAR2LZqJ1S21DgQdiboOm5OUc3Vwsy/Vkva/\n8hJAbbfqsGBnWvWs3T0YS70L1TEHMRgd1PV0jUdKP/U3V8+06MMKB/WkYxZwdwLW\nDCxtKNLTqDJr9UYvzS+qqPJ6AKwcAzo9k3t+5sSEF/qQJlQaYQKCAQEA9mNc17zw\nX/m8Kl6xLqyrkPO3lGNggYbF09rxtIt56GuU6/NegeqabUI3+WQ0aO2lYayIGbKV\n07Fh5P0jxxFKA2x/ceWHxGEENTqfTz1f+uhyYZkQWuxLC+Bn1rSmZ7XYhrBBBQH4\nHO/JHf764rNncFGGDN5CIOBzmwqjOi8tI47CGHtaJ6/uIA3QSpIcRnI6B2y+XiqB\nE62Zi6Pvvem/i88AHMwDILf+t5zeIOl2Jg+3OOJYeM8+QoUnYjlYN1JO4ch5hB38\nhwTxRIQ0HG2vYNFOaeITjBiUlYd9iauj9UA1RjmW2pabVef4rBgPuqASzQWFQrMy\nBVHBaQyfhw1gtQKCAQEA2OiRdNVWOT7ZpWtDRcor1Vx8umpckEKgX14Q3wjy2rxR\nj/NpsMbvKACVA6En9l4T1XeGb6AzklpCH4gIwJfaFFykfsWzt1OmB5y4/T4qmImm\noo7SKlTOac5IOkZvq21POLh6LK00AaavdHCb+bUKjyI27CHvT4ixjD50nX8imI2y\nZ+6gQtXzBCaKmvRst1ESkQlu8Kjqvd2Xdw1oCwOVSy69LNT1fKTQRTBldY++iTdI\n16yJqnimdbCaa7KBmzeF5EK5/xWhbEaJ6l6QuUWlC1rnKXjr3kvNfYwI9jrF+Ev8\ndB148Nlpu+lHPg72C8WMwFsmp9lfksgPr62+TXKnEQKCAQBfhaY4/cczbl7FDnEi\n506co+BIZz7C+L05pWiP1oMs2Iij8XZyZ8H3GfUZev/Wrkuv0nPxgosTx3FslbeJ\nNeq76RR/070PJY28FT4uOl07qWKu8+ijV3jA4f9f/imFi7qmfS64Aktu7IfMWU5r\nFaLEqNaPppt8o4guU7F0nVWis3YMh8YsHMj1JGXcg0jzBtX4ivBDEbd5KZSdzrDj\nr/vS+C3S/EU6KhtWlDqKB/kGSJGAPECQlchNgdIDArhwujha0J5L2lRNBF2FcrvB\nScT0hDEiW8O4TM9OKXlCJokqS/pbf3g8xqbyZzzBe5Kdh/HCCqHfehmqdkfFsl8m\nvMkBAoIBACJZIb+IkPDbvQdu5uZET5Cn5rB2a1YE1vSDJtTKy/pLW/cZqqDtv61r\n+izq9jMTTry9ArLmgYQpetMXbZth8IzM5teUYxMi0eXesYSYtZIWKicPysQdTdp/\nafygbsqMssDepaoxe0zPonUj5+8w9g0Yc+4MqaLEmKkGstzF5yqVNa0zonky/Bqq\nJ8TF8/lLg08oWUMO3K1YS8X0f4dO9/NB4X8Z8g3c44YxVb+6FYgREx3kGqC+Hk4A\nu1NWnyN/NVqiHmYAfOFygiB0/SH5n1se+AAfP+qhwlahOEIVqIfbMI0Kpb4uD4pM\nNJXlNJ1EmTdUt9hYmcPannFJaBtHjqECggEBAKzPjjwFxuZWT0gJyovxDZRCch2O\n1lFcRxJs6F89yVpDCL16CS2kGH08/QDWC6JmMQaBym31dUIE8WKapeSx03iU4XME\nkMB9ze5f9u/Dm+Va8d5y9IPElVoTE81a5vc5IY+DP2oWKlCHHj7D3NFL2sY/3EVm\n8RibGlX+Ofe2Yoc+w41MG/Eq6EgJ2FvodTNrwBKvX3oUJtpc5GCYLQASlZP7AxGQ\nQicEs9pGQVvoaWdyMT4aiWvrhkbSKVWBcsKkZEr4IuNABJ0Ula3ciBit+L1/urFR\noqpCtSVg6GHGSV2J5LCMeKu8X/Gv41y988hwR/gaLF+/r0sdqftu4qG36x0=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T07:01:50.535 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:50 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:01:51.401 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:51 smithi027 ceph-mon[30779]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:51.401 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:51 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:01:51.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T07:01:53.405 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T07:01:53.405 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2024-09-21T07:01:53.650 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:53 smithi027 ceph-mon[30779]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:53.650 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:53 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:01:54.985 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7Ew3Tir/OqaQQ3VG4IvSnOj6RrJEhSooxYxcJ6G43bJgHYev7N8ti", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726902114, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T07:01:54.986 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2024-09-21T07:01:55.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:55 smithi027 ceph-mon[30779]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:01:55.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:55 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:01:56.503 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:56 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/620439353' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T07:01:56.510 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T07:01:56.530 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T07:01:56.530 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T07:01:56.545 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T07:01:56.545 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T07:01:56.545 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T07:01:56.545 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T07:01:56.547 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2024-09-21T07:01:56.548 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:56.548 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2024-09-21T07:01:56.548 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2024-09-21T07:01:56.548 INFO:teuthology.orchestra.run.smithi027.stdout: Password: 3b98dc697o 2024-09-21T07:01:56.548 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:56.548 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config directory 2024-09-21T07:01:58.108 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T07:01:58.108 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T07:01:58.108 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.108 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:01:58.108 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.108 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:01:58.109 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2024-09-21T07:01:58.169 INFO:tasks.cephadm:Fetching config... 2024-09-21T07:01:58.169 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:01:58.169 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T07:01:58.206 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T07:01:58.206 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:01:58.206 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T07:01:58.285 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T07:01:58.285 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:01:58.285 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T07:01:58.384 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T07:01:58.384 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:01:58.384 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T07:01:58.460 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T07:01:58.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7ecjv5ww6ZDShW0zcRgr9s8qnzys7uL/z0gJ9Nsm/3EIjNT04OYJtCS29oGsBM/HQxEko6zRqZ9UFF5VenOaBXXlOawGyU7rSq59weD2042XDLTXYNv96jex/Jm8fBcKXt7UZB2dVpgAa3wWHJVsFb/SmsookYaUIEMotH2fGIPSgR/ERMZE23emn8yUeGhdqKnro7tOKBYkiI3RkSyWtoUVfe4C8bNysoRdH2LM0VJPaA9fqEUwIFOt+J7ggf+V+8GZkvL14d9fKVljlNz6wEsNL0l3KxVSHILSCz9sP497w+mNWiXLLqFA3GizFfKoHcn7M6cuZ/O4ZLGBax+XqlpEH0mPlvR6kbvP1PCtJ3/i3Y3iZoLKOiQwcom/gzyqBaIiiuif5+W7Hi4RPrz+aN4RU1DUS+Bv5pOszCJG0j/ik+hYf34h3Wwdthnc5JeDQ7jYYAwEerouCfWY9IOdJuYNDeUNgZcRDzBaZqjqj0Hzi/1MpCEcDB7jM7i7RpbM= ceph-0ff1e276-77e7-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T07:01:58.582 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7ecjv5ww6ZDShW0zcRgr9s8qnzys7uL/z0gJ9Nsm/3EIjNT04OYJtCS29oGsBM/HQxEko6zRqZ9UFF5VenOaBXXlOawGyU7rSq59weD2042XDLTXYNv96jex/Jm8fBcKXt7UZB2dVpgAa3wWHJVsFb/SmsookYaUIEMotH2fGIPSgR/ERMZE23emn8yUeGhdqKnro7tOKBYkiI3RkSyWtoUVfe4C8bNysoRdH2LM0VJPaA9fqEUwIFOt+J7ggf+V+8GZkvL14d9fKVljlNz6wEsNL0l3KxVSHILSCz9sP497w+mNWiXLLqFA3GizFfKoHcn7M6cuZ/O4ZLGBax+XqlpEH0mPlvR6kbvP1PCtJ3/i3Y3iZoLKOiQwcom/gzyqBaIiiuif5+W7Hi4RPrz+aN4RU1DUS+Bv5pOszCJG0j/ik+hYf34h3Wwdthnc5JeDQ7jYYAwEerouCfWY9IOdJuYNDeUNgZcRDzBaZqjqj0Hzi/1MpCEcDB7jM7i7RpbM= ceph-0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:01:58.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:58 smithi027 ceph-mon[30779]: mgrmap e14: x(active, since 12s) 2024-09-21T07:01:58.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:01:58 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/515029013' entity='client.admin' 2024-09-21T07:01:58.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7ecjv5ww6ZDShW0zcRgr9s8qnzys7uL/z0gJ9Nsm/3EIjNT04OYJtCS29oGsBM/HQxEko6zRqZ9UFF5VenOaBXXlOawGyU7rSq59weD2042XDLTXYNv96jex/Jm8fBcKXt7UZB2dVpgAa3wWHJVsFb/SmsookYaUIEMotH2fGIPSgR/ERMZE23emn8yUeGhdqKnro7tOKBYkiI3RkSyWtoUVfe4C8bNysoRdH2LM0VJPaA9fqEUwIFOt+J7ggf+V+8GZkvL14d9fKVljlNz6wEsNL0l3KxVSHILSCz9sP497w+mNWiXLLqFA3GizFfKoHcn7M6cuZ/O4ZLGBax+XqlpEH0mPlvR6kbvP1PCtJ3/i3Y3iZoLKOiQwcom/gzyqBaIiiuif5+W7Hi4RPrz+aN4RU1DUS+Bv5pOszCJG0j/ik+hYf34h3Wwdthnc5JeDQ7jYYAwEerouCfWY9IOdJuYNDeUNgZcRDzBaZqjqj0Hzi/1MpCEcDB7jM7i7RpbM= ceph-0ff1e276-77e7-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T07:01:58.652 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7ecjv5ww6ZDShW0zcRgr9s8qnzys7uL/z0gJ9Nsm/3EIjNT04OYJtCS29oGsBM/HQxEko6zRqZ9UFF5VenOaBXXlOawGyU7rSq59weD2042XDLTXYNv96jex/Jm8fBcKXt7UZB2dVpgAa3wWHJVsFb/SmsookYaUIEMotH2fGIPSgR/ERMZE23emn8yUeGhdqKnro7tOKBYkiI3RkSyWtoUVfe4C8bNysoRdH2LM0VJPaA9fqEUwIFOt+J7ggf+V+8GZkvL14d9fKVljlNz6wEsNL0l3KxVSHILSCz9sP497w+mNWiXLLqFA3GizFfKoHcn7M6cuZ/O4ZLGBax+XqlpEH0mPlvR6kbvP1PCtJ3/i3Y3iZoLKOiQwcom/gzyqBaIiiuif5+W7Hi4RPrz+aN4RU1DUS+Bv5pOszCJG0j/ik+hYf34h3Wwdthnc5JeDQ7jYYAwEerouCfWY9IOdJuYNDeUNgZcRDzBaZqjqj0Hzi/1MpCEcDB7jM7i7RpbM= ceph-0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:01:58.665 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7ecjv5ww6ZDShW0zcRgr9s8qnzys7uL/z0gJ9Nsm/3EIjNT04OYJtCS29oGsBM/HQxEko6zRqZ9UFF5VenOaBXXlOawGyU7rSq59weD2042XDLTXYNv96jex/Jm8fBcKXt7UZB2dVpgAa3wWHJVsFb/SmsookYaUIEMotH2fGIPSgR/ERMZE23emn8yUeGhdqKnro7tOKBYkiI3RkSyWtoUVfe4C8bNysoRdH2LM0VJPaA9fqEUwIFOt+J7ggf+V+8GZkvL14d9fKVljlNz6wEsNL0l3KxVSHILSCz9sP497w+mNWiXLLqFA3GizFfKoHcn7M6cuZ/O4ZLGBax+XqlpEH0mPlvR6kbvP1PCtJ3/i3Y3iZoLKOiQwcom/gzyqBaIiiuif5+W7Hi4RPrz+aN4RU1DUS+Bv5pOszCJG0j/ik+hYf34h3Wwdthnc5JeDQ7jYYAwEerouCfWY9IOdJuYNDeUNgZcRDzBaZqjqj0Hzi/1MpCEcDB7jM7i7RpbM= ceph-0ff1e276-77e7-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T07:01:58.709 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7ecjv5ww6ZDShW0zcRgr9s8qnzys7uL/z0gJ9Nsm/3EIjNT04OYJtCS29oGsBM/HQxEko6zRqZ9UFF5VenOaBXXlOawGyU7rSq59weD2042XDLTXYNv96jex/Jm8fBcKXt7UZB2dVpgAa3wWHJVsFb/SmsookYaUIEMotH2fGIPSgR/ERMZE23emn8yUeGhdqKnro7tOKBYkiI3RkSyWtoUVfe4C8bNysoRdH2LM0VJPaA9fqEUwIFOt+J7ggf+V+8GZkvL14d9fKVljlNz6wEsNL0l3KxVSHILSCz9sP497w+mNWiXLLqFA3GizFfKoHcn7M6cuZ/O4ZLGBax+XqlpEH0mPlvR6kbvP1PCtJ3/i3Y3iZoLKOiQwcom/gzyqBaIiiuif5+W7Hi4RPrz+aN4RU1DUS+Bv5pOszCJG0j/ik+hYf34h3Wwdthnc5JeDQ7jYYAwEerouCfWY9IOdJuYNDeUNgZcRDzBaZqjqj0Hzi/1MpCEcDB7jM7i7RpbM= ceph-0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:01:58.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T07:01:59.102 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:00.950 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T07:02:00.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T07:02:01.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:00 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3609199820' entity='client.admin' 2024-09-21T07:02:01.343 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:02 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:02 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:02 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:02 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:02 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:02 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:03.269 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi119 2024-09-21T07:02:03.270 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T07:02:03.270 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.conf 2024-09-21T07:02:03.289 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T07:02:03.290 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:03.348 INFO:tasks.cephadm:Adding host smithi119 to orchestrator... 2024-09-21T07:02:03.348 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch host add smithi119 2024-09-21T07:02:03.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:03.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:03.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:03.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:03.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:03.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T07:02:03.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:03.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:03 smithi027 ceph-mon[30779]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:03.970 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:04 smithi027 ceph-mon[30779]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:02:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:04 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:04 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:04 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:05.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:05 smithi027 ceph-mon[30779]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:06.814 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:06 smithi027 ceph-mon[30779]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:06.814 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:06 smithi027 ceph-mon[30779]: Deploying cephadm binary to smithi119 2024-09-21T07:02:08.584 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi119' with addr '172.21.15.119' 2024-09-21T07:02:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:08 smithi027 ceph-mon[30779]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:09.146 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T07:02:09.522 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:09.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:09 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:09.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:09 smithi027 ceph-mon[30779]: Added host smithi119 2024-09-21T07:02:09.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:09 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:09.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:09 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:09.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:09 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:02:10.343 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}] 2024-09-21T07:02:10.944 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:10 smithi027 ceph-mon[30779]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:10.944 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:10 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:10.982 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi153 2024-09-21T07:02:10.982 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T07:02:10.982 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.conf 2024-09-21T07:02:11.002 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T07:02:11.002 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:11.061 INFO:tasks.cephadm:Adding host smithi153 to orchestrator... 2024-09-21T07:02:11.061 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch host add smithi153 2024-09-21T07:02:11.431 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:12.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:11 smithi027 ceph-mon[30779]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:12 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:02:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:02:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:14.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:13 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:15.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:14 smithi027 ceph-mon[30779]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:15.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:14 smithi027 ceph-mon[30779]: Deploying cephadm binary to smithi153 2024-09-21T07:02:16.031 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi153' with addr '172.21.15.153' 2024-09-21T07:02:16.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T07:02:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:17.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:17 smithi027 ceph-mon[30779]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:17.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:17 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:17.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:17 smithi027 ceph-mon[30779]: Added host smithi153 2024-09-21T07:02:17.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:17 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:17.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:17 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:17.211 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:17 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:17.880 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:02:17.880 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}] 2024-09-21T07:02:18.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:18 smithi027 ceph-mon[30779]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:18.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:18 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:18.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:18 smithi027 ceph-mon[30779]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:02:18.588 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T07:02:18.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T07:02:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:20.359 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4137972220' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:20.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:20 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:21.019 INFO:tasks.cephadm:Adding mon.a on smithi027 2024-09-21T07:02:21.019 INFO:tasks.cephadm:Adding mon.b on smithi119 2024-09-21T07:02:21.020 INFO:tasks.cephadm:Adding mon.c on smithi153 2024-09-21T07:02:21.020 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c' 2024-09-21T07:02:21.253 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:21.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:02:21.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:21.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4137972220' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T07:02:21.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:21.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:21.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:02:21.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:21.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:21.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:21 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:21.950 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mon update... 2024-09-21T07:02:22.557 DEBUG:teuthology.orchestra.run.smithi119:mon.b> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T07:02:22.559 DEBUG:teuthology.orchestra.run.smithi153:mon.c> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T07:02:22.562 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T07:02:22.562 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T07:02:23.241 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c;count:3 2024-09-21T07:02:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:23.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:23.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:23.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:23.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:23.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:22 smithi027 ceph-mon[30779]: Deploying daemon mon.c on smithi153 2024-09-21T07:02:23.918 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:02:23.918 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":1,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","modified":"2024-09-21T07:00:41.226200Z","created":"2024-09-21T07:00:41.226200Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T07:02:23.918 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 1 2024-09-21T07:02:24.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:23 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/3993215864' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:24 smithi027 ceph-mon[30779]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:25.507 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T07:02:25.507 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T07:02:25.775 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:02:26.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:26 smithi153 systemd[1]: Starting Ceph mon.c for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:02:26.679 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:02:26.679 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":1,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","modified":"2024-09-21T07:00:41.226200Z","created":"2024-09-21T07:00:41.226200Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T07:02:26.679 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 1 2024-09-21T07:02:26.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:26 smithi153 podman[34715]: 2024-09-21 07:02:26.640156368 +0000 UTC m=+0.111296136 container create 51e0fe7fc2e44923133069353694036923720e3ab18c792dd6d5c2c4b7932b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:02:26.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:26 smithi153 podman[34715]: 2024-09-21 07:02:26.544767944 +0000 UTC m=+0.015907710 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:02:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:26 smithi027 ceph-mon[30779]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:26 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/3736814281' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:27.615 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 podman[34715]: 2024-09-21 07:02:27.185616302 +0000 UTC m=+0.656756066 container init 51e0fe7fc2e44923133069353694036923720e3ab18c792dd6d5c2c4b7932b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T07:02:27.615 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 podman[34715]: 2024-09-21 07:02:27.188972545 +0000 UTC m=+0.660112318 container start 51e0fe7fc2e44923133069353694036923720e3ab18c792dd6d5c2c4b7932b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:02:27.615 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T07:02:27.615 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T07:02:27.615 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pidfile_write: ignore empty --pid-file 2024-09-21T07:02:27.615 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: load: jerasure load: lrc 2024-09-21T07:02:27.615 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: RocksDB version: 7.9.2 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Git sha 0 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: DB SUMMARY 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: DB Session ID: TNQO3YCLPEJINSQID40Q 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: CURRENT file: CURRENT 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.error_if_exists: 0 2024-09-21T07:02:27.616 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.create_if_missing: 0 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.paranoid_checks: 1 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.env: 0x5596aa58cd60 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.info_log: 0x5596aeee9360 2024-09-21T07:02:27.617 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T07:02:27.618 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.statistics: (nil) 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.use_fsync: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_log_file_size: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.allow_fallocate: 1 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.use_direct_reads: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T07:02:27.619 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.db_log_dir: 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.wal_dir: 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T07:02:27.620 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.write_buffer_manager: 0x5596aeeedd60 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.unordered_write: 0 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T07:02:27.621 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.row_cache: None 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.wal_filter: None 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.two_write_queues: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.wal_compression: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.atomic_flush: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.log_readahead_size: 0 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T07:02:27.622 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_background_jobs: 2 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_background_compactions: -1 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_subcompactions: 1 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T07:02:27.623 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_open_files: -1 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_background_flushes: -1 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Compression algorithms supported: 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kZSTD supported: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kXpressCompression supported: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kBZip2Compression supported: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T07:02:27.624 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kLZ4Compression supported: 1 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kZlibCompression supported: 1 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: kSnappyCompression supported: 1 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.merge_operator: 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_filter: None 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T07:02:27.625 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5596aeee8fa0) 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: cache_index_and_filter_blocks: 1 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: pin_top_level_index_and_filter: 1 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: index_type: 0 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: data_block_index_type: 0 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: index_shortening: 1 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: checksum: 4 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: no_block_cache: 0 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_cache: 0x5596aaead610 2024-09-21T07:02:27.626 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_cache_name: BinnedLRUCache 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_cache_options: 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: capacity : 536870912 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: num_shard_bits : 4 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: strict_capacity_limit : 0 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: high_pri_pool_ratio: 0.000 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_cache_compressed: (nil) 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: persistent_cache: (nil) 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_size: 4096 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_size_deviation: 10 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_restart_interval: 16 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: index_block_restart_interval: 1 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: metadata_block_size: 4096 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: partition_filters: 0 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: use_delta_encoding: 1 2024-09-21T07:02:27.627 INFO:journalctl@ceph.mon.c.smithi153.stdout: filter_policy: bloomfilter 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: whole_key_filtering: 1 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: verify_compression: 0 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: read_amp_bytes_per_bit: 0 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: format_version: 5 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: enable_index_compression: 1 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: block_align: 0 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: max_auto_readahead_size: 262144 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: prepopulate_block_cache: 0 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: initial_auto_readahead_size: 8192 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression: NoCompression 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T07:02:27.628 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.num_levels: 7 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T07:02:27.629 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T07:02:27.630 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T07:02:27.631 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T07:02:27.632 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.inplace_update_support: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.bloom_locality: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.max_successive_merges: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.ttl: 2592000 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T07:02:27.633 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.enable_blob_files: false 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.min_blob_size: 0 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T07:02:27.634 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8170fc8e-346f-4782-9ff7-f4b04775950c 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902147228321, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902147228942, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726902147, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8170fc8e-346f-4782-9ff7-f4b04775950c", "db_session_id": "TNQO3YCLPEJINSQID40Q", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902147229073, "job": 1, "event": "recovery_finished"} 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5596aef0ca80 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: DB pointer 0x5596afcb2000 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout: ** DB Stats ** 2024-09-21T07:02:27.635 INFO:journalctl@ceph.mon.c.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: ** Compaction Stats [default] ** 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:02:27.636 INFO:journalctl@ceph.mon.c.smithi153.stdout: 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: ** Compaction Stats [default] ** 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T07:02:27.637 INFO:journalctl@ceph.mon.c.smithi153.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: Cumulative compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: Interval compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: Block cache BinnedLRUCache@0x5596aaead610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: using public_addr v2:172.21.15.153:0/0 -> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: starting mon.c rank -1 at public addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] at bind addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(???) e0 preinit fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T07:02:27.638 INFO:journalctl@ceph.mon.c.smithi153.stdout: e1 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout: btime 2024-09-21T07:00:45:000882+0000 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout: legacy client fscid: -1 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout: 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout: No filesystems configured 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:02:27.639 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:02:27.640 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:02:27.640 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:27.640 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.640 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Deploying cephadm binary to smithi119 2024-09-21T07:02:27.640 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.640 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Added host smithi119 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:27.641 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Deploying cephadm binary to smithi153 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.642 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Added host smithi153 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4137972220' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4137972220' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c;count:3 2024-09-21T07:02:27.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: Deploying daemon mon.c on smithi153 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/3993215864' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/3736814281' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 ceph-mon[34747]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 bash[34715]: 51e0fe7fc2e44923133069353694036923720e3ab18c792dd6d5c2c4b7932b52 2024-09-21T07:02:27.645 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:27 smithi153 systemd[1]: Started Ceph mon.c for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:02:28.650 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T07:02:28.650 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T07:02:28.905 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:02:32.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: mon.a calling monitor election 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: mon.c calling monitor election 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: monmap epoch 2 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: last_changed 2024-09-21T07:02:27.261073+0000 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: min_mon_release 19 (squid) 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: election_strategy: 1 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: fsmap 2024-09-21T07:02:32.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:32.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: mgrmap e14: x(active, since 46s) 2024-09-21T07:02:32.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: overall HEALTH_OK 2024-09-21T07:02:32.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:32.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:32.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:32.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:32.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:32 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:32.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:02:32.605 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":2,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","modified":"2024-09-21T07:02:27.261073Z","created":"2024-09-21T07:00:41.226200Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T07:02:32.605 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 2 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: mon.a calling monitor election 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: mon.c calling monitor election 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:32.606 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: monmap epoch 2 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: last_changed 2024-09-21T07:02:27.261073+0000 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:32.607 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: min_mon_release 19 (squid) 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: election_strategy: 1 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: fsmap 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: mgrmap e14: x(active, since 46s) 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: overall HEALTH_OK 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:32.608 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:32 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:33 smithi153 ceph-mon[34747]: Deploying daemon mon.b on smithi119 2024-09-21T07:02:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:33 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/74105921' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:33 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:33 smithi027 ceph-mon[30779]: Deploying daemon mon.b on smithi119 2024-09-21T07:02:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:33 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/74105921' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:33 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:34.217 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T07:02:34.217 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T07:02:34.472 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:02:34.588 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:34 smithi153 ceph-mon[34747]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:34.742 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:34 smithi027 ceph-mon[30779]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:35.146 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:02:35.146 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":2,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","modified":"2024-09-21T07:02:27.261073Z","created":"2024-09-21T07:00:41.226200Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T07:02:35.146 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 2 2024-09-21T07:02:35.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:35 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/286377928' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:35.766 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:35 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/286377928' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:35.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:35 smithi119 podman[34500]: 2024-09-21 07:02:35.658736531 +0000 UTC m=+0.015674188 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:35 smithi119 podman[34500]: 2024-09-21 07:02:35.964352412 +0000 UTC m=+0.321290069 container init fcfb2e88df905fc54d4ccd1d691c80571186dae25445f550413b74536d67e6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:35 smithi119 podman[34500]: 2024-09-21 07:02:35.967537411 +0000 UTC m=+0.324475067 container start fcfb2e88df905fc54d4ccd1d691c80571186dae25445f550413b74536d67e6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:35 smithi119 ceph-mon[34519]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:35 smithi119 ceph-mon[34519]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:35 smithi119 ceph-mon[34519]: pidfile_write: ignore empty --pid-file 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: load: jerasure load: lrc 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: RocksDB version: 7.9.2 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Git sha 0 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: DB SUMMARY 2024-09-21T07:02:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: DB Session ID: FINCNN8A3KFBIHJ274K6 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: CURRENT file: CURRENT 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.error_if_exists: 0 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.create_if_missing: 0 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.paranoid_checks: 1 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.env: 0x55c9f9eb3d60 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.info_log: 0x55c9fe935360 2024-09-21T07:02:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.statistics: (nil) 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.use_fsync: 0 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_log_file_size: 0 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.allow_fallocate: 1 2024-09-21T07:02:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.use_direct_reads: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.db_log_dir: 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.wal_dir: 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T07:02:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.write_buffer_manager: 0x55c9fe939d60 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.unordered_write: 0 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.row_cache: None 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.wal_filter: None 2024-09-21T07:02:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.two_write_queues: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.wal_compression: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.atomic_flush: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.log_readahead_size: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T07:02:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_background_jobs: 2 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_background_compactions: -1 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_subcompactions: 1 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_open_files: -1 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T07:02:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_background_flushes: -1 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Compression algorithms supported: 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kZSTD supported: 0 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kXpressCompression supported: 0 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kBZip2Compression supported: 0 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kLZ4Compression supported: 1 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kZlibCompression supported: 1 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: kSnappyCompression supported: 1 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T07:02:36.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.merge_operator: 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_filter: None 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c9fe934fa0) 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks: 1 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_top_level_index_and_filter: 1 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_type: 0 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_index_type: 0 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_shortening: 1 2024-09-21T07:02:36.294 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: checksum: 4 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: no_block_cache: 0 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache: 0x55c9fa8f9610 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_name: BinnedLRUCache 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_options: 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: capacity : 536870912 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_shard_bits : 4 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: strict_capacity_limit : 0 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: high_pri_pool_ratio: 0.000 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_compressed: (nil) 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: persistent_cache: (nil) 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size: 4096 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size_deviation: 10 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_restart_interval: 16 2024-09-21T07:02:36.295 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_block_restart_interval: 1 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: metadata_block_size: 4096 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: partition_filters: 0 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: use_delta_encoding: 1 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: filter_policy: bloomfilter 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: whole_key_filtering: 1 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: verify_compression: 0 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: read_amp_bytes_per_bit: 0 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: format_version: 5 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_index_compression: 1 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_align: 0 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: max_auto_readahead_size: 262144 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: prepopulate_block_cache: 0 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: initial_auto_readahead_size: 8192 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T07:02:36.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression: NoCompression 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.num_levels: 7 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T07:02:36.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T07:02:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T07:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T07:02:36.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.inplace_update_support: 0 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.bloom_locality: 0 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.max_successive_merges: 0 2024-09-21T07:02:36.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.ttl: 2592000 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.enable_blob_files: false 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.min_blob_size: 0 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T07:02:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 50b4c6ee-1e34-44db-a0bd-51834c25bff9 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902156004630, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902156005048, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726902156, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "50b4c6ee-1e34-44db-a0bd-51834c25bff9", "db_session_id": "FINCNN8A3KFBIHJ274K6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726902156005142, "job": 1, "event": "recovery_finished"} 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c9fe958a80 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: DB pointer 0x55c9ff6fe000 2024-09-21T07:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** DB Stats ** 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T07:02:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T07:02:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache BinnedLRUCache@0x55c9fa8f9610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: using public_addr v2:172.21.15.119:0/0 -> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: starting mon.b rank -1 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(???) e0 preinit fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T07:02:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout: e1 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout: btime 2024-09-21T07:00:45:000882+0000 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout: legacy client fscid: -1 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout: No filesystems configured 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T07:02:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Deploying cephadm binary to smithi119 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Added host smithi119 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:02:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Deploying cephadm binary to smithi153 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Added host smithi153 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:02:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4137972220' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4137972220' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:02:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c;count:3 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Deploying daemon mon.c on smithi153 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/3993215864' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/3736814281' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.a calling monitor election 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.c calling monitor election 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: monmap epoch 2 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: last_changed 2024-09-21T07:02:27.261073+0000 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: min_mon_release 19 (squid) 2024-09-21T07:02:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: election_strategy: 1 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: fsmap 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mgrmap e14: x(active, since 46s) 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: overall HEALTH_OK 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: Deploying daemon mon.b on smithi119 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/74105921' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:36.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:36.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/286377928' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:36.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 ceph-mon[34519]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T07:02:36.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 bash[34500]: fcfb2e88df905fc54d4ccd1d691c80571186dae25445f550413b74536d67e6c8 2024-09-21T07:02:36.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:36 smithi119 systemd[1]: Started Ceph mon.b for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:02:36.790 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T07:02:36.790 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T07:02:37.038 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:02:41.350 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:02:41.350 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":3,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","modified":"2024-09-21T07:02:36.034042Z","created":"2024-09-21T07:00:41.226200Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T07:02:41.350 INFO:teuthology.orchestra.run.smithi153.stderr:dumped monmap epoch 3 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: mon.a calling monitor election 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: mon.c calling monitor election 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:41.351 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: monmap epoch 3 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: last_changed 2024-09-21T07:02:36.034042+0000 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: min_mon_release 19 (squid) 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: election_strategy: 1 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: fsmap 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: mgrmap e14: x(active, since 55s) 2024-09-21T07:02:41.352 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: overall HEALTH_OK 2024-09-21T07:02:41.353 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.353 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.353 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.353 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:41 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: mon.a calling monitor election 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: mon.c calling monitor election 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:41.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: monmap epoch 3 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: last_changed 2024-09-21T07:02:36.034042+0000 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: min_mon_release 19 (squid) 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: election_strategy: 1 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: fsmap 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: mgrmap e14: x(active, since 55s) 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: overall HEALTH_OK 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:41 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:41.918 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T07:02:41.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T07:02:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:42 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:42 smithi153 ceph-mon[34747]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:42 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/1444566004' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:42 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:42 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:42 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:42.304 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:42 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:42 smithi027 ceph-mon[30779]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:42 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/1444566004' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:42 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:42 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:42 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:43.111 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:43.111 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-09-21T07:02:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:43.112 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] 2024-09-21T07:02:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: mon.a calling monitor election 2024-09-21T07:02:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: mon.c calling monitor election 2024-09-21T07:02:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:43.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:43.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:43.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:43.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:43.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:43.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:43.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:43.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:43.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T07:02:43.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: monmap epoch 3 2024-09-21T07:02:43.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:43.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: last_changed 2024-09-21T07:02:36.034042+0000 2024-09-21T07:02:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: min_mon_release 19 (squid) 2024-09-21T07:02:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: election_strategy: 1 2024-09-21T07:02:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T07:02:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: fsmap 2024-09-21T07:02:43.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:43.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: mgrmap e14: x(active, since 55s) 2024-09-21T07:02:43.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: overall HEALTH_OK 2024-09-21T07:02:43.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:43.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:43.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:43.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/1444566004' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T07:02:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:43 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:43.819 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T07:02:43.819 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:02:43.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T07:02:43.882 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:02:43.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:43.979 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T07:02:43.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T07:02:44.009 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T07:02:44.010 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:44.075 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T07:02:44.075 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T07:02:44.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: mon.b calling monitor election 2024-09-21T07:02:44.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: mon.b calling monitor election 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: mon.a calling monitor election 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: mon.c calling monitor election 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: monmap epoch 3 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: last_changed 2024-09-21T07:02:36.034042+0000 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: min_mon_release 19 (squid) 2024-09-21T07:02:44.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: election_strategy: 1 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: fsmap 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: mgrmap e14: x(active, since 57s) 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: overall HEALTH_OK 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1528357688' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:44.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:44.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:44 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:44.108 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T07:02:44.108 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:02:44.178 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T07:02:44.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: mon.b calling monitor election 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: mon.b calling monitor election 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: mon.a calling monitor election 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: mon.c calling monitor election 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: monmap epoch 3 2024-09-21T07:02:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: last_changed 2024-09-21T07:02:36.034042+0000 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: min_mon_release 19 (squid) 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: election_strategy: 1 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: fsmap 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: mgrmap e14: x(active, since 57s) 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: overall HEALTH_OK 2024-09-21T07:02:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1528357688' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:44.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:44.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:44 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: mon.b calling monitor election 2024-09-21T07:02:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T07:02:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:02:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:02:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:02:44.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: mon.b calling monitor election 2024-09-21T07:02:44.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: mon.a calling monitor election 2024-09-21T07:02:44.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: mon.c calling monitor election 2024-09-21T07:02:44.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T07:02:44.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: monmap epoch 3 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: last_changed 2024-09-21T07:02:36.034042+0000 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: created 2024-09-21T07:00:41.226200+0000 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: min_mon_release 19 (squid) 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: election_strategy: 1 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: 1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.c 2024-09-21T07:02:44.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T07:02:44.431 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: fsmap 2024-09-21T07:02:44.431 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T07:02:44.431 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: mgrmap e14: x(active, since 57s) 2024-09-21T07:02:44.431 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: overall HEALTH_OK 2024-09-21T07:02:44.431 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.431 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.431 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1528357688' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:44.432 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:44.433 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:44 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:45.050 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:45 smithi119 ceph-mon[34519]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:45 smithi119 ceph-mon[34519]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T07:02:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:45 smithi119 ceph-mon[34519]: Reconfiguring daemon mon.a on smithi027 2024-09-21T07:02:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:45 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:45.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:45 smithi027 ceph-mon[30779]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:45.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:45 smithi027 ceph-mon[30779]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T07:02:45.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:45 smithi027 ceph-mon[30779]: Reconfiguring daemon mon.a on smithi027 2024-09-21T07:02:45.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:45 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:45 smithi153 ceph-mon[34747]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:45 smithi153 ceph-mon[34747]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T07:02:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:45 smithi153 ceph-mon[34747]: Reconfiguring daemon mon.a on smithi027 2024-09-21T07:02:45.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:45 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: Reconfiguring mon.b (monmap changed)... 2024-09-21T07:02:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:46.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: Reconfiguring daemon mon.b on smithi119 2024-09-21T07:02:46.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/974639883' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T07:02:46.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T07:02:46.405 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.405 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.405 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: Reconfiguring mon.b (monmap changed)... 2024-09-21T07:02:46.405 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:46.405 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:46.405 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:46.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: Reconfiguring daemon mon.b on smithi119 2024-09-21T07:02:46.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/974639883' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T07:02:46.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:46 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T07:02:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: Reconfiguring mon.b (monmap changed)... 2024-09-21T07:02:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:46.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:46.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: Reconfiguring daemon mon.b on smithi119 2024-09-21T07:02:46.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:46.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/974639883' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T07:02:46.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:46 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T07:02:47.072 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ignoring --setuser ceph since I am not root 2024-09-21T07:02:47.072 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ignoring --setgroup ceph since I am not root 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T07:02:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:47 smithi153 ceph-mon[34747]: mgrmap e15: x(active, since 61s) 2024-09-21T07:02:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T07:02:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:47 smithi119 ceph-mon[34519]: mgrmap e15: x(active, since 61s) 2024-09-21T07:02:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:02:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' 2024-09-21T07:02:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:02:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:02:47.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: from='mgr.14154 172.21.15.27:0/383019350' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:47.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T07:02:47.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:47 smithi027 ceph-mon[30779]: mgrmap e15: x(active, since 61s) 2024-09-21T07:02:47.503 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-09-21T07:02:47.503 INFO:tasks.cephadm:Adding mgr.y on smithi119 2024-09-21T07:02:47.503 INFO:tasks.cephadm:Adding mgr.z on smithi153 2024-09-21T07:02:47.504 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi027=x;smithi119=y;smithi153=z' 2024-09-21T07:02:47.742 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:02:53.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: Active manager daemon x restarted 2024-09-21T07:02:53.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: Activating manager daemon x 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: mgrmap e16: x(active, starting, since 0.00575448s) 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T07:02:53.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:53 smithi027 ceph-mon[30779]: Manager daemon x is now available 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: Active manager daemon x restarted 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: Activating manager daemon x 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: mgrmap e16: x(active, starting, since 0.00575448s) 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:53.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T07:02:53.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T07:02:53.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T07:02:53.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T07:02:53.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:53 smithi153 ceph-mon[34747]: Manager daemon x is now available 2024-09-21T07:02:53.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: Active manager daemon x restarted 2024-09-21T07:02:53.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: Activating manager daemon x 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: mgrmap e16: x(active, starting, since 0.00575448s) 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T07:02:53.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:53 smithi119 ceph-mon[34519]: Manager daemon x is now available 2024-09-21T07:02:54.291 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mgr update... 2024-09-21T07:02:54.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:54.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T07:02:54.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T07:02:54.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:54 smithi027 ceph-mon[30779]: mgrmap e17: x(active, since 1.01345s) 2024-09-21T07:02:54.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:54.597 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:54.597 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T07:02:54.597 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T07:02:54.597 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:54 smithi153 ceph-mon[34747]: mgrmap e17: x(active, since 1.01345s) 2024-09-21T07:02:54.597 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:54.678 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:54.678 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T07:02:54.678 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T07:02:54.678 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:54 smithi119 ceph-mon[34519]: mgrmap e17: x(active, since 1.01345s) 2024-09-21T07:02:54.678 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:54.885 DEBUG:teuthology.orchestra.run.smithi119:mgr.y> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T07:02:54.888 DEBUG:teuthology.orchestra.run.smithi153:mgr.z> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T07:02:54.890 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T07:02:54.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T07:02:54.894 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T07:02:54.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T07:02:54.940 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T07:02:54.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T07:02:54.987 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T07:02:54.987 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T07:02:55.033 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T07:02:55.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T07:02:55.079 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T07:02:55.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T07:02:55.130 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T07:02:55.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T07:02:55.183 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T07:02:55.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T07:02:55.233 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T07:02:55.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T07:02:55.291 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T07:02:55.291 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T07:02:55.344 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T07:02:55.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T07:02:55.403 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T07:02:55.403 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T07:02:55.456 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T07:02:55.456 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T07:02:55.512 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T07:02:55.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T07:02:55.564 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T07:02:55.564 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T07:02:55.623 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T07:02:55.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T07:02:55.689 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T07:02:55.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T07:02:55.756 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T07:02:55.756 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T07:02:55.816 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T07:02:55.816 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T07:02:55.897 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T07:02:55.897 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T07:02:55.957 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T07:02:55.957 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T07:02:56.004 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:56.031 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T07:02:56.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T07:02:56.083 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T07:02:56.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T07:02:56.164 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T07:02:56.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T07:02:56.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: Saving service mgr spec with placement smithi027=x;smithi119=y;smithi153=z;count:3 2024-09-21T07:02:56.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:56.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.166 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:02:55.875+0000 7f761b400640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:07:02:55] ENGINE Error in HTTPServer.serve 2024-09-21T07:02:56.166 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: Traceback (most recent call last): 2024-09-21T07:02:56.166 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:02:56.166 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._connections.run(self.expiration_interval) 2024-09-21T07:02:56.166 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:02:56.166 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._run(expiration_interval) 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: s = self.context.wrap_socket( 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: return self.sslsocket_class._create( 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self.do_handshake() 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:02:56.167 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: self._sslobj.do_handshake() 2024-09-21T07:02:56.168 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:02:56.168 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:02:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:02:56.225 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T07:02:56.226 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T07:02:56.248 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:56.256 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T07:02:56.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T07:02:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: Saving service mgr spec with placement smithi027=x;smithi119=y;smithi153=z;count:3 2024-09-21T07:02:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.319 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T07:02:56.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T07:02:56.377 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T07:02:56.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T07:02:56.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: Saving service mgr spec with placement smithi027=x;smithi119=y;smithi153=z;count:3 2024-09-21T07:02:56.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:56.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:56.446 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T07:02:56.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T07:02:56.474 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:56.492 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T07:02:56.492 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T07:02:56.553 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T07:02:56.553 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi027') 2024-09-21T07:02:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: [21/Sep/2024:07:02:55] ENGINE Bus STARTING 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: [21/Sep/2024:07:02:55] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: [21/Sep/2024:07:02:55] ENGINE Error in HTTPServer.serve 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: Traceback (most recent call last): 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: self._connections.run(self.expiration_interval) 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: self._run(expiration_interval) 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:02:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: s = self.context.wrap_socket( 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: return self.sslsocket_class._create( 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: self.do_handshake() 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: self._sslobj.do_handshake() 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: mgrmap e18: x(active, since 2s) 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: [21/Sep/2024:07:02:55] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: [21/Sep/2024:07:02:55] ENGINE Bus STARTED 2024-09-21T07:02:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:57.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:56 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: [21/Sep/2024:07:02:55] ENGINE Bus STARTING 2024-09-21T07:02:57.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: [21/Sep/2024:07:02:55] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T07:02:57.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: [21/Sep/2024:07:02:55] ENGINE Error in HTTPServer.serve 2024-09-21T07:02:57.308 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T07:02:57.308 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:02:57.309 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: mgrmap e18: x(active, since 2s) 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: [21/Sep/2024:07:02:55] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: [21/Sep/2024:07:02:55] ENGINE Bus STARTED 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:56 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: [21/Sep/2024:07:02:55] ENGINE Bus STARTING 2024-09-21T07:02:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: [21/Sep/2024:07:02:55] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T07:02:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: [21/Sep/2024:07:02:55] ENGINE Error in HTTPServer.serve 2024-09-21T07:02:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout: Traceback (most recent call last): 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: self._connections.run(self.expiration_interval) 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: self._run(expiration_interval) 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: s = self.context.wrap_socket( 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: return self.sslsocket_class._create( 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: self.do_handshake() 2024-09-21T07:02:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout: self._sslobj.do_handshake() 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: mgrmap e18: x(active, since 2s) 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: [21/Sep/2024:07:02:55] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: [21/Sep/2024:07:02:55] ENGINE Bus STARTED 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"} : dispatch 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:57.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:56 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 -- 172.21.15.27:0/1034977263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa70403fb70 msgr2=0x7fa704042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 --2- 172.21.15.27:0/1034977263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa70403fb70 0x7fa704042080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa7100095f0 tx=0x7fa710005f90 comp rx=0 tx=0).stop 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 -- 172.21.15.27:0/1034977263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa71c0770e0 msgr2=0x7fa71c1a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 --2- 172.21.15.27:0/1034977263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa71c0770e0 0x7fa71c1a08d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7080057c0 tx=0x7fa708005e60 comp rx=0 tx=0).stop 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 -- 172.21.15.27:0/1034977263 shutdown_connections 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 --2- 172.21.15.27:0/1034977263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa71c10fee0 0x7fa71c1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 --2- 172.21.15.27:0/1034977263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa70403fb70 0x7fa704042080 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 --2- 172.21.15.27:0/1034977263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa71c0770e0 0x7fa71c1a08d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 --2- 172.21.15.27:0/1034977263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa71c075cf0 0x7fa71c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 -- 172.21.15.27:0/1034977263 >> 172.21.15.27:0/1034977263 conn(0x7fa71c0ff0e0 msgr2=0x7fa71c103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 -- 172.21.15.27:0/1034977263 shutdown_connections 2024-09-21T07:02:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.149+0000 7fa724200640 1 -- 172.21.15.27:0/1034977263 wait complete. 2024-09-21T07:02:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 -- 172.21.15.27:0/527083657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d101066f0 msgr2=0x7f5d10108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:58.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/527083657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d101066f0 0x7f5d10108b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009900 tx=0x7f5cfc02a410 comp rx=0 tx=0).stop 2024-09-21T07:02:58.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 -- 172.21.15.27:0/527083657 shutdown_connections 2024-09-21T07:02:58.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/527083657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d10109290 0x7f5d1010b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/527083657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d101066f0 0x7f5d10108b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/527083657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d1006b980 0x7f5d101061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 -- 172.21.15.27:0/527083657 >> 172.21.15.27:0/527083657 conn(0x7f5d100fddd0 msgr2=0x7f5d10100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:58.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 -- 172.21.15.27:0/527083657 shutdown_connections 2024-09-21T07:02:58.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.762+0000 7f5d0fe00640 1 -- 172.21.15.27:0/527083657 wait complete. 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.763+0000 7f5d0fe00640 1 Processor -- start 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.763+0000 7f5d0fe00640 1 -- start start 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d1006b980 0x7f5d101037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d101066f0 0x7f5d101020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d10109290 0x7f5d101025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1010eeb0 con 0x7f5d1006b980 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1010ed30 con 0x7f5d101066f0 2024-09-21T07:02:58.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1010f030 con 0x7f5d10109290 2024-09-21T07:02:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d10109290 0x7f5d101025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d101066f0 0x7f5d101020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d1006b980 0x7f5d101037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d10109290 0x7f5d101025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45688/0 (socket says 172.21.15.27:45688) 2024-09-21T07:02:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d101066f0 0x7f5d101020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52990/0 (socket says 172.21.15.27:52990) 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.764+0000 7f5d0f400640 1 -- 172.21.15.27:0/3636108081 learned_addr learned my addr 172.21.15.27:0/3636108081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d0ea00640 1 -- 172.21.15.27:0/3636108081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d10109290 msgr2=0x7f5d101025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d0ea00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d10109290 0x7f5d101025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d0ea00640 1 -- 172.21.15.27:0/3636108081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d101066f0 msgr2=0x7f5d101020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d0ea00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d101066f0 0x7f5d101020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d0ea00640 1 -- 172.21.15.27:0/3636108081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d10102cf0 con 0x7f5d1006b980 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d0e000640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d101066f0 0x7f5d101020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d0ea00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d1006b980 0x7f5d101037e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e9f0 tx=0x7f5d0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.765+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000cd80 con 0x7f5d1006b980 2024-09-21T07:02:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.766+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f5d00004480 con 0x7f5d1006b980 2024-09-21T07:02:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.766+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00010430 con 0x7f5d1006b980 2024-09-21T07:02:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.766+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d101a3bd0 con 0x7f5d1006b980 2024-09-21T07:02:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.766+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d101a4030 con 0x7f5d1006b980 2024-09-21T07:02:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.767+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f5d000105d0 con 0x7f5d1006b980 2024-09-21T07:02:58.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.767+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d10106b40 con 0x7f5d1006b980 2024-09-21T07:02:58.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.768+0000 7f5d07400640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5cf403fb70 0x7f5cf4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:58.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.768+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5d000549b0 con 0x7f5d1006b980 2024-09-21T07:02:58.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.768+0000 7f5d0e000640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5cf403fb70 0x7f5cf4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:58.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.769+0000 7f5d0e000640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5cf403fb70 0x7f5cf4042080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc005ab0 tx=0x7f5cfc005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.773+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d00009420 con 0x7f5d1006b980 2024-09-21T07:02:58.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.978+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f5d10063fc0 con 0x7f5d1006b980 2024-09-21T07:02:58.980 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:58 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:02:58.980 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:58 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1034977263' entity='client.admin' 2024-09-21T07:02:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.982+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f5d000035f0 con 0x7f5d1006b980 2024-09-21T07:02:58.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.983+0000 7f5d07400640 1 -- 172.21.15.27:0/3636108081 <== mon.0 v2:172.21.15.27:3300/0 8 ==== config(24 keys) ==== 1045+0+0 (secure 0 0 0) 0x7f5d0005a080 con 0x7f5d1006b980 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5cf403fb70 msgr2=0x7f5cf4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5cf403fb70 0x7f5cf4042080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc005ab0 tx=0x7f5cfc005a20 comp rx=0 tx=0).stop 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d1006b980 msgr2=0x7f5d101037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d1006b980 0x7f5d101037e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e9f0 tx=0x7f5d0000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 shutdown_connections 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d10109290 0x7f5d101025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5cf403fb70 0x7f5cf4042080 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d101066f0 0x7f5d101020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 --2- 172.21.15.27:0/3636108081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d1006b980 0x7f5d101037e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 >> 172.21.15.27:0/3636108081 conn(0x7f5d100fddd0 msgr2=0x7f5d10104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 shutdown_connections 2024-09-21T07:02:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:02:58.985+0000 7f5d0fe00640 1 -- 172.21.15.27:0/3636108081 wait complete. 2024-09-21T07:02:59.029 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi027') 2024-09-21T07:02:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:58 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:02:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:58 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1034977263' entity='client.admin' 2024-09-21T07:02:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:58 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:02:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:58 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1034977263' entity='client.admin' 2024-09-21T07:03:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3636108081' entity='client.admin' 2024-09-21T07:03:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T07:03:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:02:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3636108081' entity='client.admin' 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:02:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3636108081' entity='client.admin' 2024-09-21T07:03:00.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T07:03:00.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:00.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:00.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:02:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:00.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.726+0000 7f3295000640 1 -- 172.21.15.27:0/362641400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290106820 msgr2=0x7f3290106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.726+0000 7f3295000640 1 --2- 172.21.15.27:0/362641400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290106820 0x7f3290106c30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3278009900 tx=0x7f327802a480 comp rx=0 tx=0).stop 2024-09-21T07:03:00.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.727+0000 7f3295000640 1 -- 172.21.15.27:0/362641400 shutdown_connections 2024-09-21T07:03:00.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.727+0000 7f3295000640 1 --2- 172.21.15.27:0/362641400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32901071c0 0x7f329010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.727+0000 7f3295000640 1 --2- 172.21.15.27:0/362641400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290106820 0x7f3290106c30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.727+0000 7f3295000640 1 --2- 172.21.15.27:0/362641400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3290105790 0x7f3290105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.727+0000 7f3295000640 1 -- 172.21.15.27:0/362641400 >> 172.21.15.27:0/362641400 conn(0x7f3290101100 msgr2=0x7f32901035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.727+0000 7f3295000640 1 -- 172.21.15.27:0/362641400 shutdown_connections 2024-09-21T07:03:00.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.727+0000 7f3295000640 1 -- 172.21.15.27:0/362641400 wait complete. 2024-09-21T07:03:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 Processor -- start 2024-09-21T07:03:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 -- start start 2024-09-21T07:03:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290105790 0x7f329019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3290106820 0x7f329019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32901071c0 0x7f329019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3290111220 con 0x7f3290105790 2024-09-21T07:03:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32901110a0 con 0x7f32901071c0 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3295000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32901113a0 con 0x7f3290106820 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f3286a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3290106820 0x7f329019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.728+0000 7f328f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290105790 0x7f329019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f328fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32901071c0 0x7f329019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3290106820 0x7f329019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45708/0 (socket says 172.21.15.27:45708) 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 -- 172.21.15.27:0/2432352951 learned_addr learned my addr 172.21.15.27:0/2432352951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 -- 172.21.15.27:0/2432352951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32901071c0 msgr2=0x7f329019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32901071c0 0x7f329019d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 -- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290105790 msgr2=0x7f329019be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290105790 0x7f329019be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 -- 172.21.15.27:0/2432352951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329019d8b0 con 0x7f3290106820 2024-09-21T07:03:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f328fe00640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32901071c0 0x7f329019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:00.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.729+0000 7f3286a00640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3290106820 0x7f329019c4f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f327802aa70 tx=0x7f3278003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.730+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327803f070 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.730+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(24 keys) ==== 1045+0+0 (secure 0 0 0) 0x7f3278031af0 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.730+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329019db40 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.730+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278035030 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.730+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32901a42a0 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.731+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f3278042070 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.731+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329010c970 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.731+0000 7f328cc00640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f327403fb70 0x7f3274042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.732+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3278074e50 con 0x7f3290106820 2024-09-21T07:03:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.732+0000 7f328f400640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f327403fb70 0x7f3274042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.732+0000 7f328f400640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f327403fb70 0x7f3274042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3280009bd0 tx=0x7f3280005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.737+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f327804c350 con 0x7f3290106820 2024-09-21T07:03:00.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.942+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f329002e640 con 0x7f3290106820 2024-09-21T07:03:00.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.949+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 7 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f327803b460 con 0x7f3290106820 2024-09-21T07:03:00.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.949+0000 7f328cc00640 1 -- 172.21.15.27:0/2432352951 <== mon.1 v2:172.21.15.153:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v13)=0 v13) ==== 114+0+0 (secure 0 0 0) 0x7f327803ae60 con 0x7f3290106820 2024-09-21T07:03:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f327403fb70 msgr2=0x7f3274042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f327403fb70 0x7f3274042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3280009bd0 tx=0x7f3280005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3290106820 msgr2=0x7f329019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3290106820 0x7f329019c4f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f327802aa70 tx=0x7f3278003c10 comp rx=0 tx=0).stop 2024-09-21T07:03:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 shutdown_connections 2024-09-21T07:03:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3290106820 0x7f329019c4f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f327403fb70 0x7f3274042080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32901071c0 0x7f329019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 --2- 172.21.15.27:0/2432352951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3290105790 0x7f329019be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 >> 172.21.15.27:0/2432352951 conn(0x7f3290101100 msgr2=0x7f3290109b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 shutdown_connections 2024-09-21T07:03:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:00.952+0000 7f3295000640 1 -- 172.21.15.27:0/2432352951 wait complete. 2024-09-21T07:03:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T07:03:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:01.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T07:03:01.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:01.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:01.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: Deploying daemon mgr.z on smithi153 2024-09-21T07:03:01.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:00 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:01.251 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: Deploying daemon mgr.z on smithi153 2024-09-21T07:03:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:00 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:01.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T07:03:01.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Updating smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:01.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T07:03:01.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:01.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:01.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: Deploying daemon mgr.z on smithi153 2024-09-21T07:03:01.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:00 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:01.470 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.765 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.775 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.793 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:01.804 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi027') 2024-09-21T07:03:01.804 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T07:03:01.804 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T07:03:01.832 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:02.232 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:02.232 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 -- 172.21.15.27:0/4000317574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 msgr2=0x7f26dc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 --2- 172.21.15.27:0/4000317574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc103f70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009900 tx=0x7f26cc02a410 comp rx=0 tx=0).stop 2024-09-21T07:03:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 -- 172.21.15.27:0/4000317574 shutdown_connections 2024-09-21T07:03:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 --2- 172.21.15.27:0/4000317574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc1094a0 0x7f26dc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 --2- 172.21.15.27:0/4000317574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc109df0 0x7f26dc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 --2- 172.21.15.27:0/4000317574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc103f70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 -- 172.21.15.27:0/4000317574 >> 172.21.15.27:0/4000317574 conn(0x7f26dc07b850 msgr2=0x7f26dc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 -- 172.21.15.27:0/4000317574 shutdown_connections 2024-09-21T07:03:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.893+0000 7f26e0a00640 1 -- 172.21.15.27:0/4000317574 wait complete. 2024-09-21T07:03:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.895+0000 7f26e0a00640 1 Processor -- start 2024-09-21T07:03:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.895+0000 7f26e0a00640 1 -- start start 2024-09-21T07:03:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26e0a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc1094a0 0x7f26dc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc109df0 0x7f26dc1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26e0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1139e0 con 0x7f26dc109df0 2024-09-21T07:03:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26e0a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc113860 con 0x7f26dc103b60 2024-09-21T07:03:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26e0a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc113b60 con 0x7f26dc1094a0 2024-09-21T07:03:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26d9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26d9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc19c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53026/0 (socket says 172.21.15.27:53026) 2024-09-21T07:03:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26d9e00640 1 -- 172.21.15.27:0/253204880 learned_addr learned my addr 172.21.15.27:0/253204880 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26da800640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc109df0 0x7f26dc1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.896+0000 7f26d9400640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc1094a0 0x7f26dc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.897+0000 7f26d9e00640 1 -- 172.21.15.27:0/253204880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc1094a0 msgr2=0x7f26dc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.897+0000 7f26d9e00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc1094a0 0x7f26dc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.897+0000 7f26d9e00640 1 -- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc109df0 msgr2=0x7f26dc1a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.897+0000 7f26d9e00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc109df0 0x7f26dc1a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.897+0000 7f26d9e00640 1 -- 172.21.15.27:0/253204880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc1a2560 con 0x7f26dc103b60 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.897+0000 7f26d9400640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc1094a0 0x7f26dc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.897+0000 7f26d9e00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc19c5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02aa00 tx=0x7f26cc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.898+0000 7f26d2a00640 1 -- 172.21.15.27:0/253204880 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc03f070 con 0x7f26dc103b60 2024-09-21T07:03:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.898+0000 7f26d2a00640 1 -- 172.21.15.27:0/253204880 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f26cc03bb00 con 0x7f26dc103b60 2024-09-21T07:03:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.898+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc1a27f0 con 0x7f26dc103b60 2024-09-21T07:03:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.898+0000 7f26d2a00640 1 -- 172.21.15.27:0/253204880 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc035030 con 0x7f26dc103b60 2024-09-21T07:03:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.898+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc1a8fb0 con 0x7f26dc103b60 2024-09-21T07:03:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.899+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26bc005290 con 0x7f26dc103b60 2024-09-21T07:03:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.899+0000 7f26d2a00640 1 -- 172.21.15.27:0/253204880 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f26cc042070 con 0x7f26dc103b60 2024-09-21T07:03:02.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.900+0000 7f26d2a00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26c003fbc0 0x7f26c00420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:02.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.900+0000 7f26d2a00640 1 -- 172.21.15.27:0/253204880 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f26cc074b80 con 0x7f26dc103b60 2024-09-21T07:03:02.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.900+0000 7f26d9400640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26c003fbc0 0x7f26c00420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.901+0000 7f26d9400640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26c003fbc0 0x7f26c00420d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26dc1065a0 tx=0x7f26c8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:02.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:02.905+0000 7f26d2a00640 1 -- 172.21.15.27:0/253204880 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26cc04c350 con 0x7f26dc103b60 2024-09-21T07:03:02.927 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 21 07:03:02 smithi153 systemd[1]: Started Ceph mgr.z for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 -- 172.21.15.27:0/3132045224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4109df0 msgr2=0x7fedc4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 --2- 172.21.15.27:0/3132045224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4109df0 0x7fedc4110620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009ca0 tx=0x7fedbc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 -- 172.21.15.27:0/3132045224 shutdown_connections 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 --2- 172.21.15.27:0/3132045224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4109df0 0x7fedc4110620 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 --2- 172.21.15.27:0/3132045224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc41094a0 0x7fedc41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 --2- 172.21.15.27:0/3132045224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedc4103b60 0x7fedc4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 -- 172.21.15.27:0/3132045224 >> 172.21.15.27:0/3132045224 conn(0x7fedc407b850 msgr2=0x7fedc407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.075+0000 7fedcb400640 1 -- 172.21.15.27:0/3132045224 shutdown_connections 2024-09-21T07:03:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.076+0000 7fedcb400640 1 -- 172.21.15.27:0/3132045224 wait complete. 2024-09-21T07:03:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.077+0000 7fedcb400640 1 Processor -- start 2024-09-21T07:03:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.077+0000 7fedcb400640 1 -- start start 2024-09-21T07:03:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedcb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4103b60 0x7fedc419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedcb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc41094a0 0x7fedc419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedcb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedc4109df0 0x7fedc41a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedcb400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fedc4113990 con 0x7fedc4103b60 2024-09-21T07:03:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedcb400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fedc4113810 con 0x7fedc41094a0 2024-09-21T07:03:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedcb400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fedc4113b10 con 0x7fedc4109df0 2024-09-21T07:03:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedc8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4103b60 0x7fedc419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedc8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4103b60 0x7fedc419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38090/0 (socket says 172.21.15.27:38090) 2024-09-21T07:03:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedc3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc41094a0 0x7fedc419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedc8a00640 1 -- 172.21.15.27:0/212008978 learned_addr learned my addr 172.21.15.27:0/212008978 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.078+0000 7fedc9400640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedc4109df0 0x7fedc41a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc3e00640 1 -- 172.21.15.27:0/212008978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedc4109df0 msgr2=0x7fedc41a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc3e00640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedc4109df0 0x7fedc41a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc3e00640 1 -- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4103b60 msgr2=0x7fedc419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc3e00640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4103b60 0x7fedc419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc3e00640 1 -- 172.21.15.27:0/212008978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc41a24d0 con 0x7fedc41094a0 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc8a00640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4103b60 0x7fedc419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc3e00640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc41094a0 0x7fedc419cb00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedc406c9f0 tx=0x7fedb000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.079+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb000cd00 con 0x7fedc41094a0 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.080+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fedb00035f0 con 0x7fedc41094a0 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.080+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc41a27c0 con 0x7fedc41094a0 2024-09-21T07:03:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.080+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0010690 con 0x7fedc41094a0 2024-09-21T07:03:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.080+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc41a8f20 con 0x7fedc41094a0 2024-09-21T07:03:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.081+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc4103f70 con 0x7fedc41094a0 2024-09-21T07:03:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.082+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fedb0010830 con 0x7fedc41094a0 2024-09-21T07:03:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.082+0000 7fedc1600640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedac03fb70 0x7fedac042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.082+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fedb00549b0 con 0x7fedc41094a0 2024-09-21T07:03:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.083+0000 7fedc8a00640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedac03fb70 0x7fedac042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.083+0000 7fedc8a00640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedac03fb70 0x7fedac042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fedb8011500 tx=0x7fedb8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:03.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.088+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fedb0003810 con 0x7fedc41094a0 2024-09-21T07:03:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.118+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f26bc005580 con 0x7f26dc103b60 2024-09-21T07:03:03.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.123+0000 7f26d2a00640 1 -- 172.21.15.27:0/253204880 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v14)=0 v14) ==== 108+0+0 (secure 0 0 0) 0x7f26cc040d50 con 0x7f26dc103b60 2024-09-21T07:03:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26c003fbc0 msgr2=0x7f26c00420d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26c003fbc0 0x7f26c00420d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26dc1065a0 tx=0x7f26c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 msgr2=0x7f26dc19c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc19c5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02aa00 tx=0x7f26cc003c10 comp rx=0 tx=0).stop 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 shutdown_connections 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26dc1094a0 0x7f26dc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26c003fbc0 0x7f26c00420d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc109df0 0x7f26dc1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 --2- 172.21.15.27:0/253204880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26dc103b60 0x7f26dc19c5f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 >> 172.21.15.27:0/253204880 conn(0x7f26dc07b850 msgr2=0x7f26dc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 shutdown_connections 2024-09-21T07:03:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.126+0000 7f26e0a00640 1 -- 172.21.15.27:0/253204880 wait complete. 2024-09-21T07:03:03.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.303+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fedc41a9250 con 0x7fedc41094a0 2024-09-21T07:03:03.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.311+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb00093f0 con 0x7fedc41094a0 2024-09-21T07:03:03.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.311+0000 7fedc1600640 1 -- 172.21.15.27:0/212008978 <== mon.2 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7fedb0021950 con 0x7fedc41094a0 2024-09-21T07:03:03.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedac03fb70 msgr2=0x7fedac042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedac03fb70 0x7fedac042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fedb8011500 tx=0x7fedb8005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:03.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc41094a0 msgr2=0x7fedc419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc41094a0 0x7fedc419cb00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedc406c9f0 tx=0x7fedb000ee50 comp rx=0 tx=0).stop 2024-09-21T07:03:03.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 shutdown_connections 2024-09-21T07:03:03.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedc4109df0 0x7fedc41a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedac03fb70 0x7fedac042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc41094a0 0x7fedc419cb00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 --2- 172.21.15.27:0/212008978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedc4103b60 0x7fedc419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 >> 172.21.15.27:0/212008978 conn(0x7fedc407b850 msgr2=0x7fedc41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:03.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.313+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 shutdown_connections 2024-09-21T07:03:03.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.314+0000 7fedcb400640 1 -- 172.21.15.27:0/212008978 wait complete. 2024-09-21T07:03:03.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 -- 172.21.15.27:0/1875935415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 msgr2=0x7ffb6010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 --2- 172.21.15.27:0/1875935415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 0x7ffb6010d870 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009900 tx=0x7ffb4c02a410 comp rx=0 tx=0).stop 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 -- 172.21.15.27:0/1875935415 shutdown_connections 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 --2- 172.21.15.27:0/1875935415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6010de30 0x7ffb60110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 --2- 172.21.15.27:0/1875935415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 0x7ffb6010d870 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 --2- 172.21.15.27:0/1875935415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 0x7ffb60104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 -- 172.21.15.27:0/1875935415 >> 172.21.15.27:0/1875935415 conn(0x7ffb60079740 msgr2=0x7ffb60079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 -- 172.21.15.27:0/1875935415 shutdown_connections 2024-09-21T07:03:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.913+0000 7ffb67e00640 1 -- 172.21.15.27:0/1875935415 wait complete. 2024-09-21T07:03:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.914+0000 7ffb67e00640 1 Processor -- start 2024-09-21T07:03:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb67e00640 1 -- start start 2024-09-21T07:03:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb67e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 0x7ffb6019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb67e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 0x7ffb6019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb67e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6010de30 0x7ffb6019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb67e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60113a90 con 0x7ffb6010de30 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb67e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60113910 con 0x7ffb60105320 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb67e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60113c10 con 0x7ffb601049d0 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.915+0000 7ffb65400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 0x7ffb6019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 0x7ffb6019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45754/0 (socket says 172.21.15.27:45754) 2024-09-21T07:03:03.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 -- 172.21.15.27:0/765556077 learned_addr learned my addr 172.21.15.27:0/765556077 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6010de30 0x7ffb6019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb64a00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 0x7ffb6019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 -- 172.21.15.27:0/765556077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 msgr2=0x7ffb6019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 0x7ffb6019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 -- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6010de30 msgr2=0x7ffb6019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6010de30 0x7ffb6019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 -- 172.21.15.27:0/765556077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb601a0450 con 0x7ffb601049d0 2024-09-21T07:03:03.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb64a00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 0x7ffb6019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:03:03.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6010de30 0x7ffb6019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:03.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.916+0000 7ffb65400640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 0x7ffb6019a4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9c0 tx=0x7ffb5400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:03.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.917+0000 7ffb5e000640 1 -- 172.21.15.27:0/765556077 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400cd80 con 0x7ffb601049d0 2024-09-21T07:03:03.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.917+0000 7ffb5e000640 1 -- 172.21.15.27:0/765556077 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb540035f0 con 0x7ffb601049d0 2024-09-21T07:03:03.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.917+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb601a0740 con 0x7ffb601049d0 2024-09-21T07:03:03.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.917+0000 7ffb5e000640 1 -- 172.21.15.27:0/765556077 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54010690 con 0x7ffb601049d0 2024-09-21T07:03:03.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.917+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb601025f0 con 0x7ffb601049d0 2024-09-21T07:03:03.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.918+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6006c240 con 0x7ffb601049d0 2024-09-21T07:03:03.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.919+0000 7ffb5e000640 1 -- 172.21.15.27:0/765556077 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7ffb54010830 con 0x7ffb601049d0 2024-09-21T07:03:03.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.919+0000 7ffb5e000640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb4803fb70 0x7ffb48042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:03.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.919+0000 7ffb5e000640 1 -- 172.21.15.27:0/765556077 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ffb54054b90 con 0x7ffb601049d0 2024-09-21T07:03:03.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.919+0000 7ffb64a00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb4803fb70 0x7ffb48042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:03.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.920+0000 7ffb64a00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb4803fb70 0x7ffb48042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c000c00 tx=0x7ffb4c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:03.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:03.924+0000 7ffb5e000640 1 -- 172.21.15.27:0/765556077 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffb5401f5a0 con 0x7ffb601049d0 2024-09-21T07:03:04.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:04.010 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T07:03:04.010 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:04.010 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:04.010 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: Deploying daemon mgr.y on smithi119 2024-09-21T07:03:04.010 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:04.010 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:03 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:04.024 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.024 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: Deploying daemon mgr.y on smithi119 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:04.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:03 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:04.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.129+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7ffb60104de0 con 0x7ffb601049d0 2024-09-21T07:03:04.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.136+0000 7ffb5e000640 1 -- 172.21.15.27:0/765556077 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v16)=0 v16) ==== 120+0+0 (secure 0 0 0) 0x7ffb5401f340 con 0x7ffb601049d0 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.138+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb4803fb70 msgr2=0x7ffb48042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.138+0000 7ffb67e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb4803fb70 0x7ffb48042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c000c00 tx=0x7ffb4c031400 comp rx=0 tx=0).stop 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.138+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 msgr2=0x7ffb6019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.138+0000 7ffb67e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 0x7ffb6019a4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9c0 tx=0x7ffb5400ee80 comp rx=0 tx=0).stop 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 shutdown_connections 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb4803fb70 0x7ffb48042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6010de30 0x7ffb6019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb60105320 0x7ffb6019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 --2- 172.21.15.27:0/765556077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb601049d0 0x7ffb6019a4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 >> 172.21.15.27:0/765556077 conn(0x7ffb60079740 msgr2=0x7ffb601007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 shutdown_connections 2024-09-21T07:03:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.139+0000 7ffb67e00640 1 -- 172.21.15.27:0/765556077 wait complete. 2024-09-21T07:03:04.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:04.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:04.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T07:03:04.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:04.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:04.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: Deploying daemon mgr.y on smithi119 2024-09-21T07:03:04.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:04.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:03 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.645+0000 7f8163e00640 1 -- 172.21.15.27:0/3927388723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f815c076100 msgr2=0x7f815c10fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.645+0000 7f8163e00640 1 --2- 172.21.15.27:0/3927388723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f815c076100 0x7f815c10fb90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8148009900 tx=0x7f814802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.645+0000 7f8163e00640 1 -- 172.21.15.27:0/3927388723 shutdown_connections 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.645+0000 7f8163e00640 1 --2- 172.21.15.27:0/3927388723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c1100d0 0x7f815c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.645+0000 7f8163e00640 1 --2- 172.21.15.27:0/3927388723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f815c076100 0x7f815c10fb90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.645+0000 7f8163e00640 1 --2- 172.21.15.27:0/3927388723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f815c0757b0 0x7f815c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.645+0000 7f8163e00640 1 -- 172.21.15.27:0/3927388723 >> 172.21.15.27:0/3927388723 conn(0x7f815c0ff5e0 msgr2=0x7f815c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.646+0000 7f8163e00640 1 -- 172.21.15.27:0/3927388723 shutdown_connections 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.646+0000 7f8163e00640 1 -- 172.21.15.27:0/3927388723 wait complete. 2024-09-21T07:03:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 Processor -- start 2024-09-21T07:03:04.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 -- start start 2024-09-21T07:03:04.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f815c0757b0 0x7f815c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c076100 0x7f815c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f815c1100d0 0x7f815c1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f815c115cd0 con 0x7f815c1100d0 2024-09-21T07:03:04.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f815c115b50 con 0x7f815c0757b0 2024-09-21T07:03:04.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.647+0000 7f8163e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f815c115e50 con 0x7f815c076100 2024-09-21T07:03:04.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8161400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f815c0757b0 0x7f815c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:04.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c076100 0x7f815c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:04.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c076100 0x7f815c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45796/0 (socket says 172.21.15.27:45796) 2024-09-21T07:03:04.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8161400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f815c0757b0 0x7f815c19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53134/0 (socket says 172.21.15.27:53134) 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 -- 172.21.15.27:0/3017501903 learned_addr learned my addr 172.21.15.27:0/3017501903 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 -- 172.21.15.27:0/3017501903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f815c0757b0 msgr2=0x7f815c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f815c0757b0 0x7f815c19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 -- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f815c1100d0 msgr2=0x7f815c1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f815c1100d0 0x7f815c1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.648+0000 7f8160a00640 1 -- 172.21.15.27:0/3017501903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c1a25f0 con 0x7f815c076100 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.649+0000 7f8160a00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c076100 0x7f815c19cc20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8148005ab0 tx=0x7f8148003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.649+0000 7f815a000640 1 -- 172.21.15.27:0/3017501903 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814803f070 con 0x7f815c076100 2024-09-21T07:03:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.649+0000 7f815a000640 1 -- 172.21.15.27:0/3017501903 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8148031b50 con 0x7f815c076100 2024-09-21T07:03:04.652 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 systemd[1]: Starting Ceph mgr.y for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:03:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.649+0000 7f815a000640 1 -- 172.21.15.27:0/3017501903 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148035030 con 0x7f815c076100 2024-09-21T07:03:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.649+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c1a2880 con 0x7f815c076100 2024-09-21T07:03:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.649+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c1a9040 con 0x7f815c076100 2024-09-21T07:03:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.651+0000 7f815a000640 1 -- 172.21.15.27:0/3017501903 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f8148042070 con 0x7f815c076100 2024-09-21T07:03:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.651+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c075bc0 con 0x7f815c076100 2024-09-21T07:03:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.651+0000 7f815a000640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f814403fb70 0x7f8144042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.652+0000 7f815a000640 1 -- 172.21.15.27:0/3017501903 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8148074cc0 con 0x7f815c076100 2024-09-21T07:03:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.652+0000 7f8161400640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f814403fb70 0x7f8144042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.652+0000 7f8161400640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f814403fb70 0x7f8144042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f81500095f0 tx=0x7f8150005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:04.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.657+0000 7f815a000640 1 -- 172.21.15.27:0/3017501903 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f814803b460 con 0x7f815c076100 2024-09-21T07:03:04.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 -- 172.21.15.27:0/866968010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81092f0 msgr2=0x7fc2e8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 --2- 172.21.15.27:0/866968010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81092f0 0x7fc2e8109700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009900 tx=0x7fc2d002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 -- 172.21.15.27:0/866968010 shutdown_connections 2024-09-21T07:03:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 --2- 172.21.15.27:0/866968010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e8109c40 0x7fc2e8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 --2- 172.21.15.27:0/866968010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81092f0 0x7fc2e8109700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 --2- 172.21.15.27:0/866968010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e81039b0 0x7fc2e8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 -- 172.21.15.27:0/866968010 >> 172.21.15.27:0/866968010 conn(0x7fc2e80ff440 msgr2=0x7fc2e8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 -- 172.21.15.27:0/866968010 shutdown_connections 2024-09-21T07:03:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.826+0000 7fc2eda00640 1 -- 172.21.15.27:0/866968010 wait complete. 2024-09-21T07:03:04.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.827+0000 7fc2eda00640 1 Processor -- start 2024-09-21T07:03:04.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2eda00640 1 -- start start 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2eda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81039b0 0x7fc2e819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2eda00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e81092f0 0x7fc2e819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2eda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8109c40 0x7fc2e819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2eda00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8113900 con 0x7fc2e81039b0 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2eda00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8113780 con 0x7fc2e8109c40 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2eda00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8113a80 con 0x7fc2e81092f0 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81039b0 0x7fc2e819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2dfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e81092f0 0x7fc2e819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81039b0 0x7fc2e819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38184/0 (socket says 172.21.15.27:38184) 2024-09-21T07:03:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.828+0000 7fc2e7e00640 1 -- 172.21.15.27:0/4284669812 learned_addr learned my addr 172.21.15.27:0/4284669812 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.829+0000 7fc2ed000640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8109c40 0x7fc2e819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.829+0000 7fc2dfe00640 1 -- 172.21.15.27:0/4284669812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8109c40 msgr2=0x7fc2e819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.829+0000 7fc2dfe00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8109c40 0x7fc2e819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.829+0000 7fc2dfe00640 1 -- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81039b0 msgr2=0x7fc2e819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.829+0000 7fc2dfe00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81039b0 0x7fc2e819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.829+0000 7fc2dfe00640 1 -- 172.21.15.27:0/4284669812 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e81a02a0 con 0x7fc2e81092f0 2024-09-21T07:03:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.829+0000 7fc2dfe00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e81092f0 0x7fc2e819a8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2d002a910 tx=0x7fc2d0031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.830+0000 7fc2e5600640 1 -- 172.21.15.27:0/4284669812 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d003f070 con 0x7fc2e81092f0 2024-09-21T07:03:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.830+0000 7fc2e5600640 1 -- 172.21.15.27:0/4284669812 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d0004490 con 0x7fc2e81092f0 2024-09-21T07:03:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.830+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e81a0530 con 0x7fc2e81092f0 2024-09-21T07:03:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.830+0000 7fc2e5600640 1 -- 172.21.15.27:0/4284669812 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0035070 con 0x7fc2e81092f0 2024-09-21T07:03:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.830+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e80755f0 con 0x7fc2e81092f0 2024-09-21T07:03:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.831+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e8104c30 con 0x7fc2e81092f0 2024-09-21T07:03:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.831+0000 7fc2e5600640 1 -- 172.21.15.27:0/4284669812 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fc2d00037f0 con 0x7fc2e81092f0 2024-09-21T07:03:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.832+0000 7fc2e5600640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2cc03fb70 0x7fc2cc042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.832+0000 7fc2e5600640 1 -- 172.21.15.27:0/4284669812 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc2d0074970 con 0x7fc2e81092f0 2024-09-21T07:03:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.832+0000 7fc2e7e00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2cc03fb70 0x7fc2cc042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.833+0000 7fc2e7e00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2cc03fb70 0x7fc2cc042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80095f0 tx=0x7fc2d8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:04.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.837+0000 7fc2e5600640 1 -- 172.21.15.27:0/4284669812 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2d0003ae0 con 0x7fc2e81092f0 2024-09-21T07:03:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.874+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f815c1a9370 con 0x7f815c076100 2024-09-21T07:03:04.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.881+0000 7f815a000640 1 -- 172.21.15.27:0/3017501903 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v18)=0 v18) ==== 127+0+0 (secure 0 0 0) 0x7f8148031d00 con 0x7f815c076100 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.884+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f814403fb70 msgr2=0x7f8144042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.884+0000 7f8163e00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f814403fb70 0x7f8144042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f81500095f0 tx=0x7f8150005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.884+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c076100 msgr2=0x7f815c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.884+0000 7f8163e00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c076100 0x7f815c19cc20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8148005ab0 tx=0x7f8148003b90 comp rx=0 tx=0).stop 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 shutdown_connections 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f815c076100 0x7f815c19cc20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f814403fb70 0x7f8144042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f815c1100d0 0x7f815c1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 --2- 172.21.15.27:0/3017501903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f815c0757b0 0x7f815c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:04.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 >> 172.21.15.27:0/3017501903 conn(0x7f815c0ff5e0 msgr2=0x7f815c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:04.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 shutdown_connections 2024-09-21T07:03:04.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.885+0000 7f8163e00640 1 -- 172.21.15.27:0/3017501903 wait complete. 2024-09-21T07:03:04.922 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 podman[36025]: 2024-09-21 07:03:04.650508132 +0000 UTC m=+0.102486552 container create b4e81ec188d43802e3b7d660dcd7d9ad0c5985e5c3fca023144cd59685444d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-y, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T07:03:04.922 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 podman[36025]: 2024-09-21 07:03:04.563383526 +0000 UTC m=+0.015361957 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:03:04.922 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 podman[36025]: 2024-09-21 07:03:04.836417157 +0000 UTC m=+0.288395577 container init b4e81ec188d43802e3b7d660dcd7d9ad0c5985e5c3fca023144cd59685444d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T07:03:04.922 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 podman[36025]: 2024-09-21 07:03:04.839232164 +0000 UTC m=+0.291210585 container start b4e81ec188d43802e3b7d660dcd7d9ad0c5985e5c3fca023144cd59685444d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T07:03:04.922 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 ceph-mgr[36044]: -- 172.21.15.119:0/1443280577 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572f3c390e0 con 0x5572f89dc480 2024-09-21T07:03:04.922 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 bash[36025]: b4e81ec188d43802e3b7d660dcd7d9ad0c5985e5c3fca023144cd59685444d67 2024-09-21T07:03:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.076+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fc2e810ba30 con 0x7fc2e81092f0 2024-09-21T07:03:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.082+0000 7fc2e5600640 1 -- 172.21.15.27:0/4284669812 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v19)=0 v19) ==== 116+0+0 (secure 0 0 0) 0x7fc2d0005590 con 0x7fc2e81092f0 2024-09-21T07:03:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2cc03fb70 msgr2=0x7fc2cc042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2cc03fb70 0x7fc2cc042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80095f0 tx=0x7fc2d8005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e81092f0 msgr2=0x7fc2e819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e81092f0 0x7fc2e819a8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2d002a910 tx=0x7fc2d0031a20 comp rx=0 tx=0).stop 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 shutdown_connections 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e81092f0 0x7fc2e819a8d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2cc03fb70 0x7fc2cc042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e8109c40 0x7fc2e819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 --2- 172.21.15.27:0/4284669812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2e81039b0 0x7fc2e819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 >> 172.21.15.27:0/4284669812 conn(0x7fc2e80ff440 msgr2=0x7fc2e8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.084+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 shutdown_connections 2024-09-21T07:03:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.085+0000 7fc2eda00640 1 -- 172.21.15.27:0/4284669812 wait complete. 2024-09-21T07:03:05.284 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:03:04 smithi119 systemd[1]: Started Ceph mgr.y for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:05.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:05.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:05.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.816+0000 7f8f31e00640 1 -- 172.21.15.27:0/3584009789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0757b0 msgr2=0x7f8f2c075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:05.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.816+0000 7f8f31e00640 1 --2- 172.21.15.27:0/3584009789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0757b0 0x7f8f2c075bc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009900 tx=0x7f8f1c02a410 comp rx=0 tx=0).stop 2024-09-21T07:03:05.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.816+0000 7f8f31e00640 1 -- 172.21.15.27:0/3584009789 shutdown_connections 2024-09-21T07:03:05.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.816+0000 7f8f31e00640 1 --2- 172.21.15.27:0/3584009789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f2c1100e0 0x7f8f2c112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.816+0000 7f8f31e00640 1 --2- 172.21.15.27:0/3584009789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 0x7f8f2c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.816+0000 7f8f31e00640 1 --2- 172.21.15.27:0/3584009789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0757b0 0x7f8f2c075bc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.816+0000 7f8f31e00640 1 -- 172.21.15.27:0/3584009789 >> 172.21.15.27:0/3584009789 conn(0x7f8f2c0ff590 msgr2=0x7f8f2c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:05.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.817+0000 7f8f31e00640 1 -- 172.21.15.27:0/3584009789 shutdown_connections 2024-09-21T07:03:05.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.817+0000 7f8f31e00640 1 -- 172.21.15.27:0/3584009789 wait complete. 2024-09-21T07:03:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.819+0000 7f8f31e00640 1 Processor -- start 2024-09-21T07:03:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.819+0000 7f8f31e00640 1 -- start start 2024-09-21T07:03:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f31e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f2c0757b0 0x7f8f2c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f31e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 0x7f8f2c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c1100e0 0x7f8f2c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f31e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c115ce0 con 0x7f8f2c1100e0 2024-09-21T07:03:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f31e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c115b60 con 0x7f8f2c076100 2024-09-21T07:03:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f31e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c115e60 con 0x7f8f2c0757b0 2024-09-21T07:03:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f2a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 0x7f8f2c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f2a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 0x7f8f2c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53180/0 (socket says 172.21.15.27:53180) 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f2a000640 1 -- 172.21.15.27:0/741046710 learned_addr learned my addr 172.21.15.27:0/741046710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f2aa00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f2c0757b0 0x7f8f2c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.820+0000 7f8f2b400640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c1100e0 0x7f8f2c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.821+0000 7f8f2a000640 1 -- 172.21.15.27:0/741046710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f2c0757b0 msgr2=0x7f8f2c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.821+0000 7f8f2a000640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f2c0757b0 0x7f8f2c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.821+0000 7f8f2a000640 1 -- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c1100e0 msgr2=0x7f8f2c1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.821+0000 7f8f2a000640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c1100e0 0x7f8f2c1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:05.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.821+0000 7f8f2a000640 1 -- 172.21.15.27:0/741046710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c1a25e0 con 0x7f8f2c076100 2024-09-21T07:03:05.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.822+0000 7f8f2a000640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 0x7f8f2c19cc10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800e9c0 tx=0x7f8f1800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:05.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.822+0000 7f8f30a00640 1 -- 172.21.15.27:0/741046710 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800cd80 con 0x7f8f2c076100 2024-09-21T07:03:05.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.822+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c1a28d0 con 0x7f8f2c076100 2024-09-21T07:03:05.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.822+0000 7f8f30a00640 1 -- 172.21.15.27:0/741046710 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f180035f0 con 0x7f8f2c076100 2024-09-21T07:03:05.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.822+0000 7f8f30a00640 1 -- 172.21.15.27:0/741046710 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18010690 con 0x7f8f2c076100 2024-09-21T07:03:05.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.822+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c1a9030 con 0x7f8f2c076100 2024-09-21T07:03:05.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.824+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2c075bc0 con 0x7f8f2c076100 2024-09-21T07:03:05.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.824+0000 7f8f30a00640 1 -- 172.21.15.27:0/741046710 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f8f18004020 con 0x7f8f2c076100 2024-09-21T07:03:05.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.825+0000 7f8f30a00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f1003fb70 0x7f8f10042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:05.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.825+0000 7f8f30a00640 1 -- 172.21.15.27:0/741046710 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8f180549a0 con 0x7f8f2c076100 2024-09-21T07:03:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.826+0000 7f8f2aa00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f1003fb70 0x7f8f10042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.827+0000 7f8f2aa00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f1003fb70 0x7f8f10042080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c000c00 tx=0x7f8f1c005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:05.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.831+0000 7f8f30a00640 1 -- 172.21.15.27:0/741046710 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f180037a0 con 0x7f8f2c076100 2024-09-21T07:03:06.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.056+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f8f2c1a93e0 con 0x7f8f2c076100 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.057+0000 7f8f30a00640 1 -- 172.21.15.27:0/741046710 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v21)=0 v21) ==== 106+0+0 (secure 0 0 0) 0x7f8f18003ac0 con 0x7f8f2c076100 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.060+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f1003fb70 msgr2=0x7f8f10042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.060+0000 7f8f31e00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f1003fb70 0x7f8f10042080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c000c00 tx=0x7f8f1c005d50 comp rx=0 tx=0).stop 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.060+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 msgr2=0x7f8f2c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.060+0000 7f8f31e00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 0x7f8f2c19cc10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800e9c0 tx=0x7f8f1800ee80 comp rx=0 tx=0).stop 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 shutdown_connections 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f1003fb70 0x7f8f10042080 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c1100e0 0x7f8f2c1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f2c076100 0x7f8f2c19cc10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 --2- 172.21.15.27:0/741046710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f2c0757b0 0x7f8f2c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 >> 172.21.15.27:0/741046710 conn(0x7f8f2c0ff590 msgr2=0x7f8f2c107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 shutdown_connections 2024-09-21T07:03:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.061+0000 7f8f31e00640 1 -- 172.21.15.27:0/741046710 wait complete. 2024-09-21T07:03:06.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:06.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:06.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:06.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:06.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:06.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:06.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:06.939 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:06.939 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:06.939 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:06.939 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:06.939 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:06.940 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi027') 2024-09-21T07:03:06.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.446+0000 7f5207e00640 1 -- 172.21.15.27:0/3716110082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 msgr2=0x7f520810f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.446+0000 7f5207e00640 1 --2- 172.21.15.27:0/3716110082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 0x7f520810f9b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009900 tx=0x7f51f402a410 comp rx=0 tx=0).stop 2024-09-21T07:03:06.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.446+0000 7f5207e00640 1 -- 172.21.15.27:0/3716110082 shutdown_connections 2024-09-21T07:03:06.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.446+0000 7f5207e00640 1 --2- 172.21.15.27:0/3716110082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520810fef0 0x7f5208112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.447+0000 7f5207e00640 1 --2- 172.21.15.27:0/3716110082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 0x7f520810f9b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.447+0000 7f5207e00640 1 --2- 172.21.15.27:0/3716110082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52080757b0 0x7f5208075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.447+0000 7f5207e00640 1 -- 172.21.15.27:0/3716110082 >> 172.21.15.27:0/3716110082 conn(0x7f52080ff3e0 msgr2=0x7f5208101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.447+0000 7f5207e00640 1 -- 172.21.15.27:0/3716110082 shutdown_connections 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.447+0000 7f5207e00640 1 -- 172.21.15.27:0/3716110082 wait complete. 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 Processor -- start 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 -- start start 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52080757b0 0x7f520819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 0x7f520819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520810fef0 0x7f52081a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5208115a60 con 0x7f5208076100 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52081158e0 con 0x7f520810fef0 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.448+0000 7f5207e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5208115be0 con 0x7f52080757b0 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5206a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52080757b0 0x7f520819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5206000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 0x7f520819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5207400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520810fef0 0x7f52081a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5206a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52080757b0 0x7f520819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45774/0 (socket says 172.21.15.27:45774) 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5206a00640 1 -- 172.21.15.27:0/3105712376 learned_addr learned my addr 172.21.15.27:0/3105712376 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5207400640 1 -- 172.21.15.27:0/3105712376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52080757b0 msgr2=0x7f520819c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5207400640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52080757b0 0x7f520819c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5207400640 1 -- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 msgr2=0x7f520819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5207400640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 0x7f520819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5207400640 1 -- 172.21.15.27:0/3105712376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52081a2440 con 0x7f520810fef0 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f5207400640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520810fef0 0x7f52081a1d10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5200009fd0 tx=0x7f520000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.449+0000 7f520ca00640 1 -- 172.21.15.27:0/3105712376 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520000ce20 con 0x7f520810fef0 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.450+0000 7f520ca00640 1 -- 172.21.15.27:0/3105712376 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5200005020 con 0x7f520810fef0 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.450+0000 7f520ca00640 1 -- 172.21.15.27:0/3105712376 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200010690 con 0x7f520810fef0 2024-09-21T07:03:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.450+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52081a2730 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.450+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52081a8e90 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.451+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5208075bc0 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.451+0000 7f520ca00640 1 -- 172.21.15.27:0/3105712376 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f5200004b20 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.452+0000 7f520ca00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51ec03fb70 0x7f51ec042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.452+0000 7f520ca00640 1 -- 172.21.15.27:0/3105712376 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5200054be0 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.452+0000 7f5206a00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51ec03fb70 0x7f51ec042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.453+0000 7f5206a00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51ec03fb70 0x7f51ec042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009c90 tx=0x7f51f8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.457+0000 7f520ca00640 1 -- 172.21.15.27:0/3105712376 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5200011dc0 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.671+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f52081a9170 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.677+0000 7f520ca00640 1 -- 172.21.15.27:0/3105712376 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v17)=0 v17) ==== 106+0+0 (secure 0 0 0) 0x7f520001fd00 con 0x7f520810fef0 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51ec03fb70 msgr2=0x7f51ec042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51ec03fb70 0x7f51ec042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009c90 tx=0x7f51f8005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520810fef0 msgr2=0x7f52081a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520810fef0 0x7f52081a1d10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5200009fd0 tx=0x7f520000ef10 comp rx=0 tx=0).stop 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 shutdown_connections 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51ec03fb70 0x7f51ec042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f520810fef0 0x7f52081a1d10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5208076100 0x7f520819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 --2- 172.21.15.27:0/3105712376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52080757b0 0x7f520819c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 >> 172.21.15.27:0/3105712376 conn(0x7f52080ff3e0 msgr2=0x7f5208108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 shutdown_connections 2024-09-21T07:03:06.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:04.680+0000 7f5207e00640 1 -- 172.21.15.27:0/3105712376 wait complete. 2024-09-21T07:03:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T07:03:07.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:07.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 -- 172.21.15.27:0/1737432987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 msgr2=0x7fcad410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 --2- 172.21.15.27:0/1737432987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 0x7fcad410d660 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009900 tx=0x7fcac002a410 comp rx=0 tx=0).stop 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 -- 172.21.15.27:0/1737432987 shutdown_connections 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 --2- 172.21.15.27:0/1737432987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 0x7fcad4110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 --2- 172.21.15.27:0/1737432987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 0x7fcad410d660 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 --2- 172.21.15.27:0/1737432987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcad41047c0 0x7fcad4104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 -- 172.21.15.27:0/1737432987 >> 172.21.15.27:0/1737432987 conn(0x7fcad40fe2c0 msgr2=0x7fcad4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 -- 172.21.15.27:0/1737432987 shutdown_connections 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.483+0000 7fcad3e00640 1 -- 172.21.15.27:0/1737432987 wait complete. 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.485+0000 7fcad3e00640 1 Processor -- start 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.485+0000 7fcad3e00640 1 -- start start 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.485+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcad41047c0 0x7fcad419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.485+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 0x7fcad419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 0x7fcad419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41138d0 con 0x7fcad4105110 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113750 con 0x7fcad410dc20 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113a50 con 0x7fcad41047c0 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 0x7fcad419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 0x7fcad419fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53162/0 (socket says 172.21.15.27:53162) 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad3400640 1 -- 172.21.15.27:0/2911857836 learned_addr learned my addr 172.21.15.27:0/2911857836 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad2000640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 0x7fcad419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.486+0000 7fcad2a00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcad41047c0 0x7fcad419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.487+0000 7fcad3400640 1 -- 172.21.15.27:0/2911857836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcad41047c0 msgr2=0x7fcad419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.487+0000 7fcad3400640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcad41047c0 0x7fcad419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.487+0000 7fcad3400640 1 -- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 msgr2=0x7fcad419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.487+0000 7fcad3400640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 0x7fcad419a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.487+0000 7fcad3400640 1 -- 172.21.15.27:0/2911857836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad41a0220 con 0x7fcad410dc20 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.487+0000 7fcad3400640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 0x7fcad419fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00db20 tx=0x7fcacc00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.487+0000 7fcacb400640 1 -- 172.21.15.27:0/2911857836 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc017070 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.488+0000 7fcacb400640 1 -- 172.21.15.27:0/2911857836 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcacc003e20 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.488+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41a0510 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.488+0000 7fcacb400640 1 -- 172.21.15.27:0/2911857836 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc010460 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.488+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad406cde0 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.489+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad4104bd0 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.490+0000 7fcacb400640 1 -- 172.21.15.27:0/2911857836 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fcacc010600 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.490+0000 7fcacb400640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcab803fb70 0x7fcab8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.490+0000 7fcacb400640 1 -- 172.21.15.27:0/2911857836 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcacc054400 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.490+0000 7fcad2a00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcab803fb70 0x7fcab8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.491+0000 7fcad2a00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcab803fb70 0x7fcab8042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009b60 tx=0x7fcac4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.495+0000 7fcacb400640 1 -- 172.21.15.27:0/2911857836 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcacc00fe90 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.743+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fcad406d040 con 0x7fcad410dc20 2024-09-21T07:03:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.751+0000 7fcacb400640 1 -- 172.21.15.27:0/2911857836 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7fcacc00e350 con 0x7fcad410dc20 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcab803fb70 msgr2=0x7fcab8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcab803fb70 0x7fcab8042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009b60 tx=0x7fcac4005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 msgr2=0x7fcad419fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 0x7fcad419fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00db20 tx=0x7fcacc00b260 comp rx=0 tx=0).stop 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 shutdown_connections 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcab803fb70 0x7fcab8042080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad410dc20 0x7fcad419fb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad4105110 0x7fcad419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 --2- 172.21.15.27:0/2911857836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcad41047c0 0x7fcad419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 >> 172.21.15.27:0/2911857836 conn(0x7fcad40fe2c0 msgr2=0x7fcad4106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 shutdown_connections 2024-09-21T07:03:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.755+0000 7fcad3e00640 1 -- 172.21.15.27:0/2911857836 wait complete. 2024-09-21T07:03:07.771 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T07:03:07.771 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T07:03:07.771 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:07.771 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: Standby manager daemon z started 2024-09-21T07:03:07.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T07:03:07.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T07:03:07.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T07:03:07.772 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:07 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T07:03:07.865 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:07.866 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:08.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T07:03:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T07:03:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: Standby manager daemon z started 2024-09-21T07:03:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T07:03:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T07:03:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T07:03:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:07 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T07:03:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T07:03:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T07:03:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: Standby manager daemon z started 2024-09-21T07:03:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T07:03:08.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T07:03:08.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T07:03:08.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:07 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.153:0/472728950' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T07:03:08.495 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi027') 2024-09-21T07:03:09.512 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:09 smithi027 ceph-mon[30779]: mgrmap e19: x(active, since 14s), standbys: z 2024-09-21T07:03:09.512 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T07:03:09.512 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:09 smithi119 ceph-mon[34519]: mgrmap e19: x(active, since 14s), standbys: z 2024-09-21T07:03:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T07:03:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:09.551 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:09.552 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:09.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:09 smithi153 ceph-mon[34747]: mgrmap e19: x(active, since 14s), standbys: z 2024-09-21T07:03:09.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T07:03:09.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 -- 172.21.15.27:0/489557570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be0105110 msgr2=0x7f9be010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 --2- 172.21.15.27:0/489557570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be0105110 0x7f9be010d660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009900 tx=0x7f9bcc02a410 comp rx=0 tx=0).stop 2024-09-21T07:03:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 -- 172.21.15.27:0/489557570 shutdown_connections 2024-09-21T07:03:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 --2- 172.21.15.27:0/489557570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be010dc20 0x7f9be0110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 --2- 172.21.15.27:0/489557570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be0105110 0x7f9be010d660 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 --2- 172.21.15.27:0/489557570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be01047c0 0x7f9be0104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 -- 172.21.15.27:0/489557570 >> 172.21.15.27:0/489557570 conn(0x7f9be00fe2c0 msgr2=0x7f9be0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.689+0000 7f9be5200640 1 -- 172.21.15.27:0/489557570 shutdown_connections 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.690+0000 7f9be5200640 1 -- 172.21.15.27:0/489557570 wait complete. 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.691+0000 7f9be5200640 1 Processor -- start 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.691+0000 7f9be5200640 1 -- start start 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9be5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be01047c0 0x7f9be0102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9be5200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0105110 0x7f9be0102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9be5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be010dc20 0x7f9be0102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9be5200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0113890 con 0x7f9be01047c0 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9be5200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0113710 con 0x7f9be010dc20 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9be5200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0113a10 con 0x7f9be0105110 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9bdf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be01047c0 0x7f9be0102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.692+0000 7f9bdea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0105110 0x7f9be0102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0105110 0x7f9be0102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45838/0 (socket says 172.21.15.27:45838) 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdea00640 1 -- 172.21.15.27:0/1363310500 learned_addr learned my addr 172.21.15.27:0/1363310500 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdfe00640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be010dc20 0x7f9be0102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdea00640 1 -- 172.21.15.27:0/1363310500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be010dc20 msgr2=0x7f9be0102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdea00640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be010dc20 0x7f9be0102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdea00640 1 -- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be01047c0 msgr2=0x7f9be0102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdea00640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be01047c0 0x7f9be0102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.693+0000 7f9bdea00640 1 -- 172.21.15.27:0/1363310500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be01a85c0 con 0x7f9be0105110 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.694+0000 7f9bdea00640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0105110 0x7f9be0102a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc005ab0 tx=0x7f9bcc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.694+0000 7f9bd7e00640 1 -- 172.21.15.27:0/1363310500 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc03f070 con 0x7f9be0105110 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.694+0000 7f9bd7e00640 1 -- 172.21.15.27:0/1363310500 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc031b50 con 0x7f9be0105110 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.694+0000 7f9bd7e00640 1 -- 172.21.15.27:0/1363310500 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc035030 con 0x7f9be0105110 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.694+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be01a8850 con 0x7f9be0105110 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.694+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be01a8d80 con 0x7f9be0105110 2024-09-21T07:03:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.696+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be0104bd0 con 0x7f9be0105110 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.696+0000 7f9bd7e00640 1 -- 172.21.15.27:0/1363310500 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f9bcc042070 con 0x7f9be0105110 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.697+0000 7f9bd7e00640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bc403fb70 0x7f9bc4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.697+0000 7f9bd7e00640 1 -- 172.21.15.27:0/1363310500 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9bcc074c70 con 0x7f9be0105110 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.697+0000 7f9bdf400640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bc403fb70 0x7f9bc4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.698+0000 7f9bdf400640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bc403fb70 0x7f9bc4042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9be0103f50 tx=0x7f9bd0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.702+0000 7f9bd7e00640 1 -- 172.21.15.27:0/1363310500 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9bcc03b460 con 0x7f9be0105110 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.949+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f9be0103800 con 0x7f9be0105110 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.954+0000 7f9bd7e00640 1 -- 172.21.15.27:0/1363310500 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v21)=0 v21) ==== 111+0+0 (secure 0 0 0) 0x7f9bcc0316a0 con 0x7f9be0105110 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bc403fb70 msgr2=0x7f9bc4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bc403fb70 0x7f9bc4042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9be0103f50 tx=0x7f9bd0005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0105110 msgr2=0x7f9be0102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0105110 0x7f9be0102a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc005ab0 tx=0x7f9bcc003b90 comp rx=0 tx=0).stop 2024-09-21T07:03:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 shutdown_connections 2024-09-21T07:03:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0105110 0x7f9be0102a40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bc403fb70 0x7f9bc4042080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be010dc20 0x7f9be0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 --2- 172.21.15.27:0/1363310500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be01047c0 0x7f9be0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.957+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 >> 172.21.15.27:0/1363310500 conn(0x7f9be00fe2c0 msgr2=0x7f9be0107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.958+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 shutdown_connections 2024-09-21T07:03:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:05.958+0000 7f9be5200640 1 -- 172.21.15.27:0/1363310500 wait complete. 2024-09-21T07:03:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 -- 172.21.15.27:0/1301090547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 msgr2=0x7f20d4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:11.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 --2- 172.21.15.27:0/1301090547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 0x7f20d4103dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 -- 172.21.15.27:0/1301090547 shutdown_connections 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 --2- 172.21.15.27:0/1301090547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d4109c40 0x7f20d4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 --2- 172.21.15.27:0/1301090547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 0x7f20d4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 --2- 172.21.15.27:0/1301090547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 0x7f20d4103dc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 -- 172.21.15.27:0/1301090547 >> 172.21.15.27:0/1301090547 conn(0x7f20d40ff440 msgr2=0x7f20d4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 -- 172.21.15.27:0/1301090547 shutdown_connections 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.989+0000 7f20d9e00640 1 -- 172.21.15.27:0/1301090547 wait complete. 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.990+0000 7f20d9e00640 1 Processor -- start 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.990+0000 7f20d9e00640 1 -- start start 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 0x7f20d4077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 0x7f20d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d4109c40 0x7f20d4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:11.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d9e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41139b0 con 0x7f20d41039b0 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d9e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4113830 con 0x7f20d41092f0 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d9e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4113b30 con 0x7f20d4109c40 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 0x7f20d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 0x7f20d4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53198/0 (socket says 172.21.15.27:53198) 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 -- 172.21.15.27:0/2513412979 learned_addr learned my addr 172.21.15.27:0/2513412979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d9400640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d4109c40 0x7f20d4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d8a00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 0x7f20d4077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 -- 172.21.15.27:0/2513412979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d4109c40 msgr2=0x7f20d4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d4109c40 0x7f20d4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 -- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 msgr2=0x7f20d4077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 0x7f20d4077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.991+0000 7f20d3e00640 1 -- 172.21.15.27:0/2513412979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d4076470 con 0x7f20d41092f0 2024-09-21T07:03:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.992+0000 7f20d3e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 0x7f20d4075710 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f20c000e970 tx=0x7f20c000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.992+0000 7f20d1600640 1 -- 172.21.15.27:0/2513412979 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c000cd80 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.992+0000 7f20d1600640 1 -- 172.21.15.27:0/2513412979 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c00035f0 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.992+0000 7f20d1600640 1 -- 172.21.15.27:0/2513412979 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0010690 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.992+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d41acd40 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.992+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41ad1f0 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.993+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d4103dc0 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.993+0000 7f20d1600640 1 -- 172.21.15.27:0/2513412979 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f20c0004020 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.994+0000 7f20d1600640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20bc03fb70 0x7f20bc042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.994+0000 7f20d1600640 1 -- 172.21.15.27:0/2513412979 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f20c00549b0 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.994+0000 7f20d8a00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20bc03fb70 0x7f20bc042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.995+0000 7f20d8a00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20bc03fb70 0x7f20bc042080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f20c8006010 tx=0x7f20c8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:06.999+0000 7f20d1600640 1 -- 172.21.15.27:0/2513412979 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f20c0004460 con 0x7f20d41092f0 2024-09-21T07:03:11.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.211+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f20d4076a10 con 0x7f20d41092f0 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.218+0000 7f20d1600640 1 -- 172.21.15.27:0/2513412979 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v22)=0 v22) ==== 117+0+0 (secure 0 0 0) 0x7f20c0013190 con 0x7f20d41092f0 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20bc03fb70 msgr2=0x7f20bc042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20bc03fb70 0x7f20bc042080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f20c8006010 tx=0x7f20c8031400 comp rx=0 tx=0).stop 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 msgr2=0x7f20d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 0x7f20d4075710 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f20c000e970 tx=0x7f20c000ee30 comp rx=0 tx=0).stop 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 shutdown_connections 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20d4109c40 0x7f20d4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20bc03fb70 0x7f20bc042080 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d41092f0 0x7f20d4075710 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 --2- 172.21.15.27:0/2513412979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d41039b0 0x7f20d4077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 >> 172.21.15.27:0/2513412979 conn(0x7f20d40ff440 msgr2=0x7f20d4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 shutdown_connections 2024-09-21T07:03:11.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:07.221+0000 7f20d9e00640 1 -- 172.21.15.27:0/2513412979 wait complete. 2024-09-21T07:03:11.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: Reconfiguring mon.c (monmap changed)... 2024-09-21T07:03:11.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: Reconfiguring daemon mon.c on smithi153 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: Standby manager daemon y started 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:10 smithi119 ceph-mon[34519]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: Reconfiguring mon.c (monmap changed)... 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:03:11.047 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: Reconfiguring daemon mon.c on smithi153 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: Standby manager daemon y started 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:10 smithi153 ceph-mon[34747]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: Reconfiguring mon.c (monmap changed)... 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:11.048 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: Reconfiguring daemon mon.c on smithi153 2024-09-21T07:03:11.049 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: Standby manager daemon y started 2024-09-21T07:03:11.049 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T07:03:11.049 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T07:03:11.049 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T07:03:11.049 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:10 smithi027 ceph-mon[30779]: from='mgr.? 172.21.15.119:0/1864631124' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: mgrmap e20: x(active, since 17s), standbys: z, y 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:11.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: mgrmap e20: x(active, since 17s), standbys: z, y 2024-09-21T07:03:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T07:03:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: mgrmap e20: x(active, since 17s), standbys: z, y 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:11.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:12.479 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:12.479 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:12.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 -- 172.21.15.27:0/3783300172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d74109df0 msgr2=0x7f8d74110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 --2- 172.21.15.27:0/3783300172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d74109df0 0x7f8d74110620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009ca0 tx=0x7f8d6c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 -- 172.21.15.27:0/3783300172 shutdown_connections 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 --2- 172.21.15.27:0/3783300172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d741094a0 0x7f8d741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 --2- 172.21.15.27:0/3783300172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d74109df0 0x7f8d74110620 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 --2- 172.21.15.27:0/3783300172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d74103b60 0x7f8d74103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 -- 172.21.15.27:0/3783300172 >> 172.21.15.27:0/3783300172 conn(0x7f8d7407b850 msgr2=0x7f8d7407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 -- 172.21.15.27:0/3783300172 shutdown_connections 2024-09-21T07:03:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.872+0000 7f8d7b000640 1 -- 172.21.15.27:0/3783300172 wait complete. 2024-09-21T07:03:12.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.873+0000 7f8d7b000640 1 Processor -- start 2024-09-21T07:03:12.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.873+0000 7f8d7b000640 1 -- start start 2024-09-21T07:03:12.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d7b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d74103b60 0x7f8d7419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d7b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d741094a0 0x7f8d7419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d7b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d74109df0 0x7f8d7419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d7b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74113ab0 con 0x7f8d741094a0 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d7b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74113930 con 0x7f8d74109df0 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d7b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74113c30 con 0x7f8d74103b60 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d741094a0 0x7f8d7419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d741094a0 0x7f8d7419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41968/0 (socket says 172.21.15.27:41968) 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d6be00640 1 -- 172.21.15.27:0/338805617 learned_addr learned my addr 172.21.15.27:0/338805617 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d79000640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d74109df0 0x7f8d7419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.874+0000 7f8d73e00640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d74103b60 0x7f8d7419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d6be00640 1 -- 172.21.15.27:0/338805617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d74103b60 msgr2=0x7f8d7419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d6be00640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d74103b60 0x7f8d7419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d6be00640 1 -- 172.21.15.27:0/338805617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d74109df0 msgr2=0x7f8d7419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d6be00640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d74109df0 0x7f8d7419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d6be00640 1 -- 172.21.15.27:0/338805617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d741a03b0 con 0x7f8d741094a0 2024-09-21T07:03:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d6be00640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d741094a0 0x7f8d7419a9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e9f0 tx=0x7f8d5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d71600640 1 -- 172.21.15.27:0/338805617 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c00ccc0 con 0x7f8d741094a0 2024-09-21T07:03:12.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d741a06a0 con 0x7f8d741094a0 2024-09-21T07:03:12.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d71600640 1 -- 172.21.15.27:0/338805617 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5c0108e0 con 0x7f8d741094a0 2024-09-21T07:03:12.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.875+0000 7f8d71600640 1 -- 172.21.15.27:0/338805617 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c018860 con 0x7f8d741094a0 2024-09-21T07:03:12.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.876+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d741a6e00 con 0x7f8d741094a0 2024-09-21T07:03:12.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.877+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d74103f70 con 0x7f8d741094a0 2024-09-21T07:03:12.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.879+0000 7f8d71600640 1 -- 172.21.15.27:0/338805617 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f8d5c010430 con 0x7f8d741094a0 2024-09-21T07:03:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.881+0000 7f8d71600640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d580b64d0 0x7f8d580b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.881+0000 7f8d71600640 1 -- 172.21.15.27:0/338805617 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8d5c0e5a50 con 0x7f8d741094a0 2024-09-21T07:03:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.881+0000 7f8d73e00640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d580b64d0 0x7f8d580b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.882+0000 7f8d73e00640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d580b64d0 0x7f8d580b89e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009bd0 tx=0x7f8d64005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:12.885+0000 7f8d71600640 1 -- 172.21.15.27:0/338805617 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d5c0b2220 con 0x7f8d741094a0 2024-09-21T07:03:13.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.105+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f8d7402e640 con 0x7f8d741094a0 2024-09-21T07:03:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.109+0000 7f8d71600640 1 -- 172.21.15.27:0/338805617 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v23)=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f8d5c0b7160 con 0x7f8d741094a0 2024-09-21T07:03:13.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d580b64d0 msgr2=0x7f8d580b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:13.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d580b64d0 0x7f8d580b89e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009bd0 tx=0x7f8d64005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:13.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d741094a0 msgr2=0x7f8d7419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d741094a0 0x7f8d7419a9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e9f0 tx=0x7f8d5c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 shutdown_connections 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d580b64d0 0x7f8d580b89e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d74109df0 0x7f8d7419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d741094a0 0x7f8d7419a9e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 --2- 172.21.15.27:0/338805617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d74103b60 0x7f8d7419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 >> 172.21.15.27:0/338805617 conn(0x7f8d7407b850 msgr2=0x7f8d74105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 shutdown_connections 2024-09-21T07:03:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.112+0000 7f8d7b000640 1 -- 172.21.15.27:0/338805617 wait complete. 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 -- 172.21.15.27:0/4069828029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c741094b0 msgr2=0x7f4c741098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 --2- 172.21.15.27:0/4069828029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c741094b0 0x7f4c741098c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009900 tx=0x7f4c5c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 -- 172.21.15.27:0/4069828029 shutdown_connections 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 --2- 172.21.15.27:0/4069828029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c74109e00 0x7f4c74110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 --2- 172.21.15.27:0/4069828029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c741094b0 0x7f4c741098c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 --2- 172.21.15.27:0/4069828029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74103b70 0x7f4c74103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 -- 172.21.15.27:0/4069828029 >> 172.21.15.27:0/4069828029 conn(0x7f4c740ff5e0 msgr2=0x7f4c74101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.236+0000 7f4c7a600640 1 -- 172.21.15.27:0/4069828029 shutdown_connections 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.237+0000 7f4c7a600640 1 -- 172.21.15.27:0/4069828029 wait complete. 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 Processor -- start 2024-09-21T07:03:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 -- start start 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c74103b70 0x7f4c7419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c741094b0 0x7f4c7419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c74109e00 0x7f4c741a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74113ac0 con 0x7f4c74103b70 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74113940 con 0x7f4c741094b0 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.238+0000 7f4c7a600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74113c40 con 0x7f4c74109e00 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c74103b70 0x7f4c7419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c74103b70 0x7f4c7419c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41992/0 (socket says 172.21.15.27:41992) 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 -- 172.21.15.27:0/88555526 learned_addr learned my addr 172.21.15.27:0/88555526 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73e00640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c74109e00 0x7f4c741a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c6be00640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c741094b0 0x7f4c7419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 -- 172.21.15.27:0/88555526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c74109e00 msgr2=0x7f4c741a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c74109e00 0x7f4c741a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 -- 172.21.15.27:0/88555526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c741094b0 msgr2=0x7f4c7419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c741094b0 0x7f4c7419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73400640 1 -- 172.21.15.27:0/88555526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c741a26c0 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.239+0000 7f4c73e00640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c74109e00 0x7f4c741a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.240+0000 7f4c73400640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c74103b70 0x7f4c7419c750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400ea60 tx=0x7f4c6400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.240+0000 7f4c70c00640 1 -- 172.21.15.27:0/88555526 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400cde0 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.240+0000 7f4c70c00640 1 -- 172.21.15.27:0/88555526 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c640035f0 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.240+0000 7f4c70c00640 1 -- 172.21.15.27:0/88555526 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64010690 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.240+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c741a29b0 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.240+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c741a9110 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.241+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c74103f80 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.243+0000 7f4c70c00640 1 -- 172.21.15.27:0/88555526 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f4c640037a0 con 0x7f4c74103b70 2024-09-21T07:03:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.245+0000 7f4c70c00640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4c580b6550 0x7f4c580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.245+0000 7f4c70c00640 1 -- 172.21.15.27:0/88555526 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4c640e5a20 con 0x7f4c74103b70 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.245+0000 7f4c6be00640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4c580b6550 0x7f4c580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.246+0000 7f4c6be00640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4c580b6550 0x7f4c580b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009230 tx=0x7f4c5c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.249+0000 7f4c70c00640 1 -- 172.21.15.27:0/88555526 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c640ade10 con 0x7f4c74103b70 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.458+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f4c741a94e0 con 0x7f4c74103b70 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.458+0000 7f4c70c00640 1 -- 172.21.15.27:0/88555526 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v23)=0 v23) ==== 108+0+0 (secure 0 0 0) 0x7f4c640b21f0 con 0x7f4c74103b70 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4c580b6550 msgr2=0x7f4c580b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4c580b6550 0x7f4c580b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009230 tx=0x7f4c5c031400 comp rx=0 tx=0).stop 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c74103b70 msgr2=0x7f4c7419c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c74103b70 0x7f4c7419c750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400ea60 tx=0x7f4c6400ef20 comp rx=0 tx=0).stop 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 shutdown_connections 2024-09-21T07:03:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c74109e00 0x7f4c741a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4c580b6550 0x7f4c580b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c741094b0 0x7f4c7419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 --2- 172.21.15.27:0/88555526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c74103b70 0x7f4c7419c750 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 >> 172.21.15.27:0/88555526 conn(0x7f4c740ff5e0 msgr2=0x7f4c74106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 shutdown_connections 2024-09-21T07:03:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.461+0000 7f4c7a600640 1 -- 172.21.15.27:0/88555526 wait complete. 2024-09-21T07:03:13.714 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:13.715 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:13.717 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:13.720 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:14.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:14 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/338805617' entity='client.admin' 2024-09-21T07:03:14.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:14 smithi027 ceph-mon[30779]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:14.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:14 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1811058121' entity='client.admin' 2024-09-21T07:03:14.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:14 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/338805617' entity='client.admin' 2024-09-21T07:03:14.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:14 smithi153 ceph-mon[34747]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:14.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:14 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1811058121' entity='client.admin' 2024-09-21T07:03:14.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:14 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/338805617' entity='client.admin' 2024-09-21T07:03:14.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:14 smithi119 ceph-mon[34519]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:14.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:14 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1811058121' entity='client.admin' 2024-09-21T07:03:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 -- 172.21.15.27:0/4011679806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18075dd0 msgr2=0x7f4c180761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 --2- 172.21.15.27:0/4011679806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18075dd0 0x7f4c180761e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009900 tx=0x7f4c0402a410 comp rx=0 tx=0).stop 2024-09-21T07:03:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 -- 172.21.15.27:0/4011679806 shutdown_connections 2024-09-21T07:03:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 --2- 172.21.15.27:0/4011679806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18114280 0x7f4c18116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 --2- 172.21.15.27:0/4011679806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18075dd0 0x7f4c180761e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 --2- 172.21.15.27:0/4011679806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c18077380 0x7f4c180757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 -- 172.21.15.27:0/4011679806 >> 172.21.15.27:0/4011679806 conn(0x7f4c180ff440 msgr2=0x7f4c18101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 -- 172.21.15.27:0/4011679806 shutdown_connections 2024-09-21T07:03:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.291+0000 7f4c17200640 1 -- 172.21.15.27:0/4011679806 wait complete. 2024-09-21T07:03:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 Processor -- start 2024-09-21T07:03:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 -- start start 2024-09-21T07:03:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18075dd0 0x7f4c181a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c18077380 0x7f4c181a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18114280 0x7f4c181a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c18119f10 con 0x7f4c18114280 2024-09-21T07:03:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c18119d90 con 0x7f4c18077380 2024-09-21T07:03:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.293+0000 7f4c17200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1811a090 con 0x7f4c18075dd0 2024-09-21T07:03:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18075dd0 0x7f4c181a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18075dd0 0x7f4c181a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51252/0 (socket says 172.21.15.27:51252) 2024-09-21T07:03:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 -- 172.21.15.27:0/356623250 learned_addr learned my addr 172.21.15.27:0/356623250 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15400640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c18077380 0x7f4c181a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c16800640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18114280 0x7f4c181a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 -- 172.21.15.27:0/356623250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c18077380 msgr2=0x7f4c181a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c18077380 0x7f4c181a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 -- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18114280 msgr2=0x7f4c181a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18114280 0x7f4c181a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15e00640 1 -- 172.21.15.27:0/356623250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c181a6830 con 0x7f4c18075dd0 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.294+0000 7f4c15400640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c18077380 0x7f4c181a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.295+0000 7f4c15e00640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18075dd0 0x7f4c181a08c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e9f0 tx=0x7f4c0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.295+0000 7f4c0ea00640 1 -- 172.21.15.27:0/356623250 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0800cd80 con 0x7f4c18075dd0 2024-09-21T07:03:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.295+0000 7f4c0ea00640 1 -- 172.21.15.27:0/356623250 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c080108e0 con 0x7f4c18075dd0 2024-09-21T07:03:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.295+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c181a6b20 con 0x7f4c18075dd0 2024-09-21T07:03:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.295+0000 7f4c0ea00640 1 -- 172.21.15.27:0/356623250 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08018980 con 0x7f4c18075dd0 2024-09-21T07:03:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.295+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c181ad280 con 0x7f4c18075dd0 2024-09-21T07:03:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.297+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c180761e0 con 0x7f4c18075dd0 2024-09-21T07:03:15.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.300+0000 7f4c0ea00640 1 -- 172.21.15.27:0/356623250 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f4c08010430 con 0x7f4c18075dd0 2024-09-21T07:03:15.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.301+0000 7f4c0ea00640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bfc0b6550 0x7f4bfc0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.302+0000 7f4c0ea00640 1 -- 172.21.15.27:0/356623250 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4c080e5b00 con 0x7f4c18075dd0 2024-09-21T07:03:15.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.302+0000 7f4c15400640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bfc0b6550 0x7f4bfc0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.302+0000 7f4c15400640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bfc0b6550 0x7f4bfc0b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c04031440 tx=0x7f4c04005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:15.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.305+0000 7f4c0ea00640 1 -- 172.21.15.27:0/356623250 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c080b22d0 con 0x7f4c18075dd0 2024-09-21T07:03:15.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.609+0000 7f8d6a800640 1 -- 172.21.15.27:0/2456720953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d641094a0 msgr2=0x7f8d641098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.609+0000 7f8d6a800640 1 --2- 172.21.15.27:0/2456720953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d641094a0 0x7f8d641098b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009900 tx=0x7f8d5002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.609+0000 7f8d6a800640 1 -- 172.21.15.27:0/2456720953 shutdown_connections 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.609+0000 7f8d6a800640 1 --2- 172.21.15.27:0/2456720953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64109df0 0x7f8d64110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.609+0000 7f8d6a800640 1 --2- 172.21.15.27:0/2456720953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d641094a0 0x7f8d641098b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.609+0000 7f8d6a800640 1 --2- 172.21.15.27:0/2456720953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d64103b60 0x7f8d64103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.609+0000 7f8d6a800640 1 -- 172.21.15.27:0/2456720953 >> 172.21.15.27:0/2456720953 conn(0x7f8d6407b850 msgr2=0x7f8d6407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.610+0000 7f8d6a800640 1 -- 172.21.15.27:0/2456720953 shutdown_connections 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.610+0000 7f8d6a800640 1 -- 172.21.15.27:0/2456720953 wait complete. 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.611+0000 7f8d6a800640 1 Processor -- start 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.611+0000 7f8d6a800640 1 -- start start 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d6a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64103b60 0x7f8d6419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d6a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d641094a0 0x7f8d6419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d6a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d64109df0 0x7f8d641a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d6a800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64113b00 con 0x7f8d64109df0 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d6a800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64113980 con 0x7f8d641094a0 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d6a800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64113c80 con 0x7f8d64103b60 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d63400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64103b60 0x7f8d6419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d63400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64103b60 0x7f8d6419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51234/0 (socket says 172.21.15.27:51234) 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d63400640 1 -- 172.21.15.27:0/1505863099 learned_addr learned my addr 172.21.15.27:0/1505863099 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d63e00640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d64109df0 0x7f8d641a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.612+0000 7f8d62a00640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d641094a0 0x7f8d6419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d63400640 1 -- 172.21.15.27:0/1505863099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d641094a0 msgr2=0x7f8d6419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d63400640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d641094a0 0x7f8d6419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d63400640 1 -- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d64109df0 msgr2=0x7f8d641a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d63400640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d64109df0 0x7f8d641a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d63400640 1 -- 172.21.15.27:0/1505863099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d641a2720 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d63400640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64103b60 0x7f8d6419c7b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400e9f0 tx=0x7f8d5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d5be00640 1 -- 172.21.15.27:0/1505863099 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5400cd80 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d5be00640 1 -- 172.21.15.27:0/1505863099 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d540108e0 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d5be00640 1 -- 172.21.15.27:0/1505863099 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54018980 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.613+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d641a2a10 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.614+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d641a9170 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.615+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d64104de0 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.617+0000 7f8d5be00640 1 -- 172.21.15.27:0/1505863099 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f8d54010430 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.619+0000 7f8d5be00640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d480b64d0 0x7f8d480b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.619+0000 7f8d5be00640 1 -- 172.21.15.27:0/1505863099 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8d540e5fa0 con 0x7f8d64103b60 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.619+0000 7f8d62a00640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d480b64d0 0x7f8d480b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.620+0000 7f8d62a00640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d480b64d0 0x7f8d480b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d5002a9e0 tx=0x7f8d50005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.620+0000 7f8d5be00640 1 -- 172.21.15.27:0/1505863099 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d540b2770 con 0x7f8d64103b60 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.821+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f8d64103f70 con 0x7f8d64103b60 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.828+0000 7f8d5be00640 1 -- 172.21.15.27:0/1505863099 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v25)=0 v25) ==== 119+0+0 (secure 0 0 0) 0x7f8d540b76b0 con 0x7f8d64103b60 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d480b64d0 msgr2=0x7f8d480b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d480b64d0 0x7f8d480b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d5002a9e0 tx=0x7f8d50005a20 comp rx=0 tx=0).stop 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64103b60 msgr2=0x7f8d6419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64103b60 0x7f8d6419c7b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400e9f0 tx=0x7f8d5400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 shutdown_connections 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d480b64d0 0x7f8d480b89e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d64109df0 0x7f8d641a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d641094a0 0x7f8d6419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 --2- 172.21.15.27:0/1505863099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64103b60 0x7f8d6419c7b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 >> 172.21.15.27:0/1505863099 conn(0x7f8d6407b850 msgr2=0x7f8d641073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 shutdown_connections 2024-09-21T07:03:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:14.830+0000 7f8d6a800640 1 -- 172.21.15.27:0/1505863099 wait complete. 2024-09-21T07:03:15.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.510+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4c1802e640 con 0x7f4c18075dd0 2024-09-21T07:03:15.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.517+0000 7f4c0ea00640 1 -- 172.21.15.27:0/356623250 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v26)=0 v26) ==== 133+0+0 (secure 0 0 0) 0x7f4c080b7210 con 0x7f4c18075dd0 2024-09-21T07:03:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bfc0b6550 msgr2=0x7f4bfc0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bfc0b6550 0x7f4bfc0b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c04031440 tx=0x7f4c04005a20 comp rx=0 tx=0).stop 2024-09-21T07:03:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18075dd0 msgr2=0x7f4c181a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18075dd0 0x7f4c181a08c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e9f0 tx=0x7f4c0800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 shutdown_connections 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bfc0b6550 0x7f4bfc0b8a60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c18114280 0x7f4c181a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c18077380 0x7f4c181a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 --2- 172.21.15.27:0/356623250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c18075dd0 0x7f4c181a08c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.520+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 >> 172.21.15.27:0/356623250 conn(0x7f4c180ff440 msgr2=0x7f4c18110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.521+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 shutdown_connections 2024-09-21T07:03:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:15.521+0000 7f4c17200640 1 -- 172.21.15.27:0/356623250 wait complete. 2024-09-21T07:03:16.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:16.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:16.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:16.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:16.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:16.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:16.084 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:16.084 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:16.084 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:16.084 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:16.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:16.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi027') 2024-09-21T07:03:16.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi027') 2024-09-21T07:03:16.330 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi027') 2024-09-21T07:03:16.330 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi027') 2024-09-21T07:03:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 -- 172.21.15.27:0/527275118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80109e20 msgr2=0x7f7a80110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 --2- 172.21.15.27:0/527275118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80109e20 0x7f7a80110650 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009be0 tx=0x7f7a7802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 -- 172.21.15.27:0/527275118 shutdown_connections 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 --2- 172.21.15.27:0/527275118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80109e20 0x7f7a80110650 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 --2- 172.21.15.27:0/527275118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a801094d0 0x7f7a801098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 --2- 172.21.15.27:0/527275118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a80103b90 0x7f7a80103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 -- 172.21.15.27:0/527275118 >> 172.21.15.27:0/527275118 conn(0x7f7a800ff5e0 msgr2=0x7f7a80101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 -- 172.21.15.27:0/527275118 shutdown_connections 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.346+0000 7f7a86a00640 1 -- 172.21.15.27:0/527275118 wait complete. 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.347+0000 7f7a86a00640 1 Processor -- start 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.347+0000 7f7a86a00640 1 -- start start 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a86a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80103b90 0x7f7a8019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a86a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a801094d0 0x7f7a8019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a86a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a80109e20 0x7f7a8019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a86a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80113ae0 con 0x7f7a801094d0 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a86a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80113960 con 0x7f7a80109e20 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a86a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80113c60 con 0x7f7a80103b90 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a801094d0 0x7f7a8019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a801094d0 0x7f7a8019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42014/0 (socket says 172.21.15.27:42014) 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a77e00640 1 -- 172.21.15.27:0/1811058121 learned_addr learned my addr 172.21.15.27:0/1811058121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a84a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a80109e20 0x7f7a8019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.348+0000 7f7a7fe00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80103b90 0x7f7a8019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a77e00640 1 -- 172.21.15.27:0/1811058121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80103b90 msgr2=0x7f7a8019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a77e00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80103b90 0x7f7a8019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a77e00640 1 -- 172.21.15.27:0/1811058121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a80109e20 msgr2=0x7f7a8019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a77e00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a80109e20 0x7f7a8019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a77e00640 1 -- 172.21.15.27:0/1811058121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a801a03a0 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a84a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a80109e20 0x7f7a8019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a7fe00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80103b90 0x7f7a8019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a77e00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a801094d0 0x7f7a8019a9d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e9f0 tx=0x7f7a6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a7d600640 1 -- 172.21.15.27:0/1811058121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800ccc0 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a7d600640 1 -- 172.21.15.27:0/1811058121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a680108e0 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a801a0690 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a7d600640 1 -- 172.21.15.27:0/1811058121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68018880 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.349+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a801a6df0 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.351+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a80103fa0 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.355+0000 7f7a7d600640 1 -- 172.21.15.27:0/1811058121 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f7a68010430 con 0x7f7a801094d0 2024-09-21T07:03:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.357+0000 7f7a7d600640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a640b6550 0x7f7a640b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.357+0000 7f7a7d600640 1 -- 172.21.15.27:0/1811058121 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7a680e5aa0 con 0x7f7a801094d0 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.358+0000 7f7a7fe00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a640b6550 0x7f7a640b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.358+0000 7f7a7fe00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a640b6550 0x7f7a640b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a70005fd0 tx=0x7f7a70009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.362+0000 7f7a7d600640 1 -- 172.21.15.27:0/1811058121 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7a680ade90 con 0x7f7a801094d0 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.565+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f7a8002e640 con 0x7f7a801094d0 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.570+0000 7f7a7d600640 1 -- 172.21.15.27:0/1811058121 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f7a680b2270 con 0x7f7a801094d0 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.572+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a640b6550 msgr2=0x7f7a640b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.572+0000 7f7a86a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a640b6550 0x7f7a640b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a70005fd0 tx=0x7f7a70009260 comp rx=0 tx=0).stop 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.572+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a801094d0 msgr2=0x7f7a8019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.572+0000 7f7a86a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a801094d0 0x7f7a8019a9d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e9f0 tx=0x7f7a6800edf0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 shutdown_connections 2024-09-21T07:03:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a640b6550 0x7f7a640b8a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a80109e20 0x7f7a8019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a801094d0 0x7f7a8019a9d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 --2- 172.21.15.27:0/1811058121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a80103b90 0x7f7a8019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 >> 172.21.15.27:0/1811058121 conn(0x7f7a800ff5e0 msgr2=0x7f7a80101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 shutdown_connections 2024-09-21T07:03:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:13.573+0000 7f7a86a00640 1 -- 172.21.15.27:0/1811058121 wait complete. 2024-09-21T07:03:17.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:16 smithi027 ceph-mon[30779]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:16 smithi153 ceph-mon[34747]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:16 smithi119 ceph-mon[34519]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:19.001 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:18 smithi027 ceph-mon[30779]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:19.001 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:18 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3221627968' entity='client.admin' 2024-09-21T07:03:19.001 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:18 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/288593216' entity='client.admin' 2024-09-21T07:03:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:18 smithi153 ceph-mon[34747]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:18 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3221627968' entity='client.admin' 2024-09-21T07:03:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:18 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/288593216' entity='client.admin' 2024-09-21T07:03:19.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:18 smithi119 ceph-mon[34519]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:19.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:18 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3221627968' entity='client.admin' 2024-09-21T07:03:19.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:18 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/288593216' entity='client.admin' 2024-09-21T07:03:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 -- 172.21.15.27:0/3055763007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0109df0 msgr2=0x7f26b0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 --2- 172.21.15.27:0/3055763007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0109df0 0x7f26b0110620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009900 tx=0x7f26a802a410 comp rx=0 tx=0).stop 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 -- 172.21.15.27:0/3055763007 shutdown_connections 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 --2- 172.21.15.27:0/3055763007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0109df0 0x7f26b0110620 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 --2- 172.21.15.27:0/3055763007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b01094a0 0x7f26b01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 --2- 172.21.15.27:0/3055763007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103b60 0x7f26b0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 -- 172.21.15.27:0/3055763007 >> 172.21.15.27:0/3055763007 conn(0x7f26b007b850 msgr2=0x7f26b007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 -- 172.21.15.27:0/3055763007 shutdown_connections 2024-09-21T07:03:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.471+0000 7f26b7200640 1 -- 172.21.15.27:0/3055763007 wait complete. 2024-09-21T07:03:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.472+0000 7f26b7200640 1 Processor -- start 2024-09-21T07:03:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b7200640 1 -- start start 2024-09-21T07:03:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0103b60 0x7f26b019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b7200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b01094a0 0x7f26b019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b0109df0 0x7f26b01a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0113b00 con 0x7f26b0103b60 2024-09-21T07:03:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b7200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0113980 con 0x7f26b0109df0 2024-09-21T07:03:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b7200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0113c80 con 0x7f26b01094a0 2024-09-21T07:03:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26afe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0103b60 0x7f26b019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26afe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0103b60 0x7f26b019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54288/0 (socket says 172.21.15.27:54288) 2024-09-21T07:03:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26afe00640 1 -- 172.21.15.27:0/2933920992 learned_addr learned my addr 172.21.15.27:0/2933920992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26a7400640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b01094a0 0x7f26b019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.473+0000 7f26b5200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b0109df0 0x7f26b01a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26afe00640 1 -- 172.21.15.27:0/2933920992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b01094a0 msgr2=0x7f26b019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26afe00640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b01094a0 0x7f26b019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26afe00640 1 -- 172.21.15.27:0/2933920992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b0109df0 msgr2=0x7f26b01a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26afe00640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b0109df0 0x7f26b01a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26afe00640 1 -- 172.21.15.27:0/2933920992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b01a25c0 con 0x7f26b0103b60 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26a7400640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b01094a0 0x7f26b019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26b5200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b0109df0 0x7f26b01a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26afe00640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0103b60 0x7f26b019c650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26a000eb90 tx=0x7f26a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.474+0000 7f26ad600640 1 -- 172.21.15.27:0/2933920992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0004d20 con 0x7f26b0103b60 2024-09-21T07:03:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.475+0000 7f26ad600640 1 -- 172.21.15.27:0/2933920992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a000ee90 con 0x7f26b0103b60 2024-09-21T07:03:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.475+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b01a28b0 con 0x7f26b0103b60 2024-09-21T07:03:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.475+0000 7f26ad600640 1 -- 172.21.15.27:0/2933920992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0010670 con 0x7f26b0103b60 2024-09-21T07:03:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.475+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b01a9010 con 0x7f26b0103b60 2024-09-21T07:03:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.476+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b0103f70 con 0x7f26b0103b60 2024-09-21T07:03:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.478+0000 7f26ad600640 1 -- 172.21.15.27:0/2933920992 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f26a0005020 con 0x7f26b0103b60 2024-09-21T07:03:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.480+0000 7f26ad600640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26940b65d0 0x7f26940b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.480+0000 7f26ad600640 1 -- 172.21.15.27:0/2933920992 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f26a00e5da0 con 0x7f26b0103b60 2024-09-21T07:03:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.480+0000 7f26a7400640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26940b65d0 0x7f26940b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.481+0000 7f26a7400640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26940b65d0 0x7f26940b8ae0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26b0107120 tx=0x7f2698005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.484+0000 7f26ad600640 1 -- 172.21.15.27:0/2933920992 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26a00ae190 con 0x7f26b0103b60 2024-09-21T07:03:19.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.694+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f26b019d780 con 0x7f26b0103b60 2024-09-21T07:03:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.699+0000 7f26ad600640 1 -- 172.21.15.27:0/2933920992 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f26a00b2570 con 0x7f26b0103b60 2024-09-21T07:03:19.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.701+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26940b65d0 msgr2=0x7f26940b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.701+0000 7f26b7200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26940b65d0 0x7f26940b8ae0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26b0107120 tx=0x7f2698005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:19.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.701+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0103b60 msgr2=0x7f26b019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.701+0000 7f26b7200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0103b60 0x7f26b019c650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26a000eb90 tx=0x7f26a000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 shutdown_connections 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b01094a0 0x7f26b019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26940b65d0 0x7f26940b8ae0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b0109df0 0x7f26b01a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 --2- 172.21.15.27:0/2933920992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26b0103b60 0x7f26b019c650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 >> 172.21.15.27:0/2933920992 conn(0x7f26b007b850 msgr2=0x7f26b0105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 shutdown_connections 2024-09-21T07:03:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.702+0000 7f26b7200640 1 -- 172.21.15.27:0/2933920992 wait complete. 2024-09-21T07:03:19.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 -- 172.21.15.27:0/3475536026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18101092f0 msgr2=0x7f1810109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 --2- 172.21.15.27:0/3475536026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18101092f0 0x7f1810109700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009900 tx=0x7f17fc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 -- 172.21.15.27:0/3475536026 shutdown_connections 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 --2- 172.21.15.27:0/3475536026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1810109c40 0x7f1810110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 --2- 172.21.15.27:0/3475536026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18101092f0 0x7f1810109700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 --2- 172.21.15.27:0/3475536026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 0x7f1810103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 -- 172.21.15.27:0/3475536026 >> 172.21.15.27:0/3475536026 conn(0x7f18100ff440 msgr2=0x7f1810101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 -- 172.21.15.27:0/3475536026 shutdown_connections 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.899+0000 7f180f200640 1 -- 172.21.15.27:0/3475536026 wait complete. 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.900+0000 7f180f200640 1 Processor -- start 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.900+0000 7f180f200640 1 -- start start 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 0x7f181019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18101092f0 0x7f181019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810109c40 0x7f18101a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113900 con 0x7f1810109c40 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113780 con 0x7f18101039b0 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113a80 con 0x7f18101092f0 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810109c40 0x7f18101a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 0x7f181019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 0x7f181019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:43630/0 (socket says 172.21.15.27:43630) 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180de00640 1 -- 172.21.15.27:0/3221627968 learned_addr learned my addr 172.21.15.27:0/3221627968 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180d400640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18101092f0 0x7f181019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180e800640 1 -- 172.21.15.27:0/3221627968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18101092f0 msgr2=0x7f181019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.901+0000 7f180e800640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18101092f0 0x7f181019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f180e800640 1 -- 172.21.15.27:0/3221627968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 msgr2=0x7f181019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f180e800640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 0x7f181019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f180e800640 1 -- 172.21.15.27:0/3221627968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18101a24c0 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f180de00640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 0x7f181019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f180e800640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810109c40 0x7f18101a1d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1808009fd0 tx=0x7f180800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f1806a00640 1 -- 172.21.15.27:0/3221627968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180800ce20 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f1806a00640 1 -- 172.21.15.27:0/3221627968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1808005020 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f1806a00640 1 -- 172.21.15.27:0/3221627968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808010690 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18101a27b0 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.902+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18101a8f10 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.903+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1810103dc0 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.906+0000 7f1806a00640 1 -- 172.21.15.27:0/3221627968 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f18080054d0 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.908+0000 7f1806a00640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17f40b6550 0x7f17f40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.908+0000 7f1806a00640 1 -- 172.21.15.27:0/3221627968 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f18080e5d40 con 0x7f1810109c40 2024-09-21T07:03:19.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.908+0000 7f180de00640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17f40b6550 0x7f17f40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.909+0000 7f180de00640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17f40b6550 0x7f17f40b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1800005fd0 tx=0x7f180000b4a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:17.910+0000 7f1806a00640 1 -- 172.21.15.27:0/3221627968 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18080ae130 con 0x7f1810109c40 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.122+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f181019d680 con 0x7f1810109c40 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.127+0000 7f1806a00640 1 -- 172.21.15.27:0/3221627968 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7f181019d680 con 0x7f1810109c40 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.130+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17f40b6550 msgr2=0x7f17f40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.130+0000 7f180f200640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17f40b6550 0x7f17f40b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1800005fd0 tx=0x7f180000b4a0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.130+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810109c40 msgr2=0x7f18101a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.130+0000 7f180f200640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810109c40 0x7f18101a1d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1808009fd0 tx=0x7f180800ef10 comp rx=0 tx=0).stop 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 shutdown_connections 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18101092f0 0x7f181019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17f40b6550 0x7f17f40b8a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810109c40 0x7f18101a1d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 --2- 172.21.15.27:0/3221627968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18101039b0 0x7f181019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 >> 172.21.15.27:0/3221627968 conn(0x7f18100ff440 msgr2=0x7f18101089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 shutdown_connections 2024-09-21T07:03:19.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.131+0000 7f180f200640 1 -- 172.21.15.27:0/3221627968 wait complete. 2024-09-21T07:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/874006166' entity='client.admin' 2024-09-21T07:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2933920992' entity='client.admin' 2024-09-21T07:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:20.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/874006166' entity='client.admin' 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2933920992' entity='client.admin' 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:20.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:20.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:20.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/874006166' entity='client.admin' 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2933920992' entity='client.admin' 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:20.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 -- 172.21.15.27:0/1842869984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d41039b0 msgr2=0x7fd2d4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1842869984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d41039b0 0x7fd2d4103dc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009860 tx=0x7fd2c802a340 comp rx=0 tx=0).stop 2024-09-21T07:03:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 -- 172.21.15.27:0/1842869984 shutdown_connections 2024-09-21T07:03:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1842869984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d4109c40 0x7fd2d4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1842869984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d41092f0 0x7fd2d4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1842869984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d41039b0 0x7fd2d4103dc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 -- 172.21.15.27:0/1842869984 >> 172.21.15.27:0/1842869984 conn(0x7fd2d40ff440 msgr2=0x7fd2d4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 -- 172.21.15.27:0/1842869984 shutdown_connections 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.998+0000 7fd2dac00640 1 -- 172.21.15.27:0/1842869984 wait complete. 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 Processor -- start 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 -- start start 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d41039b0 0x7fd2d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d41092f0 0x7fd2d4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d4109c40 0x7fd2d4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4113900 con 0x7fd2d41092f0 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4113780 con 0x7fd2d41039b0 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.000+0000 7fd2dac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4113a80 con 0x7fd2d4109c40 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2d8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d41092f0 0x7fd2d4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d4109c40 0x7fd2d4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d4109c40 0x7fd2d4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52946/0 (socket says 172.21.15.27:52946) 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 -- 172.21.15.27:0/1625773460 learned_addr learned my addr 172.21.15.27:0/1625773460 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2d9800640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d41039b0 0x7fd2d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 -- 172.21.15.27:0/1625773460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d41039b0 msgr2=0x7fd2d4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d41039b0 0x7fd2d4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 -- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d41092f0 msgr2=0x7fd2d4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d41092f0 0x7fd2d4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2da200640 1 -- 172.21.15.27:0/1625773460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d41a85d0 con 0x7fd2d4109c40 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.001+0000 7fd2d9800640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d41039b0 0x7fd2d4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.002+0000 7fd2da200640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d4109c40 0x7fd2d4076190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00eb10 tx=0x7fd2cc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.002+0000 7fd2d2000640 1 -- 172.21.15.27:0/1625773460 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004b20 con 0x7fd2d4109c40 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.002+0000 7fd2d2000640 1 -- 172.21.15.27:0/1625773460 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2cc004cc0 con 0x7fd2d4109c40 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.002+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d41a88c0 con 0x7fd2d4109c40 2024-09-21T07:03:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.002+0000 7fd2d2000640 1 -- 172.21.15.27:0/1625773460 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc010670 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.002+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d41a8e50 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.003+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d4103dc0 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.006+0000 7fd2d2000640 1 -- 172.21.15.27:0/1625773460 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fd2cc005020 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.007+0000 7fd2d2000640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2bc0b6550 0x7fd2bc0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.008+0000 7fd2d2000640 1 -- 172.21.15.27:0/1625773460 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd2cc0e5db0 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.008+0000 7fd2d9800640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2bc0b6550 0x7fd2bc0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.008+0000 7fd2d9800640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2bc0b6550 0x7fd2bc0b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd2c802a930 tx=0x7fd2c80058b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.012+0000 7fd2d2000640 1 -- 172.21.15.27:0/1625773460 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2cc0ae1a0 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.224+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fd2d4076a10 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.233+0000 7fd2d2000640 1 -- 172.21.15.27:0/1625773460 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7fd2cc0b2580 con 0x7fd2d4109c40 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2bc0b6550 msgr2=0x7fd2bc0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2bc0b6550 0x7fd2bc0b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd2c802a930 tx=0x7fd2c80058b0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d4109c40 msgr2=0x7fd2d4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d4109c40 0x7fd2d4076190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00eb10 tx=0x7fd2cc00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 shutdown_connections 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2d4109c40 0x7fd2d4076190 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2bc0b6550 0x7fd2bc0b8a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d41092f0 0x7fd2d4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 --2- 172.21.15.27:0/1625773460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d41039b0 0x7fd2d4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 >> 172.21.15.27:0/1625773460 conn(0x7fd2d40ff440 msgr2=0x7fd2d4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 shutdown_connections 2024-09-21T07:03:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.237+0000 7fd2dac00640 1 -- 172.21.15.27:0/1625773460 wait complete. 2024-09-21T07:03:20.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 -- 172.21.15.27:0/2803674369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04076080 msgr2=0x7f5c0410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 --2- 172.21.15.27:0/2803674369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04076080 0x7f5c0410fc00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009900 tx=0x7f5bf002a410 comp rx=0 tx=0).stop 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 -- 172.21.15.27:0/2803674369 shutdown_connections 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 --2- 172.21.15.27:0/2803674369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c04110140 0x7f5c04112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 --2- 172.21.15.27:0/2803674369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04076080 0x7f5c0410fc00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 --2- 172.21.15.27:0/2803674369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c04075730 0x7f5c04075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 -- 172.21.15.27:0/2803674369 >> 172.21.15.27:0/2803674369 conn(0x7f5c0407b850 msgr2=0x7f5c0407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 -- 172.21.15.27:0/2803674369 shutdown_connections 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.091+0000 7f5c0c400640 1 -- 172.21.15.27:0/2803674369 wait complete. 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.092+0000 7f5c0c400640 1 Processor -- start 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.092+0000 7f5c0c400640 1 -- start start 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0c400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04075730 0x7f5c0419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0c400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c04076080 0x7f5c0419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0c400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c04110140 0x7f5c041a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0c400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04115c80 con 0x7f5c04075730 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0c400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04115b00 con 0x7f5c04110140 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0c400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04115e00 con 0x7f5c04076080 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04075730 0x7f5c0419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04075730 0x7f5c0419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54268/0 (socket says 172.21.15.27:54268) 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 -- 172.21.15.27:0/874006166 learned_addr learned my addr 172.21.15.27:0/874006166 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09000640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c04076080 0x7f5c0419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0a400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c04110140 0x7f5c041a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 -- 172.21.15.27:0/874006166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c04076080 msgr2=0x7f5c0419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c04076080 0x7f5c0419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 -- 172.21.15.27:0/874006166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c04110140 msgr2=0x7f5c041a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c04110140 0x7f5c041a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c09a00640 1 -- 172.21.15.27:0/874006166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c041a25e0 con 0x7f5c04075730 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.093+0000 7f5c0a400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c04110140 0x7f5c041a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.094+0000 7f5c09a00640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04075730 0x7f5c0419c670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800ea60 tx=0x7f5bf800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.094+0000 7f5c02000640 1 -- 172.21.15.27:0/874006166 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf800cde0 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.094+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c041a28d0 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.094+0000 7f5c02000640 1 -- 172.21.15.27:0/874006166 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf80035f0 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.094+0000 7f5c02000640 1 -- 172.21.15.27:0/874006166 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8010690 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.094+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c041a9030 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.095+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c04075b40 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.100+0000 7f5c02000640 1 -- 172.21.15.27:0/874006166 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f5bf80037a0 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.102+0000 7f5c02000640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bec0b64d0 0x7f5bec0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.102+0000 7f5c02000640 1 -- 172.21.15.27:0/874006166 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5bf80e5eb0 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.102+0000 7f5c02000640 1 -- 172.21.15.27:0/874006166 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5bf80e6390 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.102+0000 7f5c09000640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bec0b64d0 0x7f5bec0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.103+0000 7f5c09000640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bec0b64d0 0x7f5bec0b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009230 tx=0x7f5bf0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.334+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f5c0402e640 con 0x7f5c04075730 2024-09-21T07:03:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.339+0000 7f5c02000640 1 -- 172.21.15.27:0/874006166 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7f5bf80ae340 con 0x7f5c04075730 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bec0b64d0 msgr2=0x7f5bec0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bec0b64d0 0x7f5bec0b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009230 tx=0x7f5bf0031400 comp rx=0 tx=0).stop 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04075730 msgr2=0x7f5c0419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04075730 0x7f5c0419c670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800ea60 tx=0x7f5bf800ef20 comp rx=0 tx=0).stop 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 shutdown_connections 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c04076080 0x7f5c0419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bec0b64d0 0x7f5bec0b89e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c04110140 0x7f5c041a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 --2- 172.21.15.27:0/874006166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04075730 0x7f5c0419c670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 >> 172.21.15.27:0/874006166 conn(0x7f5c0407b850 msgr2=0x7f5c04101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 shutdown_connections 2024-09-21T07:03:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:19.342+0000 7f5c0c400640 1 -- 172.21.15.27:0/874006166 wait complete. 2024-09-21T07:03:21.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 -- 172.21.15.27:0/1585603974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d0105bb0 msgr2=0x7fc9d0105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/1585603974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d0105bb0 0x7fc9d0105fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009900 tx=0x7fc9c002a410 comp rx=0 tx=0).stop 2024-09-21T07:03:21.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 -- 172.21.15.27:0/1585603974 shutdown_connections 2024-09-21T07:03:21.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/1585603974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d010be40 0x7fc9d0112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/1585603974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d010b4f0 0x7fc9d010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/1585603974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d0105bb0 0x7fc9d0105fc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 -- 172.21.15.27:0/1585603974 >> 172.21.15.27:0/1585603974 conn(0x7fc9d0101680 msgr2=0x7fc9d0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 -- 172.21.15.27:0/1585603974 shutdown_connections 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.934+0000 7fc9cfe00640 1 -- 172.21.15.27:0/1585603974 wait complete. 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.935+0000 7fc9cfe00640 1 Processor -- start 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.935+0000 7fc9cfe00640 1 -- start start 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d0105bb0 0x7fc9d019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d010b4f0 0x7fc9d019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d010be40 0x7fc9d01a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cfe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0115b00 con 0x7fc9d010be40 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0115980 con 0x7fc9d0105bb0 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cfe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0115c80 con 0x7fc9d010b4f0 2024-09-21T07:03:21.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d010be40 0x7fc9d01a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d0105bb0 0x7fc9d019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d010be40 0x7fc9d01a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54302/0 (socket says 172.21.15.27:54302) 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cf400640 1 -- 172.21.15.27:0/770426589 learned_addr learned my addr 172.21.15.27:0/770426589 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d0105bb0 0x7fc9d019c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51654/0 (socket says 172.21.15.27:51654) 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.936+0000 7fc9ce000640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d010b4f0 0x7fc9d019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9cea00640 1 -- 172.21.15.27:0/770426589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d010b4f0 msgr2=0x7fc9d019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9cea00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d010b4f0 0x7fc9d019cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9cea00640 1 -- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d010be40 msgr2=0x7fc9d01a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9cea00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d010be40 0x7fc9d01a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9cea00640 1 -- 172.21.15.27:0/770426589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d01a24e0 con 0x7fc9d0105bb0 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9cea00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d0105bb0 0x7fc9d019c570 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006010 tx=0x7fc9c0004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9c7400640 1 -- 172.21.15.27:0/770426589 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c003f070 con 0x7fc9d0105bb0 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9c7400640 1 -- 172.21.15.27:0/770426589 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c0003ca0 con 0x7fc9d0105bb0 2024-09-21T07:03:21.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9c7400640 1 -- 172.21.15.27:0/770426589 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0035070 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.937+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d01a2770 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.938+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a8f30 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.938+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d0106e30 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.941+0000 7fc9c7400640 1 -- 172.21.15.27:0/770426589 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fc9c003b460 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.942+0000 7fc9c7400640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9b40b6550 0x7fc9b40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.943+0000 7fc9c7400640 1 -- 172.21.15.27:0/770426589 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc9c0105df0 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.943+0000 7fc9ce000640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9b40b6550 0x7fc9b40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.943+0000 7fc9ce000640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9b40b6550 0x7fc9b40b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9d010aff0 tx=0x7fc9bc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:20.945+0000 7fc9c7400640 1 -- 172.21.15.27:0/770426589 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9c00d25c0 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.150+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fc9d010dc40 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.157+0000 7fc9c7400640 1 -- 172.21.15.27:0/770426589 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v32)=0 v32) ==== 115+0+0 (secure 0 0 0) 0x7fc9c00d7500 con 0x7fc9d0105bb0 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9b40b6550 msgr2=0x7fc9b40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9b40b6550 0x7fc9b40b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9d010aff0 tx=0x7fc9bc005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:21.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d0105bb0 msgr2=0x7fc9d019c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d0105bb0 0x7fc9d019c570 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006010 tx=0x7fc9c0004060 comp rx=0 tx=0).stop 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 shutdown_connections 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d010b4f0 0x7fc9d019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9b40b6550 0x7fc9b40b8a60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d010be40 0x7fc9d01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 --2- 172.21.15.27:0/770426589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d0105bb0 0x7fc9d019c570 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 >> 172.21.15.27:0/770426589 conn(0x7fc9d0101680 msgr2=0x7fc9d0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 shutdown_connections 2024-09-21T07:03:21.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.160+0000 7fc9cfe00640 1 -- 172.21.15.27:0/770426589 wait complete. 2024-09-21T07:03:21.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 -- 172.21.15.27:0/2786732502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 msgr2=0x7feb6c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 --2- 172.21.15.27:0/2786732502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 0x7feb6c106190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feb60009860 tx=0x7feb6002a340 comp rx=0 tx=0).stop 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 -- 172.21.15.27:0/2786732502 shutdown_connections 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 --2- 172.21.15.27:0/2786732502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb6c10b6c0 0x7feb6c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 --2- 172.21.15.27:0/2786732502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 0x7feb6c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 --2- 172.21.15.27:0/2786732502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 0x7feb6c106190 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 -- 172.21.15.27:0/2786732502 >> 172.21.15.27:0/2786732502 conn(0x7feb6c101810 msgr2=0x7feb6c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 -- 172.21.15.27:0/2786732502 shutdown_connections 2024-09-21T07:03:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.846+0000 7feb74200640 1 -- 172.21.15.27:0/2786732502 wait complete. 2024-09-21T07:03:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.847+0000 7feb74200640 1 Processor -- start 2024-09-21T07:03:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb74200640 1 -- start start 2024-09-21T07:03:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb74200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 0x7feb6c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb74200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb6c10b6c0 0x7feb6c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb74200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 0x7feb6c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb74200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c115cd0 con 0x7feb6c10c010 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb74200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c115b50 con 0x7feb6c105d80 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb74200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c115e50 con 0x7feb6c10b6c0 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb72200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 0x7feb6c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb71800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 0x7feb6c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb72200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 0x7feb6c1a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54328/0 (socket says 172.21.15.27:54328) 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb72200640 1 -- 172.21.15.27:0/2511632377 learned_addr learned my addr 172.21.15.27:0/2511632377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.848+0000 7feb70e00640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb6c10b6c0 0x7feb6c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb72200640 1 -- 172.21.15.27:0/2511632377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb6c10b6c0 msgr2=0x7feb6c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb72200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb6c10b6c0 0x7feb6c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb72200640 1 -- 172.21.15.27:0/2511632377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 msgr2=0x7feb6c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb72200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 0x7feb6c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb72200640 1 -- 172.21.15.27:0/2511632377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c1a2630 con 0x7feb6c10c010 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb71800640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 0x7feb6c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb72200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 0x7feb6c1a1f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb6400efc0 tx=0x7feb6400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.849+0000 7feb6a000640 1 -- 172.21.15.27:0/2511632377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64003970 con 0x7feb6c10c010 2024-09-21T07:03:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.850+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a2920 con 0x7feb6c10c010 2024-09-21T07:03:21.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.850+0000 7feb6a000640 1 -- 172.21.15.27:0/2511632377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb64004020 con 0x7feb6c10c010 2024-09-21T07:03:21.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.850+0000 7feb6a000640 1 -- 172.21.15.27:0/2511632377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6400b940 con 0x7feb6c10c010 2024-09-21T07:03:21.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.850+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a9080 con 0x7feb6c10c010 2024-09-21T07:03:21.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.851+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c106190 con 0x7feb6c10c010 2024-09-21T07:03:21.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.853+0000 7feb6a000640 1 -- 172.21.15.27:0/2511632377 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7feb6401d020 con 0x7feb6c10c010 2024-09-21T07:03:21.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.855+0000 7feb6a000640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb540b6550 0x7feb540b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.855+0000 7feb6a000640 1 -- 172.21.15.27:0/2511632377 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7feb640e5c00 con 0x7feb6c10c010 2024-09-21T07:03:21.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.855+0000 7feb71800640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb540b6550 0x7feb540b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.856+0000 7feb71800640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb540b6550 0x7feb540b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feb60009830 tx=0x7feb60030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:21.857+0000 7feb6a000640 1 -- 172.21.15.27:0/2511632377 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feb640adff0 con 0x7feb6c10c010 2024-09-21T07:03:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.063+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7feb6c19d7f0 con 0x7feb6c10c010 2024-09-21T07:03:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.068+0000 7feb6a000640 1 -- 172.21.15.27:0/2511632377 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v33)=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7feb6c19d7f0 con 0x7feb6c10c010 2024-09-21T07:03:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb540b6550 msgr2=0x7feb540b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb540b6550 0x7feb540b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feb60009830 tx=0x7feb60030340 comp rx=0 tx=0).stop 2024-09-21T07:03:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 msgr2=0x7feb6c1a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 0x7feb6c1a1f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb6400efc0 tx=0x7feb6400c800 comp rx=0 tx=0).stop 2024-09-21T07:03:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 shutdown_connections 2024-09-21T07:03:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb6c10b6c0 0x7feb6c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb540b6550 0x7feb540b8a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb6c10c010 0x7feb6c1a1f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 --2- 172.21.15.27:0/2511632377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb6c105d80 0x7feb6c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 >> 172.21.15.27:0/2511632377 conn(0x7feb6c101810 msgr2=0x7feb6c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 shutdown_connections 2024-09-21T07:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.071+0000 7feb74200640 1 -- 172.21.15.27:0/2511632377 wait complete. 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.233+0000 7fcbc1a00640 1 -- 172.21.15.27:0/1258697548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 msgr2=0x7fcbbc116730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.233+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/1258697548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 0x7fcbbc116730 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009ca0 tx=0x7fcbb402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.234+0000 7fcbc1a00640 1 -- 172.21.15.27:0/1258697548 shutdown_connections 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.234+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/1258697548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 0x7fcbbc116730 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.234+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/1258697548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbbc075dd0 0x7fcbbc0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.234+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/1258697548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc077380 0x7fcbbc0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.234+0000 7fcbc1a00640 1 -- 172.21.15.27:0/1258697548 >> 172.21.15.27:0/1258697548 conn(0x7fcbbc0ff440 msgr2=0x7fcbbc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.234+0000 7fcbc1a00640 1 -- 172.21.15.27:0/1258697548 shutdown_connections 2024-09-21T07:03:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.234+0000 7fcbc1a00640 1 -- 172.21.15.27:0/1258697548 wait complete. 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 Processor -- start 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 -- start start 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc075dd0 0x7fcbbc1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbbc077380 0x7fcbbc1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 0x7fcbbc1a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc119ea0 con 0x7fcbbc114260 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc119d20 con 0x7fcbbc077380 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.235+0000 7fcbc1a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc11a020 con 0x7fcbbc075dd0 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc075dd0 0x7fcbbc1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc075dd0 0x7fcbbc1a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53034/0 (socket says 172.21.15.27:53034) 2024-09-21T07:03:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 -- 172.21.15.27:0/28778539 learned_addr learned my addr 172.21.15.27:0/28778539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbb400640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbbc077380 0x7fcbbc1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbc1000640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 0x7fcbbc1a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 -- 172.21.15.27:0/28778539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbbc077380 msgr2=0x7fcbbc1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbbc077380 0x7fcbbc1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 -- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 msgr2=0x7fcbbc1a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 0x7fcbbc1a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 -- 172.21.15.27:0/28778539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc1a67c0 con 0x7fcbbc075dd0 2024-09-21T07:03:22.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.236+0000 7fcbbbe00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc075dd0 0x7fcbbc1a0850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00e9f0 tx=0x7fcbac00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:22.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.237+0000 7fcbb8c00640 1 -- 172.21.15.27:0/28778539 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00cd80 con 0x7fcbbc075dd0 2024-09-21T07:03:22.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.237+0000 7fcbb8c00640 1 -- 172.21.15.27:0/28778539 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbac0054d0 con 0x7fcbbc075dd0 2024-09-21T07:03:22.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.237+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc1a6ab0 con 0x7fcbbc075dd0 2024-09-21T07:03:22.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.237+0000 7fcbb8c00640 1 -- 172.21.15.27:0/28778539 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010690 con 0x7fcbbc075dd0 2024-09-21T07:03:22.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.237+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc1ad210 con 0x7fcbbc075dd0 2024-09-21T07:03:22.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.238+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc0761e0 con 0x7fcbbc075dd0 2024-09-21T07:03:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.240+0000 7fcbb8c00640 1 -- 172.21.15.27:0/28778539 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fcbac004e90 con 0x7fcbbc075dd0 2024-09-21T07:03:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.242+0000 7fcbb8c00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcba00b64d0 0x7fcba00b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.242+0000 7fcbb8c00640 1 -- 172.21.15.27:0/28778539 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcbac0e5b90 con 0x7fcbbc075dd0 2024-09-21T07:03:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.242+0000 7fcbbb400640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcba00b64d0 0x7fcba00b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.243+0000 7fcbbb400640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcba00b64d0 0x7fcba00b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc116110 tx=0x7fcba800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:22.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.246+0000 7fcbb8c00640 1 -- 172.21.15.27:0/28778539 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcbac0ae000 con 0x7fcbbc075dd0 2024-09-21T07:03:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:22 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:22 smithi027 ceph-mon[30779]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:22 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2511632377' entity='client.admin' 2024-09-21T07:03:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:22 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:22 smithi153 ceph-mon[34747]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:22.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:22 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2511632377' entity='client.admin' 2024-09-21T07:03:22.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.445+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fcbbc02e640 con 0x7fcbbc075dd0 2024-09-21T07:03:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.454+0000 7fcbb8c00640 1 -- 172.21.15.27:0/28778539 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v34)=0 v34) ==== 119+0+0 (secure 0 0 0) 0x7fcbac0b23e0 con 0x7fcbbc075dd0 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcba00b64d0 msgr2=0x7fcba00b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcba00b64d0 0x7fcba00b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc116110 tx=0x7fcba800b410 comp rx=0 tx=0).stop 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc075dd0 msgr2=0x7fcbbc1a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc075dd0 0x7fcbbc1a0850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00e9f0 tx=0x7fcbac00edf0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 shutdown_connections 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcba00b64d0 0x7fcba00b89e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc114260 0x7fcbbc1a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbbc077380 0x7fcbbc1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 --2- 172.21.15.27:0/28778539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbbc075dd0 0x7fcbbc1a0850 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.456+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 >> 172.21.15.27:0/28778539 conn(0x7fcbbc0ff440 msgr2=0x7fcbbc110160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.457+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 shutdown_connections 2024-09-21T07:03:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:22.457+0000 7fcbc1a00640 1 -- 172.21.15.27:0/28778539 wait complete. 2024-09-21T07:03:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:22 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:22 smithi119 ceph-mon[34519]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:22.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:22 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2511632377' entity='client.admin' 2024-09-21T07:03:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 -- 172.21.15.27:0/2806053809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 msgr2=0x7f09c4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 --2- 172.21.15.27:0/2806053809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 0x7f09c4110470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009be0 tx=0x7f09bc02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 -- 172.21.15.27:0/2806053809 shutdown_connections 2024-09-21T07:03:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 --2- 172.21.15.27:0/2806053809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 0x7f09c4110470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 --2- 172.21.15.27:0/2806053809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c41092f0 0x7f09c4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 --2- 172.21.15.27:0/2806053809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 0x7f09c4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 -- 172.21.15.27:0/2806053809 >> 172.21.15.27:0/2806053809 conn(0x7f09c40ff440 msgr2=0x7f09c4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 -- 172.21.15.27:0/2806053809 shutdown_connections 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.402+0000 7f09c9600640 1 -- 172.21.15.27:0/2806053809 wait complete. 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.403+0000 7f09c9600640 1 Processor -- start 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c9600640 1 -- start start 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 0x7f09c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c41092f0 0x7f09c4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 0x7f09c4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c9600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4113900 con 0x7f09c41039b0 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c9600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4113780 con 0x7f09c41092f0 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c9600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4113a80 con 0x7f09c4109c40 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 0x7f09c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 0x7f09c4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54220/0 (socket says 172.21.15.27:54220) 2024-09-21T07:03:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c3e00640 1 -- 172.21.15.27:0/288593216 learned_addr learned my addr 172.21.15.27:0/288593216 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.404+0000 7f09c3400640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c41092f0 0x7f09c4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c8c00640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 0x7f09c4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c3e00640 1 -- 172.21.15.27:0/288593216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 msgr2=0x7f09c4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c3e00640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 0x7f09c4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c3e00640 1 -- 172.21.15.27:0/288593216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c41092f0 msgr2=0x7f09c4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c3e00640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c41092f0 0x7f09c4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c3e00640 1 -- 172.21.15.27:0/288593216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c41a8570 con 0x7f09c41039b0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c8c00640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 0x7f09c4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c3e00640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 0x7f09c4075710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09b400e9c0 tx=0x7f09b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.405+0000 7f09c0c00640 1 -- 172.21.15.27:0/288593216 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400cde0 con 0x7f09c41039b0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.406+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c41a8860 con 0x7f09c41039b0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.406+0000 7f09c0c00640 1 -- 172.21.15.27:0/288593216 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b40035f0 con 0x7f09c41039b0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.406+0000 7f09c0c00640 1 -- 172.21.15.27:0/288593216 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4010690 con 0x7f09c41039b0 2024-09-21T07:03:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.406+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c41a8da0 con 0x7f09c41039b0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.407+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c4103dc0 con 0x7f09c41039b0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.409+0000 7f09c0c00640 1 -- 172.21.15.27:0/288593216 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f09b4004000 con 0x7f09c41039b0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.411+0000 7f09c0c00640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09a80b64d0 0x7f09a80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.411+0000 7f09c0c00640 1 -- 172.21.15.27:0/288593216 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f09b40e5bd0 con 0x7f09c41039b0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.411+0000 7f09c3400640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09a80b64d0 0x7f09a80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.412+0000 7f09c3400640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09a80b64d0 0x7f09a80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f09c4105c10 tx=0x7f09b0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.413+0000 7f09c0c00640 1 -- 172.21.15.27:0/288593216 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09b40ae0f0 con 0x7f09c41039b0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.620+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f09c4076a10 con 0x7f09c41039b0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.626+0000 7f09c0c00640 1 -- 172.21.15.27:0/288593216 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f09c4076a10 con 0x7f09c41039b0 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09a80b64d0 msgr2=0x7f09a80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09a80b64d0 0x7f09a80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f09c4105c10 tx=0x7f09b0005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 msgr2=0x7f09c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 0x7f09c4075710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09b400e9c0 tx=0x7f09b400ee80 comp rx=0 tx=0).stop 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 shutdown_connections 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09c4109c40 0x7f09c4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09a80b64d0 0x7f09a80b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09c41092f0 0x7f09c4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 --2- 172.21.15.27:0/288593216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09c41039b0 0x7f09c4075710 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 >> 172.21.15.27:0/288593216 conn(0x7f09c40ff440 msgr2=0x7f09c4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.629+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 shutdown_connections 2024-09-21T07:03:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:18.630+0000 7f09c9600640 1 -- 172.21.15.27:0/288593216 wait complete. 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.550+0000 7fb024400640 1 -- 172.21.15.27:0/1573639093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10b6c0 msgr2=0x7fb01c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.550+0000 7fb024400640 1 --2- 172.21.15.27:0/1573639093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10b6c0 0x7fb01c10bad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb008009900 tx=0x7fb00802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.550+0000 7fb024400640 1 -- 172.21.15.27:0/1573639093 shutdown_connections 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.550+0000 7fb024400640 1 --2- 172.21.15.27:0/1573639093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c10c010 0x7fb01c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.550+0000 7fb024400640 1 --2- 172.21.15.27:0/1573639093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10b6c0 0x7fb01c10bad0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.550+0000 7fb024400640 1 --2- 172.21.15.27:0/1573639093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 0x7fb01c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.550+0000 7fb024400640 1 -- 172.21.15.27:0/1573639093 >> 172.21.15.27:0/1573639093 conn(0x7fb01c101810 msgr2=0x7fb01c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.551+0000 7fb024400640 1 -- 172.21.15.27:0/1573639093 shutdown_connections 2024-09-21T07:03:23.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:23 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.551+0000 7fb024400640 1 -- 172.21.15.27:0/1573639093 wait complete. 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 Processor -- start 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 -- start start 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 0x7fb01c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c10b6c0 0x7fb01c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10c010 0x7fb01c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c115cd0 con 0x7fb01c10b6c0 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c115b50 con 0x7fb01c10c010 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.552+0000 7fb024400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c115e50 con 0x7fb01c105d80 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 0x7fb01c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c10b6c0 0x7fb01c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 0x7fb01c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53064/0 (socket says 172.21.15.27:53064) 2024-09-21T07:03:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 -- 172.21.15.27:0/579590977 learned_addr learned my addr 172.21.15.27:0/579590977 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:23.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb022400640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10c010 0x7fb01c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 -- 172.21.15.27:0/579590977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10c010 msgr2=0x7fb01c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10c010 0x7fb01c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 -- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c10b6c0 msgr2=0x7fb01c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c10b6c0 0x7fb01c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021a00640 1 -- 172.21.15.27:0/579590977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c1a2630 con 0x7fb01c105d80 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.553+0000 7fb021000640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c10b6c0 0x7fb01c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.554+0000 7fb021a00640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 0x7fb01c19c6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb01000d770 tx=0x7fb01000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.554+0000 7fb01a000640 1 -- 172.21.15.27:0/579590977 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010017070 con 0x7fb01c105d80 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.554+0000 7fb01a000640 1 -- 172.21.15.27:0/579590977 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0100054d0 con 0x7fb01c105d80 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.554+0000 7fb01a000640 1 -- 172.21.15.27:0/579590977 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010004c30 con 0x7fb01c105d80 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.554+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c1a2920 con 0x7fb01c105d80 2024-09-21T07:03:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.554+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a9080 con 0x7fb01c105d80 2024-09-21T07:03:23.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.555+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c106190 con 0x7fb01c105d80 2024-09-21T07:03:23.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.557+0000 7fb01a000640 1 -- 172.21.15.27:0/579590977 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fb010004e10 con 0x7fb01c105d80 2024-09-21T07:03:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.559+0000 7fb01a000640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0040b65d0 0x7fb0040b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.559+0000 7fb01a000640 1 -- 172.21.15.27:0/579590977 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb0100e51a0 con 0x7fb01c105d80 2024-09-21T07:03:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.559+0000 7fb021000640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0040b65d0 0x7fb0040b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.560+0000 7fb021000640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0040b65d0 0x7fb0040b8ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb008009230 tx=0x7fb008031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:23.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.563+0000 7fb01a000640 1 -- 172.21.15.27:0/579590977 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0100b1970 con 0x7fb01c105d80 2024-09-21T07:03:23.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.767+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fb01c10dde0 con 0x7fb01c105d80 2024-09-21T07:03:23.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.778+0000 7fb01a000640 1 -- 172.21.15.27:0/579590977 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v35)=0 v35) ==== 125+0+0 (secure 0 0 0) 0x7fb0100b68b0 con 0x7fb01c105d80 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0040b65d0 msgr2=0x7fb0040b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0040b65d0 0x7fb0040b8ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb008009230 tx=0x7fb008031400 comp rx=0 tx=0).stop 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 msgr2=0x7fb01c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 0x7fb01c19c6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb01000d770 tx=0x7fb01000dc30 comp rx=0 tx=0).stop 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 shutdown_connections 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0040b65d0 0x7fb0040b8ae0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb01c10c010 0x7fb01c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c10b6c0 0x7fb01c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 --2- 172.21.15.27:0/579590977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c105d80 0x7fb01c19c6c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.781+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 >> 172.21.15.27:0/579590977 conn(0x7fb01c101810 msgr2=0x7fb01c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:23.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.782+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 shutdown_connections 2024-09-21T07:03:23.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:23.782+0000 7fb024400640 1 -- 172.21.15.27:0/579590977 wait complete. 2024-09-21T07:03:23.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:23 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:23.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:23 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 -- 172.21.15.27:0/322561024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c105d80 msgr2=0x7f6e6c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 --2- 172.21.15.27:0/322561024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c105d80 0x7f6e6c106190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c02a410 comp rx=0 tx=0).stop 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 -- 172.21.15.27:0/322561024 shutdown_connections 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 --2- 172.21.15.27:0/322561024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e6c10b6c0 0x7f6e6c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 --2- 172.21.15.27:0/322561024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e6c10c010 0x7f6e6c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 --2- 172.21.15.27:0/322561024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c105d80 0x7f6e6c106190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 -- 172.21.15.27:0/322561024 >> 172.21.15.27:0/322561024 conn(0x7f6e6c101810 msgr2=0x7f6e6c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.520+0000 7f6e70a00640 1 -- 172.21.15.27:0/322561024 shutdown_connections 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.521+0000 7f6e70a00640 1 -- 172.21.15.27:0/322561024 wait complete. 2024-09-21T07:03:24.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 Processor -- start 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 -- start start 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e6c105d80 0x7f6e6c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e6c10b6c0 0x7f6e6c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c10c010 0x7f6e6c1a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c115d20 con 0x7f6e6c10c010 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c115ba0 con 0x7f6e6c105d80 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e70a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c115ea0 con 0x7f6e6c10b6c0 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.522+0000 7f6e6a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c10c010 0x7f6e6c1a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c10c010 0x7f6e6c1a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54374/0 (socket says 172.21.15.27:54374) 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 -- 172.21.15.27:0/3128275916 learned_addr learned my addr 172.21.15.27:0/3128275916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e69e00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e6c105d80 0x7f6e6c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e69400640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e6c10b6c0 0x7f6e6c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 -- 172.21.15.27:0/3128275916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e6c10b6c0 msgr2=0x7f6e6c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e6c10b6c0 0x7f6e6c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 -- 172.21.15.27:0/3128275916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e6c105d80 msgr2=0x7f6e6c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e6c105d80 0x7f6e6c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 -- 172.21.15.27:0/3128275916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c1a26a0 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.523+0000 7f6e6a800640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c10c010 0x7f6e6c1a1f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009fd0 tx=0x7f6e6400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.524+0000 7f6e62a00640 1 -- 172.21.15.27:0/3128275916 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64017070 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.524+0000 7f6e62a00640 1 -- 172.21.15.27:0/3128275916 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e64003e20 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.524+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c1a2990 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.524+0000 7f6e62a00640 1 -- 172.21.15.27:0/3128275916 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64010460 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.524+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c1a90f0 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.525+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c106190 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.527+0000 7f6e62a00640 1 -- 172.21.15.27:0/3128275916 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f6e64003970 con 0x7f6e6c10c010 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.529+0000 7f6e62a00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e500b64d0 0x7f6e500b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.529+0000 7f6e62a00640 1 -- 172.21.15.27:0/3128275916 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6e640e54b0 con 0x7f6e6c10c010 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.529+0000 7f6e69e00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e500b64d0 0x7f6e500b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.529+0000 7f6e69e00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e500b64d0 0x7f6e500b89e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005ab0 tx=0x7f6e5c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.530+0000 7f6e62a00640 1 -- 172.21.15.27:0/3128275916 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e640b1c80 con 0x7f6e6c10c010 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.729+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f6e6c19d860 con 0x7f6e6c10c010 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.734+0000 7f6e62a00640 1 -- 172.21.15.27:0/3128275916 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v36)=0 v36) ==== 116+0+0 (secure 0 0 0) 0x7f6e6c19d860 con 0x7f6e6c10c010 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e500b64d0 msgr2=0x7f6e500b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e500b64d0 0x7f6e500b89e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005ab0 tx=0x7f6e5c005a20 comp rx=0 tx=0).stop 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c10c010 msgr2=0x7f6e6c1a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c10c010 0x7f6e6c1a1f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009fd0 tx=0x7f6e6400b260 comp rx=0 tx=0).stop 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 shutdown_connections 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e6c10b6c0 0x7f6e6c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e500b64d0 0x7f6e500b89e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e6c10c010 0x7f6e6c1a1f70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 --2- 172.21.15.27:0/3128275916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e6c105d80 0x7f6e6c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 >> 172.21.15.27:0/3128275916 conn(0x7f6e6c101810 msgr2=0x7f6e6c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 shutdown_connections 2024-09-21T07:03:24.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:24.737+0000 7f6e70a00640 1 -- 172.21.15.27:0/3128275916 wait complete. 2024-09-21T07:03:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:24 smithi119 ceph-mon[34519]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:24 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:24 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3128275916' entity='client.admin' 2024-09-21T07:03:25.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:24 smithi027 ceph-mon[30779]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:25.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:24 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:25.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:25.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:25.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:25.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:25.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:24 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3128275916' entity='client.admin' 2024-09-21T07:03:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:24 smithi153 ceph-mon[34747]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:24 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:25.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:25.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:25.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:25.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:24 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3128275916' entity='client.admin' 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 -- 172.21.15.27:0/81796433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c109df0 msgr2=0x7fa17c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 --2- 172.21.15.27:0/81796433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c109df0 0x7fa17c110620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa174009ca0 tx=0x7fa17402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 -- 172.21.15.27:0/81796433 shutdown_connections 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 --2- 172.21.15.27:0/81796433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c109df0 0x7fa17c110620 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 --2- 172.21.15.27:0/81796433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c1094a0 0x7fa17c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 --2- 172.21.15.27:0/81796433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa17c103b60 0x7fa17c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 -- 172.21.15.27:0/81796433 >> 172.21.15.27:0/81796433 conn(0x7fa17c07b850 msgr2=0x7fa17c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 -- 172.21.15.27:0/81796433 shutdown_connections 2024-09-21T07:03:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.542+0000 7fa180a00640 1 -- 172.21.15.27:0/81796433 wait complete. 2024-09-21T07:03:26.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.543+0000 7fa180a00640 1 Processor -- start 2024-09-21T07:03:26.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa180a00640 1 -- start start 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa180a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c103b60 0x7fa17c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa180a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c1094a0 0x7fa17c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa180a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa17c109df0 0x7fa17c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa180a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c113b00 con 0x7fa17c1094a0 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa180a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c113980 con 0x7fa17c103b60 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa180a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c113c80 con 0x7fa17c109df0 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa179e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c103b60 0x7fa17c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa173e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c1094a0 0x7fa17c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa179e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c103b60 0x7fa17c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51734/0 (socket says 172.21.15.27:51734) 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa179e00640 1 -- 172.21.15.27:0/667698956 learned_addr learned my addr 172.21.15.27:0/667698956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.544+0000 7fa17a800640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa17c109df0 0x7fa17c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:26.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.545+0000 7fa179e00640 1 -- 172.21.15.27:0/667698956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa17c109df0 msgr2=0x7fa17c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:26.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.545+0000 7fa179e00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa17c109df0 0x7fa17c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.545+0000 7fa179e00640 1 -- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c1094a0 msgr2=0x7fa17c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:26.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.545+0000 7fa179e00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c1094a0 0x7fa17c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.545+0000 7fa179e00640 1 -- 172.21.15.27:0/667698956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c1a86f0 con 0x7fa17c103b60 2024-09-21T07:03:26.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.545+0000 7fa179e00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c103b60 0x7fa17c075710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00e9f0 tx=0x7fa16c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:26.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.545+0000 7fa172a00640 1 -- 172.21.15.27:0/667698956 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00cd80 con 0x7fa17c103b60 2024-09-21T07:03:26.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.546+0000 7fa172a00640 1 -- 172.21.15.27:0/667698956 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa16c0108e0 con 0x7fa17c103b60 2024-09-21T07:03:26.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.546+0000 7fa172a00640 1 -- 172.21.15.27:0/667698956 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c018980 con 0x7fa17c103b60 2024-09-21T07:03:26.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.546+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1a89e0 con 0x7fa17c103b60 2024-09-21T07:03:26.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.546+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c1a8ed0 con 0x7fa17c103b60 2024-09-21T07:03:26.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.547+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c103f70 con 0x7fa17c103b60 2024-09-21T07:03:26.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.549+0000 7fa172a00640 1 -- 172.21.15.27:0/667698956 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fa16c010430 con 0x7fa17c103b60 2024-09-21T07:03:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.551+0000 7fa172a00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1600b64d0 0x7fa1600b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.551+0000 7fa172a00640 1 -- 172.21.15.27:0/667698956 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa16c0e5f80 con 0x7fa17c103b60 2024-09-21T07:03:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.551+0000 7fa173e00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1600b64d0 0x7fa1600b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.552+0000 7fa173e00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1600b64d0 0x7fa1600b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa17c109060 tx=0x7fa164005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.553+0000 7fa172a00640 1 -- 172.21.15.27:0/667698956 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa16c0b2750 con 0x7fa17c103b60 2024-09-21T07:03:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.763+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fa17c076a10 con 0x7fa17c103b60 2024-09-21T07:03:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.770+0000 7fa172a00640 1 -- 172.21.15.27:0/667698956 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v37)=0 v37) ==== 115+0+0 (secure 0 0 0) 0x7fa17c076a10 con 0x7fa17c103b60 2024-09-21T07:03:26.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.772+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1600b64d0 msgr2=0x7fa1600b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.772+0000 7fa180a00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1600b64d0 0x7fa1600b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa17c109060 tx=0x7fa164005f90 comp rx=0 tx=0).stop 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.772+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c103b60 msgr2=0x7fa17c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.772+0000 7fa180a00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c103b60 0x7fa17c075710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00e9f0 tx=0x7fa16c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.772+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 shutdown_connections 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.773+0000 7fa180a00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa17c109df0 0x7fa17c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.773+0000 7fa180a00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1600b64d0 0x7fa1600b89e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.773+0000 7fa180a00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa17c1094a0 0x7fa17c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.773+0000 7fa180a00640 1 --2- 172.21.15.27:0/667698956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa17c103b60 0x7fa17c075710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.773+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 >> 172.21.15.27:0/667698956 conn(0x7fa17c07b850 msgr2=0x7fa17c107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.773+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 shutdown_connections 2024-09-21T07:03:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:26.773+0000 7fa180a00640 1 -- 172.21.15.27:0/667698956 wait complete. 2024-09-21T07:03:26.797 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:26 smithi027 ceph-mon[30779]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:26.797 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:26 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' 2024-09-21T07:03:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:26 smithi119 ceph-mon[34519]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:26 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' 2024-09-21T07:03:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:26 smithi153 ceph-mon[34747]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:26 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' 2024-09-21T07:03:28.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:28.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:28.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:28.009 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:28.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:28.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:28.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:28.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:28.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:28.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:28.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:28.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:29.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:28 smithi027 ceph-mon[30779]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:29.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.043 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.043 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.043 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.043 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.078 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:28 smithi153 ceph-mon[34747]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:28 smithi119 ceph-mon[34519]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:29.472 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.472 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.472 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.472 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.472 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.472 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.472 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.473 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.473 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0ff1e276-77e7-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T07:03:29.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T07:03:29.847 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:30.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 -- 172.21.15.27:0/3334202450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 msgr2=0x7f4f501098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:30.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 --2- 172.21.15.27:0/3334202450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 0x7f4f501098b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009900 tx=0x7f4f3c02a410 comp rx=0 tx=0).stop 2024-09-21T07:03:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 -- 172.21.15.27:0/3334202450 shutdown_connections 2024-09-21T07:03:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 --2- 172.21.15.27:0/3334202450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50109df0 0x7f4f50110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 --2- 172.21.15.27:0/3334202450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 0x7f4f501098b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 --2- 172.21.15.27:0/3334202450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f50103b60 0x7f4f50103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 -- 172.21.15.27:0/3334202450 >> 172.21.15.27:0/3334202450 conn(0x7f4f5007b850 msgr2=0x7f4f5007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 -- 172.21.15.27:0/3334202450 shutdown_connections 2024-09-21T07:03:30.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.408+0000 7f4f58600640 1 -- 172.21.15.27:0/3334202450 wait complete. 2024-09-21T07:03:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.409+0000 7f4f58600640 1 Processor -- start 2024-09-21T07:03:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.409+0000 7f4f58600640 1 -- start start 2024-09-21T07:03:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f58600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50103b60 0x7f4f50075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f58600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 0x7f4f50075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f58600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f50109df0 0x7f4f50076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f58600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50113ab0 con 0x7f4f501094a0 2024-09-21T07:03:30.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f58600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50113930 con 0x7f4f50109df0 2024-09-21T07:03:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f58600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50113c30 con 0x7f4f50103b60 2024-09-21T07:03:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f55c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50103b60 0x7f4f50075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f55200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 0x7f4f50075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f55c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50103b60 0x7f4f50075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59980/0 (socket says 172.21.15.27:59980) 2024-09-21T07:03:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f55c00640 1 -- 172.21.15.27:0/3222332544 learned_addr learned my addr 172.21.15.27:0/3222332544 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.410+0000 7f4f56600640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f50109df0 0x7f4f50076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f55c00640 1 -- 172.21.15.27:0/3222332544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f50109df0 msgr2=0x7f4f50076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f55c00640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f50109df0 0x7f4f50076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f55c00640 1 -- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 msgr2=0x7f4f50075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f55c00640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 0x7f4f50075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f55c00640 1 -- 172.21.15.27:0/3222332544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f501a8740 con 0x7f4f50103b60 2024-09-21T07:03:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f55200640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 0x7f4f50075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f55c00640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50103b60 0x7f4f50075710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d950 tx=0x7f4f4400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f4e000640 1 -- 172.21.15.27:0/3222332544 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44017030 con 0x7f4f50103b60 2024-09-21T07:03:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f4e000640 1 -- 172.21.15.27:0/3222332544 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f44004020 con 0x7f4f50103b60 2024-09-21T07:03:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f501a8a30 con 0x7f4f50103b60 2024-09-21T07:03:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f4e000640 1 -- 172.21.15.27:0/3222332544 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f440047a0 con 0x7f4f50103b60 2024-09-21T07:03:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.411+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f501a8f40 con 0x7f4f50103b60 2024-09-21T07:03:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.412+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f50103f70 con 0x7f4f50103b60 2024-09-21T07:03:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.415+0000 7f4f4e000640 1 -- 172.21.15.27:0/3222332544 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f4f44009a30 con 0x7f4f50103b60 2024-09-21T07:03:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.416+0000 7f4f4e000640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4f380b64d0 0x7f4f380b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.416+0000 7f4f4e000640 1 -- 172.21.15.27:0/3222332544 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4f440e5850 con 0x7f4f50103b60 2024-09-21T07:03:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.417+0000 7f4f55200640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4f380b64d0 0x7f4f380b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.417+0000 7f4f55200640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4f380b64d0 0x7f4f380b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009230 tx=0x7f4f3c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.418+0000 7f4f4e000640 1 -- 172.21.15.27:0/3222332544 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f440adc40 con 0x7f4f50103b60 2024-09-21T07:03:30.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.616+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4f50076a10 con 0x7f4f50103b60 2024-09-21T07:03:30.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.617+0000 7f4f4e000640 1 -- 172.21.15.27:0/3222332544 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7f4f50076a10 con 0x7f4f50103b60 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global dev auth_debug true 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_file true 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_journald false 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_stderr false 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_allow_pool_delete true 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T07:03:30.620 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_bug true 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_old_message true 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_skipped_message false 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T07:03:30.621 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_mon 20/20 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_ms 1/1 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_paxos 20/20 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T07:03:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_client 20/20 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T07:03:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bdev_debug_aio true 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_bluefs 20/20 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_bluestore 20/20 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_ms 1/1 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_osd 20/20 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_op_order true 2024-09-21T07:03:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_shutdown true 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T07:03:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_mds 20/20 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_ms 2/2 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2024-09-21T07:03:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_split_size 100 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_debug_frag true 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_verify_scatter true 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced client_mount_timeout 600 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced debug_client 20/20 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced debug_ms 1/1 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:03:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T07:03:30.628 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T07:03:30.628 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T07:03:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.619+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4f380b64d0 msgr2=0x7f4f380b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.619+0000 7f4f58600640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4f380b64d0 0x7f4f380b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009230 tx=0x7f4f3c031400 comp rx=0 tx=0).stop 2024-09-21T07:03:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.619+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50103b60 msgr2=0x7f4f50075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.619+0000 7f4f58600640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50103b60 0x7f4f50075710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d950 tx=0x7f4f4400de10 comp rx=0 tx=0).stop 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 shutdown_connections 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4f380b64d0 0x7f4f380b89e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f50109df0 0x7f4f50076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f501094a0 0x7f4f50075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 --2- 172.21.15.27:0/3222332544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f50103b60 0x7f4f50075710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 >> 172.21.15.27:0/3222332544 conn(0x7f4f5007b850 msgr2=0x7f4f50107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 shutdown_connections 2024-09-21T07:03:30.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:30.620+0000 7f4f58600640 1 -- 172.21.15.27:0/3222332544 wait complete. 2024-09-21T07:03:30.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:30 smithi027 ceph-mon[30779]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:30.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:30 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3222332544' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T07:03:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:30 smithi153 ceph-mon[34747]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:30 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3222332544' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T07:03:31.259 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T07:03:31.259 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:03:31.259 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:03:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:30 smithi119 ceph-mon[34519]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:30 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3222332544' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T07:03:31.296 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:03:31.296 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-09-21T07:03:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:03:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:31.372 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-21T07:03:31.372 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:31.372 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:31.373 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 07:02:02.665539660 +0000 2024-09-21T07:03:31.373 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 06:57:15.196191624 +0000 2024-09-21T07:03:31.373 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 06:57:15.196191624 +0000 2024-09-21T07:03:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 06:57:15.196191624 +0000 2024-09-21T07:03:31.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:03:31.466 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T07:03:31.466 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T07:03:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000290708 s, 1.8 MB/s 2024-09-21T07:03:31.469 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:03:31.506 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 07:02:02.720538262 +0000 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 06:57:15.658179870 +0000 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 06:57:15.658179870 +0000 2024-09-21T07:03:31.583 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 06:57:15.658179870 +0000 2024-09-21T07:03:31.583 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:03:31.677 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T07:03:31.677 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T07:03:31.677 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000271615 s, 1.9 MB/s 2024-09-21T07:03:31.679 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:03:31.717 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-09-21T07:03:31.793 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 07:02:02.771536965 +0000 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 06:57:16.129167887 +0000 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 06:57:16.129167887 +0000 2024-09-21T07:03:31.794 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 06:57:16.129167887 +0000 2024-09-21T07:03:31.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:03:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T07:03:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T07:03:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000279518 s, 1.8 MB/s 2024-09-21T07:03:31.890 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:03:31.927 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-09-21T07:03:32.003 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 07:02:02.828535516 +0000 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 06:57:16.575156540 +0000 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 06:57:16.575156540 +0000 2024-09-21T07:03:32.004 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 06:57:16.575156540 +0000 2024-09-21T07:03:32.004 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:03:32.097 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T07:03:32.097 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T07:03:32.097 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000271593 s, 1.9 MB/s 2024-09-21T07:03:32.100 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:03:32.137 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T07:03:32.137 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:03:32.157 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:03:32.157 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-09-21T07:03:32.214 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:03:32.214 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:32.214 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T07:03:32.215 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:32.215 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:32.215 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 07:03:04.396005146 +0000 2024-09-21T07:03:32.215 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 06:56:32.943385343 +0000 2024-09-21T07:03:32.215 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 06:56:32.943385343 +0000 2024-09-21T07:03:32.215 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 06:56:32.943385343 +0000 2024-09-21T07:03:32.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:03:32.284 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T07:03:32.284 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T07:03:32.284 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000233631 s, 2.2 MB/s 2024-09-21T07:03:32.285 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:03:32.344 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 07:03:04.397005121 +0000 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 06:56:33.275376674 +0000 2024-09-21T07:03:32.401 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 06:56:33.275376674 +0000 2024-09-21T07:03:32.402 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 06:56:33.275376674 +0000 2024-09-21T07:03:32.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:03:32.466 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T07:03:32.466 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T07:03:32.467 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000177021 s, 2.9 MB/s 2024-09-21T07:03:32.468 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:03:32.524 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 07:03:04.397005121 +0000 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 06:56:33.609367953 +0000 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 06:56:33.609367953 +0000 2024-09-21T07:03:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 06:56:33.609367953 +0000 2024-09-21T07:03:32.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:03:32.645 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T07:03:32.645 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T07:03:32.645 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000201934 s, 2.5 MB/s 2024-09-21T07:03:32.646 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:03:32.703 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-09-21T07:03:32.758 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 07:03:04.397005121 +0000 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 06:56:33.930359571 +0000 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 06:56:33.930359571 +0000 2024-09-21T07:03:32.759 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 06:56:33.930359571 +0000 2024-09-21T07:03:32.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:03:32.822 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T07:03:32.822 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T07:03:32.822 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000189999 s, 2.7 MB/s 2024-09-21T07:03:32.823 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:03:32.879 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-21T07:03:32.879 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:03:32.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:03:32.900 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-09-21T07:03:32.959 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:03:32.959 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:32.959 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T07:03:32.959 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:32.959 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:32.960 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-21 07:03:02.133572709 +0000 2024-09-21T07:03:32.960 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-21 06:56:36.739119617 +0000 2024-09-21T07:03:32.960 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-21 06:56:36.739119617 +0000 2024-09-21T07:03:32.960 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-21 06:56:36.739119617 +0000 2024-09-21T07:03:32.960 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:03:33.028 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-21T07:03:33.029 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-21T07:03:33.029 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000335745 s, 1.5 MB/s 2024-09-21T07:03:33.030 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:03:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:32 smithi119 ceph-mon[34519]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:33.089 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-09-21T07:03:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:32 smithi027 ceph-mon[30779]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:33.145 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:03:33.145 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:33.145 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T07:03:33.145 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:33.145 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:33.145 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-21 07:03:02.133572709 +0000 2024-09-21T07:03:33.145 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-21 06:56:37.073113663 +0000 2024-09-21T07:03:33.146 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-21 06:56:37.073113663 +0000 2024-09-21T07:03:33.146 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-21 06:56:37.073113663 +0000 2024-09-21T07:03:33.146 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:03:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:32 smithi153 ceph-mon[34747]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-21T07:03:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-21T07:03:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000191488 s, 2.7 MB/s 2024-09-21T07:03:33.203 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:03:33.260 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-09-21T07:03:33.316 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:03:33.316 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:33.316 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T07:03:33.316 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:33.317 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:33.317 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-21 07:03:02.133572709 +0000 2024-09-21T07:03:33.317 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-21 06:56:37.408107690 +0000 2024-09-21T07:03:33.317 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-21 06:56:37.408107690 +0000 2024-09-21T07:03:33.317 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-21 06:56:37.408107690 +0000 2024-09-21T07:03:33.317 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:03:33.380 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-21T07:03:33.381 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-21T07:03:33.381 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000193077 s, 2.7 MB/s 2024-09-21T07:03:33.382 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:03:33.438 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-09-21T07:03:33.493 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-21 07:03:02.134572692 +0000 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-21 06:56:37.734101879 +0000 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-21 06:56:37.734101879 +0000 2024-09-21T07:03:33.494 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-21 06:56:37.734101879 +0000 2024-09-21T07:03:33.494 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:03:33.557 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-21T07:03:33.557 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-21T07:03:33.557 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000161741 s, 3.2 MB/s 2024-09-21T07:03:33.559 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:03:33.614 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2024-09-21T07:03:33.615 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T07:03:33.993 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:35.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:34 smithi027 ceph-mon[30779]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:34 smithi153 ceph-mon[34747]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:34 smithi119 ceph-mon[34519]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:03:36.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-09-21T07:03:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:36 smithi027 ceph-mon[30779]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:36 smithi153 ceph-mon[34747]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:37.193 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:36 smithi119 ceph-mon[34519]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:37.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 -- 172.21.15.27:0/1880617890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e60109c20 msgr2=0x7f6e60110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:37.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 --2- 172.21.15.27:0/1880617890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e60109c20 0x7f6e60110450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009be0 tx=0x7f6e5c02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:03:37.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 -- 172.21.15.27:0/1880617890 shutdown_connections 2024-09-21T07:03:37.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 --2- 172.21.15.27:0/1880617890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e60109c20 0x7f6e60110450 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:37.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 --2- 172.21.15.27:0/1880617890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e601092d0 0x7f6e601096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 --2- 172.21.15.27:0/1880617890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 0x7f6e60103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 -- 172.21.15.27:0/1880617890 >> 172.21.15.27:0/1880617890 conn(0x7f6e600ff3e0 msgr2=0x7f6e60101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 -- 172.21.15.27:0/1880617890 shutdown_connections 2024-09-21T07:03:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.857+0000 7f6e67200640 1 -- 172.21.15.27:0/1880617890 wait complete. 2024-09-21T07:03:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.858+0000 7f6e67200640 1 Processor -- start 2024-09-21T07:03:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.859+0000 7f6e67200640 1 -- start start 2024-09-21T07:03:37.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.859+0000 7f6e67200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 0x7f6e60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:37.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.859+0000 7f6e67200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e601092d0 0x7f6e60075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:37.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.859+0000 7f6e67200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e60109c20 0x7f6e60076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:37.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.859+0000 7f6e67200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601138e0 con 0x7f6e60109c20 2024-09-21T07:03:37.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.859+0000 7f6e67200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6e60113760 con 0x7f6e601092d0 2024-09-21T07:03:37.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.859+0000 7f6e67200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e60113a60 con 0x7f6e60103990 2024-09-21T07:03:37.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 0x7f6e60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:37.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 0x7f6e60075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59992/0 (socket says 172.21.15.27:59992) 2024-09-21T07:03:37.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 -- 172.21.15.27:0/1978190953 learned_addr learned my addr 172.21.15.27:0/1978190953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:03:37.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e66800640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e60109c20 0x7f6e60076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65400640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e601092d0 0x7f6e60075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 -- 172.21.15.27:0/1978190953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e601092d0 msgr2=0x7f6e60075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e601092d0 0x7f6e60075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 -- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e60109c20 msgr2=0x7f6e60076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e60109c20 0x7f6e60076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65e00640 1 -- 172.21.15.27:0/1978190953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e601a8550 con 0x7f6e60103990 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.860+0000 7f6e65400640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e601092d0 0x7f6e60075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.861+0000 7f6e65e00640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 0x7f6e60075710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400e9f0 tx=0x7f6e5400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.861+0000 7f6e5aa00640 1 -- 172.21.15.27:0/1978190953 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400ccc0 con 0x7f6e60103990 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.861+0000 7f6e5aa00640 1 -- 172.21.15.27:0/1978190953 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e540108e0 con 0x7f6e60103990 2024-09-21T07:03:37.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.861+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e601a8840 con 0x7f6e60103990 2024-09-21T07:03:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.861+0000 7f6e5aa00640 1 -- 172.21.15.27:0/1978190953 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54018880 con 0x7f6e60103990 2024-09-21T07:03:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.861+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e601a8dd0 con 0x7f6e60103990 2024-09-21T07:03:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.862+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e60103da0 con 0x7f6e60103990 2024-09-21T07:03:37.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.864+0000 7f6e5aa00640 1 -- 172.21.15.27:0/1978190953 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f6e54010430 con 0x7f6e60103990 2024-09-21T07:03:37.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.866+0000 7f6e5aa00640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e480b6550 0x7f6e480b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:37.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.866+0000 7f6e5aa00640 1 -- 172.21.15.27:0/1978190953 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6e540e5b40 con 0x7f6e60103990 2024-09-21T07:03:37.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.866+0000 7f6e65400640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e480b6550 0x7f6e480b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:37.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.867+0000 7f6e65400640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e480b6550 0x7f6e480b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e60106be0 tx=0x7f6e50009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:37.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:37.870+0000 7f6e5aa00640 1 -- 172.21.15.27:0/1978190953 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e540adf30 con 0x7f6e60103990 2024-09-21T07:03:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:03:38.076+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6e60063800 con 0x7f6e480b6550 2024-09-21T07:03:38.916 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:38 smithi027 ceph-mon[30779]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:38.916 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:03:38.916 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:03:38.916 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:38 smithi153 ceph-mon[34747]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:03:39.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:03:39.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:38 smithi119 ceph-mon[34519]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:39.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:03:39.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:03:39.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:39.937 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:39 smithi027 ceph-mon[30779]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:03:40.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:39 smithi153 ceph-mon[34747]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:03:40.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:39 smithi119 ceph-mon[34519]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:03:41.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:40 smithi027 ceph-mon[30779]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:40 smithi153 ceph-mon[34747]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:40 smithi119 ceph-mon[34519]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:42.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2123774018' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d999c06a-d17f-4840-9bec-67d0325b7546"} : dispatch 2024-09-21T07:03:42.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2123774018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d999c06a-d17f-4840-9bec-67d0325b7546"}]': finished 2024-09-21T07:03:42.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:41 smithi027 ceph-mon[30779]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T07:03:42.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:03:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2123774018' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d999c06a-d17f-4840-9bec-67d0325b7546"} : dispatch 2024-09-21T07:03:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2123774018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d999c06a-d17f-4840-9bec-67d0325b7546"}]': finished 2024-09-21T07:03:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:41 smithi153 ceph-mon[34747]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T07:03:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:03:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2123774018' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d999c06a-d17f-4840-9bec-67d0325b7546"} : dispatch 2024-09-21T07:03:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2123774018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d999c06a-d17f-4840-9bec-67d0325b7546"}]': finished 2024-09-21T07:03:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:41 smithi119 ceph-mon[34519]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T07:03:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:03:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:42 smithi027 ceph-mon[30779]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:42 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4198976727' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:03:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:42 smithi153 ceph-mon[34747]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:42 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4198976727' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:03:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:42 smithi119 ceph-mon[34519]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:42 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4198976727' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:03:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:44 smithi027 ceph-mon[30779]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:44 smithi153 ceph-mon[34747]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:44 smithi119 ceph-mon[34519]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:47.021 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:46 smithi027 ceph-mon[30779]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:46 smithi153 ceph-mon[34747]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:46 smithi119 ceph-mon[34519]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:48 smithi027 ceph-mon[30779]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:48 smithi153 ceph-mon[34747]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:48 smithi119 ceph-mon[34519]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:50.945 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:50 smithi027 ceph-mon[30779]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:50.945 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T07:03:50.945 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:50 smithi153 ceph-mon[34747]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T07:03:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:50 smithi119 ceph-mon[34519]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T07:03:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:52.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:51 smithi027 ceph-mon[30779]: Deploying daemon osd.0 on smithi027 2024-09-21T07:03:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:51 smithi153 ceph-mon[34747]: Deploying daemon osd.0 on smithi027 2024-09-21T07:03:52.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:51 smithi119 ceph-mon[34519]: Deploying daemon osd.0 on smithi027 2024-09-21T07:03:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:52 smithi027 ceph-mon[30779]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:52 smithi153 ceph-mon[34747]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:52 smithi119 ceph-mon[34519]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:54 smithi153 ceph-mon[34747]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:54 smithi119 ceph-mon[34519]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:54 smithi027 ceph-mon[30779]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:57.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:56 smithi027 ceph-mon[30779]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:56 smithi153 ceph-mon[34747]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:56 smithi119 ceph-mon[34519]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:58 smithi153 ceph-mon[34747]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:59.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:03:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:59.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:58 smithi027 ceph-mon[30779]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:59.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:59.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:59.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:03:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:58 smithi119 ceph-mon[34519]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:03:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:03:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:03:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:01.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:00 smithi027 ceph-mon[30779]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:00 smithi153 ceph-mon[34747]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:00 smithi119 ceph-mon[34519]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:02.238 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.238 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.239 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:02 smithi153 ceph-mon[34747]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:02.239 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:02.239 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:02.239 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:02 smithi119 ceph-mon[34519]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:02.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:02.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:02.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:02 smithi027 ceph-mon[30779]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:02.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.745+0000 7f6e5aa00640 1 -- 172.21.15.27:0/1978190953 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6e60063800 con 0x7f6e480b6550 2024-09-21T07:04:03.747 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2024-09-21T07:04:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e480b6550 msgr2=0x7f6e480b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e480b6550 0x7f6e480b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e60106be0 tx=0x7f6e50009260 comp rx=0 tx=0).stop 2024-09-21T07:04:04.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 msgr2=0x7f6e60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:04.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 0x7f6e60075710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400e9f0 tx=0x7f6e5400edf0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 shutdown_connections 2024-09-21T07:04:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6e480b6550 0x7f6e480b8a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e60109c20 0x7f6e60076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e601092d0 0x7f6e60075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 --2- 172.21.15.27:0/1978190953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e60103990 0x7f6e60075710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.747+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 >> 172.21.15.27:0/1978190953 conn(0x7f6e600ff3e0 msgr2=0x7f6e60101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.748+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 shutdown_connections 2024-09-21T07:04:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:03.748+0000 7f6e67200640 1 -- 172.21.15.27:0/1978190953 wait complete. 2024-09-21T07:04:04.480 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T07:04:04.484 INFO:tasks.cephadm:Deploying osd.1 on smithi119 with /dev/vg_nvme/lv_4... 2024-09-21T07:04:04.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T07:04:04.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:04 smithi027 ceph-mon[30779]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:04.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:04.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:04.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:04.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:04 smithi153 ceph-mon[34747]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:04.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:04.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:04.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:04:04.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:04 smithi119 ceph-mon[34519]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:04.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:04.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:04.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:04.870 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:04:04 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0[48167]: 2024-09-21T07:04:04.613+0000 7f3c3289e640 -1 osd.0 0 log_to_monitors true 2024-09-21T07:04:05.642 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:05 smithi119 ceph-mon[34519]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T07:04:05.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:05 smithi153 ceph-mon[34747]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T07:04:05.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:05 smithi027 ceph-mon[30779]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T07:04:06.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:06 smithi119 ceph-mon[34519]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:06.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:06 smithi119 ceph-mon[34519]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T07:04:06.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:06 smithi119 ceph-mon[34519]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T07:04:06.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:06 smithi119 ceph-mon[34519]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:04:06.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:06.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:06 smithi153 ceph-mon[34747]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:06.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:06 smithi153 ceph-mon[34747]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T07:04:06.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:06 smithi153 ceph-mon[34747]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T07:04:06.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:06 smithi153 ceph-mon[34747]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:04:06.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:06 smithi027 ceph-mon[30779]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:06.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:06 smithi027 ceph-mon[30779]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T07:04:06.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:06 smithi027 ceph-mon[30779]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T07:04:06.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:06 smithi027 ceph-mon[30779]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:04:06.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:06.842 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:04:06 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0[48167]: 2024-09-21T07:04:06.375+0000 7f3c2d800640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T07:04:06.842 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:04:06 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0[48167]: 2024-09-21T07:04:06.380+0000 7f3c28800640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:04:06.860 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T07:04:06.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2024-09-21T07:04:07.126 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:04:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:07 smithi119 ceph-mon[34519]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:04:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:07 smithi119 ceph-mon[34519]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T07:04:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 -- 172.21.15.119:0/1102890583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef8102a60 msgr2=0x7f7ef8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/1102890583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef8102a60 0x7f7ef8102e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009900 tx=0x7f7ef402a410 comp rx=0 tx=0).stop 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 -- 172.21.15.119:0/1102890583 shutdown_connections 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/1102890583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef81083a0 0x7f7ef81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/1102890583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef8108cf0 0x7f7ef810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/1102890583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef8102a60 0x7f7ef8102e70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 -- 172.21.15.119:0/1102890583 >> 172.21.15.119:0/1102890583 conn(0x7f7ef8079730 msgr2=0x7f7ef8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 -- 172.21.15.119:0/1102890583 shutdown_connections 2024-09-21T07:04:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 -- 172.21.15.119:0/1102890583 wait complete. 2024-09-21T07:04:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.571+0000 7f7eff3e3640 1 Processor -- start 2024-09-21T07:04:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7eff3e3640 1 -- start start 2024-09-21T07:04:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7eff3e3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef8102a60 0x7f7ef819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7eff3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81083a0 0x7f7ef819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7eff3e3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef8108cf0 0x7f7ef819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7eff3e3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81129e0 con 0x7f7ef8108cf0 2024-09-21T07:04:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7eff3e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112860 con 0x7f7ef81083a0 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7eff3e3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112b60 con 0x7f7ef8102a60 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81083a0 0x7f7ef819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81083a0 0x7f7ef819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53130/0 (socket says 172.21.15.119:53130) 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 -- 172.21.15.119:0/2163100696 learned_addr learned my addr 172.21.15.119:0/2163100696 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efd17f640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef8102a60 0x7f7ef819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efd980640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef8108cf0 0x7f7ef819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 -- 172.21.15.119:0/2163100696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef8102a60 msgr2=0x7f7ef819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef8102a60 0x7f7ef819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 -- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef8108cf0 msgr2=0x7f7ef819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef8108cf0 0x7f7ef819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efc97e640 1 -- 172.21.15.119:0/2163100696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef81a0470 con 0x7f7ef81083a0 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.572+0000 7f7efd980640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef8108cf0 0x7f7ef819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.573+0000 7f7efc97e640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81083a0 0x7f7ef819aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800c970 tx=0x7f7ee800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.573+0000 7f7eee7fc640 1 -- 172.21.15.119:0/2163100696 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8007bb0 con 0x7f7ef81083a0 2024-09-21T07:04:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.573+0000 7f7eee7fc640 1 -- 172.21.15.119:0/2163100696 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee8007d50 con 0x7f7ef81083a0 2024-09-21T07:04:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.573+0000 7f7eee7fc640 1 -- 172.21.15.119:0/2163100696 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8018730 con 0x7f7ef81083a0 2024-09-21T07:04:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.573+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef81a0760 con 0x7f7ef81083a0 2024-09-21T07:04:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.573+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef8104130 con 0x7f7ef81083a0 2024-09-21T07:04:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.574+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef8102e70 con 0x7f7ef81083a0 2024-09-21T07:04:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.575+0000 7f7eee7fc640 1 -- 172.21.15.119:0/2163100696 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f7ee800f990 con 0x7f7ef81083a0 2024-09-21T07:04:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.576+0000 7f7eee7fc640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ee40b6550 0x7f7ee40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.576+0000 7f7eee7fc640 1 -- 172.21.15.119:0/2163100696 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1982+0+0 (secure 0 0 0) 0x7f7ee80e5e10 con 0x7f7ef81083a0 2024-09-21T07:04:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.577+0000 7f7efd17f640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ee40b6550 0x7f7ee40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:07.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.577+0000 7f7efd17f640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ee40b6550 0x7f7ee40b8a60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4031440 tx=0x7f7ef4005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:07.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.580+0000 7f7eee7fc640 1 -- 172.21.15.119:0/2163100696 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ee8014030 con 0x7f7ef81083a0 2024-09-21T07:04:07.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:07 smithi153 ceph-mon[34747]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:04:07.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:07 smithi153 ceph-mon[34747]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T07:04:07.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:07.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:07.749 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:07 smithi027 ceph-mon[30779]: from='osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:04:07.749 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:07 smithi027 ceph-mon[30779]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T07:04:07.749 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:07.749 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:07.775+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7ef806ae90 con 0x7f7ee40b6550 2024-09-21T07:04:08.482 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:04:08.482 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:04:08.482 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] boot 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:08.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] boot 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:08.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:08.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:04:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:04:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: osd.0 [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] boot 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:08.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:09.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:09 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi027 2024-09-21T07:04:09.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:09 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T07:04:09.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:09 smithi119 ceph-mon[34519]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T07:04:09.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:09 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi027 2024-09-21T07:04:09.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:09 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T07:04:09.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:09 smithi153 ceph-mon[34747]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T07:04:09.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:09 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi027 2024-09-21T07:04:09.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:09 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T07:04:09.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:09 smithi027 ceph-mon[30779]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T07:04:10.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:10 smithi119 ceph-mon[34519]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:10.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:10 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/3076589060' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"} : dispatch 2024-09-21T07:04:10.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:10 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"} : dispatch 2024-09-21T07:04:10.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:10 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"}]': finished 2024-09-21T07:04:10.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:10 smithi119 ceph-mon[34519]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T07:04:10.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:10 smithi153 ceph-mon[34747]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:10 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/3076589060' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"} : dispatch 2024-09-21T07:04:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:10 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"} : dispatch 2024-09-21T07:04:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:10 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"}]': finished 2024-09-21T07:04:10.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:10 smithi153 ceph-mon[34747]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T07:04:10.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:10 smithi027 ceph-mon[30779]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:10 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/3076589060' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"} : dispatch 2024-09-21T07:04:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:10 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"} : dispatch 2024-09-21T07:04:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:10 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4063fc7c-3ba2-4b37-b188-45faf89d4127"}]': finished 2024-09-21T07:04:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:10 smithi027 ceph-mon[30779]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T07:04:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:11.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:11 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/2733679721' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:04:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:11 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/2733679721' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:04:11.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:11 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/2733679721' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:04:12.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:12 smithi153 ceph-mon[34747]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:12.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:12 smithi119 ceph-mon[34519]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:12.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:12 smithi027 ceph-mon[30779]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:14.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:14 smithi153 ceph-mon[34747]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:14.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:14 smithi119 ceph-mon[34519]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:14 smithi027 ceph-mon[30779]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:16.497 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:16 smithi119 ceph-mon[34519]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:16.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:16 smithi027 ceph-mon[30779]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:16.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:16 smithi153 ceph-mon[34747]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:18.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:18 smithi153 ceph-mon[34747]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:18.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T07:04:18.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:18.692 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:18 smithi119 ceph-mon[34519]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:18.692 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T07:04:18.692 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:18 smithi027 ceph-mon[30779]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T07:04:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:19.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:19 smithi153 ceph-mon[34747]: Deploying daemon osd.1 on smithi119 2024-09-21T07:04:19.715 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:19 smithi119 ceph-mon[34519]: Deploying daemon osd.1 on smithi119 2024-09-21T07:04:19.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:19 smithi027 ceph-mon[30779]: Deploying daemon osd.1 on smithi119 2024-09-21T07:04:20.460 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:20 smithi119 ceph-mon[34519]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:20.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:20 smithi153 ceph-mon[34747]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:20 smithi027 ceph-mon[30779]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:22.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:22 smithi153 ceph-mon[34747]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:22 smithi119 ceph-mon[34519]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:22 smithi027 ceph-mon[30779]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:24 smithi119 ceph-mon[34519]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:24.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:24 smithi153 ceph-mon[34747]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:24.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:24 smithi027 ceph-mon[30779]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:26.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:26 smithi153 ceph-mon[34747]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:26.684 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:26 smithi119 ceph-mon[34519]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:26.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:26 smithi027 ceph-mon[30779]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:27.742 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.742 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.742 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:27.742 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:27.742 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:27.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:27.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:27.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:28.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.336+0000 7f7eee7fc640 1 -- 172.21.15.119:0/2163100696 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ef806ae90 con 0x7f7ee40b6550 2024-09-21T07:04:28.340 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 1 on host 'smithi119' 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ee40b6550 msgr2=0x7f7ee40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ee40b6550 0x7f7ee40b8a60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4031440 tx=0x7f7ef4005d50 comp rx=0 tx=0).stop 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81083a0 msgr2=0x7f7ef819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81083a0 0x7f7ef819aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800c970 tx=0x7f7ee800ce30 comp rx=0 tx=0).stop 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 shutdown_connections 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ee40b6550 0x7f7ee40b8a60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef8108cf0 0x7f7ef819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef81083a0 0x7f7ef819aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 --2- 172.21.15.119:0/2163100696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef8102a60 0x7f7ef819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 >> 172.21.15.119:0/2163100696 conn(0x7f7ef8079730 msgr2=0x7f7ef8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:28.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 shutdown_connections 2024-09-21T07:04:28.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:04:28.339+0000 7f7eff3e3640 1 -- 172.21.15.119:0/2163100696 wait complete. 2024-09-21T07:04:28.691 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:28 smithi119 ceph-mon[34519]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:28.691 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:28.691 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:28.691 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:28.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:28 smithi027 ceph-mon[30779]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:28.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:28.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:28.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:28 smithi153 ceph-mon[34747]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:29.052 DEBUG:teuthology.orchestra.run.smithi119:osd.1> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T07:04:29.054 INFO:tasks.cephadm:Deploying osd.2 on smithi153 with /dev/vg_nvme/lv_4... 2024-09-21T07:04:29.054 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T07:04:29.306 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:04:30.581 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:30 smithi153 ceph-mon[34747]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:30.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:30 smithi119 ceph-mon[34519]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:30.785 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:04:30 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1[39229]: 2024-09-21T07:04:30.606+0000 7f94dcdc3640 -1 osd.1 0 log_to_monitors true 2024-09-21T07:04:30.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:30 smithi027 ceph-mon[30779]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:31.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:31 smithi153 ceph-mon[34747]: from='osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T07:04:31.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:31 smithi153 ceph-mon[34747]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T07:04:31.705 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:04:31.731 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2024-09-21T07:04:31.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:31 smithi119 ceph-mon[34519]: from='osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T07:04:31.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:31 smithi119 ceph-mon[34519]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T07:04:31.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:31 smithi027 ceph-mon[30779]: from='osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T07:04:31.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:31 smithi027 ceph-mon[30779]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T07:04:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:04:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 -- 172.21.15.153:0/187483850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218109df0 msgr2=0x7f2218110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 --2- 172.21.15.153:0/187483850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218109df0 0x7f2218110620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2210009ca0 tx=0x7f221002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 -- 172.21.15.153:0/187483850 shutdown_connections 2024-09-21T07:04:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 --2- 172.21.15.153:0/187483850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218109df0 0x7f2218110620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 --2- 172.21.15.153:0/187483850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22181094a0 0x7f22181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 --2- 172.21.15.153:0/187483850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2218103b60 0x7f2218103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 -- 172.21.15.153:0/187483850 >> 172.21.15.153:0/187483850 conn(0x7f221807b850 msgr2=0x7f221807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 -- 172.21.15.153:0/187483850 shutdown_connections 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.439+0000 7f2220116640 1 -- 172.21.15.153:0/187483850 wait complete. 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 Processor -- start 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 -- start start 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218103b60 0x7f221819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22181094a0 0x7f221819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2218109df0 0x7f221819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2218113ab0 con 0x7f22181094a0 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2218113930 con 0x7f2218109df0 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f2220116640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2218113c30 con 0x7f2218103b60 2024-09-21T07:04:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218103b60 0x7f221819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218103b60 0x7f221819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43292/0 (socket says 172.21.15.153:43292) 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 -- 172.21.15.153:0/182895558 learned_addr learned my addr 172.21.15.153:0/182895558 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 -- 172.21.15.153:0/182895558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2218109df0 msgr2=0x7f221819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221e6b3640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2218109df0 0x7f221819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2218109df0 0x7f221819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 -- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22181094a0 msgr2=0x7f221819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22181094a0 0x7f221819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.440+0000 7f221deb2640 1 -- 172.21.15.153:0/182895558 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22181a0390 con 0x7f2218103b60 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.441+0000 7f221deb2640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218103b60 0x7f221819a420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f221400c890 tx=0x7f221400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.441+0000 7f220effd640 1 -- 172.21.15.153:0/182895558 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214004220 con 0x7f2218103b60 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.441+0000 7f220effd640 1 -- 172.21.15.153:0/182895558 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22140043c0 con 0x7f2218103b60 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.441+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22181a0680 con 0x7f2218103b60 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.441+0000 7f220effd640 1 -- 172.21.15.153:0/182895558 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214004a00 con 0x7f2218103b60 2024-09-21T07:04:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.441+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22181a6de0 con 0x7f2218103b60 2024-09-21T07:04:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.442+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2218103f70 con 0x7f2218103b60 2024-09-21T07:04:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.443+0000 7f220effd640 1 -- 172.21.15.153:0/182895558 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f22140079b0 con 0x7f2218103b60 2024-09-21T07:04:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.444+0000 7f220effd640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22040b6550 0x7f22040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.445+0000 7f220effd640 1 -- 172.21.15.153:0/182895558 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2118+0+0 (secure 0 0 0) 0x7f22140e5be0 con 0x7f2218103b60 2024-09-21T07:04:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.445+0000 7f221d6b1640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22040b6550 0x7f22040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.446+0000 7f221d6b1640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22040b6550 0x7f22040b8a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22080058c0 tx=0x7f2208005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.448+0000 7f220effd640 1 -- 172.21.15.153:0/182895558 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2214014030 con 0x7f2218103b60 2024-09-21T07:04:32.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:32.640+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2218063800 con 0x7f22040b6550 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi119 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi119 to 17088M 2024-09-21T07:04:32.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:32.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:32.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:32.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T07:04:32.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi119 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi119 to 17088M 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T07:04:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T07:04:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi119 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi119 to 17088M 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:33.499 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1[39229]: 2024-09-21T07:04:33.181+0000 7f94d8d45640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T07:04:33.499 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1[39229]: 2024-09-21T07:04:33.185+0000 7f94d0b3b640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:04:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:04:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T07:04:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:33.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:33.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:33.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:33.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:33 smithi153 ceph-mon[34747]: from='osd.1 ' entity='osd.1' 2024-09-21T07:04:33.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:04:33.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T07:04:33.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:33.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:33.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:33.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:33.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:33.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:33.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:33 smithi119 ceph-mon[34519]: from='osd.1 ' entity='osd.1' 2024-09-21T07:04:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:04:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T07:04:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:33.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:33.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:33 smithi027 ceph-mon[30779]: from='osd.1 ' entity='osd.1' 2024-09-21T07:04:34.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:34 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:04:34.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:34 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:04:34.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:34 smithi119 ceph-mon[34519]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:34.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:34.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:34 smithi119 ceph-mon[34519]: osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] boot 2024-09-21T07:04:34.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:34 smithi119 ceph-mon[34519]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T07:04:34.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:34 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:04:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:34 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:04:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:34 smithi027 ceph-mon[30779]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:34 smithi027 ceph-mon[30779]: osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] boot 2024-09-21T07:04:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:34 smithi027 ceph-mon[30779]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T07:04:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:34 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:34 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:34 smithi153 ceph-mon[34747]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:34 smithi153 ceph-mon[34747]: osd.1 [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] boot 2024-09-21T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:34 smithi153 ceph-mon[34747]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T07:04:35.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:35 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/4245781752' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"} : dispatch 2024-09-21T07:04:35.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:35 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"} : dispatch 2024-09-21T07:04:35.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:35 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"}]': finished 2024-09-21T07:04:35.763 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:35 smithi027 ceph-mon[30779]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T07:04:35.763 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:35 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:35.763 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:35 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/1421050688' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:04:35.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:35 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/4245781752' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"} : dispatch 2024-09-21T07:04:35.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:35 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"} : dispatch 2024-09-21T07:04:35.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:35 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"}]': finished 2024-09-21T07:04:35.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:35 smithi119 ceph-mon[34519]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T07:04:35.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:35 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:35.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:35 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/1421050688' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:04:35.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:35 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/4245781752' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"} : dispatch 2024-09-21T07:04:35.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:35 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"} : dispatch 2024-09-21T07:04:35.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:35 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f871d34f-c35e-4ce0-9c20-09911568fc95"}]': finished 2024-09-21T07:04:35.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:35 smithi153 ceph-mon[34747]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T07:04:35.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:35 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:35.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:35 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/1421050688' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:04:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:36 smithi119 ceph-mon[34519]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:37.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:36 smithi027 ceph-mon[30779]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:36 smithi153 ceph-mon[34747]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:38 smithi119 ceph-mon[34519]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:38 smithi027 ceph-mon[30779]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:38 smithi153 ceph-mon[34747]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:41.263 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:40 smithi153 ceph-mon[34747]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:40 smithi119 ceph-mon[34519]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:40 smithi027 ceph-mon[30779]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:42 smithi153 ceph-mon[34747]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:42 smithi119 ceph-mon[34519]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:42 smithi027 ceph-mon[30779]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:44.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:43 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T07:04:44.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:43 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:44.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:43 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T07:04:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:43 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:44.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:43 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T07:04:44.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:43 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:44 smithi153 ceph-mon[34747]: Deploying daemon osd.2 on smithi153 2024-09-21T07:04:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:44 smithi153 ceph-mon[34747]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:44 smithi119 ceph-mon[34519]: Deploying daemon osd.2 on smithi153 2024-09-21T07:04:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:44 smithi119 ceph-mon[34519]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:44 smithi027 ceph-mon[30779]: Deploying daemon osd.2 on smithi153 2024-09-21T07:04:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:44 smithi027 ceph-mon[30779]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:46 smithi027 ceph-mon[30779]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:46 smithi119 ceph-mon[34519]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:47.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:46 smithi153 ceph-mon[34747]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:48 smithi153 ceph-mon[34747]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:48 smithi119 ceph-mon[34519]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:49.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:48 smithi027 ceph-mon[30779]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:50.147 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:50.147 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:50.147 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:51.110 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:50 smithi153 ceph-mon[34747]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:50 smithi119 ceph-mon[34519]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:51.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:50 smithi027 ceph-mon[30779]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:52.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.927+0000 7f220effd640 1 -- 172.21.15.153:0/182895558 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2218063800 con 0x7f22040b6550 2024-09-21T07:04:52.929 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 2 on host 'smithi153' 2024-09-21T07:04:52.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22040b6550 msgr2=0x7f22040b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:52.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22040b6550 0x7f22040b8a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22080058c0 tx=0x7f2208005850 comp rx=0 tx=0).stop 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218103b60 msgr2=0x7f221819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218103b60 0x7f221819a420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f221400c890 tx=0x7f221400cd50 comp rx=0 tx=0).stop 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 shutdown_connections 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22040b6550 0x7f22040b8a60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2218109df0 0x7f221819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22181094a0 0x7f221819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 --2- 172.21.15.153:0/182895558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2218103b60 0x7f221819a420 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 >> 172.21.15.153:0/182895558 conn(0x7f221807b850 msgr2=0x7f22181074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 shutdown_connections 2024-09-21T07:04:52.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:04:52.930+0000 7f2220116640 1 -- 172.21.15.153:0/182895558 wait complete. 2024-09-21T07:04:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:53.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:53.550 DEBUG:teuthology.orchestra.run.smithi153:osd.2> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T07:04:53.552 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2024-09-21T07:04:53.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T07:04:53.942 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:55.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:54 smithi027 ceph-mon[30779]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:54 smithi153 ceph-mon[34747]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:54 smithi119 ceph-mon[34519]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:55.624 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:04:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2[40027]: 2024-09-21T07:04:55.315+0000 7fee13f2c640 -1 osd.2 0 log_to_monitors true 2024-09-21T07:04:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:55 smithi119 ceph-mon[34519]: from='osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T07:04:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:55 smithi119 ceph-mon[34519]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T07:04:56.292 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:55 smithi153 ceph-mon[34747]: from='osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T07:04:56.292 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:55 smithi153 ceph-mon[34747]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T07:04:56.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:55 smithi027 ceph-mon[30779]: from='osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T07:04:56.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:55 smithi027 ceph-mon[30779]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T07:04:56.826 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:04:56.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-09-21T07:04:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T07:04:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:04:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.260 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T07:04:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:04:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T07:04:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:04:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T07:04:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:04:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T07:04:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:04:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:04:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:57.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:04:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.841+0000 7f4feb800640 1 -- 172.21.15.27:0/1125165806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4077940 msgr2=0x7f4fe4077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.841+0000 7f4feb800640 1 --2- 172.21.15.27:0/1125165806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4077940 0x7f4fe4077d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009900 tx=0x7f4fd402a400 comp rx=0 tx=0).stop 2024-09-21T07:04:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.842+0000 7f4feb800640 1 -- 172.21.15.27:0/1125165806 shutdown_connections 2024-09-21T07:04:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.842+0000 7f4feb800640 1 --2- 172.21.15.27:0/1125165806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4114750 0x7f4fe4116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.842+0000 7f4feb800640 1 --2- 172.21.15.27:0/1125165806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe4078290 0x7f4fe4114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.842+0000 7f4feb800640 1 --2- 172.21.15.27:0/1125165806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4077940 0x7f4fe4077d50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.842+0000 7f4feb800640 1 -- 172.21.15.27:0/1125165806 >> 172.21.15.27:0/1125165806 conn(0x7f4fe4101860 msgr2=0x7f4fe4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:57.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.842+0000 7f4feb800640 1 -- 172.21.15.27:0/1125165806 shutdown_connections 2024-09-21T07:04:57.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.842+0000 7f4feb800640 1 -- 172.21.15.27:0/1125165806 wait complete. 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 Processor -- start 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 -- start start 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4077940 0x7f4fe41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4078290 0x7f4fe41a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe4114750 0x7f4fe41a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe411a0e0 con 0x7f4fe4078290 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4119f60 con 0x7f4fe4114750 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.843+0000 7f4feb800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe411a260 con 0x7f4fe4077940 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4077940 0x7f4fe41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4078290 0x7f4fe41a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:57.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4077940 0x7f4fe41a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45690/0 (socket says 172.21.15.27:45690) 2024-09-21T07:04:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4078290 0x7f4fe41a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47580/0 (socket says 172.21.15.27:47580) 2024-09-21T07:04:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fdbe00640 1 -- 172.21.15.27:0/1738196949 learned_addr learned my addr 172.21.15.27:0/1738196949 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:04:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe4114750 0x7f4fe41a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe9800640 1 -- 172.21.15.27:0/1738196949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4077940 msgr2=0x7f4fe41a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe9800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4077940 0x7f4fe41a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe9800640 1 -- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4078290 msgr2=0x7f4fe41a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe9800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4078290 0x7f4fe41a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe9800640 1 -- 172.21.15.27:0/1738196949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe41a6a20 con 0x7f4fe4114750 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.844+0000 7f4fe8e00640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4077940 0x7f4fe41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.845+0000 7f4fe9800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe4114750 0x7f4fe41a62f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009fd0 tx=0x7f4fdc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.845+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00ce20 con 0x7f4fe4114750 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.845+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41a6d10 con 0x7f4fe4114750 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.845+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fdc005020 con 0x7f4fe4114750 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.845+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc010690 con 0x7f4fe4114750 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.845+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41ad470 con 0x7f4fe4114750 2024-09-21T07:04:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.846+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe4077d50 con 0x7f4fe4114750 2024-09-21T07:04:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.849+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f4fdc0054d0 con 0x7f4fe4114750 2024-09-21T07:04:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.850+0000 7f4fe2000640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc80b64d0 0x7f4fc80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.850+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2786+0+0 (secure 0 0 0) 0x7f4fdc0e63f0 con 0x7f4fe4114750 2024-09-21T07:04:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.850+0000 7f4fe8e00640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc80b64d0 0x7f4fc80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.851+0000 7f4fe8e00640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc80b64d0 0x7f4fc80b89e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4031440 tx=0x7f4fd4032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:57.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:57.852+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4fdc0ae250 con 0x7f4fe4114750 2024-09-21T07:04:58.007 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2[40027]: 2024-09-21T07:04:57.700+0000 7fee106c3640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T07:04:58.007 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:04:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2[40027]: 2024-09-21T07:04:57.705+0000 7fee07ca4640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:04:58.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:04:58.064+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4fe4063800 con 0x7f4fc80b64d0 2024-09-21T07:04:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:58 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi153 2024-09-21T07:04:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:58 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-21T07:04:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:58 smithi119 ceph-mon[34519]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:04:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:58 smithi119 ceph-mon[34519]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T07:04:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:58 smithi119 ceph-mon[34519]: from='osd.2 ' entity='osd.2' 2024-09-21T07:04:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:58 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi153 2024-09-21T07:04:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:58 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-21T07:04:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:58 smithi027 ceph-mon[30779]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:04:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:58 smithi027 ceph-mon[30779]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T07:04:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:58 smithi027 ceph-mon[30779]: from='osd.2 ' entity='osd.2' 2024-09-21T07:04:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:58 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi153 2024-09-21T07:04:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:58 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-21T07:04:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:58 smithi153 ceph-mon[34747]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:04:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:58 smithi153 ceph-mon[34747]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T07:04:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:58 smithi153 ceph-mon[34747]: from='osd.2 ' entity='osd.2' 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:59.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] boot 2024-09-21T07:04:59.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T07:04:59.268 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:04:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:04:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:04:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] boot 2024-09-21T07:04:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T07:04:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:04:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:04:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: osd.2 [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] boot 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T07:04:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:04:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T07:05:00.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:00 smithi027 ceph-mon[30779]: pgmap v69: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:00.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:05:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:00 smithi119 ceph-mon[34519]: pgmap v69: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:05:00.329 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:00 smithi153 ceph-mon[34747]: pgmap v69: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:00.329 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:05:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T07:05:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 ceph-mon[30779]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T07:05:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:05:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T07:05:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 ceph-mon[34519]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T07:05:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43217]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43217]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43217]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43220]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43220]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43220]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.286 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T07:05:01.286 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 ceph-mon[34747]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T07:05:01.286 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:05:01.368 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52767]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T07:05:01.368 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52767]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.368 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52767]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.369 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52767]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.369 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52770]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T07:05:01.369 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.369 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52770]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.369 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52770]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.578 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43974]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T07:05:01.578 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43974]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.578 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43974]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.579 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43974]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.579 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T07:05:01.579 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.579 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43977]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.579 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43977]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.579 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43980]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T07:05:01.579 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43980]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.579 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43980]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.579 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43980]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.579 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43983]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T07:05:01.579 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43983]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.580 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43983]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.580 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:01 smithi153 sudo[43983]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52779]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52779]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52779]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52779]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52782]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52782]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52782]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.746 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:01 smithi027 sudo[52782]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43223]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T07:05:01.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43223]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43223]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:01.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T07:05:01.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T07:05:01.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43226]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T07:05:01.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:01 smithi119 sudo[43226]: pam_unix(sudo:session): session closed for user root 2024-09-21T07:05:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/848665210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"} : dispatch 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"} : dispatch 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"}]': finished 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:02 smithi119 ceph-mon[34519]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:02.283+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4fdc0a9dc0 con 0x7f4fe4114750 2024-09-21T07:05:02.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:05:02.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T07:05:02.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/848665210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"} : dispatch 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"} : dispatch 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"}]': finished 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:02 smithi027 ceph-mon[30779]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/848665210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"} : dispatch 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"} : dispatch 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db7d7254-c55f-4b12-8c53-9292a254ffcb"}]': finished 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T07:05:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T07:05:02.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:02 smithi153 ceph-mon[34747]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T07:05:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:03 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1949440739' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:03 smithi119 ceph-mon[34519]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T07:05:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:03 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1949440739' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:03 smithi027 ceph-mon[30779]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T07:05:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:03 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1949440739' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:03 smithi153 ceph-mon[34747]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T07:05:04.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:04 smithi027 ceph-mon[30779]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:04.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:04 smithi153 ceph-mon[34747]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:04.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:04 smithi119 ceph-mon[34519]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:06.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:06 smithi153 ceph-mon[34747]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:06.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:06 smithi119 ceph-mon[34519]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:06 smithi027 ceph-mon[30779]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:08.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:08 smithi153 ceph-mon[34747]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:08.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:08 smithi119 ceph-mon[34519]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:08.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:08 smithi027 ceph-mon[30779]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:10.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:10 smithi027 ceph-mon[30779]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:10 smithi153 ceph-mon[34747]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:10.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:10 smithi119 ceph-mon[34519]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:11.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T07:05:11.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:11.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:11 smithi027 ceph-mon[30779]: Deploying daemon osd.3 on smithi027 2024-09-21T07:05:11.652 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T07:05:11.652 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:11.653 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:11 smithi153 ceph-mon[34747]: Deploying daemon osd.3 on smithi027 2024-09-21T07:05:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T07:05:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:11 smithi119 ceph-mon[34519]: Deploying daemon osd.3 on smithi027 2024-09-21T07:05:12.546 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:12 smithi027 ceph-mon[30779]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:12.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:12 smithi153 ceph-mon[34747]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:12.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:12 smithi119 ceph-mon[34519]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:14.461 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:14 smithi027 ceph-mon[30779]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:14.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:14 smithi153 ceph-mon[34747]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:14.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:14 smithi119 ceph-mon[34519]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:16.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:16 smithi153 ceph-mon[34747]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:16.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:16 smithi119 ceph-mon[34519]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:16.844 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:16 smithi027 ceph-mon[30779]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:18.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:18 smithi153 ceph-mon[34747]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:18.729 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:18 smithi027 ceph-mon[30779]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:18.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:18 smithi119 ceph-mon[34519]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:19.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:19 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:19.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:19 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:19.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:19 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:19.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:19 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:19.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:19 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:19.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:19 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:19.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:19 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:19.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:19 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:19.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:19 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:20.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:20 smithi153 ceph-mon[34747]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:20.699 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:20 smithi027 ceph-mon[30779]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:20.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:20 smithi119 ceph-mon[34519]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:22 smithi119 ceph-mon[34519]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:23.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:23.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:22 smithi027 ceph-mon[30779]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:22 smithi153 ceph-mon[34747]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:23.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:23.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:24.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.090+0000 7f4fe2000640 1 -- 172.21.15.27:0/1738196949 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4fe4063800 con 0x7f4fc80b64d0 2024-09-21T07:05:24.093 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-09-21T07:05:24.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc80b64d0 msgr2=0x7f4fc80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:24.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc80b64d0 0x7f4fc80b89e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4031440 tx=0x7f4fd4032ed0 comp rx=0 tx=0).stop 2024-09-21T07:05:24.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe4114750 msgr2=0x7f4fe41a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe4114750 0x7f4fe41a62f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009fd0 tx=0x7f4fdc00ef10 comp rx=0 tx=0).stop 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 shutdown_connections 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc80b64d0 0x7f4fc80b89e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe4114750 0x7f4fe41a62f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe4078290 0x7f4fe41a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 --2- 172.21.15.27:0/1738196949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe4077940 0x7f4fe41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 >> 172.21.15.27:0/1738196949 conn(0x7f4fe4101860 msgr2=0x7f4fe41167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 shutdown_connections 2024-09-21T07:05:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:05:24.093+0000 7f4feb800640 1 -- 172.21.15.27:0/1738196949 wait complete. 2024-09-21T07:05:24.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:24 smithi027 ceph-mon[30779]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:24.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:24.823 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:24.823 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:24.859 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T07:05:24.863 INFO:tasks.cephadm:Deploying osd.4 on smithi119 with /dev/vg_nvme/lv_3... 2024-09-21T07:05:24.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T07:05:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:24 smithi119 ceph-mon[34519]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:25.091 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:05:24 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:05:24.982+0000 7f7c1602f640 -1 osd.3 0 log_to_monitors true 2024-09-21T07:05:25.115 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:05:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:24 smithi153 ceph-mon[34747]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:26.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:25 smithi119 ceph-mon[34519]: from='osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T07:05:26.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:25 smithi119 ceph-mon[34519]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T07:05:26.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:25 smithi027 ceph-mon[30779]: from='osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T07:05:26.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:25 smithi027 ceph-mon[30779]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T07:05:26.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:25 smithi153 ceph-mon[34747]: from='osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T07:05:26.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:25 smithi153 ceph-mon[34747]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T07:05:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:26 smithi119 ceph-mon[34519]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:26 smithi119 ceph-mon[34519]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T07:05:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:26 smithi119 ceph-mon[34519]: from='osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:05:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:26 smithi119 ceph-mon[34519]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T07:05:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:26 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:26 smithi119 ceph-mon[34519]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:05:27.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:26 smithi027 ceph-mon[30779]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:27.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:26 smithi027 ceph-mon[30779]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T07:05:27.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:26 smithi027 ceph-mon[30779]: from='osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:05:27.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:26 smithi027 ceph-mon[30779]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T07:05:27.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:26 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:26 smithi027 ceph-mon[30779]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:05:27.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:26 smithi153 ceph-mon[34747]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:27.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:26 smithi153 ceph-mon[34747]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T07:05:27.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:26 smithi153 ceph-mon[34747]: from='osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:05:27.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:26 smithi153 ceph-mon[34747]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T07:05:27.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:26 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:26 smithi153 ceph-mon[34747]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:05:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T07:05:27.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2024-09-21T07:05:27.745 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:05:27.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:05:27.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T07:05:27.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: from='osd.3 ' entity='osd.3' 2024-09-21T07:05:27.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] boot 2024-09-21T07:05:27.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T07:05:27.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:05:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T07:05:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: from='osd.3 ' entity='osd.3' 2024-09-21T07:05:27.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] boot 2024-09-21T07:05:27.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T07:05:27.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:27.842 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:05:27.523+0000 7f7c10800640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T07:05:27.842 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:05:27 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:05:27.545+0000 7f7c06800640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: from='osd.3 ' entity='osd.3' 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: osd.3 [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] boot 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T07:05:28.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T07:05:28.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 -- 172.21.15.119:0/797671759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a68102ae0 msgr2=0x7f1a68102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:28.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/797671759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a68102ae0 0x7f1a68102ef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009900 tx=0x7f1a5802a410 comp rx=0 tx=0).stop 2024-09-21T07:05:28.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 -- 172.21.15.119:0/797671759 shutdown_connections 2024-09-21T07:05:28.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/797671759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a68108d20 0x7f1a6810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:28.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/797671759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a681083d0 0x7f1a681087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:28.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/797671759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a68102ae0 0x7f1a68102ef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:28.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 -- 172.21.15.119:0/797671759 >> 172.21.15.119:0/797671759 conn(0x7f1a68079790 msgr2=0x7f1a68079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:28.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 -- 172.21.15.119:0/797671759 shutdown_connections 2024-09-21T07:05:28.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.179+0000 7f1a6cb66640 1 -- 172.21.15.119:0/797671759 wait complete. 2024-09-21T07:05:28.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 Processor -- start 2024-09-21T07:05:28.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 -- start start 2024-09-21T07:05:28.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a68102ae0 0x7f1a6819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:28.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a681083d0 0x7f1a6819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a68108d20 0x7f1a6819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681129e0 con 0x7f1a68108d20 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68112860 con 0x7f1a681083d0 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.180+0000 7f1a6cb66640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68112b60 con 0x7f1a68102ae0 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a681083d0 0x7f1a6819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a681083d0 0x7f1a6819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33924/0 (socket says 172.21.15.119:33924) 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 -- 172.21.15.119:0/2573239318 learned_addr learned my addr 172.21.15.119:0/2573239318 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a66d9d640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a68108d20 0x7f1a6819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a6659c640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a68102ae0 0x7f1a6819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 -- 172.21.15.119:0/2573239318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a68102ae0 msgr2=0x7f1a6819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a68102ae0 0x7f1a6819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 -- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a68108d20 msgr2=0x7f1a6819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a68108d20 0x7f1a6819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:28.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 -- 172.21.15.119:0/2573239318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a681a64d0 con 0x7f1a681083d0 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a66d9d640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a68108d20 0x7f1a6819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a65d9b640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a681083d0 0x7f1a6819b810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c8c0 tx=0x7f1a5c00cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a577fe640 1 -- 172.21.15.119:0/2573239318 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c007bb0 con 0x7f1a681083d0 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a577fe640 1 -- 172.21.15.119:0/2573239318 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a5c007d50 con 0x7f1a681083d0 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a681a67c0 con 0x7f1a681083d0 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.181+0000 7f1a577fe640 1 -- 172.21.15.119:0/2573239318 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0187c0 con 0x7f1a681083d0 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.182+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a681a6d50 con 0x7f1a681083d0 2024-09-21T07:05:28.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.182+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a68102ef0 con 0x7f1a681083d0 2024-09-21T07:05:28.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.185+0000 7f1a577fe640 1 -- 172.21.15.119:0/2573239318 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f1a5c00fa20 con 0x7f1a681083d0 2024-09-21T07:05:28.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.186+0000 7f1a577fe640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a500b6610 0x7f1a500b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:28.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.186+0000 7f1a577fe640 1 -- 172.21.15.119:0/2573239318 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3917+0+0 (secure 0 0 0) 0x7f1a5c0e65c0 con 0x7f1a681083d0 2024-09-21T07:05:28.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.186+0000 7f1a6659c640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a500b6610 0x7f1a500b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:28.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.187+0000 7f1a6659c640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a500b6610 0x7f1a500b8b20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:28.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.189+0000 7f1a577fe640 1 -- 172.21.15.119:0/2573239318 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a5c014030 con 0x7f1a681083d0 2024-09-21T07:05:28.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:28.379+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1a68063800 con 0x7f1a500b6610 2024-09-21T07:05:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:28 smithi119 ceph-mon[34519]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T07:05:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:05:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:28 smithi027 ceph-mon[30779]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T07:05:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:29.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:29.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:28 smithi153 ceph-mon[34747]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T07:05:29.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:29 smithi119 ceph-mon[34519]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:05:29.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:29 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi027 2024-09-21T07:05:29.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:29 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T07:05:29.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:29 smithi119 ceph-mon[34519]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T07:05:30.078 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:29 smithi153 ceph-mon[34747]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:05:30.079 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:29 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi027 2024-09-21T07:05:30.079 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:29 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T07:05:30.079 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:29 smithi153 ceph-mon[34747]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T07:05:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:29 smithi027 ceph-mon[30779]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:05:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:29 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi027 2024-09-21T07:05:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:29 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T07:05:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:29 smithi027 ceph-mon[30779]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T07:05:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:30 smithi119 ceph-mon[34519]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 472 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:30 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/2581118165' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"} : dispatch 2024-09-21T07:05:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:30 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"} : dispatch 2024-09-21T07:05:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:30 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"}]': finished 2024-09-21T07:05:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:30 smithi119 ceph-mon[34519]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T07:05:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:30 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:30 smithi027 ceph-mon[30779]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 472 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:30 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/2581118165' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"} : dispatch 2024-09-21T07:05:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:30 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"} : dispatch 2024-09-21T07:05:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:30 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"}]': finished 2024-09-21T07:05:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:30 smithi027 ceph-mon[30779]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T07:05:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:30 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:30 smithi153 ceph-mon[34747]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 472 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:30 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/2581118165' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"} : dispatch 2024-09-21T07:05:31.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:30 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"} : dispatch 2024-09-21T07:05:31.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:30 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa23aaaf-55c1-4c48-bf7b-c2c2039624ab"}]': finished 2024-09-21T07:05:31.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:30 smithi153 ceph-mon[34747]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T07:05:31.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:30 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:32.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:31 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/51924855' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:32.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:31 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/51924855' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:32.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:31 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/51924855' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:32 smithi119 ceph-mon[34519]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 472 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:32 smithi027 ceph-mon[30779]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 472 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:32 smithi153 ceph-mon[34747]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 472 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:34 smithi119 ceph-mon[34519]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:34 smithi027 ceph-mon[30779]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:34 smithi153 ceph-mon[34747]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:36 smithi119 ceph-mon[34519]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T07:05:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:36 smithi027 ceph-mon[30779]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T07:05:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:36 smithi153 ceph-mon[34747]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T07:05:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:38 smithi119 ceph-mon[34519]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:05:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:38 smithi027 ceph-mon[30779]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:05:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:38 smithi153 ceph-mon[34747]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:05:40.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T07:05:40.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:40.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T07:05:40.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:40.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T07:05:40.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:40.939 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:40 smithi119 ceph-mon[34519]: Deploying daemon osd.4 on smithi119 2024-09-21T07:05:40.939 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:40 smithi119 ceph-mon[34519]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T07:05:41.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:40 smithi027 ceph-mon[30779]: Deploying daemon osd.4 on smithi119 2024-09-21T07:05:41.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:40 smithi027 ceph-mon[30779]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T07:05:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:40 smithi153 ceph-mon[34747]: Deploying daemon osd.4 on smithi119 2024-09-21T07:05:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:40 smithi153 ceph-mon[34747]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T07:05:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:42 smithi119 ceph-mon[34519]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T07:05:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:42 smithi027 ceph-mon[30779]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T07:05:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:42 smithi153 ceph-mon[34747]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T07:05:44.968 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:44 smithi119 ceph-mon[34519]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:05:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:44 smithi027 ceph-mon[30779]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:05:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:44 smithi153 ceph-mon[34747]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:05:45.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:45.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:45.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:46.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:46.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:46.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:46.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:46 smithi119 ceph-mon[34519]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:05:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:46 smithi027 ceph-mon[30779]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:05:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:46 smithi153 ceph-mon[34747]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:05:49.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:48 smithi119 ceph-mon[34519]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:49.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:49.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:49.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.049+0000 7f1a577fe640 1 -- 172.21.15.119:0/2573239318 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1a68063800 con 0x7f1a500b6610 2024-09-21T07:05:49.051 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 4 on host 'smithi119' 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a500b6610 msgr2=0x7f1a500b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a500b6610 0x7f1a500b8b20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58031400 comp rx=0 tx=0).stop 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a681083d0 msgr2=0x7f1a6819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a681083d0 0x7f1a6819b810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c8c0 tx=0x7f1a5c00cd80 comp rx=0 tx=0).stop 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 shutdown_connections 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a500b6610 0x7f1a500b8b20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a68108d20 0x7f1a6819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a681083d0 0x7f1a6819b810 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 --2- 172.21.15.119:0/2573239318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a68102ae0 0x7f1a6819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 >> 172.21.15.119:0/2573239318 conn(0x7f1a68079790 msgr2=0x7f1a68106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 shutdown_connections 2024-09-21T07:05:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:05:49.052+0000 7f1a6cb66640 1 -- 172.21.15.119:0/2573239318 wait complete. 2024-09-21T07:05:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:48 smithi027 ceph-mon[30779]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:48 smithi153 ceph-mon[34747]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:49.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.697 DEBUG:teuthology.orchestra.run.smithi119:osd.4> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T07:05:49.699 INFO:tasks.cephadm:Deploying osd.5 on smithi153 with /dev/vg_nvme/lv_3... 2024-09-21T07:05:49.699 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T07:05:49.954 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:05:49.985 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:49.985 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:49.985 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:50.090 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:50.090 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:50.091 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:50.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:50.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:50.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:50.982 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:50 smithi119 ceph-mon[34519]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:50 smithi027 ceph-mon[30779]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:51.115 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:50 smithi153 ceph-mon[34747]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:51.284 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:05:50 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:05:50.979+0000 7f2b18fde640 -1 osd.4 0 log_to_monitors true 2024-09-21T07:05:52.017 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:51 smithi119 ceph-mon[34519]: from='osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T07:05:52.017 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:51 smithi119 ceph-mon[34519]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T07:05:52.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:51 smithi027 ceph-mon[30779]: from='osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T07:05:52.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:51 smithi027 ceph-mon[30779]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T07:05:52.167 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:51 smithi153 ceph-mon[34747]: from='osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T07:05:52.167 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:51 smithi153 ceph-mon[34747]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T07:05:52.504 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:05:52.529 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2024-09-21T07:05:52.772 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.016 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:53.017 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:53.017 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.017 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:53.017 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:53.017 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T07:05:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T07:05:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T07:05:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T07:05:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:53.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/690579679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08108d20 msgr2=0x7f4f0810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/690579679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08108d20 0x7f4f0810f550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009ca0 tx=0x7f4f0002a8e0 comp rx=0 tx=0).stop 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/690579679 shutdown_connections 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/690579679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08108d20 0x7f4f0810f550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/690579679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f081083d0 0x7f4f081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/690579679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f08102ae0 0x7f4f08102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/690579679 >> 172.21.15.153:0/690579679 conn(0x7f4f08079790 msgr2=0x7f4f08079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/690579679 shutdown_connections 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/690579679 wait complete. 2024-09-21T07:05:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.255+0000 7f4f0dcdc640 1 Processor -- start 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f0dcdc640 1 -- start start 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f0dcdc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08102ae0 0x7f4f0819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f0dcdc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f081083d0 0x7f4f0819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f0dcdc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f08108d20 0x7f4f0819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f0dcdc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08112a20 con 0x7f4f08108d20 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f0dcdc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081128a0 con 0x7f4f081083d0 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f0dcdc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08112ba0 con 0x7f4f08102ae0 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08102ae0 0x7f4f0819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08102ae0 0x7f4f0819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58992/0 (socket says 172.21.15.153:58992) 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 -- 172.21.15.153:0/719560112 learned_addr learned my addr 172.21.15.153:0/719560112 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f06ffd640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f081083d0 0x7f4f0819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f07fff640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f08108d20 0x7f4f0819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 -- 172.21.15.153:0/719560112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f081083d0 msgr2=0x7f4f0819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:53.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f081083d0 0x7f4f0819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 -- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f08108d20 msgr2=0x7f4f0819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f08108d20 0x7f4f0819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 -- 172.21.15.153:0/719560112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a6560 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f07fff640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f08108d20 0x7f4f0819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.256+0000 7f4f077fe640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08102ae0 0x7f4f0819b2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.257+0000 7f4f04ff9640 1 -- 172.21.15.153:0/719560112 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8017070 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.257+0000 7f4f04ff9640 1 -- 172.21.15.153:0/719560112 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef800de80 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.257+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a6850 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.257+0000 7f4f04ff9640 1 -- 172.21.15.153:0/719560112 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800f620 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.257+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a6e60 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.258+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08102ef0 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.260+0000 7f4f04ff9640 1 -- 172.21.15.153:0/719560112 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4ef8009c30 con 0x7f4f08102ae0 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.261+0000 7f4f04ff9640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ef00b6610 0x7f4ef00b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.261+0000 7f4f04ff9640 1 -- 172.21.15.153:0/719560112 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4036+0+0 (secure 0 0 0) 0x7f4ef80e5e30 con 0x7f4f08102ae0 2024-09-21T07:05:53.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.261+0000 7f4f06ffd640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ef00b6610 0x7f4ef00b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:53.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.262+0000 7f4f06ffd640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ef00b6610 0x7f4ef00b8b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f0806b1d0 tx=0x7f4efc009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:53.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.264+0000 7f4f04ff9640 1 -- 172.21.15.153:0/719560112 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4ef80ad7f0 con 0x7f4f08102ae0 2024-09-21T07:05:53.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:05:53.452+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4f08063800 con 0x7f4ef00b6610 2024-09-21T07:05:53.770 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:05:53.454+0000 7f2b14f60640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T07:05:53.771 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:05:53.460+0000 7f2b0cd56640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:05:54.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi119 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi119 to 8544M 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: from='osd.4 ' entity='osd.4' 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:05:54.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:54.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi119 2024-09-21T07:05:54.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi119 to 8544M 2024-09-21T07:05:54.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:05:54.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T07:05:54.066 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:54.066 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:54.066 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:05:54.066 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: from='osd.4 ' entity='osd.4' 2024-09-21T07:05:54.066 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:05:54.066 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi119 2024-09-21T07:05:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi119 to 8544M 2024-09-21T07:05:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:05:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T07:05:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:05:54.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: from='osd.4 ' entity='osd.4' 2024-09-21T07:05:54.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:05:54.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:55.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:05:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:05:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:05:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] boot 2024-09-21T07:05:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T07:05:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] boot 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T07:05:55.083 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:05:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:05:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T07:05:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:05:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:55.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: osd.4 [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] boot 2024-09-21T07:05:55.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T07:05:55.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T07:05:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:05:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T07:05:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/902142439' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"} : dispatch 2024-09-21T07:05:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"} : dispatch 2024-09-21T07:05:56.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"}]': finished 2024-09-21T07:05:56.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T07:05:56.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:05:56.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:56 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/1317854770' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:05:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T07:05:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/902142439' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"} : dispatch 2024-09-21T07:05:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"} : dispatch 2024-09-21T07:05:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"}]': finished 2024-09-21T07:05:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T07:05:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:05:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:56 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/1317854770' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:05:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T07:05:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/902142439' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"} : dispatch 2024-09-21T07:05:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"} : dispatch 2024-09-21T07:05:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ab0114f-b98e-4b90-b112-aa1dec91830c"}]': finished 2024-09-21T07:05:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T07:05:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:05:57.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:56 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/1317854770' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:05:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:05:58 smithi027 ceph-mon[30779]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:05:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:05:58 smithi153 ceph-mon[34747]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:05:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:05:58 smithi119 ceph-mon[34519]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:00 smithi027 ceph-mon[30779]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:01.158 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:00 smithi153 ceph-mon[34747]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:00 smithi119 ceph-mon[34519]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:02 smithi027 ceph-mon[30779]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:02 smithi153 ceph-mon[34747]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:02 smithi119 ceph-mon[34519]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:05.051 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:04 smithi153 ceph-mon[34747]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:05.051 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T07:06:05.051 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:05.085 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:04 smithi119 ceph-mon[34519]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:05.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T07:06:05.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:04 smithi027 ceph-mon[30779]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T07:06:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:06.071 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:05 smithi153 ceph-mon[34747]: Deploying daemon osd.5 on smithi153 2024-09-21T07:06:06.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:05 smithi027 ceph-mon[30779]: Deploying daemon osd.5 on smithi153 2024-09-21T07:06:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:05 smithi119 ceph-mon[34519]: Deploying daemon osd.5 on smithi153 2024-09-21T07:06:07.074 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:06 smithi153 ceph-mon[34747]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:06 smithi027 ceph-mon[30779]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:06 smithi119 ceph-mon[34519]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:08 smithi027 ceph-mon[30779]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:08 smithi153 ceph-mon[34747]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:08 smithi119 ceph-mon[34519]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:10 smithi027 ceph-mon[30779]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:10 smithi153 ceph-mon[34747]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:10 smithi119 ceph-mon[34519]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:11.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:11.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:11.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:12.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:12.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:12.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:12 smithi153 ceph-mon[34747]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:12 smithi119 ceph-mon[34519]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:12 smithi027 ceph-mon[30779]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:14.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.497+0000 7f4f04ff9640 1 -- 172.21.15.153:0/719560112 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f08063800 con 0x7f4ef00b6610 2024-09-21T07:06:14.499 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 5 on host 'smithi153' 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ef00b6610 msgr2=0x7f4ef00b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ef00b6610 0x7f4ef00b8b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f0806b1d0 tx=0x7f4efc009110 comp rx=0 tx=0).stop 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08102ae0 msgr2=0x7f4f0819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08102ae0 0x7f4f0819b2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800dd50 comp rx=0 tx=0).stop 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 shutdown_connections 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ef00b6610 0x7f4ef00b8b20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f08108d20 0x7f4f0819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:14.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f081083d0 0x7f4f0819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:14.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 --2- 172.21.15.153:0/719560112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f08102ae0 0x7f4f0819b2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:14.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.499+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 >> 172.21.15.153:0/719560112 conn(0x7f4f08079790 msgr2=0x7f4f08106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:14.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.500+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 shutdown_connections 2024-09-21T07:06:14.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:06:14.500+0000 7f4f0dcdc640 1 -- 172.21.15.153:0/719560112 wait complete. 2024-09-21T07:06:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:14.893 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:15.195 DEBUG:teuthology.orchestra.run.smithi153:osd.5> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T07:06:15.198 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2024-09-21T07:06:15.198 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T07:06:15.588 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:16.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:16 smithi119 ceph-mon[34519]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:16.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:16 smithi027 ceph-mon[30779]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:16.870 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:16 smithi153 ceph-mon[34747]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:17.178 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:06:16 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:06:16.949+0000 7f2f682ef640 -1 osd.5 0 log_to_monitors true 2024-09-21T07:06:17.673 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:17 smithi153 ceph-mon[34747]: from='osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T07:06:17.673 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:17 smithi153 ceph-mon[34747]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T07:06:17.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:17 smithi027 ceph-mon[30779]: from='osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T07:06:17.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:17 smithi027 ceph-mon[30779]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T07:06:18.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:17 smithi119 ceph-mon[34519]: from='osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T07:06:18.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:17 smithi119 ceph-mon[34519]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T07:06:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T07:06:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:06:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi153 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi153 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.929 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:18.929 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:18.929 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:18.951 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:06:18.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-09-21T07:06:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi153 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:19.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:19.606 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:06:19.327+0000 7f2f64271640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T07:06:19.606 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:06:19.334+0000 7f2f5c067640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:06:19.765 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:19 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-21T07:06:19.765 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:19 smithi027 ceph-mon[30779]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:06:19.765 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:19 smithi027 ceph-mon[30779]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T07:06:19.765 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:19 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:19.765 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:19 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:19.765 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:19 smithi027 ceph-mon[30779]: from='osd.5 ' entity='osd.5' 2024-09-21T07:06:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-21T07:06:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-mon[34747]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:06:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-mon[34747]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T07:06:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:19.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:19 smithi153 ceph-mon[34747]: from='osd.5 ' entity='osd.5' 2024-09-21T07:06:19.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.950+0000 7fb380200640 1 -- 172.21.15.27:0/3475810311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb378103b80 msgr2=0x7fb378103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.950+0000 7fb380200640 1 --2- 172.21.15.27:0/3475810311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb378103b80 0x7fb378103f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009900 tx=0x7fb36c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.950+0000 7fb380200640 1 -- 172.21.15.27:0/3475810311 shutdown_connections 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.950+0000 7fb380200640 1 --2- 172.21.15.27:0/3475810311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378109e10 0x7fb378110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.950+0000 7fb380200640 1 --2- 172.21.15.27:0/3475810311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3781094c0 0x7fb3781098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.950+0000 7fb380200640 1 --2- 172.21.15.27:0/3475810311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb378103b80 0x7fb378103f90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.950+0000 7fb380200640 1 -- 172.21.15.27:0/3475810311 >> 172.21.15.27:0/3475810311 conn(0x7fb3780ff610 msgr2=0x7fb378101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.951+0000 7fb380200640 1 -- 172.21.15.27:0/3475810311 shutdown_connections 2024-09-21T07:06:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.951+0000 7fb380200640 1 -- 172.21.15.27:0/3475810311 wait complete. 2024-09-21T07:06:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.952+0000 7fb380200640 1 Processor -- start 2024-09-21T07:06:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.952+0000 7fb380200640 1 -- start start 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb380200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378103b80 0x7fb37819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb380200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3781094c0 0x7fb37819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb380200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb378109e10 0x7fb37819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb380200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb378113b20 con 0x7fb3781094c0 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb380200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781139a0 con 0x7fb378109e10 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb380200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb378113ca0 con 0x7fb378103b80 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb37d800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378103b80 0x7fb37819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb37ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3781094c0 0x7fb37819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb37d800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378103b80 0x7fb37819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:44208/0 (socket says 172.21.15.27:44208) 2024-09-21T07:06:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb37d800640 1 -- 172.21.15.27:0/903581668 learned_addr learned my addr 172.21.15.27:0/903581668 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:06:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.953+0000 7fb37e200640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb378109e10 0x7fb37819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb37d800640 1 -- 172.21.15.27:0/903581668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb378109e10 msgr2=0x7fb37819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb37d800640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb378109e10 0x7fb37819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb37d800640 1 -- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3781094c0 msgr2=0x7fb37819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb37d800640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3781094c0 0x7fb37819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb37d800640 1 -- 172.21.15.27:0/903581668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3781a04a0 con 0x7fb378103b80 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb37ce00640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3781094c0 0x7fb37819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb37d800640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378103b80 0x7fb37819a530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb36c005ab0 tx=0x7fb36c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb376000640 1 -- 172.21.15.27:0/903581668 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c03f070 con 0x7fb378103b80 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb376000640 1 -- 172.21.15.27:0/903581668 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36c031b60 con 0x7fb378103b80 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781a0730 con 0x7fb378103b80 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.954+0000 7fb376000640 1 -- 172.21.15.27:0/903581668 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c035030 con 0x7fb378103b80 2024-09-21T07:06:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.955+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a6ef0 con 0x7fb378103b80 2024-09-21T07:06:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.956+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378104e00 con 0x7fb378103b80 2024-09-21T07:06:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.958+0000 7fb376000640 1 -- 172.21.15.27:0/903581668 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fb36c042070 con 0x7fb378103b80 2024-09-21T07:06:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.959+0000 7fb376000640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb3600b6560 0x7fb3600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.960+0000 7fb376000640 1 -- 172.21.15.27:0/903581668 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4781+0+0 (secure 0 0 0) 0x7fb36c1070e0 con 0x7fb378103b80 2024-09-21T07:06:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.960+0000 7fb37ce00640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb3600b6560 0x7fb3600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.960+0000 7fb37ce00640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb3600b6560 0x7fb3600b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb378102d10 tx=0x7fb364005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:19.961+0000 7fb376000640 1 -- 172.21.15.27:0/903581668 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb36c0d2a80 con 0x7fb378103b80 2024-09-21T07:06:20.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:19 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-21T07:06:20.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:19 smithi119 ceph-mon[34519]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:06:20.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:19 smithi119 ceph-mon[34519]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T07:06:20.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:19 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:20.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:19 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:20.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:19 smithi119 ceph-mon[34519]: from='osd.5 ' entity='osd.5' 2024-09-21T07:06:20.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:20.171+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb378063800 con 0x7fb3600b6560 2024-09-21T07:06:20.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:06:20.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:06:20.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:20.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:20.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] boot 2024-09-21T07:06:20.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T07:06:20.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:20.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:06:20.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:06:20.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:20 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:20.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:06:20.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:06:20.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:20.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:20.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] boot 2024-09-21T07:06:20.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T07:06:20.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:20.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:06:20.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:06:20.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:20 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:06:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:06:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T07:06:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: osd.5 [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] boot 2024-09-21T07:06:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T07:06:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T07:06:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:06:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:06:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:20 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:21.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:21 smithi153 ceph-mon[34747]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:06:21.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:21 smithi153 ceph-mon[34747]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T07:06:21.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:21 smithi027 ceph-mon[30779]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:06:21.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:21 smithi027 ceph-mon[30779]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T07:06:22.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:21 smithi119 ceph-mon[34519]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:06:22.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:21 smithi119 ceph-mon[34519]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T07:06:22.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:22 smithi153 ceph-mon[34747]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:22.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:22 smithi153 ceph-mon[34747]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T07:06:22.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:22 smithi027 ceph-mon[30779]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:22.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:22 smithi027 ceph-mon[30779]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T07:06:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:22 smithi119 ceph-mon[34519]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:22 smithi119 ceph-mon[34519]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T07:06:23.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:23 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2314147411' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d7bb5b9d-337b-41e4-ae7a-93db10aa151e"} : dispatch 2024-09-21T07:06:23.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:23 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2314147411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7bb5b9d-337b-41e4-ae7a-93db10aa151e"}]': finished 2024-09-21T07:06:23.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:23 smithi027 ceph-mon[30779]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T07:06:23.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:23 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:23.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:23 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2314147411' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d7bb5b9d-337b-41e4-ae7a-93db10aa151e"} : dispatch 2024-09-21T07:06:23.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:23 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2314147411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7bb5b9d-337b-41e4-ae7a-93db10aa151e"}]': finished 2024-09-21T07:06:23.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:23 smithi153 ceph-mon[34747]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T07:06:23.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:23 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:24.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:23 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2314147411' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d7bb5b9d-337b-41e4-ae7a-93db10aa151e"} : dispatch 2024-09-21T07:06:24.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:23 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2314147411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7bb5b9d-337b-41e4-ae7a-93db10aa151e"}]': finished 2024-09-21T07:06:24.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:23 smithi119 ceph-mon[34519]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T07:06:24.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:23 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:24.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:24 smithi153 ceph-mon[34747]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:24.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:24 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3963526608' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:06:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:24 smithi119 ceph-mon[34519]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:24 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3963526608' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:06:25.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:24 smithi027 ceph-mon[30779]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:25.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:24 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3963526608' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:06:26.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:26 smithi153 ceph-mon[34747]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T07:06:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:26 smithi119 ceph-mon[34519]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T07:06:27.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:26 smithi027 ceph-mon[30779]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T07:06:28.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:28 smithi027 ceph-mon[30779]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T07:06:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:28 smithi153 ceph-mon[34747]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T07:06:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:28 smithi119 ceph-mon[34519]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T07:06:30.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:30 smithi153 ceph-mon[34747]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:06:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:30 smithi119 ceph-mon[34519]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:06:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:30 smithi027 ceph-mon[30779]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:06:32.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:32 smithi153 ceph-mon[34747]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T07:06:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:32 smithi119 ceph-mon[34519]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T07:06:33.064 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:32 smithi027 ceph-mon[30779]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T07:06:33.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T07:06:33.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:33.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T07:06:33.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:34.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T07:06:34.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:34 smithi153 ceph-mon[34747]: Deploying daemon osd.6 on smithi027 2024-09-21T07:06:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:34 smithi153 ceph-mon[34747]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:06:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:34 smithi119 ceph-mon[34519]: Deploying daemon osd.6 on smithi027 2024-09-21T07:06:35.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:34 smithi119 ceph-mon[34519]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:06:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:34 smithi027 ceph-mon[30779]: Deploying daemon osd.6 on smithi027 2024-09-21T07:06:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:34 smithi027 ceph-mon[30779]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:06:36.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:36 smithi027 ceph-mon[30779]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:06:36.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:36 smithi153 ceph-mon[34747]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:06:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:36 smithi119 ceph-mon[34519]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:06:38.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:38 smithi153 ceph-mon[34747]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:38 smithi119 ceph-mon[34519]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:38 smithi027 ceph-mon[30779]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:40.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:40 smithi153 ceph-mon[34747]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:40 smithi119 ceph-mon[34519]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:41.084 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:40 smithi027 ceph-mon[30779]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:41.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:41.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:41.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:42.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:42.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:42.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:42.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:42 smithi153 ceph-mon[34747]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:42 smithi119 ceph-mon[34519]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:42 smithi027 ceph-mon[30779]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:44.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:44 smithi153 ceph-mon[34747]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:44.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:44.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:44 smithi119 ceph-mon[34519]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:44 smithi027 ceph-mon[30779]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:45.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:45.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:45.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:46.023 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:46.023 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:46.023 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:46.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.544+0000 7fb376000640 1 -- 172.21.15.27:0/903581668 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb378063800 con 0x7fb3600b6560 2024-09-21T07:06:46.546 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb3600b6560 msgr2=0x7fb3600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb3600b6560 0x7fb3600b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb378102d10 tx=0x7fb364005f90 comp rx=0 tx=0).stop 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378103b80 msgr2=0x7fb37819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378103b80 0x7fb37819a530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb36c005ab0 tx=0x7fb36c003c10 comp rx=0 tx=0).stop 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 shutdown_connections 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb3600b6560 0x7fb3600b8a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb378109e10 0x7fb37819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3781094c0 0x7fb37819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 --2- 172.21.15.27:0/903581668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb378103b80 0x7fb37819a530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 >> 172.21.15.27:0/903581668 conn(0x7fb3780ff610 msgr2=0x7fb378108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 shutdown_connections 2024-09-21T07:06:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:06:46.547+0000 7fb380200640 1 -- 172.21.15.27:0/903581668 wait complete. 2024-09-21T07:06:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:46 smithi027 ceph-mon[30779]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:46 smithi153 ceph-mon[34747]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:46 smithi119 ceph-mon[34519]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:47.304 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T07:06:47.307 INFO:tasks.cephadm:Deploying osd.7 on smithi119 with /dev/vg_nvme/lv_2... 2024-09-21T07:06:47.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T07:06:47.562 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:06:47.984 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:47 smithi119 ceph-mon[34519]: from='osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T07:06:47.984 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:47 smithi119 ceph-mon[34519]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T07:06:48.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:47 smithi027 ceph-mon[30779]: from='osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T07:06:48.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:47 smithi027 ceph-mon[30779]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T07:06:48.091 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:06:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:06:47.790+0000 7f4fc242e640 -1 osd.6 0 log_to_monitors true 2024-09-21T07:06:48.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:47 smithi153 ceph-mon[34747]: from='osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T07:06:48.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:47 smithi153 ceph-mon[34747]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T07:06:49.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-mon[30779]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:49.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-mon[30779]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T07:06:49.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-mon[30779]: from='osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:06:49.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-mon[30779]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T07:06:49.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:49.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-mon[30779]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:06:49.120 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:06:48.876+0000 7f4fbcc00640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T07:06:49.120 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:06:48 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:06:48.887+0000 7f4fb7c00640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:06:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:48 smithi153 ceph-mon[34747]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:48 smithi153 ceph-mon[34747]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T07:06:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:48 smithi153 ceph-mon[34747]: from='osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:06:49.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:48 smithi153 ceph-mon[34747]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T07:06:49.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:49.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:48 smithi153 ceph-mon[34747]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:06:49.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:48 smithi119 ceph-mon[34519]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:49.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:48 smithi119 ceph-mon[34519]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T07:06:49.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:48 smithi119 ceph-mon[34519]: from='osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:06:49.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:48 smithi119 ceph-mon[34519]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T07:06:49.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:49.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:48 smithi119 ceph-mon[34519]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:06:49.982 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T07:06:50.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2024-09-21T07:06:50.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:49 smithi153 ceph-mon[34747]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:06:50.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:49 smithi153 ceph-mon[34747]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T07:06:50.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:50.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:50.242 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:49 smithi027 ceph-mon[30779]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:06:50.242 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:49 smithi027 ceph-mon[30779]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T07:06:50.242 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:50.242 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:50.257 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:06:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:49 smithi119 ceph-mon[34519]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:06:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:49 smithi119 ceph-mon[34519]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T07:06:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 -- 172.21.15.119:0/15098224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 msgr2=0x7f91641098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 --2- 172.21.15.119:0/15098224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f91641098b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9158009900 tx=0x7f915802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 -- 172.21.15.119:0/15098224 shutdown_connections 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 --2- 172.21.15.119:0/15098224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9164109df0 0x7f9164110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 --2- 172.21.15.119:0/15098224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f91641098b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 --2- 172.21.15.119:0/15098224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9164103b60 0x7f9164103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 -- 172.21.15.119:0/15098224 >> 172.21.15.119:0/15098224 conn(0x7f916407b850 msgr2=0x7f916407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 -- 172.21.15.119:0/15098224 shutdown_connections 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.723+0000 7f9168dae640 1 -- 172.21.15.119:0/15098224 wait complete. 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.724+0000 7f9168dae640 1 Processor -- start 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.724+0000 7f9168dae640 1 -- start start 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.724+0000 7f9168dae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9164103b60 0x7f9164075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.724+0000 7f9168dae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f9164075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.724+0000 7f9168dae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9164109df0 0x7f9164076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.724+0000 7f9168dae640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9164113ab0 con 0x7f9164109df0 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.724+0000 7f9168dae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9164113930 con 0x7f91641094a0 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9168dae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9164113c30 con 0x7f9164103b60 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f9164075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f9164075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46848/0 (socket says 172.21.15.119:46848) 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f916259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9164103b60 0x7f9164075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 -- 172.21.15.119:0/3504835422 learned_addr learned my addr 172.21.15.119:0/3504835422 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9162d9d640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9164109df0 0x7f9164076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 -- 172.21.15.119:0/3504835422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9164103b60 msgr2=0x7f9164075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9164103b60 0x7f9164075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 -- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9164109df0 msgr2=0x7f9164076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9164109df0 0x7f9164076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 -- 172.21.15.119:0/3504835422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91641a8670 con 0x7f91641094a0 2024-09-21T07:06:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9161d9b640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f9164075c50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f915802a910 tx=0x7f9158004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f91537fe640 1 -- 172.21.15.119:0/3504835422 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915803f030 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f91537fe640 1 -- 172.21.15.119:0/3504835422 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9158003aa0 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91641a8900 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.725+0000 7f91537fe640 1 -- 172.21.15.119:0/3504835422 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158035070 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.726+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91641a8e30 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.726+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9164103f70 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.729+0000 7f91537fe640 1 -- 172.21.15.119:0/3504835422 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f9158042030 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.729+0000 7f91537fe640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f914c0b65e0 0x7f914c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.730+0000 7f91537fe640 1 -- 172.21.15.119:0/3504835422 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5180+0+0 (secure 0 0 0) 0x7f91581071f0 con 0x7f91641094a0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.730+0000 7f916259c640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f914c0b65e0 0x7f914c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.730+0000 7f916259c640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f914c0b65e0 0x7f914c0b8af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9164077160 tx=0x7f915400a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.733+0000 7f91537fe640 1 -- 172.21.15.119:0/3504835422 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91580ce6c0 con 0x7f91641094a0 2024-09-21T07:06:50.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:06:50.921+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9164063800 con 0x7f914c0b65e0 2024-09-21T07:06:50.924 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:50 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:06:50.924 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:50 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:06:50.924 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:50 smithi119 ceph-mon[34519]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:50.924 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:50 smithi119 ceph-mon[34519]: osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] boot 2024-09-21T07:06:50.924 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:50 smithi119 ceph-mon[34519]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T07:06:50.924 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:50.924 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:50 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:06:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:50 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:06:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:50 smithi027 ceph-mon[30779]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:50 smithi027 ceph-mon[30779]: osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] boot 2024-09-21T07:06:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:50 smithi027 ceph-mon[30779]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T07:06:51.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:51.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:50 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:06:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:50 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:06:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:50 smithi153 ceph-mon[34747]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-21T07:06:51.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:50 smithi153 ceph-mon[34747]: osd.6 [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] boot 2024-09-21T07:06:51.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:50 smithi153 ceph-mon[34747]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T07:06:51.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:51.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T07:06:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T07:06:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:06:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:06:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:52.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.231 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T07:06:52.231 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:06:52.231 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:52.232 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T07:06:52.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:06:52.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:06:52.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:52.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:06:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:52 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi027 2024-09-21T07:06:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:52 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T07:06:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:52 smithi153 ceph-mon[34747]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:52 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi027 2024-09-21T07:06:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:52 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T07:06:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:52 smithi119 ceph-mon[34519]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:52 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi027 2024-09-21T07:06:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:52 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T07:06:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:52 smithi027 ceph-mon[30779]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:54.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:53 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/1914836524' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"} : dispatch 2024-09-21T07:06:54.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:53 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"} : dispatch 2024-09-21T07:06:54.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:53 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"}]': finished 2024-09-21T07:06:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:53 smithi153 ceph-mon[34747]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T07:06:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:06:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:53 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/1879464626' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:06:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:53 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/1914836524' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"} : dispatch 2024-09-21T07:06:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:53 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"} : dispatch 2024-09-21T07:06:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:53 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"}]': finished 2024-09-21T07:06:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:53 smithi119 ceph-mon[34519]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T07:06:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:06:54.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:53 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/1879464626' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:06:54.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:53 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/1914836524' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"} : dispatch 2024-09-21T07:06:54.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:53 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"} : dispatch 2024-09-21T07:06:54.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:53 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cb69c8e-353c-49e9-87f9-802de8728313"}]': finished 2024-09-21T07:06:54.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:53 smithi027 ceph-mon[30779]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T07:06:54.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:06:54.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:53 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/1879464626' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:06:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:54 smithi153 ceph-mon[34747]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:54 smithi119 ceph-mon[34519]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:54 smithi027 ceph-mon[30779]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:56 smithi153 ceph-mon[34747]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:56 smithi119 ceph-mon[34519]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:57.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:56 smithi027 ceph-mon[30779]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:06:58 smithi027 ceph-mon[30779]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:06:58 smithi153 ceph-mon[34747]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:06:59.177 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:06:58 smithi119 ceph-mon[34519]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:00 smithi119 ceph-mon[34519]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:00 smithi153 ceph-mon[34747]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:00 smithi027 ceph-mon[30779]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:02.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T07:07:02.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:02.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T07:07:02.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:02.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T07:07:02.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:03.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:02 smithi119 ceph-mon[34519]: Deploying daemon osd.7 on smithi119 2024-09-21T07:07:03.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:02 smithi119 ceph-mon[34519]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:02 smithi153 ceph-mon[34747]: Deploying daemon osd.7 on smithi119 2024-09-21T07:07:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:02 smithi153 ceph-mon[34747]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:02 smithi027 ceph-mon[30779]: Deploying daemon osd.7 on smithi119 2024-09-21T07:07:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:02 smithi027 ceph-mon[30779]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:04 smithi119 ceph-mon[34519]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:05.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:04 smithi027 ceph-mon[30779]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:04 smithi153 ceph-mon[34747]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:06 smithi027 ceph-mon[30779]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:06 smithi153 ceph-mon[34747]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:06 smithi119 ceph-mon[34519]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:08.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:08.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:08.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:08.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:08.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:08.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:09 smithi119 ceph-mon[34519]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:09 smithi027 ceph-mon[30779]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:09 smithi153 ceph-mon[34747]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:10.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.767+0000 7f91537fe640 1 -- 172.21.15.119:0/3504835422 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9164063800 con 0x7f914c0b65e0 2024-09-21T07:07:10.769 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 7 on host 'smithi119' 2024-09-21T07:07:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.769+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f914c0b65e0 msgr2=0x7f914c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.769+0000 7f9168dae640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f914c0b65e0 0x7f914c0b8af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9164077160 tx=0x7f915400a2d0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 msgr2=0x7f9164075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f9164075c50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f915802a910 tx=0x7f9158004270 comp rx=0 tx=0).stop 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 shutdown_connections 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f914c0b65e0 0x7f914c0b8af0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9164109df0 0x7f9164076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91641094a0 0x7f9164075c50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 --2- 172.21.15.119:0/3504835422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9164103b60 0x7f9164075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 >> 172.21.15.119:0/3504835422 conn(0x7f916407b850 msgr2=0x7f9164107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 shutdown_connections 2024-09-21T07:07:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:07:10.770+0000 7f9168dae640 1 -- 172.21.15.119:0/3504835422 wait complete. 2024-09-21T07:07:10.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:10 smithi153 ceph-mon[34747]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:10.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:10.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:10.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:10.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:10.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:10 smithi119 ceph-mon[34519]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:11.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:10 smithi027 ceph-mon[30779]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:11.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:11.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.487 DEBUG:teuthology.orchestra.run.smithi119:osd.7> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T07:07:11.489 INFO:tasks.cephadm:Deploying osd.8 on smithi153 with /dev/vg_nvme/lv_2... 2024-09-21T07:07:11.489 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T07:07:11.747 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:07:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.872 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:11.873 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:11.873 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:12.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:12.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:12.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:12.914 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:12 smithi119 ceph-mon[34519]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:12.972 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:12 smithi153 ceph-mon[34747]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:12 smithi027 ceph-mon[30779]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:13.284 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:07:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:07:13.060+0000 7efc7154a640 -1 osd.7 0 log_to_monitors true 2024-09-21T07:07:13.881 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:13 smithi153 ceph-mon[34747]: from='osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T07:07:13.881 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:13 smithi153 ceph-mon[34747]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T07:07:13.915 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:13 smithi119 ceph-mon[34519]: from='osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T07:07:13.915 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:13 smithi119 ceph-mon[34519]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T07:07:14.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:13 smithi027 ceph-mon[30779]: from='osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T07:07:14.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:13 smithi027 ceph-mon[30779]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T07:07:14.095 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:07:14.122 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2024-09-21T07:07:14.374 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:07:14.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 -- 172.21.15.153:0/491565003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 msgr2=0x7f379c109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:14.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 --2- 172.21.15.153:0/491565003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c109980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3790009900 tx=0x7f379002a410 comp rx=0 tx=0).stop 2024-09-21T07:07:14.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 -- 172.21.15.153:0/491565003 shutdown_connections 2024-09-21T07:07:14.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 --2- 172.21.15.153:0/491565003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c109980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:14.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 --2- 172.21.15.153:0/491565003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f379c109ec0 0x7f379c1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 --2- 172.21.15.153:0/491565003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f379c103c30 0x7f379c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 -- 172.21.15.153:0/491565003 >> 172.21.15.153:0/491565003 conn(0x7f379c07b8d0 msgr2=0x7f379c07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 -- 172.21.15.153:0/491565003 shutdown_connections 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.859+0000 7f37a3548640 1 -- 172.21.15.153:0/491565003 wait complete. 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 Processor -- start 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 -- start start 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f379c103c30 0x7f379c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f379c109ec0 0x7f379c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f379c113ae0 con 0x7f379c109ec0 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f379c113960 con 0x7f379c103c30 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a3548640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f379c113c60 con 0x7f379c109570 2024-09-21T07:07:14.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a0ae3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a0ae3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50628/0 (socket says 172.21.15.153:50628) 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a0ae3640 1 -- 172.21.15.153:0/2062457965 learned_addr learned my addr 172.21.15.153:0/2062457965 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.860+0000 7f37a12e4640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f379c103c30 0x7f379c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a0ae3640 1 -- 172.21.15.153:0/2062457965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f379c103c30 msgr2=0x7f379c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a0ae3640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f379c103c30 0x7f379c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a1ae5640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f379c109ec0 0x7f379c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a0ae3640 1 -- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f379c109ec0 msgr2=0x7f379c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a0ae3640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f379c109ec0 0x7f379c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a0ae3640 1 -- 172.21.15.153:0/2062457965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1a8680 con 0x7f379c109570 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a0ae3640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c075c50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f379002a930 tx=0x7f3790004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f379a7fc640 1 -- 172.21.15.153:0/2062457965 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379003f030 con 0x7f379c109570 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f379a7fc640 1 -- 172.21.15.153:0/2062457965 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3790003aa0 con 0x7f379c109570 2024-09-21T07:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1a8910 con 0x7f379c109570 2024-09-21T07:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f379a7fc640 1 -- 172.21.15.153:0/2062457965 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790035070 con 0x7f379c109570 2024-09-21T07:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.861+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c1a8df0 con 0x7f379c109570 2024-09-21T07:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.863+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c104040 con 0x7f379c109570 2024-09-21T07:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.864+0000 7f379a7fc640 1 -- 172.21.15.153:0/2062457965 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3790042030 con 0x7f379c109570 2024-09-21T07:07:14.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.865+0000 7f379a7fc640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37840b6560 0x7f37840b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:14.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.865+0000 7f379a7fc640 1 -- 172.21.15.153:0/2062457965 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5332+0+0 (secure 0 0 0) 0x7f3790106ef0 con 0x7f379c109570 2024-09-21T07:07:14.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.865+0000 7f37a12e4640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37840b6560 0x7f37840b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:14.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.866+0000 7f37a12e4640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37840b6560 0x7f37840b8a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f379c077160 tx=0x7f37940075b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:14.869+0000 7f379a7fc640 1 -- 172.21.15.153:0/2062457965 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37900ce320 con 0x7f379c109570 2024-09-21T07:07:15.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:15.062+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f379c063800 con 0x7f37840b6560 2024-09-21T07:07:15.064 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:15.064 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T07:07:15.064 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:07:15.064 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T07:07:15.064 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:15.065 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T07:07:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:07:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T07:07:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.285 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:07:14.791+0000 7efc6d4cc640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T07:07:15.285 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:07:14.802+0000 7efc692ca640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:16.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi119 2024-09-21T07:07:16.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi119 to 5696M 2024-09-21T07:07:16.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:07:16.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T07:07:16.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:07:16.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:07:16.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:16.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] boot 2024-09-21T07:07:16.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T07:07:16.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi119 2024-09-21T07:07:16.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi119 to 5696M 2024-09-21T07:07:16.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:07:16.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] boot 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T07:07:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi119 2024-09-21T07:07:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi119 to 5696M 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: osd.7 [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] boot 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T07:07:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T07:07:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:16 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:07:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:16 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:07:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:16 smithi027 ceph-mon[30779]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:07:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:16 smithi027 ceph-mon[30779]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:17.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:16 smithi027 ceph-mon[30779]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T07:07:17.176 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:16 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:07:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:16 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:07:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:16 smithi153 ceph-mon[34747]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:07:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:16 smithi153 ceph-mon[34747]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:16 smithi153 ceph-mon[34747]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T07:07:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:16 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:07:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:16 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:07:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:16 smithi119 ceph-mon[34519]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:07:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:16 smithi119 ceph-mon[34519]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T07:07:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:16 smithi119 ceph-mon[34519]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T07:07:18.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:17 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/3785662152' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"} : dispatch 2024-09-21T07:07:18.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:17 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"} : dispatch 2024-09-21T07:07:18.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:17 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"}]': finished 2024-09-21T07:07:18.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:17 smithi027 ceph-mon[30779]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T07:07:18.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:17 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:18.099 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:17 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/3785662152' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"} : dispatch 2024-09-21T07:07:18.099 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:17 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"} : dispatch 2024-09-21T07:07:18.099 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:17 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"}]': finished 2024-09-21T07:07:18.099 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:17 smithi153 ceph-mon[34747]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T07:07:18.099 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:17 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:17 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/3785662152' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"} : dispatch 2024-09-21T07:07:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:17 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"} : dispatch 2024-09-21T07:07:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:17 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13844110-df7e-451b-b574-38419af2df5d"}]': finished 2024-09-21T07:07:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:17 smithi119 ceph-mon[34519]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T07:07:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:17 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:18 smithi027 ceph-mon[30779]: pgmap v167: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:18 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/201552403' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:07:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:18 smithi153 ceph-mon[34747]: pgmap v167: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:18 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/201552403' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:07:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:18 smithi119 ceph-mon[34519]: pgmap v167: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:18 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/201552403' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:07:20.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:19 smithi153 ceph-mon[34747]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-21T07:07:20.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:19 smithi119 ceph-mon[34519]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-21T07:07:20.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:19 smithi027 ceph-mon[30779]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-21T07:07:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:20 smithi153 ceph-mon[34747]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:20 smithi119 ceph-mon[34519]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:21.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:20 smithi027 ceph-mon[30779]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:22.168 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:21 smithi153 ceph-mon[34747]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-21T07:07:22.168 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:21 smithi153 ceph-mon[34747]: Cluster is now healthy 2024-09-21T07:07:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:21 smithi119 ceph-mon[34519]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-21T07:07:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:21 smithi119 ceph-mon[34519]: Cluster is now healthy 2024-09-21T07:07:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:21 smithi027 ceph-mon[30779]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-21T07:07:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:21 smithi027 ceph-mon[30779]: Cluster is now healthy 2024-09-21T07:07:23.105 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:22 smithi153 ceph-mon[34747]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:22 smithi119 ceph-mon[34519]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:22 smithi027 ceph-mon[30779]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:25.055 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:24 smithi153 ceph-mon[34747]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-09-21T07:07:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:24 smithi119 ceph-mon[34519]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-09-21T07:07:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:24 smithi027 ceph-mon[30779]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-09-21T07:07:26.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:25 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T07:07:26.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:25 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:25 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T07:07:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:25 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:26.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:25 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T07:07:26.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:25 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:26 smithi153 ceph-mon[34747]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-21T07:07:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:26 smithi153 ceph-mon[34747]: Deploying daemon osd.8 on smithi153 2024-09-21T07:07:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:26 smithi119 ceph-mon[34519]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-21T07:07:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:26 smithi119 ceph-mon[34519]: Deploying daemon osd.8 on smithi153 2024-09-21T07:07:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:26 smithi027 ceph-mon[30779]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-21T07:07:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:26 smithi027 ceph-mon[30779]: Deploying daemon osd.8 on smithi153 2024-09-21T07:07:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:28 smithi153 ceph-mon[34747]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T07:07:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:28 smithi119 ceph-mon[34519]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T07:07:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:28 smithi027 ceph-mon[30779]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T07:07:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:30 smithi119 ceph-mon[34519]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T07:07:31.326 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:30 smithi153 ceph-mon[34747]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T07:07:31.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:30 smithi027 ceph-mon[30779]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T07:07:32.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:31 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:32.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:31 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:32.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:31 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:31 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:31 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:31 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:32.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:31 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:32.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:31 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:32.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:31 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:33.236 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:32 smithi153 ceph-mon[34747]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:32 smithi119 ceph-mon[34519]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:32 smithi027 ceph-mon[30779]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:35.215 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:34 smithi153 ceph-mon[34747]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:35.215 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.215 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.216 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:35.216 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:35.216 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:34 smithi119 ceph-mon[34519]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:34 smithi027 ceph-mon[30779]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.963+0000 7f379a7fc640 1 -- 172.21.15.153:0/2062457965 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f379c063800 con 0x7f37840b6560 2024-09-21T07:07:35.965 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 8 on host 'smithi153' 2024-09-21T07:07:35.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37840b6560 msgr2=0x7f37840b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:35.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37840b6560 0x7f37840b8a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f379c077160 tx=0x7f37940075b0 comp rx=0 tx=0).stop 2024-09-21T07:07:35.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 msgr2=0x7f379c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c075c50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f379002a930 tx=0x7f3790004270 comp rx=0 tx=0).stop 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 shutdown_connections 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379c109570 0x7f379c075c50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37840b6560 0x7f37840b8a70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f379c109ec0 0x7f379c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 --2- 172.21.15.153:0/2062457965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f379c103c30 0x7f379c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 >> 172.21.15.153:0/2062457965 conn(0x7f379c07b8d0 msgr2=0x7f379c1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 shutdown_connections 2024-09-21T07:07:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:07:35.966+0000 7f37a3548640 1 -- 172.21.15.153:0/2062457965 wait complete. 2024-09-21T07:07:36.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:35 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:36.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:35 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:35 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:36.661 DEBUG:teuthology.orchestra.run.smithi153:osd.8> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T07:07:36.663 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2024-09-21T07:07:36.664 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T07:07:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:36 smithi153 ceph-mon[34747]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:37.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:37.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:36 smithi027 ceph-mon[30779]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:37.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:37.228 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:36 smithi119 ceph-mon[34519]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:37.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:38.088 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:07:37 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:07:37.981+0000 7fa03c34a640 -1 osd.8 0 log_to_monitors true 2024-09-21T07:07:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:38 smithi119 ceph-mon[34519]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:38 smithi119 ceph-mon[34519]: from='osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T07:07:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:38 smithi119 ceph-mon[34519]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T07:07:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:38 smithi027 ceph-mon[30779]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:38 smithi027 ceph-mon[30779]: from='osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T07:07:39.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:38 smithi027 ceph-mon[30779]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T07:07:39.349 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:38 smithi153 ceph-mon[34747]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:39.349 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:38 smithi153 ceph-mon[34747]: from='osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T07:07:39.349 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:38 smithi153 ceph-mon[34747]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T07:07:40.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:40.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:40.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:39 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T07:07:40.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T07:07:40.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:40.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:07:40.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:07:40.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:40.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:39 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.300 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:07:40.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-09-21T07:07:40.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:40.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:40.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:39 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:07:40.711 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:40.983 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:07:40.688+0000 7fa0382cc640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T07:07:40.983 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:07:40.698+0000 7fa0300c2640 -1 osd.8 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi153 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:41.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:40 smithi027 ceph-mon[30779]: from='osd.8 ' entity='osd.8' 2024-09-21T07:07:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi153 2024-09-21T07:07:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-21T07:07:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:07:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T07:07:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:40 smithi119 ceph-mon[34519]: from='osd.8 ' entity='osd.8' 2024-09-21T07:07:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 -- 172.21.15.27:0/193112693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 msgr2=0x7f100410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 --2- 172.21.15.27:0/193112693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100410bad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009900 tx=0x7f0ff002a410 comp rx=0 tx=0).stop 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 -- 172.21.15.27:0/193112693 shutdown_connections 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 --2- 172.21.15.27:0/193112693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100410c010 0x7f1004112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 --2- 172.21.15.27:0/193112693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100410bad0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 --2- 172.21.15.27:0/193112693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004105d80 0x7f1004106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 -- 172.21.15.27:0/193112693 >> 172.21.15.27:0/193112693 conn(0x7f1004101810 msgr2=0x7f1004103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 -- 172.21.15.27:0/193112693 shutdown_connections 2024-09-21T07:07:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.309+0000 7f100c000640 1 -- 172.21.15.27:0/193112693 wait complete. 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.310+0000 7f100c000640 1 Processor -- start 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100c000640 1 -- start start 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100c000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004105d80 0x7f100419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100c000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100c000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100410c010 0x7f10041a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100c000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1004115d20 con 0x7f100410b6c0 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100c000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1004115ba0 con 0x7f100410c010 2024-09-21T07:07:41.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100c000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1004115ea0 con 0x7f1004105d80 2024-09-21T07:07:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f1008c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f1008c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45554/0 (socket says 172.21.15.27:45554) 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f1008c00640 1 -- 172.21.15.27:0/2224236921 learned_addr learned my addr 172.21.15.27:0/2224236921 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f1009600640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004105d80 0x7f100419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.311+0000 7f100a000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100410c010 0x7f10041a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f1008c00640 1 -- 172.21.15.27:0/2224236921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004105d80 msgr2=0x7f100419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f1008c00640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004105d80 0x7f100419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f1008c00640 1 -- 172.21.15.27:0/2224236921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100410c010 msgr2=0x7f10041a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f1008c00640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100410c010 0x7f10041a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f1008c00640 1 -- 172.21.15.27:0/2224236921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10041a2660 con 0x7f100410b6c0 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f100a000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100410c010 0x7f10041a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f1008c00640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100419cc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005ab0 tx=0x7f0ff0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.312+0000 7f1002000640 1 -- 172.21.15.27:0/2224236921 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff003f070 con 0x7f100410b6c0 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.313+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10041a28f0 con 0x7f100410b6c0 2024-09-21T07:07:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.313+0000 7f1002000640 1 -- 172.21.15.27:0/2224236921 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff0031b50 con 0x7f100410b6c0 2024-09-21T07:07:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.313+0000 7f1002000640 1 -- 172.21.15.27:0/2224236921 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0035030 con 0x7f100410b6c0 2024-09-21T07:07:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.313+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10041a90b0 con 0x7f100410b6c0 2024-09-21T07:07:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.314+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1004107000 con 0x7f100410b6c0 2024-09-21T07:07:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.316+0000 7f1002000640 1 -- 172.21.15.27:0/2224236921 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0ff0042070 con 0x7f100410b6c0 2024-09-21T07:07:41.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.319+0000 7f1002000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fec0b6560 0x7f0fec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.319+0000 7f1002000640 1 -- 172.21.15.27:0/2224236921 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6077+0+0 (secure 0 0 0) 0x7f0ff0107570 con 0x7f100410b6c0 2024-09-21T07:07:41.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.319+0000 7f1009600640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fec0b6560 0x7f0fec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.319+0000 7f1009600640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fec0b6560 0x7f0fec0b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009bd0 tx=0x7f0ff8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.320+0000 7f1002000640 1 -- 172.21.15.27:0/2224236921 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ff00ce620 con 0x7f100410b6c0 2024-09-21T07:07:41.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:41.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi153 2024-09-21T07:07:41.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-21T07:07:41.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:07:41.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T07:07:41.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:41.404 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:41.405 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:40 smithi153 ceph-mon[34747]: from='osd.8 ' entity='osd.8' 2024-09-21T07:07:41.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:07:41.534+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1004063800 con 0x7f0fec0b6560 2024-09-21T07:07:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:07:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:07:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] boot 2024-09-21T07:07:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T07:07:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T07:07:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:07:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:07:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] boot 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:07:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:07:42.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:42.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:07:42.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:07:42.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: osd.8 [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] boot 2024-09-21T07:07:42.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T07:07:42.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T07:07:42.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T07:07:42.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:07:42.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:07:42.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:43.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:42 smithi119 ceph-mon[34519]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:43.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:42 smithi119 ceph-mon[34519]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:07:43.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:42 smithi119 ceph-mon[34519]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T07:07:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:42 smithi027 ceph-mon[30779]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:42 smithi027 ceph-mon[30779]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:07:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:42 smithi027 ceph-mon[30779]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T07:07:43.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:42 smithi153 ceph-mon[34747]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T07:07:43.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:42 smithi153 ceph-mon[34747]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:07:43.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:42 smithi153 ceph-mon[34747]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T07:07:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:44 smithi027 ceph-mon[30779]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:07:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:44 smithi119 ceph-mon[34519]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:07:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:44 smithi153 ceph-mon[34747]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:07:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:45 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/170997363' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5c254d35-aa04-43e4-844e-5d6d2adb11ed"} : dispatch 2024-09-21T07:07:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:45 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/170997363' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c254d35-aa04-43e4-844e-5d6d2adb11ed"}]': finished 2024-09-21T07:07:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:45 smithi119 ceph-mon[34519]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T07:07:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:07:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:45 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4118012412' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:07:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:45 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/170997363' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5c254d35-aa04-43e4-844e-5d6d2adb11ed"} : dispatch 2024-09-21T07:07:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:45 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/170997363' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c254d35-aa04-43e4-844e-5d6d2adb11ed"}]': finished 2024-09-21T07:07:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:45 smithi027 ceph-mon[30779]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T07:07:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:07:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:45 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4118012412' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:07:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:45 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/170997363' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5c254d35-aa04-43e4-844e-5d6d2adb11ed"} : dispatch 2024-09-21T07:07:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:45 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/170997363' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c254d35-aa04-43e4-844e-5d6d2adb11ed"}]': finished 2024-09-21T07:07:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:45 smithi153 ceph-mon[34747]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T07:07:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:07:46.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:45 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4118012412' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:07:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:46 smithi027 ceph-mon[30779]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:07:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:46 smithi119 ceph-mon[34519]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:07:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:46 smithi153 ceph-mon[34747]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:07:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:49 smithi119 ceph-mon[34519]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:49.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:48 smithi027 ceph-mon[30779]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:49 smithi153 ceph-mon[34747]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T07:07:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:51 smithi119 ceph-mon[34519]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T07:07:51.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:51 smithi027 ceph-mon[30779]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T07:07:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:51 smithi153 ceph-mon[34747]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T07:07:53.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:53 smithi027 ceph-mon[30779]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T07:07:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:53 smithi119 ceph-mon[34519]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T07:07:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:53 smithi153 ceph-mon[34747]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T07:07:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:55 smithi119 ceph-mon[34519]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:07:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T07:07:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:55 smithi027 ceph-mon[30779]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:07:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T07:07:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:55 smithi153 ceph-mon[34747]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:07:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T07:07:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:56 smithi119 ceph-mon[34519]: Deploying daemon osd.9 on smithi027 2024-09-21T07:07:56.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:56 smithi027 ceph-mon[30779]: Deploying daemon osd.9 on smithi027 2024-09-21T07:07:56.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:56 smithi153 ceph-mon[34747]: Deploying daemon osd.9 on smithi027 2024-09-21T07:07:57.127 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:57 smithi027 ceph-mon[30779]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T07:07:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:57 smithi119 ceph-mon[34519]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T07:07:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:57 smithi153 ceph-mon[34747]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T07:07:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:07:59 smithi027 ceph-mon[30779]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:07:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:07:59 smithi119 ceph-mon[34519]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:07:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:07:59 smithi153 ceph-mon[34747]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T07:08:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:01 smithi119 ceph-mon[34519]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:01 smithi027 ceph-mon[30779]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:01 smithi153 ceph-mon[34747]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:03 smithi027 ceph-mon[30779]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:03 smithi119 ceph-mon[34519]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:03 smithi153 ceph-mon[34747]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:03.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:05 smithi027 ceph-mon[30779]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:05 smithi119 ceph-mon[34519]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:05 smithi153 ceph-mon[34747]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:07 smithi027 ceph-mon[30779]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:07.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:07 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:07 smithi153 ceph-mon[34747]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:07.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:07.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:07 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:07 smithi119 ceph-mon[34519]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:07.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:07 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:07.982 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2024-09-21T07:08:07.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.979+0000 7f1002000640 1 -- 172.21.15.27:0/2224236921 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1004063800 con 0x7f0fec0b6560 2024-09-21T07:08:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.982+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fec0b6560 msgr2=0x7f0fec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.982+0000 7f100c000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fec0b6560 0x7f0fec0b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009bd0 tx=0x7f0ff8005f90 comp rx=0 tx=0).stop 2024-09-21T07:08:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.982+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 msgr2=0x7f100419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.982+0000 7f100c000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100419cc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005ab0 tx=0x7f0ff0003b90 comp rx=0 tx=0).stop 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 shutdown_connections 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fec0b6560 0x7f0fec0b8a70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100410c010 0x7f10041a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100410b6c0 0x7f100419cc90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 --2- 172.21.15.27:0/2224236921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1004105d80 0x7f100419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 >> 172.21.15.27:0/2224236921 conn(0x7f1004101810 msgr2=0x7f1004103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 shutdown_connections 2024-09-21T07:08:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:07.984+0000 7f100c000640 1 -- 172.21.15.27:0/2224236921 wait complete. 2024-09-21T07:08:08.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:08.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:08.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:08.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:08.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:08.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:08.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:08.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:08.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:08.813 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T07:08:08.817 INFO:tasks.cephadm:Deploying osd.10 on smithi119 with /dev/vg_nvme/lv_1... 2024-09-21T07:08:08.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T07:08:09.074 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:08:09.195 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:09 smithi119 ceph-mon[34519]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:09.335 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:09 smithi027 ceph-mon[30779]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:09 smithi153 ceph-mon[34747]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:09.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:08:09 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:08:09.540+0000 7f5f89069640 -1 osd.9 0 log_to_monitors true 2024-09-21T07:08:10.247 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:10 smithi027 ceph-mon[30779]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:10.247 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:10 smithi027 ceph-mon[30779]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T07:08:10.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:10 smithi153 ceph-mon[34747]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:10.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:10 smithi153 ceph-mon[34747]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T07:08:10.435 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:10 smithi119 ceph-mon[34519]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:10.435 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:10 smithi119 ceph-mon[34519]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T07:08:11.082 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:11 smithi027 ceph-mon[30779]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T07:08:11.082 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:11 smithi027 ceph-mon[30779]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T07:08:11.082 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:11 smithi027 ceph-mon[30779]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:08:11.082 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:11 smithi119 ceph-mon[34519]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T07:08:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:11 smithi119 ceph-mon[34519]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T07:08:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:11 smithi119 ceph-mon[34519]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:08:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:11.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:08:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:08:11.079+0000 7f5f83c00640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T07:08:11.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:08:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:08:11.098+0000 7f5f7ec00640 -1 osd.9 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:08:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:11 smithi153 ceph-mon[34747]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T07:08:11.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:11 smithi153 ceph-mon[34747]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T07:08:11.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:11 smithi153 ceph-mon[34747]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T07:08:11.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T07:08:11.744 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_1 2024-09-21T07:08:11.991 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:08:12.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:12 smithi119 ceph-mon[34519]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:08:12.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:12 smithi119 ceph-mon[34519]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T07:08:12.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:12 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:12.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:12 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:12.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:12 smithi119 ceph-mon[34519]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:12 smithi027 ceph-mon[30779]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:08:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:12 smithi027 ceph-mon[30779]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T07:08:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:12 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:12 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:12 smithi027 ceph-mon[30779]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:12 smithi153 ceph-mon[34747]: from='osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T07:08:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:12 smithi153 ceph-mon[34747]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T07:08:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:12 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:12 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:12.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:12 smithi153 ceph-mon[34747]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T07:08:12.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 -- 172.21.15.119:0/2426569421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 msgr2=0x7f909410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 --2- 172.21.15.119:0/2426569421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909410ecb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f908c009900 tx=0x7f908c02a410 comp rx=0 tx=0).stop 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 -- 172.21.15.119:0/2426569421 shutdown_connections 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 --2- 172.21.15.119:0/2426569421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909410ecb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 --2- 172.21.15.119:0/2426569421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9094107b30 0x7f9094107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 --2- 172.21.15.119:0/2426569421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 0x7f9094102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 -- 172.21.15.119:0/2426569421 >> 172.21.15.119:0/2426569421 conn(0x7f9094077c60 msgr2=0x7f90940780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 -- 172.21.15.119:0/2426569421 shutdown_connections 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.509+0000 7f90999a3640 1 -- 172.21.15.119:0/2426569421 wait complete. 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 Processor -- start 2024-09-21T07:08:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 -- start start 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 0x7f909419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9094107b30 0x7f909419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f90941129d0 con 0x7f9094107b30 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9094112850 con 0x7f9094108480 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.510+0000 7f90999a3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9094112b50 con 0x7f90941021f0 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:12.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 0x7f909419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 0x7f909419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55602/0 (socket says 172.21.15.119:55602) 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909419bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37104/0 (socket says 172.21.15.119:37104) 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f9092ffd640 1 -- 172.21.15.119:0/183924132 learned_addr learned my addr 172.21.15.119:0/183924132 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90927fc640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9094107b30 0x7f909419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 -- 172.21.15.119:0/183924132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 msgr2=0x7f909419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 0x7f909419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 -- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9094107b30 msgr2=0x7f909419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9094107b30 0x7f909419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 -- 172.21.15.119:0/183924132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90941a65d0 con 0x7f9094108480 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f9092ffd640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 0x7f909419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90937fe640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909419bd40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f908c005940 tx=0x7f908c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90989a1640 1 -- 172.21.15.119:0/183924132 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c03f030 con 0x7f9094108480 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90989a1640 1 -- 172.21.15.119:0/183924132 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908c003cb0 con 0x7f9094108480 2024-09-21T07:08:12.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90941a6860 con 0x7f9094108480 2024-09-21T07:08:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90989a1640 1 -- 172.21.15.119:0/183924132 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c035070 con 0x7f9094108480 2024-09-21T07:08:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.511+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90941a6d40 con 0x7f9094108480 2024-09-21T07:08:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.512+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9094102600 con 0x7f9094108480 2024-09-21T07:08:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.514+0000 7f90989a1640 1 -- 172.21.15.119:0/183924132 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f908c042030 con 0x7f9094108480 2024-09-21T07:08:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.515+0000 7f90989a1640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f90800b6560 0x7f90800b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.515+0000 7f90989a1640 1 -- 172.21.15.119:0/183924132 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6476+0+0 (secure 0 0 0) 0x7f908c107280 con 0x7f9094108480 2024-09-21T07:08:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.515+0000 7f9092ffd640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f90800b6560 0x7f90800b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.516+0000 7f9092ffd640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f90800b6560 0x7f90800b8a70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f909410ff30 tx=0x7f9084009700 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:12.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.518+0000 7f90989a1640 1 -- 172.21.15.119:0/183924132 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f908c0d2620 con 0x7f9094108480 2024-09-21T07:08:12.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:12.711+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9094063800 con 0x7f90800b6560 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] boot 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi027 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:08:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:08:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:08:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] boot 2024-09-21T07:08:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T07:08:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi027 2024-09-21T07:08:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:08:13.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:08:13.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:08:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: osd.9 [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] boot 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi027 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:13.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:13.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:13.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:08:13.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:08:13.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:08:13.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:14 smithi027 ceph-mon[30779]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T07:08:14.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:14 smithi027 ceph-mon[30779]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:14.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:14 smithi119 ceph-mon[34519]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T07:08:14.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:14 smithi119 ceph-mon[34519]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:14.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:14 smithi153 ceph-mon[34747]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T07:08:14.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:14 smithi153 ceph-mon[34747]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:15.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:15 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/3829056778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"} : dispatch 2024-09-21T07:08:15.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:15 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"} : dispatch 2024-09-21T07:08:15.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:15 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"}]': finished 2024-09-21T07:08:15.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:15 smithi027 ceph-mon[30779]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T07:08:15.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:15 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/3829056778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"} : dispatch 2024-09-21T07:08:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:15 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"} : dispatch 2024-09-21T07:08:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:15 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"}]': finished 2024-09-21T07:08:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:15 smithi119 ceph-mon[34519]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T07:08:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:15.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:15 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/3829056778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"} : dispatch 2024-09-21T07:08:15.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:15 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"} : dispatch 2024-09-21T07:08:15.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:15 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87219b10-05f8-49ae-8be3-3e7d79454a39"}]': finished 2024-09-21T07:08:15.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:15 smithi153 ceph-mon[34747]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T07:08:15.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:16 smithi027 ceph-mon[30779]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:16.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:16 smithi027 ceph-mon[30779]: from='client.? 172.21.15.119:0/1834412254' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:08:16.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:16 smithi153 ceph-mon[34747]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:16.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:16 smithi153 ceph-mon[34747]: from='client.? 172.21.15.119:0/1834412254' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:08:16.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:16 smithi119 ceph-mon[34519]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:16.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:16 smithi119 ceph-mon[34519]: from='client.? 172.21.15.119:0/1834412254' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:08:18.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:18 smithi153 ceph-mon[34747]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:18.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:18 smithi119 ceph-mon[34519]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:18 smithi027 ceph-mon[30779]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:20.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:20 smithi153 ceph-mon[34747]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:20.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:20 smithi119 ceph-mon[34519]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:20 smithi027 ceph-mon[30779]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:22.666 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:22 smithi119 ceph-mon[34519]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:22.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:22 smithi153 ceph-mon[34747]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:22 smithi027 ceph-mon[30779]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:24 smithi153 ceph-mon[34747]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T07:08:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:24 smithi153 ceph-mon[34747]: Deploying daemon osd.10 on smithi119 2024-09-21T07:08:24.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:24 smithi119 ceph-mon[34519]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:24.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T07:08:24.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:24.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:24 smithi119 ceph-mon[34519]: Deploying daemon osd.10 on smithi119 2024-09-21T07:08:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:24 smithi027 ceph-mon[30779]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T07:08:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:24 smithi027 ceph-mon[30779]: Deploying daemon osd.10 on smithi119 2024-09-21T07:08:26.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:26 smithi153 ceph-mon[34747]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:26.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:26 smithi119 ceph-mon[34519]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:26.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:26 smithi027 ceph-mon[30779]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:28.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:28 smithi119 ceph-mon[34519]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:28.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:28 smithi153 ceph-mon[34747]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:28.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:28 smithi027 ceph-mon[30779]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:29.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:29.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:29.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:29.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:29.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:29.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:30.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:30 smithi153 ceph-mon[34747]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:30.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:30 smithi119 ceph-mon[34519]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:30.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:30 smithi027 ceph-mon[30779]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:32.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.470+0000 7f90989a1640 1 -- 172.21.15.119:0/183924132 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9094063800 con 0x7f90800b6560 2024-09-21T07:08:32.472 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 10 on host 'smithi119' 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f90800b6560 msgr2=0x7f90800b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f90800b6560 0x7f90800b8a70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f909410ff30 tx=0x7f9084009700 comp rx=0 tx=0).stop 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 msgr2=0x7f909419bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909419bd40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f908c005940 tx=0x7f908c0042f0 comp rx=0 tx=0).stop 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 shutdown_connections 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f90800b6560 0x7f90800b8a70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9094108480 0x7f909419bd40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9094107b30 0x7f909419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 --2- 172.21.15.119:0/183924132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90941021f0 0x7f909419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 >> 172.21.15.119:0/183924132 conn(0x7f9094077c60 msgr2=0x7f9094103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 shutdown_connections 2024-09-21T07:08:32.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T07:08:32.473+0000 7f90999a3640 1 -- 172.21.15.119:0/183924132 wait complete. 2024-09-21T07:08:32.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:32 smithi119 ceph-mon[34519]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:32 smithi027 ceph-mon[30779]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:32.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:32 smithi153 ceph-mon[34747]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:33.133 DEBUG:teuthology.orchestra.run.smithi119:osd.10> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T07:08:33.135 INFO:tasks.cephadm:Deploying osd.11 on smithi153 with /dev/vg_nvme/lv_1... 2024-09-21T07:08:33.135 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T07:08:33.390 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:08:33.719 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.719 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.719 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.720 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:33 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.766 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.766 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.766 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.766 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:33 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:33.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:33 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:34.765 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:34 smithi153 ceph-mon[34747]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:34.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:34 smithi119 ceph-mon[34519]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:34 smithi027 ceph-mon[30779]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:35.755 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:08:35 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:08:35.501+0000 7fdb4d279640 -1 osd.10 0 log_to_monitors true 2024-09-21T07:08:36.057 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-21T07:08:36.087 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi153:vg_nvme/lv_1 2024-09-21T07:08:36.339 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:08:36.724 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:36.725 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:36 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:36.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:36 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.816+0000 7f0e66c44640 1 -- 172.21.15.153:0/3993395117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60103ba0 msgr2=0x7f0e60103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.816+0000 7f0e66c44640 1 --2- 172.21.15.153:0/3993395117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60103ba0 0x7f0e60103fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009900 tx=0x7f0e5c02a410 comp rx=0 tx=0).stop 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 -- 172.21.15.153:0/3993395117 shutdown_connections 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 --2- 172.21.15.153:0/3993395117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60109e30 0x7f0e60110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 --2- 172.21.15.153:0/3993395117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e601094e0 0x7f0e601098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 --2- 172.21.15.153:0/3993395117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60103ba0 0x7f0e60103fb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 -- 172.21.15.153:0/3993395117 >> 172.21.15.153:0/3993395117 conn(0x7f0e600ff630 msgr2=0x7f0e60101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 -- 172.21.15.153:0/3993395117 shutdown_connections 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 -- 172.21.15.153:0/3993395117 wait complete. 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 Processor -- start 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.817+0000 7f0e66c44640 1 -- start start 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e66c44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e60103ba0 0x7f0e6019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e66c44640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e601094e0 0x7f0e6019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e66c44640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60109e30 0x7f0e6019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:36.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e66c44640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60113b40 con 0x7f0e601094e0 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e66c44640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601139c0 con 0x7f0e60103ba0 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e66c44640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60113cc0 con 0x7f0e60109e30 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60109e30 0x7f0e6019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60109e30 0x7f0e6019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35842/0 (socket says 172.21.15.153:35842) 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 -- 172.21.15.153:0/1548762783 learned_addr learned my addr 172.21.15.153:0/1548762783 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e649e0640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e60103ba0 0x7f0e6019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e57fff640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e601094e0 0x7f0e6019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 -- 172.21.15.153:0/1548762783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e60103ba0 msgr2=0x7f0e6019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e60103ba0 0x7f0e6019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 -- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e601094e0 msgr2=0x7f0e6019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e601094e0 0x7f0e6019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:36.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 -- 172.21.15.153:0/1548762783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e601a0460 con 0x7f0e60109e30 2024-09-21T07:08:36.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e651e1640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60109e30 0x7f0e6019fd30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009fd0 tx=0x7f0e5800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:36.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e55ffb640 1 -- 172.21.15.153:0/1548762783 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58017070 con 0x7f0e60109e30 2024-09-21T07:08:36.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.818+0000 7f0e55ffb640 1 -- 172.21.15.153:0/1548762783 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e58003e20 con 0x7f0e60109e30 2024-09-21T07:08:36.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.819+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e601a0750 con 0x7f0e60109e30 2024-09-21T07:08:36.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.819+0000 7f0e55ffb640 1 -- 172.21.15.153:0/1548762783 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58010460 con 0x7f0e60109e30 2024-09-21T07:08:36.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.819+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e601a6eb0 con 0x7f0e60109e30 2024-09-21T07:08:36.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.820+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e60103fb0 con 0x7f0e60109e30 2024-09-21T07:08:36.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.821+0000 7f0e55ffb640 1 -- 172.21.15.153:0/1548762783 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0e58003970 con 0x7f0e60109e30 2024-09-21T07:08:36.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.822+0000 7f0e55ffb640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6560 0x7f0e480b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:36.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.822+0000 7f0e55ffb640 1 -- 172.21.15.153:0/1548762783 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6613+0+0 (secure 0 0 0) 0x7f0e580e6a70 con 0x7f0e60109e30 2024-09-21T07:08:36.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.822+0000 7f0e649e0640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6560 0x7f0e480b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:36.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.823+0000 7f0e649e0640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6560 0x7f0e480b8a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005ab0 tx=0x7f0e5c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:36.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:36.826+0000 7f0e55ffb640 1 -- 172.21.15.153:0/1548762783 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e580b1ce0 con 0x7f0e60109e30 2024-09-21T07:08:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T07:08:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T07:08:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:08:36.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:36.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:36.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:36.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:36 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:37.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:37.012+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0e60063800 con 0x7f0e480b6560 2024-09-21T07:08:37.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi119 2024-09-21T07:08:37.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi119 to 4272M 2024-09-21T07:08:37.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T07:08:37.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:08:37.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:37 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:37.643 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi119 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi119 to 4272M 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:08:37.644 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:37 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:37.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi119 2024-09-21T07:08:37.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi119 to 4272M 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T07:08:37.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:37 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:38.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:08:38.242+0000 7fdb491fb640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T07:08:38.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:08:38.255+0000 7fdb40ff1640 -1 osd.10 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:08:38.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:38.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:08:38.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T07:08:38.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: from='osd.10 ' entity='osd.10' 2024-09-21T07:08:38.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] boot 2024-09-21T07:08:38.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T07:08:38.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: from='osd.10 ' entity='osd.10' 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] boot 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T07:08:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T07:08:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T07:08:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T07:08:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: from='osd.10 ' entity='osd.10' 2024-09-21T07:08:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: osd.10 [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] boot 2024-09-21T07:08:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T07:08:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T07:08:39.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:39 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:08:39.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:39 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:08:39.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:39 smithi119 ceph-mon[34519]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T07:08:39.828 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:39 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:08:39.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:39 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:08:39.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:39 smithi153 ceph-mon[34747]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T07:08:39.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:39 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:08:39.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:39 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:08:39.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:39 smithi027 ceph-mon[30779]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T07:08:40.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:40 smithi119 ceph-mon[34519]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 430 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:40.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:40 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/1589800311' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"} : dispatch 2024-09-21T07:08:40.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:40 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"} : dispatch 2024-09-21T07:08:40.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:40 smithi119 ceph-mon[34519]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"}]': finished 2024-09-21T07:08:40.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:40 smithi119 ceph-mon[34519]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T07:08:40.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:40 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:08:40.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:40 smithi119 ceph-mon[34519]: from='client.? 172.21.15.153:0/3425472182' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:08:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:40 smithi027 ceph-mon[30779]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 430 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:40 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/1589800311' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"} : dispatch 2024-09-21T07:08:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:40 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"} : dispatch 2024-09-21T07:08:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:40 smithi027 ceph-mon[30779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"}]': finished 2024-09-21T07:08:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:40 smithi027 ceph-mon[30779]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T07:08:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:40 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:08:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:40 smithi027 ceph-mon[30779]: from='client.? 172.21.15.153:0/3425472182' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:08:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:40 smithi153 ceph-mon[34747]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 430 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:40 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/1589800311' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"} : dispatch 2024-09-21T07:08:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:40 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"} : dispatch 2024-09-21T07:08:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:40 smithi153 ceph-mon[34747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ee8dc56-e77b-4969-8e32-4380b8b12b21"}]': finished 2024-09-21T07:08:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:40 smithi153 ceph-mon[34747]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T07:08:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:40 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:08:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:40 smithi153 ceph-mon[34747]: from='client.? 172.21.15.153:0/3425472182' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T07:08:42.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:42 smithi119 ceph-mon[34519]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 431 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T07:08:42.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:42 smithi027 ceph-mon[30779]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 431 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T07:08:42.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:42 smithi153 ceph-mon[34747]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 431 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T07:08:44.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:44 smithi119 ceph-mon[34519]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 430 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T07:08:44.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:44 smithi027 ceph-mon[30779]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 430 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T07:08:44.923 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:44 smithi153 ceph-mon[34747]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 430 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T07:08:46.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:46 smithi153 ceph-mon[34747]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T07:08:46.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:46 smithi119 ceph-mon[34519]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T07:08:46.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:46 smithi027 ceph-mon[30779]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T07:08:48.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:48 smithi027 ceph-mon[30779]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:08:48.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T07:08:48.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:48.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:48 smithi027 ceph-mon[30779]: Deploying daemon osd.11 on smithi153 2024-09-21T07:08:48.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:48 smithi153 ceph-mon[34747]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:08:48.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T07:08:48.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:48.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:48 smithi153 ceph-mon[34747]: Deploying daemon osd.11 on smithi153 2024-09-21T07:08:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:48 smithi119 ceph-mon[34519]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T07:08:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T07:08:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:48 smithi119 ceph-mon[34519]: Deploying daemon osd.11 on smithi153 2024-09-21T07:08:50.763 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:50 smithi153 ceph-mon[34747]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:08:50.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:50 smithi119 ceph-mon[34519]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:08:50.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:50 smithi027 ceph-mon[30779]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T07:08:52.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:52 smithi153 ceph-mon[34747]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T07:08:52.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:52 smithi119 ceph-mon[34519]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T07:08:52.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:52 smithi027 ceph-mon[30779]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T07:08:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:54 smithi119 ceph-mon[34519]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:54.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:54.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:54 smithi153 ceph-mon[34747]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:54.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:54.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:54.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:54.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:54 smithi027 ceph-mon[30779]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:54.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:54.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:54.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:56.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:56 smithi119 ceph-mon[34519]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:56.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:56 smithi027 ceph-mon[30779]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:56 smithi153 ceph-mon[34747]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:56.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.994+0000 7f0e55ffb640 1 -- 172.21.15.153:0/1548762783 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0e60063800 con 0x7f0e480b6560 2024-09-21T07:08:56.996 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 11 on host 'smithi153' 2024-09-21T07:08:56.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6560 msgr2=0x7f0e480b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6560 0x7f0e480b8a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005ab0 tx=0x7f0e5c005a20 comp rx=0 tx=0).stop 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60109e30 msgr2=0x7f0e6019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60109e30 0x7f0e6019fd30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009fd0 tx=0x7f0e5800b260 comp rx=0 tx=0).stop 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 shutdown_connections 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e60109e30 0x7f0e6019fd30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6560 0x7f0e480b8a70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e601094e0 0x7f0e6019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 --2- 172.21.15.153:0/1548762783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e60103ba0 0x7f0e6019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 >> 172.21.15.153:0/1548762783 conn(0x7f0e600ff630 msgr2=0x7f0e60108b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 shutdown_connections 2024-09-21T07:08:56.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:08:56.997+0000 7f0e66c44640 1 -- 172.21.15.153:0/1548762783 wait complete. 2024-09-21T07:08:57.980 DEBUG:teuthology.orchestra.run.smithi153:osd.11> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T07:08:57.982 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T07:08:57.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T07:08:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:58.380 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 -- 172.21.15.27:0/1523248326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94941094a0 msgr2=0x7f94941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 --2- 172.21.15.27:0/1523248326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94941094a0 0x7f94941098b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f947c009900 tx=0x7f947c02a410 comp rx=0 tx=0).stop 2024-09-21T07:08:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 -- 172.21.15.27:0/1523248326 shutdown_connections 2024-09-21T07:08:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 --2- 172.21.15.27:0/1523248326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9494109df0 0x7f9494110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 --2- 172.21.15.27:0/1523248326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94941094a0 0x7f94941098b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 --2- 172.21.15.27:0/1523248326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9494103b60 0x7f9494103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:58.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 -- 172.21.15.27:0/1523248326 >> 172.21.15.27:0/1523248326 conn(0x7f949407b850 msgr2=0x7f949407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:58.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 -- 172.21.15.27:0/1523248326 shutdown_connections 2024-09-21T07:08:58.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.949+0000 7f949a000640 1 -- 172.21.15.27:0/1523248326 wait complete. 2024-09-21T07:08:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.950+0000 7f949a000640 1 Processor -- start 2024-09-21T07:08:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.951+0000 7f949a000640 1 -- start start 2024-09-21T07:08:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.951+0000 7f949a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9494103b60 0x7f949419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.951+0000 7f949a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94941094a0 0x7f949419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.951+0000 7f949a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9494109df0 0x7f949419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.951+0000 7f949a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9494113ab0 con 0x7f9494103b60 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.951+0000 7f949a000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9494113930 con 0x7f94941094a0 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.951+0000 7f949a000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9494113c30 con 0x7f9494109df0 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f9493400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9494103b60 0x7f949419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94941094a0 0x7f949419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94941094a0 0x7f949419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34538/0 (socket says 172.21.15.27:34538) 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 -- 172.21.15.27:0/1276081572 learned_addr learned my addr 172.21.15.27:0/1276081572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f9493e00640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9494109df0 0x7f949419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 -- 172.21.15.27:0/1276081572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9494109df0 msgr2=0x7f949419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9494109df0 0x7f949419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 -- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9494103b60 msgr2=0x7f949419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9494103b60 0x7f949419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 -- 172.21.15.27:0/1276081572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94941a03f0 con 0x7f94941094a0 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.952+0000 7f948aa00640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94941094a0 0x7f949419aa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f947c02a930 tx=0x7f947c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.953+0000 7f9490c00640 1 -- 172.21.15.27:0/1276081572 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c03f030 con 0x7f94941094a0 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.953+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94941a0680 con 0x7f94941094a0 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.953+0000 7f9490c00640 1 -- 172.21.15.27:0/1276081572 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947c004490 con 0x7f94941094a0 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.953+0000 7f9490c00640 1 -- 172.21.15.27:0/1276081572 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c035070 con 0x7f94941094a0 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.953+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94941a6e40 con 0x7f94941094a0 2024-09-21T07:08:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.954+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9494104de0 con 0x7f94941094a0 2024-09-21T07:08:58.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.957+0000 7f9490c00640 1 -- 172.21.15.27:0/1276081572 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f947c042030 con 0x7f94941094a0 2024-09-21T07:08:58.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.958+0000 7f9490c00640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94780b6560 0x7f94780b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:58.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.958+0000 7f9490c00640 1 -- 172.21.15.27:0/1276081572 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7024+0+0 (secure 0 0 0) 0x7f947c1075b0 con 0x7f94941094a0 2024-09-21T07:08:58.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.958+0000 7f9493400640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94780b6560 0x7f94780b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.959+0000 7f9493400640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94780b6560 0x7f94780b8a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9484009bd0 tx=0x7f9484005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:58.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:58.960+0000 7f9490c00640 1 -- 172.21.15.27:0/1276081572 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f947c0d26b0 con 0x7f94941094a0 2024-09-21T07:08:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:58 smithi027 ceph-mon[30779]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:08:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.162+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f949410bbe0 con 0x7f94941094a0 2024-09-21T07:08:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.163+0000 7f9490c00640 1 -- 172.21.15.27:0/1276081572 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v64) ==== 74+0+133 (secure 0 0 0) 0x7f947c0d75f0 con 0x7f94941094a0 2024-09-21T07:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:08:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94780b6560 msgr2=0x7f94780b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94780b6560 0x7f94780b8a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9484009bd0 tx=0x7f9484005f90 comp rx=0 tx=0).stop 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94941094a0 msgr2=0x7f949419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94941094a0 0x7f949419aa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f947c02a930 tx=0x7f947c031a20 comp rx=0 tx=0).stop 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 shutdown_connections 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9494109df0 0x7f949419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94780b6560 0x7f94780b8a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94941094a0 0x7f949419aa20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 --2- 172.21.15.27:0/1276081572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9494103b60 0x7f949419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.165+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 >> 172.21.15.27:0/1276081572 conn(0x7f949407b850 msgr2=0x7f9494107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.166+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 shutdown_connections 2024-09-21T07:08:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:08:59.166+0000 7f949a000640 1 -- 172.21.15.27:0/1276081572 wait complete. 2024-09-21T07:08:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:58 smithi153 ceph-mon[34747]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:08:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:58 smithi119 ceph-mon[34519]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:08:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:08:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:08:59.948 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":64,"num_osds":12,"num_up_osds":11,"osd_up_since":1726902518,"num_in_osds":12,"osd_in_since":1726902519,"num_remapped_pgs":0} 2024-09-21T07:09:00.134 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:00 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1276081572' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T07:09:00.135 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:00 smithi153 ceph-mon[34747]: from='osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T07:09:00.135 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:00 smithi153 ceph-mon[34747]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T07:09:00.135 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:08:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:08:59.913+0000 7f058dedd640 -1 osd.11 0 log_to_monitors true 2024-09-21T07:09:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:00 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1276081572' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T07:09:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:00 smithi119 ceph-mon[34519]: from='osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T07:09:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:00 smithi119 ceph-mon[34519]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T07:09:00.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:00 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1276081572' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T07:09:00.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:00 smithi027 ceph-mon[30779]: from='osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T07:09:00.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:00 smithi027 ceph-mon[30779]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T07:09:00.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T07:09:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:01 smithi119 ceph-mon[34519]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:09:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:01 smithi119 ceph-mon[34519]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T07:09:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:01 smithi119 ceph-mon[34519]: from='osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:09:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:01 smithi119 ceph-mon[34519]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T07:09:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:01 smithi119 ceph-mon[34519]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:09:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:01 smithi027 ceph-mon[30779]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:09:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:01 smithi027 ceph-mon[30779]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T07:09:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:01 smithi027 ceph-mon[30779]: from='osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:09:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:01 smithi027 ceph-mon[30779]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T07:09:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:01 smithi027 ceph-mon[30779]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:09:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:01.398 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-mon[34747]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:09:01.398 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-mon[34747]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T07:09:01.398 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-mon[34747]: from='osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:09:01.398 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-mon[34747]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T07:09:01.398 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:01.398 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-mon[34747]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-21T07:09:01.399 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:09:01.010+0000 7f0589e5f640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T07:09:01.399 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:09:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:09:01.024+0000 7f0585c5d640 -1 osd.11 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:09:02.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 -- 172.21.15.27:0/2123346561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081039b0 msgr2=0x7f2508103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 --2- 172.21.15.27:0/2123346561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081039b0 0x7f2508103dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009900 tx=0x7f24fc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 -- 172.21.15.27:0/2123346561 shutdown_connections 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 --2- 172.21.15.27:0/2123346561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25081092f0 0x7f2508109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 --2- 172.21.15.27:0/2123346561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2508109c40 0x7f2508110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 --2- 172.21.15.27:0/2123346561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081039b0 0x7f2508103dc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 -- 172.21.15.27:0/2123346561 >> 172.21.15.27:0/2123346561 conn(0x7f25080ff440 msgr2=0x7f2508101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 -- 172.21.15.27:0/2123346561 shutdown_connections 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.005+0000 7f250de00640 1 -- 172.21.15.27:0/2123346561 wait complete. 2024-09-21T07:09:02.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.006+0000 7f250de00640 1 Processor -- start 2024-09-21T07:09:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.007+0000 7f250de00640 1 -- start start 2024-09-21T07:09:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.007+0000 7f250de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25081039b0 0x7f250819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.007+0000 7f250de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081092f0 0x7f250819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.007+0000 7f250de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2508109c40 0x7f25081a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.007+0000 7f250de00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2508113900 con 0x7f25081092f0 2024-09-21T07:09:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.007+0000 7f250de00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2508113780 con 0x7f25081039b0 2024-09-21T07:09:02.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.007+0000 7f250de00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2508113a80 con 0x7f2508109c40 2024-09-21T07:09:02.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25081039b0 0x7f250819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:02.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25081039b0 0x7f250819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34560/0 (socket says 172.21.15.27:34560) 2024-09-21T07:09:02.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 -- 172.21.15.27:0/3865615644 learned_addr learned my addr 172.21.15.27:0/3865615644 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:02.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f2507e00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081092f0 0x7f250819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:02.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250d400640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2508109c40 0x7f25081a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:02.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 -- 172.21.15.27:0/3865615644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2508109c40 msgr2=0x7f25081a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:02.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2508109c40 0x7f25081a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 -- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081092f0 msgr2=0x7f250819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:02.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081092f0 0x7f250819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.008+0000 7f250ca00640 1 -- 172.21.15.27:0/3865615644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25081a24a0 con 0x7f25081039b0 2024-09-21T07:09:02.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.009+0000 7f250ca00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25081039b0 0x7f250819c530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f24fc005940 tx=0x7f24fc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:02.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.009+0000 7f2505600640 1 -- 172.21.15.27:0/3865615644 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc03f030 con 0x7f25081039b0 2024-09-21T07:09:02.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.009+0000 7f2505600640 1 -- 172.21.15.27:0/3865615644 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24fc003aa0 con 0x7f25081039b0 2024-09-21T07:09:02.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.009+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25081a2730 con 0x7f25081039b0 2024-09-21T07:09:02.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.009+0000 7f2505600640 1 -- 172.21.15.27:0/3865615644 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc035070 con 0x7f25081039b0 2024-09-21T07:09:02.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.009+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081a8ef0 con 0x7f25081039b0 2024-09-21T07:09:02.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.011+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2508104c30 con 0x7f25081039b0 2024-09-21T07:09:02.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.015+0000 7f2505600640 1 -- 172.21.15.27:0/3865615644 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f24fc042030 con 0x7f25081039b0 2024-09-21T07:09:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.017+0000 7f2505600640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6560 0x7f24f00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.017+0000 7f2505600640 1 -- 172.21.15.27:0/3865615644 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7062+0+0 (secure 0 0 0) 0x7f24fc1074f0 con 0x7f25081039b0 2024-09-21T07:09:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.018+0000 7f2507e00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6560 0x7f24f00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.019+0000 7f2507e00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6560 0x7f24f00b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2508105a60 tx=0x7f24f400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:02.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.021+0000 7f2505600640 1 -- 172.21.15.27:0/3865615644 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24fc0d2640 con 0x7f25081039b0 2024-09-21T07:09:02.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.248+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f25081a9120 con 0x7f25081039b0 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:02.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:02.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:02.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:02.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:02.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.249+0000 7f2505600640 1 -- 172.21.15.27:0/3865615644 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f24fc0d7580 con 0x7f25081039b0 2024-09-21T07:09:02.252 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:02.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6560 msgr2=0x7f24f00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:02.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6560 0x7f24f00b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2508105a60 tx=0x7f24f400b390 comp rx=0 tx=0).stop 2024-09-21T07:09:02.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25081039b0 msgr2=0x7f250819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25081039b0 0x7f250819c530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f24fc005940 tx=0x7f24fc004270 comp rx=0 tx=0).stop 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 shutdown_connections 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2508109c40 0x7f25081a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6560 0x7f24f00b8a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25081092f0 0x7f250819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 --2- 172.21.15.27:0/3865615644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25081039b0 0x7f250819c530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 >> 172.21.15.27:0/3865615644 conn(0x7f25080ff440 msgr2=0x7f25081089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.251+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 shutdown_connections 2024-09-21T07:09:02.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:02.252+0000 7f250de00640 1 -- 172.21.15.27:0/3865615644 wait complete. 2024-09-21T07:09:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:02.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:02.429 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:02.887 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":67,"num_osds":12,"num_up_osds":12,"osd_up_since":1726902542,"num_in_osds":12,"osd_in_since":1726902519,"num_remapped_pgs":0} 2024-09-21T07:09:02.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:09:03.269 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: purged_snaps scrub starts 2024-09-21T07:09:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: purged_snaps scrub ok 2024-09-21T07:09:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:09:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: Detected new or changed devices on smithi153 2024-09-21T07:09:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-21T07:09:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] boot 2024-09-21T07:09:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T07:09:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:03 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3865615644' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: purged_snaps scrub starts 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: purged_snaps scrub ok 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: Detected new or changed devices on smithi153 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] boot 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:03 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3865615644' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T07:09:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: purged_snaps scrub starts 2024-09-21T07:09:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: purged_snaps scrub ok 2024-09-21T07:09:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-21T07:09:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: Detected new or changed devices on smithi153 2024-09-21T07:09:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-21T07:09:03.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: osd.11 [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] boot 2024-09-21T07:09:03.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T07:09:03.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T07:09:03.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:03 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3865615644' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.952+0000 7f5f96000640 1 -- 172.21.15.27:0/206940660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f9010b6c0 msgr2=0x7f5f9010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.952+0000 7f5f96000640 1 --2- 172.21.15.27:0/206940660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f9010b6c0 0x7f5f9010bad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009900 tx=0x7f5f7c02a410 comp rx=0 tx=0).stop 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.952+0000 7f5f96000640 1 -- 172.21.15.27:0/206940660 shutdown_connections 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.952+0000 7f5f96000640 1 --2- 172.21.15.27:0/206940660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 0x7f5f90112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.952+0000 7f5f96000640 1 --2- 172.21.15.27:0/206940660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f9010b6c0 0x7f5f9010bad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.952+0000 7f5f96000640 1 --2- 172.21.15.27:0/206940660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90105d80 0x7f5f90106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.952+0000 7f5f96000640 1 -- 172.21.15.27:0/206940660 >> 172.21.15.27:0/206940660 conn(0x7f5f90101810 msgr2=0x7f5f90103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.953+0000 7f5f96000640 1 -- 172.21.15.27:0/206940660 shutdown_connections 2024-09-21T07:09:03.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.953+0000 7f5f96000640 1 -- 172.21.15.27:0/206940660 wait complete. 2024-09-21T07:09:03.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 Processor -- start 2024-09-21T07:09:03.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 -- start start 2024-09-21T07:09:03.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f90105d80 0x7f5f9019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f9010b6c0 0x7f5f9019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 0x7f5f901a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90115d20 con 0x7f5f90105d80 2024-09-21T07:09:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90115ba0 con 0x7f5f9010b6c0 2024-09-21T07:09:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.954+0000 7f5f96000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90115ea0 con 0x7f5f9010c010 2024-09-21T07:09:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f9010b6c0 0x7f5f9019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 0x7f5f901a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f90105d80 0x7f5f9019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 0x7f5f901a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59078/0 (socket says 172.21.15.27:59078) 2024-09-21T07:09:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f9010b6c0 0x7f5f9019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34564/0 (socket says 172.21.15.27:34564) 2024-09-21T07:09:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 -- 172.21.15.27:0/1638303007 learned_addr learned my addr 172.21.15.27:0/1638303007 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 -- 172.21.15.27:0/1638303007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f9010b6c0 msgr2=0x7f5f9019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f9010b6c0 0x7f5f9019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 -- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f90105d80 msgr2=0x7f5f9019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f90105d80 0x7f5f9019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 -- 172.21.15.27:0/1638303007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f901a2620 con 0x7f5f9010c010 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.955+0000 7f5f8fe00640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 0x7f5f901a1ef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e970 tx=0x7f5f8800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.956+0000 7f5f87e00640 1 -- 172.21.15.27:0/1638303007 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800ccc0 con 0x7f5f9010c010 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.956+0000 7f5f87e00640 1 -- 172.21.15.27:0/1638303007 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8800ce60 con 0x7f5f9010c010 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.956+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f901a2910 con 0x7f5f9010c010 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.956+0000 7f5f87e00640 1 -- 172.21.15.27:0/1638303007 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88010430 con 0x7f5f9010c010 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.956+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f901a9070 con 0x7f5f9010c010 2024-09-21T07:09:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.957+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f90106190 con 0x7f5f9010c010 2024-09-21T07:09:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.959+0000 7f5f87e00640 1 -- 172.21.15.27:0/1638303007 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f5f88004f70 con 0x7f5f9010c010 2024-09-21T07:09:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.961+0000 7f5f87e00640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5f740b65e0 0x7f5f740b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.961+0000 7f5f87e00640 1 -- 172.21.15.27:0/1638303007 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f5f880e7550 con 0x7f5f9010c010 2024-09-21T07:09:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.961+0000 7f5f8f400640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5f740b65e0 0x7f5f740b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.962+0000 7f5f8f400640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5f740b65e0 0x7f5f740b8af0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005fd0 tx=0x7f5f80009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:03.962+0000 7f5f87e00640 1 -- 172.21.15.27:0/1638303007 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f88056710 con 0x7f5f9010c010 2024-09-21T07:09:04.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.164+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f9019d7e0 con 0x7f5f9010c010 2024-09-21T07:09:04.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.165+0000 7f5f87e00640 1 -- 172.21.15.27:0/1638303007 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+18794 (secure 0 0 0) 0x7f5f9019d7e0 con 0x7f5f9010c010 2024-09-21T07:09:04.167 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:04.168 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":68,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:09:03.018269+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:09:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5f740b65e0 msgr2=0x7f5f740b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5f740b65e0 0x7f5f740b8af0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005fd0 tx=0x7f5f80009500 comp rx=0 tx=0).stop 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 msgr2=0x7f5f901a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 0x7f5f901a1ef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e970 tx=0x7f5f8800ed70 comp rx=0 tx=0).stop 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 shutdown_connections 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f9010c010 0x7f5f901a1ef0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5f740b65e0 0x7f5f740b8af0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.167+0000 7f5f96000640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f9010b6c0 0x7f5f9019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.168+0000 7f5f96000640 1 --2- 172.21.15.27:0/1638303007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f90105d80 0x7f5f9019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.168+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 >> 172.21.15.27:0/1638303007 conn(0x7f5f90101810 msgr2=0x7f5f90108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.168+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 shutdown_connections 2024-09-21T07:09:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:04.168+0000 7f5f96000640 1 -- 172.21.15.27:0/1638303007 wait complete. 2024-09-21T07:09:04.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:04 smithi027 ceph-mon[30779]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T07:09:04.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:04 smithi153 ceph-mon[34747]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T07:09:04.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:04 smithi119 ceph-mon[34519]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T07:09:04.882 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:04:59.755438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:09:04.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T07:09:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:05 smithi027 ceph-mon[30779]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:05 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1638303007' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:09:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:05 smithi153 ceph-mon[34747]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:05 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1638303007' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:09:05.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:05 smithi119 ceph-mon[34519]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:05.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:05 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1638303007' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 -- 172.21.15.27:0/3112516213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010b510 msgr2=0x7f2e0010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3112516213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010b510 0x7f2e0010b920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009900 tx=0x7f2de802a410 comp rx=0 tx=0).stop 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 -- 172.21.15.27:0/3112516213 shutdown_connections 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3112516213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e0010be60 0x7f2e00112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3112516213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010b510 0x7f2e0010b920 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3112516213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00105bd0 0x7f2e00105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 -- 172.21.15.27:0/3112516213 >> 172.21.15.27:0/3112516213 conn(0x7f2e00101620 msgr2=0x7f2e00103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 -- 172.21.15.27:0/3112516213 shutdown_connections 2024-09-21T07:09:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.851+0000 7f2e04a00640 1 -- 172.21.15.27:0/3112516213 wait complete. 2024-09-21T07:09:05.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.852+0000 7f2e04a00640 1 Processor -- start 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.852+0000 7f2e04a00640 1 -- start start 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2e04a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00105bd0 0x7f2e0019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2e04a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e0010b510 0x7f2e0019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2e04a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010be60 0x7f2e001a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2e04a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00115b70 con 0x7f2e0010be60 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2e04a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001159f0 con 0x7f2e00105bd0 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2e04a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00115cf0 con 0x7f2e0010b510 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2dff200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010be60 0x7f2e001a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2dfe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00105bd0 0x7f2e0019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2df5e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e0010b510 0x7f2e0019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2dff200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010be60 0x7f2e001a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57174/0 (socket says 172.21.15.27:57174) 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.853+0000 7f2dff200640 1 -- 172.21.15.27:0/3590361631 learned_addr learned my addr 172.21.15.27:0/3590361631 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2dff200640 1 -- 172.21.15.27:0/3590361631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e0010b510 msgr2=0x7f2e0019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:05.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2dff200640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e0010b510 0x7f2e0019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:05.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2dff200640 1 -- 172.21.15.27:0/3590361631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00105bd0 msgr2=0x7f2e0019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:05.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2dff200640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00105bd0 0x7f2e0019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:05.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2dff200640 1 -- 172.21.15.27:0/3590361631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e001a2490 con 0x7f2e0010be60 2024-09-21T07:09:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2dff200640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010be60 0x7f2e001a1d60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2df800eaa0 tx=0x7f2df800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2df7e00640 1 -- 172.21.15.27:0/3590361631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df800cdb0 con 0x7f2e0010be60 2024-09-21T07:09:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2df7e00640 1 -- 172.21.15.27:0/3590361631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df8005020 con 0x7f2e0010be60 2024-09-21T07:09:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e001a2780 con 0x7f2e0010be60 2024-09-21T07:09:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.854+0000 7f2df7e00640 1 -- 172.21.15.27:0/3590361631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80116d0 con 0x7f2e0010be60 2024-09-21T07:09:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.855+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e001a8ee0 con 0x7f2e0010be60 2024-09-21T07:09:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.856+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e00105fe0 con 0x7f2e0010be60 2024-09-21T07:09:05.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.858+0000 7f2df7e00640 1 -- 172.21.15.27:0/3590361631 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f2df80051d0 con 0x7f2e0010be60 2024-09-21T07:09:05.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.860+0000 7f2df7e00640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2de40b6560 0x7f2de40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:05.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.860+0000 7f2df7e00640 1 -- 172.21.15.27:0/3590361631 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f2df80e7520 con 0x7f2e0010be60 2024-09-21T07:09:05.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.860+0000 7f2dfe800640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2de40b6560 0x7f2de40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:05.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.861+0000 7f2dfe800640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2de40b6560 0x7f2de40b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2df000bbe0 tx=0x7f2df000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:05.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:05.862+0000 7f2df7e00640 1 -- 172.21.15.27:0/3590361631 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2df8016030 con 0x7f2e0010be60 2024-09-21T07:09:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.060+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e0010dcb0 con 0x7f2e0010be60 2024-09-21T07:09:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.061+0000 7f2df7e00640 1 -- 172.21.15.27:0/3590361631 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f2df80b24c0 con 0x7f2e0010be60 2024-09-21T07:09:06.063 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-21T07:09:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2de40b6560 msgr2=0x7f2de40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2de40b6560 0x7f2de40b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2df000bbe0 tx=0x7f2df000b390 comp rx=0 tx=0).stop 2024-09-21T07:09:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010be60 msgr2=0x7f2e001a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010be60 0x7f2e001a1d60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2df800eaa0 tx=0x7f2df800ef60 comp rx=0 tx=0).stop 2024-09-21T07:09:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 shutdown_connections 2024-09-21T07:09:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e0010b510 0x7f2e0019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2de40b6560 0x7f2de40b8a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e0010be60 0x7f2e001a1d60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 --2- 172.21.15.27:0/3590361631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00105bd0 0x7f2e0019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 >> 172.21.15.27:0/3590361631 conn(0x7f2e00101620 msgr2=0x7f2e00103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:06.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 shutdown_connections 2024-09-21T07:09:06.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:06.064+0000 7f2e04a00640 1 -- 172.21.15.27:0/3590361631 wait complete. 2024-09-21T07:09:06.935 INFO:tasks.cephadm:Adding mds.a on smithi027 2024-09-21T07:09:06.935 INFO:tasks.cephadm:Adding mds.d on smithi027 2024-09-21T07:09:06.935 INFO:tasks.cephadm:Adding mds.g on smithi027 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.j on smithi027 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.b on smithi119 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.e on smithi119 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.h on smithi119 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.k on smithi119 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.c on smithi153 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.f on smithi153 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.i on smithi153 2024-09-21T07:09:06.936 INFO:tasks.cephadm:Adding mds.l on smithi153 2024-09-21T07:09:06.936 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l' 2024-09-21T07:09:07.196 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:09:07.318 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:07 smithi153 ceph-mon[34747]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:07.318 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:07 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3590361631' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:09:07.336 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:07 smithi119 ceph-mon[34519]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:07.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:07 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3590361631' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:09:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:07 smithi027 ceph-mon[30779]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:07 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3590361631' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:09:07.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 -- 172.21.15.153:0/2906435940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b4108cf0 msgr2=0x7f82b410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:07.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 --2- 172.21.15.153:0/2906435940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b4108cf0 0x7f82b410f520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009900 tx=0x7f82ac02a410 comp rx=0 tx=0).stop 2024-09-21T07:09:07.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 -- 172.21.15.153:0/2906435940 shutdown_connections 2024-09-21T07:09:07.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 --2- 172.21.15.153:0/2906435940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b4108cf0 0x7f82b410f520 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 --2- 172.21.15.153:0/2906435940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82b41083a0 0x7f82b41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 --2- 172.21.15.153:0/2906435940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b4102a60 0x7f82b4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 -- 172.21.15.153:0/2906435940 >> 172.21.15.153:0/2906435940 conn(0x7f82b4079730 msgr2=0x7f82b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:07.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 -- 172.21.15.153:0/2906435940 shutdown_connections 2024-09-21T07:09:07.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 -- 172.21.15.153:0/2906435940 wait complete. 2024-09-21T07:09:07.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 Processor -- start 2024-09-21T07:09:07.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.695+0000 7f82bb821640 1 -- start start 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82bb821640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b4102a60 0x7f82b419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82bb821640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b41083a0 0x7f82b419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82bb821640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82b4108cf0 0x7f82b419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82bb821640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4112960 con 0x7f82b4108cf0 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82bb821640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41127e0 con 0x7f82b4102a60 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82bb821640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4112ae0 con 0x7f82b41083a0 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b41083a0 0x7f82b419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b41083a0 0x7f82b419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41638/0 (socket says 172.21.15.153:41638) 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 -- 172.21.15.153:0/1847916066 learned_addr learned my addr 172.21.15.153:0/1847916066 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b95bd640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b4102a60 0x7f82b419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:07.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b9dbe640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82b4108cf0 0x7f82b419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 -- 172.21.15.153:0/1847916066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b4102a60 msgr2=0x7f82b419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b4102a60 0x7f82b419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 -- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82b4108cf0 msgr2=0x7f82b419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82b4108cf0 0x7f82b419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 -- 172.21.15.153:0/1847916066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b41a0390 con 0x7f82b41083a0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.696+0000 7f82b8dbc640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b41083a0 0x7f82b419a9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82a400c970 tx=0x7f82a400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.697+0000 7f82aa7fc640 1 -- 172.21.15.153:0/1847916066 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4007bb0 con 0x7f82b41083a0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.697+0000 7f82aa7fc640 1 -- 172.21.15.153:0/1847916066 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a4007d50 con 0x7f82b41083a0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.697+0000 7f82aa7fc640 1 -- 172.21.15.153:0/1847916066 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a400f450 con 0x7f82b41083a0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.697+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b41a0680 con 0x7f82b41083a0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.697+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b40ff550 con 0x7f82b41083a0 2024-09-21T07:09:07.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.698+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b4102e70 con 0x7f82b41083a0 2024-09-21T07:09:07.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.699+0000 7f82aa7fc640 1 -- 172.21.15.153:0/1847916066 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f82a4004f20 con 0x7f82b41083a0 2024-09-21T07:09:07.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.700+0000 7f82aa7fc640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f82a00b6560 0x7f82a00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:07.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.700+0000 7f82aa7fc640 1 -- 172.21.15.153:0/1847916066 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f82a40e6f60 con 0x7f82b41083a0 2024-09-21T07:09:07.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.700+0000 7f82b95bd640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f82a00b6560 0x7f82a00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:07.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.701+0000 7f82b95bd640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f82a00b6560 0x7f82a00b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82b0005fd0 tx=0x7f82b0009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:07.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.704+0000 7f82aa7fc640 1 -- 172.21.15.153:0/1847916066 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82a4014030 con 0x7f82b41083a0 2024-09-21T07:09:07.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.894+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}) -- 0x7f82b410bb30 con 0x7f82a00b6560 2024-09-21T07:09:07.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.904+0000 7f82aa7fc640 1 -- 172.21.15.153:0/1847916066 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f82b410bb30 con 0x7f82a00b6560 2024-09-21T07:09:07.905 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled mds.all update... 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f82a00b6560 msgr2=0x7f82a00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f82a00b6560 0x7f82a00b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82b0005fd0 tx=0x7f82b0009680 comp rx=0 tx=0).stop 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b41083a0 msgr2=0x7f82b419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b41083a0 0x7f82b419a9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82a400c970 tx=0x7f82a400ce30 comp rx=0 tx=0).stop 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 shutdown_connections 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82b41083a0 0x7f82b419a9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f82a00b6560 0x7f82a00b8a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82b4108cf0 0x7f82b419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 --2- 172.21.15.153:0/1847916066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b4102a60 0x7f82b419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:07.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 >> 172.21.15.153:0/1847916066 conn(0x7f82b4079730 msgr2=0x7f82b406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:07.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 shutdown_connections 2024-09-21T07:09:07.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-21T07:09:07.906+0000 7f82bb821640 1 -- 172.21.15.153:0/1847916066 wait complete. 2024-09-21T07:09:08.533 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T07:09:08.537 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T07:09:08.583 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T07:09:08.629 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T07:09:08.673 DEBUG:teuthology.orchestra.run.smithi119:mds.b> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T07:09:08.676 DEBUG:teuthology.orchestra.run.smithi119:mds.e> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T07:09:08.719 DEBUG:teuthology.orchestra.run.smithi119:mds.h> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T07:09:08.762 DEBUG:teuthology.orchestra.run.smithi119:mds.k> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T07:09:08.805 DEBUG:teuthology.orchestra.run.smithi153:mds.c> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T07:09:08.808 DEBUG:teuthology.orchestra.run.smithi153:mds.f> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T07:09:08.850 DEBUG:teuthology.orchestra.run.smithi153:mds.i> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T07:09:08.893 DEBUG:teuthology.orchestra.run.smithi153:mds.l> sudo journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T07:09:08.936 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T07:09:08.936 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:09.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:09.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:09.096 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:09.313 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:09:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:09.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.907+0000 7f15e9800640 1 -- 172.21.15.27:0/41896517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e4105d90 msgr2=0x7f15e41061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.907+0000 7f15e9800640 1 --2- 172.21.15.27:0/41896517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e4105d90 0x7f15e41061a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009900 tx=0x7f15d402a400 comp rx=0 tx=0).stop 2024-09-21T07:09:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.908+0000 7f15e9800640 1 -- 172.21.15.27:0/41896517 shutdown_connections 2024-09-21T07:09:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.908+0000 7f15e9800640 1 --2- 172.21.15.27:0/41896517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15e410b6d0 0x7f15e410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.908+0000 7f15e9800640 1 --2- 172.21.15.27:0/41896517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e410c020 0x7f15e4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.908+0000 7f15e9800640 1 --2- 172.21.15.27:0/41896517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e4105d90 0x7f15e41061a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:09.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.908+0000 7f15e9800640 1 -- 172.21.15.27:0/41896517 >> 172.21.15.27:0/41896517 conn(0x7f15e4101860 msgr2=0x7f15e4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:09.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.908+0000 7f15e9800640 1 -- 172.21.15.27:0/41896517 shutdown_connections 2024-09-21T07:09:09.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.908+0000 7f15e9800640 1 -- 172.21.15.27:0/41896517 wait complete. 2024-09-21T07:09:09.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.909+0000 7f15e9800640 1 Processor -- start 2024-09-21T07:09:09.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.909+0000 7f15e9800640 1 -- start start 2024-09-21T07:09:09.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.909+0000 7f15e9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e4105d90 0x7f15e419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:09.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.909+0000 7f15e9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e410b6d0 0x7f15e419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:09.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15e410c020 0x7f15e41a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:09.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e9800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4115d30 con 0x7f15e4105d90 2024-09-21T07:09:09.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e9800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4115bb0 con 0x7f15e410b6d0 2024-09-21T07:09:09.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e9800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4115eb0 con 0x7f15e410c020 2024-09-21T07:09:09.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e4105d90 0x7f15e419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:09.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e410b6d0 0x7f15e419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:09.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15e410c020 0x7f15e41a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:09.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e4105d90 0x7f15e419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34208/0 (socket says 172.21.15.27:34208) 2024-09-21T07:09:09.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e2a00640 1 -- 172.21.15.27:0/1536361438 learned_addr learned my addr 172.21.15.27:0/1536361438 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e410b6d0 0x7f15e419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56430/0 (socket says 172.21.15.27:56430) 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e3400640 1 -- 172.21.15.27:0/1536361438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e410b6d0 msgr2=0x7f15e419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e3400640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e410b6d0 0x7f15e419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e3400640 1 -- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e4105d90 msgr2=0x7f15e419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e3400640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e4105d90 0x7f15e419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.910+0000 7f15e3400640 1 -- 172.21.15.27:0/1536361438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e41a2750 con 0x7f15e410c020 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.911+0000 7f15e3400640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15e410c020 0x7f15e41a2020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009fd0 tx=0x7f15dc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.911+0000 7f15db400640 1 -- 172.21.15.27:0/1536361438 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00ce20 con 0x7f15e410c020 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.911+0000 7f15db400640 1 -- 172.21.15.27:0/1536361438 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15dc005020 con 0x7f15e410c020 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.911+0000 7f15db400640 1 -- 172.21.15.27:0/1536361438 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc010690 con 0x7f15e410c020 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.911+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41a2a40 con 0x7f15e410c020 2024-09-21T07:09:09.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.911+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41a91a0 con 0x7f15e410c020 2024-09-21T07:09:09.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.913+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e41061a0 con 0x7f15e410c020 2024-09-21T07:09:09.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.915+0000 7f15db400640 1 -- 172.21.15.27:0/1536361438 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f15dc004b20 con 0x7f15e410c020 2024-09-21T07:09:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.916+0000 7f15db400640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15c80b6560 0x7f15c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.916+0000 7f15db400640 1 -- 172.21.15.27:0/1536361438 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f15dc0e7650 con 0x7f15e410c020 2024-09-21T07:09:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.916+0000 7f15e2a00640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15c80b6560 0x7f15c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.917+0000 7f15e2a00640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15c80b6560 0x7f15c80b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f15d40096c0 tx=0x7f15d4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:09.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:09.920+0000 7f15db400640 1 -- 172.21.15.27:0/1536361438 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15dc0ae210 con 0x7f15e410c020 2024-09-21T07:09:09.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:09 smithi153 ceph-mon[34747]: Deploying daemon mds.l on smithi153 2024-09-21T07:09:10.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:09 smithi027 ceph-mon[30779]: Deploying daemon mds.l on smithi153 2024-09-21T07:09:10.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.157+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15e410de50 con 0x7f15e410c020 2024-09-21T07:09:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.158+0000 7f15db400640 1 -- 172.21.15.27:0/1536361438 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f15dc0b25f0 con 0x7f15e410c020 2024-09-21T07:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"btime":"2024-09-21T07:00:45:000882+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T07:09:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 1 2024-09-21T07:09:10.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15c80b6560 msgr2=0x7f15c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:10.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15c80b6560 0x7f15c80b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f15d40096c0 tx=0x7f15d4005a20 comp rx=0 tx=0).stop 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15e410c020 msgr2=0x7f15e41a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15e410c020 0x7f15e41a2020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009fd0 tx=0x7f15dc00ef10 comp rx=0 tx=0).stop 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 shutdown_connections 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15e410c020 0x7f15e41a2020 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15c80b6560 0x7f15c80b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15e410b6d0 0x7f15e419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 --2- 172.21.15.27:0/1536361438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e4105d90 0x7f15e419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.160+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 >> 172.21.15.27:0/1536361438 conn(0x7f15e4101860 msgr2=0x7f15e410ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.161+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 shutdown_connections 2024-09-21T07:09:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:10.161+0000 7f15e9800640 1 -- 172.21.15.27:0/1536361438 wait complete. 2024-09-21T07:09:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:09 smithi119 ceph-mon[34519]: Deploying daemon mds.l on smithi153 2024-09-21T07:09:10.651 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 systemd[1]: Starting Ceph mds.l for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:09:10.913 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 podman[64387]: 2024-09-21 07:09:10.552128124 +0000 UTC m=+0.015650640 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:10.913 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 podman[64387]: 2024-09-21 07:09:10.651741105 +0000 UTC m=+0.115263650 container create 226bdb383465a6ee7d1a1a786dfe5f76fe84343bf7be95f00c11badf76f90732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T07:09:10.913 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 podman[64387]: 2024-09-21 07:09:10.879445174 +0000 UTC m=+0.342967677 container init 226bdb383465a6ee7d1a1a786dfe5f76fe84343bf7be95f00c11badf76f90732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T07:09:10.913 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 podman[64387]: 2024-09-21 07:09:10.882126508 +0000 UTC m=+0.345649012 container start 226bdb383465a6ee7d1a1a786dfe5f76fe84343bf7be95f00c11badf76f90732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:09:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:10 smithi153 ceph-mon[34747]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:11.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:10 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1536361438' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:11.178 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 ceph-mds[64406]: -- 172.21.15.153:0/3099424957 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604f02c7860 con 0x5604f446fa80 2024-09-21T07:09:11.178 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l[64402]: starting mds.l at 2024-09-21T07:09:11.178 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 bash[64387]: 226bdb383465a6ee7d1a1a786dfe5f76fe84343bf7be95f00c11badf76f90732 2024-09-21T07:09:11.178 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:09:10 smithi153 systemd[1]: Started Ceph mds.l for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:10 smithi119 ceph-mon[34519]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:10 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1536361438' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:10 smithi027 ceph-mon[30779]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:10 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1536361438' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.153:6832/4121519709,v1:172.21.15.153:6833/4121519709] up:boot 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: fsmap 1 up:standby 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:11 smithi119 ceph-mon[34519]: Deploying daemon mds.k on smithi119 2024-09-21T07:09:12.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.153:6832/4121519709,v1:172.21.15.153:6833/4121519709] up:boot 2024-09-21T07:09:12.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: fsmap 1 up:standby 2024-09-21T07:09:12.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T07:09:12.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:12.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:12.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:12.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:11 smithi153 ceph-mon[34747]: Deploying daemon mds.k on smithi119 2024-09-21T07:09:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.153:6832/4121519709,v1:172.21.15.153:6833/4121519709] up:boot 2024-09-21T07:09:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: fsmap 1 up:standby 2024-09-21T07:09:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T07:09:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:12.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:12.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:12.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:12.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:11 smithi027 ceph-mon[30779]: Deploying daemon mds.k on smithi119 2024-09-21T07:09:12.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:13.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:13 smithi119 ceph-mon[34519]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:13.290 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 podman[62953]: 2024-09-21 07:09:13.181930642 +0000 UTC m=+0.015637788 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:13.290 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 podman[62953]: 2024-09-21 07:09:13.289145172 +0000 UTC m=+0.122852348 container create ad65fc685cca749149609856a282fe8232868f99bed884c62dd61c7964e52f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:09:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:13 smithi027 ceph-mon[30779]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:13.376 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:13 smithi153 ceph-mon[34747]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 -- 172.21.15.27:0/2109565883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 msgr2=0x7fd770110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 --2- 172.21.15.27:0/2109565883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 0x7fd770110470 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd768009ca0 tx=0x7fd76802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 -- 172.21.15.27:0/2109565883 shutdown_connections 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 --2- 172.21.15.27:0/2109565883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 0x7fd770110470 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 --2- 172.21.15.27:0/2109565883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701092f0 0x7fd770109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 --2- 172.21.15.27:0/2109565883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7701039b0 0x7fd770103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 -- 172.21.15.27:0/2109565883 >> 172.21.15.27:0/2109565883 conn(0x7fd7700ff440 msgr2=0x7fd770101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 -- 172.21.15.27:0/2109565883 shutdown_connections 2024-09-21T07:09:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.949+0000 7fd76fe00640 1 -- 172.21.15.27:0/2109565883 wait complete. 2024-09-21T07:09:13.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.950+0000 7fd76fe00640 1 Processor -- start 2024-09-21T07:09:13.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.950+0000 7fd76fe00640 1 -- start start 2024-09-21T07:09:13.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701039b0 0x7fd770075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7701092f0 0x7fd770075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 0x7fd770076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd770113900 con 0x7fd770109c40 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd770113780 con 0x7fd7701039b0 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd770113a80 con 0x7fd7701092f0 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701039b0 0x7fd770075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 0x7fd770076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701039b0 0x7fd770075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56458/0 (socket says 172.21.15.27:56458) 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 0x7fd770076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34226/0 (socket says 172.21.15.27:34226) 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd76ea00640 1 -- 172.21.15.27:0/964324964 learned_addr learned my addr 172.21.15.27:0/964324964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.951+0000 7fd767e00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7701092f0 0x7fd770075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76ea00640 1 -- 172.21.15.27:0/964324964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7701092f0 msgr2=0x7fd770075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:13.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76ea00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7701092f0 0x7fd770075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76ea00640 1 -- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 msgr2=0x7fd770076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76ea00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 0x7fd770076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76ea00640 1 -- 172.21.15.27:0/964324964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7701a8530 con 0x7fd7701039b0 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76ea00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701039b0 0x7fd770075710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd76000e9f0 tx=0x7fd76000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd774a00640 1 -- 172.21.15.27:0/964324964 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76000cd80 con 0x7fd7701039b0 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd774a00640 1 -- 172.21.15.27:0/964324964 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7600108e0 con 0x7fd7701039b0 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7701a8820 con 0x7fd7701039b0 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd774a00640 1 -- 172.21.15.27:0/964324964 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760018980 con 0x7fd7701039b0 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.952+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7701a8db0 con 0x7fd7701039b0 2024-09-21T07:09:13.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.954+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd770103dc0 con 0x7fd7701039b0 2024-09-21T07:09:13.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.956+0000 7fd774a00640 1 -- 172.21.15.27:0/964324964 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fd760010430 con 0x7fd7701039b0 2024-09-21T07:09:13.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.957+0000 7fd774a00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7540b65e0 0x7fd7540b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.957+0000 7fd774a00640 1 -- 172.21.15.27:0/964324964 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fd7600e72f0 con 0x7fd7701039b0 2024-09-21T07:09:13.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.958+0000 7fd767e00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7540b65e0 0x7fd7540b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.958+0000 7fd767e00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7540b65e0 0x7fd7540b8af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd770102d90 tx=0x7fd758005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:13.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:13.961+0000 7fd774a00640 1 -- 172.21.15.27:0/964324964 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7600b2290 con 0x7fd7701039b0 2024-09-21T07:09:14.034 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 podman[62953]: 2024-09-21 07:09:13.541767527 +0000 UTC m=+0.375474665 container init ad65fc685cca749149609856a282fe8232868f99bed884c62dd61c7964e52f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-21T07:09:14.035 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 podman[62953]: 2024-09-21 07:09:13.54494792 +0000 UTC m=+0.378655060 container start ad65fc685cca749149609856a282fe8232868f99bed884c62dd61c7964e52f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T07:09:14.035 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 ceph-mds[62972]: -- 172.21.15.119:0/444000311 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56497c6ef860 con 0x564980897180 2024-09-21T07:09:14.035 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k[62968]: starting mds.k at 2024-09-21T07:09:14.035 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 bash[62953]: ad65fc685cca749149609856a282fe8232868f99bed884c62dd61c7964e52f9e 2024-09-21T07:09:14.035 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:09:13 smithi119 systemd[1]: Started Ceph mds.k for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.210+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd77002e640 con 0x7fd7701039b0 2024-09-21T07:09:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.211+0000 7fd774a00640 1 -- 172.21.15.27:0/964324964 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7fd7600b71d0 con 0x7fd7701039b0 2024-09-21T07:09:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:14.213 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"btime":"2024-09-21T07:09:14:039172+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T07:09:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.213+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7540b65e0 msgr2=0x7fd7540b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.213+0000 7fd76fe00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7540b65e0 0x7fd7540b8af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd770102d90 tx=0x7fd758005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.213+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701039b0 msgr2=0x7fd770075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.213+0000 7fd76fe00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701039b0 0x7fd770075710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd76000e9f0 tx=0x7fd76000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.213+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 shutdown_connections 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.214+0000 7fd76fe00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7701092f0 0x7fd770075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.214+0000 7fd76fe00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7540b65e0 0x7fd7540b8af0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.214+0000 7fd76fe00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd770109c40 0x7fd770076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.214+0000 7fd76fe00640 1 --2- 172.21.15.27:0/964324964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7701039b0 0x7fd770075710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.214+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 >> 172.21.15.27:0/964324964 conn(0x7fd7700ff440 msgr2=0x7fd770106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.214+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 shutdown_connections 2024-09-21T07:09:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:14.214+0000 7fd76fe00640 1 -- 172.21.15.27:0/964324964 wait complete. 2024-09-21T07:09:14.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:14.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:14.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:14.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:14.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:14.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:14.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:14.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: Deploying daemon mds.j on smithi027 2024-09-21T07:09:14.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6832/1660738486,v1:172.21.15.119:6833/1660738486] up:boot 2024-09-21T07:09:14.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: fsmap 2 up:standby 2024-09-21T07:09:14.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T07:09:14.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:14 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/964324964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: Deploying daemon mds.j on smithi027 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6832/1660738486,v1:172.21.15.119:6833/1660738486] up:boot 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: fsmap 2 up:standby 2024-09-21T07:09:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T07:09:15.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:14 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/964324964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: Deploying daemon mds.j on smithi027 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6832/1660738486,v1:172.21.15.119:6833/1660738486] up:boot 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: fsmap 2 up:standby 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T07:09:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:14 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/964324964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:16.771 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:16 smithi027 ceph-mon[30779]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:17.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:16 smithi119 ceph-mon[34519]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:17.091 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:16 smithi027 systemd[1]: Starting Ceph mds.j for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:09:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:16 smithi153 ceph-mon[34747]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:17.271 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:17.362 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 podman[72314]: 2024-09-21 07:09:16.995320974 +0000 UTC m=+0.028576026 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:17.362 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 podman[72314]: 2024-09-21 07:09:17.107360325 +0000 UTC m=+0.140615376 container create 71965bfaefa825ebf7ac6be4c29a4472403ed77d9a8b86a69e55880215a643f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T07:09:17.362 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 podman[72314]: 2024-09-21 07:09:17.360577579 +0000 UTC m=+0.393832639 container init 71965bfaefa825ebf7ac6be4c29a4472403ed77d9a8b86a69e55880215a643f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:09:17.689 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:17.831 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 podman[72314]: 2024-09-21 07:09:17.367780077 +0000 UTC m=+0.401035132 container start 71965bfaefa825ebf7ac6be4c29a4472403ed77d9a8b86a69e55880215a643f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T07:09:17.831 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 ceph-mds[72351]: -- 172.21.15.27:0/790850954 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570c4439860 con 0x5570c85e1600 2024-09-21T07:09:17.831 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 bash[72314]: 71965bfaefa825ebf7ac6be4c29a4472403ed77d9a8b86a69e55880215a643f9 2024-09-21T07:09:17.831 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j[72329]: starting mds.j at 2024-09-21T07:09:17.831 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:09:17 smithi027 systemd[1]: Started Ceph mds.j for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:18.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.315+0000 7fbde3e00640 1 -- 172.21.15.27:0/2834836604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde410fff0 msgr2=0x7fbde4112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:18.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.315+0000 7fbde3e00640 1 --2- 172.21.15.27:0/2834836604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde410fff0 0x7fbde4112410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009be0 tx=0x7fbddc02a750 comp rx=0 tx=0).stop 2024-09-21T07:09:18.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.316+0000 7fbde3e00640 1 -- 172.21.15.27:0/2834836604 shutdown_connections 2024-09-21T07:09:18.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.316+0000 7fbde3e00640 1 --2- 172.21.15.27:0/2834836604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde410fff0 0x7fbde4112410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.316+0000 7fbde3e00640 1 --2- 172.21.15.27:0/2834836604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102d90 0x7fbde410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.316+0000 7fbde3e00640 1 --2- 172.21.15.27:0/2834836604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde4102440 0x7fbde4102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.316+0000 7fbde3e00640 1 -- 172.21.15.27:0/2834836604 >> 172.21.15.27:0/2834836604 conn(0x7fbde40fe2c0 msgr2=0x7fbde4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:18.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.316+0000 7fbde3e00640 1 -- 172.21.15.27:0/2834836604 shutdown_connections 2024-09-21T07:09:18.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.316+0000 7fbde3e00640 1 -- 172.21.15.27:0/2834836604 wait complete. 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.317+0000 7fbde3e00640 1 Processor -- start 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.317+0000 7fbde3e00640 1 -- start start 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102440 0x7fbde419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde4102d90 0x7fbde419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410fff0 0x7fbde41a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4115a90 con 0x7fbde4102440 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4115910 con 0x7fbde4102d90 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4115c10 con 0x7fbde410fff0 2024-09-21T07:09:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102440 0x7fbde419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:18.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102440 0x7fbde419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50138/0 (socket says 172.21.15.27:50138) 2024-09-21T07:09:18.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde2a00640 1 -- 172.21.15.27:0/577410806 learned_addr learned my addr 172.21.15.27:0/577410806 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:18.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde2000640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde4102d90 0x7fbde419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:18.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.318+0000 7fbde3400640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410fff0 0x7fbde41a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:18.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbde2a00640 1 -- 172.21.15.27:0/577410806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410fff0 msgr2=0x7fbde41a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbde2a00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410fff0 0x7fbde41a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbde2a00640 1 -- 172.21.15.27:0/577410806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde4102d90 msgr2=0x7fbde419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbde2a00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde4102d90 0x7fbde419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbde2a00640 1 -- 172.21.15.27:0/577410806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde41a23b0 con 0x7fbde4102440 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbde2a00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102440 0x7fbde419c440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e9f0 tx=0x7fbdd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbddb400640 1 -- 172.21.15.27:0/577410806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400cd80 con 0x7fbde4102440 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde41a26a0 con 0x7fbde4102440 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbddb400640 1 -- 172.21.15.27:0/577410806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd40054d0 con 0x7fbde4102440 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.319+0000 7fbddb400640 1 -- 172.21.15.27:0/577410806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4010690 con 0x7fbde4102440 2024-09-21T07:09:18.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.320+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde41a8e00 con 0x7fbde4102440 2024-09-21T07:09:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.321+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde4102850 con 0x7fbde4102440 2024-09-21T07:09:18.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.323+0000 7fbddb400640 1 -- 172.21.15.27:0/577410806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fbdd4004e90 con 0x7fbde4102440 2024-09-21T07:09:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.326+0000 7fbddb400640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbdc80b6560 0x7fbdc80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.326+0000 7fbddb400640 1 -- 172.21.15.27:0/577410806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fbdd40e7410 con 0x7fbde4102440 2024-09-21T07:09:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.326+0000 7fbde2000640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbdc80b6560 0x7fbdc80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.327+0000 7fbde2000640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbdc80b6560 0x7fbdc80b8a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbde4104260 tx=0x7fbdd0009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.329+0000 7fbddb400640 1 -- 172.21.15.27:0/577410806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbdd40adfd0 con 0x7fbde4102440 2024-09-21T07:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.570+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbde419d570 con 0x7fbde4102440 2024-09-21T07:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.571+0000 7fbddb400640 1 -- 172.21.15.27:0/577410806 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7fbdd40b23b0 con 0x7fbde4102440 2024-09-21T07:09:18.573 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:18.573 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"btime":"2024-09-21T07:09:17:687689+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-09-21T07:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.573+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbdc80b6560 msgr2=0x7fbdc80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbdc80b6560 0x7fbdc80b8a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbde4104260 tx=0x7fbdd0009260 comp rx=0 tx=0).stop 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102440 msgr2=0x7fbde419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102440 0x7fbde419c440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e9f0 tx=0x7fbdd400edf0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 shutdown_connections 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410fff0 0x7fbde41a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbdc80b6560 0x7fbdc80b8a70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbde4102d90 0x7fbde419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 --2- 172.21.15.27:0/577410806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde4102440 0x7fbde419c440 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 >> 172.21.15.27:0/577410806 conn(0x7fbde40fe2c0 msgr2=0x7fbde41120d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 shutdown_connections 2024-09-21T07:09:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:18.574+0000 7fbde3e00640 1 -- 172.21.15.27:0/577410806 wait complete. 2024-09-21T07:09:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: Deploying daemon mds.d on smithi027 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.27:6834/2994040514,v1:172.21.15.27:6835/2994040514] up:boot 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: fsmap 3 up:standby 2024-09-21T07:09:18.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:18 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T07:09:18.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: Deploying daemon mds.d on smithi027 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.27:6834/2994040514,v1:172.21.15.27:6835/2994040514] up:boot 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: fsmap 3 up:standby 2024-09-21T07:09:18.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:18 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: Deploying daemon mds.d on smithi027 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.27:6834/2994040514,v1:172.21.15.27:6835/2994040514] up:boot 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: fsmap 3 up:standby 2024-09-21T07:09:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:18 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T07:09:19.726 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:19 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/577410806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:19 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/577410806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:20.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:19 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/577410806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:20 smithi027 ceph-mon[30779]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:20.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:20 smithi153 ceph-mon[34747]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:20 smithi119 ceph-mon[34519]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:21.127 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:09:20 smithi027 podman[72691]: 2024-09-21 07:09:20.889286734 +0000 UTC m=+0.163002166 container create ac72738df547f0d54a7716374ca0a12aab6ca44f2470c9ded6fcc17f1a49d83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T07:09:21.127 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:09:21 smithi027 podman[72691]: 2024-09-21 07:09:21.125556762 +0000 UTC m=+0.399272204 container init ac72738df547f0d54a7716374ca0a12aab6ca44f2470c9ded6fcc17f1a49d83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T07:09:21.591 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:09:21 smithi027 podman[72691]: 2024-09-21 07:09:21.132106233 +0000 UTC m=+0.405821674 container start ac72738df547f0d54a7716374ca0a12aab6ca44f2470c9ded6fcc17f1a49d83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T07:09:21.592 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:09:21 smithi027 bash[72691]: ac72738df547f0d54a7716374ca0a12aab6ca44f2470c9ded6fcc17f1a49d83b 2024-09-21T07:09:21.592 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:09:21 smithi027 systemd[1]: Started Ceph mds.d for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:21.592 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:09:21 smithi027 ceph-mds[72710]: -- 172.21.15.27:0/467722429 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56327bf95860 con 0x56328013d600 2024-09-21T07:09:21.592 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:09:21 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d[72706]: starting mds.d at 2024-09-21T07:09:22.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:22.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:22.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:22.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:22.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: Deploying daemon mds.e on smithi119 2024-09-21T07:09:22.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:22.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.27:6836/2900670620,v1:172.21.15.27:6837/2900670620] up:boot 2024-09-21T07:09:22.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: fsmap 4 up:standby 2024-09-21T07:09:22.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:22 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T07:09:22.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: Deploying daemon mds.e on smithi119 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.27:6836/2900670620,v1:172.21.15.27:6837/2900670620] up:boot 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: fsmap 4 up:standby 2024-09-21T07:09:22.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:22 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T07:09:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:22.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:22.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: Deploying daemon mds.e on smithi119 2024-09-21T07:09:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.27:6836/2900670620,v1:172.21.15.27:6837/2900670620] up:boot 2024-09-21T07:09:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: fsmap 4 up:standby 2024-09-21T07:09:22.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:22 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 -- 172.21.15.27:0/13923228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8103b80 msgr2=0x7f5bc8103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 --2- 172.21.15.27:0/13923228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8103b80 0x7f5bc8103f90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009900 tx=0x7f5bb802a410 comp rx=0 tx=0).stop 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 -- 172.21.15.27:0/13923228 shutdown_connections 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 --2- 172.21.15.27:0/13923228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bc8109e10 0x7f5bc8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 --2- 172.21.15.27:0/13923228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc81094c0 0x7f5bc81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 --2- 172.21.15.27:0/13923228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8103b80 0x7f5bc8103f90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 -- 172.21.15.27:0/13923228 >> 172.21.15.27:0/13923228 conn(0x7f5bc80ff610 msgr2=0x7f5bc8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 -- 172.21.15.27:0/13923228 shutdown_connections 2024-09-21T07:09:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.329+0000 7f5bcf400640 1 -- 172.21.15.27:0/13923228 wait complete. 2024-09-21T07:09:23.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.330+0000 7f5bcf400640 1 Processor -- start 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.330+0000 7f5bcf400640 1 -- start start 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc8103b80 0x7f5bc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bc81094c0 0x7f5bc8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8109e10 0x7f5bc8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcf400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8113b20 con 0x7f5bc8109e10 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcf400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81139a0 con 0x7f5bc8103b80 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcf400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8113ca0 con 0x7f5bc81094c0 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc8103b80 0x7f5bc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc8103b80 0x7f5bc8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55524/0 (socket says 172.21.15.27:55524) 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcca00640 1 -- 172.21.15.27:0/3056577016 learned_addr learned my addr 172.21.15.27:0/3056577016 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bbfe00640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bc81094c0 0x7f5bc8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.331+0000 7f5bcd400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8109e10 0x7f5bc8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:23.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcca00640 1 -- 172.21.15.27:0/3056577016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bc81094c0 msgr2=0x7f5bc8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcca00640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bc81094c0 0x7f5bc8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcca00640 1 -- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8109e10 msgr2=0x7f5bc8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcca00640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8109e10 0x7f5bc8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcca00640 1 -- 172.21.15.27:0/3056577016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc81a87f0 con 0x7f5bc8103b80 2024-09-21T07:09:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcd400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8109e10 0x7f5bc8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:09:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcca00640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc8103b80 0x7f5bc8075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005ab0 tx=0x7f5bb8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bc6000640 1 -- 172.21.15.27:0/3056577016 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb803f070 con 0x7f5bc8103b80 2024-09-21T07:09:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81a8a80 con 0x7f5bc8103b80 2024-09-21T07:09:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bc6000640 1 -- 172.21.15.27:0/3056577016 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb8031b60 con 0x7f5bc8103b80 2024-09-21T07:09:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.332+0000 7f5bc6000640 1 -- 172.21.15.27:0/3056577016 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8035030 con 0x7f5bc8103b80 2024-09-21T07:09:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.333+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a8f60 con 0x7f5bc8103b80 2024-09-21T07:09:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.334+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc8104e00 con 0x7f5bc8103b80 2024-09-21T07:09:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.336+0000 7f5bc6000640 1 -- 172.21.15.27:0/3056577016 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f5bb8042070 con 0x7f5bc8103b80 2024-09-21T07:09:23.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.337+0000 7f5bc6000640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bac0b6560 0x7f5bac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.338+0000 7f5bc6000640 1 -- 172.21.15.27:0/3056577016 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f5bb8107b70 con 0x7f5bc8103b80 2024-09-21T07:09:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.338+0000 7f5bbfe00640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bac0b6560 0x7f5bac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.338+0000 7f5bbfe00640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bac0b6560 0x7f5bac0b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8102d10 tx=0x7f5bb0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:23.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.339+0000 7f5bc6000640 1 -- 172.21.15.27:0/3056577016 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5bb80ce730 con 0x7f5bc8103b80 2024-09-21T07:09:23.456 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 systemd[1]: Starting Ceph mds.e for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:09:23.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.578+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bc8076a10 con 0x7f5bc8103b80 2024-09-21T07:09:23.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.579+0000 7f5bc6000640 1 -- 172.21.15.27:0/3056577016 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f5bb80d2b10 con 0x7f5bc8103b80 2024-09-21T07:09:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:23.581 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"btime":"2024-09-21T07:09:21:575199+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-09-21T07:09:23.581 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 5 2024-09-21T07:09:23.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.582+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bac0b6560 msgr2=0x7f5bac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:23.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.582+0000 7f5bcf400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bac0b6560 0x7f5bac0b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8102d10 tx=0x7f5bb0005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:23.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.582+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc8103b80 msgr2=0x7f5bc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:23.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.582+0000 7f5bcf400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc8103b80 0x7f5bc8075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005ab0 tx=0x7f5bb8003c10 comp rx=0 tx=0).stop 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 shutdown_connections 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bc81094c0 0x7f5bc8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5bac0b6560 0x7f5bac0b8a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc8109e10 0x7f5bc8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 --2- 172.21.15.27:0/3056577016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bc8103b80 0x7f5bc8075710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 >> 172.21.15.27:0/3056577016 conn(0x7f5bc80ff610 msgr2=0x7f5bc8108b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 shutdown_connections 2024-09-21T07:09:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:23.583+0000 7f5bcf400640 1 -- 172.21.15.27:0/3056577016 wait complete. 2024-09-21T07:09:23.727 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 podman[63234]: 2024-09-21 07:09:23.362462714 +0000 UTC m=+0.019755672 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:23.727 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 podman[63234]: 2024-09-21 07:09:23.470147112 +0000 UTC m=+0.127440094 container create 0e099bd5a490a6be184f7d99fdce7b746aace8bfc3bda32c15fee84f193c110f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T07:09:23.727 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 podman[63234]: 2024-09-21 07:09:23.689391363 +0000 UTC m=+0.346684307 container init 0e099bd5a490a6be184f7d99fdce7b746aace8bfc3bda32c15fee84f193c110f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T07:09:23.727 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 podman[63234]: 2024-09-21 07:09:23.692269701 +0000 UTC m=+0.349562644 container start 0e099bd5a490a6be184f7d99fdce7b746aace8bfc3bda32c15fee84f193c110f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T07:09:24.034 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 ceph-mds[63253]: -- 172.21.15.119:0/2840660903 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558fc50d1860 con 0x558fc9279a80 2024-09-21T07:09:24.034 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e[63249]: starting mds.e at 2024-09-21T07:09:24.034 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 bash[63234]: 0e099bd5a490a6be184f7d99fdce7b746aace8bfc3bda32c15fee84f193c110f 2024-09-21T07:09:24.035 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:09:23 smithi119 systemd[1]: Started Ceph mds.e for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3056577016' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:24.509 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:24.510 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:24 smithi153 ceph-mon[34747]: Deploying daemon mds.f on smithi153 2024-09-21T07:09:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3056577016' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:24.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:24.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:24 smithi027 ceph-mon[30779]: Deploying daemon mds.f on smithi153 2024-09-21T07:09:24.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3056577016' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:24.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:24 smithi119 ceph-mon[34519]: Deploying daemon mds.f on smithi153 2024-09-21T07:09:25.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:25 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6834/3894045960,v1:172.21.15.119:6835/3894045960] up:boot 2024-09-21T07:09:25.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:25 smithi153 ceph-mon[34747]: fsmap 5 up:standby 2024-09-21T07:09:25.539 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:25 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:09:25.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:25 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6834/3894045960,v1:172.21.15.119:6835/3894045960] up:boot 2024-09-21T07:09:25.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:25 smithi119 ceph-mon[34519]: fsmap 5 up:standby 2024-09-21T07:09:25.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:25 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:09:25.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:25 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6834/3894045960,v1:172.21.15.119:6835/3894045960] up:boot 2024-09-21T07:09:25.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:25 smithi027 ceph-mon[30779]: fsmap 5 up:standby 2024-09-21T07:09:25.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:25 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:09:25.927 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:25 smithi153 systemd[1]: Starting Ceph mds.f for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:09:26.210 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:25 smithi153 podman[64657]: 2024-09-21 07:09:25.842751697 +0000 UTC m=+0.016347328 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:26.210 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:25 smithi153 podman[64657]: 2024-09-21 07:09:25.981452774 +0000 UTC m=+0.155048395 container create 69145071ef92c505c2fceb3df6b9ec17083472bc208d92afe9b2cc0b106747a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T07:09:26.210 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:26 smithi153 podman[64657]: 2024-09-21 07:09:26.209428481 +0000 UTC m=+0.383024103 container init 69145071ef92c505c2fceb3df6b9ec17083472bc208d92afe9b2cc0b106747a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T07:09:26.291 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mon[34747]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:26.678 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:26 smithi153 podman[64657]: 2024-09-21 07:09:26.212262581 +0000 UTC m=+0.385858203 container start 69145071ef92c505c2fceb3df6b9ec17083472bc208d92afe9b2cc0b106747a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T07:09:26.679 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-mds[64676]: -- 172.21.15.153:0/2556060760 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b94489860 con 0x558b98631600 2024-09-21T07:09:26.679 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:26 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f[64672]: starting mds.f at 2024-09-21T07:09:26.679 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:26 smithi153 bash[64657]: 69145071ef92c505c2fceb3df6b9ec17083472bc208d92afe9b2cc0b106747a5 2024-09-21T07:09:26.679 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:09:26 smithi153 systemd[1]: Started Ceph mds.f for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:26.692 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:26.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:26 smithi119 ceph-mon[34519]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:26.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:26 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:26 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:26 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:26 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:26.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:26 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:26.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:26 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:26.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:26 smithi027 ceph-mon[30779]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:26.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:26 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:26 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:26 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:26.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:26 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:26.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:26 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:26.833 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:26 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:27.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 -- 172.21.15.27:0/2350881366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810be50 msgr2=0x7ff468112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 --2- 172.21.15.27:0/2350881366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810be50 0x7ff468112680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff460009ca0 tx=0x7ff46002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 -- 172.21.15.27:0/2350881366 shutdown_connections 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 --2- 172.21.15.27:0/2350881366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff46810b500 0x7ff46810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 --2- 172.21.15.27:0/2350881366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810be50 0x7ff468112680 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 --2- 172.21.15.27:0/2350881366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff468105bc0 0x7ff468105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 -- 172.21.15.27:0/2350881366 >> 172.21.15.27:0/2350881366 conn(0x7ff468101670 msgr2=0x7ff468103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 -- 172.21.15.27:0/2350881366 shutdown_connections 2024-09-21T07:09:27.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.320+0000 7ff467200640 1 -- 172.21.15.27:0/2350881366 wait complete. 2024-09-21T07:09:27.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.321+0000 7ff467200640 1 Processor -- start 2024-09-21T07:09:27.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.321+0000 7ff467200640 1 -- start start 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff467200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff468105bc0 0x7ff46819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff467200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810b500 0x7ff46819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff467200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46810be50 0x7ff4681a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff467200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff468115b10 con 0x7ff46810b500 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff467200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff468115990 con 0x7ff46810be50 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff467200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff468115c90 con 0x7ff468105bc0 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810b500 0x7ff46819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff468105bc0 0x7ff46819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810b500 0x7ff46819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50194/0 (socket says 172.21.15.27:50194) 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465400640 1 -- 172.21.15.27:0/1523623715 learned_addr learned my addr 172.21.15.27:0/1523623715 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff466800640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46810be50 0x7ff4681a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465e00640 1 -- 172.21.15.27:0/1523623715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46810be50 msgr2=0x7ff4681a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465e00640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46810be50 0x7ff4681a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465e00640 1 -- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810b500 msgr2=0x7ff46819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465e00640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810b500 0x7ff46819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.322+0000 7ff465e00640 1 -- 172.21.15.27:0/1523623715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681a2530 con 0x7ff468105bc0 2024-09-21T07:09:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.323+0000 7ff465e00640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff468105bc0 0x7ff46819c5c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e9f0 tx=0x7ff45800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.323+0000 7ff45ea00640 1 -- 172.21.15.27:0/1523623715 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45800cd80 con 0x7ff468105bc0 2024-09-21T07:09:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.323+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4681a2820 con 0x7ff468105bc0 2024-09-21T07:09:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.323+0000 7ff45ea00640 1 -- 172.21.15.27:0/1523623715 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4580035f0 con 0x7ff468105bc0 2024-09-21T07:09:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.323+0000 7ff45ea00640 1 -- 172.21.15.27:0/1523623715 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458010690 con 0x7ff468105bc0 2024-09-21T07:09:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.323+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4681a8f80 con 0x7ff468105bc0 2024-09-21T07:09:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.324+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff468105fd0 con 0x7ff468105bc0 2024-09-21T07:09:27.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.327+0000 7ff45ea00640 1 -- 172.21.15.27:0/1523623715 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7ff4580037a0 con 0x7ff468105bc0 2024-09-21T07:09:27.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.328+0000 7ff45ea00640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff44c0b6560 0x7ff44c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:27.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.329+0000 7ff45ea00640 1 -- 172.21.15.27:0/1523623715 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7ff4580e7400 con 0x7ff468105bc0 2024-09-21T07:09:27.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.329+0000 7ff465400640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff44c0b6560 0x7ff44c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:27.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.329+0000 7ff465400640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff44c0b6560 0x7ff44c0b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff468104d70 tx=0x7ff454005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:27.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.330+0000 7ff45ea00640 1 -- 172.21.15.27:0/1523623715 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4580ae050 con 0x7ff468105bc0 2024-09-21T07:09:27.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.565+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff46819d6f0 con 0x7ff468105bc0 2024-09-21T07:09:27.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.566+0000 7ff45ea00640 1 -- 172.21.15.27:0/1523623715 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7ff4580b2430 con 0x7ff468105bc0 2024-09-21T07:09:27.569 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:27.569 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"btime":"2024-09-21T07:09:26:424687+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24713,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T07:09:27.569 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2024-09-21T07:09:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff44c0b6560 msgr2=0x7ff44c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff44c0b6560 0x7ff44c0b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff468104d70 tx=0x7ff454005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff468105bc0 msgr2=0x7ff46819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff468105bc0 0x7ff46819c5c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e9f0 tx=0x7ff45800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 shutdown_connections 2024-09-21T07:09:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff44c0b6560 0x7ff44c0b8a70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff46810be50 0x7ff4681a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff46810b500 0x7ff46819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 --2- 172.21.15.27:0/1523623715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff468105bc0 0x7ff46819c5c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 >> 172.21.15.27:0/1523623715 conn(0x7ff468101670 msgr2=0x7ff46810ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 shutdown_connections 2024-09-21T07:09:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:27.569+0000 7ff467200640 1 -- 172.21.15.27:0/1523623715 wait complete. 2024-09-21T07:09:27.761 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:27 smithi153 ceph-mon[34747]: Deploying daemon mds.c on smithi153 2024-09-21T07:09:27.761 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:27 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.153:6834/717203459,v1:172.21.15.153:6835/717203459] up:boot 2024-09-21T07:09:27.761 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:27 smithi153 ceph-mon[34747]: fsmap 6 up:standby 2024-09-21T07:09:27.761 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:27 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T07:09:27.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:27 smithi119 ceph-mon[34519]: Deploying daemon mds.c on smithi153 2024-09-21T07:09:27.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:27 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.153:6834/717203459,v1:172.21.15.153:6835/717203459] up:boot 2024-09-21T07:09:27.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:27 smithi119 ceph-mon[34519]: fsmap 6 up:standby 2024-09-21T07:09:27.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:27 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T07:09:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:27 smithi027 ceph-mon[30779]: Deploying daemon mds.c on smithi153 2024-09-21T07:09:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:27 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.153:6834/717203459,v1:172.21.15.153:6835/717203459] up:boot 2024-09-21T07:09:27.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:27 smithi027 ceph-mon[30779]: fsmap 6 up:standby 2024-09-21T07:09:27.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:27 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T07:09:28.428 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 systemd[1]: Starting Ceph mds.c for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:09:28.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:28 smithi027 ceph-mon[30779]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:28.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:28 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1523623715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:28.719 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:28 smithi153 ceph-mon[34747]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:28.719 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:28 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1523623715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:28.719 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 podman[64927]: 2024-09-21 07:09:28.351285933 +0000 UTC m=+0.015732736 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:28.719 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 podman[64927]: 2024-09-21 07:09:28.470637967 +0000 UTC m=+0.135084762 container create bc8dcfc1d30f1fd7bd9c54985150d98926353f015b642f7c92e495d7fee8644f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T07:09:28.719 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 podman[64927]: 2024-09-21 07:09:28.681889264 +0000 UTC m=+0.346336060 container init bc8dcfc1d30f1fd7bd9c54985150d98926353f015b642f7c92e495d7fee8644f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T07:09:28.719 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 podman[64927]: 2024-09-21 07:09:28.684618672 +0000 UTC m=+0.349065468 container start bc8dcfc1d30f1fd7bd9c54985150d98926353f015b642f7c92e495d7fee8644f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:09:28.720 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 ceph-mds[64946]: -- 172.21.15.153:0/2154666340 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5566ec4b1860 con 0x5566f0659a80 2024-09-21T07:09:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:28 smithi119 ceph-mon[34519]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:28 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1523623715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:29.177 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c[64942]: starting mds.c at 2024-09-21T07:09:29.178 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 bash[64927]: bc8dcfc1d30f1fd7bd9c54985150d98926353f015b642f7c92e495d7fee8644f 2024-09-21T07:09:29.178 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:09:28 smithi153 systemd[1]: Started Ceph mds.c for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:30.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:30.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:30.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: Deploying daemon mds.h on smithi119 2024-09-21T07:09:30.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.153:6836/1555665457,v1:172.21.15.153:6837/1555665457] up:boot 2024-09-21T07:09:30.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: fsmap 7 up:standby 2024-09-21T07:09:30.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:29 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:09:30.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: Deploying daemon mds.h on smithi119 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.153:6836/1555665457,v1:172.21.15.153:6837/1555665457] up:boot 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: fsmap 7 up:standby 2024-09-21T07:09:30.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:29 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:09:30.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:30.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:30.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:30.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:30.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: Deploying daemon mds.h on smithi119 2024-09-21T07:09:30.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.153:6836/1555665457,v1:172.21.15.153:6837/1555665457] up:boot 2024-09-21T07:09:30.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: fsmap 7 up:standby 2024-09-21T07:09:30.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:29 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:09:30.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:30.729 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:30 smithi119 ceph-mon[34519]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:31.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:30 smithi027 ceph-mon[30779]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:30 smithi153 ceph-mon[34747]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:31.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 -- 172.21.15.27:0/2637216026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810114740 msgr2=0x7f5810116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 --2- 172.21.15.27:0/2637216026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810114740 0x7f5810116b60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5808009ca0 tx=0x7f580802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 -- 172.21.15.27:0/2637216026 shutdown_connections 2024-09-21T07:09:31.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 --2- 172.21.15.27:0/2637216026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810114740 0x7f5810116b60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 --2- 172.21.15.27:0/2637216026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5810078290 0x7f5810114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 --2- 172.21.15.27:0/2637216026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5810077940 0x7f5810077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 -- 172.21.15.27:0/2637216026 >> 172.21.15.27:0/2637216026 conn(0x7f5810101810 msgr2=0x7f5810103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 -- 172.21.15.27:0/2637216026 shutdown_connections 2024-09-21T07:09:31.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.307+0000 7f5817600640 1 -- 172.21.15.27:0/2637216026 wait complete. 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.308+0000 7f5817600640 1 Processor -- start 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.308+0000 7f5817600640 1 -- start start 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5817600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5810077940 0x7f58101a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5817600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810078290 0x7f58101a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5817600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5810114740 0x7f58101a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5817600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f581011a0c0 con 0x7f5810078290 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5817600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5810119f40 con 0x7f5810077940 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5817600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f581011a240 con 0x7f5810114740 2024-09-21T07:09:31.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5807e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810078290 0x7f58101a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5807e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810078290 0x7f58101a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34902/0 (socket says 172.21.15.27:34902) 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5807e00640 1 -- 172.21.15.27:0/1052273849 learned_addr learned my addr 172.21.15.27:0/1052273849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5814c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5810077940 0x7f58101a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.309+0000 7f5815600640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5810114740 0x7f58101a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f5814c00640 1 -- 172.21.15.27:0/1052273849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5810114740 msgr2=0x7f58101a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f5814c00640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5810114740 0x7f58101a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f5814c00640 1 -- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810078290 msgr2=0x7f58101a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f5814c00640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810078290 0x7f58101a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f5814c00640 1 -- 172.21.15.27:0/1052273849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58101a69e0 con 0x7f5810077940 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f5807e00640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810078290 0x7f58101a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:31.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f5814c00640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5810077940 0x7f58101a0a70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f580000e9f0 tx=0x7f580000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.310+0000 7f580e000640 1 -- 172.21.15.27:0/1052273849 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580000ccc0 con 0x7f5810077940 2024-09-21T07:09:31.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.311+0000 7f580e000640 1 -- 172.21.15.27:0/1052273849 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800005020 con 0x7f5810077940 2024-09-21T07:09:31.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.311+0000 7f580e000640 1 -- 172.21.15.27:0/1052273849 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800004a00 con 0x7f5810077940 2024-09-21T07:09:31.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.311+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58101a6cd0 con 0x7f5810077940 2024-09-21T07:09:31.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.311+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58101ad430 con 0x7f5810077940 2024-09-21T07:09:31.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.312+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5810077d50 con 0x7f5810077940 2024-09-21T07:09:31.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.315+0000 7f580e000640 1 -- 172.21.15.27:0/1052273849 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f5800004ba0 con 0x7f5810077940 2024-09-21T07:09:31.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.316+0000 7f580e000640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57f40b65e0 0x7f57f40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.316+0000 7f580e000640 1 -- 172.21.15.27:0/1052273849 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f58000e6ff0 con 0x7f5810077940 2024-09-21T07:09:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.316+0000 7f5807e00640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57f40b65e0 0x7f57f40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.317+0000 7f5807e00640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57f40b65e0 0x7f57f40b8af0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f581007b210 tx=0x7f57f8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.318+0000 7f580e000640 1 -- 172.21.15.27:0/1052273849 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f58000adc40 con 0x7f5810077940 2024-09-21T07:09:31.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.559+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58101a1ba0 con 0x7f5810077940 2024-09-21T07:09:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.560+0000 7f580e000640 1 -- 172.21.15.27:0/1052273849 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f58000b4070 con 0x7f5810077940 2024-09-21T07:09:31.562 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:31.562 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"btime":"2024-09-21T07:09:29:439587+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24713,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T07:09:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2024-09-21T07:09:31.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57f40b65e0 msgr2=0x7f57f40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57f40b65e0 0x7f57f40b8af0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f581007b210 tx=0x7f57f8005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5810077940 msgr2=0x7f58101a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5810077940 0x7f58101a0a70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f580000e9f0 tx=0x7f580000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 shutdown_connections 2024-09-21T07:09:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5810114740 0x7f58101a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57f40b65e0 0x7f57f40b8af0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5810078290 0x7f58101a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 --2- 172.21.15.27:0/1052273849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5810077940 0x7f58101a0a70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 >> 172.21.15.27:0/1052273849 conn(0x7f5810101810 msgr2=0x7f5810116760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 shutdown_connections 2024-09-21T07:09:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:31.563+0000 7f5817600640 1 -- 172.21.15.27:0/1052273849 wait complete. 2024-09-21T07:09:31.784 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:09:31 smithi119 systemd[1]: Started Ceph mds.h for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:32.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: Deploying daemon mds.i on smithi153 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1052273849' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] up:boot 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: fsmap 8 up:standby 2024-09-21T07:09:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:32 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: Deploying daemon mds.i on smithi153 2024-09-21T07:09:32.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1052273849' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] up:boot 2024-09-21T07:09:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: fsmap 8 up:standby 2024-09-21T07:09:32.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:32 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: Deploying daemon mds.i on smithi153 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1052273849' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:32.887 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] up:boot 2024-09-21T07:09:32.888 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: fsmap 8 up:standby 2024-09-21T07:09:32.888 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:32 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:09:33.593 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 systemd[1]: Starting Ceph mds.i for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:09:33.881 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 podman[65205]: 2024-09-21 07:09:33.494472233 +0000 UTC m=+0.015851980 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:33.881 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 podman[65205]: 2024-09-21 07:09:33.618313364 +0000 UTC m=+0.139693104 container create 407106754a06791a9010037c7d4528ca97e7b009e66eae7d366be8617edd9711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T07:09:33.881 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 podman[65205]: 2024-09-21 07:09:33.787370128 +0000 UTC m=+0.308749876 container init 407106754a06791a9010037c7d4528ca97e7b009e66eae7d366be8617edd9711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T07:09:33.881 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 podman[65205]: 2024-09-21 07:09:33.7902038 +0000 UTC m=+0.311583543 container start 407106754a06791a9010037c7d4528ca97e7b009e66eae7d366be8617edd9711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:09:33.881 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 ceph-mds[65224]: -- 172.21.15.153:0/2668518373 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640e195b860 con 0x5640e5b03600 2024-09-21T07:09:33.881 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i[65220]: starting mds.i at 2024-09-21T07:09:33.881 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 bash[65205]: 407106754a06791a9010037c7d4528ca97e7b009e66eae7d366be8617edd9711 2024-09-21T07:09:34.177 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:09:33 smithi153 systemd[1]: Started Ceph mds.i for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:34.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: Deploying daemon mds.g on smithi027 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.153:6838/2081315981,v1:172.21.15.153:6839/2081315981] up:boot 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: fsmap 9 up:standby 2024-09-21T07:09:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:34 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: Deploying daemon mds.g on smithi027 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.153:6838/2081315981,v1:172.21.15.153:6839/2081315981] up:boot 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: fsmap 9 up:standby 2024-09-21T07:09:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:34 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:09:35.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:35.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: Deploying daemon mds.g on smithi027 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.153:6838/2081315981,v1:172.21.15.153:6839/2081315981] up:boot 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: fsmap 9 up:standby 2024-09-21T07:09:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:34 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:09:35.682 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:36.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.525+0000 7f69f3000640 1 -- 172.21.15.27:0/1980019238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 msgr2=0x7f69ec075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:36.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.525+0000 7f69f3000640 1 --2- 172.21.15.27:0/1980019238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec075bc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009900 tx=0x7f69e002a410 comp rx=0 tx=0).stop 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.525+0000 7f69f3000640 1 -- 172.21.15.27:0/1980019238 shutdown_connections 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.525+0000 7f69f3000640 1 --2- 172.21.15.27:0/1980019238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69ec10ff10 0x7f69ec112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.525+0000 7f69f3000640 1 --2- 172.21.15.27:0/1980019238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69ec076100 0x7f69ec10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.525+0000 7f69f3000640 1 --2- 172.21.15.27:0/1980019238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec075bc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.525+0000 7f69f3000640 1 -- 172.21.15.27:0/1980019238 >> 172.21.15.27:0/1980019238 conn(0x7f69ec0ff440 msgr2=0x7f69ec101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.526+0000 7f69f3000640 1 -- 172.21.15.27:0/1980019238 shutdown_connections 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.526+0000 7f69f3000640 1 -- 172.21.15.27:0/1980019238 wait complete. 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.527+0000 7f69f3000640 1 Processor -- start 2024-09-21T07:09:36.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.527+0000 7f69f3000640 1 -- start start 2024-09-21T07:09:36.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f3000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec1069c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:36.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f3000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69ec076100 0x7f69ec106f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:36.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f3000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69ec10ff10 0x7f69ec107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:36.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f3000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec115ac0 con 0x7f69ec0757b0 2024-09-21T07:09:36.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f3000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec115940 con 0x7f69ec10ff10 2024-09-21T07:09:36.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f3000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec115c40 con 0x7f69ec076100 2024-09-21T07:09:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec1069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec1069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34932/0 (socket says 172.21.15.27:34932) 2024-09-21T07:09:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f1c00640 1 -- 172.21.15.27:0/1653870655 learned_addr learned my addr 172.21.15.27:0/1653870655 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f1200640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69ec076100 0x7f69ec106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.528+0000 7f69f2600640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69ec10ff10 0x7f69ec107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.529+0000 7f69f1c00640 1 -- 172.21.15.27:0/1653870655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69ec076100 msgr2=0x7f69ec106f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.529+0000 7f69f1c00640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69ec076100 0x7f69ec106f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.529+0000 7f69f1c00640 1 -- 172.21.15.27:0/1653870655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69ec10ff10 msgr2=0x7f69ec107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.529+0000 7f69f1c00640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69ec10ff10 0x7f69ec107440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.529+0000 7f69f1c00640 1 -- 172.21.15.27:0/1653870655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ec1ac940 con 0x7f69ec0757b0 2024-09-21T07:09:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.529+0000 7f69f1c00640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec1069c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005940 tx=0x7f69e00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.530+0000 7f69ea000640 1 -- 172.21.15.27:0/1653870655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e003f030 con 0x7f69ec0757b0 2024-09-21T07:09:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.530+0000 7f69ea000640 1 -- 172.21.15.27:0/1653870655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e00044c0 con 0x7f69ec0757b0 2024-09-21T07:09:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.530+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ec1acbd0 con 0x7f69ec0757b0 2024-09-21T07:09:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.530+0000 7f69ea000640 1 -- 172.21.15.27:0/1653870655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0035070 con 0x7f69ec0757b0 2024-09-21T07:09:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.530+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ec1ad0b0 con 0x7f69ec0757b0 2024-09-21T07:09:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.531+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ec075bc0 con 0x7f69ec0757b0 2024-09-21T07:09:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.534+0000 7f69ea000640 1 -- 172.21.15.27:0/1653870655 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f69e0042030 con 0x7f69ec0757b0 2024-09-21T07:09:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.536+0000 7f69ea000640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f69d40b6560 0x7f69d40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.536+0000 7f69ea000640 1 -- 172.21.15.27:0/1653870655 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f69e0107600 con 0x7f69ec0757b0 2024-09-21T07:09:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.536+0000 7f69f1200640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f69d40b6560 0x7f69d40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.537+0000 7f69f1200640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f69d40b6560 0x7f69d40b8a70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f69ec1004e0 tx=0x7f69d8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.538+0000 7f69ea000640 1 -- 172.21.15.27:0/1653870655 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69e00d25a0 con 0x7f69ec0757b0 2024-09-21T07:09:36.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.795+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69ec107cc0 con 0x7f69ec0757b0 2024-09-21T07:09:36.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.796+0000 7f69ea000640 1 -- 172.21.15.27:0/1653870655 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7f69e00d74e0 con 0x7f69ec0757b0 2024-09-21T07:09:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:36.799 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"btime":"2024-09-21T07:09:34:448662+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24713,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24736,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24740,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T07:09:36.799 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f69d40b6560 msgr2=0x7f69d40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f69d40b6560 0x7f69d40b8a70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f69ec1004e0 tx=0x7f69d8005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 msgr2=0x7f69ec1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec1069c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005940 tx=0x7f69e00040c0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 shutdown_connections 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69ec076100 0x7f69ec106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f69d40b6560 0x7f69d40b8a70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69ec10ff10 0x7f69ec107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 --2- 172.21.15.27:0/1653870655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69ec0757b0 0x7f69ec1069c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:36.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 >> 172.21.15.27:0/1653870655 conn(0x7f69ec0ff440 msgr2=0x7f69ec101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:36.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 shutdown_connections 2024-09-21T07:09:36.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:36.799+0000 7f69f3000640 1 -- 172.21.15.27:0/1653870655 wait complete. 2024-09-21T07:09:37.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:36 smithi027 ceph-mon[30779]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:37.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:36 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1653870655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:37.053 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:36 smithi027 podman[73398]: 2024-09-21 07:09:36.966266886 +0000 UTC m=+0.216735444 container create a58697b2e4a96a3e42a43aac837ae5a6f604412dfe1136495994dc1736b79d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T07:09:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:36 smithi119 ceph-mon[34519]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:36 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1653870655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:36 smithi153 ceph-mon[34747]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:36 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1653870655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:37.614 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:37 smithi027 podman[73398]: 2024-09-21 07:09:37.346197222 +0000 UTC m=+0.596665781 container init a58697b2e4a96a3e42a43aac837ae5a6f604412dfe1136495994dc1736b79d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T07:09:37.615 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:37 smithi027 podman[73398]: 2024-09-21 07:09:37.352698259 +0000 UTC m=+0.603166826 container start a58697b2e4a96a3e42a43aac837ae5a6f604412dfe1136495994dc1736b79d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T07:09:37.615 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:37 smithi027 bash[73398]: a58697b2e4a96a3e42a43aac837ae5a6f604412dfe1136495994dc1736b79d25 2024-09-21T07:09:37.615 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:37 smithi027 systemd[1]: Started Ceph mds.g for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:37.615 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:37 smithi027 ceph-mds[73431]: -- 172.21.15.27:0/3155635030 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561f39fceea0 con 0x561f3e16da80 2024-09-21T07:09:37.615 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:37 smithi027 ceph-mds[73431]: -- 172.21.15.27:0/3155635030 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f39fc5860 con 0x561f3e16da80 2024-09-21T07:09:37.615 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:09:37 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g[73427]: starting mds.g at 2024-09-21T07:09:38.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:38.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: Deploying daemon mds.a on smithi027 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] up:boot 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: fsmap 10 up:standby 2024-09-21T07:09:38.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:38 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:09:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: Deploying daemon mds.a on smithi027 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] up:boot 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: fsmap 10 up:standby 2024-09-21T07:09:38.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:38 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:09:38.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: Deploying daemon mds.a on smithi027 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] up:boot 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: fsmap 10 up:standby 2024-09-21T07:09:38.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:38 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:09:39.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:40.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:40 smithi027 ceph-mon[30779]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:40.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:40 smithi119 ceph-mon[34519]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:40.802 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:40 smithi153 ceph-mon[34747]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:40.945 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:09:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-a[73730]: starting mds.a at 2024-09-21T07:09:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.420+0000 7fdb43e00640 1 -- 172.21.15.27:0/3330363772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb441039b0 msgr2=0x7fdb44103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.420+0000 7fdb43e00640 1 --2- 172.21.15.27:0/3330363772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb441039b0 0x7fdb44103dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009900 tx=0x7fdb3402a410 comp rx=0 tx=0).stop 2024-09-21T07:09:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.421+0000 7fdb43e00640 1 -- 172.21.15.27:0/3330363772 shutdown_connections 2024-09-21T07:09:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.421+0000 7fdb43e00640 1 --2- 172.21.15.27:0/3330363772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb441092f0 0x7fdb44109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.421+0000 7fdb43e00640 1 --2- 172.21.15.27:0/3330363772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb44109c40 0x7fdb44110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.421+0000 7fdb43e00640 1 --2- 172.21.15.27:0/3330363772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb441039b0 0x7fdb44103dc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.421+0000 7fdb43e00640 1 -- 172.21.15.27:0/3330363772 >> 172.21.15.27:0/3330363772 conn(0x7fdb440ff440 msgr2=0x7fdb44101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.421+0000 7fdb43e00640 1 -- 172.21.15.27:0/3330363772 shutdown_connections 2024-09-21T07:09:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.421+0000 7fdb43e00640 1 -- 172.21.15.27:0/3330363772 wait complete. 2024-09-21T07:09:41.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.422+0000 7fdb43e00640 1 Processor -- start 2024-09-21T07:09:41.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.422+0000 7fdb43e00640 1 -- start start 2024-09-21T07:09:41.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.422+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb441039b0 0x7fdb4419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:41.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.422+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb441092f0 0x7fdb4419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:41.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb44109c40 0x7fdb441a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:41.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb43e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb44113900 con 0x7fdb44109c40 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb43e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdb44113780 con 0x7fdb441092f0 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb43e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb44113a80 con 0x7fdb441039b0 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb43400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb44109c40 0x7fdb441a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb441092f0 0x7fdb4419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb441092f0 0x7fdb4419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55538/0 (socket says 172.21.15.27:55538) 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb43400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb44109c40 0x7fdb441a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43434/0 (socket says 172.21.15.27:43434) 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 -- 172.21.15.27:0/2694968402 learned_addr learned my addr 172.21.15.27:0/2694968402 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42a00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb441039b0 0x7fdb4419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 -- 172.21.15.27:0/2694968402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb441039b0 msgr2=0x7fdb4419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb441039b0 0x7fdb4419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 -- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb44109c40 msgr2=0x7fdb441a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb44109c40 0x7fdb441a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.423+0000 7fdb42000640 1 -- 172.21.15.27:0/2694968402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb441a24a0 con 0x7fdb441092f0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.424+0000 7fdb42000640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb441092f0 0x7fdb4419cad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e9f0 tx=0x7fdb3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.424+0000 7fdb48a00640 1 -- 172.21.15.27:0/2694968402 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000ccc0 con 0x7fdb441092f0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.424+0000 7fdb48a00640 1 -- 172.21.15.27:0/2694968402 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb300108e0 con 0x7fdb441092f0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.424+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb441a2790 con 0x7fdb441092f0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.424+0000 7fdb48a00640 1 -- 172.21.15.27:0/2694968402 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30018860 con 0x7fdb441092f0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.424+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb441a8ef0 con 0x7fdb441092f0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.425+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb44103dc0 con 0x7fdb441092f0 2024-09-21T07:09:41.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.427+0000 7fdb48a00640 1 -- 172.21.15.27:0/2694968402 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fdb30010430 con 0x7fdb441092f0 2024-09-21T07:09:41.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.429+0000 7fdb48a00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdb280b6560 0x7fdb280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:41.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.429+0000 7fdb48a00640 1 -- 172.21.15.27:0/2694968402 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fdb300e77a0 con 0x7fdb441092f0 2024-09-21T07:09:41.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.429+0000 7fdb42a00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdb280b6560 0x7fdb280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:41.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.430+0000 7fdb42a00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdb280b6560 0x7fdb280b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb34031440 tx=0x7fdb34009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:41.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.430+0000 7fdb48a00640 1 -- 172.21.15.27:0/2694968402 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdb300b2740 con 0x7fdb441092f0 2024-09-21T07:09:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.672+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb441a91d0 con 0x7fdb441092f0 2024-09-21T07:09:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.673+0000 7fdb48a00640 1 -- 172.21.15.27:0/2694968402 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9704 (secure 0 0 0) 0x7fdb300b7680 con 0x7fdb441092f0 2024-09-21T07:09:41.675 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":12,"btime":"2024-09-21T07:09:41:528685+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24713,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24736,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24740,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24748,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-09-21T07:09:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 12 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.675+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdb280b6560 msgr2=0x7fdb280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.675+0000 7fdb43e00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdb280b6560 0x7fdb280b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb34031440 tx=0x7fdb34009560 comp rx=0 tx=0).stop 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.675+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb441092f0 msgr2=0x7fdb4419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.675+0000 7fdb43e00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb441092f0 0x7fdb4419cad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e9f0 tx=0x7fdb3000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 shutdown_connections 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdb280b6560 0x7fdb280b8a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb44109c40 0x7fdb441a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb441092f0 0x7fdb4419cad0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 --2- 172.21.15.27:0/2694968402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb441039b0 0x7fdb4419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 >> 172.21.15.27:0/2694968402 conn(0x7fdb440ff440 msgr2=0x7fdb4410e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 shutdown_connections 2024-09-21T07:09:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:41.676+0000 7fdb43e00640 1 -- 172.21.15.27:0/2694968402 wait complete. 2024-09-21T07:09:42.001 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.001 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.001 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.001 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:42.002 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:42.002 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:42.002 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: Deploying daemon mds.b on smithi119 2024-09-21T07:09:42.002 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.27:6840/2637528669,v1:172.21.15.27:6841/2637528669] up:boot 2024-09-21T07:09:42.002 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: fsmap 11 up:standby 2024-09-21T07:09:42.002 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T07:09:42.002 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2694968402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: Deploying daemon mds.b on smithi119 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.27:6840/2637528669,v1:172.21.15.27:6841/2637528669] up:boot 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: fsmap 11 up:standby 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T07:09:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2694968402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: Deploying daemon mds.b on smithi119 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.27:6840/2637528669,v1:172.21.15.27:6841/2637528669] up:boot 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: fsmap 11 up:standby 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T07:09:42.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2694968402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:42.804 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:42 smithi119 systemd[1]: Starting Ceph mds.b for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:09:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:42 smithi027 ceph-mon[30779]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:43.145 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:42 smithi119 ceph-mon[34519]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:43.146 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:42 smithi119 podman[63789]: 2024-09-21 07:09:42.916754281 +0000 UTC m=+0.110749986 container create 88eb88a42e9ac48cef5f9a930f391ff4ac159917daba5afdf704332f162d7398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T07:09:43.146 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:42 smithi119 podman[63789]: 2024-09-21 07:09:42.82153127 +0000 UTC m=+0.015526976 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:09:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:42 smithi153 ceph-mon[34747]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:43.534 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:43 smithi119 podman[63789]: 2024-09-21 07:09:43.144504847 +0000 UTC m=+0.338500562 container init 88eb88a42e9ac48cef5f9a930f391ff4ac159917daba5afdf704332f162d7398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T07:09:43.534 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:43 smithi119 podman[63789]: 2024-09-21 07:09:43.147939383 +0000 UTC m=+0.341935096 container start 88eb88a42e9ac48cef5f9a930f391ff4ac159917daba5afdf704332f162d7398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T07:09:43.535 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:43 smithi119 ceph-mds[63808]: -- 172.21.15.119:0/3613349976 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7d35f9860 con 0x55c7d77a1180 2024-09-21T07:09:43.535 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:43 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: starting mds.b at 2024-09-21T07:09:43.535 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:43 smithi119 bash[63789]: 88eb88a42e9ac48cef5f9a930f391ff4ac159917daba5afdf704332f162d7398 2024-09-21T07:09:43.535 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:43 smithi119 systemd[1]: Started Ceph mds.b for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:09:44.340 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:44.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:44.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:44.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] up:boot 2024-09-21T07:09:44.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: fsmap 12 up:standby 2024-09-21T07:09:44.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:44 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:09:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] up:boot 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: fsmap 12 up:standby 2024-09-21T07:09:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:44 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:09:44.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] up:boot 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: fsmap 12 up:standby 2024-09-21T07:09:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:44 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:09:44.869 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 -- 172.21.15.27:0/3236340142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 msgr2=0x7f922c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 --2- 172.21.15.27:0/3236340142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 0x7f922c103dc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f921c009ca0 tx=0x7f921c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 -- 172.21.15.27:0/3236340142 shutdown_connections 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 --2- 172.21.15.27:0/3236340142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f922c1092f0 0x7f922c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 --2- 172.21.15.27:0/3236340142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c109c40 0x7f922c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 --2- 172.21.15.27:0/3236340142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 0x7f922c103dc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 -- 172.21.15.27:0/3236340142 >> 172.21.15.27:0/3236340142 conn(0x7f922c0ff440 msgr2=0x7f922c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 -- 172.21.15.27:0/3236340142 shutdown_connections 2024-09-21T07:09:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.462+0000 7f9231600640 1 -- 172.21.15.27:0/3236340142 wait complete. 2024-09-21T07:09:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.463+0000 7f9231600640 1 Processor -- start 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.463+0000 7f9231600640 1 -- start start 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9231600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 0x7f922c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9231600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c1092f0 0x7f922c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9231600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f922c109c40 0x7f922c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9231600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f922c113900 con 0x7f922c1092f0 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9231600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f922c113780 con 0x7f922c1039b0 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9231600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f922c113a80 con 0x7f922c109c40 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9223400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c1092f0 0x7f922c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f922be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 0x7f922c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9223400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c1092f0 0x7f922c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43452/0 (socket says 172.21.15.27:43452) 2024-09-21T07:09:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9223400640 1 -- 172.21.15.27:0/2566237932 learned_addr learned my addr 172.21.15.27:0/2566237932 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.464+0000 7f9230c00640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f922c109c40 0x7f922c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.465+0000 7f9223400640 1 -- 172.21.15.27:0/2566237932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f922c109c40 msgr2=0x7f922c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.465+0000 7f9223400640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f922c109c40 0x7f922c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.465+0000 7f9223400640 1 -- 172.21.15.27:0/2566237932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 msgr2=0x7f922c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.465+0000 7f9223400640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 0x7f922c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.465+0000 7f9223400640 1 -- 172.21.15.27:0/2566237932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c1a85f0 con 0x7f922c1092f0 2024-09-21T07:09:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.465+0000 7f9223400640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c1092f0 0x7f922c075c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f921400e9c0 tx=0x7f921400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.466+0000 7f9229600640 1 -- 172.21.15.27:0/2566237932 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921400ccc0 con 0x7f922c1092f0 2024-09-21T07:09:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.466+0000 7f9229600640 1 -- 172.21.15.27:0/2566237932 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92140054d0 con 0x7f922c1092f0 2024-09-21T07:09:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.466+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922c1a88e0 con 0x7f922c1092f0 2024-09-21T07:09:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.466+0000 7f9229600640 1 -- 172.21.15.27:0/2566237932 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214010430 con 0x7f922c1092f0 2024-09-21T07:09:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.466+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922c1a8dd0 con 0x7f922c1092f0 2024-09-21T07:09:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.467+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922c103dc0 con 0x7f922c1092f0 2024-09-21T07:09:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.470+0000 7f9229600640 1 -- 172.21.15.27:0/2566237932 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f9214004f20 con 0x7f922c1092f0 2024-09-21T07:09:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.472+0000 7f9229600640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f92100b6560 0x7f92100b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.473+0000 7f9229600640 1 -- 172.21.15.27:0/2566237932 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f92140e7670 con 0x7f922c1092f0 2024-09-21T07:09:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.473+0000 7f922be00640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f92100b6560 0x7f92100b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.474+0000 7f922be00640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f92100b6560 0x7f92100b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f921c005e30 tx=0x7f921c009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:45.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.474+0000 7f9229600640 1 -- 172.21.15.27:0/2566237932 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9214056710 con 0x7f922c1092f0 2024-09-21T07:09:45.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.796+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f922c076a10 con 0x7f922c1092f0 2024-09-21T07:09:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.797+0000 7f9229600640 1 -- 172.21.15.27:0/2566237932 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7f922c076a10 con 0x7f922c1092f0 2024-09-21T07:09:45.800 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:45.800 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"btime":"2024-09-21T07:09:43:802675+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24713,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24736,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24740,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24748,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24773,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T07:09:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2024-09-21T07:09:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f92100b6560 msgr2=0x7f92100b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f92100b6560 0x7f92100b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f921c005e30 tx=0x7f921c009360 comp rx=0 tx=0).stop 2024-09-21T07:09:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c1092f0 msgr2=0x7f922c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c1092f0 0x7f922c075c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f921400e9c0 tx=0x7f921400edc0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 shutdown_connections 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f922c109c40 0x7f922c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f92100b6560 0x7f92100b8a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c1092f0 0x7f922c075c50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.800+0000 7f9231600640 1 --2- 172.21.15.27:0/2566237932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922c1039b0 0x7f922c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.801+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 >> 172.21.15.27:0/2566237932 conn(0x7f922c0ff440 msgr2=0x7f922c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.801+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 shutdown_connections 2024-09-21T07:09:45.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:45.801+0000 7f9231600640 1 -- 172.21.15.27:0/2566237932 wait complete. 2024-09-21T07:09:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:45 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:45 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2566237932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:45 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:45 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2566237932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:45 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:45 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2566237932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:46.558 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T07:09:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:46 smithi119 ceph-mon[34519]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:46 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:46 smithi153 ceph-mon[34747]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:46 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:47.272 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:46 smithi027 ceph-mon[30779]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:47.272 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:47.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:47.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:47.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:46 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:47.616 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:48.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 -- 172.21.15.27:0/4139398355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad00757b0 msgr2=0x7fdad0075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 --2- 172.21.15.27:0/4139398355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad00757b0 0x7fdad0075bc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009900 tx=0x7fdac002a410 comp rx=0 tx=0).stop 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 -- 172.21.15.27:0/4139398355 shutdown_connections 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 --2- 172.21.15.27:0/4139398355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad01100d0 0x7fdad0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 --2- 172.21.15.27:0/4139398355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad0076100 0x7fdad010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 --2- 172.21.15.27:0/4139398355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad00757b0 0x7fdad0075bc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 -- 172.21.15.27:0/4139398355 >> 172.21.15.27:0/4139398355 conn(0x7fdad00ff5e0 msgr2=0x7fdad0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 -- 172.21.15.27:0/4139398355 shutdown_connections 2024-09-21T07:09:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.264+0000 7fdad4a00640 1 -- 172.21.15.27:0/4139398355 wait complete. 2024-09-21T07:09:48.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 Processor -- start 2024-09-21T07:09:48.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 -- start start 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad00757b0 0x7fdad019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad0076100 0x7fdad019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad01100d0 0x7fdad01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0115cd0 con 0x7fdad01100d0 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0115b50 con 0x7fdad0076100 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.266+0000 7fdad4a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0115e50 con 0x7fdad00757b0 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad00757b0 0x7fdad019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad00757b0 0x7fdad019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58374/0 (socket says 172.21.15.27:58374) 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 -- 172.21.15.27:0/2983468321 learned_addr learned my addr 172.21.15.27:0/2983468321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdace800640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad01100d0 0x7fdad01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdac7e00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad0076100 0x7fdad019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 -- 172.21.15.27:0/2983468321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad0076100 msgr2=0x7fdad019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad0076100 0x7fdad019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 -- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad01100d0 msgr2=0x7fdad01a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad01100d0 0x7fdad01a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.267+0000 7fdacde00640 1 -- 172.21.15.27:0/2983468321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad01a25d0 con 0x7fdad00757b0 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.268+0000 7fdacde00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad00757b0 0x7fdad019c660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005ab0 tx=0x7fdac0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.268+0000 7fdac6a00640 1 -- 172.21.15.27:0/2983468321 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac003f070 con 0x7fdad00757b0 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.268+0000 7fdac6a00640 1 -- 172.21.15.27:0/2983468321 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac0031b60 con 0x7fdad00757b0 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.268+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad01a2860 con 0x7fdad00757b0 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.268+0000 7fdac6a00640 1 -- 172.21.15.27:0/2983468321 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0035030 con 0x7fdad00757b0 2024-09-21T07:09:48.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.268+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad01a9020 con 0x7fdad00757b0 2024-09-21T07:09:48.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.269+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad0075bc0 con 0x7fdad00757b0 2024-09-21T07:09:48.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.272+0000 7fdac6a00640 1 -- 172.21.15.27:0/2983468321 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fdac0042070 con 0x7fdad00757b0 2024-09-21T07:09:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.273+0000 7fdac6a00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdab40b65e0 0x7fdab40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.274+0000 7fdac6a00640 1 -- 172.21.15.27:0/2983468321 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fdac0107680 con 0x7fdad00757b0 2024-09-21T07:09:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.274+0000 7fdac7e00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdab40b65e0 0x7fdab40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.274+0000 7fdac7e00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdab40b65e0 0x7fdab40b8af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdab80095f0 tx=0x7fdab8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.277+0000 7fdac6a00640 1 -- 172.21.15.27:0/2983468321 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdac00d2620 con 0x7fdad00757b0 2024-09-21T07:09:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.508+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdad01a9250 con 0x7fdad00757b0 2024-09-21T07:09:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.509+0000 7fdac6a00640 1 -- 172.21.15.27:0/2983468321 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7fdac00d7560 con 0x7fdad00757b0 2024-09-21T07:09:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:09:48.511 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-21T07:09:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:09:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-21T07:09:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-21T07:09:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:09:48.511 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-21T07:09:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdab40b65e0 msgr2=0x7fdab40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdab40b65e0 0x7fdab40b8af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdab80095f0 tx=0x7fdab8005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad00757b0 msgr2=0x7fdad019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad00757b0 0x7fdad019c660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005ab0 tx=0x7fdac0003c10 comp rx=0 tx=0).stop 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 shutdown_connections 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdab40b65e0 0x7fdab40b8af0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad01100d0 0x7fdad01a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad0076100 0x7fdad019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 --2- 172.21.15.27:0/2983468321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdad00757b0 0x7fdad019c660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.511+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 >> 172.21.15.27:0/2983468321 conn(0x7fdad00ff5e0 msgr2=0x7fdad0109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.512+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 shutdown_connections 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:48.512+0000 7fdad4a00640 1 -- 172.21.15.27:0/2983468321 wait complete. 2024-09-21T07:09:48.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:48 smithi027 ceph-mon[30779]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:48.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:48 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:48 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2983468321' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T07:09:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:48 smithi153 ceph-mon[34747]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:48 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:48 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2983468321' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T07:09:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:48 smithi119 ceph-mon[34519]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:49.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:49.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:49.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:48 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:49.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:48 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2983468321' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T07:09:49.206 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:09:49.207 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:09:49.603 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 -- 172.21.15.27:0/3600757171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 msgr2=0x7faffc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 --2- 172.21.15.27:0/3600757171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 0x7faffc110620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faff4009ca0 tx=0x7faff402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 -- 172.21.15.27:0/3600757171 shutdown_connections 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 --2- 172.21.15.27:0/3600757171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 0x7faffc110620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 --2- 172.21.15.27:0/3600757171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc1094a0 0x7faffc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 --2- 172.21.15.27:0/3600757171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faffc103b60 0x7faffc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 -- 172.21.15.27:0/3600757171 >> 172.21.15.27:0/3600757171 conn(0x7faffc07b850 msgr2=0x7faffc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 -- 172.21.15.27:0/3600757171 shutdown_connections 2024-09-21T07:09:50.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.242+0000 7fb002200640 1 -- 172.21.15.27:0/3600757171 wait complete. 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.243+0000 7fb002200640 1 Processor -- start 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7fb002200640 1 -- start start 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7fb002200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc103b60 0x7faffc19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7fb002200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faffc1094a0 0x7faffc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7fb002200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 0x7faffc1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7fb002200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faffc113b00 con 0x7faffc109df0 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7fb002200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faffc113980 con 0x7faffc103b60 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7fb002200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faffc113c80 con 0x7faffc1094a0 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7faffb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc103b60 0x7faffc19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7faffb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc103b60 0x7faffc19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:43790/0 (socket says 172.21.15.27:43790) 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7faffb400640 1 -- 172.21.15.27:0/3878139481 learned_addr learned my addr 172.21.15.27:0/3878139481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.244+0000 7faffbe00640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 0x7faffc1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffaa00640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faffc1094a0 0x7faffc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffb400640 1 -- 172.21.15.27:0/3878139481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faffc1094a0 msgr2=0x7faffc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffb400640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faffc1094a0 0x7faffc19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffb400640 1 -- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 msgr2=0x7faffc1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffb400640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 0x7faffc1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffb400640 1 -- 172.21.15.27:0/3878139481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc1a25e0 con 0x7faffc103b60 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffbe00640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 0x7faffc1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.245+0000 7faffb400640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc103b60 0x7faffc19c670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafec00ea60 tx=0x7fafec00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.246+0000 7faff3e00640 1 -- 172.21.15.27:0/3878139481 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec00cde0 con 0x7faffc103b60 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.246+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc1a28d0 con 0x7faffc103b60 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.246+0000 7faff3e00640 1 -- 172.21.15.27:0/3878139481 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafec0035f0 con 0x7faffc103b60 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.246+0000 7faff3e00640 1 -- 172.21.15.27:0/3878139481 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec010690 con 0x7faffc103b60 2024-09-21T07:09:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.246+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc1a9030 con 0x7faffc103b60 2024-09-21T07:09:50.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.247+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc103f70 con 0x7faffc103b60 2024-09-21T07:09:50.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.250+0000 7faff3e00640 1 -- 172.21.15.27:0/3878139481 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fafec0037a0 con 0x7faffc103b60 2024-09-21T07:09:50.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.251+0000 7faff3e00640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fafe00b6560 0x7fafe00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:50.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.251+0000 7faff3e00640 1 -- 172.21.15.27:0/3878139481 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fafec0e7740 con 0x7faffc103b60 2024-09-21T07:09:50.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.251+0000 7faffaa00640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fafe00b6560 0x7fafe00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:50.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.252+0000 7faffaa00640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fafe00b6560 0x7fafe00b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafe80095f0 tx=0x7fafe8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:50.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.253+0000 7faff3e00640 1 -- 172.21.15.27:0/3878139481 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fafec0ae300 con 0x7faffc103b60 2024-09-21T07:09:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:50 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:50 smithi027 ceph-mon[30779]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:50.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:50.456+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faffc1a9400 con 0x7faffc103b60 2024-09-21T07:09:50.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:50.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:50.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:50 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:50 smithi119 ceph-mon[34519]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:50.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:50.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:50.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:50 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:09:50.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:50 smithi153 ceph-mon[34747]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:51.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.299+0000 7faff3e00640 1 -- 172.21.15.27:0/3878139481 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7fafec0b26e0 con 0x7faffc103b60 2024-09-21T07:09:51.304 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-21T07:09:51.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.306+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fafe00b6560 msgr2=0x7fafe00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:51.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.306+0000 7fb002200640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fafe00b6560 0x7fafe00b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafe80095f0 tx=0x7fafe8005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:51.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.306+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc103b60 msgr2=0x7faffc19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:51.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.306+0000 7fb002200640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc103b60 0x7faffc19c670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafec00ea60 tx=0x7fafec00ef20 comp rx=0 tx=0).stop 2024-09-21T07:09:51.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 shutdown_connections 2024-09-21T07:09:51.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faffc1094a0 0x7faffc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:51.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fafe00b6560 0x7fafe00b8a70 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafe80095f0 tx=0x7fafe8005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:51.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faffc109df0 0x7faffc1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:51.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 --2- 172.21.15.27:0/3878139481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faffc103b60 0x7faffc19c670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:51.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 >> 172.21.15.27:0/3878139481 conn(0x7faffc07b850 msgr2=0x7faffc105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:51.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 shutdown_connections 2024-09-21T07:09:51.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:51.307+0000 7fb002200640 1 -- 172.21.15.27:0/3878139481 wait complete. 2024-09-21T07:09:51.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:51 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3878139481' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T07:09:51.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:51 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T07:09:51.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:51 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3878139481' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T07:09:51.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:51 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T07:09:51.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:51 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3878139481' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T07:09:51.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:51 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T07:09:52.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T07:09:52.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:52 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T07:09:52.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:52 smithi027 ceph-mon[30779]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T07:09:52.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:52 smithi027 ceph-mon[30779]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:52.598 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:52.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:52 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T07:09:52.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:52 smithi153 ceph-mon[34747]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T07:09:52.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:52 smithi153 ceph-mon[34747]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:52.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:52 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T07:09:52.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:52 smithi119 ceph-mon[34519]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T07:09:52.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:52 smithi119 ceph-mon[34519]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:53.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.188+0000 7fa568c00640 1 -- 172.21.15.27:0/2899851172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410b6c0 msgr2=0x7fa56410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.188+0000 7fa568c00640 1 --2- 172.21.15.27:0/2899851172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410b6c0 0x7fa56410bad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa550009900 tx=0x7fa55002a410 comp rx=0 tx=0).stop 2024-09-21T07:09:53.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.188+0000 7fa568c00640 1 -- 172.21.15.27:0/2899851172 shutdown_connections 2024-09-21T07:09:53.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.188+0000 7fa568c00640 1 --2- 172.21.15.27:0/2899851172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56410c010 0x7fa564112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.188+0000 7fa568c00640 1 --2- 172.21.15.27:0/2899851172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410b6c0 0x7fa56410bad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.188+0000 7fa568c00640 1 --2- 172.21.15.27:0/2899851172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa564105d80 0x7fa564106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.189+0000 7fa568c00640 1 -- 172.21.15.27:0/2899851172 >> 172.21.15.27:0/2899851172 conn(0x7fa564101810 msgr2=0x7fa564103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:53.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.189+0000 7fa568c00640 1 -- 172.21.15.27:0/2899851172 shutdown_connections 2024-09-21T07:09:53.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.189+0000 7fa568c00640 1 -- 172.21.15.27:0/2899851172 wait complete. 2024-09-21T07:09:53.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 Processor -- start 2024-09-21T07:09:53.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 -- start start 2024-09-21T07:09:53.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa564105d80 0x7fa56419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa56410b6c0 0x7fa56419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410c010 0x7fa5641a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa564115d20 con 0x7fa56410c010 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa564115ba0 con 0x7fa564105d80 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa568c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa564115ea0 con 0x7fa56410b6c0 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.190+0000 7fa562800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410c010 0x7fa5641a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa562800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410c010 0x7fa5641a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42846/0 (socket says 172.21.15.27:42846) 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa564105d80 0x7fa56419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa562800640 1 -- 172.21.15.27:0/1343238292 learned_addr learned my addr 172.21.15.27:0/1343238292 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561400640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa56410b6c0 0x7fa56419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561e00640 1 -- 172.21.15.27:0/1343238292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa56410b6c0 msgr2=0x7fa56419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561e00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa56410b6c0 0x7fa56419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561e00640 1 -- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410c010 msgr2=0x7fa5641a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561e00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410c010 0x7fa5641a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561e00640 1 -- 172.21.15.27:0/1343238292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5641a2640 con 0x7fa564105d80 2024-09-21T07:09:53.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa562800640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410c010 0x7fa5641a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:53.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.191+0000 7fa561e00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa564105d80 0x7fa56419c6d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa55400e9f0 tx=0x7fa55400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.192+0000 7fa55aa00640 1 -- 172.21.15.27:0/1343238292 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400ccc0 con 0x7fa564105d80 2024-09-21T07:09:53.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.192+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5641a2930 con 0x7fa564105d80 2024-09-21T07:09:53.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.192+0000 7fa55aa00640 1 -- 172.21.15.27:0/1343238292 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5540044d0 con 0x7fa564105d80 2024-09-21T07:09:53.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.192+0000 7fa55aa00640 1 -- 172.21.15.27:0/1343238292 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55401c070 con 0x7fa564105d80 2024-09-21T07:09:53.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.192+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5641a9090 con 0x7fa564105d80 2024-09-21T07:09:53.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.193+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa564106190 con 0x7fa564105d80 2024-09-21T07:09:53.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.196+0000 7fa55aa00640 1 -- 172.21.15.27:0/1343238292 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fa554005020 con 0x7fa564105d80 2024-09-21T07:09:53.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.197+0000 7fa55aa00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5480b65e0 0x7fa5480b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.197+0000 7fa55aa00640 1 -- 172.21.15.27:0/1343238292 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7712+0+0 (secure 0 0 0) 0x7fa5540e7170 con 0x7fa564105d80 2024-09-21T07:09:53.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.198+0000 7fa561400640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5480b65e0 0x7fa5480b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.198+0000 7fa561400640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5480b65e0 0x7fa5480b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa55002aa00 tx=0x7fa550005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.199+0000 7fa55aa00640 1 -- 172.21.15.27:0/1343238292 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa5540adc70 con 0x7fa564105d80 2024-09-21T07:09:53.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:53.403+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa56419d800 con 0x7fa564105d80 2024-09-21T07:09:53.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:53 smithi027 ceph-mon[30779]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T07:09:53.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:53 smithi153 ceph-mon[34747]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T07:09:53.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:53 smithi119 ceph-mon[34519]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T07:09:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.332+0000 7fa55aa00640 1 -- 172.21.15.27:0/1343238292 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v72) ==== 171+0+0 (secure 0 0 0) 0x7fa56419d800 con 0x7fa564105d80 2024-09-21T07:09:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-21T07:09:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5480b65e0 msgr2=0x7fa5480b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5480b65e0 0x7fa5480b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa55002aa00 tx=0x7fa550005a20 comp rx=0 tx=0).stop 2024-09-21T07:09:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa564105d80 msgr2=0x7fa56419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa564105d80 0x7fa56419c6d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa55400e9f0 tx=0x7fa55400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 shutdown_connections 2024-09-21T07:09:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa56410b6c0 0x7fa56419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5480b65e0 0x7fa5480b8af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa56410c010 0x7fa5641a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 --2- 172.21.15.27:0/1343238292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa564105d80 0x7fa56419c6d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 >> 172.21.15.27:0/1343238292 conn(0x7fa564101810 msgr2=0x7fa564103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.339+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 shutdown_connections 2024-09-21T07:09:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:54.340+0000 7fa568c00640 1 -- 172.21.15.27:0/1343238292 wait complete. 2024-09-21T07:09:54.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:54 smithi027 ceph-mon[30779]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T07:09:54.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:54 smithi027 ceph-mon[30779]: pgmap v269: 33 pgs: 6 creating+peering, 21 unknown, 6 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:54.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:54 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1343238292' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T07:09:54.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:54 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T07:09:54.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:54 smithi153 ceph-mon[34747]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T07:09:54.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:54 smithi153 ceph-mon[34747]: pgmap v269: 33 pgs: 6 creating+peering, 21 unknown, 6 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:54.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:54 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1343238292' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T07:09:54.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:54 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T07:09:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:54 smithi119 ceph-mon[34519]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T07:09:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:54 smithi119 ceph-mon[34519]: pgmap v269: 33 pgs: 6 creating+peering, 21 unknown, 6 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:54 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1343238292' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T07:09:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:54 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T07:09:55.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:09:55.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:55 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T07:09:55.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:55 smithi027 ceph-mon[30779]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T07:09:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:55.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:55 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T07:09:55.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:55 smithi153 ceph-mon[34747]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T07:09:55.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:55 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T07:09:55.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:55 smithi119 ceph-mon[34519]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T07:09:56.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.219+0000 7fb6b9a00640 1 -- 172.21.15.27:0/2571194737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b41049c0 msgr2=0x7fb6b4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:56.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.219+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/2571194737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b41049c0 0x7fb6b4104dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009900 tx=0x7fb6a402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:56.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.220+0000 7fb6b9a00640 1 -- 172.21.15.27:0/2571194737 shutdown_connections 2024-09-21T07:09:56.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.220+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/2571194737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b410de20 0x7fb6b4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:56.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.220+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/2571194737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 0x7fb6b410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:56.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.220+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/2571194737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b41049c0 0x7fb6b4104dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:56.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.220+0000 7fb6b9a00640 1 -- 172.21.15.27:0/2571194737 >> 172.21.15.27:0/2571194737 conn(0x7fb6b4079730 msgr2=0x7fb6b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:56.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.220+0000 7fb6b9a00640 1 -- 172.21.15.27:0/2571194737 shutdown_connections 2024-09-21T07:09:56.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.220+0000 7fb6b9a00640 1 -- 172.21.15.27:0/2571194737 wait complete. 2024-09-21T07:09:56.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.221+0000 7fb6b9a00640 1 Processor -- start 2024-09-21T07:09:56.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.221+0000 7fb6b9a00640 1 -- start start 2024-09-21T07:09:56.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.221+0000 7fb6b9a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b41049c0 0x7fb6b4102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:56.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.221+0000 7fb6b9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 0x7fb6b4102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:56.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b410de20 0x7fb6b4103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:56.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b9a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b4113ad0 con 0x7fb6b410de20 2024-09-21T07:09:56.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b9a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b4113950 con 0x7fb6b4105310 2024-09-21T07:09:56.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b9a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b4113c50 con 0x7fb6b41049c0 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b410de20 0x7fb6b4103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b41049c0 0x7fb6b4102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6abe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 0x7fb6b4102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b410de20 0x7fb6b4103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42858/0 (socket says 172.21.15.27:42858) 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b41049c0 0x7fb6b4102700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58424/0 (socket says 172.21.15.27:58424) 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6abe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 0x7fb6b4102c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:43824/0 (socket says 172.21.15.27:43824) 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b3400640 1 -- 172.21.15.27:0/3028473262 learned_addr learned my addr 172.21.15.27:0/3028473262 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6abe00640 1 -- 172.21.15.27:0/3028473262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b41049c0 msgr2=0x7fb6b4102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6abe00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b41049c0 0x7fb6b4102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6abe00640 1 -- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b410de20 msgr2=0x7fb6b4103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6abe00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b410de20 0x7fb6b4103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6abe00640 1 -- 172.21.15.27:0/3028473262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b41a8740 con 0x7fb6b4105310 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.222+0000 7fb6b3400640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b410de20 0x7fb6b4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.223+0000 7fb6abe00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 0x7fb6b4102c40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6b406b950 tx=0x7fb69c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.223+0000 7fb6ab400640 1 -- 172.21.15.27:0/3028473262 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00cc40 con 0x7fb6b4105310 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.223+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b41a8a30 con 0x7fb6b4105310 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.223+0000 7fb6ab400640 1 -- 172.21.15.27:0/3028473262 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb69c00cde0 con 0x7fb6b4105310 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.223+0000 7fb6ab400640 1 -- 172.21.15.27:0/3028473262 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c018780 con 0x7fb6b4105310 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.223+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b41a8f40 con 0x7fb6b4105310 2024-09-21T07:09:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.224+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b406c1e0 con 0x7fb6b4105310 2024-09-21T07:09:56.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.227+0000 7fb6ab400640 1 -- 172.21.15.27:0/3028473262 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fb69c010a90 con 0x7fb6b4105310 2024-09-21T07:09:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.228+0000 7fb6ab400640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6980b6560 0x7fb6980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:56.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.229+0000 7fb6ab400640 1 -- 172.21.15.27:0/3028473262 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8094+0+0 (secure 0 0 0) 0x7fb69c0e7a90 con 0x7fb6b4105310 2024-09-21T07:09:56.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.229+0000 7fb6b2a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6980b6560 0x7fb6980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:56.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.229+0000 7fb6b2a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6980b6560 0x7fb6980b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4031440 tx=0x7fb6a4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:56.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.230+0000 7fb6ab400640 1 -- 172.21.15.27:0/3028473262 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb69c0b2740 con 0x7fb6b4105310 2024-09-21T07:09:56.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:56.471+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb6b4104dd0 con 0x7fb6b4105310 2024-09-21T07:09:56.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:56 smithi027 ceph-mon[30779]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T07:09:56.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:56 smithi027 ceph-mon[30779]: pgmap v272: 97 pgs: 14 creating+peering, 67 unknown, 16 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:56.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:56 smithi153 ceph-mon[34747]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T07:09:56.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:56 smithi153 ceph-mon[34747]: pgmap v272: 97 pgs: 14 creating+peering, 67 unknown, 16 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:56.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:56 smithi119 ceph-mon[34519]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T07:09:56.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:56 smithi119 ceph-mon[34519]: pgmap v272: 97 pgs: 14 creating+peering, 67 unknown, 16 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:57.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.344+0000 7fb6ab400640 1 -- 172.21.15.27:0/3028473262 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb69c0b7680 con 0x7fb6b4105310 2024-09-21T07:09:57.346 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T07:09:57.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.348+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6980b6560 msgr2=0x7fb6980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:57.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.348+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6980b6560 0x7fb6980b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4031440 tx=0x7fb6a4005a20 comp rx=0 tx=0).stop 2024-09-21T07:09:57.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.348+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 msgr2=0x7fb6b4102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.348+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 0x7fb6b4102c40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6b406b950 tx=0x7fb69c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 shutdown_connections 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6980b6560 0x7fb6980b8a70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6b410de20 0x7fb6b4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b4105310 0x7fb6b4102c40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 --2- 172.21.15.27:0/3028473262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b41049c0 0x7fb6b4102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 >> 172.21.15.27:0/3028473262 conn(0x7fb6b4079730 msgr2=0x7fb6b4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 shutdown_connections 2024-09-21T07:09:57.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:57.349+0000 7fb6b9a00640 1 -- 172.21.15.27:0/3028473262 wait complete. 2024-09-21T07:09:57.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:57 smithi153 ceph-mon[34747]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T07:09:57.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:57 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3028473262' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T07:09:57.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:57 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T07:09:57.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:57 smithi153 ceph-mon[34747]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T07:09:57.709 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:57 smithi027 ceph-mon[30779]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T07:09:57.710 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:57 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3028473262' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T07:09:57.710 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:57 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T07:09:57.710 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:57 smithi027 ceph-mon[30779]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T07:09:57.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:57 smithi119 ceph-mon[34519]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T07:09:57.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:57 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3028473262' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T07:09:57.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:57 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T07:09:57.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:57 smithi119 ceph-mon[34519]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T07:09:58.220 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T07:09:58.221 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T07:09:58.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:58 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T07:09:58.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:58 smithi027 ceph-mon[30779]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T07:09:58.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:58 smithi027 ceph-mon[30779]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T07:09:58.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:58 smithi027 ceph-mon[30779]: pgmap v275: 97 pgs: 8 creating+peering, 33 unknown, 56 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:58.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:58 smithi027 ceph-mon[30779]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T07:09:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:58.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:58 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T07:09:58.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:58 smithi153 ceph-mon[34747]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T07:09:58.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:58 smithi153 ceph-mon[34747]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T07:09:58.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:58 smithi153 ceph-mon[34747]: pgmap v275: 97 pgs: 8 creating+peering, 33 unknown, 56 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:58.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:58 smithi153 ceph-mon[34747]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T07:09:58.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:58 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T07:09:58.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:58 smithi119 ceph-mon[34519]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T07:09:58.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:58 smithi119 ceph-mon[34519]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T07:09:58.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:58 smithi119 ceph-mon[34519]: pgmap v275: 97 pgs: 8 creating+peering, 33 unknown, 56 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:09:58.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:58 smithi119 ceph-mon[34519]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T07:09:59.284 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:09:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: 2024-09-21T07:09:58.813+0000 7fa0fea5c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.285+0000 7fde69400640 1 -- 172.21.15.27:0/3014843653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde64105d80 msgr2=0x7fde64106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.285+0000 7fde69400640 1 --2- 172.21.15.27:0/3014843653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde64105d80 0x7fde64106190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fde54009900 tx=0x7fde5402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.285+0000 7fde69400640 1 -- 172.21.15.27:0/3014843653 shutdown_connections 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.285+0000 7fde69400640 1 --2- 172.21.15.27:0/3014843653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde6410c010 0x7fde64112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.285+0000 7fde69400640 1 --2- 172.21.15.27:0/3014843653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6410b6c0 0x7fde6410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.285+0000 7fde69400640 1 --2- 172.21.15.27:0/3014843653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde64105d80 0x7fde64106190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.285+0000 7fde69400640 1 -- 172.21.15.27:0/3014843653 >> 172.21.15.27:0/3014843653 conn(0x7fde64101810 msgr2=0x7fde64103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.286+0000 7fde69400640 1 -- 172.21.15.27:0/3014843653 shutdown_connections 2024-09-21T07:09:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.286+0000 7fde69400640 1 -- 172.21.15.27:0/3014843653 wait complete. 2024-09-21T07:09:59.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 Processor -- start 2024-09-21T07:09:59.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 -- start start 2024-09-21T07:09:59.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde64105d80 0x7fde6419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6410b6c0 0x7fde6419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde6410c010 0x7fde641a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde64115cd0 con 0x7fde64105d80 2024-09-21T07:09:59.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fde64115b50 con 0x7fde6410b6c0 2024-09-21T07:09:59.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.287+0000 7fde69400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde64115e50 con 0x7fde6410c010 2024-09-21T07:09:59.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde62800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde64105d80 0x7fde6419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde61e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6410b6c0 0x7fde6419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde62800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde64105d80 0x7fde6419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49616/0 (socket says 172.21.15.27:49616) 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde62800640 1 -- 172.21.15.27:0/1093180549 learned_addr learned my addr 172.21.15.27:0/1093180549 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde63200640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde6410c010 0x7fde641a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde61e00640 1 -- 172.21.15.27:0/1093180549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde6410c010 msgr2=0x7fde641a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde61e00640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde6410c010 0x7fde641a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde61e00640 1 -- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde64105d80 msgr2=0x7fde6419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde61e00640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde64105d80 0x7fde6419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde61e00640 1 -- 172.21.15.27:0/1093180549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde641a2610 con 0x7fde6410b6c0 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde62800640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde64105d80 0x7fde6419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:59.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.288+0000 7fde61e00640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6410b6c0 0x7fde6419cc40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde6406caa0 tx=0x7fde5000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.289+0000 7fde5b400640 1 -- 172.21.15.27:0/1093180549 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000cc40 con 0x7fde6410b6c0 2024-09-21T07:09:59.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.289+0000 7fde5b400640 1 -- 172.21.15.27:0/1093180549 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde5000cde0 con 0x7fde6410b6c0 2024-09-21T07:09:59.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.289+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde641a2900 con 0x7fde6410b6c0 2024-09-21T07:09:59.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.289+0000 7fde5b400640 1 -- 172.21.15.27:0/1093180549 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50018780 con 0x7fde6410b6c0 2024-09-21T07:09:59.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.289+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde641a9060 con 0x7fde6410b6c0 2024-09-21T07:09:59.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.290+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde64106190 con 0x7fde6410b6c0 2024-09-21T07:09:59.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.293+0000 7fde5b400640 1 -- 172.21.15.27:0/1093180549 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fde50010a90 con 0x7fde6410b6c0 2024-09-21T07:09:59.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.294+0000 7fde5b400640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fde480b65e0 0x7fde480b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.294+0000 7fde5b400640 1 -- 172.21.15.27:0/1093180549 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fde500e7a80 con 0x7fde6410b6c0 2024-09-21T07:09:59.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.294+0000 7fde62800640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fde480b65e0 0x7fde480b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.295+0000 7fde62800640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fde480b65e0 0x7fde480b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde54031440 tx=0x7fde540058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.296+0000 7fde5b400640 1 -- 172.21.15.27:0/1093180549 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde500b26b0 con 0x7fde6410b6c0 2024-09-21T07:09:59.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:09:59.504+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fde6419d7d0 con 0x7fde6410b6c0 2024-09-21T07:09:59.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:59 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] up:active 2024-09-21T07:09:59.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:59 smithi153 ceph-mon[34747]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:09:59.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:09:59 smithi153 ceph-mon[34747]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:09:59.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:59 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] up:active 2024-09-21T07:09:59.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:59 smithi119 ceph-mon[34519]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:09:59.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:09:59 smithi119 ceph-mon[34519]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:09:59.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:59 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] up:active 2024-09-21T07:09:59.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:59 smithi027 ceph-mon[30779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:09:59.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:09:59 smithi027 ceph-mon[30779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:00.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.371+0000 7fde5b400640 1 -- 172.21.15.27:0/1093180549 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v76) ==== 157+0+0 (secure 0 0 0) 0x7fde6419d7d0 con 0x7fde6410b6c0 2024-09-21T07:10:00.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fde480b65e0 msgr2=0x7fde480b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:00.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fde480b65e0 0x7fde480b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde54031440 tx=0x7fde540058d0 comp rx=0 tx=0).stop 2024-09-21T07:10:00.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6410b6c0 msgr2=0x7fde6419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:00.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6410b6c0 0x7fde6419cc40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde6406caa0 tx=0x7fde5000ee50 comp rx=0 tx=0).stop 2024-09-21T07:10:00.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 shutdown_connections 2024-09-21T07:10:00.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde6410c010 0x7fde641a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:00.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fde480b65e0 0x7fde480b8af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:00.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6410b6c0 0x7fde6419cc40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:00.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 --2- 172.21.15.27:0/1093180549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde64105d80 0x7fde6419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:00.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 >> 172.21.15.27:0/1093180549 conn(0x7fde64101810 msgr2=0x7fde64103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:00.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 shutdown_connections 2024-09-21T07:10:00.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:00.374+0000 7fde69400640 1 -- 172.21.15.27:0/1093180549 wait complete. 2024-09-21T07:10:00.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:00 smithi153 ceph-mon[34747]: pgmap v276: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-21T07:10:00.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:00 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1093180549' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T07:10:00.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:00 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T07:10:00.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:00 smithi153 ceph-mon[34747]: overall HEALTH_OK 2024-09-21T07:10:00.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:00 smithi027 ceph-mon[30779]: pgmap v276: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-21T07:10:00.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:00 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1093180549' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T07:10:00.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:00 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T07:10:00.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:00 smithi027 ceph-mon[30779]: overall HEALTH_OK 2024-09-21T07:10:00.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:00 smithi119 ceph-mon[34519]: pgmap v276: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-21T07:10:00.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:00 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1093180549' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T07:10:00.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:00 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T07:10:00.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:00 smithi119 ceph-mon[34519]: overall HEALTH_OK 2024-09-21T07:10:01.248 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:10:01.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:01 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T07:10:01.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:01 smithi027 ceph-mon[30779]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:01.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:01 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T07:10:01.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:01 smithi153 ceph-mon[34747]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T07:10:01.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:01 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T07:10:01.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:01 smithi119 ceph-mon[34519]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T07:10:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 -- 172.21.15.27:0/3475231340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 msgr2=0x7f2ab810fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 --2- 172.21.15.27:0/3475231340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab810fa70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009900 tx=0x7f2aa402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 -- 172.21.15.27:0/3475231340 shutdown_connections 2024-09-21T07:10:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 --2- 172.21.15.27:0/3475231340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ab810ffb0 0x7f2ab81123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 --2- 172.21.15.27:0/3475231340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab810fa70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 --2- 172.21.15.27:0/3475231340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8075750 0x7f2ab8075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 -- 172.21.15.27:0/3475231340 >> 172.21.15.27:0/3475231340 conn(0x7f2ab80ff480 msgr2=0x7f2ab81018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 -- 172.21.15.27:0/3475231340 shutdown_connections 2024-09-21T07:10:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.259+0000 7f2abe600640 1 -- 172.21.15.27:0/3475231340 wait complete. 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 Processor -- start 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 -- start start 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8075750 0x7f2ab819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ab810ffb0 0x7f2ab81a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8115ae0 con 0x7f2ab80760a0 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8115960 con 0x7f2ab810ffb0 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.260+0000 7f2abe600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8115c60 con 0x7f2ab8075750 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2ab7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2abd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8075750 0x7f2ab819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2abd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8075750 0x7f2ab819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45160/0 (socket says 172.21.15.27:45160) 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2ab7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49622/0 (socket says 172.21.15.27:49622) 2024-09-21T07:10:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2abd200640 1 -- 172.21.15.27:0/158355035 learned_addr learned my addr 172.21.15.27:0/158355035 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2abdc00640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ab810ffb0 0x7f2ab81a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2ab7e00640 1 -- 172.21.15.27:0/158355035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8075750 msgr2=0x7f2ab819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2ab7e00640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8075750 0x7f2ab819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2ab7e00640 1 -- 172.21.15.27:0/158355035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ab810ffb0 msgr2=0x7f2ab81a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2ab7e00640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ab810ffb0 0x7f2ab81a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.261+0000 7f2ab7e00640 1 -- 172.21.15.27:0/158355035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab81a2480 con 0x7f2ab80760a0 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.262+0000 7f2ab7e00640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab819cab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2aa402a910 tx=0x7f2aa4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.262+0000 7f2ab5600640 1 -- 172.21.15.27:0/158355035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa403f070 con 0x7f2ab80760a0 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.262+0000 7f2ab5600640 1 -- 172.21.15.27:0/158355035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa4003ca0 con 0x7f2ab80760a0 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.262+0000 7f2ab5600640 1 -- 172.21.15.27:0/158355035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4035070 con 0x7f2ab80760a0 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.262+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab81a2710 con 0x7f2ab80760a0 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.262+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81a2ba0 con 0x7f2ab80760a0 2024-09-21T07:10:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.263+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab8075b60 con 0x7f2ab80760a0 2024-09-21T07:10:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.266+0000 7f2ab5600640 1 -- 172.21.15.27:0/158355035 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f2aa403b4f0 con 0x7f2ab80760a0 2024-09-21T07:10:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.268+0000 7f2ab5600640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2aa00b6550 0x7f2aa00b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.268+0000 7f2ab5600640 1 -- 172.21.15.27:0/158355035 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8492+0+0 (secure 0 0 0) 0x7f2aa4107dc0 con 0x7f2ab80760a0 2024-09-21T07:10:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.268+0000 7f2abd200640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2aa00b6550 0x7f2aa00b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.269+0000 7f2abd200640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2aa00b6550 0x7f2aa00b8a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005fd0 tx=0x7f2aac00b4a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.270+0000 7f2ab5600640 1 -- 172.21.15.27:0/158355035 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ab81a2ba0 con 0x7f2ab80760a0 2024-09-21T07:10:02.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:02.473+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2ab81a8ff0 con 0x7f2ab80760a0 2024-09-21T07:10:02.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:02 smithi153 ceph-mon[34747]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-21T07:10:02.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:02 smithi119 ceph-mon[34519]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-21T07:10:02.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:02 smithi027 ceph-mon[30779]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-21T07:10:03.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:03 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/158355035' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T07:10:03.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:03 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/158355035' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T07:10:03.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:03 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/158355035' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T07:10:04.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.383+0000 7f2ab5600640 1 -- 172.21.15.27:0/158355035 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v78) ==== 228+0+0 (secure 0 0 0) 0x7f2aa40d28d0 con 0x7f2ab80760a0 2024-09-21T07:10:04.385 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' created 2024-09-21T07:10:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.388+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2aa00b6550 msgr2=0x7f2aa00b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.388+0000 7f2abe600640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2aa00b6550 0x7f2aa00b8a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005fd0 tx=0x7f2aac00b4a0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.388+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 msgr2=0x7f2ab819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.388+0000 7f2abe600640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab819cab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2aa402a910 tx=0x7f2aa4004270 comp rx=0 tx=0).stop 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.390+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 shutdown_connections 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.390+0000 7f2abe600640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2aa00b6550 0x7f2aa00b8a60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.390+0000 7f2abe600640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ab810ffb0 0x7f2ab81a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.390+0000 7f2abe600640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ab80760a0 0x7f2ab819cab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.390+0000 7f2abe600640 1 --2- 172.21.15.27:0/158355035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ab8075750 0x7f2ab819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.390+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 >> 172.21.15.27:0/158355035 conn(0x7f2ab80ff480 msgr2=0x7f2ab8109b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.391+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 shutdown_connections 2024-09-21T07:10:04.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:04.392+0000 7f2abe600640 1 -- 172.21.15.27:0/158355035 wait complete. 2024-09-21T07:10:04.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:04 smithi153 ceph-mon[34747]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-21T07:10:04.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:04 smithi153 ceph-mon[34747]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T07:10:04.781 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:04 smithi027 ceph-mon[30779]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-21T07:10:04.781 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:04 smithi027 ceph-mon[30779]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T07:10:04.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:04 smithi119 ceph-mon[34519]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-21T07:10:04.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:04 smithi119 ceph-mon[34519]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T07:10:05.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T07:10:05.415 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:05 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/158355035' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T07:10:05.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:05 smithi027 ceph-mon[30779]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T07:10:05.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:05 smithi027 ceph-mon[30779]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T07:10:05.554 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:05.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:05 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/158355035' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T07:10:05.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:05 smithi153 ceph-mon[34747]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T07:10:05.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:05 smithi153 ceph-mon[34747]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T07:10:05.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:05 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/158355035' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T07:10:05.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:05 smithi119 ceph-mon[34519]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T07:10:05.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:05 smithi119 ceph-mon[34519]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 -- 172.21.15.27:0/3764761101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c105110 msgr2=0x7f0d1c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3764761101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c105110 0x7f0d1c10d660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009900 tx=0x7f0d0402a410 comp rx=0 tx=0).stop 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 -- 172.21.15.27:0/3764761101 shutdown_connections 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3764761101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c10dc20 0x7f0d1c110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3764761101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c105110 0x7f0d1c10d660 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3764761101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1c1047c0 0x7f0d1c104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 -- 172.21.15.27:0/3764761101 >> 172.21.15.27:0/3764761101 conn(0x7f0d1c0fe2c0 msgr2=0x7f0d1c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 -- 172.21.15.27:0/3764761101 shutdown_connections 2024-09-21T07:10:06.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.178+0000 7f0d20a00640 1 -- 172.21.15.27:0/3764761101 wait complete. 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.180+0000 7f0d20a00640 1 Processor -- start 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.180+0000 7f0d20a00640 1 -- start start 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c1047c0 0x7f0d1c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1c105110 0x7f0d1c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c10dc20 0x7f0d1c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d20a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c113840 con 0x7f0d1c10dc20 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d20a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c1136c0 con 0x7f0d1c1047c0 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d20a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c1139c0 con 0x7f0d1c105110 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c10dc20 0x7f0d1c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c1047c0 0x7f0d1c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d11e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1c105110 0x7f0d1c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c1047c0 0x7f0d1c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47076/0 (socket says 172.21.15.27:47076) 2024-09-21T07:10:06.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c10dc20 0x7f0d1c19fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49642/0 (socket says 172.21.15.27:49642) 2024-09-21T07:10:06.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 -- 172.21.15.27:0/3476234591 learned_addr learned my addr 172.21.15.27:0/3476234591 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:06.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 -- 172.21.15.27:0/3476234591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1c105110 msgr2=0x7f0d1c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1c105110 0x7f0d1c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 -- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c10dc20 msgr2=0x7f0d1c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c10dc20 0x7f0d1c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.181+0000 7f0d1a800640 1 -- 172.21.15.27:0/3476234591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c1a0280 con 0x7f0d1c1047c0 2024-09-21T07:10:06.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.182+0000 7f0d1a800640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c1047c0 0x7f0d1c19a310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e9f0 tx=0x7f0d0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:06.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.182+0000 7f0d13e00640 1 -- 172.21.15.27:0/3476234591 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00cd80 con 0x7f0d1c1047c0 2024-09-21T07:10:06.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.182+0000 7f0d13e00640 1 -- 172.21.15.27:0/3476234591 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c0108e0 con 0x7f0d1c1047c0 2024-09-21T07:10:06.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.182+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1a0570 con 0x7f0d1c1047c0 2024-09-21T07:10:06.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.182+0000 7f0d13e00640 1 -- 172.21.15.27:0/3476234591 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c018980 con 0x7f0d1c1047c0 2024-09-21T07:10:06.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.182+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1023e0 con 0x7f0d1c1047c0 2024-09-21T07:10:06.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.183+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c104bd0 con 0x7f0d1c1047c0 2024-09-21T07:10:06.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.186+0000 7f0d13e00640 1 -- 172.21.15.27:0/3476234591 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0d0c010430 con 0x7f0d1c1047c0 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.188+0000 7f0d13e00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0d000b65e0 0x7f0d000b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.188+0000 7f0d13e00640 1 -- 172.21.15.27:0/3476234591 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8985+0+0 (secure 0 0 0) 0x7f0d0c0e7e10 con 0x7f0d1c1047c0 2024-09-21T07:10:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.188+0000 7f0d11e00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0d000b65e0 0x7f0d000b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.189+0000 7f0d11e00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0d000b65e0 0x7f0d000b8af0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009230 tx=0x7f0d04031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.189+0000 7f0d13e00640 1 -- 172.21.15.27:0/3476234591 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0d0c0b2730 con 0x7f0d1c1047c0 2024-09-21T07:10:06.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:06.388+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f0d1c19b440 con 0x7f0d1c1047c0 2024-09-21T07:10:06.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:06 smithi153 ceph-mon[34747]: pgmap v282: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 5 op/s 2024-09-21T07:10:06.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:06 smithi153 ceph-mon[34747]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T07:10:06.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:06 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3476234591' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T07:10:06.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:06 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T07:10:06.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:06 smithi119 ceph-mon[34519]: pgmap v282: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 5 op/s 2024-09-21T07:10:06.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:06 smithi119 ceph-mon[34519]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T07:10:06.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:06 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3476234591' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T07:10:06.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:06 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T07:10:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:06 smithi027 ceph-mon[30779]: pgmap v282: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 5 op/s 2024-09-21T07:10:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:06 smithi027 ceph-mon[30779]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T07:10:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:06 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3476234591' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T07:10:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:06 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.397+0000 7f0d13e00640 1 -- 172.21.15.27:0/3476234591 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v81) ==== 169+0+0 (secure 0 0 0) 0x7f0d0c0b7670 con 0x7f0d1c1047c0 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.399+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0d000b65e0 msgr2=0x7f0d000b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.399+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0d000b65e0 0x7f0d000b8af0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009230 tx=0x7f0d04031400 comp rx=0 tx=0).stop 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.399+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c1047c0 msgr2=0x7f0d1c19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.399+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c1047c0 0x7f0d1c19a310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e9f0 tx=0x7f0d0c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 shutdown_connections 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d1c105110 0x7f0d1c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0d000b65e0 0x7f0d000b8af0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d1c10dc20 0x7f0d1c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 --2- 172.21.15.27:0/3476234591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1c1047c0 0x7f0d1c19a310 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 >> 172.21.15.27:0/3476234591 conn(0x7f0d1c0fe2c0 msgr2=0x7f0d1c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 shutdown_connections 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:07.400+0000 7f0d20a00640 1 -- 172.21.15.27:0/3476234591 wait complete. 2024-09-21T07:10:08.333 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T07:10:08.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:08 smithi027 ceph-mon[30779]: pgmap v285: 129 pgs: 8 creating+peering, 15 unknown, 106 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:08.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:08 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T07:10:08.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:08 smithi027 ceph-mon[30779]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T07:10:08.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:08 smithi153 ceph-mon[34747]: pgmap v285: 129 pgs: 8 creating+peering, 15 unknown, 106 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:08.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:08 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T07:10:08.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:08 smithi153 ceph-mon[34747]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:08.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:08 smithi119 ceph-mon[34519]: pgmap v285: 129 pgs: 8 creating+peering, 15 unknown, 106 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:08.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:08 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T07:10:08.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:08 smithi119 ceph-mon[34519]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 -- 172.21.15.27:0/2028342005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 msgr2=0x7f4058110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 --2- 172.21.15.27:0/2028342005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 0x7f4058110320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4050009ca0 tx=0x7f405002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 -- 172.21.15.27:0/2028342005 shutdown_connections 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 --2- 172.21.15.27:0/2028342005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 0x7f4058110320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 --2- 172.21.15.27:0/2028342005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4058105320 0x7f405810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 --2- 172.21.15.27:0/2028342005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40581049d0 0x7f4058104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 -- 172.21.15.27:0/2028342005 >> 172.21.15.27:0/2028342005 conn(0x7f4058079740 msgr2=0x7f4058079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 -- 172.21.15.27:0/2028342005 shutdown_connections 2024-09-21T07:10:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.370+0000 7f405fc00640 1 -- 172.21.15.27:0/2028342005 wait complete. 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.371+0000 7f405fc00640 1 Processor -- start 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405fc00640 1 -- start start 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405fc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40581049d0 0x7f405819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405fc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4058105320 0x7f405819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 0x7f40581a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405fc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4058113ab0 con 0x7f40581049d0 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405fc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4058113930 con 0x7f405810de30 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405fc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4058113c30 con 0x7f4058105320 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40581049d0 0x7f405819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40581049d0 0x7f405819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35860/0 (socket says 172.21.15.27:35860) 2024-09-21T07:10:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405d200640 1 -- 172.21.15.27:0/3503126057 learned_addr learned my addr 172.21.15.27:0/3503126057 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f405dc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 0x7f40581a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.372+0000 7f4057e00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4058105320 0x7f405819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f405dc00640 1 -- 172.21.15.27:0/3503126057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4058105320 msgr2=0x7f405819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f405dc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4058105320 0x7f405819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f405dc00640 1 -- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40581049d0 msgr2=0x7f405819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f405dc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40581049d0 0x7f405819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f405dc00640 1 -- 172.21.15.27:0/3503126057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40581a2630 con 0x7f405810de30 2024-09-21T07:10:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f405d200640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40581049d0 0x7f405819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f405dc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 0x7f40581a1f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f405002ade0 tx=0x7f4050031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.373+0000 7f4055600640 1 -- 172.21.15.27:0/3503126057 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050003b70 con 0x7f405810de30 2024-09-21T07:10:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.374+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40581a28c0 con 0x7f405810de30 2024-09-21T07:10:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.374+0000 7f4055600640 1 -- 172.21.15.27:0/3503126057 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4050031910 con 0x7f405810de30 2024-09-21T07:10:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.374+0000 7f4055600640 1 -- 172.21.15.27:0/3503126057 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405002f950 con 0x7f405810de30 2024-09-21T07:10:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.374+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40581a9080 con 0x7f405810de30 2024-09-21T07:10:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.375+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405806c240 con 0x7f405810de30 2024-09-21T07:10:09.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.377+0000 7f4055600640 1 -- 172.21.15.27:0/3503126057 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4050004b40 con 0x7f405810de30 2024-09-21T07:10:09.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.379+0000 7f4055600640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40400b6560 0x7f40400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:09.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.379+0000 7f4055600640 1 -- 172.21.15.27:0/3503126057 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8985+0+0 (secure 0 0 0) 0x7f40501088a0 con 0x7f405810de30 2024-09-21T07:10:09.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.379+0000 7f405d200640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40400b6560 0x7f40400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:09.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.380+0000 7f405d200640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40400b6560 0x7f40400b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f404c0058c0 tx=0x7f404c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.385+0000 7f4055600640 1 -- 172.21.15.27:0/3503126057 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40500cede0 con 0x7f405810de30 2024-09-21T07:10:09.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:09.629+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f4058104de0 con 0x7f405810de30 2024-09-21T07:10:10.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.409+0000 7f4055600640 1 -- 172.21.15.27:0/3503126057 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f4050003d10 con 0x7f405810de30 2024-09-21T07:10:10.412 INFO:teuthology.orchestra.run.smithi027.stderr:added data pool 4 to fsmap 2024-09-21T07:10:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40400b6560 msgr2=0x7f40400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40400b6560 0x7f40400b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f404c0058c0 tx=0x7f404c005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 msgr2=0x7f40581a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 0x7f40581a1f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f405002ade0 tx=0x7f4050031660 comp rx=0 tx=0).stop 2024-09-21T07:10:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 shutdown_connections 2024-09-21T07:10:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4058105320 0x7f405819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40400b6560 0x7f40400b8a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f405810de30 0x7f40581a1f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 --2- 172.21.15.27:0/3503126057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40581049d0 0x7f405819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 >> 172.21.15.27:0/3503126057 conn(0x7f4058079740 msgr2=0x7f4058103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 shutdown_connections 2024-09-21T07:10:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:10.413+0000 7f405fc00640 1 -- 172.21.15.27:0/3503126057 wait complete. 2024-09-21T07:10:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:10 smithi153 ceph-mon[34747]: pgmap v287: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:10 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3503126057' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T07:10:10.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:10 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T07:10:10.782 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:10 smithi027 ceph-mon[30779]: pgmap v287: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:10.782 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:10 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3503126057' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T07:10:10.782 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:10 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T07:10:10.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:10 smithi119 ceph-mon[34519]: pgmap v287: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:10.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:10 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3503126057' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T07:10:10.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:10 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T07:10:11.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:10:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:11 smithi027 ceph-mon[30779]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T07:10:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:11 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T07:10:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:11 smithi027 ceph-mon[30779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:11.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:11 smithi153 ceph-mon[34747]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T07:10:11.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:11 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T07:10:11.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:11 smithi153 ceph-mon[34747]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:11.678 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:11 smithi119 ceph-mon[34519]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T07:10:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:11 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T07:10:11.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:11 smithi119 ceph-mon[34519]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:12.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.305+0000 7fda3fe00640 1 -- 172.21.15.27:0/576568576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda38107be0 msgr2=0x7fda381080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:12.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.305+0000 7fda3fe00640 1 --2- 172.21.15.27:0/576568576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda38107be0 0x7fda381080c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda30009cd0 tx=0x7fda3002a8e0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.306+0000 7fda3fe00640 1 -- 172.21.15.27:0/576568576 shutdown_connections 2024-09-21T07:10:12.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.306+0000 7fda3fe00640 1 --2- 172.21.15.27:0/576568576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda38107be0 0x7fda381080c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.306+0000 7fda3fe00640 1 --2- 172.21.15.27:0/576568576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda381030f0 0x7fda38107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.306+0000 7fda3fe00640 1 --2- 172.21.15.27:0/576568576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda381027a0 0x7fda38102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.306+0000 7fda3fe00640 1 -- 172.21.15.27:0/576568576 >> 172.21.15.27:0/576568576 conn(0x7fda38079760 msgr2=0x7fda38079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:12.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.306+0000 7fda3fe00640 1 -- 172.21.15.27:0/576568576 shutdown_connections 2024-09-21T07:10:12.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.306+0000 7fda3fe00640 1 -- 172.21.15.27:0/576568576 wait complete. 2024-09-21T07:10:12.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 Processor -- start 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 -- start start 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda381027a0 0x7fda381a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda381030f0 0x7fda381a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda38107be0 0x7fda381a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda3811a090 con 0x7fda381027a0 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda38119f10 con 0x7fda38107be0 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.307+0000 7fda3fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda3811a210 con 0x7fda381030f0 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda381027a0 0x7fda381a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda38107be0 0x7fda381a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda38107be0 0x7fda381a6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33440/0 (socket says 172.21.15.27:33440) 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 -- 172.21.15.27:0/219858459 learned_addr learned my addr 172.21.15.27:0/219858459 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:12.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3ca00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda381030f0 0x7fda381a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 -- 172.21.15.27:0/219858459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda381030f0 msgr2=0x7fda381a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda381030f0 0x7fda381a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 -- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda381027a0 msgr2=0x7fda381a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda381027a0 0x7fda381a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3de00640 1 -- 172.21.15.27:0/219858459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda381a6a70 con 0x7fda38107be0 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3d400640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda381027a0 0x7fda381a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.308+0000 7fda3ca00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda381030f0 0x7fda381a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.309+0000 7fda3de00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda38107be0 0x7fda381a6340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda30033040 tx=0x7fda30009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.309+0000 7fda36000640 1 -- 172.21.15.27:0/219858459 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30004040 con 0x7fda38107be0 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.309+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda381a6d00 con 0x7fda38107be0 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.309+0000 7fda36000640 1 -- 172.21.15.27:0/219858459 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda30004690 con 0x7fda38107be0 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.309+0000 7fda36000640 1 -- 172.21.15.27:0/219858459 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30030d90 con 0x7fda38107be0 2024-09-21T07:10:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.309+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda381ad4c0 con 0x7fda38107be0 2024-09-21T07:10:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.310+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3806c2c0 con 0x7fda38107be0 2024-09-21T07:10:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.313+0000 7fda36000640 1 -- 172.21.15.27:0/219858459 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fda300041e0 con 0x7fda38107be0 2024-09-21T07:10:12.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.315+0000 7fda36000640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda200b6560 0x7fda200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:12.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.315+0000 7fda36000640 1 -- 172.21.15.27:0/219858459 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7fda3002e030 con 0x7fda38107be0 2024-09-21T07:10:12.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.315+0000 7fda3d400640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda200b6560 0x7fda200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:12.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.316+0000 7fda3d400640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda200b6560 0x7fda200b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0095f0 tx=0x7fda2c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:12.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.316+0000 7fda36000640 1 -- 172.21.15.27:0/219858459 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda300cecd0 con 0x7fda38107be0 2024-09-21T07:10:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.555+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda38107620 con 0x7fda38107be0 2024-09-21T07:10:12.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.556+0000 7fda36000640 1 -- 172.21.15.27:0/219858459 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7fda38107620 con 0x7fda38107be0 2024-09-21T07:10:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:12.558 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"btime":"2024-09-21T07:10:10:395744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24713,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24736,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24740,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24748,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:09.632393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24773,"qdb_cluster":[24773]},"id":1}]} 2024-09-21T07:10:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-09-21T07:10:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda200b6560 msgr2=0x7fda200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda200b6560 0x7fda200b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0095f0 tx=0x7fda2c005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda38107be0 msgr2=0x7fda381a6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda38107be0 0x7fda381a6340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda30033040 tx=0x7fda30009360 comp rx=0 tx=0).stop 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 shutdown_connections 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda381030f0 0x7fda381a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda200b6560 0x7fda200b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda38107be0 0x7fda381a6340 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 --2- 172.21.15.27:0/219858459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda381027a0 0x7fda381a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.558+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 >> 172.21.15.27:0/219858459 conn(0x7fda38079760 msgr2=0x7fda381055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.559+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 shutdown_connections 2024-09-21T07:10:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:12.559+0000 7fda3fe00640 1 -- 172.21.15.27:0/219858459 wait complete. 2024-09-21T07:10:12.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:12 smithi153 ceph-mon[34747]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:12.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:12 smithi119 ceph-mon[34519]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:12.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:12 smithi027 ceph-mon[30779]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:13.366 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:10:13.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:13 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/219858459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:13.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:13 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/219858459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:13.753 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:13.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:13 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/219858459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.369+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1280737202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 msgr2=0x7fb5e4116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.369+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1280737202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 0x7fb5e4116b60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009ca0 tx=0x7fb5dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.369+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1280737202 shutdown_connections 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.369+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1280737202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 0x7fb5e4116b60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.370+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1280737202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e4078290 0x7fb5e4114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.370+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1280737202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4077940 0x7fb5e4077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.370+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1280737202 >> 172.21.15.27:0/1280737202 conn(0x7fb5e4101810 msgr2=0x7fb5e4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.370+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1280737202 shutdown_connections 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.370+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1280737202 wait complete. 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.371+0000 7fb5e8c00640 1 Processor -- start 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.371+0000 7fb5e8c00640 1 -- start start 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e8c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e4077940 0x7fb5e41a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e8c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4078290 0x7fb5e41a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 0x7fb5e41a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e8c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e411a0c0 con 0x7fb5e4114740 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e8c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4119f40 con 0x7fb5e4077940 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e8c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e411a240 con 0x7fb5e4078290 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 0x7fb5e41a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5d9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4078290 0x7fb5e41a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5d9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4078290 0x7fb5e41a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45534/0 (socket says 172.21.15.27:45534) 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5d9400640 1 -- 172.21.15.27:0/1783046056 learned_addr learned my addr 172.21.15.27:0/1783046056 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 0x7fb5e41a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35888/0 (socket says 172.21.15.27:35888) 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.372+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e4077940 0x7fb5e41a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5d9400640 1 -- 172.21.15.27:0/1783046056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e4077940 msgr2=0x7fb5e41a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5d9400640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e4077940 0x7fb5e41a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5d9400640 1 -- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 msgr2=0x7fb5e41a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5d9400640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 0x7fb5e41a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5d9400640 1 -- 172.21.15.27:0/1783046056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e41a69e0 con 0x7fb5e4078290 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5d9400640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4078290 0x7fb5e41a1010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00d740 tx=0x7fb5cc00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5db400640 1 -- 172.21.15.27:0/1783046056 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc017070 con 0x7fb5e4078290 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e41a6cd0 con 0x7fb5e4078290 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5db400640 1 -- 172.21.15.27:0/1783046056 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5cc00de50 con 0x7fb5e4078290 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.373+0000 7fb5db400640 1 -- 172.21.15.27:0/1783046056 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00f620 con 0x7fb5e4078290 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.374+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e41ad430 con 0x7fb5e4078290 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.374+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4077d50 con 0x7fb5e4078290 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.377+0000 7fb5db400640 1 -- 172.21.15.27:0/1783046056 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fb5cc01a070 con 0x7fb5e4078290 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.379+0000 7fb5db400640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6560 0x7fb5c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.379+0000 7fb5db400640 1 -- 172.21.15.27:0/1783046056 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7fb5cc0e6ff0 con 0x7fb5e4078290 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.379+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6560 0x7fb5c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.380+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6560 0x7fb5c80b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009bd0 tx=0x7fb5d4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:14.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.381+0000 7fb5db400640 1 -- 172.21.15.27:0/1783046056 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5cc0b18f0 con 0x7fb5e4078290 2024-09-21T07:10:14.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.580+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5e41a1ba0 con 0x7fb5e4078290 2024-09-21T07:10:14.582 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:14 smithi027 ceph-mon[30779]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:14.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.582+0000 7fb5db400640 1 -- 172.21.15.27:0/1783046056 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24538 (secure 0 0 0) 0x7fb5cc0cee20 con 0x7fb5e4078290 2024-09-21T07:10:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:14.584 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:09.632242+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:10:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6560 msgr2=0x7fb5c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6560 0x7fb5c80b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009bd0 tx=0x7fb5d4005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4078290 msgr2=0x7fb5e41a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4078290 0x7fb5e41a1010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00d740 tx=0x7fb5cc00dc00 comp rx=0 tx=0).stop 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 shutdown_connections 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4078290 0x7fb5e41a1010 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6560 0x7fb5c80b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4114740 0x7fb5e41a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 --2- 172.21.15.27:0/1783046056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e4077940 0x7fb5e41a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.586+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 >> 172.21.15.27:0/1783046056 conn(0x7fb5e4101810 msgr2=0x7fb5e4116760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.587+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 shutdown_connections 2024-09-21T07:10:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:14.587+0000 7fb5e8c00640 1 -- 172.21.15.27:0/1783046056 wait complete. 2024-09-21T07:10:14.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:14 smithi153 ceph-mon[34747]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:14.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:14 smithi119 ceph-mon[34519]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:15.340 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:10:15.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:15 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1783046056' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:15.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:15 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1783046056' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:15.729 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:15.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:15 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1783046056' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:16.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.351+0000 7f3e7a000640 1 -- 172.21.15.27:0/38981500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7410b6c0 msgr2=0x7f3e7410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:16.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.351+0000 7f3e7a000640 1 --2- 172.21.15.27:0/38981500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7410b6c0 0x7f3e7410bad0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009900 tx=0x7f3e5c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.352+0000 7f3e7a000640 1 -- 172.21.15.27:0/38981500 shutdown_connections 2024-09-21T07:10:16.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.352+0000 7f3e7a000640 1 --2- 172.21.15.27:0/38981500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e7410c010 0x7f3e74112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.352+0000 7f3e7a000640 1 --2- 172.21.15.27:0/38981500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7410b6c0 0x7f3e7410bad0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.352+0000 7f3e7a000640 1 --2- 172.21.15.27:0/38981500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e74105d80 0x7f3e74106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.352+0000 7f3e7a000640 1 -- 172.21.15.27:0/38981500 >> 172.21.15.27:0/38981500 conn(0x7f3e74101810 msgr2=0x7f3e74103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:16.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.352+0000 7f3e7a000640 1 -- 172.21.15.27:0/38981500 shutdown_connections 2024-09-21T07:10:16.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.352+0000 7f3e7a000640 1 -- 172.21.15.27:0/38981500 wait complete. 2024-09-21T07:10:16.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 Processor -- start 2024-09-21T07:10:16.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 -- start start 2024-09-21T07:10:16.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e74105d80 0x7f3e7419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:16.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e7410b6c0 0x7f3e7419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:16.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e7410c010 0x7f3e741a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:16.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74115d20 con 0x7f3e74105d80 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74115ba0 con 0x7f3e7410c010 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.353+0000 7f3e7a000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74115ea0 con 0x7f3e7410b6c0 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e73400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e74105d80 0x7f3e7419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e73400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e74105d80 0x7f3e7419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35906/0 (socket says 172.21.15.27:35906) 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e73400640 1 -- 172.21.15.27:0/668065984 learned_addr learned my addr 172.21.15.27:0/668065984 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e6aa00640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e7410b6c0 0x7f3e7419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e73e00640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e7410c010 0x7f3e741a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e6aa00640 1 -- 172.21.15.27:0/668065984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e7410c010 msgr2=0x7f3e741a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e6aa00640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e7410c010 0x7f3e741a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e6aa00640 1 -- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e74105d80 msgr2=0x7f3e7419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e6aa00640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e74105d80 0x7f3e7419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e6aa00640 1 -- 172.21.15.27:0/668065984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e741a2660 con 0x7f3e7410b6c0 2024-09-21T07:10:16.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e73400640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e74105d80 0x7f3e7419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:16.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.354+0000 7f3e6aa00640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e7410b6c0 0x7f3e7419cc90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009230 tx=0x7f3e5c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:16.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.355+0000 7f3e70c00640 1 -- 172.21.15.27:0/668065984 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c03f070 con 0x7f3e7410b6c0 2024-09-21T07:10:16.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.355+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e741a28f0 con 0x7f3e7410b6c0 2024-09-21T07:10:16.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.355+0000 7f3e70c00640 1 -- 172.21.15.27:0/668065984 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5c031b60 con 0x7f3e7410b6c0 2024-09-21T07:10:16.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.355+0000 7f3e70c00640 1 -- 172.21.15.27:0/668065984 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c035030 con 0x7f3e7410b6c0 2024-09-21T07:10:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.355+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e741a90b0 con 0x7f3e7410b6c0 2024-09-21T07:10:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.356+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e74107000 con 0x7f3e7410b6c0 2024-09-21T07:10:16.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.359+0000 7f3e70c00640 1 -- 172.21.15.27:0/668065984 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3e5c042070 con 0x7f3e7410b6c0 2024-09-21T07:10:16.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.361+0000 7f3e70c00640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e580b6560 0x7f3e580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:16.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.361+0000 7f3e70c00640 1 -- 172.21.15.27:0/668065984 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f3e5c1084b0 con 0x7f3e7410b6c0 2024-09-21T07:10:16.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.361+0000 7f3e73400640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e580b6560 0x7f3e580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:16.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.362+0000 7f3e73400640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e580b6560 0x7f3e580b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e64011500 tx=0x7f3e64005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:16.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.363+0000 7f3e70c00640 1 -- 172.21.15.27:0/668065984 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3e5c0d2db0 con 0x7f3e7410b6c0 2024-09-21T07:10:16.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.565+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e7410dde0 con 0x7f3e7410b6c0 2024-09-21T07:10:16.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.567+0000 7f3e70c00640 1 -- 172.21.15.27:0/668065984 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24538 (secure 0 0 0) 0x7f3e5c0d7cf0 con 0x7f3e7410b6c0 2024-09-21T07:10:16.572 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:16.572 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:09.632242+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:10:16.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e580b6560 msgr2=0x7f3e580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e580b6560 0x7f3e580b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e64011500 tx=0x7f3e64005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e7410b6c0 msgr2=0x7f3e7419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e7410b6c0 0x7f3e7419cc90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009230 tx=0x7f3e5c003c10 comp rx=0 tx=0).stop 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 shutdown_connections 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e7410b6c0 0x7f3e7419cc90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e580b6560 0x7f3e580b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e7410c010 0x7f3e741a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 --2- 172.21.15.27:0/668065984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e74105d80 0x7f3e7419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 >> 172.21.15.27:0/668065984 conn(0x7f3e74101810 msgr2=0x7f3e74103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 shutdown_connections 2024-09-21T07:10:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:16.570+0000 7f3e7a000640 1 -- 172.21.15.27:0/668065984 wait complete. 2024-09-21T07:10:16.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:16 smithi119 ceph-mon[34519]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:16.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:16 smithi027 ceph-mon[30779]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:16.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:16 smithi153 ceph-mon[34747]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:17.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:10:17.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:17 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/668065984' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:17.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:17 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/668065984' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:17.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:17 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/668065984' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 -- 172.21.15.27:0/3097858959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410b6c0 msgr2=0x7f94c410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 --2- 172.21.15.27:0/3097858959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410b6c0 0x7f94c410bad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009900 tx=0x7f94ac02a410 comp rx=0 tx=0).stop 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 -- 172.21.15.27:0/3097858959 shutdown_connections 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 --2- 172.21.15.27:0/3097858959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c410c010 0x7f94c4112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 --2- 172.21.15.27:0/3097858959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410b6c0 0x7f94c410bad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 --2- 172.21.15.27:0/3097858959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 0x7f94c4106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 -- 172.21.15.27:0/3097858959 >> 172.21.15.27:0/3097858959 conn(0x7f94c4101810 msgr2=0x7f94c4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 -- 172.21.15.27:0/3097858959 shutdown_connections 2024-09-21T07:10:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.418+0000 7f94cbc00640 1 -- 172.21.15.27:0/3097858959 wait complete. 2024-09-21T07:10:18.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.420+0000 7f94cbc00640 1 Processor -- start 2024-09-21T07:10:18.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.420+0000 7f94cbc00640 1 -- start start 2024-09-21T07:10:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94cbc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 0x7f94c419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94cbc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c410b6c0 0x7f94c419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94cbc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410c010 0x7f94c41a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94cbc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4115d20 con 0x7f94c410c010 2024-09-21T07:10:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94cbc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4115ba0 con 0x7f94c410b6c0 2024-09-21T07:10:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94cbc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4115ea0 con 0x7f94c4105d80 2024-09-21T07:10:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94c9200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 0x7f94c419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c410b6c0 0x7f94c419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94c9200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 0x7f94c419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50268/0 (socket says 172.21.15.27:50268) 2024-09-21T07:10:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94c9200640 1 -- 172.21.15.27:0/1209353844 learned_addr learned my addr 172.21.15.27:0/1209353844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.421+0000 7f94c9c00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410c010 0x7f94c41a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c9200640 1 -- 172.21.15.27:0/1209353844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c410b6c0 msgr2=0x7f94c419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c9200640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c410b6c0 0x7f94c419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c9200640 1 -- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410c010 msgr2=0x7f94c41a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c9200640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410c010 0x7f94c41a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c9200640 1 -- 172.21.15.27:0/1209353844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c41a2620 con 0x7f94c4105d80 2024-09-21T07:10:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c9200640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 0x7f94c419c6b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f94b400e9f0 tx=0x7f94b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c2000640 1 -- 172.21.15.27:0/1209353844 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b400cd80 con 0x7f94c4105d80 2024-09-21T07:10:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c41a2910 con 0x7f94c4105d80 2024-09-21T07:10:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c2000640 1 -- 172.21.15.27:0/1209353844 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b40108e0 con 0x7f94c4105d80 2024-09-21T07:10:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.422+0000 7f94c2000640 1 -- 172.21.15.27:0/1209353844 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4018980 con 0x7f94c4105d80 2024-09-21T07:10:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.423+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c41a9070 con 0x7f94c4105d80 2024-09-21T07:10:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.424+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c4106190 con 0x7f94c4105d80 2024-09-21T07:10:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.426+0000 7f94c2000640 1 -- 172.21.15.27:0/1209353844 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f94b4010430 con 0x7f94c4105d80 2024-09-21T07:10:18.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.428+0000 7f94c2000640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94a80b6560 0x7f94a80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:18.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.428+0000 7f94c2000640 1 -- 172.21.15.27:0/1209353844 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f94b40e7e90 con 0x7f94c4105d80 2024-09-21T07:10:18.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.428+0000 7f94bbe00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94a80b6560 0x7f94a80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:18.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.429+0000 7f94bbe00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94a80b6560 0x7f94a80b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94ac02aa00 tx=0x7f94ac005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:18.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.429+0000 7f94c2000640 1 -- 172.21.15.27:0/1209353844 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94b40b2790 con 0x7f94c4105d80 2024-09-21T07:10:18.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.671+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94c419d7e0 con 0x7f94c4105d80 2024-09-21T07:10:18.674 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:18 smithi027 ceph-mon[30779]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:18.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.672+0000 7f94c2000640 1 -- 172.21.15.27:0/1209353844 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7f94b40b76d0 con 0x7f94c4105d80 2024-09-21T07:10:18.675 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:18.675 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"btime":"2024-09-21T07:10:10:395744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24713,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24736,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24740,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24748,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:09.632393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24773,"qdb_cluster":[24773]},"id":1}]} 2024-09-21T07:10:18.675 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-09-21T07:10:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94a80b6560 msgr2=0x7f94a80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94a80b6560 0x7f94a80b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94ac02aa00 tx=0x7f94ac005a20 comp rx=0 tx=0).stop 2024-09-21T07:10:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 msgr2=0x7f94c419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 0x7f94c419c6b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f94b400e9f0 tx=0x7f94b400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 shutdown_connections 2024-09-21T07:10:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f94a80b6560 0x7f94a80b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94c410c010 0x7f94c41a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c410b6c0 0x7f94c419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 --2- 172.21.15.27:0/1209353844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94c4105d80 0x7f94c419c6b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:18.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 >> 172.21.15.27:0/1209353844 conn(0x7f94c4101810 msgr2=0x7f94c4103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:18.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 shutdown_connections 2024-09-21T07:10:18.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:18.675+0000 7f94cbc00640 1 -- 172.21.15.27:0/1209353844 wait complete. 2024-09-21T07:10:18.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:18 smithi119 ceph-mon[34519]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:18.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:18 smithi153 ceph-mon[34747]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:19.348 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:10:19.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:19 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1209353844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:19.734 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:19.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:19 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1209353844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:19.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:19 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1209353844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:20.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 -- 172.21.15.27:0/995173775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410c020 msgr2=0x7f2b04112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:20.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 --2- 172.21.15.27:0/995173775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410c020 0x7f2b04112850 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009ca0 tx=0x7f2afc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 -- 172.21.15.27:0/995173775 shutdown_connections 2024-09-21T07:10:20.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 --2- 172.21.15.27:0/995173775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410c020 0x7f2b04112850 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 --2- 172.21.15.27:0/995173775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b0410b6d0 0x7f2b0410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 --2- 172.21.15.27:0/995173775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b04105d90 0x7f2b041061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 -- 172.21.15.27:0/995173775 >> 172.21.15.27:0/995173775 conn(0x7f2b04101860 msgr2=0x7f2b04103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:20.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 -- 172.21.15.27:0/995173775 shutdown_connections 2024-09-21T07:10:20.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.346+0000 7f2b09e00640 1 -- 172.21.15.27:0/995173775 wait complete. 2024-09-21T07:10:20.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.347+0000 7f2b09e00640 1 Processor -- start 2024-09-21T07:10:20.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.347+0000 7f2b09e00640 1 -- start start 2024-09-21T07:10:20.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b09e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b04105d90 0x7f2b0419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:20.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b09e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410b6d0 0x7f2b0419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:20.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b09e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b0410c020 0x7f2b041a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:20.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b09e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04115ce0 con 0x7f2b0410b6d0 2024-09-21T07:10:20.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b09e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04115b60 con 0x7f2b0410c020 2024-09-21T07:10:20.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b09e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04115e60 con 0x7f2b04105d90 2024-09-21T07:10:20.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410b6d0 0x7f2b0419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:20.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b02a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b04105d90 0x7f2b0419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:20.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b02a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b04105d90 0x7f2b0419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50284/0 (socket says 172.21.15.27:50284) 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b02a00640 1 -- 172.21.15.27:0/3360474051 learned_addr learned my addr 172.21.15.27:0/3360474051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.348+0000 7f2b03400640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b0410c020 0x7f2b041a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2afbe00640 1 -- 172.21.15.27:0/3360474051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b04105d90 msgr2=0x7f2b0419c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2afbe00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b04105d90 0x7f2b0419c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2afbe00640 1 -- 172.21.15.27:0/3360474051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b0410c020 msgr2=0x7f2b041a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2afbe00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b0410c020 0x7f2b041a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2afbe00640 1 -- 172.21.15.27:0/3360474051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b041a26e0 con 0x7f2b0410b6d0 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2afbe00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410b6d0 0x7f2b0419cd10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b0406ca80 tx=0x7f2aec00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2b08a00640 1 -- 172.21.15.27:0/3360474051 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00cc40 con 0x7f2b0410b6d0 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b041a29d0 con 0x7f2b0410b6d0 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2b08a00640 1 -- 172.21.15.27:0/3360474051 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec00cde0 con 0x7f2b0410b6d0 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2b08a00640 1 -- 172.21.15.27:0/3360474051 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec018780 con 0x7f2b0410b6d0 2024-09-21T07:10:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.349+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b041a9130 con 0x7f2b0410b6d0 2024-09-21T07:10:20.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.350+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b041061a0 con 0x7f2b0410b6d0 2024-09-21T07:10:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.355+0000 7f2b08a00640 1 -- 172.21.15.27:0/3360474051 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f2aec010a90 con 0x7f2b0410b6d0 2024-09-21T07:10:20.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.356+0000 7f2b08a00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2ae80b6560 0x7f2ae80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:20.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.356+0000 7f2b08a00640 1 -- 172.21.15.27:0/3360474051 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f2aec0e7d50 con 0x7f2b0410b6d0 2024-09-21T07:10:20.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.357+0000 7f2b02a00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2ae80b6560 0x7f2ae80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:20.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.357+0000 7f2b02a00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2ae80b6560 0x7f2ae80b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009cd0 tx=0x7f2af4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:20.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.359+0000 7f2b08a00640 1 -- 172.21.15.27:0/3360474051 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2aec0b2650 con 0x7f2b0410b6d0 2024-09-21T07:10:20.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.563+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b0419d8a0 con 0x7f2b0410b6d0 2024-09-21T07:10:20.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.565+0000 7f2b08a00640 1 -- 172.21.15.27:0/3360474051 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24538 (secure 0 0 0) 0x7f2aec0b7590 con 0x7f2b0410b6d0 2024-09-21T07:10:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:20.567 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:09.632242+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:10:20.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2ae80b6560 msgr2=0x7f2ae80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2ae80b6560 0x7f2ae80b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009cd0 tx=0x7f2af4005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410b6d0 msgr2=0x7f2b0419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410b6d0 0x7f2b0419cd10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b0406ca80 tx=0x7f2aec00ee50 comp rx=0 tx=0).stop 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 shutdown_connections 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2ae80b6560 0x7f2ae80b8a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b0410c020 0x7f2b041a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b0410b6d0 0x7f2b0419cd10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 --2- 172.21.15.27:0/3360474051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b04105d90 0x7f2b0419c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.567+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 >> 172.21.15.27:0/3360474051 conn(0x7f2b04101860 msgr2=0x7f2b04108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.568+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 shutdown_connections 2024-09-21T07:10:20.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:20.568+0000 7f2b09e00640 1 -- 172.21.15.27:0/3360474051 wait complete. 2024-09-21T07:10:20.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:20 smithi119 ceph-mon[34519]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:20 smithi027 ceph-mon[30779]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:20.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:20 smithi153 ceph-mon[34747]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:21.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:09:56.473755+0000', 'modified': '2024-09-21T07:10:09.632393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24773}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24773': {'gid': 24773, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.119:6839/1565967470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1565967470}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1565967470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24773, 'qdb_cluster': [24773]} 2024-09-21T07:10:21.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:10:21.301 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T07:10:21.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:21 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3360474051' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:21.685 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:21.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:21 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3360474051' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:21.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:21 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3360474051' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:22.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 -- 172.21.15.27:0/1402970553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838076100 msgr2=0x7f983810fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 --2- 172.21.15.27:0/1402970553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838076100 0x7f983810fb80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9820009900 tx=0x7f982002a410 comp rx=0 tx=0).stop 2024-09-21T07:10:22.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 -- 172.21.15.27:0/1402970553 shutdown_connections 2024-09-21T07:10:22.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 --2- 172.21.15.27:0/1402970553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98381100c0 0x7f9838112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 --2- 172.21.15.27:0/1402970553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838076100 0x7f983810fb80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 --2- 172.21.15.27:0/1402970553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98380757b0 0x7f9838075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 -- 172.21.15.27:0/1402970553 >> 172.21.15.27:0/1402970553 conn(0x7f983807b850 msgr2=0x7f983807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:22.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 -- 172.21.15.27:0/1402970553 shutdown_connections 2024-09-21T07:10:22.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.295+0000 7f983d800640 1 -- 172.21.15.27:0/1402970553 wait complete. 2024-09-21T07:10:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.296+0000 7f983d800640 1 Processor -- start 2024-09-21T07:10:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.296+0000 7f983d800640 1 -- start start 2024-09-21T07:10:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f983d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98380757b0 0x7f983819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f983d800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9838076100 0x7f983819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f983d800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98381100c0 0x7f98381a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f983d800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9838115c70 con 0x7f98380757b0 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f983d800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9838115af0 con 0x7f98381100c0 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f983d800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9838115df0 con 0x7f9838076100 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f9836a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98380757b0 0x7f983819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f9836a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98380757b0 0x7f983819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51712/0 (socket says 172.21.15.27:51712) 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f9836a00640 1 -- 172.21.15.27:0/469221527 learned_addr learned my addr 172.21.15.27:0/469221527 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f982e000640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9838076100 0x7f983819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.297+0000 7f9837400640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98381100c0 0x7f98381a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f9836a00640 1 -- 172.21.15.27:0/469221527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9838076100 msgr2=0x7f983819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f9836a00640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9838076100 0x7f983819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f9836a00640 1 -- 172.21.15.27:0/469221527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98381100c0 msgr2=0x7f98381a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f9836a00640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98381100c0 0x7f98381a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f9836a00640 1 -- 172.21.15.27:0/469221527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98381a2630 con 0x7f98380757b0 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f9836a00640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98380757b0 0x7f983819c6c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f982800e9f0 tx=0x7f982800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f982fe00640 1 -- 172.21.15.27:0/469221527 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982800cd80 con 0x7f98380757b0 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f982fe00640 1 -- 172.21.15.27:0/469221527 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98280108e0 con 0x7f98380757b0 2024-09-21T07:10:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381a2920 con 0x7f98380757b0 2024-09-21T07:10:22.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.298+0000 7f982fe00640 1 -- 172.21.15.27:0/469221527 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828018980 con 0x7f98380757b0 2024-09-21T07:10:22.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.299+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a9080 con 0x7f98380757b0 2024-09-21T07:10:22.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.302+0000 7f984441c740 1 Processor -- start 2024-09-21T07:10:22.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.302+0000 7f984441c740 1 -- start start 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.302+0000 7f984441c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d830a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.302+0000 7f984441c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d830a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.302+0000 7f984441c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d830af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f984441c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8281d50 con 0x55f9d7f63f60 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f984441c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8271180 con 0x55f9d7f3b0a0 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f984441c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8274f90 con 0x55f9d7f43280 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d830a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d830a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51714/0 (socket says 172.21.15.27:51714) 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 -- 172.21.15.27:0/2209186148 learned_addr learned my addr 172.21.15.27:0/2209186148 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f982e000640 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d830af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f982fe00640 1 -- 172.21.15.27:0/469221527 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f9828010430 con 0x7f98380757b0 2024-09-21T07:10:22.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 -- 172.21.15.27:0/2209186148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 msgr2=0x55f9d830af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d830af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 -- 172.21.15.27:0/2209186148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 msgr2=0x55f9d830a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d830a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.303+0000 7f9837400640 1 -- 172.21.15.27:0/2209186148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d823ba70 con 0x55f9d7f63f60 2024-09-21T07:10:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.304+0000 7f9837400640 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d830a490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9830009f50 tx=0x7f98300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4072bd3bd8016e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.304+0000 7f9827e00640 1 -- 172.21.15.27:0/2209186148 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830004d20 con 0x55f9d7f63f60 2024-09-21T07:10:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.304+0000 7f9827e00640 1 -- 172.21.15.27:0/2209186148 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9830004050 con 0x55f9d7f63f60 2024-09-21T07:10:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.304+0000 7f9827e00640 1 -- 172.21.15.27:0/2209186148 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830005220 con 0x55f9d7f63f60 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f982fe00640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f981c0b6560 0x7f981c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f984441c740 1 -- 172.21.15.27:0/2209186148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 msgr2=0x55f9d830a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f984441c740 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d830a490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9830009f50 tx=0x7f98300118d0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f984441c740 1 -- 172.21.15.27:0/2209186148 shutdown_connections 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f984441c740 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d830af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f984441c740 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d830a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f984441c740 1 --2- 172.21.15.27:0/2209186148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d830a490 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f984441c740 1 -- 172.21.15.27:0/2209186148 >> 172.21.15.27:0/2209186148 conn(0x55f9d7f58350 msgr2=0x55f9d8308380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f9836a00640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f981c0b6560 0x7f981c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.305+0000 7f982fe00640 1 -- 172.21.15.27:0/469221527 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f98280e79a0 con 0x7f98380757b0 2024-09-21T07:10:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.306+0000 7f984441c740 1 -- 172.21.15.27:0/2209186148 shutdown_connections 2024-09-21T07:10:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.306+0000 7f984441c740 1 -- 172.21.15.27:0/2209186148 wait complete. 2024-09-21T07:10:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.306+0000 7f9836a00640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f981c0b6560 0x7f981c0b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98280ad860 tx=0x7f98280246c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.306+0000 7f984441c740 1 -- start start 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 1 Processor -- start 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8316010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8316550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f63f60 0x55f9d8321f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f9d80d25f0 con 0x55f9d7f43280 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f9d820c460 con 0x55f9d7f63f60 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8318c70 con 0x55f9d7f3b0a0 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f984441c740 10 client.0 _ll_register_callbacks cb 0x55f9d821c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f982e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8316550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f982e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8316550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51724/0 (socket says 172.21.15.27:51724) 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.307+0000 7f982e000640 1 -- 172.21.15.27:0/3944252757 learned_addr learned my addr 172.21.15.27:0/3944252757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f9837400640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8316010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f9837400640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f63f60 0x55f9d8321f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f982e000640 1 -- 172.21.15.27:0/3944252757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f3b0a0 msgr2=0x55f9d8316010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f982e000640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8316010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f982e000640 1 -- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f63f60 msgr2=0x55f9d8321f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f982e000640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f63f60 0x55f9d8321f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f982e000640 1 -- 172.21.15.27:0/3944252757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d7fcf540 con 0x55f9d7f43280 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f9837400640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f63f60 0x55f9d8321f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f982e000640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8316550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f983806c760 tx=0x7f9820032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b222f15b5f9e9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.308+0000 7f9824c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f9824c00640 1 -- 172.21.15.27:0/3944252757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820035040 con 0x55f9d7f43280 2024-09-21T07:10:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f9824c00640 1 -- 172.21.15.27:0/3944252757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f9820031760 con 0x55f9d7f43280 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9d820ea70 con 0x55f9d7f43280 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f9824c00640 1 -- 172.21.15.27:0/3944252757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9820004040 con 0x55f9d7f43280 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f984441c740 10 client.14835 fetch_fsmap learned FSMap version 17 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f9d82296f0 con 0x55f9d7f43280 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f9824c00640 1 -- 172.21.15.27:0/3944252757 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f9d820ea70 con 0x55f9d7f43280 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.309+0000 7f9824c00640 1 -- 172.21.15.27:0/3944252757 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f9820004750 con 0x55f9d7f43280 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.310+0000 7f984441c740 10 client.14835 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.310+0000 7f984441c740 10 client.14835 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.310+0000 7f984441c740 20 client.14835 populate_metadata read hostname 'smithi027' 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.310+0000 7f984441c740 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7f228e0 0x55f9d8328530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.310+0000 7f984441c740 4 client.14835 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.310+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9d8283ca0 con 0x55f9d7f228e0 2024-09-21T07:10:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.310+0000 7f9837400640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7f228e0 0x55f9d8328530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.311+0000 7f9837400640 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7f228e0 0x55f9d8328530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d742d5f26cd05ce4 server_cookie=d90f6beda2b36bf0 in_seq=0 out_seq=0 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.311+0000 7f9824c00640 10 client.14835 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f9824c00640 1 -- 172.21.15.27:0/3944252757 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9d8283ca0 con 0x55f9d7f228e0 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f9824c00640 10 client.14835 handle_command_reply: tid=0 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 client.14835 shutdown 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 msgr2=0x55f9d8316550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8316550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f983806c760 tx=0x7f9820032e90 comp rx=0 tx=0).stop 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 shutdown_connections 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7f228e0 0x55f9d8328530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f63f60 0x55f9d8321f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f9824c00640 0 client.14835 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8316550 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 --2- 172.21.15.27:0/3944252757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8316010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 >> 172.21.15.27:0/3944252757 conn(0x55f9d7f58350 msgr2=0x55f9d830d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 shutdown_connections 2024-09-21T07:10:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.312+0000 7f984441c740 1 -- 172.21.15.27:0/3944252757 wait complete. 2024-09-21T07:10:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.313+0000 7f984441c740 20 client.14835 trim_cache size 0 max 16384 2024-09-21T07:10:22.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 Processor -- start 2024-09-21T07:10:22.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 -- start start 2024-09-21T07:10:22.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8327e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8321f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f63f60 0x55f9d8322440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8326870 con 0x55f9d7f43280 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8326f40 con 0x55f9d7f3b0a0 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.332+0000 7f984441c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8327610 con 0x55f9d7f63f60 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8321f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f9837400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8327e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8321f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51726/0 (socket says 172.21.15.27:51726) 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 -- 172.21.15.27:0/2157828372 learned_addr learned my addr 172.21.15.27:0/2157828372 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f9837400640 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f63f60 0x55f9d8322440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 -- 172.21.15.27:0/2157828372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f63f60 msgr2=0x55f9d8322440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f63f60 0x55f9d8322440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 -- 172.21.15.27:0/2157828372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 msgr2=0x55f9d8327e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8327e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f982e000640 1 -- 172.21.15.27:0/2157828372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d8315630 con 0x55f9d7f43280 2024-09-21T07:10:22.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.333+0000 7f9837400640 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8327e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:22.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.334+0000 7f982e000640 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8321f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f982002a930 tx=0x7f982002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50848d8e1ac1ee3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:22.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.334+0000 7f9827400640 1 -- 172.21.15.27:0/2157828372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820035030 con 0x55f9d7f43280 2024-09-21T07:10:22.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.334+0000 7f9827400640 1 -- 172.21.15.27:0/2157828372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98200054b0 con 0x55f9d7f43280 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.334+0000 7f9827400640 1 -- 172.21.15.27:0/2157828372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820003670 con 0x55f9d7f43280 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 -- 172.21.15.27:0/2157828372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 msgr2=0x55f9d8321f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8321f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f982002a930 tx=0x7f982002a410 comp rx=0 tx=0).stop 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 -- 172.21.15.27:0/2157828372 shutdown_connections 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f63f60 0x55f9d8322440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f43280 0x55f9d8321f00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 --2- 172.21.15.27:0/2157828372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d8327e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 -- 172.21.15.27:0/2157828372 >> 172.21.15.27:0/2157828372 conn(0x55f9d7f58350 msgr2=0x55f9d8310fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:22.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 -- 172.21.15.27:0/2157828372 shutdown_connections 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 -- 172.21.15.27:0/2157828372 wait complete. 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.335+0000 7f984441c740 1 -- start start 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 1 Processor -- start 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d832a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d832ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d832b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f9d83283b0 con 0x55f9d7f63f60 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8327ce0 con 0x55f9d7f3b0a0 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8328a80 con 0x55f9d7f43280 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f984441c740 10 client.0 _ll_register_callbacks cb 0x7f9820032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f9837400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d832b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f982e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d832ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f9837400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d832b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51728/0 (socket says 172.21.15.27:51728) 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f9837400640 1 -- 172.21.15.27:0/3625316420 learned_addr learned my addr 172.21.15.27:0/3625316420 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f982e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d832ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50342/0 (socket says 172.21.15.27:50342) 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.336+0000 7f9837400640 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d832a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9837400640 1 -- 172.21.15.27:0/3625316420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 msgr2=0x55f9d832ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9837400640 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d832ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9837400640 1 -- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 msgr2=0x55f9d832a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9837400640 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d832a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9837400640 1 -- 172.21.15.27:0/3625316420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d832feb0 con 0x55f9d7f63f60 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9837400640 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d832b1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9830009270 tx=0x7f9830009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c11e43424f4ee40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9824c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9824c00640 1 -- 172.21.15.27:0/3625316420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830004920 con 0x55f9d7f63f60 2024-09-21T07:10:22.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.337+0000 7f9824c00640 1 -- 172.21.15.27:0/3625316420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f9830004ac0 con 0x55f9d7f63f60 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98200054b0 con 0x55f9d7f63f60 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f9824c00640 1 -- 172.21.15.27:0/3625316420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9830028450 con 0x55f9d7f63f60 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f984441c740 10 client.14841 fetch_fsmap learned FSMap version 17 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f9d8393be0 con 0x55f9d7f63f60 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f9824c00640 1 -- 172.21.15.27:0/3625316420 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98200054b0 con 0x55f9d7f63f60 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f9824c00640 1 -- 172.21.15.27:0/3625316420 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f9830025040 con 0x55f9d7f63f60 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f984441c740 10 client.14841 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.338+0000 7f984441c740 10 client.14841 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.339+0000 7f984441c740 20 client.14841 populate_metadata read hostname 'smithi027' 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.339+0000 7f984441c740 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7ef7c90 0x55f9d83971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.339+0000 7f984441c740 4 client.14841 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.339+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "status"}) -- 0x55f9d827f750 con 0x55f9d7ef7c90 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.339+0000 7f982e000640 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7ef7c90 0x55f9d83971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.339+0000 7f982e000640 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7ef7c90 0x55f9d83971e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f7a7672a426cb5 server_cookie=8951cf170ea0fa0c in_seq=0 out_seq=0 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.339+0000 7f9824c00640 10 client.14841 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f9824c00640 1 -- 172.21.15.27:0/3625316420 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f9d827f750 con 0x55f9d7ef7c90 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f9824c00640 10 client.14841 handle_command_reply: tid=0 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 client.14841 shutdown 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 msgr2=0x55f9d832b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d832b1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9830009270 tx=0x7f9830009500 comp rx=0 tx=0).stop 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 shutdown_connections 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f9d7ef7c90 0x55f9d83971e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9d7f43280 0x55f9d832ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f9d7f63f60 0x55f9d832b1a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 --2- 172.21.15.27:0/3625316420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9d7f3b0a0 0x55f9d832a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 >> 172.21.15.27:0/3625316420 conn(0x55f9d7f58350 msgr2=0x55f9d830b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f9824c00640 0 client.14841 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 shutdown_connections 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 1 -- 172.21.15.27:0/3625316420 wait complete. 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.340+0000 7f984441c740 20 client.14841 trim_cache size 0 max 16384 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24773, 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:10:22.349 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:10:22.350 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 24.999825852000001, 2024-09-21T07:10:22.350 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-09-21T07:10:22.350 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-09-21T07:10:22.350 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T07:10:22.350 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 39.157137075000001 2024-09-21T07:10:22.350 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f981c0b6560 msgr2=0x7f981c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f981c0b6560 0x7f981c0b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98280ad860 tx=0x7f98280246c0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98380757b0 msgr2=0x7f983819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98380757b0 0x7f983819c6c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f982800e9f0 tx=0x7f982800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 shutdown_connections 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9838076100 0x7f983819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f981c0b6560 0x7f981c0b8a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98381100c0 0x7f98381a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 --2- 172.21.15.27:0/469221527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98380757b0 0x7f983819c6c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.341+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 >> 172.21.15.27:0/469221527 conn(0x7f983807b850 msgr2=0x7f9838101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.342+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 shutdown_connections 2024-09-21T07:10:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:22.342+0000 7f983d800640 1 -- 172.21.15.27:0/469221527 wait complete. 2024-09-21T07:10:22.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:22 smithi119 ceph-mon[34519]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:22 smithi027 ceph-mon[30779]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:22.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:22 smithi153 ceph-mon[34747]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:23.379 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:10:23.379 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:23.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T07:10:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:24.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 -- 172.21.15.27:0/2956509792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea04105d90 msgr2=0x7fea041061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 --2- 172.21.15.27:0/2956509792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea04105d90 0x7fea041061a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009900 tx=0x7fe9f402a410 comp rx=0 tx=0).stop 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 -- 172.21.15.27:0/2956509792 shutdown_connections 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 --2- 172.21.15.27:0/2956509792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea0410c020 0x7fea04112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 --2- 172.21.15.27:0/2956509792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0410b6d0 0x7fea0410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 --2- 172.21.15.27:0/2956509792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea04105d90 0x7fea041061a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 -- 172.21.15.27:0/2956509792 >> 172.21.15.27:0/2956509792 conn(0x7fea04101860 msgr2=0x7fea04103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 -- 172.21.15.27:0/2956509792 shutdown_connections 2024-09-21T07:10:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.373+0000 7fea0ae00640 1 -- 172.21.15.27:0/2956509792 wait complete. 2024-09-21T07:10:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.374+0000 7fea0ae00640 1 Processor -- start 2024-09-21T07:10:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.374+0000 7fea0ae00640 1 -- start start 2024-09-21T07:10:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea0ae00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea04105d90 0x7fea0419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea0ae00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea0410b6d0 0x7fea0419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea0ae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea0410c020 0x7fea041a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea0ae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea04115ce0 con 0x7fea0410c020 2024-09-21T07:10:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea0ae00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fea04115b60 con 0x7fea04105d90 2024-09-21T07:10:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea0ae00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea04115e60 con 0x7fea0410b6d0 2024-09-21T07:10:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea03e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea04105d90 0x7fea0419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea08e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea0410c020 0x7fea041a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea03e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea04105d90 0x7fea0419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45038/0 (socket says 172.21.15.27:45038) 2024-09-21T07:10:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea03e00640 1 -- 172.21.15.27:0/1433860700 learned_addr learned my addr 172.21.15.27:0/1433860700 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.375+0000 7fea03400640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea0410b6d0 0x7fea0419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea03e00640 1 -- 172.21.15.27:0/1433860700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea0410b6d0 msgr2=0x7fea0419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea03e00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea0410b6d0 0x7fea0419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea03e00640 1 -- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea0410c020 msgr2=0x7fea041a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea03e00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea0410c020 0x7fea041a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea03e00640 1 -- 172.21.15.27:0/1433860700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea041a26e0 con 0x7fea04105d90 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea03400640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea0410b6d0 0x7fea0419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea03e00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea04105d90 0x7fea0419c770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4005940 tx=0x7fe9f40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea00c00640 1 -- 172.21.15.27:0/1433860700 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f403f030 con 0x7fea04105d90 2024-09-21T07:10:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea041a2970 con 0x7fea04105d90 2024-09-21T07:10:24.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea00c00640 1 -- 172.21.15.27:0/1433860700 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f40044c0 con 0x7fea04105d90 2024-09-21T07:10:24.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea00c00640 1 -- 172.21.15.27:0/1433860700 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4035070 con 0x7fea04105d90 2024-09-21T07:10:24.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.376+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea041a9130 con 0x7fea04105d90 2024-09-21T07:10:24.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.378+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea041061a0 con 0x7fea04105d90 2024-09-21T07:10:24.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.380+0000 7fea00c00640 1 -- 172.21.15.27:0/1433860700 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fe9f4042030 con 0x7fea04105d90 2024-09-21T07:10:24.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.382+0000 7fea00c00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e80b6560 0x7fe9e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:24.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.382+0000 7fea00c00640 1 -- 172.21.15.27:0/1433860700 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7fe9f4107a50 con 0x7fea04105d90 2024-09-21T07:10:24.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.382+0000 7fea03400640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e80b6560 0x7fe9e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:24.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.382+0000 7fea03400640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e80b6560 0x7fe9e80b8a70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea04105110 tx=0x7fe9f0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:24.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.386+0000 7fea00c00640 1 -- 172.21.15.27:0/1433860700 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9f40d2350 con 0x7fea04105d90 2024-09-21T07:10:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.627+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea0410de20 con 0x7fea04105d90 2024-09-21T07:10:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.627+0000 7fea00c00640 1 -- 172.21.15.27:0/1433860700 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fe9f40d7290 con 0x7fea04105d90 2024-09-21T07:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.629+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e80b6560 msgr2=0x7fe9e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.629+0000 7fea0ae00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e80b6560 0x7fe9e80b8a70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea04105110 tx=0x7fe9f0005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.629+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea04105d90 msgr2=0x7fea0419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.629+0000 7fea0ae00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea04105d90 0x7fea0419c770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4005940 tx=0x7fe9f40040c0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 shutdown_connections 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea0410b6d0 0x7fea0419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e80b6560 0x7fe9e80b8a70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea0410c020 0x7fea041a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 --2- 172.21.15.27:0/1433860700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea04105d90 0x7fea0419c770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 >> 172.21.15.27:0/1433860700 conn(0x7fea04101860 msgr2=0x7fea0410ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 shutdown_connections 2024-09-21T07:10:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:24.630+0000 7fea0ae00640 1 -- 172.21.15.27:0/1433860700 wait complete. 2024-09-21T07:10:24.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:24 smithi119 ceph-mon[34519]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:24 smithi027 ceph-mon[30779]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:24.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:24 smithi153 ceph-mon[34747]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:25.424 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:10:25.425 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T07:10:25.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:25.425 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-21T07:10:25.464 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:10:25.464 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:10:25.464 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:10:25.464 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:10:25.464 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42123sec preferred_lft 42123sec 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T07:10:25.465 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:10:25.466 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:10:25.466 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-21T07:10:25.466 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-21T07:10:25.467 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:10:25.467 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:10:25.467 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:10:25.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:10:25.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-09-21T07:10:25.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-09-21T07:10:25.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:10:25.467 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:10:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:10:25.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:25 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1433860700' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T07:10:25.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:25 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1433860700' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T07:10:25.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:10:25.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:25.893 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:10:25.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:25 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1433860700' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T07:10:25.952 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-21T07:10:25.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:25.955 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-21T07:10:26.001 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-21T07:10:26.001 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:10:26.001 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:10:26.002 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:10:26.002 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:10:26.002 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:10:26.002 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:10:26.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:10:26.003 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:10:26.003 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:10:26.003 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:10:26.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:10:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:10:26.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:26.647 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T07:10:26.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:26.684 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-21T07:10:26.760 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:10:26.760 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:10:26.760 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:10:26.760 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T07:10:26.760 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:10:26.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:26 smithi119 ceph-mon[34519]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:26.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:26 smithi027 ceph-mon[30779]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:10:26.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:26 smithi153 ceph-mon[34747]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:10:26.942 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T07:10:26.942 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:10:27.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:10:27.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:10:27.223 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:10:27.223 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:10:27.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-21T07:10:27.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:10:27.223 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:10:27.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:10:27.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:10:27.400 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T07:10:27.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:10:27.400 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:27.436 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T07:10:27.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:10:27.436 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:27.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T07:10:27.616 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T07:10:27.653 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T07:10:27.653 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T07:10:27.653 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T07:10:27.653 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T07:10:27.653 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T07:10:27.654 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/be1b8c0208886fbd149dd8d1782423391826eeae8d4988b184c9b550e622299a/merged 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5da5ab1f6956328ce9246a8cb43275d95c616336ca9d6f358d129123ba42614f/merged 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/e4c715cd937effa02ffec9838ea1e00ed4457ab50ed37925bc200f11b4875750/merged 2024-09-21T07:10:27.655 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9dc8dbd4e2725d113c5dedf6857d4076f47aab9df618f943334c7900553a3d71/merged 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9e70f9b9e351326781a119a5c83e6adfea78055eac34ca563672df31d5a828e1/merged 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f9b332879b284a4f6d50ccde918f1dcc7d44d04b8316ff524c5dfeef2fb6eebb/merged 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5639ac135acb31e01ced4aed58abdeb49ee1836550550f56c4f98c2149678f7f/merged 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6ca8766a252795522a888b5f7186242e4db289fefafd59b8be83b109f97fa012/merged 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/8981d44ad3e6a886208d0c18c3ce230732fd80e8d1bb35fb6d4d32b4208ccade/merged 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/293518490d5f19bc808e114424cdb4bcb9e759db707f681a81f4ed9d0fd398f1/merged 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:10:27.656 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:10:27.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:27.657 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T07:10:27.733 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T07:10:27.734 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T07:10:27.778 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T07:10:27.835 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T07:10:27.873 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T07:10:27.873 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T07:10:27.873 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T07:10:27.874 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T07:10:27.875 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/be1b8c0208886fbd149dd8d1782423391826eeae8d4988b184c9b550e622299a/merged 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5da5ab1f6956328ce9246a8cb43275d95c616336ca9d6f358d129123ba42614f/merged 2024-09-21T07:10:27.876 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/e4c715cd937effa02ffec9838ea1e00ed4457ab50ed37925bc200f11b4875750/merged 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9dc8dbd4e2725d113c5dedf6857d4076f47aab9df618f943334c7900553a3d71/merged 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9e70f9b9e351326781a119a5c83e6adfea78055eac34ca563672df31d5a828e1/merged 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f9b332879b284a4f6d50ccde918f1dcc7d44d04b8316ff524c5dfeef2fb6eebb/merged 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5639ac135acb31e01ced4aed58abdeb49ee1836550550f56c4f98c2149678f7f/merged 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6ca8766a252795522a888b5f7186242e4db289fefafd59b8be83b109f97fa012/merged 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/8981d44ad3e6a886208d0c18c3ce230732fd80e8d1bb35fb6d4d32b4208ccade/merged 2024-09-21T07:10:27.877 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/293518490d5f19bc808e114424cdb4bcb9e759db707f681a81f4ed9d0fd398f1/merged 2024-09-21T07:10:27.878 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T07:10:27.878 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:10:27.878 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:10:27.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:27.879 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T07:10:28.044 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:2024-09-21T07:10:28.042+0000 7fd2bce22480 -1 init, newargv = 0x7fd2a0054370 newargc=15 2024-09-21T07:10:28.044 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[76725]: starting ceph client 2024-09-21T07:10:28.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[76725]: starting fuse 2024-09-21T07:10:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:28 smithi119 ceph-mon[34519]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:28.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:28 smithi027 ceph-mon[30779]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:28 smithi153 ceph-mon[34747]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:29.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T07:10:29.124 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T07:10:29.163 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T07:10:29.164 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/be1b8c0208886fbd149dd8d1782423391826eeae8d4988b184c9b550e622299a/merged 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5da5ab1f6956328ce9246a8cb43275d95c616336ca9d6f358d129123ba42614f/merged 2024-09-21T07:10:29.165 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/e4c715cd937effa02ffec9838ea1e00ed4457ab50ed37925bc200f11b4875750/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9dc8dbd4e2725d113c5dedf6857d4076f47aab9df618f943334c7900553a3d71/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/9e70f9b9e351326781a119a5c83e6adfea78055eac34ca563672df31d5a828e1/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f9b332879b284a4f6d50ccde918f1dcc7d44d04b8316ff524c5dfeef2fb6eebb/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5639ac135acb31e01ced4aed58abdeb49ee1836550550f56c4f98c2149678f7f/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6ca8766a252795522a888b5f7186242e4db289fefafd59b8be83b109f97fa012/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/8981d44ad3e6a886208d0c18c3ce230732fd80e8d1bb35fb6d4d32b4208ccade/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/293518490d5f19bc808e114424cdb4bcb9e759db707f681a81f4ed9d0fd398f1/merged 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:10:29.166 INFO:teuthology.orchestra.run.smithi027.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:29.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:29.168 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T07:10:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:120 2024-09-21T07:10:29.245 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T07:10:29.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2024-09-21T07:10:29.245 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2024-09-21T07:10:29.245 DEBUG:teuthology.orchestra.run.smithi027:> import re 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> import os 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T07:10:29.246 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> return f 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.admin")) 2024-09-21T07:10:29.247 DEBUG:teuthology.orchestra.run.smithi027:> ' 2024-09-21T07:10:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.admin.76725.asok 2024-09-21T07:10:29.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:10:29.430 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76725.asok 2024-09-21T07:10:29.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:29.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76725.asok status 2024-09-21T07:10:29.686 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "admin", 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "76725", 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24884, 2024-09-21T07:10:29.687 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 24884 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2288821098 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:10:29.688 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2288821098 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.24884 172.21.15.27:0/2288821098", 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "172.21.15.27:0/2288821098", 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 17, 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 82, 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2024-09-21T07:10:29.689 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:10:29.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:29.707 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2024-09-21T07:10:29.707 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:29.743 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2024-09-21T07:10:29.744 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:29.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:29.744 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:29.842 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T07:10:29.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:10:29.951 INFO:teuthology.orchestra.run.smithi027.stdout:# file: . 2024-09-21T07:10:29.951 INFO:teuthology.orchestra.run.smithi027.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T07:10:29.951 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:29.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:10:29.960 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:10:29.960 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:10:29.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T07:10:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:30.729 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:30 smithi027 ceph-mon[30779]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:30.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:30 smithi119 ceph-mon[34519]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:30.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:30 smithi153 ceph-mon[34747]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 -- 172.21.15.27:0/4270232284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c109c40 msgr2=0x7fc33c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 --2- 172.21.15.27:0/4270232284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c109c40 0x7fc33c110470 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc334009ca0 tx=0x7fc33402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 -- 172.21.15.27:0/4270232284 shutdown_connections 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 --2- 172.21.15.27:0/4270232284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc33c1092f0 0x7fc33c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 --2- 172.21.15.27:0/4270232284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c109c40 0x7fc33c110470 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 --2- 172.21.15.27:0/4270232284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc33c1039b0 0x7fc33c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 -- 172.21.15.27:0/4270232284 >> 172.21.15.27:0/4270232284 conn(0x7fc33c0ff400 msgr2=0x7fc33c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 -- 172.21.15.27:0/4270232284 shutdown_connections 2024-09-21T07:10:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.061+0000 7fc340c00640 1 -- 172.21.15.27:0/4270232284 wait complete. 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.062+0000 7fc340c00640 1 Processor -- start 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc340c00640 1 -- start start 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc340c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c1039b0 0x7fc33c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc340c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc33c1092f0 0x7fc33c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc340c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc33c109c40 0x7fc33c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc340c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c113950 con 0x7fc33c1039b0 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc340c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c1137d0 con 0x7fc33c109c40 2024-09-21T07:10:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc340c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c113ad0 con 0x7fc33c1092f0 2024-09-21T07:10:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc33b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c1039b0 0x7fc33c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc33b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c1039b0 0x7fc33c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45952/0 (socket says 172.21.15.27:45952) 2024-09-21T07:10:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc33b400640 1 -- 172.21.15.27:0/999380479 learned_addr learned my addr 172.21.15.27:0/999380479 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc33aa00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc33c1092f0 0x7fc33c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.063+0000 7fc33be00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc33c109c40 0x7fc33c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.064+0000 7fc33b400640 1 -- 172.21.15.27:0/999380479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc33c1092f0 msgr2=0x7fc33c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.064+0000 7fc33b400640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc33c1092f0 0x7fc33c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.064+0000 7fc33b400640 1 -- 172.21.15.27:0/999380479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc33c109c40 msgr2=0x7fc33c1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.064+0000 7fc33b400640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc33c109c40 0x7fc33c1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.064+0000 7fc33b400640 1 -- 172.21.15.27:0/999380479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c1a2410 con 0x7fc33c1039b0 2024-09-21T07:10:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.064+0000 7fc33aa00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc33c1092f0 0x7fc33c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.064+0000 7fc33b400640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c1039b0 0x7fc33c19c4a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00e9f0 tx=0x7fc32c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.065+0000 7fc333e00640 1 -- 172.21.15.27:0/999380479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00cd80 con 0x7fc33c1039b0 2024-09-21T07:10:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.065+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33c1a2700 con 0x7fc33c1039b0 2024-09-21T07:10:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.065+0000 7fc333e00640 1 -- 172.21.15.27:0/999380479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c0108e0 con 0x7fc33c1039b0 2024-09-21T07:10:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.065+0000 7fc333e00640 1 -- 172.21.15.27:0/999380479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c018980 con 0x7fc33c1039b0 2024-09-21T07:10:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.065+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33c1a8e60 con 0x7fc33c1039b0 2024-09-21T07:10:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.066+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33c103dc0 con 0x7fc33c1039b0 2024-09-21T07:10:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.068+0000 7fc333e00640 1 -- 172.21.15.27:0/999380479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fc32c010430 con 0x7fc33c1039b0 2024-09-21T07:10:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.070+0000 7fc333e00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3200b65e0 0x7fc3200b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.070+0000 7fc333e00640 1 -- 172.21.15.27:0/999380479 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7fc32c0e7e10 con 0x7fc33c1039b0 2024-09-21T07:10:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.070+0000 7fc33aa00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3200b65e0 0x7fc3200b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.071+0000 7fc33aa00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3200b65e0 0x7fc3200b8af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc33c105a60 tx=0x7fc328005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.072+0000 7fc333e00640 1 -- 172.21.15.27:0/999380479 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc32c0b2710 con 0x7fc33c1039b0 2024-09-21T07:10:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.287+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc33c19d5d0 con 0x7fc33c1039b0 2024-09-21T07:10:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.287+0000 7fc333e00640 1 -- 172.21.15.27:0/999380479 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v82) ==== 81+0+884 (secure 0 0 0) 0x7fc32c0b7650 con 0x7fc33c1039b0 2024-09-21T07:10:31.289 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/785463945 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2240730116 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2151540061 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1830236521 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2513510737 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/727966245 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2811364463 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2513510737 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2151540061 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3503043760 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2989055970 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1823929888 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/727966245 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4193484002 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:31.290 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542737158 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:31.291 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3200b65e0 msgr2=0x7fc3200b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3200b65e0 0x7fc3200b8af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc33c105a60 tx=0x7fc328005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c1039b0 msgr2=0x7fc33c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c1039b0 0x7fc33c19c4a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00e9f0 tx=0x7fc32c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 shutdown_connections 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc33c1092f0 0x7fc33c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3200b65e0 0x7fc3200b8af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc33c109c40 0x7fc33c1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 --2- 172.21.15.27:0/999380479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc33c1039b0 0x7fc33c19c4a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 >> 172.21.15.27:0/999380479 conn(0x7fc33c0ff400 msgr2=0x7fc33c1078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 shutdown_connections 2024-09-21T07:10:31.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:31.290+0000 7fc340c00640 1 -- 172.21.15.27:0/999380479 wait complete. 2024-09-21T07:10:31.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:31 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/999380479' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:31.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:31 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/999380479' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:31.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:31 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/999380479' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:32.072 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:10:32.072 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:10:32.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T07:10:32.537 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:32.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:32 smithi119 ceph-mon[34519]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:32.800 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:32 smithi027 ceph-mon[30779]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:32.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:32 smithi153 ceph-mon[34747]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:33.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 -- 172.21.15.27:0/3269531006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf30105300 msgr2=0x7faf3010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 --2- 172.21.15.27:0/3269531006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf30105300 0x7faf3010d850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf18009860 tx=0x7faf1802a340 comp rx=0 tx=0).stop 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 -- 172.21.15.27:0/3269531006 shutdown_connections 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 --2- 172.21.15.27:0/3269531006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf3010de10 0x7faf30110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 --2- 172.21.15.27:0/3269531006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf30105300 0x7faf3010d850 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 --2- 172.21.15.27:0/3269531006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 0x7faf30104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 -- 172.21.15.27:0/3269531006 >> 172.21.15.27:0/3269531006 conn(0x7faf30079760 msgr2=0x7faf30079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 -- 172.21.15.27:0/3269531006 shutdown_connections 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.152+0000 7faf35800640 1 -- 172.21.15.27:0/3269531006 wait complete. 2024-09-21T07:10:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.153+0000 7faf35800640 1 Processor -- start 2024-09-21T07:10:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf35800640 1 -- start start 2024-09-21T07:10:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf35800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 0x7faf3019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf35800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf30105300 0x7faf3019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf35800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3010de10 0x7faf3019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf35800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf30113a70 con 0x7faf30105300 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf35800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faf301138f0 con 0x7faf3010de10 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf35800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faf30113bf0 con 0x7faf301049b0 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf27e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf30105300 0x7faf3019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf2ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 0x7faf3019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3010de10 0x7faf3019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf2ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 0x7faf3019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54670/0 (socket says 172.21.15.27:54670) 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf2ea00640 1 -- 172.21.15.27:0/4136147700 learned_addr learned my addr 172.21.15.27:0/4136147700 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.154+0000 7faf2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3010de10 0x7faf3019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:42416/0 (socket says 172.21.15.27:42416) 2024-09-21T07:10:33.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf2ea00640 1 -- 172.21.15.27:0/4136147700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3010de10 msgr2=0x7faf3019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf2ea00640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3010de10 0x7faf3019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf2ea00640 1 -- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf30105300 msgr2=0x7faf3019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf2ea00640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf30105300 0x7faf3019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf2ea00640 1 -- 172.21.15.27:0/4136147700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf301a0490 con 0x7faf301049b0 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf27e00640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf30105300 0x7faf3019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.155+0000 7faf2ea00640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 0x7faf3019a520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf2000e9f0 tx=0x7faf2000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.156+0000 7faf27400640 1 -- 172.21.15.27:0/4136147700 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2000cd80 con 0x7faf301049b0 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.156+0000 7faf27400640 1 -- 172.21.15.27:0/4136147700 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf200054d0 con 0x7faf301049b0 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.156+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf301a0780 con 0x7faf301049b0 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.156+0000 7faf27400640 1 -- 172.21.15.27:0/4136147700 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20010690 con 0x7faf301049b0 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.156+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf301a6ee0 con 0x7faf301049b0 2024-09-21T07:10:33.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.157+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3006c2c0 con 0x7faf301049b0 2024-09-21T07:10:33.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.160+0000 7faf27400640 1 -- 172.21.15.27:0/4136147700 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7faf20004e30 con 0x7faf301049b0 2024-09-21T07:10:33.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.161+0000 7faf27400640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf140b6560 0x7faf140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.162+0000 7faf27400640 1 -- 172.21.15.27:0/4136147700 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7faf200e7c90 con 0x7faf301049b0 2024-09-21T07:10:33.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.162+0000 7faf27e00640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf140b6560 0x7faf140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.162+0000 7faf27e00640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf140b6560 0x7faf140b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf18005940 tx=0x7faf180058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:33.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.163+0000 7faf27400640 1 -- 172.21.15.27:0/4136147700 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf200ae1b0 con 0x7faf301049b0 2024-09-21T07:10:33.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.375+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf30104dc0 con 0x7faf301049b0 2024-09-21T07:10:33.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.376+0000 7faf27400640 1 -- 172.21.15.27:0/4136147700 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v82) ==== 81+0+884 (secure 0 0 0) 0x7faf30104dc0 con 0x7faf301049b0 2024-09-21T07:10:33.378 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/785463945 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:33.378 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2240730116 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2151540061 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1830236521 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2513510737 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/727966245 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2811364463 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2513510737 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2151540061 2024-09-22T07:01:14.297512+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3503043760 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2989055970 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1823929888 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/727966245 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4193484002 2024-09-22T07:01:45.325213+0000 2024-09-21T07:10:33.379 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542737158 2024-09-22T07:02:53.267293+0000 2024-09-21T07:10:33.380 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.378+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf140b6560 msgr2=0x7faf140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.378+0000 7faf35800640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf140b6560 0x7faf140b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf18005940 tx=0x7faf180058d0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.378+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 msgr2=0x7faf3019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.378+0000 7faf35800640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 0x7faf3019a520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf2000e9f0 tx=0x7faf2000edf0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 shutdown_connections 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf140b6560 0x7faf140b8a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3010de10 0x7faf3019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf30105300 0x7faf3019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 --2- 172.21.15.27:0/4136147700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf301049b0 0x7faf3019a520 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 >> 172.21.15.27:0/4136147700 conn(0x7faf30079760 msgr2=0x7faf301008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 shutdown_connections 2024-09-21T07:10:33.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:33.379+0000 7faf35800640 1 -- 172.21.15.27:0/4136147700 wait complete. 2024-09-21T07:10:33.726 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:33 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4136147700' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:33.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:33 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4136147700' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:33.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:33 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4136147700' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:34.100 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2024-09-21T07:10:34.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:34.100 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:10:34.156 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[76725]: fuse finished with error 0 and tester_r 0 2024-09-21T07:10:34.160 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:10:34.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:34 smithi119 ceph-mon[34519]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:34.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:34 smithi027 ceph-mon[30779]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:34 smithi153 ceph-mon[34747]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T07:10:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:36 smithi119 ceph-mon[34519]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T07:10:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:36 smithi027 ceph-mon[30779]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T07:10:36.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:36 smithi153 ceph-mon[34747]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T07:10:38.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:38 smithi119 ceph-mon[34519]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T07:10:38.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:38 smithi027 ceph-mon[30779]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T07:10:38.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:38 smithi153 ceph-mon[34747]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T07:10:40.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:10:40.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:40.262 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T07:10:40.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:10:40.713 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:40.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:40 smithi119 ceph-mon[34519]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T07:10:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:40 smithi027 ceph-mon[30779]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T07:10:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:40 smithi153 ceph-mon[34747]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T07:10:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 -- 172.21.15.27:0/677526969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4077380 msgr2=0x7fc8d40757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 --2- 172.21.15.27:0/677526969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4077380 0x7fc8d40757b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009900 tx=0x7fc8c402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 -- 172.21.15.27:0/677526969 shutdown_connections 2024-09-21T07:10:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 --2- 172.21.15.27:0/677526969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8d4114430 0x7fc8d4116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 --2- 172.21.15.27:0/677526969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 0x7fc8d40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 --2- 172.21.15.27:0/677526969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4077380 0x7fc8d40757b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 -- 172.21.15.27:0/677526969 >> 172.21.15.27:0/677526969 conn(0x7fc8d407b850 msgr2=0x7fc8d407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.378+0000 7fc8db000640 1 -- 172.21.15.27:0/677526969 shutdown_connections 2024-09-21T07:10:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.379+0000 7fc8db000640 1 -- 172.21.15.27:0/677526969 wait complete. 2024-09-21T07:10:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 Processor -- start 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 -- start start 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 0x7fc8d41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8d4077380 0x7fc8d41a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4114430 0x7fc8d41a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d411a080 con 0x7fc8d4114430 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d4119f00 con 0x7fc8d4075dd0 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.380+0000 7fc8db000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d411a200 con 0x7fc8d4077380 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 0x7fc8d41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4114430 0x7fc8d41a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 0x7fc8d41a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:42722/0 (socket says 172.21.15.27:42722) 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8d4077380 0x7fc8d41a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 -- 172.21.15.27:0/1754493626 learned_addr learned my addr 172.21.15.27:0/1754493626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 -- 172.21.15.27:0/1754493626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8d4077380 msgr2=0x7fc8d41a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8d4077380 0x7fc8d41a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 -- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4114430 msgr2=0x7fc8d41a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4114430 0x7fc8d41a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 -- 172.21.15.27:0/1754493626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d41a69a0 con 0x7fc8d4075dd0 2024-09-21T07:10:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.381+0000 7fc8d3e00640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 0x7fc8d41a0a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005940 tx=0x7fc8c40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.382+0000 7fc8d1600640 1 -- 172.21.15.27:0/1754493626 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c403f030 con 0x7fc8d4075dd0 2024-09-21T07:10:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.382+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d41a6c30 con 0x7fc8d4075dd0 2024-09-21T07:10:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.382+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d41ad3f0 con 0x7fc8d4075dd0 2024-09-21T07:10:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.382+0000 7fc8d1600640 1 -- 172.21.15.27:0/1754493626 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c40044c0 con 0x7fc8d4075dd0 2024-09-21T07:10:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.382+0000 7fc8d1600640 1 -- 172.21.15.27:0/1754493626 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4049060 con 0x7fc8d4075dd0 2024-09-21T07:10:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.383+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b00052c0 con 0x7fc8d4075dd0 2024-09-21T07:10:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.386+0000 7fc8d1600640 1 -- 172.21.15.27:0/1754493626 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fc8c4042030 con 0x7fc8d4075dd0 2024-09-21T07:10:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.387+0000 7fc8d1600640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b80b6560 0x7fc8b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.388+0000 7fc8d1600640 1 -- 172.21.15.27:0/1754493626 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7fc8c4107640 con 0x7fc8d4075dd0 2024-09-21T07:10:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.388+0000 7fc8cbe00640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b80b6560 0x7fc8b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.388+0000 7fc8cbe00640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b80b6560 0x7fc8b80b8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4115b00 tx=0x7fc8bc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.389+0000 7fc8d1600640 1 -- 172.21.15.27:0/1754493626 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8c40d1f40 con 0x7fc8d4075dd0 2024-09-21T07:10:41.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.590+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8b0005880 con 0x7fc8d4075dd0 2024-09-21T07:10:41.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.591+0000 7fc8d1600640 1 -- 172.21.15.27:0/1754493626 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24538 (secure 0 0 0) 0x7fc8c40d6e80 con 0x7fc8d4075dd0 2024-09-21T07:10:41.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:41.594 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:09.632242+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:10:41.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b80b6560 msgr2=0x7fc8b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:41.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b80b6560 0x7fc8b80b8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4115b00 tx=0x7fc8bc005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:41.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 msgr2=0x7fc8d41a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:41.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 0x7fc8d41a0a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005940 tx=0x7fc8c40040c0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 shutdown_connections 2024-09-21T07:10:41.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8d4077380 0x7fc8d41a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b80b6560 0x7fc8b80b8a70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d4114430 0x7fc8d41a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 --2- 172.21.15.27:0/1754493626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8d4075dd0 0x7fc8d41a0a30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 >> 172.21.15.27:0/1754493626 conn(0x7fc8d407b850 msgr2=0x7fc8d4113060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 shutdown_connections 2024-09-21T07:10:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:41.594+0000 7fc8db000640 1 -- 172.21.15.27:0/1754493626 wait complete. 2024-09-21T07:10:42.313 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:10:42.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:42 smithi027 ceph-mon[30779]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:42.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:42 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1754493626' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:42.703 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:42 smithi119 ceph-mon[34519]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:42 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1754493626' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:42.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:42 smithi153 ceph-mon[34747]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:42.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:42 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1754493626' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.380+0000 7ff935c00640 1 -- 172.21.15.27:0/2953939253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93010b6c0 msgr2=0x7ff93010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.380+0000 7ff935c00640 1 --2- 172.21.15.27:0/2953939253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93010b6c0 0x7ff93010bad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009900 tx=0x7ff91c02a410 comp rx=0 tx=0).stop 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.380+0000 7ff935c00640 1 -- 172.21.15.27:0/2953939253 shutdown_connections 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.380+0000 7ff935c00640 1 --2- 172.21.15.27:0/2953939253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff93010c010 0x7ff930112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.380+0000 7ff935c00640 1 --2- 172.21.15.27:0/2953939253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff93010b6c0 0x7ff93010bad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.380+0000 7ff935c00640 1 --2- 172.21.15.27:0/2953939253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff930105d80 0x7ff930106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.380+0000 7ff935c00640 1 -- 172.21.15.27:0/2953939253 >> 172.21.15.27:0/2953939253 conn(0x7ff930101810 msgr2=0x7ff930103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.381+0000 7ff935c00640 1 -- 172.21.15.27:0/2953939253 shutdown_connections 2024-09-21T07:10:43.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.381+0000 7ff935c00640 1 -- 172.21.15.27:0/2953939253 wait complete. 2024-09-21T07:10:43.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.382+0000 7ff935c00640 1 Processor -- start 2024-09-21T07:10:43.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.382+0000 7ff935c00640 1 -- start start 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff935c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff930105d80 0x7ff93007b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff935c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff93010b6c0 0x7ff930077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff935c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff93010c010 0x7ff930078040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff935c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff930115d20 con 0x7ff930105d80 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff935c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff930115ba0 con 0x7ff93010b6c0 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff935c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff930115ea0 con 0x7ff93010c010 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff92ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff930105d80 0x7ff93007b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff92ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff930105d80 0x7ff93007b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39080/0 (socket says 172.21.15.27:39080) 2024-09-21T07:10:43.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff92ea00640 1 -- 172.21.15.27:0/1956293085 learned_addr learned my addr 172.21.15.27:0/1956293085 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:43.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff92e000640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff93010b6c0 0x7ff930077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.383+0000 7ff92f400640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff93010c010 0x7ff930078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff92ea00640 1 -- 172.21.15.27:0/1956293085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff93010c010 msgr2=0x7ff930078040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff92ea00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff93010c010 0x7ff930078040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff92ea00640 1 -- 172.21.15.27:0/1956293085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff93010b6c0 msgr2=0x7ff930077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff92ea00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff93010b6c0 0x7ff930077920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff92ea00640 1 -- 172.21.15.27:0/1956293085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9301acc40 con 0x7ff930105d80 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff92ea00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff930105d80 0x7ff93007b640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff92000e9f0 tx=0x7ff92000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff927400640 1 -- 172.21.15.27:0/1956293085 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92000cd80 con 0x7ff930105d80 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff927400640 1 -- 172.21.15.27:0/1956293085 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9200108e0 con 0x7ff930105d80 2024-09-21T07:10:43.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9301ace70 con 0x7ff930105d80 2024-09-21T07:10:43.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff927400640 1 -- 172.21.15.27:0/1956293085 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920018980 con 0x7ff930105d80 2024-09-21T07:10:43.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.384+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9301ad270 con 0x7ff930105d80 2024-09-21T07:10:43.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.386+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff930106190 con 0x7ff930105d80 2024-09-21T07:10:43.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.388+0000 7ff927400640 1 -- 172.21.15.27:0/1956293085 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7ff920010430 con 0x7ff930105d80 2024-09-21T07:10:43.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.390+0000 7ff927400640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff9140b6560 0x7ff9140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.390+0000 7ff927400640 1 -- 172.21.15.27:0/1956293085 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7ff9200e7e40 con 0x7ff930105d80 2024-09-21T07:10:43.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.390+0000 7ff92e000640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff9140b6560 0x7ff9140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.391+0000 7ff92e000640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff9140b6560 0x7ff9140b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff91c02aa00 tx=0x7ff91c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:43.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.392+0000 7ff927400640 1 -- 172.21.15.27:0/1956293085 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9200b2740 con 0x7ff930105d80 2024-09-21T07:10:43.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.596+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff93002e6e0 con 0x7ff930105d80 2024-09-21T07:10:43.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.598+0000 7ff927400640 1 -- 172.21.15.27:0/1956293085 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24538 (secure 0 0 0) 0x7ff9200b7680 con 0x7ff930105d80 2024-09-21T07:10:43.600 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:10:43.600 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:09.632242+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:10:43.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff9140b6560 msgr2=0x7ff9140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff9140b6560 0x7ff9140b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff91c02aa00 tx=0x7ff91c005a20 comp rx=0 tx=0).stop 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff930105d80 msgr2=0x7ff93007b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff930105d80 0x7ff93007b640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff92000e9f0 tx=0x7ff92000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 shutdown_connections 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff93010c010 0x7ff930078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff9140b6560 0x7ff9140b8a70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff93010b6c0 0x7ff930077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 --2- 172.21.15.27:0/1956293085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff930105d80 0x7ff93007b640 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.600+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 >> 172.21.15.27:0/1956293085 conn(0x7ff930101810 msgr2=0x7ff930103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.601+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 shutdown_connections 2024-09-21T07:10:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:43.601+0000 7ff935c00640 1 -- 172.21.15.27:0/1956293085 wait complete. 2024-09-21T07:10:44.260 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:10:44.646 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:44.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:44 smithi119 ceph-mon[34519]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:44.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:44 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1956293085' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:44.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:44 smithi027 ceph-mon[30779]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:44.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:44 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1956293085' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:44.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:44 smithi153 ceph-mon[34747]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:44.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:44 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1956293085' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:45.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 -- 172.21.15.27:0/386231798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 msgr2=0x7ff560112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 --2- 172.21.15.27:0/386231798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 0x7ff560112670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff558009ca0 tx=0x7ff55802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 -- 172.21.15.27:0/386231798 shutdown_connections 2024-09-21T07:10:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 --2- 172.21.15.27:0/386231798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff56010b4f0 0x7ff56010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 --2- 172.21.15.27:0/386231798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 0x7ff560112670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 --2- 172.21.15.27:0/386231798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff560105bb0 0x7ff560105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 -- 172.21.15.27:0/386231798 >> 172.21.15.27:0/386231798 conn(0x7ff560101680 msgr2=0x7ff560103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 -- 172.21.15.27:0/386231798 shutdown_connections 2024-09-21T07:10:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.225+0000 7ff55fe00640 1 -- 172.21.15.27:0/386231798 wait complete. 2024-09-21T07:10:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.226+0000 7ff55fe00640 1 Processor -- start 2024-09-21T07:10:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.226+0000 7ff55fe00640 1 -- start start 2024-09-21T07:10:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff560105bb0 0x7ff56019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff56010b4f0 0x7ff56019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 0x7ff5601a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff560115b50 con 0x7ff56010be40 2024-09-21T07:10:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601159d0 con 0x7ff560105bb0 2024-09-21T07:10:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff560115cd0 con 0x7ff56010b4f0 2024-09-21T07:10:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff557e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff56010b4f0 0x7ff56019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 0x7ff5601a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff55ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff560105bb0 0x7ff56019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff557e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff56010b4f0 0x7ff56019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45374/0 (socket says 172.21.15.27:45374) 2024-09-21T07:10:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.227+0000 7ff557e00640 1 -- 172.21.15.27:0/1414217510 learned_addr learned my addr 172.21.15.27:0/1414217510 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.228+0000 7ff55f400640 1 -- 172.21.15.27:0/1414217510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff56010b4f0 msgr2=0x7ff56019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.228+0000 7ff55f400640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff56010b4f0 0x7ff56019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.228+0000 7ff55f400640 1 -- 172.21.15.27:0/1414217510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff560105bb0 msgr2=0x7ff56019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.228+0000 7ff55f400640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff560105bb0 0x7ff56019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.228+0000 7ff55f400640 1 -- 172.21.15.27:0/1414217510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601a2430 con 0x7ff56010be40 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.228+0000 7ff55ea00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff560105bb0 0x7ff56019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.228+0000 7ff55f400640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 0x7ff5601a1d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff558005e30 tx=0x7ff5580317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.229+0000 7ff557400640 1 -- 172.21.15.27:0/1414217510 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558003b70 con 0x7ff56010be40 2024-09-21T07:10:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.229+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5601a26c0 con 0x7ff56010be40 2024-09-21T07:10:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.229+0000 7ff557400640 1 -- 172.21.15.27:0/1414217510 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff558003d10 con 0x7ff56010be40 2024-09-21T07:10:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.229+0000 7ff557400640 1 -- 172.21.15.27:0/1414217510 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55802f970 con 0x7ff56010be40 2024-09-21T07:10:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.229+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601a8e80 con 0x7ff56010be40 2024-09-21T07:10:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.230+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff560106e30 con 0x7ff56010be40 2024-09-21T07:10:45.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.232+0000 7ff557400640 1 -- 172.21.15.27:0/1414217510 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7ff558004b40 con 0x7ff56010be40 2024-09-21T07:10:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.234+0000 7ff557400640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5440b6560 0x7ff5440b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.234+0000 7ff557400640 1 -- 172.21.15.27:0/1414217510 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7ff558108890 con 0x7ff56010be40 2024-09-21T07:10:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.234+0000 7ff55ea00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5440b6560 0x7ff5440b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.236+0000 7ff55ea00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5440b6560 0x7ff5440b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5500095f0 tx=0x7ff550005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:45.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.236+0000 7ff557400640 1 -- 172.21.15.27:0/1414217510 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5580cee40 con 0x7ff56010be40 2024-09-21T07:10:45.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.479+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff560105fc0 con 0x7ff56010be40 2024-09-21T07:10:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.510+0000 7ff557400640 1 -- 172.21.15.27:0/1414217510 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ff560105fc0 con 0x7ff56010be40 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5440b6560 msgr2=0x7ff5440b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5440b6560 0x7ff5440b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5500095f0 tx=0x7ff550005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 msgr2=0x7ff5601a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 0x7ff5601a1d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff558005e30 tx=0x7ff5580317c0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 shutdown_connections 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff56010b4f0 0x7ff56019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5440b6560 0x7ff5440b8a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56010be40 0x7ff5601a1d00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 --2- 172.21.15.27:0/1414217510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff560105bb0 0x7ff56019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 >> 172.21.15.27:0/1414217510 conn(0x7ff560101680 msgr2=0x7ff56010ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:45.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 shutdown_connections 2024-09-21T07:10:45.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:45.513+0000 7ff55fe00640 1 -- 172.21.15.27:0/1414217510 wait complete. 2024-09-21T07:10:45.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:45 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1414217510' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T07:10:45.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:45 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1414217510' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T07:10:45.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:45 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1414217510' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T07:10:46.174 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-21T07:10:46.174 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 3 2024-09-21T07:10:46.559 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:46.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:46 smithi119 ceph-mon[34519]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:46.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:46 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1414217510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T07:10:46.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:46 smithi119 ceph-mon[34519]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:46.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:46 smithi027 ceph-mon[30779]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:46.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:46 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1414217510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T07:10:46.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:46 smithi027 ceph-mon[30779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:46.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:46 smithi153 ceph-mon[34747]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T07:10:46.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:46 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1414217510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T07:10:46.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:46 smithi153 ceph-mon[34747]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:47.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 -- 172.21.15.27:0/553301818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1039b0 msgr2=0x7f026c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:47.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 --2- 172.21.15.27:0/553301818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1039b0 0x7f026c103dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f025c009900 tx=0x7f025c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 -- 172.21.15.27:0/553301818 shutdown_connections 2024-09-21T07:10:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 --2- 172.21.15.27:0/553301818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f026c1092f0 0x7f026c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 --2- 172.21.15.27:0/553301818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026c109c40 0x7f026c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 --2- 172.21.15.27:0/553301818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1039b0 0x7f026c103dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 -- 172.21.15.27:0/553301818 >> 172.21.15.27:0/553301818 conn(0x7f026c0ff400 msgr2=0x7f026c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 -- 172.21.15.27:0/553301818 shutdown_connections 2024-09-21T07:10:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.204+0000 7f026b200640 1 -- 172.21.15.27:0/553301818 wait complete. 2024-09-21T07:10:47.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.205+0000 7f026b200640 1 Processor -- start 2024-09-21T07:10:47.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.205+0000 7f026b200640 1 -- start start 2024-09-21T07:10:47.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f026b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f026c1039b0 0x7f026c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:47.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f026b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1092f0 0x7f026c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f026b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026c109c40 0x7f026c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f026b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f026c113900 con 0x7f026c1092f0 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f026b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f026c113780 con 0x7f026c109c40 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f026b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f026c113a80 con 0x7f026c1039b0 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f0269400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1092f0 0x7f026c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f0269400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1092f0 0x7f026c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39102/0 (socket says 172.21.15.27:39102) 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f0269400640 1 -- 172.21.15.27:0/3517832580 learned_addr learned my addr 172.21.15.27:0/3517832580 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f0269e00640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f026c1039b0 0x7f026c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:47.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.206+0000 7f026a800640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026c109c40 0x7f026c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:47.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0269400640 1 -- 172.21.15.27:0/3517832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f026c1039b0 msgr2=0x7f026c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:47.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0269400640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f026c1039b0 0x7f026c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0269400640 1 -- 172.21.15.27:0/3517832580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026c109c40 msgr2=0x7f026c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:47.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0269400640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026c109c40 0x7f026c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0269400640 1 -- 172.21.15.27:0/3517832580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026c1a85b0 con 0x7f026c1092f0 2024-09-21T07:10:47.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0269400640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1092f0 0x7f026c075c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f025800e9c0 tx=0x7f025800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0262a00640 1 -- 172.21.15.27:0/3517832580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025800ccc0 con 0x7f026c1092f0 2024-09-21T07:10:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026c1a88a0 con 0x7f026c1092f0 2024-09-21T07:10:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.207+0000 7f0262a00640 1 -- 172.21.15.27:0/3517832580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02580054d0 con 0x7f026c1092f0 2024-09-21T07:10:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.208+0000 7f0262a00640 1 -- 172.21.15.27:0/3517832580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258018430 con 0x7f026c1092f0 2024-09-21T07:10:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.208+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026c1a8d90 con 0x7f026c1092f0 2024-09-21T07:10:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.209+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026c103dc0 con 0x7f026c1092f0 2024-09-21T07:10:47.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.211+0000 7f0262a00640 1 -- 172.21.15.27:0/3517832580 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0258004fa0 con 0x7f026c1092f0 2024-09-21T07:10:47.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.214+0000 7f0262a00640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02500b6560 0x7f02500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.214+0000 7f0262a00640 1 -- 172.21.15.27:0/3517832580 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f02580e8150 con 0x7f026c1092f0 2024-09-21T07:10:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.214+0000 7f0269e00640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02500b6560 0x7f02500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.215+0000 7f0269e00640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02500b6560 0x7f02500b8a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:47.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.216+0000 7f0262a00640 1 -- 172.21.15.27:0/3517832580 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02580ae670 con 0x7f026c1092f0 2024-09-21T07:10:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.453+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f026c02e640 con 0x7f026c1092f0 2024-09-21T07:10:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.518+0000 7f0262a00640 1 -- 172.21.15.27:0/3517832580 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f02580b2a50 con 0x7f026c1092f0 2024-09-21T07:10:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02500b6560 msgr2=0x7f02500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02500b6560 0x7f02500b8a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c031400 comp rx=0 tx=0).stop 2024-09-21T07:10:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1092f0 msgr2=0x7f026c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1092f0 0x7f026c075c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f025800e9c0 tx=0x7f025800ee80 comp rx=0 tx=0).stop 2024-09-21T07:10:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 shutdown_connections 2024-09-21T07:10:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02500b6560 0x7f02500b8a70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f026c109c40 0x7f026c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f026c1092f0 0x7f026c075c50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 --2- 172.21.15.27:0/3517832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f026c1039b0 0x7f026c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 >> 172.21.15.27:0/3517832580 conn(0x7f026c0ff400 msgr2=0x7f026c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 shutdown_connections 2024-09-21T07:10:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:47.521+0000 7f026b200640 1 -- 172.21.15.27:0/3517832580 wait complete. 2024-09-21T07:10:47.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:47 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3517832580' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T07:10:47.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:47 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3517832580' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T07:10:47.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:47 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3517832580' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T07:10:48.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T07:10:48.653 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-mon[34519]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T07:10:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-mon[34519]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T07:10:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3517832580' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T07:10:48.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-mon[34519]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T07:10:48.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-mon[34519]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:48.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-mon[34519]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-21T07:10:48.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-mon[34519]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-21T07:10:48.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:48 smithi027 ceph-mon[30779]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T07:10:48.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:48 smithi027 ceph-mon[30779]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T07:10:48.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:48 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3517832580' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T07:10:48.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:48 smithi027 ceph-mon[30779]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T07:10:48.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:48 smithi027 ceph-mon[30779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:48.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:48 smithi027 ceph-mon[30779]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-21T07:10:48.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:48 smithi027 ceph-mon[30779]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-21T07:10:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:48 smithi153 ceph-mon[34747]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T07:10:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:48 smithi153 ceph-mon[34747]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T07:10:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:48 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3517832580' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T07:10:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:48 smithi153 ceph-mon[34747]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T07:10:48.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:48 smithi153 ceph-mon[34747]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T07:10:48.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:48 smithi153 ceph-mon[34747]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-21T07:10:48.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:48 smithi153 ceph-mon[34747]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-21T07:10:49.284 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:10:48 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: 2024-09-21T07:10:48.817+0000 7fa0fea5c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:10:49.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.291+0000 7f8001600640 1 -- 172.21.15.27:0/662722155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 msgr2=0x7f7ffc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.291+0000 7f8001600640 1 --2- 172.21.15.27:0/662722155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 0x7f7ffc110620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009ca0 tx=0x7f7ff402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.291+0000 7f8001600640 1 -- 172.21.15.27:0/662722155 shutdown_connections 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.291+0000 7f8001600640 1 --2- 172.21.15.27:0/662722155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ffc1094a0 0x7f7ffc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.291+0000 7f8001600640 1 --2- 172.21.15.27:0/662722155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 0x7f7ffc110620 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.291+0000 7f8001600640 1 --2- 172.21.15.27:0/662722155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 0x7f7ffc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.291+0000 7f8001600640 1 -- 172.21.15.27:0/662722155 >> 172.21.15.27:0/662722155 conn(0x7f7ffc07b850 msgr2=0x7f7ffc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.292+0000 7f8001600640 1 -- 172.21.15.27:0/662722155 shutdown_connections 2024-09-21T07:10:49.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.292+0000 7f8001600640 1 -- 172.21.15.27:0/662722155 wait complete. 2024-09-21T07:10:49.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 Processor -- start 2024-09-21T07:10:49.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 -- start start 2024-09-21T07:10:49.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 0x7f7ffc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ffc1094a0 0x7f7ffc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 0x7f7ffc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc113b00 con 0x7f7ffc109df0 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc113980 con 0x7f7ffc103b60 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.293+0000 7f8001600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc113c80 con 0x7f7ffc1094a0 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 0x7f7ffc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 0x7f7ffc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 0x7f7ffc19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55634/0 (socket says 172.21.15.27:55634) 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 -- 172.21.15.27:0/1924755824 learned_addr learned my addr 172.21.15.27:0/1924755824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ffc1094a0 0x7f7ffc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 -- 172.21.15.27:0/1924755824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ffc1094a0 msgr2=0x7f7ffc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ffc1094a0 0x7f7ffc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 -- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 msgr2=0x7f7ffc19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 0x7f7ffc19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 -- 172.21.15.27:0/1924755824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc1a0380 con 0x7f7ffc103b60 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.294+0000 7f7ffaa00640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 0x7f7ffc19a410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00ea60 tx=0x7f7fec00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.295+0000 7f7ff3400640 1 -- 172.21.15.27:0/1924755824 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00cde0 con 0x7f7ffc103b60 2024-09-21T07:10:49.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.295+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc1a0670 con 0x7f7ffc103b60 2024-09-21T07:10:49.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.295+0000 7f7ff3400640 1 -- 172.21.15.27:0/1924755824 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fec0035f0 con 0x7f7ffc103b60 2024-09-21T07:10:49.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.295+0000 7f7ff3400640 1 -- 172.21.15.27:0/1924755824 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec010690 con 0x7f7ffc103b60 2024-09-21T07:10:49.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.295+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1a6dd0 con 0x7f7ffc103b60 2024-09-21T07:10:49.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.296+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc103f70 con 0x7f7ffc103b60 2024-09-21T07:10:49.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.298+0000 7f7ff3400640 1 -- 172.21.15.27:0/1924755824 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f7fec0037a0 con 0x7f7ffc103b60 2024-09-21T07:10:49.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.300+0000 7f7ff3400640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7fe00b6560 0x7f7fe00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:49.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.300+0000 7f7ff3400640 1 -- 172.21.15.27:0/1924755824 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f7fec0e7de0 con 0x7f7ffc103b60 2024-09-21T07:10:49.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.300+0000 7f7ff3e00640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7fe00b6560 0x7f7fe00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:49.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.301+0000 7f7ff3e00640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7fe00b6560 0x7f7fe00b8a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc109110 tx=0x7f7fe4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:49.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.302+0000 7f7ff3400640 1 -- 172.21.15.27:0/1924755824 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7fec0ae300 con 0x7f7ffc103b60 2024-09-21T07:10:49.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:49.549+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f7ffc19b540 con 0x7f7ffc103b60 2024-09-21T07:10:49.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] up:active 2024-09-21T07:10:49.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T07:10:49.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T07:10:49.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: Cluster is now healthy 2024-09-21T07:10:49.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-21T07:10:49.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-21T07:10:49.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T07:10:49.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:49 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:10:49.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] up:active 2024-09-21T07:10:49.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T07:10:49.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T07:10:49.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: Cluster is now healthy 2024-09-21T07:10:49.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-21T07:10:49.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-21T07:10:49.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T07:10:49.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:49 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:10:49.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] up:active 2024-09-21T07:10:49.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T07:10:49.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T07:10:49.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: Cluster is now healthy 2024-09-21T07:10:49.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-21T07:10:49.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-21T07:10:49.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T07:10:49.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:49 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:10:50.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.547+0000 7f7ff3400640 1 -- 172.21.15.27:0/1924755824 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f7fec0b26e0 con 0x7f7ffc103b60 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7fe00b6560 msgr2=0x7f7fe00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7fe00b6560 0x7f7fe00b8a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc109110 tx=0x7f7fe4005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 msgr2=0x7f7ffc19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 0x7f7ffc19a410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00ea60 tx=0x7f7fec00ef20 comp rx=0 tx=0).stop 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 shutdown_connections 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ffc1094a0 0x7f7ffc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7fe00b6560 0x7f7fe00b8a70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ffc109df0 0x7f7ffc19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.550+0000 7f8001600640 1 --2- 172.21.15.27:0/1924755824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ffc103b60 0x7f7ffc19a410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.551+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 >> 172.21.15.27:0/1924755824 conn(0x7f7ffc07b850 msgr2=0x7f7ffc107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:50.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.551+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 shutdown_connections 2024-09-21T07:10:50.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:50.551+0000 7f8001600640 1 -- 172.21.15.27:0/1924755824 wait complete. 2024-09-21T07:10:50.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:50 smithi027 ceph-mon[30779]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-09-21T07:10:50.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:50 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] up:active 2024-09-21T07:10:50.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:50 smithi027 ceph-mon[30779]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T07:10:50.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:50 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1924755824' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T07:10:50.806 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:50 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T07:10:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:50 smithi153 ceph-mon[34747]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-09-21T07:10:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:50 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] up:active 2024-09-21T07:10:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:50 smithi153 ceph-mon[34747]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T07:10:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:50 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1924755824' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T07:10:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:50 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T07:10:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:50 smithi119 ceph-mon[34519]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-09-21T07:10:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:50 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] up:active 2024-09-21T07:10:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:50 smithi119 ceph-mon[34519]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T07:10:51.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:50 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1924755824' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T07:10:51.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:50 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T07:10:51.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T07:10:51.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:51 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T07:10:51.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:51 smithi027 ceph-mon[30779]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T07:10:51.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:51 smithi027 ceph-mon[30779]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:10:51.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:51.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:51.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:51.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:51 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:51.908 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:51.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:51 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T07:10:51.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:51 smithi153 ceph-mon[34747]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T07:10:51.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:51 smithi153 ceph-mon[34747]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:10:51.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:51.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:51.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:51.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:51 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:52.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:51 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T07:10:52.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:51 smithi119 ceph-mon[34519]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T07:10:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:51 smithi119 ceph-mon[34519]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:10:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:51 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:52.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 -- 172.21.15.27:0/1331636489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8109df0 msgr2=0x7f93f8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:52.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 --2- 172.21.15.27:0/1331636489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8109df0 0x7f93f8110620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009ca0 tx=0x7f93f002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 -- 172.21.15.27:0/1331636489 shutdown_connections 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 --2- 172.21.15.27:0/1331636489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8109df0 0x7f93f8110620 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 --2- 172.21.15.27:0/1331636489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f81094a0 0x7f93f81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 --2- 172.21.15.27:0/1331636489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8103b60 0x7f93f8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 -- 172.21.15.27:0/1331636489 >> 172.21.15.27:0/1331636489 conn(0x7f93f807b850 msgr2=0x7f93f807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 -- 172.21.15.27:0/1331636489 shutdown_connections 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.531+0000 7f93fdc00640 1 -- 172.21.15.27:0/1331636489 wait complete. 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.532+0000 7f93fdc00640 1 Processor -- start 2024-09-21T07:10:52.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.532+0000 7f93fdc00640 1 -- start start 2024-09-21T07:10:52.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93fdc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8103b60 0x7f93f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:52.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93fdc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f81094a0 0x7f93f8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:52.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93fdc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8109df0 0x7f93f8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:52.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93fdc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8113ab0 con 0x7f93f81094a0 2024-09-21T07:10:52.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93fdc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8113930 con 0x7f93f8103b60 2024-09-21T07:10:52.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93fdc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8113c30 con 0x7f93f8109df0 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93f6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8103b60 0x7f93f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93f6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f81094a0 0x7f93f8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93f6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8103b60 0x7f93f8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55656/0 (socket says 172.21.15.27:55656) 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.533+0000 7f93f6a00640 1 -- 172.21.15.27:0/60710589 learned_addr learned my addr 172.21.15.27:0/60710589 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.534+0000 7f93f7400640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8109df0 0x7f93f8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.534+0000 7f93f6a00640 1 -- 172.21.15.27:0/60710589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8109df0 msgr2=0x7f93f8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.534+0000 7f93f6a00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8109df0 0x7f93f8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.534+0000 7f93f6a00640 1 -- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f81094a0 msgr2=0x7f93f8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.534+0000 7f93f6a00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f81094a0 0x7f93f8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.534+0000 7f93f6a00640 1 -- 172.21.15.27:0/60710589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f81a87a0 con 0x7f93f8103b60 2024-09-21T07:10:52.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.534+0000 7f93f6a00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8103b60 0x7f93f8075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f93e800e9f0 tx=0x7f93e800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:52.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.535+0000 7f93ef400640 1 -- 172.21.15.27:0/60710589 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e800cd80 con 0x7f93f8103b60 2024-09-21T07:10:52.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.535+0000 7f93ef400640 1 -- 172.21.15.27:0/60710589 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e80054d0 con 0x7f93f8103b60 2024-09-21T07:10:52.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.535+0000 7f93ef400640 1 -- 172.21.15.27:0/60710589 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8010690 con 0x7f93f8103b60 2024-09-21T07:10:52.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.535+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f81a8a90 con 0x7f93f8103b60 2024-09-21T07:10:52.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.535+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a8f80 con 0x7f93f8103b60 2024-09-21T07:10:52.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.536+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f8103f70 con 0x7f93f8103b60 2024-09-21T07:10:52.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.539+0000 7f93ef400640 1 -- 172.21.15.27:0/60710589 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f93e8004e90 con 0x7f93f8103b60 2024-09-21T07:10:52.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.541+0000 7f93ef400640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93dc0b6560 0x7f93dc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:52.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.541+0000 7f93ef400640 1 -- 172.21.15.27:0/60710589 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9017+0+0 (secure 0 0 0) 0x7f93e80e7e30 con 0x7f93f8103b60 2024-09-21T07:10:52.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.541+0000 7f93f6000640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93dc0b6560 0x7f93dc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:52.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.542+0000 7f93ef400640 1 -- 172.21.15.27:0/60710589 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93e80ae350 con 0x7f93f8103b60 2024-09-21T07:10:52.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.542+0000 7f93f6000640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93dc0b6560 0x7f93dc0b8a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93f81074b0 tx=0x7f93e4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:52.788+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f93f8076a10 con 0x7f93f8103b60 2024-09-21T07:10:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:52 smithi027 ceph-mon[30779]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T07:10:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:52 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:52 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/60710589' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T07:10:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:52 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T07:10:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:52 smithi153 ceph-mon[34747]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T07:10:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:52 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:52 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/60710589' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T07:10:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:52 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T07:10:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:52 smithi119 ceph-mon[34519]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T07:10:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:52 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:52 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/60710589' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T07:10:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:52 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T07:10:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.649+0000 7f93ef400640 1 -- 172.21.15.27:0/60710589 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f93f8076a10 con 0x7f93f8103b60 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93dc0b6560 msgr2=0x7f93dc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93dc0b6560 0x7f93dc0b8a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93f81074b0 tx=0x7f93e4005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8103b60 msgr2=0x7f93f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8103b60 0x7f93f8075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f93e800e9f0 tx=0x7f93e800edf0 comp rx=0 tx=0).stop 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 shutdown_connections 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8109df0 0x7f93f8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93dc0b6560 0x7f93dc0b8a70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f81094a0 0x7f93f8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 --2- 172.21.15.27:0/60710589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93f8103b60 0x7f93f8075710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 >> 172.21.15.27:0/60710589 conn(0x7f93f807b850 msgr2=0x7f93f8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 shutdown_connections 2024-09-21T07:10:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:53.652+0000 7f93fdc00640 1 -- 172.21.15.27:0/60710589 wait complete. 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T07:10:54.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:10:54.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:10:54.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:54.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T07:10:54.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:53 smithi027 ceph-mon[30779]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:10:54.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:10:54.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:54.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T07:10:54.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:53 smithi153 ceph-mon[34747]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:10:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:10:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:10:54.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T07:10:54.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:53 smithi119 ceph-mon[34519]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:10:54.450 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T07:10:54.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T07:10:54.836 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:55.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-21T07:10:55.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T07:10:55.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T07:10:55.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T07:10:55.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:54 smithi027 ceph-mon[30779]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T07:10:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T07:10:55.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:54 smithi153 ceph-mon[34747]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T07:10:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T07:10:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:54 smithi119 ceph-mon[34519]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T07:10:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 -- 172.21.15.27:0/3208591403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 msgr2=0x7f1c7c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 --2- 172.21.15.27:0/3208591403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 0x7f1c7c1098d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009900 tx=0x7f1c6402a410 comp rx=0 tx=0).stop 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 -- 172.21.15.27:0/3208591403 shutdown_connections 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 --2- 172.21.15.27:0/3208591403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c7c109e10 0x7f1c7c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 --2- 172.21.15.27:0/3208591403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 0x7f1c7c1098d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 --2- 172.21.15.27:0/3208591403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 0x7f1c7c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 -- 172.21.15.27:0/3208591403 >> 172.21.15.27:0/3208591403 conn(0x7f1c7c0ff5d0 msgr2=0x7f1c7c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 -- 172.21.15.27:0/3208591403 shutdown_connections 2024-09-21T07:10:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.447+0000 7f1c81200640 1 -- 172.21.15.27:0/3208591403 wait complete. 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.448+0000 7f1c81200640 1 Processor -- start 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c81200640 1 -- start start 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c81200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 0x7f1c7c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c81200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 0x7f1c7c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c81200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c7c109e10 0x7f1c7c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c81200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c113ad0 con 0x7f1c7c1094c0 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c81200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c113950 con 0x7f1c7c103b80 2024-09-21T07:10:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c81200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c113c50 con 0x7f1c7c109e10 2024-09-21T07:10:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c71400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 0x7f1c7c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c79e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 0x7f1c7c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c71400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 0x7f1c7c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60548/0 (socket says 172.21.15.27:60548) 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c79e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 0x7f1c7c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55680/0 (socket says 172.21.15.27:55680) 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.449+0000 7f1c71400640 1 -- 172.21.15.27:0/1645710249 learned_addr learned my addr 172.21.15.27:0/1645710249 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c7a800640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c7c109e10 0x7f1c7c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c79e00640 1 -- 172.21.15.27:0/1645710249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c7c109e10 msgr2=0x7f1c7c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c79e00640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c7c109e10 0x7f1c7c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c79e00640 1 -- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 msgr2=0x7f1c7c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c79e00640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 0x7f1c7c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c79e00640 1 -- 172.21.15.27:0/1645710249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c1a2630 con 0x7f1c7c103b80 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c79e00640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 0x7f1c7c19c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e9f0 tx=0x7f1c6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.450+0000 7f1c73400640 1 -- 172.21.15.27:0/1645710249 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00ccc0 con 0x7f1c7c103b80 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.451+0000 7f1c73400640 1 -- 172.21.15.27:0/1645710249 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c0108e0 con 0x7f1c7c103b80 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.451+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c1a2920 con 0x7f1c7c103b80 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.451+0000 7f1c73400640 1 -- 172.21.15.27:0/1645710249 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c018880 con 0x7f1c7c103b80 2024-09-21T07:10:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.451+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c1a9080 con 0x7f1c7c103b80 2024-09-21T07:10:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.452+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c103f90 con 0x7f1c7c103b80 2024-09-21T07:10:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.454+0000 7f1c73400640 1 -- 172.21.15.27:0/1645710249 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f1c6c010430 con 0x7f1c7c103b80 2024-09-21T07:10:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.456+0000 7f1c73400640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c600b6560 0x7f1c600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.456+0000 7f1c73400640 1 -- 172.21.15.27:0/1645710249 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f1c6c0e7fc0 con 0x7f1c7c103b80 2024-09-21T07:10:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.456+0000 7f1c71400640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c600b6560 0x7f1c600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:55.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.457+0000 7f1c71400640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c600b6560 0x7f1c600b8a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c108b10 tx=0x7f1c64009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:55.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.458+0000 7f1c73400640 1 -- 172.21.15.27:0/1645710249 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1c6c0b27a0 con 0x7f1c7c103b80 2024-09-21T07:10:55.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.681+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1c7c10f8f0 con 0x7f1c600b6560 2024-09-21T07:10:55.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.711+0000 7f1c73400640 1 -- 172.21.15.27:0/1645710249 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1c7c10f8f0 con 0x7f1c600b6560 2024-09-21T07:10:55.714 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:10:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:10:55.688+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:10:55.714 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:10:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:10:55.688+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:10:55.714 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:10:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:10:55.688+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:10:55.714 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:10:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:10:55.688+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:10:55.714 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:10:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:10:55.688+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:10:55.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c600b6560 msgr2=0x7f1c600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:55.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c600b6560 0x7f1c600b8a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c108b10 tx=0x7f1c64009560 comp rx=0 tx=0).stop 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 msgr2=0x7f1c7c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 0x7f1c7c19c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e9f0 tx=0x7f1c6c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 shutdown_connections 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c7c109e10 0x7f1c7c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c600b6560 0x7f1c600b8a70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c7c1094c0 0x7f1c7c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 --2- 172.21.15.27:0/1645710249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c7c103b80 0x7f1c7c19c6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 >> 172.21.15.27:0/1645710249 conn(0x7f1c7c0ff5d0 msgr2=0x7f1c7c106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 shutdown_connections 2024-09-21T07:10:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:55.714+0000 7f1c81200640 1 -- 172.21.15.27:0/1645710249 wait complete. 2024-09-21T07:10:56.376 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T07:10:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:56 smithi153 ceph-mon[34747]: pgmap v313: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-21T07:10:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:56 smithi153 ceph-mon[34747]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:57.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:56 smithi027 ceph-mon[30779]: pgmap v313: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-21T07:10:57.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:56 smithi027 ceph-mon[30779]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:56 smithi119 ceph-mon[34519]: pgmap v313: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-21T07:10:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:56 smithi119 ceph-mon[34519]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:57.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 -- 172.21.15.27:0/392663480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c010be40 msgr2=0x7fb4c0112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/392663480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c010be40 0x7fb4c0112670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009ca0 tx=0x7fb4b802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 -- 172.21.15.27:0/392663480 shutdown_connections 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/392663480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4c010b4f0 0x7fb4c010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/392663480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c010be40 0x7fb4c0112670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/392663480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c0105bb0 0x7fb4c0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 -- 172.21.15.27:0/392663480 >> 172.21.15.27:0/392663480 conn(0x7fb4c0101680 msgr2=0x7fb4c0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 -- 172.21.15.27:0/392663480 shutdown_connections 2024-09-21T07:10:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.393+0000 7fb4c5c00640 1 -- 172.21.15.27:0/392663480 wait complete. 2024-09-21T07:10:57.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.395+0000 7fb4c5c00640 1 Processor -- start 2024-09-21T07:10:57.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.395+0000 7fb4c5c00640 1 -- start start 2024-09-21T07:10:57.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4c5c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c0105bb0 0x7fb4c019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:57.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4c5c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c010b4f0 0x7fb4c019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:57.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4c5c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4c010be40 0x7fb4c01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4c5c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0115b50 con 0x7fb4c0105bb0 2024-09-21T07:10:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4c5c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01159d0 con 0x7fb4c010b4f0 2024-09-21T07:10:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4c5c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0115cd0 con 0x7fb4c010be40 2024-09-21T07:10:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c0105bb0 0x7fb4c019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c0105bb0 0x7fb4c019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60588/0 (socket says 172.21.15.27:60588) 2024-09-21T07:10:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4bfe00640 1 -- 172.21.15.27:0/3883536010 learned_addr learned my addr 172.21.15.27:0/3883536010 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4c5200640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4c010be40 0x7fb4c01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:57.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.396+0000 7fb4b7400640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c010b4f0 0x7fb4c019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:57.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bfe00640 1 -- 172.21.15.27:0/3883536010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4c010be40 msgr2=0x7fb4c01a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:57.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bfe00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4c010be40 0x7fb4c01a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bfe00640 1 -- 172.21.15.27:0/3883536010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c010b4f0 msgr2=0x7fb4c019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bfe00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c010b4f0 0x7fb4c019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bfe00640 1 -- 172.21.15.27:0/3883536010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c01a2410 con 0x7fb4c0105bb0 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bfe00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c0105bb0 0x7fb4c019c4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e9f0 tx=0x7fb4b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bd600640 1 -- 172.21.15.27:0/3883536010 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000cd80 con 0x7fb4c0105bb0 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bd600640 1 -- 172.21.15.27:0/3883536010 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b00108e0 con 0x7fb4c0105bb0 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4bd600640 1 -- 172.21.15.27:0/3883536010 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0018980 con 0x7fb4c0105bb0 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.397+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c01a2700 con 0x7fb4c0105bb0 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.398+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c01a8e60 con 0x7fb4c0105bb0 2024-09-21T07:10:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.399+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c0105fc0 con 0x7fb4c0105bb0 2024-09-21T07:10:57.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.401+0000 7fb4bd600640 1 -- 172.21.15.27:0/3883536010 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fb4b0010430 con 0x7fb4c0105bb0 2024-09-21T07:10:57.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.403+0000 7fb4bd600640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb4a40b65e0 0x7fb4a40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:57.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.403+0000 7fb4bd600640 1 -- 172.21.15.27:0/3883536010 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fb4b00e7b50 con 0x7fb4c0105bb0 2024-09-21T07:10:57.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.403+0000 7fb4b7400640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb4a40b65e0 0x7fb4a40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:57.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.404+0000 7fb4b7400640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb4a40b65e0 0x7fb4a40b8af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0104f40 tx=0x7fb4a8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:57.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.407+0000 7fb4bd600640 1 -- 172.21.15.27:0/3883536010 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4b00adf50 con 0x7fb4c0105bb0 2024-09-21T07:10:57.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.634+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb4c0063800 con 0x7fb4a40b65e0 2024-09-21T07:10:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.665+0000 7fb4bd600640 1 -- 172.21.15.27:0/3883536010 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4c0063800 con 0x7fb4a40b65e0 2024-09-21T07:10:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb4a40b65e0 msgr2=0x7fb4a40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb4a40b65e0 0x7fb4a40b8af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0104f40 tx=0x7fb4a8005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c0105bb0 msgr2=0x7fb4c019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c0105bb0 0x7fb4c019c4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e9f0 tx=0x7fb4b000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 shutdown_connections 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4c010be40 0x7fb4c01a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb4a40b65e0 0x7fb4a40b8af0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c010b4f0 0x7fb4c019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 --2- 172.21.15.27:0/3883536010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4c0105bb0 0x7fb4c019c4a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 >> 172.21.15.27:0/3883536010 conn(0x7fb4c0101680 msgr2=0x7fb4c0108a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 shutdown_connections 2024-09-21T07:10:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:57.667+0000 7fb4c5c00640 1 -- 172.21.15.27:0/3883536010 wait complete. 2024-09-21T07:10:58.453 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T07:10:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:59.125 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:58 smithi027 ceph-mon[30779]: pgmap v314: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-21T07:10:59.125 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:58 smithi027 ceph-mon[30779]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:59.125 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:10:58 smithi027 ceph-mon[30779]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T07:10:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:58 smithi153 ceph-mon[34747]: pgmap v314: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-21T07:10:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:58 smithi153 ceph-mon[34747]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:10:58 smithi153 ceph-mon[34747]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T07:10:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:58 smithi119 ceph-mon[34519]: pgmap v314: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-21T07:10:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:58 smithi119 ceph-mon[34519]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:10:58 smithi119 ceph-mon[34519]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.449+0000 7f76ac600640 1 -- 172.21.15.27:0/3609733369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 msgr2=0x7f76a41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.449+0000 7f76ac600640 1 --2- 172.21.15.27:0/3609733369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a41098d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7690009900 tx=0x7f769002a410 comp rx=0 tx=0).stop 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.450+0000 7f76ac600640 1 -- 172.21.15.27:0/3609733369 shutdown_connections 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.450+0000 7f76ac600640 1 --2- 172.21.15.27:0/3609733369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4109e10 0x7f76a4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.450+0000 7f76ac600640 1 --2- 172.21.15.27:0/3609733369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a41098d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.450+0000 7f76ac600640 1 --2- 172.21.15.27:0/3609733369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a4103b80 0x7f76a4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.450+0000 7f76ac600640 1 -- 172.21.15.27:0/3609733369 >> 172.21.15.27:0/3609733369 conn(0x7f76a40ff610 msgr2=0x7f76a4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:59.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.450+0000 7f76ac600640 1 -- 172.21.15.27:0/3609733369 shutdown_connections 2024-09-21T07:10:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.450+0000 7f76ac600640 1 -- 172.21.15.27:0/3609733369 wait complete. 2024-09-21T07:10:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.451+0000 7f76ac600640 1 Processor -- start 2024-09-21T07:10:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.451+0000 7f76ac600640 1 -- start start 2024-09-21T07:10:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.451+0000 7f76ac600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4103b80 0x7f76a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.451+0000 7f76ac600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.451+0000 7f76ac600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a4109e10 0x7f76a4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76ac600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4113b20 con 0x7f76a41094c0 2024-09-21T07:10:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76ac600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41139a0 con 0x7f76a4109e10 2024-09-21T07:10:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76ac600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4113ca0 con 0x7f76a4103b80 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76a9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76a9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37060/0 (socket says 172.21.15.27:37060) 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76a9200640 1 -- 172.21.15.27:0/593919234 learned_addr learned my addr 172.21.15.27:0/593919234 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76aa600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a4109e10 0x7f76a4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76a9c00640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4103b80 0x7f76a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76aa600640 1 -- 172.21.15.27:0/593919234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4103b80 msgr2=0x7f76a4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76aa600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4103b80 0x7f76a4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76aa600640 1 -- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 msgr2=0x7f76a4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76aa600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76aa600640 1 -- 172.21.15.27:0/593919234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a41a8790 con 0x7f76a4109e10 2024-09-21T07:10:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.452+0000 7f76a9200640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.453+0000 7f76aa600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a4109e10 0x7f76a4076190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e970 tx=0x7f769c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.453+0000 7f76a2000640 1 -- 172.21.15.27:0/593919234 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c00ccc0 con 0x7f76a4109e10 2024-09-21T07:10:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.453+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a41a8a80 con 0x7f76a4109e10 2024-09-21T07:10:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.453+0000 7f76a2000640 1 -- 172.21.15.27:0/593919234 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f769c005020 con 0x7f76a4109e10 2024-09-21T07:10:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.453+0000 7f76a2000640 1 -- 172.21.15.27:0/593919234 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c004a00 con 0x7f76a4109e10 2024-09-21T07:10:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.453+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a41a8fe0 con 0x7f76a4109e10 2024-09-21T07:10:59.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.454+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a4104e00 con 0x7f76a4109e10 2024-09-21T07:10:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.457+0000 7f76a2000640 1 -- 172.21.15.27:0/593919234 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f769c004ba0 con 0x7f76a4109e10 2024-09-21T07:10:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.459+0000 7f76a2000640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f768c0b65f0 0x7f768c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.459+0000 7f76a2000640 1 -- 172.21.15.27:0/593919234 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f769c0e7990 con 0x7f76a4109e10 2024-09-21T07:10:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.459+0000 7f76a9c00640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f768c0b65f0 0x7f768c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.460+0000 7f76a9c00640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f768c0b65f0 0x7f768c0b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76a4077160 tx=0x7f7698009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.460+0000 7f76a2000640 1 -- 172.21.15.27:0/593919234 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f769c0adcc0 con 0x7f76a4109e10 2024-09-21T07:10:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.685+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f76a4063800 con 0x7f768c0b65f0 2024-09-21T07:10:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.694+0000 7f76a2000640 1 -- 172.21.15.27:0/593919234 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f76a4063800 con 0x7f768c0b65f0 2024-09-21T07:10:59.696 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_0/40a72e3d-f2d8-42fe-bf06-7a940a38aea6 2024-09-21T07:10:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f768c0b65f0 msgr2=0x7f768c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f768c0b65f0 0x7f768c0b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76a4077160 tx=0x7f7698009500 comp rx=0 tx=0).stop 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a4109e10 msgr2=0x7f76a4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a4109e10 0x7f76a4076190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e970 tx=0x7f769c00ee30 comp rx=0 tx=0).stop 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 shutdown_connections 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f768c0b65f0 0x7f768c0b8b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a4109e10 0x7f76a4076190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a41094c0 0x7f76a4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 --2- 172.21.15.27:0/593919234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4103b80 0x7f76a4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 >> 172.21.15.27:0/593919234 conn(0x7f76a40ff610 msgr2=0x7f76a4106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 shutdown_connections 2024-09-21T07:10:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:10:59.696+0000 7f76ac600640 1 -- 172.21.15.27:0/593919234 wait complete. 2024-09-21T07:11:00.361 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T07:11:00.749 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:01.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:00 smithi027 ceph-mon[30779]: pgmap v315: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.1 KiB/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:01.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:00 smithi027 ceph-mon[30779]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:00 smithi153 ceph-mon[34747]: pgmap v315: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.1 KiB/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:00 smithi153 ceph-mon[34747]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:00 smithi119 ceph-mon[34519]: pgmap v315: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.1 KiB/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:00 smithi119 ceph-mon[34519]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.348+0000 7f2218600640 1 -- 172.21.15.27:0/2675202532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f221010de20 msgr2=0x7f2210110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.348+0000 7f2218600640 1 --2- 172.21.15.27:0/2675202532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f221010de20 0x7f2210110310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2208009ca0 tx=0x7f220802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.348+0000 7f2218600640 1 -- 172.21.15.27:0/2675202532 shutdown_connections 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.348+0000 7f2218600640 1 --2- 172.21.15.27:0/2675202532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2210105310 0x7f221010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.348+0000 7f2218600640 1 --2- 172.21.15.27:0/2675202532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f221010de20 0x7f2210110310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.348+0000 7f2218600640 1 --2- 172.21.15.27:0/2675202532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22101049c0 0x7f2210104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.348+0000 7f2218600640 1 -- 172.21.15.27:0/2675202532 >> 172.21.15.27:0/2675202532 conn(0x7f2210079730 msgr2=0x7f2210079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.349+0000 7f2218600640 1 -- 172.21.15.27:0/2675202532 shutdown_connections 2024-09-21T07:11:01.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.349+0000 7f2218600640 1 -- 172.21.15.27:0/2675202532 wait complete. 2024-09-21T07:11:01.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 Processor -- start 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 -- start start 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22101049c0 0x7f2210102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2210105310 0x7f2210102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f221010de20 0x7f2210103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2210113a80 con 0x7f2210105310 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2210113900 con 0x7f221010de20 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.350+0000 7f2218600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2210113c00 con 0x7f22101049c0 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2210105310 0x7f2210102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22101049c0 0x7f2210102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22101049c0 0x7f2210102700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34154/0 (socket says 172.21.15.27:34154) 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215c00640 1 -- 172.21.15.27:0/3503062548 learned_addr learned my addr 172.21.15.27:0/3503062548 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:01.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2216600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f221010de20 0x7f2210103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215200640 1 -- 172.21.15.27:0/3503062548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22101049c0 msgr2=0x7f2210102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215200640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22101049c0 0x7f2210102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215200640 1 -- 172.21.15.27:0/3503062548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f221010de20 msgr2=0x7f2210103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215200640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f221010de20 0x7f2210103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215200640 1 -- 172.21.15.27:0/3503062548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22101a86f0 con 0x7f2210105310 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2216600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f221010de20 0x7f2210103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.351+0000 7f2215c00640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22101049c0 0x7f2210102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.352+0000 7f2215200640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2210105310 0x7f2210102c40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f221006b950 tx=0x7f21fc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.352+0000 7f220e000640 1 -- 172.21.15.27:0/3503062548 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00cc40 con 0x7f2210105310 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.352+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22101a89e0 con 0x7f2210105310 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.352+0000 7f220e000640 1 -- 172.21.15.27:0/3503062548 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21fc00cde0 con 0x7f2210105310 2024-09-21T07:11:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.352+0000 7f220e000640 1 -- 172.21.15.27:0/3503062548 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc018760 con 0x7f2210105310 2024-09-21T07:11:01.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.352+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22101a8f20 con 0x7f2210105310 2024-09-21T07:11:01.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.353+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221006c1e0 con 0x7f2210105310 2024-09-21T07:11:01.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.357+0000 7f220e000640 1 -- 172.21.15.27:0/3503062548 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f21fc010a90 con 0x7f2210105310 2024-09-21T07:11:01.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.358+0000 7f220e000640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21f80b65f0 0x7f21f80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:01.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.358+0000 7f220e000640 1 -- 172.21.15.27:0/3503062548 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f21fc0e80e0 con 0x7f2210105310 2024-09-21T07:11:01.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.358+0000 7f2215c00640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21f80b65f0 0x7f21f80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:01.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.359+0000 7f2215c00640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21f80b65f0 0x7f21f80b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2210104150 tx=0x7f2204005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:01.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.359+0000 7f220e000640 1 -- 172.21.15.27:0/3503062548 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f21fc0b2760 con 0x7f2210105310 2024-09-21T07:11:01.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.591+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2210063800 con 0x7f21f80b65f0 2024-09-21T07:11:01.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.621+0000 7f220e000640 1 -- 172.21.15.27:0/3503062548 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2210063800 con 0x7f21f80b65f0 2024-09-21T07:11:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21f80b65f0 msgr2=0x7f21f80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21f80b65f0 0x7f21f80b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2210104150 tx=0x7f2204005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2210105310 msgr2=0x7f2210102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2210105310 0x7f2210102c40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f221006b950 tx=0x7f21fc00ee50 comp rx=0 tx=0).stop 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 shutdown_connections 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21f80b65f0 0x7f21f80b8b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f221010de20 0x7f2210103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2210105310 0x7f2210102c40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 --2- 172.21.15.27:0/3503062548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22101049c0 0x7f2210102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 >> 172.21.15.27:0/3503062548 conn(0x7f2210079730 msgr2=0x7f22101007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 shutdown_connections 2024-09-21T07:11:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:01.623+0000 7f2218600640 1 -- 172.21.15.27:0/3503062548 wait complete. 2024-09-21T07:11:02.349 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T07:11:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:02 smithi153 ceph-mon[34747]: pgmap v316: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:02 smithi153 ceph-mon[34747]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:03.199 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:02 smithi027 ceph-mon[30779]: pgmap v316: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:03.201 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:02 smithi027 ceph-mon[30779]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:02 smithi119 ceph-mon[34519]: pgmap v316: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:02 smithi119 ceph-mon[34519]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:03.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 -- 172.21.15.27:0/2968412409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 msgr2=0x7f340010f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:03.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 --2- 172.21.15.27:0/2968412409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340010f9b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009860 tx=0x7f33ec02a340 comp rx=0 tx=0).stop 2024-09-21T07:11:03.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 -- 172.21.15.27:0/2968412409 shutdown_connections 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 --2- 172.21.15.27:0/2968412409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340010f9b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 --2- 172.21.15.27:0/2968412409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340010fef0 0x7f3400112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 --2- 172.21.15.27:0/2968412409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34000757b0 0x7f3400075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 -- 172.21.15.27:0/2968412409 >> 172.21.15.27:0/2968412409 conn(0x7f34000ff3e0 msgr2=0x7f3400101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 -- 172.21.15.27:0/2968412409 shutdown_connections 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.385+0000 7f3406200640 1 -- 172.21.15.27:0/2968412409 wait complete. 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.386+0000 7f3406200640 1 Processor -- start 2024-09-21T07:11:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.386+0000 7f3406200640 1 -- start start 2024-09-21T07:11:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3406200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34000757b0 0x7f340019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3406200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3406200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f340010fef0 0x7f34001a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3406200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3400115aa0 con 0x7f340010fef0 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3406200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3400115920 con 0x7f34000757b0 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3406200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3400115c20 con 0x7f3400076100 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3405800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f340010fef0 0x7f34001a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34162/0 (socket says 172.21.15.27:34162) 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f33ffe00640 1 -- 172.21.15.27:0/952116480 learned_addr learned my addr 172.21.15.27:0/952116480 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.387+0000 7f3404e00640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34000757b0 0x7f340019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.388+0000 7f33ffe00640 1 -- 172.21.15.27:0/952116480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34000757b0 msgr2=0x7f340019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.388+0000 7f33ffe00640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34000757b0 0x7f340019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.388+0000 7f33ffe00640 1 -- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f340010fef0 msgr2=0x7f34001a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.388+0000 7f33ffe00640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f340010fef0 0x7f34001a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.388+0000 7f33ffe00640 1 -- 172.21.15.27:0/952116480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34001a2480 con 0x7f3400076100 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.388+0000 7f3405800640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f340010fef0 0x7f34001a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.388+0000 7f33ffe00640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340019cab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33ec000c00 tx=0x7f33ec032e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.389+0000 7f33fd600640 1 -- 172.21.15.27:0/952116480 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec040020 con 0x7f3400076100 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.389+0000 7f33fd600640 1 -- 172.21.15.27:0/952116480 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33ec030e50 con 0x7f3400076100 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.389+0000 7f33fd600640 1 -- 172.21.15.27:0/952116480 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec037040 con 0x7f3400076100 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.389+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34001a2710 con 0x7f3400076100 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.389+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34001a2ba0 con 0x7f3400076100 2024-09-21T07:11:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.390+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3400075bc0 con 0x7f3400076100 2024-09-21T07:11:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.392+0000 7f33fd600640 1 -- 172.21.15.27:0/952116480 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f33ec043020 con 0x7f3400076100 2024-09-21T07:11:03.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.394+0000 7f33fd600640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33e80b65f0 0x7f33e80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:03.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.394+0000 7f33fd600640 1 -- 172.21.15.27:0/952116480 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f33ec107630 con 0x7f3400076100 2024-09-21T07:11:03.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.394+0000 7f3404e00640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33e80b65f0 0x7f33e80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:03.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.395+0000 7f3404e00640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33e80b65f0 0x7f33e80b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005fd0 tx=0x7f33f4009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:03.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.395+0000 7f33fd600640 1 -- 172.21.15.27:0/952116480 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34001a2ba0 con 0x7f3400076100 2024-09-21T07:11:03.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.624+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f340010a1e0 con 0x7f33e80b65f0 2024-09-21T07:11:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.628+0000 7f33fd600640 1 -- 172.21.15.27:0/952116480 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f340010a1e0 con 0x7f33e80b65f0 2024-09-21T07:11:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_1/c9dc85fb-de11-40b4-8b5e-6af2fcaf6fa0 2024-09-21T07:11:03.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33e80b65f0 msgr2=0x7f33e80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:03.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33e80b65f0 0x7f33e80b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005fd0 tx=0x7f33f4009260 comp rx=0 tx=0).stop 2024-09-21T07:11:03.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 msgr2=0x7f340019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:03.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340019cab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33ec000c00 tx=0x7f33ec032e10 comp rx=0 tx=0).stop 2024-09-21T07:11:03.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 shutdown_connections 2024-09-21T07:11:03.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3400076100 0x7f340019cab0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33e80b65f0 0x7f33e80b8b00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f340010fef0 0x7f34001a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 --2- 172.21.15.27:0/952116480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34000757b0 0x7f340019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:03.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 >> 172.21.15.27:0/952116480 conn(0x7f34000ff3e0 msgr2=0x7f3400109ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:03.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 shutdown_connections 2024-09-21T07:11:03.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:03.631+0000 7f3406200640 1 -- 172.21.15.27:0/952116480 wait complete. 2024-09-21T07:11:04.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:11:04.742 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:04 smithi153 ceph-mon[34747]: pgmap v317: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:04 smithi153 ceph-mon[34747]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:05.199 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:04 smithi027 ceph-mon[30779]: pgmap v317: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:05.199 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:04 smithi027 ceph-mon[30779]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:04 smithi119 ceph-mon[34519]: pgmap v317: 129 pgs: 129 active+clean; 611 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:04 smithi119 ceph-mon[34519]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:11:05.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 -- 172.21.15.27:0/419888129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 msgr2=0x7f09781098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:05.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 --2- 172.21.15.27:0/419888129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f09781098b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0960009900 tx=0x7f096002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 -- 172.21.15.27:0/419888129 shutdown_connections 2024-09-21T07:11:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 --2- 172.21.15.27:0/419888129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978109df0 0x7f0978110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 --2- 172.21.15.27:0/419888129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f09781098b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 --2- 172.21.15.27:0/419888129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0978103b60 0x7f0978103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 -- 172.21.15.27:0/419888129 >> 172.21.15.27:0/419888129 conn(0x7f097807b850 msgr2=0x7f097807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 -- 172.21.15.27:0/419888129 shutdown_connections 2024-09-21T07:11:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.379+0000 7f097e000640 1 -- 172.21.15.27:0/419888129 wait complete. 2024-09-21T07:11:05.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.380+0000 7f097e000640 1 Processor -- start 2024-09-21T07:11:05.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.380+0000 7f097e000640 1 -- start start 2024-09-21T07:11:05.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f097e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978103b60 0x7f0978075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:05.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f097e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f0978075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f097e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0978109df0 0x7f0978076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f097e000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0978113ab0 con 0x7f09781094a0 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f097e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0978113930 con 0x7f0978103b60 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f097e000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0978113c30 con 0x7f0978109df0 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f096fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f0978075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f096fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f0978075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37114/0 (socket says 172.21.15.27:37114) 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f096fe00640 1 -- 172.21.15.27:0/1524020664 learned_addr learned my addr 172.21.15.27:0/1524020664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f0977400640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978103b60 0x7f0978075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:05.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.381+0000 7f0977e00640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0978109df0 0x7f0978076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:05.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.382+0000 7f0977400640 1 -- 172.21.15.27:0/1524020664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0978109df0 msgr2=0x7f0978076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:05.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.382+0000 7f0977400640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0978109df0 0x7f0978076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.382+0000 7f0977400640 1 -- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 msgr2=0x7f0978075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:05.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.382+0000 7f0977400640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f0978075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.382+0000 7f0977400640 1 -- 172.21.15.27:0/1524020664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781a8760 con 0x7f0978103b60 2024-09-21T07:11:05.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.382+0000 7f096fe00640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f0978075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:05.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.382+0000 7f0977400640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978103b60 0x7f0978075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f096800e9f0 tx=0x7f096800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:05.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.383+0000 7f0974c00640 1 -- 172.21.15.27:0/1524020664 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096800ccc0 con 0x7f0978103b60 2024-09-21T07:11:05.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.383+0000 7f0974c00640 1 -- 172.21.15.27:0/1524020664 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09680108e0 con 0x7f0978103b60 2024-09-21T07:11:05.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.383+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781a8a50 con 0x7f0978103b60 2024-09-21T07:11:05.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.383+0000 7f0974c00640 1 -- 172.21.15.27:0/1524020664 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968018880 con 0x7f0978103b60 2024-09-21T07:11:05.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.383+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781a8f90 con 0x7f0978103b60 2024-09-21T07:11:05.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.384+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0978103f70 con 0x7f0978103b60 2024-09-21T07:11:05.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.386+0000 7f0974c00640 1 -- 172.21.15.27:0/1524020664 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0968010430 con 0x7f0978103b60 2024-09-21T07:11:05.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.388+0000 7f0974c00640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f095c0b65f0 0x7f095c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:05.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.388+0000 7f0974c00640 1 -- 172.21.15.27:0/1524020664 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f09680e8120 con 0x7f0978103b60 2024-09-21T07:11:05.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.388+0000 7f096fe00640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f095c0b65f0 0x7f095c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:05.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.389+0000 7f096fe00640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f095c0b65f0 0x7f095c0b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0978107590 tx=0x7f0960032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.389+0000 7f0974c00640 1 -- 172.21.15.27:0/1524020664 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09680b27a0 con 0x7f0978103b60 2024-09-21T07:11:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.629+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0978076a10 con 0x7f0978103b60 2024-09-21T07:11:05.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.630+0000 7f0974c00640 1 -- 172.21.15.27:0/1524020664 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7f0978076a10 con 0x7f0978103b60 2024-09-21T07:11:05.633 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:05.633 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T07:10:53:642840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:52.789806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24773,24748,24740]},"id":1}]} 2024-09-21T07:11:05.633 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T07:11:05.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f095c0b65f0 msgr2=0x7f095c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:05.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f095c0b65f0 0x7f095c0b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0978107590 tx=0x7f0960032ed0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978103b60 msgr2=0x7f0978075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:05.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978103b60 0x7f0978075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f096800e9f0 tx=0x7f096800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 shutdown_connections 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0978109df0 0x7f0978076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f095c0b65f0 0x7f095c0b8b00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09781094a0 0x7f0978075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 --2- 172.21.15.27:0/1524020664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978103b60 0x7f0978075710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 >> 172.21.15.27:0/1524020664 conn(0x7f097807b850 msgr2=0x7f0978105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 shutdown_connections 2024-09-21T07:11:05.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:05.633+0000 7f097e000640 1 -- 172.21.15.27:0/1524020664 wait complete. 2024-09-21T07:11:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:05 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1524020664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:11:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:05 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1524020664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:11:06.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:05 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1524020664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:11:06.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:11:06.770 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:06 smithi153 ceph-mon[34747]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:07.209 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:06 smithi027 ceph-mon[30779]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:06 smithi119 ceph-mon[34519]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T07:11:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.397+0000 7fc0dbe00640 1 -- 172.21.15.27:0/3513929250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc109c40 msgr2=0x7fc0dc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.397+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/3513929250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc109c40 0x7fc0dc110470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009ca0 tx=0x7fc0d402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.398+0000 7fc0dbe00640 1 -- 172.21.15.27:0/3513929250 shutdown_connections 2024-09-21T07:11:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.398+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/3513929250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0dc1092f0 0x7fc0dc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.398+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/3513929250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc109c40 0x7fc0dc110470 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.398+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/3513929250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0dc1039b0 0x7fc0dc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.398+0000 7fc0dbe00640 1 -- 172.21.15.27:0/3513929250 >> 172.21.15.27:0/3513929250 conn(0x7fc0dc0ff440 msgr2=0x7fc0dc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.398+0000 7fc0dbe00640 1 -- 172.21.15.27:0/3513929250 shutdown_connections 2024-09-21T07:11:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.398+0000 7fc0dbe00640 1 -- 172.21.15.27:0/3513929250 wait complete. 2024-09-21T07:11:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 Processor -- start 2024-09-21T07:11:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 -- start start 2024-09-21T07:11:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0dc1039b0 0x7fc0dc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc1092f0 0x7fc0dc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0dc109c40 0x7fc0dc1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc113900 con 0x7fc0dc1092f0 2024-09-21T07:11:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc113780 con 0x7fc0dc1039b0 2024-09-21T07:11:07.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.399+0000 7fc0dbe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc113a80 con 0x7fc0dc109c40 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc1092f0 0x7fc0dc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0daa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0dc1039b0 0x7fc0dc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc1092f0 0x7fc0dc19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37124/0 (socket says 172.21.15.27:37124) 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0d3e00640 1 -- 172.21.15.27:0/1204197688 learned_addr learned my addr 172.21.15.27:0/1204197688 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0db400640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0dc109c40 0x7fc0dc1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0daa00640 1 -- 172.21.15.27:0/1204197688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0dc109c40 msgr2=0x7fc0dc1a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0daa00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0dc109c40 0x7fc0dc1a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0daa00640 1 -- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc1092f0 msgr2=0x7fc0dc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0daa00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc1092f0 0x7fc0dc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.400+0000 7fc0daa00640 1 -- 172.21.15.27:0/1204197688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0dc1a2480 con 0x7fc0dc1039b0 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.401+0000 7fc0daa00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0dc1039b0 0x7fc0dc19c510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e9f0 tx=0x7fc0cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.401+0000 7fc0e0a00640 1 -- 172.21.15.27:0/1204197688 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00ccc0 con 0x7fc0dc1039b0 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.401+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0dc1a2770 con 0x7fc0dc1039b0 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.401+0000 7fc0e0a00640 1 -- 172.21.15.27:0/1204197688 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0cc0108e0 con 0x7fc0dc1039b0 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.401+0000 7fc0e0a00640 1 -- 172.21.15.27:0/1204197688 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc018880 con 0x7fc0dc1039b0 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.401+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0dc1a8ed0 con 0x7fc0dc1039b0 2024-09-21T07:11:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.402+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0dc103dc0 con 0x7fc0dc1039b0 2024-09-21T07:11:07.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.405+0000 7fc0e0a00640 1 -- 172.21.15.27:0/1204197688 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc0cc010430 con 0x7fc0dc1039b0 2024-09-21T07:11:07.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.406+0000 7fc0e0a00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0c00b65f0 0x7fc0c00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:07.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.407+0000 7fc0e0a00640 1 -- 172.21.15.27:0/1204197688 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fc0cc0e8150 con 0x7fc0dc1039b0 2024-09-21T07:11:07.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.407+0000 7fc0d3e00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0c00b65f0 0x7fc0c00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:07.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.408+0000 7fc0d3e00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0c00b65f0 0x7fc0c00b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc109b10 tx=0x7fc0c4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:07.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.408+0000 7fc0e0a00640 1 -- 172.21.15.27:0/1204197688 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0cc0b27d0 con 0x7fc0dc1039b0 2024-09-21T07:11:07.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.612+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0dc1a92d0 con 0x7fc0dc1039b0 2024-09-21T07:11:07.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.614+0000 7fc0e0a00640 1 -- 172.21.15.27:0/1204197688 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+25122 (secure 0 0 0) 0x7fc0cc0b7710 con 0x7fc0dc1039b0 2024-09-21T07:11:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:07.617 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:54.838649+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:11:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0c00b65f0 msgr2=0x7fc0c00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0c00b65f0 0x7fc0c00b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc109b10 tx=0x7fc0c4005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0dc1039b0 msgr2=0x7fc0dc19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0dc1039b0 0x7fc0dc19c510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e9f0 tx=0x7fc0cc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 shutdown_connections 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0dc109c40 0x7fc0dc1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0c00b65f0 0x7fc0c00b8b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0dc1092f0 0x7fc0dc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 --2- 172.21.15.27:0/1204197688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0dc1039b0 0x7fc0dc19c510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 >> 172.21.15.27:0/1204197688 conn(0x7fc0dc0ff440 msgr2=0x7fc0dc107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.617+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 shutdown_connections 2024-09-21T07:11:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:07.618+0000 7fc0dbe00640 1 -- 172.21.15.27:0/1204197688 wait complete. 2024-09-21T07:11:08.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:07 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1204197688' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:08.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:07 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1204197688' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:08.292 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:07 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1204197688' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:08.335 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T07:11:08.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T07:11:08.723 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:09.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:08 smithi027 ceph-mon[30779]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.4 KiB/s wr, 8 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T07:11:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:08 smithi153 ceph-mon[34747]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.4 KiB/s wr, 8 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T07:11:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:08 smithi119 ceph-mon[34519]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.4 KiB/s wr, 8 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.336+0000 7f507ca00640 1 -- 172.21.15.27:0/3578389454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 msgr2=0x7f5078109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.336+0000 7f507ca00640 1 --2- 172.21.15.27:0/3578389454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 0x7f5078109700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5064009900 tx=0x7f506402a410 comp rx=0 tx=0).stop 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.337+0000 7f507ca00640 1 -- 172.21.15.27:0/3578389454 shutdown_connections 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.337+0000 7f507ca00640 1 --2- 172.21.15.27:0/3578389454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5078109c40 0x7f5078110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.337+0000 7f507ca00640 1 --2- 172.21.15.27:0/3578389454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 0x7f5078109700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.337+0000 7f507ca00640 1 --2- 172.21.15.27:0/3578389454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50781039b0 0x7f5078103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.337+0000 7f507ca00640 1 -- 172.21.15.27:0/3578389454 >> 172.21.15.27:0/3578389454 conn(0x7f50780ff400 msgr2=0x7f5078101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.337+0000 7f507ca00640 1 -- 172.21.15.27:0/3578389454 shutdown_connections 2024-09-21T07:11:09.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.337+0000 7f507ca00640 1 -- 172.21.15.27:0/3578389454 wait complete. 2024-09-21T07:11:09.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 Processor -- start 2024-09-21T07:11:09.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 -- start start 2024-09-21T07:11:09.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50781039b0 0x7f507819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 0x7f507819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078109c40 0x7f50781a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5078113900 con 0x7f50781092f0 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5078113780 con 0x7f5078109c40 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.338+0000 7f507ca00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5078113a80 con 0x7f50781039b0 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5076800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50781039b0 0x7f507819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5077200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078109c40 0x7f50781a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5075e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 0x7f507819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5076800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50781039b0 0x7f507819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45818/0 (socket says 172.21.15.27:45818) 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5076800640 1 -- 172.21.15.27:0/1707994894 learned_addr learned my addr 172.21.15.27:0/1707994894 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5077200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078109c40 0x7f50781a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56182/0 (socket says 172.21.15.27:56182) 2024-09-21T07:11:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5077200640 1 -- 172.21.15.27:0/1707994894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50781039b0 msgr2=0x7f507819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5077200640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50781039b0 0x7f507819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5077200640 1 -- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 msgr2=0x7f507819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5077200640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 0x7f507819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5077200640 1 -- 172.21.15.27:0/1707994894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50781a2480 con 0x7f5078109c40 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5076800640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50781039b0 0x7f507819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.339+0000 7f5075e00640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 0x7f507819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.340+0000 7f5077200640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078109c40 0x7f50781a1d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f507000da10 tx=0x7f507000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.340+0000 7f506f400640 1 -- 172.21.15.27:0/1707994894 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507000bd70 con 0x7f5078109c40 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.340+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50781a2770 con 0x7f5078109c40 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.340+0000 7f506f400640 1 -- 172.21.15.27:0/1707994894 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5070003e20 con 0x7f5078109c40 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.340+0000 7f506f400640 1 -- 172.21.15.27:0/1707994894 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50700106c0 con 0x7f5078109c40 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.340+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781a8ed0 con 0x7f5078109c40 2024-09-21T07:11:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.341+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5078103dc0 con 0x7f5078109c40 2024-09-21T07:11:09.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.343+0000 7f506f400640 1 -- 172.21.15.27:0/1707994894 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5070004270 con 0x7f5078109c40 2024-09-21T07:11:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.345+0000 7f506f400640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f505c0b65f0 0x7f505c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.345+0000 7f506f400640 1 -- 172.21.15.27:0/1707994894 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f50700e7d00 con 0x7f5078109c40 2024-09-21T07:11:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.345+0000 7f5076800640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f505c0b65f0 0x7f505c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:09.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.346+0000 7f5076800640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f505c0b65f0 0x7f505c0b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50680095f0 tx=0x7f5068005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:09.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.346+0000 7f506f400640 1 -- 172.21.15.27:0/1707994894 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50700ae030 con 0x7f5078109c40 2024-09-21T07:11:09.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.548+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f507810fe50 con 0x7f505c0b65f0 2024-09-21T07:11:09.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.559+0000 7f506f400640 1 -- 172.21.15.27:0/1707994894 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340349 (secure 0 0 0) 0x7f507810fe50 con 0x7f505c0b65f0 2024-09-21T07:11:09.565 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:09.567 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f505c0b65f0 msgr2=0x7f505c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f505c0b65f0 0x7f505c0b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50680095f0 tx=0x7f5068005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078109c40 msgr2=0x7f50781a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078109c40 0x7f50781a1d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f507000da10 tx=0x7f507000ded0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 shutdown_connections 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f505c0b65f0 0x7f505c0b8b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078109c40 0x7f50781a1d50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50781092f0 0x7f507819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 --2- 172.21.15.27:0/1707994894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50781039b0 0x7f507819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 >> 172.21.15.27:0/1707994894 conn(0x7f50780ff400 msgr2=0x7f507810e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 shutdown_connections 2024-09-21T07:11:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:09.564+0000 7f507ca00640 1 -- 172.21.15.27:0/1707994894 wait complete. 2024-09-21T07:11:10.324 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-21T07:11:09.384733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655412,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":206,"num_read_kb":197,"num_write":286,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387616,"kb_used_data":13400,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554208,"statfs":{"total":1151940427776,"available":1151543508992,"internally_reserved":0,"allocated":13721600,"data_stored":7888247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57592,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":93,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003586"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844106+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:10:54.844106+0000","last_peered":"2024-09-21T07:10:54.844106+0000","last_clean":"2024-09-21T07:10:54.844106+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:10:54.844106+0000","last_undegraded":"2024-09-21T07:10:54.844106+0000","last_fullsized":"2024-09-21T07:10:54.844106+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857694+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:10:54.857694+0000","last_peered":"2024-09-21T07:10:54.857694+0000","last_clean":"2024-09-21T07:10:54.857694+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:10:54.857694+0000","last_undegraded":"2024-09-21T07:10:54.857694+0000","last_fullsized":"2024-09-21T07:10:54.857694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850712+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:10:54.850712+0000","last_peered":"2024-09-21T07:10:54.850712+0000","last_clean":"2024-09-21T07:10:54.850712+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:10:54.850712+0000","last_undegraded":"2024-09-21T07:10:54.850712+0000","last_fullsized":"2024-09-21T07:10:54.850712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858585+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:10:54.858585+0000","last_peered":"2024-09-21T07:10:54.858585+0000","last_clean":"2024-09-21T07:10:54.858585+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:10:54.858585+0000","last_undegraded":"2024-09-21T07:10:54.858585+0000","last_fullsized":"2024-09-21T07:10:54.858585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847175+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:10:54.847175+0000","last_peered":"2024-09-21T07:10:54.847175+0000","last_clean":"2024-09-21T07:10:54.847175+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:10:54.847175+0000","last_undegraded":"2024-09-21T07:10:54.847175+0000","last_fullsized":"2024-09-21T07:10:54.847175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854448+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:10:54.854448+0000","last_peered":"2024-09-21T07:10:54.854448+0000","last_clean":"2024-09-21T07:10:54.854448+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:10:54.854448+0000","last_undegraded":"2024-09-21T07:10:54.854448+0000","last_fullsized":"2024-09-21T07:10:54.854448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858193+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:10:54.858193+0000","last_peered":"2024-09-21T07:10:54.858193+0000","last_clean":"2024-09-21T07:10:54.858193+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:10:54.858193+0000","last_undegraded":"2024-09-21T07:10:54.858193+0000","last_fullsized":"2024-09-21T07:10:54.858193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848421+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:10:54.848421+0000","last_peered":"2024-09-21T07:10:54.848421+0000","last_clean":"2024-09-21T07:10:54.848421+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:10:54.848421+0000","last_undegraded":"2024-09-21T07:10:54.848421+0000","last_fullsized":"2024-09-21T07:10:54.848421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854828+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:10:54.854828+0000","last_peered":"2024-09-21T07:10:54.854828+0000","last_clean":"2024-09-21T07:10:54.854828+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:10:54.854828+0000","last_undegraded":"2024-09-21T07:10:54.854828+0000","last_fullsized":"2024-09-21T07:10:54.854828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857732+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:10:54.857732+0000","last_peered":"2024-09-21T07:10:54.857732+0000","last_clean":"2024-09-21T07:10:54.857732+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:10:54.857732+0000","last_undegraded":"2024-09-21T07:10:54.857732+0000","last_fullsized":"2024-09-21T07:10:54.857732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848591+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:10:54.848591+0000","last_peered":"2024-09-21T07:10:54.848591+0000","last_clean":"2024-09-21T07:10:54.848591+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:10:54.848591+0000","last_undegraded":"2024-09-21T07:10:54.848591+0000","last_fullsized":"2024-09-21T07:10:54.848591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854459+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:10:54.854459+0000","last_peered":"2024-09-21T07:10:54.854459+0000","last_clean":"2024-09-21T07:10:54.854459+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:10:54.854459+0000","last_undegraded":"2024-09-21T07:10:54.854459+0000","last_fullsized":"2024-09-21T07:10:54.854459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844977+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:10:54.844977+0000","last_peered":"2024-09-21T07:10:54.844977+0000","last_clean":"2024-09-21T07:10:54.844977+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:10:54.844977+0000","last_undegraded":"2024-09-21T07:10:54.844977+0000","last_fullsized":"2024-09-21T07:10:54.844977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857487+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:10:54.857487+0000","last_peered":"2024-09-21T07:10:54.857487+0000","last_clean":"2024-09-21T07:10:54.857487+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:10:54.857487+0000","last_undegraded":"2024-09-21T07:10:54.857487+0000","last_fullsized":"2024-09-21T07:10:54.857487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855146+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:10:54.855146+0000","last_peered":"2024-09-21T07:10:54.855146+0000","last_clean":"2024-09-21T07:10:54.855146+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:10:54.855146+0000","last_undegraded":"2024-09-21T07:10:54.855146+0000","last_fullsized":"2024-09-21T07:10:54.855146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856487+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:10:54.856487+0000","last_peered":"2024-09-21T07:10:54.856487+0000","last_clean":"2024-09-21T07:10:54.856487+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:10:54.856487+0000","last_undegraded":"2024-09-21T07:10:54.856487+0000","last_fullsized":"2024-09-21T07:10:54.856487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.867034+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:10:54.867034+0000","last_peered":"2024-09-21T07:10:54.867034+0000","last_clean":"2024-09-21T07:10:54.867034+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:10:54.867034+0000","last_undegraded":"2024-09-21T07:10:54.867034+0000","last_fullsized":"2024-09-21T07:10:54.867034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854799+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:10:54.854799+0000","last_peered":"2024-09-21T07:10:54.854799+0000","last_clean":"2024-09-21T07:10:54.854799+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:10:54.854799+0000","last_undegraded":"2024-09-21T07:10:54.854799+0000","last_fullsized":"2024-09-21T07:10:54.854799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851783+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:10:54.851783+0000","last_peered":"2024-09-21T07:10:54.851783+0000","last_clean":"2024-09-21T07:10:54.851783+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:10:54.851783+0000","last_undegraded":"2024-09-21T07:10:54.851783+0000","last_fullsized":"2024-09-21T07:10:54.851783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856047+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:10:54.856047+0000","last_peered":"2024-09-21T07:10:54.856047+0000","last_clean":"2024-09-21T07:10:54.856047+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:10:54.856047+0000","last_undegraded":"2024-09-21T07:10:54.856047+0000","last_fullsized":"2024-09-21T07:10:54.856047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843895+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:10:54.843895+0000","last_peered":"2024-09-21T07:10:54.843895+0000","last_clean":"2024-09-21T07:10:54.843895+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:10:54.843895+0000","last_undegraded":"2024-09-21T07:10:54.843895+0000","last_fullsized":"2024-09-21T07:10:54.843895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857546+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:10:54.857546+0000","last_peered":"2024-09-21T07:10:54.857546+0000","last_clean":"2024-09-21T07:10:54.857546+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:10:54.857546+0000","last_undegraded":"2024-09-21T07:10:54.857546+0000","last_fullsized":"2024-09-21T07:10:54.857546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847552+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:10:54.847552+0000","last_peered":"2024-09-21T07:10:54.847552+0000","last_clean":"2024-09-21T07:10:54.847552+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:10:54.847552+0000","last_undegraded":"2024-09-21T07:10:54.847552+0000","last_fullsized":"2024-09-21T07:10:54.847552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855277+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:10:54.855277+0000","last_peered":"2024-09-21T07:10:54.855277+0000","last_clean":"2024-09-21T07:10:54.855277+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:10:54.855277+0000","last_undegraded":"2024-09-21T07:10:54.855277+0000","last_fullsized":"2024-09-21T07:10:54.855277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847198+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:10:54.847198+0000","last_peered":"2024-09-21T07:10:54.847198+0000","last_clean":"2024-09-21T07:10:54.847198+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:10:54.847198+0000","last_undegraded":"2024-09-21T07:10:54.847198+0000","last_fullsized":"2024-09-21T07:10:54.847198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855459+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:10:54.855459+0000","last_peered":"2024-09-21T07:10:54.855459+0000","last_clean":"2024-09-21T07:10:54.855459+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:10:54.855459+0000","last_undegraded":"2024-09-21T07:10:54.855459+0000","last_fullsized":"2024-09-21T07:10:54.855459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852078+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:10:54.852078+0000","last_peered":"2024-09-21T07:10:54.852078+0000","last_clean":"2024-09-21T07:10:54.852078+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:10:54.852078+0000","last_undegraded":"2024-09-21T07:10:54.852078+0000","last_fullsized":"2024-09-21T07:10:54.852078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858105+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:10:54.858105+0000","last_peered":"2024-09-21T07:10:54.858105+0000","last_clean":"2024-09-21T07:10:54.858105+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:10:54.858105+0000","last_undegraded":"2024-09-21T07:10:54.858105+0000","last_fullsized":"2024-09-21T07:10:54.858105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:05.358727+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:11:05.358727+0000","last_peered":"2024-09-21T07:11:05.358727+0000","last_clean":"2024-09-21T07:11:05.358727+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:11:05.358727+0000","last_undegraded":"2024-09-21T07:11:05.358727+0000","last_fullsized":"2024-09-21T07:11:05.358727+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855905+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:10:54.855905+0000","last_peered":"2024-09-21T07:10:54.855905+0000","last_clean":"2024-09-21T07:10:54.855905+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:10:54.855905+0000","last_undegraded":"2024-09-21T07:10:54.855905+0000","last_fullsized":"2024-09-21T07:10:54.855905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043363+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:10:55.043363+0000","last_peered":"2024-09-21T07:10:55.043363+0000","last_clean":"2024-09-21T07:10:55.043363+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:10:55.043363+0000","last_undegraded":"2024-09-21T07:10:55.043363+0000","last_fullsized":"2024-09-21T07:10:55.043363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858281+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:10:54.858281+0000","last_peered":"2024-09-21T07:10:54.858281+0000","last_clean":"2024-09-21T07:10:54.858281+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:10:54.858281+0000","last_undegraded":"2024-09-21T07:10:54.858281+0000","last_fullsized":"2024-09-21T07:10:54.858281+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851612+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:10:54.851612+0000","last_peered":"2024-09-21T07:10:54.851612+0000","last_clean":"2024-09-21T07:10:54.851612+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:10:54.851612+0000","last_undegraded":"2024-09-21T07:10:54.851612+0000","last_fullsized":"2024-09-21T07:10:54.851612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846903+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:10:54.846903+0000","last_peered":"2024-09-21T07:10:54.846903+0000","last_clean":"2024-09-21T07:10:54.846903+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:10:54.846903+0000","last_undegraded":"2024-09-21T07:10:54.846903+0000","last_fullsized":"2024-09-21T07:10:54.846903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843652+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:10:54.843652+0000","last_peered":"2024-09-21T07:10:54.843652+0000","last_clean":"2024-09-21T07:10:54.843652+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:10:54.843652+0000","last_undegraded":"2024-09-21T07:10:54.843652+0000","last_fullsized":"2024-09-21T07:10:54.843652+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857264+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:10:54.857264+0000","last_peered":"2024-09-21T07:10:54.857264+0000","last_clean":"2024-09-21T07:10:54.857264+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:10:54.857264+0000","last_undegraded":"2024-09-21T07:10:54.857264+0000","last_fullsized":"2024-09-21T07:10:54.857264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847609+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:10:54.847609+0000","last_peered":"2024-09-21T07:10:54.847609+0000","last_clean":"2024-09-21T07:10:54.847609+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:10:54.847609+0000","last_undegraded":"2024-09-21T07:10:54.847609+0000","last_fullsized":"2024-09-21T07:10:54.847609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845174+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:10:54.845174+0000","last_peered":"2024-09-21T07:10:54.845174+0000","last_clean":"2024-09-21T07:10:54.845174+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:10:54.845174+0000","last_undegraded":"2024-09-21T07:10:54.845174+0000","last_fullsized":"2024-09-21T07:10:54.845174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043157+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:10:55.043157+0000","last_peered":"2024-09-21T07:10:55.043157+0000","last_clean":"2024-09-21T07:10:55.043157+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:10:55.043157+0000","last_undegraded":"2024-09-21T07:10:55.043157+0000","last_fullsized":"2024-09-21T07:10:55.043157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847646+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:10:54.847646+0000","last_peered":"2024-09-21T07:10:54.847646+0000","last_clean":"2024-09-21T07:10:54.847646+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:10:54.847646+0000","last_undegraded":"2024-09-21T07:10:54.847646+0000","last_fullsized":"2024-09-21T07:10:54.847646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:04.578734+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:11:04.578734+0000","last_peered":"2024-09-21T07:11:04.578734+0000","last_clean":"2024-09-21T07:11:04.578734+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:11:04.578734+0000","last_undegraded":"2024-09-21T07:11:04.578734+0000","last_fullsized":"2024-09-21T07:11:04.578734+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043137+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:10:55.043137+0000","last_peered":"2024-09-21T07:10:55.043137+0000","last_clean":"2024-09-21T07:10:55.043137+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:10:55.043137+0000","last_undegraded":"2024-09-21T07:10:55.043137+0000","last_fullsized":"2024-09-21T07:10:55.043137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042862+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:10:55.042862+0000","last_peered":"2024-09-21T07:10:55.042862+0000","last_clean":"2024-09-21T07:10:55.042862+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:10:55.042862+0000","last_undegraded":"2024-09-21T07:10:55.042862+0000","last_fullsized":"2024-09-21T07:10:55.042862+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845586+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:10:54.845586+0000","last_peered":"2024-09-21T07:10:54.845586+0000","last_clean":"2024-09-21T07:10:54.845586+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:10:54.845586+0000","last_undegraded":"2024-09-21T07:10:54.845586+0000","last_fullsized":"2024-09-21T07:10:54.845586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848102+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:10:54.848102+0000","last_peered":"2024-09-21T07:10:54.848102+0000","last_clean":"2024-09-21T07:10:54.848102+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:10:54.848102+0000","last_undegraded":"2024-09-21T07:10:54.848102+0000","last_fullsized":"2024-09-21T07:10:54.848102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856634+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:10:54.856634+0000","last_peered":"2024-09-21T07:10:54.856634+0000","last_clean":"2024-09-21T07:10:54.856634+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:10:54.856634+0000","last_undegraded":"2024-09-21T07:10:54.856634+0000","last_fullsized":"2024-09-21T07:10:54.856634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855230+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:10:54.855230+0000","last_peered":"2024-09-21T07:10:54.855230+0000","last_clean":"2024-09-21T07:10:54.855230+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:10:54.855230+0000","last_undegraded":"2024-09-21T07:10:54.855230+0000","last_fullsized":"2024-09-21T07:10:54.855230+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":112,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846411+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:10:54.846411+0000","last_peered":"2024-09-21T07:10:54.846411+0000","last_clean":"2024-09-21T07:10:54.846411+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:10:54.846411+0000","last_undegraded":"2024-09-21T07:10:54.846411+0000","last_fullsized":"2024-09-21T07:10:54.846411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845250+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:10:54.845250+0000","last_peered":"2024-09-21T07:10:54.845250+0000","last_clean":"2024-09-21T07:10:54.845250+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:10:54.845250+0000","last_undegraded":"2024-09-21T07:10:54.845250+0000","last_fullsized":"2024-09-21T07:10:54.845250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846201+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:10:54.846201+0000","last_peered":"2024-09-21T07:10:54.846201+0000","last_clean":"2024-09-21T07:10:54.846201+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:10:54.846201+0000","last_undegraded":"2024-09-21T07:10:54.846201+0000","last_fullsized":"2024-09-21T07:10:54.846201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852850+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:10:54.852850+0000","last_peered":"2024-09-21T07:10:54.852850+0000","last_clean":"2024-09-21T07:10:54.852850+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:10:54.852850+0000","last_undegraded":"2024-09-21T07:10:54.852850+0000","last_fullsized":"2024-09-21T07:10:54.852850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847470+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:10:54.847470+0000","last_peered":"2024-09-21T07:10:54.847470+0000","last_clean":"2024-09-21T07:10:54.847470+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:10:54.847470+0000","last_undegraded":"2024-09-21T07:10:54.847470+0000","last_fullsized":"2024-09-21T07:10:54.847470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845395+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:10:54.845395+0000","last_peered":"2024-09-21T07:10:54.845395+0000","last_clean":"2024-09-21T07:10:54.845395+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:10:54.845395+0000","last_undegraded":"2024-09-21T07:10:54.845395+0000","last_fullsized":"2024-09-21T07:10:54.845395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848748+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:10:54.848748+0000","last_peered":"2024-09-21T07:10:54.848748+0000","last_clean":"2024-09-21T07:10:54.848748+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:10:54.848748+0000","last_undegraded":"2024-09-21T07:10:54.848748+0000","last_fullsized":"2024-09-21T07:10:54.848748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847352+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:10:54.847352+0000","last_peered":"2024-09-21T07:10:54.847352+0000","last_clean":"2024-09-21T07:10:54.847352+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:10:54.847352+0000","last_undegraded":"2024-09-21T07:10:54.847352+0000","last_fullsized":"2024-09-21T07:10:54.847352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845402+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:10:54.845402+0000","last_peered":"2024-09-21T07:10:54.845402+0000","last_clean":"2024-09-21T07:10:54.845402+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:10:54.845402+0000","last_undegraded":"2024-09-21T07:10:54.845402+0000","last_fullsized":"2024-09-21T07:10:54.845402+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043503+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:10:55.043503+0000","last_peered":"2024-09-21T07:10:55.043503+0000","last_clean":"2024-09-21T07:10:55.043503+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:10:55.043503+0000","last_undegraded":"2024-09-21T07:10:55.043503+0000","last_fullsized":"2024-09-21T07:10:55.043503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847367+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:10:54.847367+0000","last_peered":"2024-09-21T07:10:54.847367+0000","last_clean":"2024-09-21T07:10:54.847367+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:10:54.847367+0000","last_undegraded":"2024-09-21T07:10:54.847367+0000","last_fullsized":"2024-09-21T07:10:54.847367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856776+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:10:54.856776+0000","last_peered":"2024-09-21T07:10:54.856776+0000","last_clean":"2024-09-21T07:10:54.856776+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:10:54.856776+0000","last_undegraded":"2024-09-21T07:10:54.856776+0000","last_fullsized":"2024-09-21T07:10:54.856776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.864304+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:10:54.864304+0000","last_peered":"2024-09-21T07:10:54.864304+0000","last_clean":"2024-09-21T07:10:54.864304+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:10:54.864304+0000","last_undegraded":"2024-09-21T07:10:54.864304+0000","last_fullsized":"2024-09-21T07:10:54.864304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847539+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:10:54.847539+0000","last_peered":"2024-09-21T07:10:54.847539+0000","last_clean":"2024-09-21T07:10:54.847539+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:10:54.847539+0000","last_undegraded":"2024-09-21T07:10:54.847539+0000","last_fullsized":"2024-09-21T07:10:54.847539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.861825+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:10:54.861825+0000","last_peered":"2024-09-21T07:10:54.861825+0000","last_clean":"2024-09-21T07:10:54.861825+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:10:54.861825+0000","last_undegraded":"2024-09-21T07:10:54.861825+0000","last_fullsized":"2024-09-21T07:10:54.861825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":56,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856848+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:10:54.856848+0000","last_peered":"2024-09-21T07:10:54.856848+0000","last_clean":"2024-09-21T07:10:54.856848+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:10:54.856848+0000","last_undegraded":"2024-09-21T07:10:54.856848+0000","last_fullsized":"2024-09-21T07:10:54.856848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043501+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:10:55.043501+0000","last_peered":"2024-09-21T07:10:55.043501+0000","last_clean":"2024-09-21T07:10:55.043501+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:10:55.043501+0000","last_undegraded":"2024-09-21T07:10:55.043501+0000","last_fullsized":"2024-09-21T07:10:55.043501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845481+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:10:54.845481+0000","last_peered":"2024-09-21T07:10:54.845481+0000","last_clean":"2024-09-21T07:10:54.845481+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:10:54.845481+0000","last_undegraded":"2024-09-21T07:10:54.845481+0000","last_fullsized":"2024-09-21T07:10:54.845481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:03.577991+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:11:03.577991+0000","last_peered":"2024-09-21T07:11:03.577991+0000","last_clean":"2024-09-21T07:11:03.577991+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:11:03.577991+0000","last_undegraded":"2024-09-21T07:11:03.577991+0000","last_fullsized":"2024-09-21T07:11:03.577991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843898+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:10:54.843898+0000","last_peered":"2024-09-21T07:10:54.843898+0000","last_clean":"2024-09-21T07:10:54.843898+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:10:54.843898+0000","last_undegraded":"2024-09-21T07:10:54.843898+0000","last_fullsized":"2024-09-21T07:10:54.843898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845127+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:10:54.845127+0000","last_peered":"2024-09-21T07:10:54.845127+0000","last_clean":"2024-09-21T07:10:54.845127+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:10:54.845127+0000","last_undegraded":"2024-09-21T07:10:54.845127+0000","last_fullsized":"2024-09-21T07:10:54.845127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848528+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:10:54.848528+0000","last_peered":"2024-09-21T07:10:54.848528+0000","last_clean":"2024-09-21T07:10:54.848528+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:10:54.848528+0000","last_undegraded":"2024-09-21T07:10:54.848528+0000","last_fullsized":"2024-09-21T07:10:54.848528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852002+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:10:54.852002+0000","last_peered":"2024-09-21T07:10:54.852002+0000","last_clean":"2024-09-21T07:10:54.852002+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:10:54.852002+0000","last_undegraded":"2024-09-21T07:10:54.852002+0000","last_fullsized":"2024-09-21T07:10:54.852002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042934+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:10:55.042934+0000","last_peered":"2024-09-21T07:10:55.042934+0000","last_clean":"2024-09-21T07:10:55.042934+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:10:55.042934+0000","last_undegraded":"2024-09-21T07:10:55.042934+0000","last_fullsized":"2024-09-21T07:10:55.042934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852516+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:10:54.852516+0000","last_peered":"2024-09-21T07:10:54.852516+0000","last_clean":"2024-09-21T07:10:54.852516+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:10:54.852516+0000","last_undegraded":"2024-09-21T07:10:54.852516+0000","last_fullsized":"2024-09-21T07:10:54.852516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848450+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:10:54.848450+0000","last_peered":"2024-09-21T07:10:54.848450+0000","last_clean":"2024-09-21T07:10:54.848450+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:10:54.848450+0000","last_undegraded":"2024-09-21T07:10:54.848450+0000","last_fullsized":"2024-09-21T07:10:54.848450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857933+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:10:54.857933+0000","last_peered":"2024-09-21T07:10:54.857933+0000","last_clean":"2024-09-21T07:10:54.857933+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:10:54.857933+0000","last_undegraded":"2024-09-21T07:10:54.857933+0000","last_fullsized":"2024-09-21T07:10:54.857933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'30","reported_seq":112,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:07.585014+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:11:07.585014+0000","last_peered":"2024-09-21T07:11:07.585014+0000","last_clean":"2024-09-21T07:11:07.585014+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:11:07.585014+0000","last_undegraded":"2024-09-21T07:11:07.585014+0000","last_fullsized":"2024-09-21T07:11:07.585014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60809,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":76,"num_write":31,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.862517+0000","last_change":"2024-09-21T07:10:54.861948+0000","last_active":"2024-09-21T07:10:54.862517+0000","last_peered":"2024-09-21T07:10:54.862517+0000","last_clean":"2024-09-21T07:10:54.862517+0000","last_became_active":"2024-09-21T07:10:54.861488+0000","last_became_peered":"2024-09-21T07:10:54.861488+0000","last_unstale":"2024-09-21T07:10:54.862517+0000","last_undegraded":"2024-09-21T07:10:54.862517+0000","last_fullsized":"2024-09-21T07:10:54.862517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:04.439180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848454+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:10:54.848454+0000","last_peered":"2024-09-21T07:10:54.848454+0000","last_clean":"2024-09-21T07:10:54.848454+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:10:54.848454+0000","last_undegraded":"2024-09-21T07:10:54.848454+0000","last_fullsized":"2024-09-21T07:10:54.848454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848684+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:10:54.848684+0000","last_peered":"2024-09-21T07:10:54.848684+0000","last_clean":"2024-09-21T07:10:54.848684+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:10:54.848684+0000","last_undegraded":"2024-09-21T07:10:54.848684+0000","last_fullsized":"2024-09-21T07:10:54.848684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855058+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:10:54.855058+0000","last_peered":"2024-09-21T07:10:54.855058+0000","last_clean":"2024-09-21T07:10:54.855058+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:10:54.855058+0000","last_undegraded":"2024-09-21T07:10:54.855058+0000","last_fullsized":"2024-09-21T07:10:54.855058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847228+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:10:54.847228+0000","last_peered":"2024-09-21T07:10:54.847228+0000","last_clean":"2024-09-21T07:10:54.847228+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:10:54.847228+0000","last_undegraded":"2024-09-21T07:10:54.847228+0000","last_fullsized":"2024-09-21T07:10:54.847228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845213+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:10:54.845213+0000","last_peered":"2024-09-21T07:10:54.845213+0000","last_clean":"2024-09-21T07:10:54.845213+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:10:54.845213+0000","last_undegraded":"2024-09-21T07:10:54.845213+0000","last_fullsized":"2024-09-21T07:10:54.845213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842687+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:10:54.842687+0000","last_peered":"2024-09-21T07:10:54.842687+0000","last_clean":"2024-09-21T07:10:54.842687+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:10:54.842687+0000","last_undegraded":"2024-09-21T07:10:54.842687+0000","last_fullsized":"2024-09-21T07:10:54.842687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848487+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:10:54.848487+0000","last_peered":"2024-09-21T07:10:54.848487+0000","last_clean":"2024-09-21T07:10:54.848487+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:10:54.848487+0000","last_undegraded":"2024-09-21T07:10:54.848487+0000","last_fullsized":"2024-09-21T07:10:54.848487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:06.582694+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:11:06.582694+0000","last_peered":"2024-09-21T07:11:06.582694+0000","last_clean":"2024-09-21T07:11:06.582694+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:11:06.582694+0000","last_undegraded":"2024-09-21T07:11:06.582694+0000","last_fullsized":"2024-09-21T07:11:06.582694+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855623+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:10:54.855623+0000","last_peered":"2024-09-21T07:10:54.855623+0000","last_clean":"2024-09-21T07:10:54.855623+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:10:54.855623+0000","last_undegraded":"2024-09-21T07:10:54.855623+0000","last_fullsized":"2024-09-21T07:10:54.855623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846947+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:10:54.846947+0000","last_peered":"2024-09-21T07:10:54.846947+0000","last_clean":"2024-09-21T07:10:54.846947+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:10:54.846947+0000","last_undegraded":"2024-09-21T07:10:54.846947+0000","last_fullsized":"2024-09-21T07:10:54.846947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842939+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:10:54.842939+0000","last_peered":"2024-09-21T07:10:54.842939+0000","last_clean":"2024-09-21T07:10:54.842939+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:10:54.842939+0000","last_undegraded":"2024-09-21T07:10:54.842939+0000","last_fullsized":"2024-09-21T07:10:54.842939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848215+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:10:54.848215+0000","last_peered":"2024-09-21T07:10:54.848215+0000","last_clean":"2024-09-21T07:10:54.848215+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:10:54.848215+0000","last_undegraded":"2024-09-21T07:10:54.848215+0000","last_fullsized":"2024-09-21T07:10:54.848215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852789+0000","last_change":"2024-09-21T07:10:05.407223+0000","last_active":"2024-09-21T07:10:54.852789+0000","last_peered":"2024-09-21T07:10:54.852789+0000","last_clean":"2024-09-21T07:10:54.852789+0000","last_became_active":"2024-09-21T07:10:05.406718+0000","last_became_peered":"2024-09-21T07:10:05.406718+0000","last_unstale":"2024-09-21T07:10:54.852789+0000","last_undegraded":"2024-09-21T07:10:54.852789+0000","last_fullsized":"2024-09-21T07:10:54.852789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:41.827806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856134+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:10:54.856134+0000","last_peered":"2024-09-21T07:10:54.856134+0000","last_clean":"2024-09-21T07:10:54.856134+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:10:54.856134+0000","last_undegraded":"2024-09-21T07:10:54.856134+0000","last_fullsized":"2024-09-21T07:10:54.856134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043016+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:10:55.043016+0000","last_peered":"2024-09-21T07:10:55.043016+0000","last_clean":"2024-09-21T07:10:55.043016+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:10:55.043016+0000","last_undegraded":"2024-09-21T07:10:55.043016+0000","last_fullsized":"2024-09-21T07:10:55.043016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847711+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:10:54.847711+0000","last_peered":"2024-09-21T07:10:54.847711+0000","last_clean":"2024-09-21T07:10:54.847711+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:10:54.847711+0000","last_undegraded":"2024-09-21T07:10:54.847711+0000","last_fullsized":"2024-09-21T07:10:54.847711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848010+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:10:54.848010+0000","last_peered":"2024-09-21T07:10:54.848010+0000","last_clean":"2024-09-21T07:10:54.848010+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:10:54.848010+0000","last_undegraded":"2024-09-21T07:10:54.848010+0000","last_fullsized":"2024-09-21T07:10:54.848010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850541+0000","last_change":"2024-09-21T07:10:05.426469+0000","last_active":"2024-09-21T07:10:54.850541+0000","last_peered":"2024-09-21T07:10:54.850541+0000","last_clean":"2024-09-21T07:10:54.850541+0000","last_became_active":"2024-09-21T07:10:05.426039+0000","last_became_peered":"2024-09-21T07:10:05.426039+0000","last_unstale":"2024-09-21T07:10:54.850541+0000","last_undegraded":"2024-09-21T07:10:54.850541+0000","last_fullsized":"2024-09-21T07:10:54.850541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:04.579927+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:11:04.579927+0000","last_peered":"2024-09-21T07:11:04.579927+0000","last_clean":"2024-09-21T07:11:04.579927+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:11:04.579927+0000","last_undegraded":"2024-09-21T07:11:04.579927+0000","last_fullsized":"2024-09-21T07:11:04.579927+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857504+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:10:54.857504+0000","last_peered":"2024-09-21T07:10:54.857504+0000","last_clean":"2024-09-21T07:10:54.857504+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:10:54.857504+0000","last_undegraded":"2024-09-21T07:10:54.857504+0000","last_fullsized":"2024-09-21T07:10:54.857504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851084+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:10:54.851084+0000","last_peered":"2024-09-21T07:10:54.851084+0000","last_clean":"2024-09-21T07:10:54.851084+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:10:54.851084+0000","last_undegraded":"2024-09-21T07:10:54.851084+0000","last_fullsized":"2024-09-21T07:10:54.851084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851654+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:10:54.851654+0000","last_peered":"2024-09-21T07:10:54.851654+0000","last_clean":"2024-09-21T07:10:54.851654+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:10:54.851654+0000","last_undegraded":"2024-09-21T07:10:54.851654+0000","last_fullsized":"2024-09-21T07:10:54.851654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.044649+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:10:55.044649+0000","last_peered":"2024-09-21T07:10:55.044649+0000","last_clean":"2024-09-21T07:10:55.044649+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:10:55.044649+0000","last_undegraded":"2024-09-21T07:10:55.044649+0000","last_fullsized":"2024-09-21T07:10:55.044649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850408+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:10:54.850408+0000","last_peered":"2024-09-21T07:10:54.850408+0000","last_clean":"2024-09-21T07:10:54.850408+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:10:54.850408+0000","last_undegraded":"2024-09-21T07:10:54.850408+0000","last_fullsized":"2024-09-21T07:10:54.850408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847360+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:10:54.847360+0000","last_peered":"2024-09-21T07:10:54.847360+0000","last_clean":"2024-09-21T07:10:54.847360+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:10:54.847360+0000","last_undegraded":"2024-09-21T07:10:54.847360+0000","last_fullsized":"2024-09-21T07:10:54.847360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043281+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:10:55.043281+0000","last_peered":"2024-09-21T07:10:55.043281+0000","last_clean":"2024-09-21T07:10:55.043281+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:10:55.043281+0000","last_undegraded":"2024-09-21T07:10:55.043281+0000","last_fullsized":"2024-09-21T07:10:55.043281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852709+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:10:54.852709+0000","last_peered":"2024-09-21T07:10:54.852709+0000","last_clean":"2024-09-21T07:10:54.852709+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:10:54.852709+0000","last_undegraded":"2024-09-21T07:10:54.852709+0000","last_fullsized":"2024-09-21T07:10:54.852709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851607+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:10:54.851607+0000","last_peered":"2024-09-21T07:10:54.851607+0000","last_clean":"2024-09-21T07:10:54.851607+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:10:54.851607+0000","last_undegraded":"2024-09-21T07:10:54.851607+0000","last_fullsized":"2024-09-21T07:10:54.851607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857988+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:10:54.857988+0000","last_peered":"2024-09-21T07:10:54.857988+0000","last_clean":"2024-09-21T07:10:54.857988+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:10:54.857988+0000","last_undegraded":"2024-09-21T07:10:54.857988+0000","last_fullsized":"2024-09-21T07:10:54.857988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858404+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:10:54.858404+0000","last_peered":"2024-09-21T07:10:54.858404+0000","last_clean":"2024-09-21T07:10:54.858404+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:10:54.858404+0000","last_undegraded":"2024-09-21T07:10:54.858404+0000","last_fullsized":"2024-09-21T07:10:54.858404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855648+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:10:54.855648+0000","last_peered":"2024-09-21T07:10:54.855648+0000","last_clean":"2024-09-21T07:10:54.855648+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:10:54.855648+0000","last_undegraded":"2024-09-21T07:10:54.855648+0000","last_fullsized":"2024-09-21T07:10:54.855648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847712+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:10:54.847712+0000","last_peered":"2024-09-21T07:10:54.847712+0000","last_clean":"2024-09-21T07:10:54.847712+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:10:54.847712+0000","last_undegraded":"2024-09-21T07:10:54.847712+0000","last_fullsized":"2024-09-21T07:10:54.847712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043249+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:10:55.043249+0000","last_peered":"2024-09-21T07:10:55.043249+0000","last_clean":"2024-09-21T07:10:55.043249+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:10:55.043249+0000","last_undegraded":"2024-09-21T07:10:55.043249+0000","last_fullsized":"2024-09-21T07:10:55.043249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852135+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:10:54.852135+0000","last_peered":"2024-09-21T07:10:54.852135+0000","last_clean":"2024-09-21T07:10:54.852135+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:10:54.852135+0000","last_undegraded":"2024-09-21T07:10:54.852135+0000","last_fullsized":"2024-09-21T07:10:54.852135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851861+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:10:54.851861+0000","last_peered":"2024-09-21T07:10:54.851861+0000","last_clean":"2024-09-21T07:10:54.851861+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:10:54.851861+0000","last_undegraded":"2024-09-21T07:10:54.851861+0000","last_fullsized":"2024-09-21T07:10:54.851861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843362+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:10:54.843362+0000","last_peered":"2024-09-21T07:10:54.843362+0000","last_clean":"2024-09-21T07:10:54.843362+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:10:54.843362+0000","last_undegraded":"2024-09-21T07:10:54.843362+0000","last_fullsized":"2024-09-21T07:10:54.843362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853323+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:10:54.853323+0000","last_peered":"2024-09-21T07:10:54.853323+0000","last_clean":"2024-09-21T07:10:54.853323+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:10:54.853323+0000","last_undegraded":"2024-09-21T07:10:54.853323+0000","last_fullsized":"2024-09-21T07:10:54.853323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848242+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:10:54.848242+0000","last_peered":"2024-09-21T07:10:54.848242+0000","last_clean":"2024-09-21T07:10:54.848242+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:10:54.848242+0000","last_undegraded":"2024-09-21T07:10:54.848242+0000","last_fullsized":"2024-09-21T07:10:54.848242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.861506+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:10:54.861506+0000","last_peered":"2024-09-21T07:10:54.861506+0000","last_clean":"2024-09-21T07:10:54.861506+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:10:54.861506+0000","last_undegraded":"2024-09-21T07:10:54.861506+0000","last_fullsized":"2024-09-21T07:10:54.861506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043140+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:10:55.043140+0000","last_peered":"2024-09-21T07:10:55.043140+0000","last_clean":"2024-09-21T07:10:55.043140+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:10:55.043140+0000","last_undegraded":"2024-09-21T07:10:55.043140+0000","last_fullsized":"2024-09-21T07:10:55.043140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.860207+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:10:54.860207+0000","last_peered":"2024-09-21T07:10:54.860207+0000","last_clean":"2024-09-21T07:10:54.860207+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:10:54.860207+0000","last_undegraded":"2024-09-21T07:10:54.860207+0000","last_fullsized":"2024-09-21T07:10:54.860207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845743+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:10:54.845743+0000","last_peered":"2024-09-21T07:10:54.845743+0000","last_clean":"2024-09-21T07:10:54.845743+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:10:54.845743+0000","last_undegraded":"2024-09-21T07:10:54.845743+0000","last_fullsized":"2024-09-21T07:10:54.845743+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847513+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:10:54.847513+0000","last_peered":"2024-09-21T07:10:54.847513+0000","last_clean":"2024-09-21T07:10:54.847513+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:10:54.847513+0000","last_undegraded":"2024-09-21T07:10:54.847513+0000","last_fullsized":"2024-09-21T07:10:54.847513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848613+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:10:54.848613+0000","last_peered":"2024-09-21T07:10:54.848613+0000","last_clean":"2024-09-21T07:10:54.848613+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:10:54.848613+0000","last_undegraded":"2024-09-21T07:10:54.848613+0000","last_fullsized":"2024-09-21T07:10:54.848613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856045+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:10:54.856045+0000","last_peered":"2024-09-21T07:10:54.856045+0000","last_clean":"2024-09-21T07:10:54.856045+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:10:54.856045+0000","last_undegraded":"2024-09-21T07:10:54.856045+0000","last_fullsized":"2024-09-21T07:10:54.856045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844872+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:10:54.844872+0000","last_peered":"2024-09-21T07:10:54.844872+0000","last_clean":"2024-09-21T07:10:54.844872+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:10:54.844872+0000","last_undegraded":"2024-09-21T07:10:54.844872+0000","last_fullsized":"2024-09-21T07:10:54.844872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847825+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:10:54.847825+0000","last_peered":"2024-09-21T07:10:54.847825+0000","last_clean":"2024-09-21T07:10:54.847825+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:10:54.847825+0000","last_undegraded":"2024-09-21T07:10:54.847825+0000","last_fullsized":"2024-09-21T07:10:54.847825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857433+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:10:54.857433+0000","last_peered":"2024-09-21T07:10:54.857433+0000","last_clean":"2024-09-21T07:10:54.857433+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:10:54.857433+0000","last_undegraded":"2024-09-21T07:10:54.857433+0000","last_fullsized":"2024-09-21T07:10:54.857433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855117+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:10:54.855117+0000","last_peered":"2024-09-21T07:10:54.855117+0000","last_clean":"2024-09-21T07:10:54.855117+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:10:54.855117+0000","last_undegraded":"2024-09-21T07:10:54.855117+0000","last_fullsized":"2024-09-21T07:10:54.855117+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847084+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:10:54.847084+0000","last_peered":"2024-09-21T07:10:54.847084+0000","last_clean":"2024-09-21T07:10:54.847084+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:10:54.847084+0000","last_undegraded":"2024-09-21T07:10:54.847084+0000","last_fullsized":"2024-09-21T07:10:54.847084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853198+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:10:54.853198+0000","last_peered":"2024-09-21T07:10:54.853198+0000","last_clean":"2024-09-21T07:10:54.853198+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:10:54.853198+0000","last_undegraded":"2024-09-21T07:10:54.853198+0000","last_fullsized":"2024-09-21T07:10:54.853198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64810,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":125,"num_read_kb":128,"num_write":161,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":190165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":496595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1650688,"data_stored":1140776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135910,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":500905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332140,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495663,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1092638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659189,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":491985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855420,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1087835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":497026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":492179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":549789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":548439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":493373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:11:10.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T07:11:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:11.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:10 smithi027 ceph-mon[30779]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T07:11:11.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:10 smithi027 ceph-mon[30779]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:10 smithi153 ceph-mon[34747]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T07:11:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:10 smithi153 ceph-mon[34747]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:10 smithi119 ceph-mon[34519]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T07:11:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:10 smithi119 ceph-mon[34519]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 -- 172.21.15.27:0/3816002289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc078290 msgr2=0x7fccfc114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 --2- 172.21.15.27:0/3816002289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc078290 0x7fccfc114030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009900 tx=0x7fcce402a410 comp rx=0 tx=0).stop 2024-09-21T07:11:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 -- 172.21.15.27:0/3816002289 shutdown_connections 2024-09-21T07:11:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 --2- 172.21.15.27:0/3816002289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc114570 0x7fccfc116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 --2- 172.21.15.27:0/3816002289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc078290 0x7fccfc114030 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 --2- 172.21.15.27:0/3816002289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc077940 0x7fccfc077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 -- 172.21.15.27:0/3816002289 >> 172.21.15.27:0/3816002289 conn(0x7fccfc101680 msgr2=0x7fccfc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 -- 172.21.15.27:0/3816002289 shutdown_connections 2024-09-21T07:11:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.320+0000 7fcd02000640 1 -- 172.21.15.27:0/3816002289 wait complete. 2024-09-21T07:11:11.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.321+0000 7fcd02000640 1 Processor -- start 2024-09-21T07:11:11.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.321+0000 7fcd02000640 1 -- start start 2024-09-21T07:11:11.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd02000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc077940 0x7fccfc1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:11.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd02000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc078290 0x7fccfc1a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:11.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd02000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc114570 0x7fccfc1a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:11.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd02000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc119ed0 con 0x7fccfc114570 2024-09-21T07:11:11.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd02000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc119d50 con 0x7fccfc077940 2024-09-21T07:11:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd02000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc11a050 con 0x7fccfc078290 2024-09-21T07:11:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd01600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc114570 0x7fccfc1a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd01600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc114570 0x7fccfc1a6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39672/0 (socket says 172.21.15.27:39672) 2024-09-21T07:11:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd01600640 1 -- 172.21.15.27:0/1691341041 learned_addr learned my addr 172.21.15.27:0/1691341041 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fcd00c00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc077940 0x7fccfc1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.322+0000 7fccf3e00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc078290 0x7fccfc1a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd00c00640 1 -- 172.21.15.27:0/1691341041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc078290 msgr2=0x7fccfc1a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd00c00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc078290 0x7fccfc1a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd00c00640 1 -- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc114570 msgr2=0x7fccfc1a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd00c00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc114570 0x7fccfc1a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd00c00640 1 -- 172.21.15.27:0/1691341041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc1a6890 con 0x7fccfc077940 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fccf3e00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc078290 0x7fccfc1a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd01600640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc114570 0x7fccfc1a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd00c00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc077940 0x7fccfc1a0920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fccec005fd0 tx=0x7fccec00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fccfa000640 1 -- 172.21.15.27:0/1691341041 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec009c30 con 0x7fccfc077940 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc1a6b80 con 0x7fccfc077940 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fccfa000640 1 -- 172.21.15.27:0/1691341041 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec009dd0 con 0x7fccfc077940 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fccfa000640 1 -- 172.21.15.27:0/1691341041 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec003e20 con 0x7fccfc077940 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.323+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc1ad2e0 con 0x7fccfc077940 2024-09-21T07:11:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.324+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc077d50 con 0x7fccfc077940 2024-09-21T07:11:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.327+0000 7fccfa000640 1 -- 172.21.15.27:0/1691341041 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fccec003fc0 con 0x7fccfc077940 2024-09-21T07:11:11.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.328+0000 7fccfa000640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcce00b65f0 0x7fcce00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:11.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.329+0000 7fccfa000640 1 -- 172.21.15.27:0/1691341041 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fccec0e7480 con 0x7fccfc077940 2024-09-21T07:11:11.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.329+0000 7fccf3e00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcce00b65f0 0x7fcce00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:11.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.329+0000 7fccf3e00640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcce00b65f0 0x7fcce00b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009230 tx=0x7fcce4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:11.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.330+0000 7fccfa000640 1 -- 172.21.15.27:0/1691341041 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fccec017020 con 0x7fccfc077940 2024-09-21T07:11:11.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.526+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fccfc10f760 con 0x7fcce00b65f0 2024-09-21T07:11:11.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.534+0000 7fccfa000640 1 -- 172.21.15.27:0/1691341041 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340348 (secure 0 0 0) 0x7fccfc10f760 con 0x7fcce00b65f0 2024-09-21T07:11:11.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:11.543 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcce00b65f0 msgr2=0x7fcce00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcce00b65f0 0x7fcce00b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009230 tx=0x7fcce4031400 comp rx=0 tx=0).stop 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc077940 msgr2=0x7fccfc1a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc077940 0x7fccfc1a0920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fccec005fd0 tx=0x7fccec00dde0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 shutdown_connections 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc078290 0x7fccfc1a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcce00b65f0 0x7fcce00b8b00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccfc114570 0x7fccfc1a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 --2- 172.21.15.27:0/1691341041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc077940 0x7fccfc1a0920 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 >> 172.21.15.27:0/1691341041 conn(0x7fccfc101680 msgr2=0x7fccfc116660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 shutdown_connections 2024-09-21T07:11:11.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:11.539+0000 7fcd02000640 1 -- 172.21.15.27:0/1691341041 wait complete. 2024-09-21T07:11:12.354 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-21T07:11:11.385297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655412,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":221,"num_read_kb":202,"num_write":286,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387620,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554204,"statfs":{"total":1151940427776,"available":1151543504896,"internally_reserved":0,"allocated":13725696,"data_stored":7892512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30157,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":82,"num_write":23,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003596"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844106+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:10:54.844106+0000","last_peered":"2024-09-21T07:10:54.844106+0000","last_clean":"2024-09-21T07:10:54.844106+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:10:54.844106+0000","last_undegraded":"2024-09-21T07:10:54.844106+0000","last_fullsized":"2024-09-21T07:10:54.844106+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857694+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:10:54.857694+0000","last_peered":"2024-09-21T07:10:54.857694+0000","last_clean":"2024-09-21T07:10:54.857694+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:10:54.857694+0000","last_undegraded":"2024-09-21T07:10:54.857694+0000","last_fullsized":"2024-09-21T07:10:54.857694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850712+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:10:54.850712+0000","last_peered":"2024-09-21T07:10:54.850712+0000","last_clean":"2024-09-21T07:10:54.850712+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:10:54.850712+0000","last_undegraded":"2024-09-21T07:10:54.850712+0000","last_fullsized":"2024-09-21T07:10:54.850712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858585+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:10:54.858585+0000","last_peered":"2024-09-21T07:10:54.858585+0000","last_clean":"2024-09-21T07:10:54.858585+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:10:54.858585+0000","last_undegraded":"2024-09-21T07:10:54.858585+0000","last_fullsized":"2024-09-21T07:10:54.858585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847175+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:10:54.847175+0000","last_peered":"2024-09-21T07:10:54.847175+0000","last_clean":"2024-09-21T07:10:54.847175+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:10:54.847175+0000","last_undegraded":"2024-09-21T07:10:54.847175+0000","last_fullsized":"2024-09-21T07:10:54.847175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854448+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:10:54.854448+0000","last_peered":"2024-09-21T07:10:54.854448+0000","last_clean":"2024-09-21T07:10:54.854448+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:10:54.854448+0000","last_undegraded":"2024-09-21T07:10:54.854448+0000","last_fullsized":"2024-09-21T07:10:54.854448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858193+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:10:54.858193+0000","last_peered":"2024-09-21T07:10:54.858193+0000","last_clean":"2024-09-21T07:10:54.858193+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:10:54.858193+0000","last_undegraded":"2024-09-21T07:10:54.858193+0000","last_fullsized":"2024-09-21T07:10:54.858193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848421+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:10:54.848421+0000","last_peered":"2024-09-21T07:10:54.848421+0000","last_clean":"2024-09-21T07:10:54.848421+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:10:54.848421+0000","last_undegraded":"2024-09-21T07:10:54.848421+0000","last_fullsized":"2024-09-21T07:10:54.848421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854828+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:10:54.854828+0000","last_peered":"2024-09-21T07:10:54.854828+0000","last_clean":"2024-09-21T07:10:54.854828+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:10:54.854828+0000","last_undegraded":"2024-09-21T07:10:54.854828+0000","last_fullsized":"2024-09-21T07:10:54.854828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857732+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:10:54.857732+0000","last_peered":"2024-09-21T07:10:54.857732+0000","last_clean":"2024-09-21T07:10:54.857732+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:10:54.857732+0000","last_undegraded":"2024-09-21T07:10:54.857732+0000","last_fullsized":"2024-09-21T07:10:54.857732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848591+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:10:54.848591+0000","last_peered":"2024-09-21T07:10:54.848591+0000","last_clean":"2024-09-21T07:10:54.848591+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:10:54.848591+0000","last_undegraded":"2024-09-21T07:10:54.848591+0000","last_fullsized":"2024-09-21T07:10:54.848591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854459+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:10:54.854459+0000","last_peered":"2024-09-21T07:10:54.854459+0000","last_clean":"2024-09-21T07:10:54.854459+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:10:54.854459+0000","last_undegraded":"2024-09-21T07:10:54.854459+0000","last_fullsized":"2024-09-21T07:10:54.854459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844977+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:10:54.844977+0000","last_peered":"2024-09-21T07:10:54.844977+0000","last_clean":"2024-09-21T07:10:54.844977+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:10:54.844977+0000","last_undegraded":"2024-09-21T07:10:54.844977+0000","last_fullsized":"2024-09-21T07:10:54.844977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857487+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:10:54.857487+0000","last_peered":"2024-09-21T07:10:54.857487+0000","last_clean":"2024-09-21T07:10:54.857487+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:10:54.857487+0000","last_undegraded":"2024-09-21T07:10:54.857487+0000","last_fullsized":"2024-09-21T07:10:54.857487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855146+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:10:54.855146+0000","last_peered":"2024-09-21T07:10:54.855146+0000","last_clean":"2024-09-21T07:10:54.855146+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:10:54.855146+0000","last_undegraded":"2024-09-21T07:10:54.855146+0000","last_fullsized":"2024-09-21T07:10:54.855146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856487+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:10:54.856487+0000","last_peered":"2024-09-21T07:10:54.856487+0000","last_clean":"2024-09-21T07:10:54.856487+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:10:54.856487+0000","last_undegraded":"2024-09-21T07:10:54.856487+0000","last_fullsized":"2024-09-21T07:10:54.856487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.867034+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:10:54.867034+0000","last_peered":"2024-09-21T07:10:54.867034+0000","last_clean":"2024-09-21T07:10:54.867034+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:10:54.867034+0000","last_undegraded":"2024-09-21T07:10:54.867034+0000","last_fullsized":"2024-09-21T07:10:54.867034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854799+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:10:54.854799+0000","last_peered":"2024-09-21T07:10:54.854799+0000","last_clean":"2024-09-21T07:10:54.854799+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:10:54.854799+0000","last_undegraded":"2024-09-21T07:10:54.854799+0000","last_fullsized":"2024-09-21T07:10:54.854799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851783+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:10:54.851783+0000","last_peered":"2024-09-21T07:10:54.851783+0000","last_clean":"2024-09-21T07:10:54.851783+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:10:54.851783+0000","last_undegraded":"2024-09-21T07:10:54.851783+0000","last_fullsized":"2024-09-21T07:10:54.851783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856047+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:10:54.856047+0000","last_peered":"2024-09-21T07:10:54.856047+0000","last_clean":"2024-09-21T07:10:54.856047+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:10:54.856047+0000","last_undegraded":"2024-09-21T07:10:54.856047+0000","last_fullsized":"2024-09-21T07:10:54.856047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843895+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:10:54.843895+0000","last_peered":"2024-09-21T07:10:54.843895+0000","last_clean":"2024-09-21T07:10:54.843895+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:10:54.843895+0000","last_undegraded":"2024-09-21T07:10:54.843895+0000","last_fullsized":"2024-09-21T07:10:54.843895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857546+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:10:54.857546+0000","last_peered":"2024-09-21T07:10:54.857546+0000","last_clean":"2024-09-21T07:10:54.857546+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:10:54.857546+0000","last_undegraded":"2024-09-21T07:10:54.857546+0000","last_fullsized":"2024-09-21T07:10:54.857546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847552+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:10:54.847552+0000","last_peered":"2024-09-21T07:10:54.847552+0000","last_clean":"2024-09-21T07:10:54.847552+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:10:54.847552+0000","last_undegraded":"2024-09-21T07:10:54.847552+0000","last_fullsized":"2024-09-21T07:10:54.847552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855277+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:10:54.855277+0000","last_peered":"2024-09-21T07:10:54.855277+0000","last_clean":"2024-09-21T07:10:54.855277+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:10:54.855277+0000","last_undegraded":"2024-09-21T07:10:54.855277+0000","last_fullsized":"2024-09-21T07:10:54.855277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847198+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:10:54.847198+0000","last_peered":"2024-09-21T07:10:54.847198+0000","last_clean":"2024-09-21T07:10:54.847198+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:10:54.847198+0000","last_undegraded":"2024-09-21T07:10:54.847198+0000","last_fullsized":"2024-09-21T07:10:54.847198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855459+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:10:54.855459+0000","last_peered":"2024-09-21T07:10:54.855459+0000","last_clean":"2024-09-21T07:10:54.855459+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:10:54.855459+0000","last_undegraded":"2024-09-21T07:10:54.855459+0000","last_fullsized":"2024-09-21T07:10:54.855459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852078+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:10:54.852078+0000","last_peered":"2024-09-21T07:10:54.852078+0000","last_clean":"2024-09-21T07:10:54.852078+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:10:54.852078+0000","last_undegraded":"2024-09-21T07:10:54.852078+0000","last_fullsized":"2024-09-21T07:10:54.852078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858105+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:10:54.858105+0000","last_peered":"2024-09-21T07:10:54.858105+0000","last_clean":"2024-09-21T07:10:54.858105+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:10:54.858105+0000","last_undegraded":"2024-09-21T07:10:54.858105+0000","last_fullsized":"2024-09-21T07:10:54.858105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:05.358727+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:11:05.358727+0000","last_peered":"2024-09-21T07:11:05.358727+0000","last_clean":"2024-09-21T07:11:05.358727+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:11:05.358727+0000","last_undegraded":"2024-09-21T07:11:05.358727+0000","last_fullsized":"2024-09-21T07:11:05.358727+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855905+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:10:54.855905+0000","last_peered":"2024-09-21T07:10:54.855905+0000","last_clean":"2024-09-21T07:10:54.855905+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:10:54.855905+0000","last_undegraded":"2024-09-21T07:10:54.855905+0000","last_fullsized":"2024-09-21T07:10:54.855905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043363+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:10:55.043363+0000","last_peered":"2024-09-21T07:10:55.043363+0000","last_clean":"2024-09-21T07:10:55.043363+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:10:55.043363+0000","last_undegraded":"2024-09-21T07:10:55.043363+0000","last_fullsized":"2024-09-21T07:10:55.043363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858281+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:10:54.858281+0000","last_peered":"2024-09-21T07:10:54.858281+0000","last_clean":"2024-09-21T07:10:54.858281+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:10:54.858281+0000","last_undegraded":"2024-09-21T07:10:54.858281+0000","last_fullsized":"2024-09-21T07:10:54.858281+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851612+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:10:54.851612+0000","last_peered":"2024-09-21T07:10:54.851612+0000","last_clean":"2024-09-21T07:10:54.851612+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:10:54.851612+0000","last_undegraded":"2024-09-21T07:10:54.851612+0000","last_fullsized":"2024-09-21T07:10:54.851612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846903+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:10:54.846903+0000","last_peered":"2024-09-21T07:10:54.846903+0000","last_clean":"2024-09-21T07:10:54.846903+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:10:54.846903+0000","last_undegraded":"2024-09-21T07:10:54.846903+0000","last_fullsized":"2024-09-21T07:10:54.846903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843652+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:10:54.843652+0000","last_peered":"2024-09-21T07:10:54.843652+0000","last_clean":"2024-09-21T07:10:54.843652+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:10:54.843652+0000","last_undegraded":"2024-09-21T07:10:54.843652+0000","last_fullsized":"2024-09-21T07:10:54.843652+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857264+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:10:54.857264+0000","last_peered":"2024-09-21T07:10:54.857264+0000","last_clean":"2024-09-21T07:10:54.857264+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:10:54.857264+0000","last_undegraded":"2024-09-21T07:10:54.857264+0000","last_fullsized":"2024-09-21T07:10:54.857264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847609+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:10:54.847609+0000","last_peered":"2024-09-21T07:10:54.847609+0000","last_clean":"2024-09-21T07:10:54.847609+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:10:54.847609+0000","last_undegraded":"2024-09-21T07:10:54.847609+0000","last_fullsized":"2024-09-21T07:10:54.847609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845174+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:10:54.845174+0000","last_peered":"2024-09-21T07:10:54.845174+0000","last_clean":"2024-09-21T07:10:54.845174+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:10:54.845174+0000","last_undegraded":"2024-09-21T07:10:54.845174+0000","last_fullsized":"2024-09-21T07:10:54.845174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043157+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:10:55.043157+0000","last_peered":"2024-09-21T07:10:55.043157+0000","last_clean":"2024-09-21T07:10:55.043157+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:10:55.043157+0000","last_undegraded":"2024-09-21T07:10:55.043157+0000","last_fullsized":"2024-09-21T07:10:55.043157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847646+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:10:54.847646+0000","last_peered":"2024-09-21T07:10:54.847646+0000","last_clean":"2024-09-21T07:10:54.847646+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:10:54.847646+0000","last_undegraded":"2024-09-21T07:10:54.847646+0000","last_fullsized":"2024-09-21T07:10:54.847646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:09.586712+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:11:09.586712+0000","last_peered":"2024-09-21T07:11:09.586712+0000","last_clean":"2024-09-21T07:11:09.586712+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:11:09.586712+0000","last_undegraded":"2024-09-21T07:11:09.586712+0000","last_fullsized":"2024-09-21T07:11:09.586712+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043137+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:10:55.043137+0000","last_peered":"2024-09-21T07:10:55.043137+0000","last_clean":"2024-09-21T07:10:55.043137+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:10:55.043137+0000","last_undegraded":"2024-09-21T07:10:55.043137+0000","last_fullsized":"2024-09-21T07:10:55.043137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042862+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:10:55.042862+0000","last_peered":"2024-09-21T07:10:55.042862+0000","last_clean":"2024-09-21T07:10:55.042862+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:10:55.042862+0000","last_undegraded":"2024-09-21T07:10:55.042862+0000","last_fullsized":"2024-09-21T07:10:55.042862+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845586+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:10:54.845586+0000","last_peered":"2024-09-21T07:10:54.845586+0000","last_clean":"2024-09-21T07:10:54.845586+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:10:54.845586+0000","last_undegraded":"2024-09-21T07:10:54.845586+0000","last_fullsized":"2024-09-21T07:10:54.845586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848102+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:10:54.848102+0000","last_peered":"2024-09-21T07:10:54.848102+0000","last_clean":"2024-09-21T07:10:54.848102+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:10:54.848102+0000","last_undegraded":"2024-09-21T07:10:54.848102+0000","last_fullsized":"2024-09-21T07:10:54.848102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856634+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:10:54.856634+0000","last_peered":"2024-09-21T07:10:54.856634+0000","last_clean":"2024-09-21T07:10:54.856634+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:10:54.856634+0000","last_undegraded":"2024-09-21T07:10:54.856634+0000","last_fullsized":"2024-09-21T07:10:54.856634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855230+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:10:54.855230+0000","last_peered":"2024-09-21T07:10:54.855230+0000","last_clean":"2024-09-21T07:10:54.855230+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:10:54.855230+0000","last_undegraded":"2024-09-21T07:10:54.855230+0000","last_fullsized":"2024-09-21T07:10:54.855230+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":112,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846411+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:10:54.846411+0000","last_peered":"2024-09-21T07:10:54.846411+0000","last_clean":"2024-09-21T07:10:54.846411+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:10:54.846411+0000","last_undegraded":"2024-09-21T07:10:54.846411+0000","last_fullsized":"2024-09-21T07:10:54.846411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845250+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:10:54.845250+0000","last_peered":"2024-09-21T07:10:54.845250+0000","last_clean":"2024-09-21T07:10:54.845250+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:10:54.845250+0000","last_undegraded":"2024-09-21T07:10:54.845250+0000","last_fullsized":"2024-09-21T07:10:54.845250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846201+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:10:54.846201+0000","last_peered":"2024-09-21T07:10:54.846201+0000","last_clean":"2024-09-21T07:10:54.846201+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:10:54.846201+0000","last_undegraded":"2024-09-21T07:10:54.846201+0000","last_fullsized":"2024-09-21T07:10:54.846201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852850+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:10:54.852850+0000","last_peered":"2024-09-21T07:10:54.852850+0000","last_clean":"2024-09-21T07:10:54.852850+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:10:54.852850+0000","last_undegraded":"2024-09-21T07:10:54.852850+0000","last_fullsized":"2024-09-21T07:10:54.852850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847470+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:10:54.847470+0000","last_peered":"2024-09-21T07:10:54.847470+0000","last_clean":"2024-09-21T07:10:54.847470+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:10:54.847470+0000","last_undegraded":"2024-09-21T07:10:54.847470+0000","last_fullsized":"2024-09-21T07:10:54.847470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845395+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:10:54.845395+0000","last_peered":"2024-09-21T07:10:54.845395+0000","last_clean":"2024-09-21T07:10:54.845395+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:10:54.845395+0000","last_undegraded":"2024-09-21T07:10:54.845395+0000","last_fullsized":"2024-09-21T07:10:54.845395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848748+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:10:54.848748+0000","last_peered":"2024-09-21T07:10:54.848748+0000","last_clean":"2024-09-21T07:10:54.848748+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:10:54.848748+0000","last_undegraded":"2024-09-21T07:10:54.848748+0000","last_fullsized":"2024-09-21T07:10:54.848748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847352+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:10:54.847352+0000","last_peered":"2024-09-21T07:10:54.847352+0000","last_clean":"2024-09-21T07:10:54.847352+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:10:54.847352+0000","last_undegraded":"2024-09-21T07:10:54.847352+0000","last_fullsized":"2024-09-21T07:10:54.847352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845402+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:10:54.845402+0000","last_peered":"2024-09-21T07:10:54.845402+0000","last_clean":"2024-09-21T07:10:54.845402+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:10:54.845402+0000","last_undegraded":"2024-09-21T07:10:54.845402+0000","last_fullsized":"2024-09-21T07:10:54.845402+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043503+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:10:55.043503+0000","last_peered":"2024-09-21T07:10:55.043503+0000","last_clean":"2024-09-21T07:10:55.043503+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:10:55.043503+0000","last_undegraded":"2024-09-21T07:10:55.043503+0000","last_fullsized":"2024-09-21T07:10:55.043503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847367+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:10:54.847367+0000","last_peered":"2024-09-21T07:10:54.847367+0000","last_clean":"2024-09-21T07:10:54.847367+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:10:54.847367+0000","last_undegraded":"2024-09-21T07:10:54.847367+0000","last_fullsized":"2024-09-21T07:10:54.847367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856776+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:10:54.856776+0000","last_peered":"2024-09-21T07:10:54.856776+0000","last_clean":"2024-09-21T07:10:54.856776+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:10:54.856776+0000","last_undegraded":"2024-09-21T07:10:54.856776+0000","last_fullsized":"2024-09-21T07:10:54.856776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.864304+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:10:54.864304+0000","last_peered":"2024-09-21T07:10:54.864304+0000","last_clean":"2024-09-21T07:10:54.864304+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:10:54.864304+0000","last_undegraded":"2024-09-21T07:10:54.864304+0000","last_fullsized":"2024-09-21T07:10:54.864304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847539+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:10:54.847539+0000","last_peered":"2024-09-21T07:10:54.847539+0000","last_clean":"2024-09-21T07:10:54.847539+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:10:54.847539+0000","last_undegraded":"2024-09-21T07:10:54.847539+0000","last_fullsized":"2024-09-21T07:10:54.847539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.861825+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:10:54.861825+0000","last_peered":"2024-09-21T07:10:54.861825+0000","last_clean":"2024-09-21T07:10:54.861825+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:10:54.861825+0000","last_undegraded":"2024-09-21T07:10:54.861825+0000","last_fullsized":"2024-09-21T07:10:54.861825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":56,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856848+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:10:54.856848+0000","last_peered":"2024-09-21T07:10:54.856848+0000","last_clean":"2024-09-21T07:10:54.856848+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:10:54.856848+0000","last_undegraded":"2024-09-21T07:10:54.856848+0000","last_fullsized":"2024-09-21T07:10:54.856848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043501+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:10:55.043501+0000","last_peered":"2024-09-21T07:10:55.043501+0000","last_clean":"2024-09-21T07:10:55.043501+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:10:55.043501+0000","last_undegraded":"2024-09-21T07:10:55.043501+0000","last_fullsized":"2024-09-21T07:10:55.043501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845481+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:10:54.845481+0000","last_peered":"2024-09-21T07:10:54.845481+0000","last_clean":"2024-09-21T07:10:54.845481+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:10:54.845481+0000","last_undegraded":"2024-09-21T07:10:54.845481+0000","last_fullsized":"2024-09-21T07:10:54.845481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":67,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:08.585888+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:11:08.585888+0000","last_peered":"2024-09-21T07:11:08.585888+0000","last_clean":"2024-09-21T07:11:08.585888+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:11:08.585888+0000","last_undegraded":"2024-09-21T07:11:08.585888+0000","last_fullsized":"2024-09-21T07:11:08.585888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843898+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:10:54.843898+0000","last_peered":"2024-09-21T07:10:54.843898+0000","last_clean":"2024-09-21T07:10:54.843898+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:10:54.843898+0000","last_undegraded":"2024-09-21T07:10:54.843898+0000","last_fullsized":"2024-09-21T07:10:54.843898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845127+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:10:54.845127+0000","last_peered":"2024-09-21T07:10:54.845127+0000","last_clean":"2024-09-21T07:10:54.845127+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:10:54.845127+0000","last_undegraded":"2024-09-21T07:10:54.845127+0000","last_fullsized":"2024-09-21T07:10:54.845127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848528+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:10:54.848528+0000","last_peered":"2024-09-21T07:10:54.848528+0000","last_clean":"2024-09-21T07:10:54.848528+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:10:54.848528+0000","last_undegraded":"2024-09-21T07:10:54.848528+0000","last_fullsized":"2024-09-21T07:10:54.848528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852002+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:10:54.852002+0000","last_peered":"2024-09-21T07:10:54.852002+0000","last_clean":"2024-09-21T07:10:54.852002+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:10:54.852002+0000","last_undegraded":"2024-09-21T07:10:54.852002+0000","last_fullsized":"2024-09-21T07:10:54.852002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042934+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:10:55.042934+0000","last_peered":"2024-09-21T07:10:55.042934+0000","last_clean":"2024-09-21T07:10:55.042934+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:10:55.042934+0000","last_undegraded":"2024-09-21T07:10:55.042934+0000","last_fullsized":"2024-09-21T07:10:55.042934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852516+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:10:54.852516+0000","last_peered":"2024-09-21T07:10:54.852516+0000","last_clean":"2024-09-21T07:10:54.852516+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:10:54.852516+0000","last_undegraded":"2024-09-21T07:10:54.852516+0000","last_fullsized":"2024-09-21T07:10:54.852516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848450+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:10:54.848450+0000","last_peered":"2024-09-21T07:10:54.848450+0000","last_clean":"2024-09-21T07:10:54.848450+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:10:54.848450+0000","last_undegraded":"2024-09-21T07:10:54.848450+0000","last_fullsized":"2024-09-21T07:10:54.848450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857933+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:10:54.857933+0000","last_peered":"2024-09-21T07:10:54.857933+0000","last_clean":"2024-09-21T07:10:54.857933+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:10:54.857933+0000","last_undegraded":"2024-09-21T07:10:54.857933+0000","last_fullsized":"2024-09-21T07:10:54.857933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'30","reported_seq":112,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:07.585014+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:11:07.585014+0000","last_peered":"2024-09-21T07:11:07.585014+0000","last_clean":"2024-09-21T07:11:07.585014+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:11:07.585014+0000","last_undegraded":"2024-09-21T07:11:07.585014+0000","last_fullsized":"2024-09-21T07:11:07.585014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60809,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":76,"num_write":31,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.862517+0000","last_change":"2024-09-21T07:10:54.861948+0000","last_active":"2024-09-21T07:10:54.862517+0000","last_peered":"2024-09-21T07:10:54.862517+0000","last_clean":"2024-09-21T07:10:54.862517+0000","last_became_active":"2024-09-21T07:10:54.861488+0000","last_became_peered":"2024-09-21T07:10:54.861488+0000","last_unstale":"2024-09-21T07:10:54.862517+0000","last_undegraded":"2024-09-21T07:10:54.862517+0000","last_fullsized":"2024-09-21T07:10:54.862517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:04.439180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848454+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:10:54.848454+0000","last_peered":"2024-09-21T07:10:54.848454+0000","last_clean":"2024-09-21T07:10:54.848454+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:10:54.848454+0000","last_undegraded":"2024-09-21T07:10:54.848454+0000","last_fullsized":"2024-09-21T07:10:54.848454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848684+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:10:54.848684+0000","last_peered":"2024-09-21T07:10:54.848684+0000","last_clean":"2024-09-21T07:10:54.848684+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:10:54.848684+0000","last_undegraded":"2024-09-21T07:10:54.848684+0000","last_fullsized":"2024-09-21T07:10:54.848684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855058+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:10:54.855058+0000","last_peered":"2024-09-21T07:10:54.855058+0000","last_clean":"2024-09-21T07:10:54.855058+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:10:54.855058+0000","last_undegraded":"2024-09-21T07:10:54.855058+0000","last_fullsized":"2024-09-21T07:10:54.855058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847228+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:10:54.847228+0000","last_peered":"2024-09-21T07:10:54.847228+0000","last_clean":"2024-09-21T07:10:54.847228+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:10:54.847228+0000","last_undegraded":"2024-09-21T07:10:54.847228+0000","last_fullsized":"2024-09-21T07:10:54.847228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845213+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:10:54.845213+0000","last_peered":"2024-09-21T07:10:54.845213+0000","last_clean":"2024-09-21T07:10:54.845213+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:10:54.845213+0000","last_undegraded":"2024-09-21T07:10:54.845213+0000","last_fullsized":"2024-09-21T07:10:54.845213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842687+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:10:54.842687+0000","last_peered":"2024-09-21T07:10:54.842687+0000","last_clean":"2024-09-21T07:10:54.842687+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:10:54.842687+0000","last_undegraded":"2024-09-21T07:10:54.842687+0000","last_fullsized":"2024-09-21T07:10:54.842687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848487+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:10:54.848487+0000","last_peered":"2024-09-21T07:10:54.848487+0000","last_clean":"2024-09-21T07:10:54.848487+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:10:54.848487+0000","last_undegraded":"2024-09-21T07:10:54.848487+0000","last_fullsized":"2024-09-21T07:10:54.848487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:06.582694+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:11:06.582694+0000","last_peered":"2024-09-21T07:11:06.582694+0000","last_clean":"2024-09-21T07:11:06.582694+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:11:06.582694+0000","last_undegraded":"2024-09-21T07:11:06.582694+0000","last_fullsized":"2024-09-21T07:11:06.582694+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855623+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:10:54.855623+0000","last_peered":"2024-09-21T07:10:54.855623+0000","last_clean":"2024-09-21T07:10:54.855623+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:10:54.855623+0000","last_undegraded":"2024-09-21T07:10:54.855623+0000","last_fullsized":"2024-09-21T07:10:54.855623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846947+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:10:54.846947+0000","last_peered":"2024-09-21T07:10:54.846947+0000","last_clean":"2024-09-21T07:10:54.846947+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:10:54.846947+0000","last_undegraded":"2024-09-21T07:10:54.846947+0000","last_fullsized":"2024-09-21T07:10:54.846947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842939+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:10:54.842939+0000","last_peered":"2024-09-21T07:10:54.842939+0000","last_clean":"2024-09-21T07:10:54.842939+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:10:54.842939+0000","last_undegraded":"2024-09-21T07:10:54.842939+0000","last_fullsized":"2024-09-21T07:10:54.842939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848215+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:10:54.848215+0000","last_peered":"2024-09-21T07:10:54.848215+0000","last_clean":"2024-09-21T07:10:54.848215+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:10:54.848215+0000","last_undegraded":"2024-09-21T07:10:54.848215+0000","last_fullsized":"2024-09-21T07:10:54.848215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852789+0000","last_change":"2024-09-21T07:10:05.407223+0000","last_active":"2024-09-21T07:10:54.852789+0000","last_peered":"2024-09-21T07:10:54.852789+0000","last_clean":"2024-09-21T07:10:54.852789+0000","last_became_active":"2024-09-21T07:10:05.406718+0000","last_became_peered":"2024-09-21T07:10:05.406718+0000","last_unstale":"2024-09-21T07:10:54.852789+0000","last_undegraded":"2024-09-21T07:10:54.852789+0000","last_fullsized":"2024-09-21T07:10:54.852789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:41.827806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856134+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:10:54.856134+0000","last_peered":"2024-09-21T07:10:54.856134+0000","last_clean":"2024-09-21T07:10:54.856134+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:10:54.856134+0000","last_undegraded":"2024-09-21T07:10:54.856134+0000","last_fullsized":"2024-09-21T07:10:54.856134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043016+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:10:55.043016+0000","last_peered":"2024-09-21T07:10:55.043016+0000","last_clean":"2024-09-21T07:10:55.043016+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:10:55.043016+0000","last_undegraded":"2024-09-21T07:10:55.043016+0000","last_fullsized":"2024-09-21T07:10:55.043016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847711+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:10:54.847711+0000","last_peered":"2024-09-21T07:10:54.847711+0000","last_clean":"2024-09-21T07:10:54.847711+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:10:54.847711+0000","last_undegraded":"2024-09-21T07:10:54.847711+0000","last_fullsized":"2024-09-21T07:10:54.847711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848010+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:10:54.848010+0000","last_peered":"2024-09-21T07:10:54.848010+0000","last_clean":"2024-09-21T07:10:54.848010+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:10:54.848010+0000","last_undegraded":"2024-09-21T07:10:54.848010+0000","last_fullsized":"2024-09-21T07:10:54.848010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850541+0000","last_change":"2024-09-21T07:10:05.426469+0000","last_active":"2024-09-21T07:10:54.850541+0000","last_peered":"2024-09-21T07:10:54.850541+0000","last_clean":"2024-09-21T07:10:54.850541+0000","last_became_active":"2024-09-21T07:10:05.426039+0000","last_became_peered":"2024-09-21T07:10:05.426039+0000","last_unstale":"2024-09-21T07:10:54.850541+0000","last_undegraded":"2024-09-21T07:10:54.850541+0000","last_fullsized":"2024-09-21T07:10:54.850541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:09.587920+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:11:09.587920+0000","last_peered":"2024-09-21T07:11:09.587920+0000","last_clean":"2024-09-21T07:11:09.587920+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:11:09.587920+0000","last_undegraded":"2024-09-21T07:11:09.587920+0000","last_fullsized":"2024-09-21T07:11:09.587920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857504+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:10:54.857504+0000","last_peered":"2024-09-21T07:10:54.857504+0000","last_clean":"2024-09-21T07:10:54.857504+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:10:54.857504+0000","last_undegraded":"2024-09-21T07:10:54.857504+0000","last_fullsized":"2024-09-21T07:10:54.857504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851084+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:10:54.851084+0000","last_peered":"2024-09-21T07:10:54.851084+0000","last_clean":"2024-09-21T07:10:54.851084+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:10:54.851084+0000","last_undegraded":"2024-09-21T07:10:54.851084+0000","last_fullsized":"2024-09-21T07:10:54.851084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851654+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:10:54.851654+0000","last_peered":"2024-09-21T07:10:54.851654+0000","last_clean":"2024-09-21T07:10:54.851654+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:10:54.851654+0000","last_undegraded":"2024-09-21T07:10:54.851654+0000","last_fullsized":"2024-09-21T07:10:54.851654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.044649+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:10:55.044649+0000","last_peered":"2024-09-21T07:10:55.044649+0000","last_clean":"2024-09-21T07:10:55.044649+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:10:55.044649+0000","last_undegraded":"2024-09-21T07:10:55.044649+0000","last_fullsized":"2024-09-21T07:10:55.044649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850408+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:10:54.850408+0000","last_peered":"2024-09-21T07:10:54.850408+0000","last_clean":"2024-09-21T07:10:54.850408+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:10:54.850408+0000","last_undegraded":"2024-09-21T07:10:54.850408+0000","last_fullsized":"2024-09-21T07:10:54.850408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847360+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:10:54.847360+0000","last_peered":"2024-09-21T07:10:54.847360+0000","last_clean":"2024-09-21T07:10:54.847360+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:10:54.847360+0000","last_undegraded":"2024-09-21T07:10:54.847360+0000","last_fullsized":"2024-09-21T07:10:54.847360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043281+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:10:55.043281+0000","last_peered":"2024-09-21T07:10:55.043281+0000","last_clean":"2024-09-21T07:10:55.043281+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:10:55.043281+0000","last_undegraded":"2024-09-21T07:10:55.043281+0000","last_fullsized":"2024-09-21T07:10:55.043281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852709+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:10:54.852709+0000","last_peered":"2024-09-21T07:10:54.852709+0000","last_clean":"2024-09-21T07:10:54.852709+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:10:54.852709+0000","last_undegraded":"2024-09-21T07:10:54.852709+0000","last_fullsized":"2024-09-21T07:10:54.852709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851607+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:10:54.851607+0000","last_peered":"2024-09-21T07:10:54.851607+0000","last_clean":"2024-09-21T07:10:54.851607+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:10:54.851607+0000","last_undegraded":"2024-09-21T07:10:54.851607+0000","last_fullsized":"2024-09-21T07:10:54.851607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857988+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:10:54.857988+0000","last_peered":"2024-09-21T07:10:54.857988+0000","last_clean":"2024-09-21T07:10:54.857988+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:10:54.857988+0000","last_undegraded":"2024-09-21T07:10:54.857988+0000","last_fullsized":"2024-09-21T07:10:54.857988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858404+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:10:54.858404+0000","last_peered":"2024-09-21T07:10:54.858404+0000","last_clean":"2024-09-21T07:10:54.858404+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:10:54.858404+0000","last_undegraded":"2024-09-21T07:10:54.858404+0000","last_fullsized":"2024-09-21T07:10:54.858404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855648+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:10:54.855648+0000","last_peered":"2024-09-21T07:10:54.855648+0000","last_clean":"2024-09-21T07:10:54.855648+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:10:54.855648+0000","last_undegraded":"2024-09-21T07:10:54.855648+0000","last_fullsized":"2024-09-21T07:10:54.855648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847712+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:10:54.847712+0000","last_peered":"2024-09-21T07:10:54.847712+0000","last_clean":"2024-09-21T07:10:54.847712+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:10:54.847712+0000","last_undegraded":"2024-09-21T07:10:54.847712+0000","last_fullsized":"2024-09-21T07:10:54.847712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043249+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:10:55.043249+0000","last_peered":"2024-09-21T07:10:55.043249+0000","last_clean":"2024-09-21T07:10:55.043249+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:10:55.043249+0000","last_undegraded":"2024-09-21T07:10:55.043249+0000","last_fullsized":"2024-09-21T07:10:55.043249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852135+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:10:54.852135+0000","last_peered":"2024-09-21T07:10:54.852135+0000","last_clean":"2024-09-21T07:10:54.852135+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:10:54.852135+0000","last_undegraded":"2024-09-21T07:10:54.852135+0000","last_fullsized":"2024-09-21T07:10:54.852135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851861+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:10:54.851861+0000","last_peered":"2024-09-21T07:10:54.851861+0000","last_clean":"2024-09-21T07:10:54.851861+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:10:54.851861+0000","last_undegraded":"2024-09-21T07:10:54.851861+0000","last_fullsized":"2024-09-21T07:10:54.851861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843362+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:10:54.843362+0000","last_peered":"2024-09-21T07:10:54.843362+0000","last_clean":"2024-09-21T07:10:54.843362+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:10:54.843362+0000","last_undegraded":"2024-09-21T07:10:54.843362+0000","last_fullsized":"2024-09-21T07:10:54.843362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853323+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:10:54.853323+0000","last_peered":"2024-09-21T07:10:54.853323+0000","last_clean":"2024-09-21T07:10:54.853323+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:10:54.853323+0000","last_undegraded":"2024-09-21T07:10:54.853323+0000","last_fullsized":"2024-09-21T07:10:54.853323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848242+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:10:54.848242+0000","last_peered":"2024-09-21T07:10:54.848242+0000","last_clean":"2024-09-21T07:10:54.848242+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:10:54.848242+0000","last_undegraded":"2024-09-21T07:10:54.848242+0000","last_fullsized":"2024-09-21T07:10:54.848242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.861506+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:10:54.861506+0000","last_peered":"2024-09-21T07:10:54.861506+0000","last_clean":"2024-09-21T07:10:54.861506+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:10:54.861506+0000","last_undegraded":"2024-09-21T07:10:54.861506+0000","last_fullsized":"2024-09-21T07:10:54.861506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043140+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:10:55.043140+0000","last_peered":"2024-09-21T07:10:55.043140+0000","last_clean":"2024-09-21T07:10:55.043140+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:10:55.043140+0000","last_undegraded":"2024-09-21T07:10:55.043140+0000","last_fullsized":"2024-09-21T07:10:55.043140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.860207+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:10:54.860207+0000","last_peered":"2024-09-21T07:10:54.860207+0000","last_clean":"2024-09-21T07:10:54.860207+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:10:54.860207+0000","last_undegraded":"2024-09-21T07:10:54.860207+0000","last_fullsized":"2024-09-21T07:10:54.860207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845743+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:10:54.845743+0000","last_peered":"2024-09-21T07:10:54.845743+0000","last_clean":"2024-09-21T07:10:54.845743+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:10:54.845743+0000","last_undegraded":"2024-09-21T07:10:54.845743+0000","last_fullsized":"2024-09-21T07:10:54.845743+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847513+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:10:54.847513+0000","last_peered":"2024-09-21T07:10:54.847513+0000","last_clean":"2024-09-21T07:10:54.847513+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:10:54.847513+0000","last_undegraded":"2024-09-21T07:10:54.847513+0000","last_fullsized":"2024-09-21T07:10:54.847513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848613+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:10:54.848613+0000","last_peered":"2024-09-21T07:10:54.848613+0000","last_clean":"2024-09-21T07:10:54.848613+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:10:54.848613+0000","last_undegraded":"2024-09-21T07:10:54.848613+0000","last_fullsized":"2024-09-21T07:10:54.848613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856045+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:10:54.856045+0000","last_peered":"2024-09-21T07:10:54.856045+0000","last_clean":"2024-09-21T07:10:54.856045+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:10:54.856045+0000","last_undegraded":"2024-09-21T07:10:54.856045+0000","last_fullsized":"2024-09-21T07:10:54.856045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844872+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:10:54.844872+0000","last_peered":"2024-09-21T07:10:54.844872+0000","last_clean":"2024-09-21T07:10:54.844872+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:10:54.844872+0000","last_undegraded":"2024-09-21T07:10:54.844872+0000","last_fullsized":"2024-09-21T07:10:54.844872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847825+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:10:54.847825+0000","last_peered":"2024-09-21T07:10:54.847825+0000","last_clean":"2024-09-21T07:10:54.847825+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:10:54.847825+0000","last_undegraded":"2024-09-21T07:10:54.847825+0000","last_fullsized":"2024-09-21T07:10:54.847825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857433+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:10:54.857433+0000","last_peered":"2024-09-21T07:10:54.857433+0000","last_clean":"2024-09-21T07:10:54.857433+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:10:54.857433+0000","last_undegraded":"2024-09-21T07:10:54.857433+0000","last_fullsized":"2024-09-21T07:10:54.857433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855117+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:10:54.855117+0000","last_peered":"2024-09-21T07:10:54.855117+0000","last_clean":"2024-09-21T07:10:54.855117+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:10:54.855117+0000","last_undegraded":"2024-09-21T07:10:54.855117+0000","last_fullsized":"2024-09-21T07:10:54.855117+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847084+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:10:54.847084+0000","last_peered":"2024-09-21T07:10:54.847084+0000","last_clean":"2024-09-21T07:10:54.847084+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:10:54.847084+0000","last_undegraded":"2024-09-21T07:10:54.847084+0000","last_fullsized":"2024-09-21T07:10:54.847084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853198+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:10:54.853198+0000","last_peered":"2024-09-21T07:10:54.853198+0000","last_clean":"2024-09-21T07:10:54.853198+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:10:54.853198+0000","last_undegraded":"2024-09-21T07:10:54.853198+0000","last_fullsized":"2024-09-21T07:10:54.853198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64810,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":140,"num_read_kb":133,"num_write":161,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":496595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1650688,"data_stored":1140776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135910,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":500905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332140,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495664,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1092638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659190,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":491985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855421,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1087835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018945,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":497026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":492179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":549789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":552704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":493373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:11:12.358 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T07:11:12.359 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T07:11:12.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T07:11:12.746 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:12 smithi153 ceph-mon[34747]: pgmap v321: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-21T07:11:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:12 smithi153 ceph-mon[34747]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:13.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:12 smithi027 ceph-mon[30779]: pgmap v321: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-21T07:11:13.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:12 smithi027 ceph-mon[30779]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:12 smithi119 ceph-mon[34519]: pgmap v321: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-21T07:11:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:12 smithi119 ceph-mon[34519]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:13.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 -- 172.21.15.27:0/434001274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce4105d80 msgr2=0x7efce4106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 --2- 172.21.15.27:0/434001274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce4105d80 0x7efce4106190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009ca0 tx=0x7efcd802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 -- 172.21.15.27:0/434001274 shutdown_connections 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 --2- 172.21.15.27:0/434001274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce410c010 0x7efce4112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 --2- 172.21.15.27:0/434001274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce410b6c0 0x7efce410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 --2- 172.21.15.27:0/434001274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce4105d80 0x7efce4106190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 -- 172.21.15.27:0/434001274 >> 172.21.15.27:0/434001274 conn(0x7efce41017d0 msgr2=0x7efce4103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.399+0000 7efcec600640 1 -- 172.21.15.27:0/434001274 shutdown_connections 2024-09-21T07:11:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.400+0000 7efcec600640 1 -- 172.21.15.27:0/434001274 wait complete. 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.400+0000 7efcec600640 1 Processor -- start 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efcec600640 1 -- start start 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efcec600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce4105d80 0x7efce419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efcec600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce410b6c0 0x7efce419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efcec600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce410c010 0x7efce41a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efcec600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efce4115d20 con 0x7efce410b6c0 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efcec600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efce4115ba0 con 0x7efce4105d80 2024-09-21T07:11:13.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efcec600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efce4115ea0 con 0x7efce410c010 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efce9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce410b6c0 0x7efce419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efce9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce410b6c0 0x7efce419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39694/0 (socket says 172.21.15.27:39694) 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.401+0000 7efce9200640 1 -- 172.21.15.27:0/3641699100 learned_addr learned my addr 172.21.15.27:0/3641699100 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9c00640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce4105d80 0x7efce419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efcea600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce410c010 0x7efce41a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9c00640 1 -- 172.21.15.27:0/3641699100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce410c010 msgr2=0x7efce41a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9c00640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce410c010 0x7efce41a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9c00640 1 -- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce410b6c0 msgr2=0x7efce419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9c00640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce410b6c0 0x7efce419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9c00640 1 -- 172.21.15.27:0/3641699100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce41a2680 con 0x7efce4105d80 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efcea600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce410c010 0x7efce41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:13.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9200640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce410b6c0 0x7efce419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:13.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.402+0000 7efce9c00640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce4105d80 0x7efce419c710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000f80 tx=0x7efcd8031750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:13.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.403+0000 7efce2000640 1 -- 172.21.15.27:0/3641699100 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd8003b70 con 0x7efce4105d80 2024-09-21T07:11:13.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.403+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce41a2910 con 0x7efce4105d80 2024-09-21T07:11:13.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.403+0000 7efce2000640 1 -- 172.21.15.27:0/3641699100 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd8003d10 con 0x7efce4105d80 2024-09-21T07:11:13.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.403+0000 7efce2000640 1 -- 172.21.15.27:0/3641699100 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd802fa00 con 0x7efce4105d80 2024-09-21T07:11:13.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.403+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce41a90d0 con 0x7efce4105d80 2024-09-21T07:11:13.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.404+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce4106190 con 0x7efce4105d80 2024-09-21T07:11:13.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.407+0000 7efce2000640 1 -- 172.21.15.27:0/3641699100 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7efcd8004580 con 0x7efce4105d80 2024-09-21T07:11:13.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.408+0000 7efce2000640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efccc0b6670 0x7efccc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:13.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.408+0000 7efce2000640 1 -- 172.21.15.27:0/3641699100 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7efcd8108ac0 con 0x7efce4105d80 2024-09-21T07:11:13.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.408+0000 7efce9200640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efccc0b6670 0x7efccc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:13.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.409+0000 7efce9200640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efccc0b6670 0x7efccc0b8b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efce410b030 tx=0x7efcd0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:13.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.410+0000 7efce2000640 1 -- 172.21.15.27:0/3641699100 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efcd80ced60 con 0x7efce4105d80 2024-09-21T07:11:13.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.686+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7efce419d840 con 0x7efce4105d80 2024-09-21T07:11:13.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.694+0000 7efce2000640 1 -- 172.21.15.27:0/3641699100 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7efce419d840 con 0x7efce4105d80 2024-09-21T07:11:13.696 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-21T07:11:13.696 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCRce5mDGwPKRAADLwRROyxVowBm1xrjrKK3g== 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efccc0b6670 msgr2=0x7efccc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efccc0b6670 0x7efccc0b8b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efce410b030 tx=0x7efcd0005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce4105d80 msgr2=0x7efce419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce4105d80 0x7efce419c710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000f80 tx=0x7efcd8031750 comp rx=0 tx=0).stop 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 shutdown_connections 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce410c010 0x7efce41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efccc0b6670 0x7efccc0b8b80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce410b6c0 0x7efce419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 --2- 172.21.15.27:0/3641699100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce4105d80 0x7efce419c710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:13.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.696+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 >> 172.21.15.27:0/3641699100 conn(0x7efce41017d0 msgr2=0x7efce4103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:13.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.697+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 shutdown_connections 2024-09-21T07:11:13.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:13.697+0000 7efcec600640 1 -- 172.21.15.27:0/3641699100 wait complete. 2024-09-21T07:11:14.046 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:13 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3641699100' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T07:11:14.046 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:13 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T07:11:14.046 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:13 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T07:11:14.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:13 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3641699100' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T07:11:14.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:13 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T07:11:14.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:13 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T07:11:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:13 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3641699100' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T07:11:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:13 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T07:11:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:13 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T07:11:14.377 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:11:14.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T07:11:14.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T07:11:14.459 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T07:11:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:15.173 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:14 smithi027 ceph-mon[30779]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-09-21T07:11:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:14 smithi153 ceph-mon[34747]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-09-21T07:11:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:14 smithi119 ceph-mon[34519]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-09-21T07:11:15.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.494+0000 7f011b600640 1 -- 172.21.15.27:0/1733802715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01141049b0 msgr2=0x7f0114104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:15.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.494+0000 7f011b600640 1 --2- 172.21.15.27:0/1733802715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01141049b0 0x7f0114104dc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0108009900 tx=0x7f010802a410 comp rx=0 tx=0).stop 2024-09-21T07:11:15.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.494+0000 7f011b600640 1 -- 172.21.15.27:0/1733802715 shutdown_connections 2024-09-21T07:11:15.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.494+0000 7f011b600640 1 --2- 172.21.15.27:0/1733802715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0114105300 0x7f011410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.495+0000 7f011b600640 1 --2- 172.21.15.27:0/1733802715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011410de10 0x7f0114110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.495+0000 7f011b600640 1 --2- 172.21.15.27:0/1733802715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01141049b0 0x7f0114104dc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.495+0000 7f011b600640 1 -- 172.21.15.27:0/1733802715 >> 172.21.15.27:0/1733802715 conn(0x7f0114079760 msgr2=0x7f0114079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:15.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.495+0000 7f011b600640 1 -- 172.21.15.27:0/1733802715 shutdown_connections 2024-09-21T07:11:15.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.495+0000 7f011b600640 1 -- 172.21.15.27:0/1733802715 wait complete. 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.496+0000 7f011b600640 1 Processor -- start 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.496+0000 7f011b600640 1 -- start start 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f011b600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01141049b0 0x7f011419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f011b600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0114105300 0x7f011419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f011b600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011410de10 0x7f011419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f011b600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0114113ae0 con 0x7f0114105300 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f011b600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0114113960 con 0x7f011410de10 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f011b600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0114113c60 con 0x7f01141049b0 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0113e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0114105300 0x7f011419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0113e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0114105300 0x7f011419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39702/0 (socket says 172.21.15.27:39702) 2024-09-21T07:11:15.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0118c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01141049b0 0x7f011419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0113e00640 1 -- 172.21.15.27:0/3643764372 learned_addr learned my addr 172.21.15.27:0/3643764372 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0119600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011410de10 0x7f011419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0113e00640 1 -- 172.21.15.27:0/3643764372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01141049b0 msgr2=0x7f011419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0113e00640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01141049b0 0x7f011419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0113e00640 1 -- 172.21.15.27:0/3643764372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011410de10 msgr2=0x7f011419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.497+0000 7f0113e00640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011410de10 0x7f011419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f0113e00640 1 -- 172.21.15.27:0/3643764372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01141a03e0 con 0x7f0114105300 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f0118c00640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01141049b0 0x7f011419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f0119600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011410de10 0x7f011419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:15.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f0113e00640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0114105300 0x7f011419aa10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f010000e9c0 tx=0x7f010000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:15.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f0111600640 1 -- 172.21.15.27:0/3643764372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010000ccc0 con 0x7f0114105300 2024-09-21T07:11:15.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f0111600640 1 -- 172.21.15.27:0/3643764372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01000054d0 con 0x7f0114105300 2024-09-21T07:11:15.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f0111600640 1 -- 172.21.15.27:0/3643764372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100010430 con 0x7f0114105300 2024-09-21T07:11:15.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141a06d0 con 0x7f0114105300 2024-09-21T07:11:15.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.498+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a6e30 con 0x7f0114105300 2024-09-21T07:11:15.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.500+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011406c2c0 con 0x7f0114105300 2024-09-21T07:11:15.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.502+0000 7f0111600640 1 -- 172.21.15.27:0/3643764372 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0100005020 con 0x7f0114105300 2024-09-21T07:11:15.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.506+0000 7f0111600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00fc0b6670 0x7f00fc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:15.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.506+0000 7f0111600640 1 -- 172.21.15.27:0/3643764372 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f01000e7c70 con 0x7f0114105300 2024-09-21T07:11:15.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.506+0000 7f0118c00640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00fc0b6670 0x7f00fc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:15.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.507+0000 7f0118c00640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00fc0b6670 0x7f00fc0b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0108005ab0 tx=0x7f0108005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:15.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.510+0000 7f0111600640 1 -- 172.21.15.27:0/3643764372 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01000adf10 con 0x7f0114105300 2024-09-21T07:11:15.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.710+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f011419b5a0 con 0x7f0114105300 2024-09-21T07:11:15.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.711+0000 7f0111600640 1 -- 172.21.15.27:0/3643764372 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v57) ==== 86+0+133 (secure 0 0 0) 0x7f01000b22f0 con 0x7f0114105300 2024-09-21T07:11:15.713 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:15.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00fc0b6670 msgr2=0x7f00fc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:15.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00fc0b6670 0x7f00fc0b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0108005ab0 tx=0x7f0108005a20 comp rx=0 tx=0).stop 2024-09-21T07:11:15.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0114105300 msgr2=0x7f011419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:15.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0114105300 0x7f011419aa10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f010000e9c0 tx=0x7f010000ee80 comp rx=0 tx=0).stop 2024-09-21T07:11:15.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 shutdown_connections 2024-09-21T07:11:15.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00fc0b6670 0x7f00fc0b8b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f011410de10 0x7f011419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0114105300 0x7f011419aa10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 --2- 172.21.15.27:0/3643764372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01141049b0 0x7f011419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:15.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 >> 172.21.15.27:0/3643764372 conn(0x7f0114079760 msgr2=0x7f0114100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:15.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 shutdown_connections 2024-09-21T07:11:15.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:15.714+0000 7f011b600640 1 -- 172.21.15.27:0/3643764372 wait complete. 2024-09-21T07:11:16.064 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:15 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3643764372' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T07:11:16.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:15 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3643764372' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T07:11:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:15 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3643764372' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T07:11:16.537 INFO:teuthology.orchestra.run.smithi027.stdout:[{"version":57,"timestamp":"2024-09-21T07:09:43.360451+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T07:11:16.537 INFO:tasks.ceph_manager:config epoch is 57 2024-09-21T07:11:16.538 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T07:11:16.538 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T07:11:16.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T07:11:16.923 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:16 smithi153 ceph-mon[34747]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-21T07:11:17.245 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:16 smithi027 ceph-mon[30779]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-21T07:11:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:16 smithi119 ceph-mon[34519]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-21T07:11:17.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 -- 172.21.15.27:0/3193651448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394103990 msgr2=0x7fa394103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:17.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 --2- 172.21.15.27:0/3193651448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394103990 0x7fa394103da0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa384009900 tx=0x7fa38402a410 comp rx=0 tx=0).stop 2024-09-21T07:11:17.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 -- 172.21.15.27:0/3193651448 shutdown_connections 2024-09-21T07:11:17.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 --2- 172.21.15.27:0/3193651448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3941092d0 0x7fa3941096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 --2- 172.21.15.27:0/3193651448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa394109c20 0x7fa394110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 --2- 172.21.15.27:0/3193651448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394103990 0x7fa394103da0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 -- 172.21.15.27:0/3193651448 >> 172.21.15.27:0/3193651448 conn(0x7fa3940ff3e0 msgr2=0x7fa394101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:17.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 -- 172.21.15.27:0/3193651448 shutdown_connections 2024-09-21T07:11:17.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.560+0000 7fa393200640 1 -- 172.21.15.27:0/3193651448 wait complete. 2024-09-21T07:11:17.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.562+0000 7fa393200640 1 Processor -- start 2024-09-21T07:11:17.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.562+0000 7fa393200640 1 -- start start 2024-09-21T07:11:17.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.562+0000 7fa393200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa394103990 0x7fa39419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:17.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.562+0000 7fa393200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3941092d0 0x7fa39419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa393200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394109c20 0x7fa3941a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa393200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941138e0 con 0x7fa394109c20 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa393200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa394113760 con 0x7fa3941092d0 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa393200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa394113a60 con 0x7fa394103990 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa394103990 0x7fa39419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa394103990 0x7fa39419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45902/0 (socket says 172.21.15.27:45902) 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 -- 172.21.15.27:0/165925447 learned_addr learned my addr 172.21.15.27:0/165925447 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa392800640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394109c20 0x7fa3941a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa38be00640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3941092d0 0x7fa39419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 -- 172.21.15.27:0/165925447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3941092d0 msgr2=0x7fa39419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3941092d0 0x7fa39419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 -- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394109c20 msgr2=0x7fa3941a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:17.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394109c20 0x7fa3941a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.563+0000 7fa391e00640 1 -- 172.21.15.27:0/165925447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941a24c0 con 0x7fa394103990 2024-09-21T07:11:17.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.564+0000 7fa391e00640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa394103990 0x7fa39419c550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa384005940 tx=0x7fa3840042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:17.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.564+0000 7fa38aa00640 1 -- 172.21.15.27:0/165925447 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38403f030 con 0x7fa394103990 2024-09-21T07:11:17.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.564+0000 7fa38aa00640 1 -- 172.21.15.27:0/165925447 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa384003cb0 con 0x7fa394103990 2024-09-21T07:11:17.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.564+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3941a2750 con 0x7fa394103990 2024-09-21T07:11:17.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.564+0000 7fa38aa00640 1 -- 172.21.15.27:0/165925447 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384035070 con 0x7fa394103990 2024-09-21T07:11:17.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.564+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39410a2e0 con 0x7fa394103990 2024-09-21T07:11:17.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.566+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa394104c10 con 0x7fa394103990 2024-09-21T07:11:17.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.568+0000 7fa38aa00640 1 -- 172.21.15.27:0/165925447 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa384042030 con 0x7fa394103990 2024-09-21T07:11:17.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.570+0000 7fa38aa00640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3780b65f0 0x7fa3780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:17.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.570+0000 7fa38aa00640 1 -- 172.21.15.27:0/165925447 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fa3841079e0 con 0x7fa394103990 2024-09-21T07:11:17.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.570+0000 7fa38be00640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3780b65f0 0x7fa3780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:17.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.571+0000 7fa38be00640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3780b65f0 0x7fa3780b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa394105a70 tx=0x7fa37c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.574+0000 7fa38aa00640 1 -- 172.21.15.27:0/165925447 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3840d2060 con 0x7fa394103990 2024-09-21T07:11:17.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.828+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa39410a570 con 0x7fa394103990 2024-09-21T07:11:17.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.834+0000 7fa38aa00640 1 -- 172.21.15.27:0/165925447 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v22) ==== 74+0+296453 (secure 0 0 0) 0x7fa3840d6fa0 con 0x7fa394103990 2024-09-21T07:11:17.841 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3780b65f0 msgr2=0x7fa3780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3780b65f0 0x7fa3780b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa394105a70 tx=0x7fa37c00b390 comp rx=0 tx=0).stop 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa394103990 msgr2=0x7fa39419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa394103990 0x7fa39419c550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa384005940 tx=0x7fa3840042f0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 shutdown_connections 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3780b65f0 0x7fa3780b8b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa394109c20 0x7fa3941a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3941092d0 0x7fa39419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 --2- 172.21.15.27:0/165925447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa394103990 0x7fa39419c550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:17.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 >> 172.21.15.27:0/165925447 conn(0x7fa3940ff3e0 msgr2=0x7fa3940755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:17.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 shutdown_connections 2024-09-21T07:11:17.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:17.839+0000 7fa393200640 1 -- 172.21.15.27:0/165925447 wait complete. 2024-09-21T07:11:18.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:17 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/165925447' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T07:11:18.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:17 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/165925447' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T07:11:18.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:17 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/165925447' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T07:11:18.624 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"flags":0,"active_gid":14226,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3854502753},{"type":"v1","addr":"172.21.15.27:6801","nonce":3854502753}]},"active_addr":"172.21.15.27:6801/3854502753","active_change":"2024-09-21T07:02:53.267424+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24160,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24191,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2037380229}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":476022973}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":1400087737}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3140690905}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2501711618}]}]} 2024-09-21T07:11:18.632 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T07:11:18.632 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T07:11:18.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:11:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:19.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:18 smithi153 ceph-mon[34747]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:11:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:18 smithi119 ceph-mon[34519]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:11:19.304 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:18 smithi027 ceph-mon[30779]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 -- 172.21.15.27:0/4174444898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c8105d80 msgr2=0x7fe0c8106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4174444898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c8105d80 0x7fe0c8106190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009900 tx=0x7fe0b802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 -- 172.21.15.27:0/4174444898 shutdown_connections 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4174444898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 0x7fe0c810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4174444898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c810c010 0x7fe0c8112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4174444898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c8105d80 0x7fe0c8106190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 -- 172.21.15.27:0/4174444898 >> 172.21.15.27:0/4174444898 conn(0x7fe0c8101810 msgr2=0x7fe0c8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 -- 172.21.15.27:0/4174444898 shutdown_connections 2024-09-21T07:11:19.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.625+0000 7fe0cf800640 1 -- 172.21.15.27:0/4174444898 wait complete. 2024-09-21T07:11:19.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.626+0000 7fe0cf800640 1 Processor -- start 2024-09-21T07:11:19.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.626+0000 7fe0cf800640 1 -- start start 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.626+0000 7fe0cf800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c8105d80 0x7fe0c819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cf800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 0x7fe0c819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cf800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c810c010 0x7fe0c81a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cf800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8115cd0 con 0x7fe0c810c010 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cf800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8115b50 con 0x7fe0c8105d80 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cf800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8115e50 con 0x7fe0c810b6c0 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c8105d80 0x7fe0c819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 0x7fe0c819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cd800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c810c010 0x7fe0c81a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c8105d80 0x7fe0c819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45636/0 (socket says 172.21.15.27:45636) 2024-09-21T07:11:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 0x7fe0c819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:44464/0 (socket says 172.21.15.27:44464) 2024-09-21T07:11:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cce00640 1 -- 172.21.15.27:0/4019845972 learned_addr learned my addr 172.21.15.27:0/4019845972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cd800640 1 -- 172.21.15.27:0/4019845972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 msgr2=0x7fe0c819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cd800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 0x7fe0c819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cd800640 1 -- 172.21.15.27:0/4019845972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c8105d80 msgr2=0x7fe0c819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cd800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c8105d80 0x7fe0c819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.627+0000 7fe0cd800640 1 -- 172.21.15.27:0/4019845972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c81a26d0 con 0x7fe0c810c010 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0bfe00640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 0x7fe0c819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0cce00640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c8105d80 0x7fe0c819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0cd800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c810c010 0x7fe0c81a1fa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009fd0 tx=0x7fe0c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0c6000640 1 -- 172.21.15.27:0/4019845972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000ce20 con 0x7fe0c810c010 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0c6000640 1 -- 172.21.15.27:0/4019845972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c0005020 con 0x7fe0c810c010 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c81a29c0 con 0x7fe0c810c010 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0c6000640 1 -- 172.21.15.27:0/4019845972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0010690 con 0x7fe0c810c010 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.628+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c81a9120 con 0x7fe0c810c010 2024-09-21T07:11:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.630+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c8106190 con 0x7fe0c810c010 2024-09-21T07:11:19.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.632+0000 7fe0c6000640 1 -- 172.21.15.27:0/4019845972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe0c00054d0 con 0x7fe0c810c010 2024-09-21T07:11:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.633+0000 7fe0c6000640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.633+0000 7fe0c6000640 1 -- 172.21.15.27:0/4019845972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fe0c00e7fe0 con 0x7fe0c810c010 2024-09-21T07:11:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.634+0000 7fe0cce00640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:19.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.634+0000 7fe0cce00640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8031440 tx=0x7fe0b80058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:19.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.638+0000 7fe0c6000640 1 -- 172.21.15.27:0/4019845972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0c00ae2e0 con 0x7fe0c810c010 2024-09-21T07:11:19.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.842+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0c810d850 con 0x7fe0c810c010 2024-09-21T07:11:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.843+0000 7fe0c6000640 1 -- 172.21.15.27:0/4019845972 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+25122 (secure 0 0 0) 0x7fe0c00b26c0 con 0x7fe0c810c010 2024-09-21T07:11:19.846 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:54.838649+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.846+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0ac0b65f0 msgr2=0x7fe0ac0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.846+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8031440 tx=0x7fe0b80058d0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.846+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c810c010 msgr2=0x7fe0c81a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.846+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c810c010 0x7fe0c81a1fa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009fd0 tx=0x7fe0c000ef10 comp rx=0 tx=0).stop 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 shutdown_connections 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c810b6c0 0x7fe0c819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0c810c010 0x7fe0c81a1fa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 --2- 172.21.15.27:0/4019845972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c8105d80 0x7fe0c819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 >> 172.21.15.27:0/4019845972 conn(0x7fe0c8101810 msgr2=0x7fe0c8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 shutdown_connections 2024-09-21T07:11:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:19.847+0000 7fe0cf800640 1 -- 172.21.15.27:0/4019845972 wait complete. 2024-09-21T07:11:20.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:19 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4019845972' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:20.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:19 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4019845972' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:20.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:19 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4019845972' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:20.571 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T07:11:20.571 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:11:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:21.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:20 smithi027 ceph-mon[30779]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:11:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:20 smithi119 ceph-mon[34519]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:11:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:20 smithi153 ceph-mon[34747]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:11:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 -- 172.21.15.27:0/1830744657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c110440 msgr2=0x7f3c1c116c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 --2- 172.21.15.27:0/1830744657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c110440 0x7f3c1c116c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009ca0 tx=0x7f3c1802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 -- 172.21.15.27:0/1830744657 shutdown_connections 2024-09-21T07:11:21.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 --2- 172.21.15.27:0/1830744657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c10faf0 0x7f3c1c10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 --2- 172.21.15.27:0/1830744657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c110440 0x7f3c1c116c70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 --2- 172.21.15.27:0/1830744657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c1c10a1b0 0x7f3c1c10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 -- 172.21.15.27:0/1830744657 >> 172.21.15.27:0/1830744657 conn(0x7f3c1c0780e0 msgr2=0x7f3c1c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:21.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 -- 172.21.15.27:0/1830744657 shutdown_connections 2024-09-21T07:11:21.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.684+0000 7f3c24800640 1 -- 172.21.15.27:0/1830744657 wait complete. 2024-09-21T07:11:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.685+0000 7f3c24800640 1 Processor -- start 2024-09-21T07:11:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.685+0000 7f3c24800640 1 -- start start 2024-09-21T07:11:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c24800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c10a1b0 0x7f3c1c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c24800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c10faf0 0x7f3c1c1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c24800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c1c110440 0x7f3c1c1a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c24800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c11a160 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c24800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c119fe0 con 0x7f3c1c110440 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c24800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c11a2e0 con 0x7f3c1c10faf0 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c21e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c10a1b0 0x7f3c1c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c21e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c10a1b0 0x7f3c1c1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36750/0 (socket says 172.21.15.27:36750) 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c21e00640 1 -- 172.21.15.27:0/3831566258 learned_addr learned my addr 172.21.15.27:0/3831566258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c22800640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c1c110440 0x7f3c1c1a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.686+0000 7f3c21400640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c10faf0 0x7f3c1c1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.687+0000 7f3c21e00640 1 -- 172.21.15.27:0/3831566258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c10faf0 msgr2=0x7f3c1c1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.687+0000 7f3c21e00640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c10faf0 0x7f3c1c1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.687+0000 7f3c21e00640 1 -- 172.21.15.27:0/3831566258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c1c110440 msgr2=0x7f3c1c1a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.687+0000 7f3c21e00640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c1c110440 0x7f3c1c1a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.687+0000 7f3c21e00640 1 -- 172.21.15.27:0/3831566258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c1a69c0 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.687+0000 7f3c21e00640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c10a1b0 0x7f3c1c1a0a50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e9f0 tx=0x7f3c1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.687+0000 7f3c16a00640 1 -- 172.21.15.27:0/3831566258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000cd80 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.688+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c1a6cb0 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.688+0000 7f3c16a00640 1 -- 172.21.15.27:0/3831566258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c100108e0 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.688+0000 7f3c16a00640 1 -- 172.21.15.27:0/3831566258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10018980 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.688+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1ad410 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.689+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c10a5c0 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.691+0000 7f3c16a00640 1 -- 172.21.15.27:0/3831566258 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3c10010430 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.693+0000 7f3c16a00640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c040b65f0 0x7f3c040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:21.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.693+0000 7f3c16a00640 1 -- 172.21.15.27:0/3831566258 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f3c100e7c60 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.693+0000 7f3c21400640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c040b65f0 0x7f3c040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:21.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.694+0000 7f3c21400640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c040b65f0 0x7f3c040b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c07a300 tx=0x7f3c0c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:21.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.697+0000 7f3c16a00640 1 -- 172.21.15.27:0/3831566258 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c100b2340 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.897+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c1c111ca0 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.898+0000 7f3c16a00640 1 -- 172.21.15.27:0/3831566258 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+25122 (secure 0 0 0) 0x7f3c100b7280 con 0x7f3c1c10a1b0 2024-09-21T07:11:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:21.901 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:10:54.838649+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:11:21.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c040b65f0 msgr2=0x7f3c040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:21.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c040b65f0 0x7f3c040b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c07a300 tx=0x7f3c0c005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:21.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c10a1b0 msgr2=0x7f3c1c1a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:21.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c10a1b0 0x7f3c1c1a0a50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e9f0 tx=0x7f3c1000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 shutdown_connections 2024-09-21T07:11:21.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1c10faf0 0x7f3c1c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c040b65f0 0x7f3c040b8b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c1c110440 0x7f3c1c1a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 --2- 172.21.15.27:0/3831566258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c10a1b0 0x7f3c1c1a0a50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:21.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 >> 172.21.15.27:0/3831566258 conn(0x7f3c1c0780e0 msgr2=0x7f3c1c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:21.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 shutdown_connections 2024-09-21T07:11:21.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:21.902+0000 7f3c24800640 1 -- 172.21.15.27:0/3831566258 wait complete. 2024-09-21T07:11:22.246 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:21 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3831566258' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:21 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3831566258' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:21 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3831566258' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:11:22.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T07:11:22.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T07:11:22.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T07:11:22.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T07:11:22.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T07:11:22.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T07:11:22.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T07:11:22.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T07:11:22.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T07:11:22.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T07:11:22.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T07:11:22.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T07:11:23.242 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:22 smithi027 ceph-mon[30779]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T07:11:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:22 smithi119 ceph-mon[34519]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T07:11:23.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:22 smithi153 ceph-mon[34747]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T07:11:23.599 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.049 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.267 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.268 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.286 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.315 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:24.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 -- 172.21.15.27:0/3951154332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec0757b0 msgr2=0x7f55ec075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:24.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 --2- 172.21.15.27:0/3951154332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec0757b0 0x7f55ec075bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a400 comp rx=0 tx=0).stop 2024-09-21T07:11:24.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 -- 172.21.15.27:0/3951154332 shutdown_connections 2024-09-21T07:11:24.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 --2- 172.21.15.27:0/3951154332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec076100 0x7f55ec10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:24.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 --2- 172.21.15.27:0/3951154332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55ec10ff10 0x7f55ec112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:24.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 --2- 172.21.15.27:0/3951154332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec0757b0 0x7f55ec075bc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:24.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 -- 172.21.15.27:0/3951154332 >> 172.21.15.27:0/3951154332 conn(0x7f55ec0ff440 msgr2=0x7f55ec101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:24.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.371+0000 7f55f0a00640 1 -- 172.21.15.27:0/3951154332 shutdown_connections 2024-09-21T07:11:24.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.372+0000 7f55f0a00640 1 -- 172.21.15.27:0/3951154332 wait complete. 2024-09-21T07:11:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 Processor -- start 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 -- start start 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec0757b0 0x7f55ec074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55ec076100 0x7f55ec06e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec10ff10 0x7f55ec06e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec115b10 con 0x7f55ec0757b0 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec115990 con 0x7f55ec076100 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.373+0000 7f55f0a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec115c90 con 0x7f55ec10ff10 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec10ff10 0x7f55ec06e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55eaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55ec076100 0x7f55ec06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55eb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec0757b0 0x7f55ec074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec10ff10 0x7f55ec06e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:44492/0 (socket says 172.21.15.27:44492) 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55eaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55ec076100 0x7f55ec06e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45674/0 (socket says 172.21.15.27:45674) 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 -- 172.21.15.27:0/3734648982 learned_addr learned my addr 172.21.15.27:0/3734648982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 -- 172.21.15.27:0/3734648982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55ec076100 msgr2=0x7f55ec06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55ec076100 0x7f55ec06e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 -- 172.21.15.27:0/3734648982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec0757b0 msgr2=0x7f55ec074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec0757b0 0x7f55ec074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.374+0000 7f55ebe00640 1 -- 172.21.15.27:0/3734648982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec06efa0 con 0x7f55ec10ff10 2024-09-21T07:11:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.375+0000 7f55ebe00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec10ff10 0x7f55ec06e780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55e400eb10 tx=0x7f55e400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.375+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4004b20 con 0x7f55ec10ff10 2024-09-21T07:11:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.375+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec073c50 con 0x7f55ec10ff10 2024-09-21T07:11:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.375+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e4004cc0 con 0x7f55ec10ff10 2024-09-21T07:11:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.375+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4010690 con 0x7f55ec10ff10 2024-09-21T07:11:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.375+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec074160 con 0x7f55ec10ff10 2024-09-21T07:11:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.376+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55ec074730 con 0x7f55ec10ff10 2024-09-21T07:11:24.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.379+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f55e4005000 con 0x7f55ec10ff10 2024-09-21T07:11:24.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.381+0000 7f55e3e00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f55d00b6670 0x7f55d00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:24.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.381+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f55e40e81c0 con 0x7f55ec10ff10 2024-09-21T07:11:24.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.381+0000 7f55e3e00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] conn(0x7f55d00c1a00 0x7f55d00c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:24.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.381+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 --> [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55e4022e50 con 0x7f55d00c1a00 2024-09-21T07:11:24.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.381+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f55e40e8600 con 0x7f55ec10ff10 2024-09-21T07:11:24.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.381+0000 7f55eb400640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f55d00b6670 0x7f55d00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:24.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.382+0000 7f55eaa00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] conn(0x7f55d00c1a00 0x7f55d00c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:24.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.382+0000 7f55eb400640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f55d00b6670 0x7f55d00b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f55dc005ab0 tx=0x7f55dc005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:24.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.382+0000 7f55eaa00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] conn(0x7f55d00c1a00 0x7f55d00c3ef0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:24.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.384+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== osd.1 v2:172.21.15.119:6800/3407674054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55e4022e50 con 0x7f55d00c1a00 2024-09-21T07:11:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.399+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 --> [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55ec076cc0 con 0x7f55d00c1a00 2024-09-21T07:11:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.400+0000 7f55e3e00640 1 -- 172.21.15.27:0/3734648982 <== osd.1 v2:172.21.15.119:6800/3407674054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f55ec076cc0 con 0x7f55d00c1a00 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.400+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 >> [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] conn(0x7f55d00c1a00 msgr2=0x7f55d00c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.400+0000 7f55f0a00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] conn(0x7f55d00c1a00 0x7f55d00c3ef0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f55d00b6670 msgr2=0x7f55d00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f55d00b6670 0x7f55d00b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f55dc005ab0 tx=0x7f55dc005a20 comp rx=0 tx=0).stop 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec10ff10 msgr2=0x7f55ec06e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 --2- 172.21.15.27:0/3734648982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec10ff10 0x7f55ec06e780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55e400eb10 tx=0x7f55e400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55ebe00640 1 -- 172.21.15.27:0/3734648982 reap_dead start 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 shutdown_connections 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 >> 172.21.15.27:0/3734648982 conn(0x7f55ec0ff440 msgr2=0x7f55ec109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 shutdown_connections 2024-09-21T07:11:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:24.401+0000 7f55f0a00640 1 -- 172.21.15.27:0/3734648982 wait complete. 2024-09-21T07:11:25.252 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:24 smithi027 ceph-mon[30779]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-21T07:11:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:24 smithi119 ceph-mon[34519]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-21T07:11:25.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:24 smithi153 ceph-mon[34747]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-21T07:11:25.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 -- 172.21.15.27:0/1701844964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581092f0 msgr2=0x7fc258109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 --2- 172.21.15.27:0/1701844964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581092f0 0x7fc258109700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc244009900 tx=0x7fc24402a410 comp rx=0 tx=0).stop 2024-09-21T07:11:25.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 -- 172.21.15.27:0/1701844964 shutdown_connections 2024-09-21T07:11:25.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 --2- 172.21.15.27:0/1701844964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc258109c40 0x7fc258110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 --2- 172.21.15.27:0/1701844964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581092f0 0x7fc258109700 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 --2- 172.21.15.27:0/1701844964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2581039b0 0x7fc258103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 -- 172.21.15.27:0/1701844964 >> 172.21.15.27:0/1701844964 conn(0x7fc2580ff440 msgr2=0x7fc258101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 -- 172.21.15.27:0/1701844964 shutdown_connections 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.581+0000 7fc25dc00640 1 -- 172.21.15.27:0/1701844964 wait complete. 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.582+0000 7fc25dc00640 1 Processor -- start 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc25dc00640 1 -- start start 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc25dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581039b0 0x7fc25819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc25dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2581092f0 0x7fc25819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc25dc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc258109c40 0x7fc25819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc25dc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc258113900 con 0x7fc2581039b0 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc25dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc258113780 con 0x7fc2581092f0 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc25dc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc258113a80 con 0x7fc258109c40 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc257e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581039b0 0x7fc25819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc257e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581039b0 0x7fc25819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36772/0 (socket says 172.21.15.27:36772) 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.583+0000 7fc257e00640 1 -- 172.21.15.27:0/3254192050 learned_addr learned my addr 172.21.15.27:0/3254192050 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc25d200640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc258109c40 0x7fc25819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc257400640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2581092f0 0x7fc25819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc257e00640 1 -- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc258109c40 msgr2=0x7fc25819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc257e00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc258109c40 0x7fc25819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc257e00640 1 -- 172.21.15.27:0/3254192050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2581092f0 msgr2=0x7fc25819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc257e00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2581092f0 0x7fc25819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc257e00640 1 -- 172.21.15.27:0/3254192050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2581a0260 con 0x7fc2581039b0 2024-09-21T07:11:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc25d200640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc258109c40 0x7fc25819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.584+0000 7fc257e00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581039b0 0x7fc25819a2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc24800e9f0 tx=0x7fc24800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.585+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24800cd80 con 0x7fc2581039b0 2024-09-21T07:11:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.585+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2480108e0 con 0x7fc2581039b0 2024-09-21T07:11:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.585+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2581a0550 con 0x7fc2581039b0 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.585+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248018980 con 0x7fc2581039b0 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.585+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2581a6cb0 con 0x7fc2581039b0 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.586+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc258063ad0 con 0x7fc2581039b0 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.588+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc248010430 con 0x7fc2581039b0 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.590+0000 7fc254c00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc23c0b65f0 0x7fc23c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.590+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fc2480e83c0 con 0x7fc2581039b0 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.590+0000 7fc257400640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc23c0b65f0 0x7fc23c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.590+0000 7fc254c00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] conn(0x7fc23c0c1980 0x7fc23c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.590+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 --> [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc24801ce70 con 0x7fc23c0c1980 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.590+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fc2480e8770 con 0x7fc2581039b0 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.591+0000 7fc25d200640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] conn(0x7fc23c0c1980 0x7fc23c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.591+0000 7fc257400640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc23c0b65f0 0x7fc23c0b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc244009230 tx=0x7fc244031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.591+0000 7fc25d200640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] conn(0x7fc23c0c1980 0x7fc23c0c3e70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.592+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== osd.8 v2:172.21.15.153:6816/2174139924 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc24801ce70 con 0x7fc23c0c1980 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.607+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 --> [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc258063f50 con 0x7fc23c0c1980 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc254c00640 1 -- 172.21.15.27:0/3254192050 <== osd.8 v2:172.21.15.153:6816/2174139924 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc258063f50 con 0x7fc23c0c1980 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] conn(0x7fc23c0c1980 msgr2=0x7fc23c0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] conn(0x7fc23c0c1980 0x7fc23c0c3e70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc23c0b65f0 msgr2=0x7fc23c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc23c0b65f0 0x7fc23c0b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc244009230 tx=0x7fc244031400 comp rx=0 tx=0).stop 2024-09-21T07:11:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581039b0 msgr2=0x7fc25819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 --2- 172.21.15.27:0/3254192050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2581039b0 0x7fc25819a2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc24800e9f0 tx=0x7fc24800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25d200640 1 -- 172.21.15.27:0/3254192050 reap_dead start 2024-09-21T07:11:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 shutdown_connections 2024-09-21T07:11:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.608+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 >> 172.21.15.27:0/3254192050 conn(0x7fc2580ff440 msgr2=0x7fc258101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.609+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 shutdown_connections 2024-09-21T07:11:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.609+0000 7fc25dc00640 1 -- 172.21.15.27:0/3254192050 wait complete. 2024-09-21T07:11:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.681+0000 7f3c9ec00640 1 -- 172.21.15.27:0/2500962890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c98110120 msgr2=0x7f3c98112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.681+0000 7f3c9ec00640 1 --2- 172.21.15.27:0/2500962890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c98110120 0x7f3c98112540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009be0 tx=0x7f3c9002a6c0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.681+0000 7f3c9ec00640 1 -- 172.21.15.27:0/2500962890 shutdown_connections 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.681+0000 7f3c9ec00640 1 --2- 172.21.15.27:0/2500962890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c98110120 0x7f3c98112540 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.681+0000 7f3c9ec00640 1 --2- 172.21.15.27:0/2500962890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c980760a0 0x7f3c9810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.681+0000 7f3c9ec00640 1 --2- 172.21.15.27:0/2500962890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c98075750 0x7f3c98075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.681+0000 7f3c9ec00640 1 -- 172.21.15.27:0/2500962890 >> 172.21.15.27:0/2500962890 conn(0x7f3c9807b850 msgr2=0x7f3c9807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.682+0000 7f3c9ec00640 1 -- 172.21.15.27:0/2500962890 shutdown_connections 2024-09-21T07:11:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.682+0000 7f3c9ec00640 1 -- 172.21.15.27:0/2500962890 wait complete. 2024-09-21T07:11:25.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.683+0000 7f3c9ec00640 1 Processor -- start 2024-09-21T07:11:25.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.683+0000 7f3c9ec00640 1 -- start start 2024-09-21T07:11:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c9ec00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c98075750 0x7f3c9819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c9ec00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c980760a0 0x7f3c9819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c9ec00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c98110120 0x7f3c981a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c9ec00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98115c90 con 0x7f3c98075750 2024-09-21T07:11:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c9ec00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98115b10 con 0x7f3c98110120 2024-09-21T07:11:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c9ec00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98115e10 con 0x7f3c980760a0 2024-09-21T07:11:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c98075750 0x7f3c9819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c97400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c980760a0 0x7f3c9819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c98075750 0x7f3c9819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36792/0 (socket says 172.21.15.27:36792) 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c97e00640 1 -- 172.21.15.27:0/3390987136 learned_addr learned my addr 172.21.15.27:0/3390987136 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.684+0000 7f3c9cc00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c98110120 0x7f3c981a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c97400640 1 -- 172.21.15.27:0/3390987136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c98110120 msgr2=0x7f3c981a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c97400640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c98110120 0x7f3c981a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c97400640 1 -- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c98075750 msgr2=0x7f3c9819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c97400640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c98075750 0x7f3c9819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c97400640 1 -- 172.21.15.27:0/3390987136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c981a25b0 con 0x7f3c980760a0 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c97e00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c98075750 0x7f3c9819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c97400640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c980760a0 0x7f3c9819cbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c980777d0 tx=0x7f3c8400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8400cc40 con 0x7f3c980760a0 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c8400cde0 con 0x7f3c980760a0 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.685+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c981a28a0 con 0x7f3c980760a0 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.686+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84018760 con 0x7f3c980760a0 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.686+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c981a9000 con 0x7f3c980760a0 2024-09-21T07:11:25.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.687+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c9810a6e0 con 0x7f3c980760a0 2024-09-21T07:11:25.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.689+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3c84010a90 con 0x7f3c980760a0 2024-09-21T07:11:25.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.692+0000 7f3c94c00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c7c0b65f0 0x7f3c7c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.692+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f3c840e84c0 con 0x7f3c980760a0 2024-09-21T07:11:25.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.692+0000 7f3c97e00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c7c0b65f0 0x7f3c7c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.692+0000 7f3c94c00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] conn(0x7f3c7c0c1980 0x7f3c7c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.692+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 --> [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c84001d30 con 0x7f3c7c0c1980 2024-09-21T07:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.692+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f3c840e8910 con 0x7f3c980760a0 2024-09-21T07:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.693+0000 7f3c9cc00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] conn(0x7f3c7c0c1980 0x7f3c7c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.693+0000 7f3c97e00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c7c0b65f0 0x7f3c7c0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3c880096c0 tx=0x7f3c88009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.693+0000 7f3c9cc00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] conn(0x7f3c7c0c1980 0x7f3c7c0c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:25.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.694+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== osd.6 v2:172.21.15.27:6818/3534557792 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c84001d30 con 0x7f3c7c0c1980 2024-09-21T07:11:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.709+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 --> [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3c98076c60 con 0x7f3c7c0c1980 2024-09-21T07:11:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.709+0000 7f3c94c00640 1 -- 172.21.15.27:0/3390987136 <== osd.6 v2:172.21.15.27:6818/3534557792 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3c98076c60 con 0x7f3c7c0c1980 2024-09-21T07:11:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] conn(0x7f3c7c0c1980 msgr2=0x7f3c7c0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] conn(0x7f3c7c0c1980 0x7f3c7c0c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c7c0b65f0 msgr2=0x7f3c7c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c7c0b65f0 0x7f3c7c0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3c880096c0 tx=0x7f3c88009260 comp rx=0 tx=0).stop 2024-09-21T07:11:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c980760a0 msgr2=0x7f3c9819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 --2- 172.21.15.27:0/3390987136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c980760a0 0x7f3c9819cbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c980777d0 tx=0x7f3c8400ee50 comp rx=0 tx=0).stop 2024-09-21T07:11:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9cc00640 1 -- 172.21.15.27:0/3390987136 reap_dead start 2024-09-21T07:11:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 shutdown_connections 2024-09-21T07:11:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 >> 172.21.15.27:0/3390987136 conn(0x7f3c9807b850 msgr2=0x7f3c98109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 shutdown_connections 2024-09-21T07:11:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:25.710+0000 7f3c9ec00640 1 -- 172.21.15.27:0/3390987136 wait complete. 2024-09-21T07:11:26.063 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.551+0000 7fb48cc00640 1 -- 172.21.15.27:0/2117444132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb488109c40 msgr2=0x7fb488110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 --2- 172.21.15.27:0/2117444132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb488109c40 0x7fb488110470 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb480009ca0 tx=0x7fb48002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 -- 172.21.15.27:0/2117444132 shutdown_connections 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 --2- 172.21.15.27:0/2117444132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb488109c40 0x7fb488110470 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 --2- 172.21.15.27:0/2117444132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4881092f0 0x7fb488109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 --2- 172.21.15.27:0/2117444132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4881039b0 0x7fb488103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 -- 172.21.15.27:0/2117444132 >> 172.21.15.27:0/2117444132 conn(0x7fb4880ff400 msgr2=0x7fb488101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 -- 172.21.15.27:0/2117444132 shutdown_connections 2024-09-21T07:11:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.552+0000 7fb48cc00640 1 -- 172.21.15.27:0/2117444132 wait complete. 2024-09-21T07:11:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.553+0000 7fb48cc00640 1 Processor -- start 2024-09-21T07:11:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.553+0000 7fb48cc00640 1 -- start start 2024-09-21T07:11:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb48cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4881039b0 0x7fb48819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb48cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4881092f0 0x7fb48819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb48cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb488109c40 0x7fb48819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb48cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb488113900 con 0x7fb4881039b0 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb48cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb488113780 con 0x7fb4881092f0 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb48cc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb488113a80 con 0x7fb488109c40 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb487400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4881039b0 0x7fb48819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb486a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4881092f0 0x7fb48819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb487400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4881039b0 0x7fb48819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36816/0 (socket says 172.21.15.27:36816) 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb487400640 1 -- 172.21.15.27:0/168860562 learned_addr learned my addr 172.21.15.27:0/168860562 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb487e00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb488109c40 0x7fb48819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb486a00640 1 -- 172.21.15.27:0/168860562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb488109c40 msgr2=0x7fb48819fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb486a00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb488109c40 0x7fb48819fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb486a00640 1 -- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4881039b0 msgr2=0x7fb48819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb486a00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4881039b0 0x7fb48819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.554+0000 7fb486a00640 1 -- 172.21.15.27:0/168860562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4881a0180 con 0x7fb4881092f0 2024-09-21T07:11:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.555+0000 7fb487e00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb488109c40 0x7fb48819fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:11:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.555+0000 7fb486a00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4881092f0 0x7fb48819a7b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb48806b930 tx=0x7fb47400db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.555+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474017070 con 0x7fb4881092f0 2024-09-21T07:11:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.555+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4881a0470 con 0x7fb4881092f0 2024-09-21T07:11:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.555+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb474009820 con 0x7fb4881092f0 2024-09-21T07:11:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.555+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474004ba0 con 0x7fb4881092f0 2024-09-21T07:11:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.555+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4881a6bd0 con 0x7fb4881092f0 2024-09-21T07:11:26.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.556+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb488063b50 con 0x7fb4881092f0 2024-09-21T07:11:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.560+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fb474009c60 con 0x7fb4881092f0 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.561+0000 7fb47fe00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb46c0b65f0 0x7fb46c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.561+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fb4740e7660 con 0x7fb4881092f0 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.562+0000 7fb487400640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb46c0b65f0 0x7fb46c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.562+0000 7fb47fe00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] conn(0x7fb46c0c1980 0x7fb46c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.562+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 --> [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb474021e50 con 0x7fb46c0c1980 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.562+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb4740e7a10 con 0x7fb4881092f0 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.562+0000 7fb487e00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] conn(0x7fb46c0c1980 0x7fb46c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.562+0000 7fb487400640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb46c0b65f0 0x7fb46c0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb478005fd0 tx=0x7fb47800b4a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.563+0000 7fb487e00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] conn(0x7fb46c0c1980 0x7fb46c0c3e70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:26.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.564+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== osd.3 v2:172.21.15.27:6810/984637291 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb474021e50 con 0x7fb46c0c1980 2024-09-21T07:11:26.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.578+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 --> [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb488063f50 con 0x7fb46c0c1980 2024-09-21T07:11:26.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb47fe00640 1 -- 172.21.15.27:0/168860562 <== osd.3 v2:172.21.15.27:6810/984637291 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb488063f50 con 0x7fb46c0c1980 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] conn(0x7fb46c0c1980 msgr2=0x7fb46c0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] conn(0x7fb46c0c1980 0x7fb46c0c3e70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb46c0b65f0 msgr2=0x7fb46c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb46c0b65f0 0x7fb46c0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb478005fd0 tx=0x7fb47800b4a0 comp rx=0 tx=0).stop 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4881092f0 msgr2=0x7fb48819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 --2- 172.21.15.27:0/168860562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4881092f0 0x7fb48819a7b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb48806b930 tx=0x7fb47400db30 comp rx=0 tx=0).stop 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb487e00640 1 -- 172.21.15.27:0/168860562 reap_dead start 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 shutdown_connections 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.579+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 >> 172.21.15.27:0/168860562 conn(0x7fb4880ff400 msgr2=0x7fb4881068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:26.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.580+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 shutdown_connections 2024-09-21T07:11:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:26.580+0000 7fb48cc00640 1 -- 172.21.15.27:0/168860562 wait complete. 2024-09-21T07:11:27.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:26 smithi027 ceph-mon[30779]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T07:11:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:26 smithi119 ceph-mon[34519]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T07:11:27.363 INFO:teuthology.orchestra.run.smithi027.stdout:60129542228 2024-09-21T07:11:27.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T07:11:27.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:26 smithi153 ceph-mon[34747]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T07:11:27.783 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:28.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.753+0000 7f1249c00640 1 -- 172.21.15.27:0/3607679733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 msgr2=0x7f1244075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.753+0000 7f1249c00640 1 --2- 172.21.15.27:0/3607679733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 0x7f1244075bc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1234009900 tx=0x7f123402a410 comp rx=0 tx=0).stop 2024-09-21T07:11:28.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.754+0000 7f1249c00640 1 -- 172.21.15.27:0/3607679733 shutdown_connections 2024-09-21T07:11:28.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.754+0000 7f1249c00640 1 --2- 172.21.15.27:0/3607679733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1244076100 0x7f124410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.754+0000 7f1249c00640 1 --2- 172.21.15.27:0/3607679733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f124410ff10 0x7f1244112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.754+0000 7f1249c00640 1 --2- 172.21.15.27:0/3607679733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 0x7f1244075bc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.754+0000 7f1249c00640 1 -- 172.21.15.27:0/3607679733 >> 172.21.15.27:0/3607679733 conn(0x7f12440ff440 msgr2=0x7f1244101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.754+0000 7f1249c00640 1 -- 172.21.15.27:0/3607679733 shutdown_connections 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.754+0000 7f1249c00640 1 -- 172.21.15.27:0/3607679733 wait complete. 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 Processor -- start 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 -- start start 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 0x7f124419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1244076100 0x7f124419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f124410ff10 0x7f12441a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1244115ac0 con 0x7f12440757b0 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1244115940 con 0x7f124410ff10 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.755+0000 7f1249c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1244115c40 con 0x7f1244076100 2024-09-21T07:11:28.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 0x7f124419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1249200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f124410ff10 0x7f12441a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 0x7f124419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36854/0 (socket says 172.21.15.27:36854) 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 -- 172.21.15.27:0/1590682361 learned_addr learned my addr 172.21.15.27:0/1590682361 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f123be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1244076100 0x7f124419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 -- 172.21.15.27:0/1590682361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1244076100 msgr2=0x7f124419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1244076100 0x7f124419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 -- 172.21.15.27:0/1590682361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f124410ff10 msgr2=0x7f12441a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f124410ff10 0x7f12441a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 -- 172.21.15.27:0/1590682361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12441a2420 con 0x7f12440757b0 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1249200640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f124410ff10 0x7f12441a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.756+0000 7f1243e00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 0x7f124419c4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1234005940 tx=0x7f1234004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.757+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123403f030 con 0x7f12440757b0 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.757+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1234003aa0 con 0x7f12440757b0 2024-09-21T07:11:28.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.757+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12441a26b0 con 0x7f12440757b0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.757+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234035070 con 0x7f12440757b0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.757+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12441a8e70 con 0x7f12440757b0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.758+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f124410a510 con 0x7f12440757b0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.761+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1234042030 con 0x7f12440757b0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.762+0000 7f1241600640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f12280b65f0 0x7f12280b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.762+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f1234108760 con 0x7f12440757b0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.762+0000 7f123be00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f12280b65f0 0x7f12280b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.763+0000 7f1241600640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] conn(0x7f12280c1980 0x7f12280c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.763+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 --> [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12280c4540 con 0x7f12280c1980 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.763+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f1234108b10 con 0x7f12440757b0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.763+0000 7f1249200640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] conn(0x7f12280c1980 0x7f12280c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.763+0000 7f123be00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f12280b65f0 0x7f12280b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1244101c10 tx=0x7f122c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.764+0000 7f1249200640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] conn(0x7f12280c1980 0x7f12280c3e70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:28.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.765+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== osd.11 v2:172.21.15.153:6824/1280738577 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12280c4540 con 0x7f12280c1980 2024-09-21T07:11:28.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.781+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 --> [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1244076cc0 con 0x7f12280c1980 2024-09-21T07:11:28.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1241600640 1 -- 172.21.15.27:0/1590682361 <== osd.11 v2:172.21.15.153:6824/1280738577 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1244076cc0 con 0x7f12280c1980 2024-09-21T07:11:28.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 >> [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] conn(0x7f12280c1980 msgr2=0x7f12280c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1249c00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] conn(0x7f12280c1980 0x7f12280c3e70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f12280b65f0 msgr2=0x7f12280b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1249c00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f12280b65f0 0x7f12280b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1244101c10 tx=0x7f122c00b390 comp rx=0 tx=0).stop 2024-09-21T07:11:28.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 msgr2=0x7f124419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1249c00640 1 --2- 172.21.15.27:0/1590682361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12440757b0 0x7f124419c4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1234005940 tx=0x7f1234004270 comp rx=0 tx=0).stop 2024-09-21T07:11:28.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.782+0000 7f1249200640 1 -- 172.21.15.27:0/1590682361 reap_dead start 2024-09-21T07:11:28.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.783+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 shutdown_connections 2024-09-21T07:11:28.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.783+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 >> 172.21.15.27:0/1590682361 conn(0x7f12440ff440 msgr2=0x7f1244109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:28.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.783+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 shutdown_connections 2024-09-21T07:11:28.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.783+0000 7f1249c00640 1 -- 172.21.15.27:0/1590682361 wait complete. 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 -- 172.21.15.27:0/3250671236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd81092f0 msgr2=0x7f9bd8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 --2- 172.21.15.27:0/3250671236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd81092f0 0x7f9bd8109700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009900 tx=0x7f9bc402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 -- 172.21.15.27:0/3250671236 shutdown_connections 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 --2- 172.21.15.27:0/3250671236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bd8109c40 0x7f9bd8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 --2- 172.21.15.27:0/3250671236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd81092f0 0x7f9bd8109700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 --2- 172.21.15.27:0/3250671236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bd81039b0 0x7f9bd8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 -- 172.21.15.27:0/3250671236 >> 172.21.15.27:0/3250671236 conn(0x7f9bd80ff400 msgr2=0x7f9bd8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 -- 172.21.15.27:0/3250671236 shutdown_connections 2024-09-21T07:11:28.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.507+0000 7f9bdd400640 1 -- 172.21.15.27:0/3250671236 wait complete. 2024-09-21T07:11:28.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.508+0000 7f9bdd400640 1 Processor -- start 2024-09-21T07:11:28.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.508+0000 7f9bdd400640 1 -- start start 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.508+0000 7f9bdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bd81039b0 0x7f9bd819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.508+0000 7f9bdd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bd81092f0 0x7f9bd819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bdd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd8109c40 0x7f9bd819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bdd400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8113950 con 0x7f9bd8109c40 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bdd400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81137d0 con 0x7f9bd81039b0 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bdd400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8113ad0 con 0x7f9bd81092f0 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bdca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd8109c40 0x7f9bd819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bdca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd8109c40 0x7f9bd819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35234/0 (socket says 172.21.15.27:35234) 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bdca00640 1 -- 172.21.15.27:0/727287525 learned_addr learned my addr 172.21.15.27:0/727287525 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:28.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bd7400640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bd81092f0 0x7f9bd819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bd7e00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bd81039b0 0x7f9bd819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bd7400640 1 -- 172.21.15.27:0/727287525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bd81039b0 msgr2=0x7f9bd819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bd7400640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bd81039b0 0x7f9bd819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bd7400640 1 -- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd8109c40 msgr2=0x7f9bd819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.509+0000 7f9bd7400640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd8109c40 0x7f9bd819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bd7400640 1 -- 172.21.15.27:0/727287525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd81a0250 con 0x7f9bd81092f0 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bdca00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd8109c40 0x7f9bd819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bd7e00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bd81039b0 0x7f9bd819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bd7400640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bd81092f0 0x7f9bd819a880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009230 tx=0x7f9bc4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc403f070 con 0x7f9bd81092f0 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd81a04e0 con 0x7f9bd81092f0 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc4031b50 con 0x7f9bd81092f0 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4035030 con 0x7f9bd81092f0 2024-09-21T07:11:28.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.510+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd81a6ca0 con 0x7f9bd81092f0 2024-09-21T07:11:28.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.511+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9bd8063b50 con 0x7f9bd81092f0 2024-09-21T07:11:28.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.514+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f9bc4042070 con 0x7f9bd81092f0 2024-09-21T07:11:28.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.516+0000 7f9bd4c00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bbc0b65f0 0x7f9bbc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.516+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f9bc4107f80 con 0x7f9bd81092f0 2024-09-21T07:11:28.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.516+0000 7f9bd7e00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bbc0b65f0 0x7f9bbc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.516+0000 7f9bd4c00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] conn(0x7f9bbc0c1980 0x7f9bbc0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:28.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.516+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 --> [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bbc0c4540 con 0x7f9bbc0c1980 2024-09-21T07:11:28.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.516+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f9bc4108330 con 0x7f9bd81092f0 2024-09-21T07:11:28.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.517+0000 7f9bdca00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] conn(0x7f9bbc0c1980 0x7f9bbc0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:28.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.517+0000 7f9bd7e00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bbc0b65f0 0x7f9bbc0b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80095f0 tx=0x7f9bc8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:28.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.517+0000 7f9bdca00640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] conn(0x7f9bbc0c1980 0x7f9bbc0c3e70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:28.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.518+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== osd.0 v2:172.21.15.27:6802/1967162431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bbc0c4540 con 0x7f9bbc0c1980 2024-09-21T07:11:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.533+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 --> [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9bd8063f50 con 0x7f9bbc0c1980 2024-09-21T07:11:28.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.533+0000 7f9bd4c00640 1 -- 172.21.15.27:0/727287525 <== osd.0 v2:172.21.15.27:6802/1967162431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9bd8063f50 con 0x7f9bbc0c1980 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.533+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] conn(0x7f9bbc0c1980 msgr2=0x7f9bbc0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.533+0000 7f9bdd400640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] conn(0x7f9bbc0c1980 0x7f9bbc0c3e70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bbc0b65f0 msgr2=0x7f9bbc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bbc0b65f0 0x7f9bbc0b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80095f0 tx=0x7f9bc8005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bd81092f0 msgr2=0x7f9bd819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 --2- 172.21.15.27:0/727287525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bd81092f0 0x7f9bd819a880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009230 tx=0x7f9bc4003b90 comp rx=0 tx=0).stop 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdca00640 1 -- 172.21.15.27:0/727287525 reap_dead start 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 shutdown_connections 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 >> 172.21.15.27:0/727287525 conn(0x7f9bd80ff400 msgr2=0x7f9bd8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 shutdown_connections 2024-09-21T07:11:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:28.534+0000 7f9bdd400640 1 -- 172.21.15.27:0/727287525 wait complete. 2024-09-21T07:11:29.019 INFO:teuthology.orchestra.run.smithi027.stdout:103079215177 2024-09-21T07:11:29.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T07:11:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:29.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:28 smithi027 ceph-mon[30779]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s 2024-09-21T07:11:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:28 smithi119 ceph-mon[34519]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s 2024-09-21T07:11:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.638+0000 7f51a9c00640 1 -- 172.21.15.27:0/692225242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51a4109e00 msgr2=0x7f51a4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.638+0000 7f51a9c00640 1 --2- 172.21.15.27:0/692225242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51a4109e00 0x7f51a4110630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f519c009ca0 tx=0x7f519c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.638+0000 7f51a9c00640 1 -- 172.21.15.27:0/692225242 shutdown_connections 2024-09-21T07:11:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.639+0000 7f51a9c00640 1 --2- 172.21.15.27:0/692225242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51a4109e00 0x7f51a4110630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.639+0000 7f51a9c00640 1 --2- 172.21.15.27:0/692225242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a41094b0 0x7f51a41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.639+0000 7f51a9c00640 1 --2- 172.21.15.27:0/692225242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103b70 0x7f51a4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.639+0000 7f51a9c00640 1 -- 172.21.15.27:0/692225242 >> 172.21.15.27:0/692225242 conn(0x7f51a40ff5e0 msgr2=0x7f51a4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.639+0000 7f51a9c00640 1 -- 172.21.15.27:0/692225242 shutdown_connections 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.639+0000 7f51a9c00640 1 -- 172.21.15.27:0/692225242 wait complete. 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.640+0000 7f51a9c00640 1 Processor -- start 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.640+0000 7f51a9c00640 1 -- start start 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.640+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103b70 0x7f51a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.640+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51a41094b0 0x7f51a4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.640+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4109e00 0x7f51a4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a9c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4113ac0 con 0x7f51a4103b70 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a9c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4113940 con 0x7f51a4109e00 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a9c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4113c40 con 0x7f51a41094b0 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103b70 0x7f51a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103b70 0x7f51a4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36832/0 (socket says 172.21.15.27:36832) 2024-09-21T07:11:29.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 -- 172.21.15.27:0/1077465996 learned_addr learned my addr 172.21.15.27:0/1077465996 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51a41094b0 0x7f51a4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a3400640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4109e00 0x7f51a4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 -- 172.21.15.27:0/1077465996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51a41094b0 msgr2=0x7f51a4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51a41094b0 0x7f51a4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 -- 172.21.15.27:0/1077465996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4109e00 msgr2=0x7f51a4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4109e00 0x7f51a4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.641+0000 7f51a2a00640 1 -- 172.21.15.27:0/1077465996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a41a86f0 con 0x7f51a4103b70 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.642+0000 7f51a2a00640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103b70 0x7f51a4075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9f0 tx=0x7f519400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.642+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519400cd80 con 0x7f51a4103b70 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.642+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41a89e0 con 0x7f51a4103b70 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.642+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51940108e0 con 0x7f51a4103b70 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.642+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a8f70 con 0x7f51a4103b70 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.642+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194018920 con 0x7f51a4103b70 2024-09-21T07:11:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.643+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51a4063b50 con 0x7f51a4103b70 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.646+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5194010430 con 0x7f51a4103b70 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.647+0000 7f519b400640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51880b65f0 0x7f51880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.648+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f51940e80d0 con 0x7f51a4103b70 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.648+0000 7f51a2000640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51880b65f0 0x7f51880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.648+0000 7f519b400640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] conn(0x7f51880c1980 0x7f51880c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.648+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 --> [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51880c4540 con 0x7f51880c1980 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.648+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f51940e8520 con 0x7f51a4103b70 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.649+0000 7f51a3400640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] conn(0x7f51880c1980 0x7f51880c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.649+0000 7f51a2000640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51880b65f0 0x7f51880b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51900095f0 tx=0x7f5190005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.649+0000 7f51a3400640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] conn(0x7f51880c1980 0x7f51880c3e70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.650+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== osd.2 v2:172.21.15.153:6800/206718354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51880c4540 con 0x7f51880c1980 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.665+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 --> [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51a4063f50 con 0x7f51880c1980 2024-09-21T07:11:29.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.665+0000 7f519b400640 1 -- 172.21.15.27:0/1077465996 <== osd.2 v2:172.21.15.153:6800/206718354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f51a4063f50 con 0x7f51880c1980 2024-09-21T07:11:29.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 >> [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] conn(0x7f51880c1980 msgr2=0x7f51880c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:29.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a9c00640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] conn(0x7f51880c1980 0x7f51880c3e70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51880b65f0 msgr2=0x7f51880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:29.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a9c00640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f51880b65f0 0x7f51880b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51900095f0 tx=0x7f5190005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:29.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103b70 msgr2=0x7f51a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:29.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a9c00640 1 --2- 172.21.15.27:0/1077465996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103b70 0x7f51a4075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9f0 tx=0x7f519400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:29.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a3400640 1 -- 172.21.15.27:0/1077465996 reap_dead start 2024-09-21T07:11:29.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.666+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 shutdown_connections 2024-09-21T07:11:29.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.667+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 >> 172.21.15.27:0/1077465996 conn(0x7f51a40ff5e0 msgr2=0x7f51a4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:29.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.667+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 shutdown_connections 2024-09-21T07:11:29.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:27.667+0000 7f51a9c00640 1 -- 172.21.15.27:0/1077465996 wait complete. 2024-09-21T07:11:29.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:28 smithi153 ceph-mon[34747]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.140+0000 7f31a0600640 1 -- 172.21.15.27:0/3541722459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3198077380 msgr2=0x7f31980757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.140+0000 7f31a0600640 1 --2- 172.21.15.27:0/3541722459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3198077380 0x7f31980757b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f318c009900 tx=0x7f318c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.141+0000 7f31a0600640 1 -- 172.21.15.27:0/3541722459 shutdown_connections 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.141+0000 7f31a0600640 1 --2- 172.21.15.27:0/3541722459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3198075dd0 0x7f31980761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.141+0000 7f31a0600640 1 --2- 172.21.15.27:0/3541722459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3198114430 0x7f3198116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.141+0000 7f31a0600640 1 --2- 172.21.15.27:0/3541722459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3198077380 0x7f31980757b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.141+0000 7f31a0600640 1 -- 172.21.15.27:0/3541722459 >> 172.21.15.27:0/3541722459 conn(0x7f319807b850 msgr2=0x7f319807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.141+0000 7f31a0600640 1 -- 172.21.15.27:0/3541722459 shutdown_connections 2024-09-21T07:11:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.141+0000 7f31a0600640 1 -- 172.21.15.27:0/3541722459 wait complete. 2024-09-21T07:11:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.142+0000 7f31a0600640 1 Processor -- start 2024-09-21T07:11:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.142+0000 7f31a0600640 1 -- start start 2024-09-21T07:11:31.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f31a0600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3198075dd0 0x7f31981a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:31.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f31a0600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3198077380 0x7f31981a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:31.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f31a0600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3198114430 0x7f31981a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:31.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f31a0600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f319811a0d0 con 0x7f3198077380 2024-09-21T07:11:31.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f31a0600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3198119f50 con 0x7f3198114430 2024-09-21T07:11:31.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f31a0600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f319811a250 con 0x7f3198075dd0 2024-09-21T07:11:31.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3198077380 0x7f31981a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:31.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3198077380 0x7f31981a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35260/0 (socket says 172.21.15.27:35260) 2024-09-21T07:11:31.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 -- 172.21.15.27:0/1054982258 learned_addr learned my addr 172.21.15.27:0/1054982258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:31.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319dc00640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3198075dd0 0x7f31981a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:31.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319e600640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3198114430 0x7f31981a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 -- 172.21.15.27:0/1054982258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3198075dd0 msgr2=0x7f31981a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3198075dd0 0x7f31981a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 -- 172.21.15.27:0/1054982258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3198114430 msgr2=0x7f31981a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3198114430 0x7f31981a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.143+0000 7f319d200640 1 -- 172.21.15.27:0/1054982258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31981a6a10 con 0x7f3198077380 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.144+0000 7f319d200640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3198077380 0x7f31981a1040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9f0 tx=0x7f318400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.144+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400ccc0 con 0x7f3198077380 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.144+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31981a6d00 con 0x7f3198077380 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.144+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31840054d0 con 0x7f3198077380 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.144+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31981ad460 con 0x7f3198077380 2024-09-21T07:11:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.145+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184021440 con 0x7f3198077380 2024-09-21T07:11:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.145+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3198113a20 con 0x7f3198077380 2024-09-21T07:11:31.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.149+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f31840046c0 con 0x7f3198077380 2024-09-21T07:11:31.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.150+0000 7f3196000640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31800b65f0 0x7f31800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.150+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f31840e8310 con 0x7f3198077380 2024-09-21T07:11:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.150+0000 7f319dc00640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31800b65f0 0x7f31800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.151+0000 7f3196000640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] conn(0x7f31800c1980 0x7f31800c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.151+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 --> [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f318401aec0 con 0x7f31800c1980 2024-09-21T07:11:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.151+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f31840e86c0 con 0x7f3198077380 2024-09-21T07:11:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.151+0000 7f319e600640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] conn(0x7f31800c1980 0x7f31800c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:31.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.151+0000 7f319dc00640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31800b65f0 0x7f31800b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f318c005ab0 tx=0x7f318c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:31.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.152+0000 7f319e600640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] conn(0x7f31800c1980 0x7f31800c3e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:31.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.153+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== osd.9 v2:172.21.15.27:6826/611699615 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f318401aec0 con 0x7f31800c1980 2024-09-21T07:11:31.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.167+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 --> [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3198076cc0 con 0x7f31800c1980 2024-09-21T07:11:31.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.168+0000 7f3196000640 1 -- 172.21.15.27:0/1054982258 <== osd.9 v2:172.21.15.27:6826/611699615 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3198076cc0 con 0x7f31800c1980 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.168+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] conn(0x7f31800c1980 msgr2=0x7f31800c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.168+0000 7f31a0600640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] conn(0x7f31800c1980 0x7f31800c3e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.168+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31800b65f0 msgr2=0x7f31800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.168+0000 7f31a0600640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31800b65f0 0x7f31800b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f318c005ab0 tx=0x7f318c005a20 comp rx=0 tx=0).stop 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.168+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3198077380 msgr2=0x7f31981a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.168+0000 7f31a0600640 1 --2- 172.21.15.27:0/1054982258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3198077380 0x7f31981a1040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9f0 tx=0x7f318400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.169+0000 7f319e600640 1 -- 172.21.15.27:0/1054982258 reap_dead start 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.169+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 shutdown_connections 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.169+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 >> 172.21.15.27:0/1054982258 conn(0x7f319807b850 msgr2=0x7f3198113040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.169+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 shutdown_connections 2024-09-21T07:11:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:31.169+0000 7f31a0600640 1 -- 172.21.15.27:0/1054982258 wait complete. 2024-09-21T07:11:31.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:30 smithi027 ceph-mon[30779]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-21T07:11:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:30 smithi119 ceph-mon[34519]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-21T07:11:31.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:30 smithi153 ceph-mon[34747]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-21T07:11:32.003 INFO:teuthology.orchestra.run.smithi027.stdout:219043332144 2024-09-21T07:11:32.003 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T07:11:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 -- 172.21.15.27:0/3934904111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 msgr2=0x7f8938103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 --2- 172.21.15.27:0/3934904111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 0x7f8938103f90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8928009900 tx=0x7f892802a410 comp rx=0 tx=0).stop 2024-09-21T07:11:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 -- 172.21.15.27:0/3934904111 shutdown_connections 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 --2- 172.21.15.27:0/3934904111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89381094c0 0x7f89381098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 --2- 172.21.15.27:0/3934904111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8938109e10 0x7f8938110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 --2- 172.21.15.27:0/3934904111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 0x7f8938103f90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 -- 172.21.15.27:0/3934904111 >> 172.21.15.27:0/3934904111 conn(0x7f89380ff5d0 msgr2=0x7f8938101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 -- 172.21.15.27:0/3934904111 shutdown_connections 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.264+0000 7f893e800640 1 -- 172.21.15.27:0/3934904111 wait complete. 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.265+0000 7f893e800640 1 Processor -- start 2024-09-21T07:11:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.266+0000 7f893e800640 1 -- start start 2024-09-21T07:11:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.266+0000 7f893e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 0x7f893819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.266+0000 7f893e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89381094c0 0x7f893819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.266+0000 7f893e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8938109e10 0x7f893819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.266+0000 7f893e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8938113ad0 con 0x7f8938103b80 2024-09-21T07:11:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.266+0000 7f893e800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8938113950 con 0x7f89381094c0 2024-09-21T07:11:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.266+0000 7f893e800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8938113c50 con 0x7f8938109e10 2024-09-21T07:11:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8937e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8938109e10 0x7f893819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8937400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 0x7f893819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8937e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8938109e10 0x7f893819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49728/0 (socket says 172.21.15.27:49728) 2024-09-21T07:11:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8937e00640 1 -- 172.21.15.27:0/712749443 learned_addr learned my addr 172.21.15.27:0/712749443 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8936a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89381094c0 0x7f893819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8936a00640 1 -- 172.21.15.27:0/712749443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8938109e10 msgr2=0x7f893819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8936a00640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8938109e10 0x7f893819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8936a00640 1 -- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 msgr2=0x7f893819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8936a00640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 0x7f893819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8936a00640 1 -- 172.21.15.27:0/712749443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89381a0430 con 0x7f89381094c0 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.267+0000 7f8937400640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8938103b80 0x7f893819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.268+0000 7f8936a00640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89381094c0 0x7f893819aa60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8924005fd0 tx=0x7f892400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.268+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924009c00 con 0x7f89381094c0 2024-09-21T07:11:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.268+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8924009da0 con 0x7f89381094c0 2024-09-21T07:11:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.268+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89381a0720 con 0x7f89381094c0 2024-09-21T07:11:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.268+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924003e10 con 0x7f89381094c0 2024-09-21T07:11:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.268+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89381a6e80 con 0x7f89381094c0 2024-09-21T07:11:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.269+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8918001060 con 0x7f89381094c0 2024-09-21T07:11:32.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.273+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8924004030 con 0x7f89381094c0 2024-09-21T07:11:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.275+0000 7f892fe00640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f891c0b65f0 0x7f891c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.275+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f89240e76d0 con 0x7f89381094c0 2024-09-21T07:11:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.275+0000 7f8937400640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f891c0b65f0 0x7f891c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.275+0000 7f892fe00640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] conn(0x7f891c0c1980 0x7f891c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.275+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 --> [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8924015ec0 con 0x7f891c0c1980 2024-09-21T07:11:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.275+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f89240e7b10 con 0x7f89381094c0 2024-09-21T07:11:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.276+0000 7f8937e00640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] conn(0x7f891c0c1980 0x7f891c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.276+0000 7f8937400640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f891c0b65f0 0x7f891c0b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8928031440 tx=0x7f8928032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.276+0000 7f8937e00640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] conn(0x7f891c0c1980 0x7f891c0c3e70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.277+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== osd.7 v2:172.21.15.119:6816/1162667989 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8924015ec0 con 0x7f891c0c1980 2024-09-21T07:11:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.292+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 --> [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8918002e00 con 0x7f891c0c1980 2024-09-21T07:11:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.293+0000 7f892fe00640 1 -- 172.21.15.27:0/712749443 <== osd.7 v2:172.21.15.119:6816/1162667989 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8918002e00 con 0x7f891c0c1980 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.293+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] conn(0x7f891c0c1980 msgr2=0x7f891c0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.293+0000 7f893e800640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] conn(0x7f891c0c1980 0x7f891c0c3e70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f891c0b65f0 msgr2=0x7f891c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f891c0b65f0 0x7f891c0b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8928031440 tx=0x7f8928032ed0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89381094c0 msgr2=0x7f893819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 --2- 172.21.15.27:0/712749443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89381094c0 0x7f893819aa60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8924005fd0 tx=0x7f892400dde0 comp rx=0 tx=0).stop 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f8937e00640 1 -- 172.21.15.27:0/712749443 reap_dead start 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 shutdown_connections 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 >> 172.21.15.27:0/712749443 conn(0x7f89380ff5d0 msgr2=0x7f89381019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 shutdown_connections 2024-09-21T07:11:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:32.294+0000 7f893e800640 1 -- 172.21.15.27:0/712749443 wait complete. 2024-09-21T07:11:32.446 INFO:teuthology.orchestra.run.smithi027.stdout:176093659193 2024-09-21T07:11:32.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T07:11:33.064 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:32 smithi027 ceph-mon[30779]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:33.230 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:32 smithi153 ceph-mon[34747]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:32 smithi119 ceph-mon[34519]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:33.530 INFO:teuthology.orchestra.run.smithi027.stdout:197568495669 2024-09-21T07:11:33.530 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T07:11:33.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.247+0000 7f4bdb200640 1 -- 172.21.15.27:0/858457197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc077940 msgr2=0x7f4bdc077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.247+0000 7f4bdb200640 1 --2- 172.21.15.27:0/858457197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc077940 0x7f4bdc077d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009900 tx=0x7f4bcc02a410 comp rx=0 tx=0).stop 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.248+0000 7f4bdb200640 1 -- 172.21.15.27:0/858457197 shutdown_connections 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.248+0000 7f4bdb200640 1 --2- 172.21.15.27:0/858457197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bdc078290 0x7f4bdc114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.248+0000 7f4bdb200640 1 --2- 172.21.15.27:0/858457197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc114570 0x7f4bdc116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.248+0000 7f4bdb200640 1 --2- 172.21.15.27:0/858457197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc077940 0x7f4bdc077d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.248+0000 7f4bdb200640 1 -- 172.21.15.27:0/858457197 >> 172.21.15.27:0/858457197 conn(0x7f4bdc101680 msgr2=0x7f4bdc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.248+0000 7f4bdb200640 1 -- 172.21.15.27:0/858457197 shutdown_connections 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.248+0000 7f4bdb200640 1 -- 172.21.15.27:0/858457197 wait complete. 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 Processor -- start 2024-09-21T07:11:33.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 -- start start 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc077940 0x7f4bdc1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc078290 0x7f4bdc1a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bdc114570 0x7f4bdc1a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc119f20 con 0x7f4bdc078290 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc119da0 con 0x7f4bdc077940 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.249+0000 7f4bdb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc11a0a0 con 0x7f4bdc114570 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc077940 0x7f4bdc1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc078290 0x7f4bdc1a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc077940 0x7f4bdc1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46220/0 (socket says 172.21.15.27:46220) 2024-09-21T07:11:33.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 -- 172.21.15.27:0/325995541 learned_addr learned my addr 172.21.15.27:0/325995541 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bda800640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bdc114570 0x7f4bdc1a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 -- 172.21.15.27:0/325995541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bdc114570 msgr2=0x7f4bdc1a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bdc114570 0x7f4bdc1a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 -- 172.21.15.27:0/325995541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc078290 msgr2=0x7f4bdc1a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc078290 0x7f4bdc1a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 -- 172.21.15.27:0/325995541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bdc1a6860 con 0x7f4bdc077940 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.250+0000 7f4bd9e00640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc077940 0x7f4bdc1a08f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005ab0 tx=0x7f4bcc003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.251+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc03f070 con 0x7f4bdc077940 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.251+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bdc1a6af0 con 0x7f4bdc077940 2024-09-21T07:11:33.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.251+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bcc031d60 con 0x7f4bdc077940 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.251+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc035030 con 0x7f4bdc077940 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.251+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bdc1ad2b0 con 0x7f4bdc077940 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.252+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bdc063b50 con 0x7f4bdc077940 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.254+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f4bcc042070 con 0x7f4bdc077940 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.256+0000 7f4bd2a00640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bc00b65f0 0x7f4bc00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.256+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f4bcc108830 con 0x7f4bdc077940 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.256+0000 7f4bd9400640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bc00b65f0 0x7f4bc00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.256+0000 7f4bd2a00640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] conn(0x7f4bc00c1980 0x7f4bc00c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.256+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 --> [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bc00c4540 con 0x7f4bc00c1980 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.256+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f4bcc108be0 con 0x7f4bdc077940 2024-09-21T07:11:33.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.257+0000 7f4bda800640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] conn(0x7f4bc00c1980 0x7f4bc00c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:33.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.257+0000 7f4bd9400640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bc00b65f0 0x7f4bc00b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc07a230 tx=0x7f4bc800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:33.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.257+0000 7f4bda800640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] conn(0x7f4bc00c1980 0x7f4bc00c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:33.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.258+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== osd.10 v2:172.21.15.119:6824/3847228474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc00c4540 con 0x7f4bc00c1980 2024-09-21T07:11:33.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.273+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 --> [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bdc063fd0 con 0x7f4bc00c1980 2024-09-21T07:11:33.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.273+0000 7f4bd2a00640 1 -- 172.21.15.27:0/325995541 <== osd.10 v2:172.21.15.119:6824/3847228474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4bdc063fd0 con 0x7f4bc00c1980 2024-09-21T07:11:33.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] conn(0x7f4bc00c1980 msgr2=0x7f4bc00c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:33.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] conn(0x7f4bc00c1980 0x7f4bc00c3e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:33.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bc00b65f0 msgr2=0x7f4bc00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4bc00b65f0 0x7f4bc00b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc07a230 tx=0x7f4bc800b390 comp rx=0 tx=0).stop 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc077940 msgr2=0x7f4bdc1a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 --2- 172.21.15.27:0/325995541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc077940 0x7f4bdc1a08f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005ab0 tx=0x7f4bcc003c80 comp rx=0 tx=0).stop 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bda800640 1 -- 172.21.15.27:0/325995541 reap_dead start 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 shutdown_connections 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 >> 172.21.15.27:0/325995541 conn(0x7f4bdc101680 msgr2=0x7f4bdc1165b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 shutdown_connections 2024-09-21T07:11:33.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:33.274+0000 7f4bdb200640 1 -- 172.21.15.27:0/325995541 wait complete. 2024-09-21T07:11:34.168 INFO:teuthology.orchestra.run.smithi027.stdout:77309411408 2024-09-21T07:11:34.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T07:11:34.770 INFO:teuthology.orchestra.run.smithi027.stdout:38654705756 2024-09-21T07:11:34.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T07:11:35.211 INFO:teuthology.orchestra.run.smithi027.stdout:266287972389 2024-09-21T07:11:35.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T07:11:35.261 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:34 smithi027 ceph-mon[30779]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:34 smithi119 ceph-mon[34519]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:35.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:34 smithi153 ceph-mon[34747]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:35.644 INFO:teuthology.orchestra.run.smithi027.stdout:287762808863 2024-09-21T07:11:35.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 -- 172.21.15.27:0/3372642871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c1049b0 msgr2=0x7fab6c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 --2- 172.21.15.27:0/3372642871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c1049b0 0x7fab6c104dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009900 tx=0x7fab5c02a410 comp rx=0 tx=0).stop 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 -- 172.21.15.27:0/3372642871 shutdown_connections 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 --2- 172.21.15.27:0/3372642871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab6c10de10 0x7fab6c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 --2- 172.21.15.27:0/3372642871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab6c105300 0x7fab6c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 --2- 172.21.15.27:0/3372642871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c1049b0 0x7fab6c104dc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 -- 172.21.15.27:0/3372642871 >> 172.21.15.27:0/3372642871 conn(0x7fab6c079760 msgr2=0x7fab6c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 -- 172.21.15.27:0/3372642871 shutdown_connections 2024-09-21T07:11:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.470+0000 7fab72000640 1 -- 172.21.15.27:0/3372642871 wait complete. 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.471+0000 7fab72000640 1 Processor -- start 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.471+0000 7fab72000640 1 -- start start 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab72000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab6c1049b0 0x7fab6c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab72000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c105300 0x7fab6c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab72000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab6c10de10 0x7fab6c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab72000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c111880 con 0x7fab6c105300 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab72000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c111700 con 0x7fab6c1049b0 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab72000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c111a00 con 0x7fab6c10de10 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c105300 0x7fab6c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c105300 0x7fab6c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35242/0 (socket says 172.21.15.27:35242) 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab62a00640 1 -- 172.21.15.27:0/313305654 learned_addr learned my addr 172.21.15.27:0/313305654 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab6b400640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab6c1049b0 0x7fab6c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:35.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.472+0000 7fab6be00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab6c10de10 0x7fab6c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab62a00640 1 -- 172.21.15.27:0/313305654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab6c10de10 msgr2=0x7fab6c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab62a00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab6c10de10 0x7fab6c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab62a00640 1 -- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab6c1049b0 msgr2=0x7fab6c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab62a00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab6c1049b0 0x7fab6c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab62a00640 1 -- 172.21.15.27:0/313305654 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c19e220 con 0x7fab6c105300 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab6be00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab6c10de10 0x7fab6c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab6b400640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab6c1049b0 0x7fab6c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.473+0000 7fab62a00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c105300 0x7fab6c198850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab54009be0 tx=0x7fab54007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.474+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54007e20 con 0x7fab6c105300 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.474+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab5401e070 con 0x7fab6c105300 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.474+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540186b0 con 0x7fab6c105300 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.474+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6c19e510 con 0x7fab6c105300 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.474+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6c1a4c70 con 0x7fab6c105300 2024-09-21T07:11:35.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.475+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab6c063b50 con 0x7fab6c105300 2024-09-21T07:11:35.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.478+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fab540193f0 con 0x7fab6c105300 2024-09-21T07:11:35.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.480+0000 7fab68c00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab500b6670 0x7fab500b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:35.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.480+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fab540ebc90 con 0x7fab6c105300 2024-09-21T07:11:35.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.480+0000 7fab6b400640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab500b6670 0x7fab500b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:35.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.481+0000 7fab68c00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] conn(0x7fab500c1a00 0x7fab500c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:35.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.481+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 --> [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab5401dec0 con 0x7fab500c1a00 2024-09-21T07:11:35.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.481+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fab540ec0e0 con 0x7fab6c105300 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.481+0000 7fab6be00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] conn(0x7fab500c1a00 0x7fab500c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.481+0000 7fab6b400640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab500b6670 0x7fab500b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab5c031440 tx=0x7fab5c032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.482+0000 7fab6be00640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] conn(0x7fab500c1a00 0x7fab500c3ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.482+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== osd.4 v2:172.21.15.119:6808/2014804508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab5401dec0 con 0x7fab500c1a00 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.497+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 --> [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab6c063f50 con 0x7fab500c1a00 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab68c00640 1 -- 172.21.15.27:0/313305654 <== osd.4 v2:172.21.15.119:6808/2014804508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab6c063f50 con 0x7fab500c1a00 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] conn(0x7fab500c1a00 msgr2=0x7fab500c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab72000640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] conn(0x7fab500c1a00 0x7fab500c3ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab500b6670 msgr2=0x7fab500b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab72000640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab500b6670 0x7fab500b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab5c031440 tx=0x7fab5c032ed0 comp rx=0 tx=0).stop 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c105300 msgr2=0x7fab6c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab72000640 1 --2- 172.21.15.27:0/313305654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab6c105300 0x7fab6c198850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab54009be0 tx=0x7fab54007590 comp rx=0 tx=0).stop 2024-09-21T07:11:35.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.498+0000 7fab6be00640 1 -- 172.21.15.27:0/313305654 reap_dead start 2024-09-21T07:11:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.499+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 shutdown_connections 2024-09-21T07:11:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.499+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 >> 172.21.15.27:0/313305654 conn(0x7fab6c079760 msgr2=0x7fab6c100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.499+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 shutdown_connections 2024-09-21T07:11:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:29.499+0000 7fab72000640 1 -- 172.21.15.27:0/313305654 wait complete. 2024-09-21T07:11:35.924 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:36.483 INFO:teuthology.orchestra.run.smithi027.stdout:128849018949 2024-09-21T07:11:36.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T07:11:37.062 INFO:teuthology.orchestra.run.smithi027.stdout:244813135915 2024-09-21T07:11:37.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T07:11:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:36 smithi119 ceph-mon[34519]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:37.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:36 smithi027 ceph-mon[30779]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:37.355 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:37.379 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.408+0000 7f673c800640 1 -- 172.21.15.27:0/420164911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6734109e00 msgr2=0x7f6734110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.408+0000 7f673c800640 1 --2- 172.21.15.27:0/420164911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6734109e00 0x7f6734110630 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6730009ca0 tx=0x7f673002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.409+0000 7f673c800640 1 -- 172.21.15.27:0/420164911 shutdown_connections 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.409+0000 7f673c800640 1 --2- 172.21.15.27:0/420164911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67341094b0 0x7f67341098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.409+0000 7f673c800640 1 --2- 172.21.15.27:0/420164911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6734109e00 0x7f6734110630 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.409+0000 7f673c800640 1 --2- 172.21.15.27:0/420164911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6734103b70 0x7f6734103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.409+0000 7f673c800640 1 -- 172.21.15.27:0/420164911 >> 172.21.15.27:0/420164911 conn(0x7f67340ff5e0 msgr2=0x7f6734101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.409+0000 7f673c800640 1 -- 172.21.15.27:0/420164911 shutdown_connections 2024-09-21T07:11:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.409+0000 7f673c800640 1 -- 172.21.15.27:0/420164911 wait complete. 2024-09-21T07:11:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.411+0000 7f673c800640 1 Processor -- start 2024-09-21T07:11:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.411+0000 7f673c800640 1 -- start start 2024-09-21T07:11:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.412+0000 7f673c800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6734103b70 0x7f6734075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.412+0000 7f673c800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67341094b0 0x7f6734075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.412+0000 7f673c800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6734109e00 0x7f6734076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.412+0000 7f673c800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6734113ac0 con 0x7f67341094b0 2024-09-21T07:11:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.412+0000 7f673c800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6734113940 con 0x7f6734103b70 2024-09-21T07:11:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.412+0000 7f673c800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6734113c40 con 0x7f6734109e00 2024-09-21T07:11:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.413+0000 7f6739400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67341094b0 0x7f6734075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.413+0000 7f6739400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67341094b0 0x7f6734075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35316/0 (socket says 172.21.15.27:35316) 2024-09-21T07:11:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.413+0000 7f6739400640 1 -- 172.21.15.27:0/2423077253 learned_addr learned my addr 172.21.15.27:0/2423077253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.413+0000 7f673a800640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6734109e00 0x7f6734076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.414+0000 7f6739400640 1 -- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6734109e00 msgr2=0x7f6734076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.414+0000 7f6739400640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6734109e00 0x7f6734076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.414+0000 7f6739400640 1 -- 172.21.15.27:0/2423077253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6734103b70 msgr2=0x7f6734075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.414+0000 7f6739400640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6734103b70 0x7f6734075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.414+0000 7f6739400640 1 -- 172.21.15.27:0/2423077253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67341a8710 con 0x7f67341094b0 2024-09-21T07:11:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.414+0000 7f6739400640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67341094b0 0x7f6734075c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f672400d740 tx=0x7f672400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.415+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724017070 con 0x7f67341094b0 2024-09-21T07:11:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.415+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f672400de50 con 0x7f67341094b0 2024-09-21T07:11:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.415+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400f620 con 0x7f67341094b0 2024-09-21T07:11:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.415+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67341a8a00 con 0x7f67341094b0 2024-09-21T07:11:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.415+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67341a8ef0 con 0x7f67341094b0 2024-09-21T07:11:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.417+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6734063b10 con 0x7f67341094b0 2024-09-21T07:11:37.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.421+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f672401a070 con 0x7f67341094b0 2024-09-21T07:11:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.424+0000 7f672ea00640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f671c0b6740 0x7f671c0b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.424+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f67240e7680 con 0x7f67341094b0 2024-09-21T07:11:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.424+0000 7f672ea00640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] conn(0x7f671c0c1ad0 0x7f671c0c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.424+0000 7f6739e00640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f671c0b6740 0x7f671c0b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.424+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 --> [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f671c0c4690 con 0x7f671c0c1ad0 2024-09-21T07:11:37.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.424+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f67240e7ad0 con 0x7f67341094b0 2024-09-21T07:11:37.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.425+0000 7f673a800640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] conn(0x7f671c0c1ad0 0x7f671c0c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:36 smithi153 ceph-mon[34747]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.425+0000 7f6739e00640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f671c0b6740 0x7f671c0b8c50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f67280058c0 tx=0x7f6728005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.426+0000 7f673a800640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] conn(0x7f671c0c1ad0 0x7f671c0c3fc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.427+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== osd.5 v2:172.21.15.153:6808/2063779127 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f671c0c4690 con 0x7f671c0c1ad0 2024-09-21T07:11:37.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.445+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 --> [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6734063fe0 con 0x7f671c0c1ad0 2024-09-21T07:11:37.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.446+0000 7f672ea00640 1 -- 172.21.15.27:0/2423077253 <== osd.5 v2:172.21.15.153:6808/2063779127 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6734063fe0 con 0x7f671c0c1ad0 2024-09-21T07:11:37.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.447+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] conn(0x7f671c0c1ad0 msgr2=0x7f671c0c3fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:37.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.447+0000 7f673c800640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] conn(0x7f671c0c1ad0 0x7f671c0c3fc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.448+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f671c0b6740 msgr2=0x7f671c0b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.448+0000 7f673c800640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f671c0b6740 0x7f671c0b8c50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f67280058c0 tx=0x7f6728005850 comp rx=0 tx=0).stop 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.448+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67341094b0 msgr2=0x7f6734075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.448+0000 7f673c800640 1 --2- 172.21.15.27:0/2423077253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67341094b0 0x7f6734075c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f672400d740 tx=0x7f672400dc00 comp rx=0 tx=0).stop 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.448+0000 7f673a800640 1 -- 172.21.15.27:0/2423077253 reap_dead start 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.448+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 shutdown_connections 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.448+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 >> 172.21.15.27:0/2423077253 conn(0x7f67340ff5e0 msgr2=0x7f6734106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.449+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 shutdown_connections 2024-09-21T07:11:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:37.449+0000 7f673c800640 1 -- 172.21.15.27:0/2423077253 wait complete. 2024-09-21T07:11:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:37.492 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 -- 172.21.15.27:0/51801076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec075750 msgr2=0x7faaec075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 --2- 172.21.15.27:0/51801076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec075750 0x7faaec075b60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faae0009900 tx=0x7faae002a410 comp rx=0 tx=0).stop 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 -- 172.21.15.27:0/51801076 shutdown_connections 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 --2- 172.21.15.27:0/51801076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faaec110120 0x7faaec112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 --2- 172.21.15.27:0/51801076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec0760a0 0x7faaec10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 --2- 172.21.15.27:0/51801076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec075750 0x7faaec075b60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 -- 172.21.15.27:0/51801076 >> 172.21.15.27:0/51801076 conn(0x7faaec07b850 msgr2=0x7faaec07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 -- 172.21.15.27:0/51801076 shutdown_connections 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.286+0000 7faaf3e00640 1 -- 172.21.15.27:0/51801076 wait complete. 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.288+0000 7faaf3e00640 1 Processor -- start 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.288+0000 7faaf3e00640 1 -- start start 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.288+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec075750 0x7faaec19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faaec0760a0 0x7faaec19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec110120 0x7faaec1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faaec115ce0 con 0x7faaec110120 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faaec115b60 con 0x7faaec075750 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faaec115e60 con 0x7faaec0760a0 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec110120 0x7faaec1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec075750 0x7faaec19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec075750 0x7faaec19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36032/0 (socket says 172.21.15.27:36032) 2024-09-21T07:11:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec110120 0x7faaec1a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38330/0 (socket says 172.21.15.27:38330) 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 -- 172.21.15.27:0/4076268028 learned_addr learned my addr 172.21.15.27:0/4076268028 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf0a00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faaec0760a0 0x7faaec19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 -- 172.21.15.27:0/4076268028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faaec0760a0 msgr2=0x7faaec19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faaec0760a0 0x7faaec19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 -- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec110120 msgr2=0x7faaec1a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec110120 0x7faaec1a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.289+0000 7faaf1400640 1 -- 172.21.15.27:0/4076268028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec1a2700 con 0x7faaec075750 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.290+0000 7faaf1400640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec075750 0x7faaec19c790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faae0005ab0 tx=0x7faae0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.290+0000 7faaea000640 1 -- 172.21.15.27:0/4076268028 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae003f070 con 0x7faaec075750 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.290+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec1a2990 con 0x7faaec075750 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.290+0000 7faaea000640 1 -- 172.21.15.27:0/4076268028 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae0031b60 con 0x7faaec075750 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.290+0000 7faaea000640 1 -- 172.21.15.27:0/4076268028 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0035030 con 0x7faaec075750 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.290+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1a9150 con 0x7faaec075750 2024-09-21T07:11:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.291+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec075b60 con 0x7faaec075750 2024-09-21T07:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.294+0000 7faaea000640 1 -- 172.21.15.27:0/4076268028 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7faae0042070 con 0x7faaec075750 2024-09-21T07:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.295+0000 7faaea000640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faad40b65f0 0x7faad40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.296+0000 7faaea000640 1 -- 172.21.15.27:0/4076268028 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7faae0108730 con 0x7faaec075750 2024-09-21T07:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.296+0000 7faaf0a00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faad40b65f0 0x7faad40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.296+0000 7faaf0a00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faad40b65f0 0x7faad40b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faaec111f90 tx=0x7faad8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.297+0000 7faaea000640 1 -- 172.21.15.27:0/4076268028 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faae00d2db0 con 0x7faaec075750 2024-09-21T07:11:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.503+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7faaec1a9320 con 0x7faaec075750 2024-09-21T07:11:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.503+0000 7faaea000640 1 -- 172.21.15.27:0/4076268028 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faae00d7cf0 con 0x7faaec075750 2024-09-21T07:11:38.506 INFO:teuthology.orchestra.run.smithi027.stdout:103079215179 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faad40b65f0 msgr2=0x7faad40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faad40b65f0 0x7faad40b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faaec111f90 tx=0x7faad8005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec075750 msgr2=0x7faaec19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec075750 0x7faaec19c790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faae0005ab0 tx=0x7faae0003c10 comp rx=0 tx=0).stop 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 shutdown_connections 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faaec0760a0 0x7faaec19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faad40b65f0 0x7faad40b8b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaec110120 0x7faaec1a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 --2- 172.21.15.27:0/4076268028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaec075750 0x7faaec19c790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 >> 172.21.15.27:0/4076268028 conn(0x7faaec07b850 msgr2=0x7faaec108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 shutdown_connections 2024-09-21T07:11:38.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:38.506+0000 7faaf3e00640 1 -- 172.21.15.27:0/4076268028 wait complete. 2024-09-21T07:11:39.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:38 smithi027 ceph-mon[30779]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:11:39.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:38 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4076268028' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T07:11:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:38 smithi119 ceph-mon[34519]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:11:39.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:38 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4076268028' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T07:11:39.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.420+0000 7f41d0400640 1 -- 172.21.15.27:0/3977979706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8109df0 msgr2=0x7f41c8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:39.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.420+0000 7f41d0400640 1 --2- 172.21.15.27:0/3977979706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8109df0 0x7f41c8110620 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009900 tx=0x7f41c002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.421+0000 7f41d0400640 1 -- 172.21.15.27:0/3977979706 shutdown_connections 2024-09-21T07:11:39.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.421+0000 7f41d0400640 1 --2- 172.21.15.27:0/3977979706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41c81094a0 0x7f41c81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.421+0000 7f41d0400640 1 --2- 172.21.15.27:0/3977979706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8109df0 0x7f41c8110620 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.421+0000 7f41d0400640 1 --2- 172.21.15.27:0/3977979706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c8103b60 0x7f41c8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.421+0000 7f41d0400640 1 -- 172.21.15.27:0/3977979706 >> 172.21.15.27:0/3977979706 conn(0x7f41c807b850 msgr2=0x7f41c807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:39.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.421+0000 7f41d0400640 1 -- 172.21.15.27:0/3977979706 shutdown_connections 2024-09-21T07:11:39.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.421+0000 7f41d0400640 1 -- 172.21.15.27:0/3977979706 wait complete. 2024-09-21T07:11:39.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.422+0000 7f41d0400640 1 Processor -- start 2024-09-21T07:11:39.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41d0400640 1 -- start start 2024-09-21T07:11:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41d0400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8103b60 0x7f41c819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41d0400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41c81094a0 0x7f41c819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41d0400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c8109df0 0x7f41c819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41d0400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f41c8113b00 con 0x7f41c8103b60 2024-09-21T07:11:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41d0400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41c8113980 con 0x7f41c8109df0 2024-09-21T07:11:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41d0400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41c8113c80 con 0x7f41c81094a0 2024-09-21T07:11:39.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41cda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8103b60 0x7f41c819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:39.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41cda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8103b60 0x7f41c819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38350/0 (socket says 172.21.15.27:38350) 2024-09-21T07:11:39.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41cda00640 1 -- 172.21.15.27:0/3038715698 learned_addr learned my addr 172.21.15.27:0/3038715698 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:39.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41cd000640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41c81094a0 0x7f41c819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:39.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.423+0000 7f41ce400640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c8109df0 0x7f41c819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:39.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.424+0000 7f41cda00640 1 -- 172.21.15.27:0/3038715698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41c81094a0 msgr2=0x7f41c819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:39.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.424+0000 7f41cda00640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41c81094a0 0x7f41c819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.424+0000 7f41cda00640 1 -- 172.21.15.27:0/3038715698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c8109df0 msgr2=0x7f41c819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:39.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.424+0000 7f41cda00640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c8109df0 0x7f41c819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.424+0000 7f41cda00640 1 -- 172.21.15.27:0/3038715698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c81a0360 con 0x7f41c8103b60 2024-09-21T07:11:39.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.424+0000 7f41cd000640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41c81094a0 0x7f41c819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:39.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.424+0000 7f41cda00640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8103b60 0x7f41c819a3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00db80 tx=0x7f41bc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:38 smithi153 ceph-mon[34747]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:11:39.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:38 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4076268028' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T07:11:39.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.425+0000 7f41c6000640 1 -- 172.21.15.27:0/3038715698 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc017070 con 0x7f41c8103b60 2024-09-21T07:11:39.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.425+0000 7f41c6000640 1 -- 172.21.15.27:0/3038715698 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41bc003b70 con 0x7f41c8103b60 2024-09-21T07:11:39.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.425+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c81a0650 con 0x7f41c8103b60 2024-09-21T07:11:39.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.425+0000 7f41c6000640 1 -- 172.21.15.27:0/3038715698 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc0106c0 con 0x7f41c8103b60 2024-09-21T07:11:39.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.425+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c80755f0 con 0x7f41c8103b60 2024-09-21T07:11:39.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.426+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c8103f70 con 0x7f41c8103b60 2024-09-21T07:11:39.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.428+0000 7f41c6000640 1 -- 172.21.15.27:0/3038715698 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f41bc004020 con 0x7f41c8103b60 2024-09-21T07:11:39.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.430+0000 7f41c6000640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41b00b65f0 0x7f41b00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:39.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.430+0000 7f41c6000640 1 -- 172.21.15.27:0/3038715698 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f41bc0e7980 con 0x7f41c8103b60 2024-09-21T07:11:39.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.430+0000 7f41cd000640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41b00b65f0 0x7f41b00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:39.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.431+0000 7f41cd000640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41b00b65f0 0x7f41b00b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41c8109130 tx=0x7f41b400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.434+0000 7f41c6000640 1 -- 172.21.15.27:0/3038715698 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41bc0adc20 con 0x7f41c8103b60 2024-09-21T07:11:39.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.639+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f41c819b520 con 0x7f41c8103b60 2024-09-21T07:11:39.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.640+0000 7f41c6000640 1 -- 172.21.15.27:0/3038715698 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41bc0b2000 con 0x7f41c8103b60 2024-09-21T07:11:39.642 INFO:teuthology.orchestra.run.smithi027.stdout:176093659196 2024-09-21T07:11:39.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.642+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41b00b65f0 msgr2=0x7f41b00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.642+0000 7f41d0400640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41b00b65f0 0x7f41b00b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41c8109130 tx=0x7f41b400b410 comp rx=0 tx=0).stop 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.642+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8103b60 msgr2=0x7f41c819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.642+0000 7f41d0400640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8103b60 0x7f41c819a3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00db80 tx=0x7f41bc00b630 comp rx=0 tx=0).stop 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.642+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 shutdown_connections 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.642+0000 7f41d0400640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41c81094a0 0x7f41c819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.642+0000 7f41d0400640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41b00b65f0 0x7f41b00b8b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.643+0000 7f41d0400640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41c8109df0 0x7f41c819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.643+0000 7f41d0400640 1 --2- 172.21.15.27:0/3038715698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41c8103b60 0x7f41c819a3f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.643+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 >> 172.21.15.27:0/3038715698 conn(0x7f41c807b850 msgr2=0x7f41c8107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.643+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 shutdown_connections 2024-09-21T07:11:39.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.643+0000 7f41d0400640 1 -- 172.21.15.27:0/3038715698 wait complete. 2024-09-21T07:11:40.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 -- 172.21.15.27:0/2802256022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc105d80 msgr2=0x7f3ddc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:40.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2802256022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc105d80 0x7f3ddc106190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009900 tx=0x7f3dcc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 -- 172.21.15.27:0/2802256022 shutdown_connections 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2802256022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ddc10b6c0 0x7f3ddc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2802256022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ddc10c010 0x7f3ddc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2802256022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc105d80 0x7f3ddc106190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 -- 172.21.15.27:0/2802256022 >> 172.21.15.27:0/2802256022 conn(0x7f3ddc101810 msgr2=0x7f3ddc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 -- 172.21.15.27:0/2802256022 shutdown_connections 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.850+0000 7f3de2a00640 1 -- 172.21.15.27:0/2802256022 wait complete. 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.851+0000 7f3de2a00640 1 Processor -- start 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.851+0000 7f3de2a00640 1 -- start start 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3de2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ddc105d80 0x7f3ddc19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3de2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc10b6c0 0x7f3ddc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3de2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ddc10c010 0x7f3ddc1a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3de2a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc115d20 con 0x7f3ddc10b6c0 2024-09-21T07:11:40.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3de2a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc115ba0 con 0x7f3ddc105d80 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3de2a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc115ea0 con 0x7f3ddc10c010 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3dd3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc10b6c0 0x7f3ddc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3ddbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ddc105d80 0x7f3ddc19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3dd3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc10b6c0 0x7f3ddc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38364/0 (socket says 172.21.15.27:38364) 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3dd3e00640 1 -- 172.21.15.27:0/2286261729 learned_addr learned my addr 172.21.15.27:0/2286261729 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.852+0000 7f3de0a00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ddc10c010 0x7f3ddc1a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3ddbe00640 1 -- 172.21.15.27:0/2286261729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ddc10c010 msgr2=0x7f3ddc1a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3ddbe00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ddc10c010 0x7f3ddc1a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3ddbe00640 1 -- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc10b6c0 msgr2=0x7f3ddc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3ddbe00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc10b6c0 0x7f3ddc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3ddbe00640 1 -- 172.21.15.27:0/2286261729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc1a26a0 con 0x7f3ddc105d80 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3dd3e00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc10b6c0 0x7f3ddc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3ddbe00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ddc105d80 0x7f3ddc19c730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc02a9e0 tx=0x7f3dcc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:40.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3dd9600640 1 -- 172.21.15.27:0/2286261729 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc03f070 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc1a2930 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc1a90f0 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3dd9600640 1 -- 172.21.15.27:0/2286261729 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dcc031b50 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.853+0000 7f3dd9600640 1 -- 172.21.15.27:0/2286261729 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc049060 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.855+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc106190 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.857+0000 7f3dd9600640 1 -- 172.21.15.27:0/2286261729 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3dcc042070 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.859+0000 7f3dd9600640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3dc00b65f0 0x7f3dc00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.859+0000 7f3dd9600640 1 -- 172.21.15.27:0/2286261729 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f3dcc107e20 con 0x7f3ddc105d80 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.859+0000 7f3dd3e00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3dc00b65f0 0x7f3dc00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.860+0000 7f3dd3e00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3dc00b65f0 0x7f3dc00b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc10b160 tx=0x7f3dc4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:40.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:39.860+0000 7f3dd9600640 1 -- 172.21.15.27:0/2286261729 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3dcc0ce0c0 con 0x7f3ddc105d80 2024-09-21T07:11:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.062+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f3ddc02e640 con 0x7f3ddc105d80 2024-09-21T07:11:40.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.063+0000 7f3dd9600640 1 -- 172.21.15.27:0/2286261729 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3dcc0d24a0 con 0x7f3ddc105d80 2024-09-21T07:11:40.066 INFO:teuthology.orchestra.run.smithi027.stdout:287762808866 2024-09-21T07:11:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3dc00b65f0 msgr2=0x7f3dc00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3dc00b65f0 0x7f3dc00b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc10b160 tx=0x7f3dc4005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ddc105d80 msgr2=0x7f3ddc19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ddc105d80 0x7f3ddc19c730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc02a9e0 tx=0x7f3dcc003b90 comp rx=0 tx=0).stop 2024-09-21T07:11:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 shutdown_connections 2024-09-21T07:11:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ddc10c010 0x7f3ddc1a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3dc00b65f0 0x7f3dc00b8b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ddc10b6c0 0x7f3ddc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 --2- 172.21.15.27:0/2286261729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ddc105d80 0x7f3ddc19c730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 >> 172.21.15.27:0/2286261729 conn(0x7f3ddc101810 msgr2=0x7f3ddc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 shutdown_connections 2024-09-21T07:11:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:40.066+0000 7f3de2a00640 1 -- 172.21.15.27:0/2286261729 wait complete. 2024-09-21T07:11:40.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:39 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3038715698' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T07:11:40.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:39 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3038715698' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T07:11:40.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:39 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3038715698' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T07:11:40.782 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215177 got 103079215179 for osd.3 2024-09-21T07:11:40.782 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:41.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:40 smithi027 ceph-mon[30779]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:41.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:40 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2286261729' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T07:11:41.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.144+0000 7fe46ae00640 1 -- 172.21.15.27:0/341345496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe464109c40 msgr2=0x7fe464110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.144+0000 7fe46ae00640 1 --2- 172.21.15.27:0/341345496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe464109c40 0x7fe464110470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009ca0 tx=0x7fe45c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.145+0000 7fe46ae00640 1 -- 172.21.15.27:0/341345496 shutdown_connections 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.145+0000 7fe46ae00640 1 --2- 172.21.15.27:0/341345496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4641092f0 0x7fe464109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.145+0000 7fe46ae00640 1 --2- 172.21.15.27:0/341345496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe464109c40 0x7fe464110470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.145+0000 7fe46ae00640 1 --2- 172.21.15.27:0/341345496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 0x7fe464103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.145+0000 7fe46ae00640 1 -- 172.21.15.27:0/341345496 >> 172.21.15.27:0/341345496 conn(0x7fe4640ff440 msgr2=0x7fe464101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.145+0000 7fe46ae00640 1 -- 172.21.15.27:0/341345496 shutdown_connections 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.145+0000 7fe46ae00640 1 -- 172.21.15.27:0/341345496 wait complete. 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 Processor -- start 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 -- start start 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 0x7fe46419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641092f0 0x7fe46419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe464109c40 0x7fe4641a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe464113950 con 0x7fe4641092f0 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe4641137d0 con 0x7fe4641039b0 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.146+0000 7fe46ae00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe464113ad0 con 0x7fe464109c40 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 0x7fe46419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 0x7fe46419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36090/0 (socket says 172.21.15.27:36090) 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 -- 172.21.15.27:0/3569191352 learned_addr learned my addr 172.21.15.27:0/3569191352 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469000640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641092f0 0x7fe46419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe46a400640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe464109c40 0x7fe4641a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 -- 172.21.15.27:0/3569191352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe464109c40 msgr2=0x7fe4641a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe464109c40 0x7fe4641a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 -- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641092f0 msgr2=0x7fe46419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641092f0 0x7fe46419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.147+0000 7fe469a00640 1 -- 172.21.15.27:0/3569191352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4641a2450 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.148+0000 7fe469a00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 0x7fe46419c4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe45800e9f0 tx=0x7fe45800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.148+0000 7fe462000640 1 -- 172.21.15.27:0/3569191352 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45800cd80 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.148+0000 7fe462000640 1 -- 172.21.15.27:0/3569191352 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4580108e0 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.148+0000 7fe462000640 1 -- 172.21.15.27:0/3569191352 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458018980 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.148+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4641a2740 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.148+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4641a8ea0 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.149+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe464103dc0 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.152+0000 7fe462000640 1 -- 172.21.15.27:0/3569191352 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe458010430 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.153+0000 7fe462000640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe44c0b65f0 0x7fe44c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.153+0000 7fe462000640 1 -- 172.21.15.27:0/3569191352 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fe4580e8140 con 0x7fe4641039b0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.153+0000 7fe469000640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe44c0b65f0 0x7fe44c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.154+0000 7fe469000640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe44c0b65f0 0x7fe44c0b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe464105a10 tx=0x7fe450005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:41.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.155+0000 7fe462000640 1 -- 172.21.15.27:0/3569191352 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4580b27c0 con 0x7fe4641039b0 2024-09-21T07:11:41.210 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:41.213 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:40 smithi119 ceph-mon[34519]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:40 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2286261729' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 -- 172.21.15.27:0/4029324453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9880757b0 msgr2=0x7fd988075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 --2- 172.21.15.27:0/4029324453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9880757b0 0x7fd988075bc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009900 tx=0x7fd97c02a410 comp rx=0 tx=0).stop 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 -- 172.21.15.27:0/4029324453 shutdown_connections 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 --2- 172.21.15.27:0/4029324453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd988076100 0x7fd98810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 --2- 172.21.15.27:0/4029324453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9881100d0 0x7fd988112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 --2- 172.21.15.27:0/4029324453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9880757b0 0x7fd988075bc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 -- 172.21.15.27:0/4029324453 >> 172.21.15.27:0/4029324453 conn(0x7fd9880ff5e0 msgr2=0x7fd988101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 -- 172.21.15.27:0/4029324453 shutdown_connections 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.262+0000 7fd98f600640 1 -- 172.21.15.27:0/4029324453 wait complete. 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.264+0000 7fd98f600640 1 Processor -- start 2024-09-21T07:11:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.264+0000 7fd98f600640 1 -- start start 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.264+0000 7fd98f600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9880757b0 0x7fd98819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.264+0000 7fd98f600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd988076100 0x7fd98819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.264+0000 7fd98f600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9881100d0 0x7fd9881a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98f600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd988115c80 con 0x7fd988076100 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98f600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd988115b00 con 0x7fd9881100d0 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98f600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd988115e00 con 0x7fd9880757b0 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd987e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd988076100 0x7fd98819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9880757b0 0x7fd98819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd987e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd988076100 0x7fd98819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38406/0 (socket says 172.21.15.27:38406) 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd987e00640 1 -- 172.21.15.27:0/1009590379 learned_addr learned my addr 172.21.15.27:0/1009590379 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98d600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9881100d0 0x7fd9881a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98cc00640 1 -- 172.21.15.27:0/1009590379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9881100d0 msgr2=0x7fd9881a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98cc00640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9881100d0 0x7fd9881a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98cc00640 1 -- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd988076100 msgr2=0x7fd98819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98cc00640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd988076100 0x7fd98819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98cc00640 1 -- 172.21.15.27:0/1009590379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9881a25c0 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.265+0000 7fd98d600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9881100d0 0x7fd9881a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.266+0000 7fd98cc00640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9880757b0 0x7fd98819c650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd97c02a930 tx=0x7fd97c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.266+0000 7fd985600640 1 -- 172.21.15.27:0/1009590379 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c03f030 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.266+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9881a2850 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.266+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881a9010 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.266+0000 7fd985600640 1 -- 172.21.15.27:0/1009590379 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd97c004490 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.266+0000 7fd985600640 1 -- 172.21.15.27:0/1009590379 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c035070 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.267+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9680052c0 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.271+0000 7fd985600640 1 -- 172.21.15.27:0/1009590379 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd97c042030 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.272+0000 7fd985600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9700b65f0 0x7fd9700b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.273+0000 7fd985600640 1 -- 172.21.15.27:0/1009590379 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fd97c107fe0 con 0x7fd9880757b0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.273+0000 7fd987e00640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9700b65f0 0x7fd9700b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.273+0000 7fd987e00640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9700b65f0 0x7fd9700b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd988112430 tx=0x7fd974005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.274+0000 7fd985600640 1 -- 172.21.15.27:0/1009590379 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd97c049030 con 0x7fd9880757b0 2024-09-21T07:11:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.383+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe4641a9250 con 0x7fe4641039b0 2024-09-21T07:11:41.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.383+0000 7fe462000640 1 -- 172.21.15.27:0/3569191352 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe4580b7700 con 0x7fe4641039b0 2024-09-21T07:11:41.386 INFO:teuthology.orchestra.run.smithi027.stdout:197568495670 2024-09-21T07:11:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe44c0b65f0 msgr2=0x7fe44c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe44c0b65f0 0x7fe44c0b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe464105a10 tx=0x7fe450005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 msgr2=0x7fe46419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 0x7fe46419c4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe45800e9f0 tx=0x7fe45800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 shutdown_connections 2024-09-21T07:11:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe464109c40 0x7fe4641a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe44c0b65f0 0x7fe44c0b8b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641092f0 0x7fe46419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 --2- 172.21.15.27:0/3569191352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4641039b0 0x7fe46419c4e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 >> 172.21.15.27:0/3569191352 conn(0x7fe4640ff440 msgr2=0x7fe464108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 shutdown_connections 2024-09-21T07:11:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.386+0000 7fe46ae00640 1 -- 172.21.15.27:0/3569191352 wait complete. 2024-09-21T07:11:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:40 smithi153 ceph-mon[34747]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:40 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2286261729' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T07:11:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.485+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd968005880 con 0x7fd9880757b0 2024-09-21T07:11:41.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.486+0000 7fd985600640 1 -- 172.21.15.27:0/1009590379 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd97c0ce2c0 con 0x7fd9880757b0 2024-09-21T07:11:41.488 INFO:teuthology.orchestra.run.smithi027.stdout:60129542231 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.488+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9700b65f0 msgr2=0x7fd9700b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.488+0000 7fd98f600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9700b65f0 0x7fd9700b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd988112430 tx=0x7fd974005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.488+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9880757b0 msgr2=0x7fd98819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.488+0000 7fd98f600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9880757b0 0x7fd98819c650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd97c02a930 tx=0x7fd97c031a20 comp rx=0 tx=0).stop 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.488+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 shutdown_connections 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.488+0000 7fd98f600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9700b65f0 0x7fd9700b8b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.489+0000 7fd98f600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9881100d0 0x7fd9881a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.489+0000 7fd98f600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd988076100 0x7fd98819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.489+0000 7fd98f600640 1 --2- 172.21.15.27:0/1009590379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9880757b0 0x7fd98819c650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.489+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 >> 172.21.15.27:0/1009590379 conn(0x7fd9880ff5e0 msgr2=0x7fd988109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.489+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 shutdown_connections 2024-09-21T07:11:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:41.489+0000 7fd98f600640 1 -- 172.21.15.27:0/1009590379 wait complete. 2024-09-21T07:11:42.059 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3569191352' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T07:11:42.060 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1009590379' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T07:11:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3569191352' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T07:11:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1009590379' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T07:11:42.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3569191352' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T07:11:42.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1009590379' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T07:11:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:42.740 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659193 got 176093659196 for osd.6 2024-09-21T07:11:42.740 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:43.189 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542228 got 60129542231 for osd.1 2024-09-21T07:11:43.190 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:43.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:42 smithi027 ceph-mon[30779]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:42 smithi119 ceph-mon[34519]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:43.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:42 smithi153 ceph-mon[34747]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:43.694 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495669 got 197568495670 for osd.7 2024-09-21T07:11:43.695 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.946+0000 7f0635000640 1 -- 172.21.15.27:0/2921787692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 msgr2=0x7f0630116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.946+0000 7f0635000640 1 --2- 172.21.15.27:0/2921787692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 0x7f0630116b60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0628009ca0 tx=0x7f062802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.946+0000 7f0635000640 1 -- 172.21.15.27:0/2921787692 shutdown_connections 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.946+0000 7f0635000640 1 --2- 172.21.15.27:0/2921787692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0630078290 0x7f0630114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.946+0000 7f0635000640 1 --2- 172.21.15.27:0/2921787692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 0x7f0630116b60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.946+0000 7f0635000640 1 --2- 172.21.15.27:0/2921787692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 0x7f0630077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.946+0000 7f0635000640 1 -- 172.21.15.27:0/2921787692 >> 172.21.15.27:0/2921787692 conn(0x7f0630101810 msgr2=0x7f0630103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.947+0000 7f0635000640 1 -- 172.21.15.27:0/2921787692 shutdown_connections 2024-09-21T07:11:43.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.947+0000 7f0635000640 1 -- 172.21.15.27:0/2921787692 wait complete. 2024-09-21T07:11:43.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.948+0000 7f0635000640 1 Processor -- start 2024-09-21T07:11:43.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.948+0000 7f0635000640 1 -- start start 2024-09-21T07:11:43.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.948+0000 7f0635000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 0x7f06301a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:43.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.948+0000 7f0635000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0630078290 0x7f06301a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:43.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f0635000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 0x7f06301a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:43.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f0635000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f063011a120 con 0x7f0630114740 2024-09-21T07:11:43.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f0635000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0630119fa0 con 0x7f0630077940 2024-09-21T07:11:43.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f0635000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f063011a2a0 con 0x7f0630078290 2024-09-21T07:11:43.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 0x7f06301a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 0x7f06301a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36142/0 (socket says 172.21.15.27:36142) 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062de00640 1 -- 172.21.15.27:0/4006153465 learned_addr learned my addr 172.21.15.27:0/4006153465 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062e800640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 0x7f06301a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f0627e00640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0630078290 0x7f06301a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062e800640 1 -- 172.21.15.27:0/4006153465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0630078290 msgr2=0x7f06301a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062e800640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0630078290 0x7f06301a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062e800640 1 -- 172.21.15.27:0/4006153465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 msgr2=0x7f06301a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062e800640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 0x7f06301a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062e800640 1 -- 172.21.15.27:0/4006153465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06301a69a0 con 0x7f0630114740 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.949+0000 7f062de00640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 0x7f06301a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.950+0000 7f062e800640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 0x7f06301a6270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0628005e30 tx=0x7f06280317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:43.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.950+0000 7f0626a00640 1 -- 172.21.15.27:0/4006153465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628003b70 con 0x7f0630114740 2024-09-21T07:11:43.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.950+0000 7f0626a00640 1 -- 172.21.15.27:0/4006153465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0628003d10 con 0x7f0630114740 2024-09-21T07:11:43.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.950+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06301a6c30 con 0x7f0630114740 2024-09-21T07:11:43.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.950+0000 7f0626a00640 1 -- 172.21.15.27:0/4006153465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062802fa00 con 0x7f0630114740 2024-09-21T07:11:43.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.950+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06301ad3f0 con 0x7f0630114740 2024-09-21T07:11:43.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.952+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0630077d50 con 0x7f0630114740 2024-09-21T07:11:43.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.954+0000 7f0626a00640 1 -- 172.21.15.27:0/4006153465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0628004580 con 0x7f0630114740 2024-09-21T07:11:43.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.956+0000 7f0626a00640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06140b65f0 0x7f06140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:43.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.956+0000 7f0626a00640 1 -- 172.21.15.27:0/4006153465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f0628108bc0 con 0x7f0630114740 2024-09-21T07:11:43.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.956+0000 7f062de00640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06140b65f0 0x7f06140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:43.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.957+0000 7f062de00640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06140b65f0 0x7f06140b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0620005fd0 tx=0x7f062000b480 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:43.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:43.958+0000 7f0626a00640 1 -- 172.21.15.27:0/4006153465 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06280cee60 con 0x7f0630114740 2024-09-21T07:11:44.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.168+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f06301a1b60 con 0x7f0630114740 2024-09-21T07:11:44.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.168+0000 7f0626a00640 1 -- 172.21.15.27:0/4006153465 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f06301a1b60 con 0x7f0630114740 2024-09-21T07:11:44.170 INFO:teuthology.orchestra.run.smithi027.stdout:219043332148 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06140b65f0 msgr2=0x7f06140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06140b65f0 0x7f06140b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0620005fd0 tx=0x7f062000b480 comp rx=0 tx=0).stop 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 msgr2=0x7f06301a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 0x7f06301a6270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0628005e30 tx=0x7f06280317c0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 shutdown_connections 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0630078290 0x7f06301a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06140b65f0 0x7f06140b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0630114740 0x7f06301a6270 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 --2- 172.21.15.27:0/4006153465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0630077940 0x7f06301a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 >> 172.21.15.27:0/4006153465 conn(0x7f0630101810 msgr2=0x7f0630116700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 shutdown_connections 2024-09-21T07:11:44.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.171+0000 7f0635000640 1 -- 172.21.15.27:0/4006153465 wait complete. 2024-09-21T07:11:44.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 -- 172.21.15.27:0/1521344477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc105bb0 msgr2=0x7fc1cc105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:44.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 --2- 172.21.15.27:0/1521344477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc105bb0 0x7fc1cc105fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009900 tx=0x7fc1bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 -- 172.21.15.27:0/1521344477 shutdown_connections 2024-09-21T07:11:44.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 --2- 172.21.15.27:0/1521344477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1cc10be40 0x7fc1cc112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 --2- 172.21.15.27:0/1521344477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1cc10b4f0 0x7fc1cc10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 --2- 172.21.15.27:0/1521344477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc105bb0 0x7fc1cc105fc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 -- 172.21.15.27:0/1521344477 >> 172.21.15.27:0/1521344477 conn(0x7fc1cc101680 msgr2=0x7fc1cc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 -- 172.21.15.27:0/1521344477 shutdown_connections 2024-09-21T07:11:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.388+0000 7fc1cb200640 1 -- 172.21.15.27:0/1521344477 wait complete. 2024-09-21T07:11:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.389+0000 7fc1cb200640 1 Processor -- start 2024-09-21T07:11:44.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1cb200640 1 -- start start 2024-09-21T07:11:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1cb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1cc105bb0 0x7fc1cc19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1cb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc10b4f0 0x7fc1cc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1cb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1cc10be40 0x7fc1cc1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1cb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc115b00 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1cb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc115980 con 0x7fc1cc105bb0 2024-09-21T07:11:44.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1cb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc115c80 con 0x7fc1cc10be40 2024-09-21T07:11:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1c9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc10b4f0 0x7fc1cc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1c9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1cc105bb0 0x7fc1cc19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1c9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc10b4f0 0x7fc1cc19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38426/0 (socket says 172.21.15.27:38426) 2024-09-21T07:11:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.390+0000 7fc1c9400640 1 -- 172.21.15.27:0/2913528613 learned_addr learned my addr 172.21.15.27:0/2913528613 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1ca800640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1cc10be40 0x7fc1cc1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1c9400640 1 -- 172.21.15.27:0/2913528613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1cc10be40 msgr2=0x7fc1cc1a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:44.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1c9400640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1cc10be40 0x7fc1cc1a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1c9400640 1 -- 172.21.15.27:0/2913528613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1cc105bb0 msgr2=0x7fc1cc19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1c9400640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1cc105bb0 0x7fc1cc19c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1c9400640 1 -- 172.21.15.27:0/2913528613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc1a24c0 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1c9e00640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1cc105bb0 0x7fc1cc19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.391+0000 7fc1c9400640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc10b4f0 0x7fc1cc19caf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800e9f0 tx=0x7fc1b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.392+0000 7fc1c2a00640 1 -- 172.21.15.27:0/2913528613 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800ccc0 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.392+0000 7fc1c2a00640 1 -- 172.21.15.27:0/2913528613 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b80108e0 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.392+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc1a27b0 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.392+0000 7fc1c2a00640 1 -- 172.21.15.27:0/2913528613 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8018860 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.392+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc1a8f10 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.393+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1cc105fc0 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.395+0000 7fc1c2a00640 1 -- 172.21.15.27:0/2913528613 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc1b8010430 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.398+0000 7fc1c2a00640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1b00b6670 0x7fc1b00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.398+0000 7fc1c2a00640 1 -- 172.21.15.27:0/2913528613 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fc1b80e8080 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.398+0000 7fc1c9e00640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1b00b6670 0x7fc1b00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.399+0000 7fc1c9e00640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1b00b6670 0x7fc1b00b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc031440 tx=0x7fc1bc032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:44.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.400+0000 7fc1c2a00640 1 -- 172.21.15.27:0/2913528613 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1b80b2700 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.603+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc1cc19d680 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.603+0000 7fc1c2a00640 1 -- 172.21.15.27:0/2913528613 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc1b80b7640 con 0x7fc1cc10b4f0 2024-09-21T07:11:44.605 INFO:teuthology.orchestra.run.smithi027.stdout:77309411411 2024-09-21T07:11:44.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1b00b6670 msgr2=0x7fc1b00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:44.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1b00b6670 0x7fc1b00b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc031440 tx=0x7fc1bc032ed0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc10b4f0 msgr2=0x7fc1cc19caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:44.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc10b4f0 0x7fc1cc19caf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800e9f0 tx=0x7fc1b800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 shutdown_connections 2024-09-21T07:11:44.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1cc10be40 0x7fc1cc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1b00b6670 0x7fc1b00b8b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc10b4f0 0x7fc1cc19caf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 --2- 172.21.15.27:0/2913528613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1cc105bb0 0x7fc1cc19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 >> 172.21.15.27:0/2913528613 conn(0x7fc1cc101680 msgr2=0x7fc1cc103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 shutdown_connections 2024-09-21T07:11:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:44.606+0000 7fc1cb200640 1 -- 172.21.15.27:0/2913528613 wait complete. 2024-09-21T07:11:44.880 INFO:teuthology.orchestra.run.smithi027.stdout:150323855427 2024-09-21T07:11:44.880 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T07:11:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:44 smithi027 ceph-mon[30779]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:11:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:44 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4006153465' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T07:11:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:44 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2913528613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T07:11:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:44 smithi119 ceph-mon[34519]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:11:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:44 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4006153465' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T07:11:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:44 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2913528613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T07:11:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:44 smithi153 ceph-mon[34747]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:11:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:44 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4006153465' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T07:11:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:44 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2913528613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T07:11:45.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411411 for osd.2 2024-09-21T07:11:45.671 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:46.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808863 got 287762808866 for osd.11 2024-09-21T07:11:46.143 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.878+0000 7f5b6f200640 1 -- 172.21.15.27:0/3196337960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68114280 msgr2=0x7f5b68116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.878+0000 7f5b6f200640 1 --2- 172.21.15.27:0/3196337960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68114280 0x7f5b68116750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009ca0 tx=0x7f5b6402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.879+0000 7f5b6f200640 1 -- 172.21.15.27:0/3196337960 shutdown_connections 2024-09-21T07:11:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.879+0000 7f5b6f200640 1 --2- 172.21.15.27:0/3196337960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b68075dd0 0x7f5b680761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.879+0000 7f5b6f200640 1 --2- 172.21.15.27:0/3196337960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68114280 0x7f5b68116750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.879+0000 7f5b6f200640 1 --2- 172.21.15.27:0/3196337960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b68077380 0x7f5b680757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.879+0000 7f5b6f200640 1 -- 172.21.15.27:0/3196337960 >> 172.21.15.27:0/3196337960 conn(0x7f5b680ff440 msgr2=0x7f5b68101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.879+0000 7f5b6f200640 1 -- 172.21.15.27:0/3196337960 shutdown_connections 2024-09-21T07:11:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.879+0000 7f5b6f200640 1 -- 172.21.15.27:0/3196337960 wait complete. 2024-09-21T07:11:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.880+0000 7f5b6f200640 1 Processor -- start 2024-09-21T07:11:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.880+0000 7f5b6f200640 1 -- start start 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.880+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68075dd0 0x7f5b681a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.880+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b68077380 0x7f5b681a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b68114280 0x7f5b681a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68119ec0 con 0x7f5b68114280 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68119d40 con 0x7f5b68075dd0 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6811a040 con 0x7f5b68077380 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b68114280 0x7f5b681a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68075dd0 0x7f5b681a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b68114280 0x7f5b681a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38470/0 (socket says 172.21.15.27:38470) 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6e800640 1 -- 172.21.15.27:0/2239291498 learned_addr learned my addr 172.21.15.27:0/2239291498 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68075dd0 0x7f5b681a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36196/0 (socket says 172.21.15.27:36196) 2024-09-21T07:11:46.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6d400640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b68077380 0x7f5b681a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6de00640 1 -- 172.21.15.27:0/2239291498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b68077380 msgr2=0x7f5b681a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:46.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6de00640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b68077380 0x7f5b681a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6de00640 1 -- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b68114280 msgr2=0x7f5b681a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6de00640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b68114280 0x7f5b681a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.881+0000 7f5b6de00640 1 -- 172.21.15.27:0/2239291498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b681a67e0 con 0x7f5b68075dd0 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.882+0000 7f5b6de00640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68075dd0 0x7f5b681a0870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e9f0 tx=0x7f5b5c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.882+0000 7f5b62a00640 1 -- 172.21.15.27:0/2239291498 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00ccc0 con 0x7f5b68075dd0 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.882+0000 7f5b62a00640 1 -- 172.21.15.27:0/2239291498 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b5c0108e0 con 0x7f5b68075dd0 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.882+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b681a6ad0 con 0x7f5b68075dd0 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.882+0000 7f5b62a00640 1 -- 172.21.15.27:0/2239291498 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c018880 con 0x7f5b68075dd0 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.882+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681ad230 con 0x7f5b68075dd0 2024-09-21T07:11:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.883+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b680761e0 con 0x7f5b68075dd0 2024-09-21T07:11:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.886+0000 7f5b62a00640 1 -- 172.21.15.27:0/2239291498 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5b5c010430 con 0x7f5b68075dd0 2024-09-21T07:11:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.888+0000 7f5b62a00640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b500b65f0 0x7f5b500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.888+0000 7f5b62a00640 1 -- 172.21.15.27:0/2239291498 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f5b5c0e81a0 con 0x7f5b68075dd0 2024-09-21T07:11:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.888+0000 7f5b6d400640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b500b65f0 0x7f5b500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.889+0000 7f5b6d400640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b500b65f0 0x7f5b500b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b68115f40 tx=0x7f5b58005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:46.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.889+0000 7f5b62a00640 1 -- 172.21.15.27:0/2239291498 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b5c0b2820 con 0x7f5b68075dd0 2024-09-21T07:11:46.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.808+0000 7fc03ac00640 1 -- 172.21.15.27:0/3123809520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 msgr2=0x7fc034075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:46.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.808+0000 7fc03ac00640 1 --2- 172.21.15.27:0/3123809520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc034075bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc024009860 tx=0x7fc02402a340 comp rx=0 tx=0).stop 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.809+0000 7fc03ac00640 1 -- 172.21.15.27:0/3123809520 shutdown_connections 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.809+0000 7fc03ac00640 1 --2- 172.21.15.27:0/3123809520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0341100e0 0x7fc034112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.809+0000 7fc03ac00640 1 --2- 172.21.15.27:0/3123809520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc034076100 0x7fc03410fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.809+0000 7fc03ac00640 1 --2- 172.21.15.27:0/3123809520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc034075bc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.809+0000 7fc03ac00640 1 -- 172.21.15.27:0/3123809520 >> 172.21.15.27:0/3123809520 conn(0x7fc0340ff590 msgr2=0x7fc0341019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.809+0000 7fc03ac00640 1 -- 172.21.15.27:0/3123809520 shutdown_connections 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.809+0000 7fc03ac00640 1 -- 172.21.15.27:0/3123809520 wait complete. 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.810+0000 7fc03ac00640 1 Processor -- start 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.810+0000 7fc03ac00640 1 -- start start 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc03ac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc03419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc03ac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc034076100 0x7fc03419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc03ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0341100e0 0x7fc0341a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc03ac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc034115c90 con 0x7fc034076100 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc03ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc034115b10 con 0x7fc0341100e0 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc03ac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc034115e10 con 0x7fc0340757b0 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc033e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc03419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc033400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc034076100 0x7fc03419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc033e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc03419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:43718/0 (socket says 172.21.15.27:43718) 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc033e00640 1 -- 172.21.15.27:0/1124966942 learned_addr learned my addr 172.21.15.27:0/1124966942 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.811+0000 7fc038c00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0341100e0 0x7fc0341a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc033e00640 1 -- 172.21.15.27:0/1124966942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0341100e0 msgr2=0x7fc0341a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc033e00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0341100e0 0x7fc0341a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc033e00640 1 -- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc034076100 msgr2=0x7fc03419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc033e00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc034076100 0x7fc03419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc033e00640 1 -- 172.21.15.27:0/1124966942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0341a2590 con 0x7fc0340757b0 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc038c00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0341100e0 0x7fc0341a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc033e00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc03419c620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0240303f0 tx=0x7fc024032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc030c00640 1 -- 172.21.15.27:0/1124966942 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024040020 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc030c00640 1 -- 172.21.15.27:0/1124966942 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc024030dd0 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0341a2820 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.812+0000 7fc030c00640 1 -- 172.21.15.27:0/1124966942 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024037040 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.813+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341a8fe0 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.814+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc034076cc0 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.816+0000 7fc030c00640 1 -- 172.21.15.27:0/1124966942 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc024043020 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.818+0000 7fc030c00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0180b6670 0x7fc0180b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.818+0000 7fc030c00640 1 -- 172.21.15.27:0/1124966942 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7fc024107be0 con 0x7fc0340757b0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.818+0000 7fc033400640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0180b6670 0x7fc0180b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.819+0000 7fc033400640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0180b6670 0x7fc0180b8b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc034102aa0 tx=0x7fc020005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:46.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:46.820+0000 7fc030c00640 1 -- 172.21.15.27:0/1124966942 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc02404a020 con 0x7fc0340757b0 2024-09-21T07:11:46.980 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:46.980 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:47.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.046+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc034075bc0 con 0x7fc0340757b0 2024-09-21T07:11:47.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.047+0000 7fc030c00640 1 -- 172.21.15.27:0/1124966942 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0240328c0 con 0x7fc0340757b0 2024-09-21T07:11:47.049 INFO:teuthology.orchestra.run.smithi027.stdout:38654705759 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0180b6670 msgr2=0x7fc0180b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0180b6670 0x7fc0180b8b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc034102aa0 tx=0x7fc020005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 msgr2=0x7fc03419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc03419c620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0240303f0 tx=0x7fc024032da0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 shutdown_connections 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0180b6670 0x7fc0180b8b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0341100e0 0x7fc0341a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc034076100 0x7fc03419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 --2- 172.21.15.27:0/1124966942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0340757b0 0x7fc03419c620 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 >> 172.21.15.27:0/1124966942 conn(0x7fc0340ff590 msgr2=0x7fc034107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:47.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 shutdown_connections 2024-09-21T07:11:47.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.050+0000 7fc03ac00640 1 -- 172.21.15.27:0/1124966942 wait complete. 2024-09-21T07:11:47.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:46 smithi027 ceph-mon[30779]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T07:11:47.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.126+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5b681a19a0 con 0x7f5b68075dd0 2024-09-21T07:11:47.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.126+0000 7f5b62a00640 1 -- 172.21.15.27:0/2239291498 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5b5c0b7760 con 0x7f5b68075dd0 2024-09-21T07:11:47.129 INFO:teuthology.orchestra.run.smithi027.stdout:266287972392 2024-09-21T07:11:47.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b500b65f0 msgr2=0x7f5b500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:47.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b500b65f0 0x7f5b500b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b68115f40 tx=0x7f5b58005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:47.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68075dd0 msgr2=0x7f5b681a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:47.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68075dd0 0x7f5b681a0870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e9f0 tx=0x7f5b5c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 shutdown_connections 2024-09-21T07:11:47.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b68077380 0x7f5b681a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b500b65f0 0x7f5b500b8b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b68114280 0x7f5b681a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 --2- 172.21.15.27:0/2239291498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b68075dd0 0x7f5b681a0870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:47.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 >> 172.21.15.27:0/2239291498 conn(0x7f5b680ff440 msgr2=0x7f5b68110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:47.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 shutdown_connections 2024-09-21T07:11:47.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:47.129+0000 7f5b6f200640 1 -- 172.21.15.27:0/2239291498 wait complete. 2024-09-21T07:11:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:46 smithi119 ceph-mon[34519]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T07:11:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:46 smithi153 ceph-mon[34747]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T07:11:47.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332144 got 219043332148 for osd.8 2024-09-21T07:11:47.807 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:48.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:48 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1124966942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T07:11:48.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:48 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2239291498' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T07:11:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:48 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1124966942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T07:11:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:48 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2239291498' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T07:11:48.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:48 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1124966942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T07:11:48.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:48 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2239291498' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T07:11:48.470 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705756 got 38654705759 for osd.0 2024-09-21T07:11:48.470 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:48.880 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:48.911 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972389 got 266287972392 for osd.10 2024-09-21T07:11:48.911 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:49.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:49 smithi027 ceph-mon[30779]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:11:49.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:49 smithi119 ceph-mon[34519]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:11:49.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:49 smithi153 ceph-mon[34747]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:11:49.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 -- 172.21.15.27:0/1017826120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 msgr2=0x7f26d8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:49.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 --2- 172.21.15.27:0/1017826120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 0x7f26d8110640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009ca0 tx=0x7f26d002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:11:49.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 -- 172.21.15.27:0/1017826120 shutdown_connections 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 --2- 172.21.15.27:0/1017826120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26d81094c0 0x7f26d81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 --2- 172.21.15.27:0/1017826120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 0x7f26d8110640 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 --2- 172.21.15.27:0/1017826120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d8103b80 0x7f26d8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 -- 172.21.15.27:0/1017826120 >> 172.21.15.27:0/1017826120 conn(0x7f26d80ff610 msgr2=0x7f26d8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 -- 172.21.15.27:0/1017826120 shutdown_connections 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.831+0000 7f26dde00640 1 -- 172.21.15.27:0/1017826120 wait complete. 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.832+0000 7f26dde00640 1 Processor -- start 2024-09-21T07:11:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.832+0000 7f26dde00640 1 -- start start 2024-09-21T07:11:49.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26dde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26d8103b80 0x7f26d819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26dde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d81094c0 0x7f26d819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26dde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 0x7f26d81a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26dde00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8113b20 con 0x7f26d8109e10 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26dde00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26d81139a0 con 0x7f26d81094c0 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26dde00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8113ca0 con 0x7f26d8103b80 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26d7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 0x7f26d81a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26d7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 0x7f26d81a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52308/0 (socket says 172.21.15.27:52308) 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26d6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26d8103b80 0x7f26d819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26d7400640 1 -- 172.21.15.27:0/3260419979 learned_addr learned my addr 172.21.15.27:0/3260419979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.833+0000 7f26cfe00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d81094c0 0x7f26d819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26d6a00640 1 -- 172.21.15.27:0/3260419979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d81094c0 msgr2=0x7f26d819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26d6a00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d81094c0 0x7f26d819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26d6a00640 1 -- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 msgr2=0x7f26d81a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26d6a00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 0x7f26d81a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26d6a00640 1 -- 172.21.15.27:0/3260419979 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d81a2680 con 0x7f26d8103b80 2024-09-21T07:11:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26d6a00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26d8103b80 0x7f26d819c710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e9f0 tx=0x7f26c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26dca00640 1 -- 172.21.15.27:0/3260419979 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c800ccc0 con 0x7f26d8103b80 2024-09-21T07:11:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d81a2970 con 0x7f26d8103b80 2024-09-21T07:11:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26dca00640 1 -- 172.21.15.27:0/3260419979 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c80108e0 con 0x7f26d8103b80 2024-09-21T07:11:49.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.834+0000 7f26dca00640 1 -- 172.21.15.27:0/3260419979 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8018880 con 0x7f26d8103b80 2024-09-21T07:11:49.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.835+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d81a90d0 con 0x7f26d8103b80 2024-09-21T07:11:49.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.836+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d8104e00 con 0x7f26d8103b80 2024-09-21T07:11:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.838+0000 7f26dca00640 1 -- 172.21.15.27:0/3260419979 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f26c8010430 con 0x7f26d8103b80 2024-09-21T07:11:49.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.840+0000 7f26dca00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26bc0b65f0 0x7f26bc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.840+0000 7f26dca00640 1 -- 172.21.15.27:0/3260419979 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f26c80e8150 con 0x7f26d8103b80 2024-09-21T07:11:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.840+0000 7f26cfe00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26bc0b65f0 0x7f26bc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.841+0000 7f26cfe00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26bc0b65f0 0x7f26bc0b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26d8105af0 tx=0x7f26c0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:49.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:49.842+0000 7f26dca00640 1 -- 172.21.15.27:0/3260419979 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26c80b27d0 con 0x7f26d8103b80 2024-09-21T07:11:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.047+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f26d8103f90 con 0x7f26d8103b80 2024-09-21T07:11:50.049 INFO:teuthology.orchestra.run.smithi027.stdout:244813135919 2024-09-21T07:11:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.047+0000 7f26dca00640 1 -- 172.21.15.27:0/3260419979 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f26c80b7710 con 0x7f26d8103b80 2024-09-21T07:11:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26bc0b65f0 msgr2=0x7f26bc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26bc0b65f0 0x7f26bc0b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26d8105af0 tx=0x7f26c0005f90 comp rx=0 tx=0).stop 2024-09-21T07:11:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26d8103b80 msgr2=0x7f26d819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26d8103b80 0x7f26d819c710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e9f0 tx=0x7f26c800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 shutdown_connections 2024-09-21T07:11:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26bc0b65f0 0x7f26bc0b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26d8109e10 0x7f26d81a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d81094c0 0x7f26d819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 --2- 172.21.15.27:0/3260419979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26d8103b80 0x7f26d819c710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.050+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 >> 172.21.15.27:0/3260419979 conn(0x7f26d80ff610 msgr2=0x7f26d8108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.051+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 shutdown_connections 2024-09-21T07:11:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.051+0000 7f26dde00640 1 -- 172.21.15.27:0/3260419979 wait complete. 2024-09-21T07:11:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.138+0000 7f8b98000640 1 -- 172.21.15.27:0/442773213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90114740 msgr2=0x7f8b90116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.138+0000 7f8b98000640 1 --2- 172.21.15.27:0/442773213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90114740 0x7f8b90116b60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009be0 tx=0x7f8b8802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.138+0000 7f8b98000640 1 -- 172.21.15.27:0/442773213 shutdown_connections 2024-09-21T07:11:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.138+0000 7f8b98000640 1 --2- 172.21.15.27:0/442773213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90114740 0x7f8b90116b60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.138+0000 7f8b98000640 1 --2- 172.21.15.27:0/442773213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90078290 0x7f8b90114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.138+0000 7f8b98000640 1 --2- 172.21.15.27:0/442773213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90077940 0x7f8b90077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.138+0000 7f8b98000640 1 -- 172.21.15.27:0/442773213 >> 172.21.15.27:0/442773213 conn(0x7f8b90101810 msgr2=0x7f8b90103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.139+0000 7f8b98000640 1 -- 172.21.15.27:0/442773213 shutdown_connections 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.139+0000 7f8b98000640 1 -- 172.21.15.27:0/442773213 wait complete. 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 Processor -- start 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 -- start start 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90077940 0x7f8b901a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90078290 0x7f8b901a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90114740 0x7f8b901a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9011a0b0 con 0x7f8b90078290 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90119f30 con 0x7f8b90114740 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b98000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9011a230 con 0x7f8b90077940 2024-09-21T07:11:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.140+0000 7f8b94c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90078290 0x7f8b901a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b95600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90077940 0x7f8b901a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b95600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90077940 0x7f8b901a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35656/0 (socket says 172.21.15.27:35656) 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b94c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90078290 0x7f8b901a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52326/0 (socket says 172.21.15.27:52326) 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b95600640 1 -- 172.21.15.27:0/4201630216 learned_addr learned my addr 172.21.15.27:0/4201630216 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b96000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90114740 0x7f8b901a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b96000640 1 -- 172.21.15.27:0/4201630216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90077940 msgr2=0x7f8b901a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b96000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90077940 0x7f8b901a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b96000640 1 -- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90078290 msgr2=0x7f8b901a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b96000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90078290 0x7f8b901a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b96000640 1 -- 172.21.15.27:0/4201630216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b901a6a10 con 0x7f8b90114740 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.141+0000 7f8b94c00640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90078290 0x7f8b901a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.142+0000 7f8b96000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90114740 0x7f8b901a62e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b8802abe0 tx=0x7f8b88003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.142+0000 7f8b8e000640 1 -- 172.21.15.27:0/4201630216 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88041020 con 0x7f8b90114740 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.142+0000 7f8b8e000640 1 -- 172.21.15.27:0/4201630216 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b88003e10 con 0x7f8b90114740 2024-09-21T07:11:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.142+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b901a6ca0 con 0x7f8b90114740 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.142+0000 7f8b8e000640 1 -- 172.21.15.27:0/4201630216 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88036040 con 0x7f8b90114740 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.142+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b901ad460 con 0x7f8b90114740 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.143+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b90078e50 con 0x7f8b90114740 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.146+0000 7f8b8e000640 1 -- 172.21.15.27:0/4201630216 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8b88044020 con 0x7f8b90114740 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.147+0000 7f8b8e000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8b780b65f0 0x7f8b780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.147+0000 7f8b8e000640 1 -- 172.21.15.27:0/4201630216 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f8b88108860 con 0x7f8b90114740 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.147+0000 7f8b95600640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8b780b65f0 0x7f8b780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.148+0000 7f8b95600640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8b780b65f0 0x7f8b780b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b840058c0 tx=0x7f8b84009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.149+0000 7f8b8e000640 1 -- 172.21.15.27:0/4201630216 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b8804b020 con 0x7f8b90114740 2024-09-21T07:11:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.358+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8b9010f8d0 con 0x7f8b90114740 2024-09-21T07:11:50.360 INFO:teuthology.orchestra.run.smithi027.stdout:128849018953 2024-09-21T07:11:50.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.358+0000 7f8b8e000640 1 -- 172.21.15.27:0/4201630216 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8b880ceb00 con 0x7f8b90114740 2024-09-21T07:11:50.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8b780b65f0 msgr2=0x7f8b780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:50.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8b780b65f0 0x7f8b780b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b840058c0 tx=0x7f8b84009260 comp rx=0 tx=0).stop 2024-09-21T07:11:50.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90114740 msgr2=0x7f8b901a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:50.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90114740 0x7f8b901a62e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b8802abe0 tx=0x7f8b88003c20 comp rx=0 tx=0).stop 2024-09-21T07:11:50.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 shutdown_connections 2024-09-21T07:11:50.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8b780b65f0 0x7f8b780b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b90114740 0x7f8b901a62e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b90078290 0x7f8b901a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 --2- 172.21.15.27:0/4201630216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b90077940 0x7f8b901a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:50.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 >> 172.21.15.27:0/4201630216 conn(0x7f8b90101810 msgr2=0x7f8b90116730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:50.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 shutdown_connections 2024-09-21T07:11:50.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.362+0000 7f8b98000640 1 -- 172.21.15.27:0/4201630216 wait complete. 2024-09-21T07:11:51.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:51 smithi027 ceph-mon[30779]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:51.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:51 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3260419979' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T07:11:51.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:51 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4201630216' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T07:11:51.219 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:51 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2624449384' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T07:11:51.255 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018949 got 128849018953 for osd.4 2024-09-21T07:11:51.255 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:51 smithi119 ceph-mon[34519]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:51 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3260419979' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T07:11:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:51 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4201630216' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T07:11:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:51 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2624449384' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T07:11:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:51 smithi153 ceph-mon[34747]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:11:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:51 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3260419979' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T07:11:51.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:51 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4201630216' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T07:11:51.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:51 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2624449384' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T07:11:51.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135915 got 244813135919 for osd.9 2024-09-21T07:11:51.534 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.239+0000 7f66b9a00640 1 -- 172.21.15.27:0/1733162816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b4103b60 msgr2=0x7f66b4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.239+0000 7f66b9a00640 1 --2- 172.21.15.27:0/1733162816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b4103b60 0x7f66b4103f70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009860 tx=0x7f66a402a340 comp rx=0 tx=0).stop 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.240+0000 7f66b9a00640 1 -- 172.21.15.27:0/1733162816 shutdown_connections 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.240+0000 7f66b9a00640 1 --2- 172.21.15.27:0/1733162816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b41094a0 0x7f66b41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.240+0000 7f66b9a00640 1 --2- 172.21.15.27:0/1733162816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b4109df0 0x7f66b4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.240+0000 7f66b9a00640 1 --2- 172.21.15.27:0/1733162816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b4103b60 0x7f66b4103f70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.240+0000 7f66b9a00640 1 -- 172.21.15.27:0/1733162816 >> 172.21.15.27:0/1733162816 conn(0x7f66b407b850 msgr2=0x7f66b407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.240+0000 7f66b9a00640 1 -- 172.21.15.27:0/1733162816 shutdown_connections 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.240+0000 7f66b9a00640 1 -- 172.21.15.27:0/1733162816 wait complete. 2024-09-21T07:11:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.241+0000 7f66b9a00640 1 Processor -- start 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.241+0000 7f66b9a00640 1 -- start start 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b9a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b4103b60 0x7f66b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b41094a0 0x7f66b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b4109df0 0x7f66b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b9a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4113ab0 con 0x7f66b4109df0 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b9a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4113930 con 0x7f66b41094a0 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b9a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4113c30 con 0x7f66b4103b60 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b4109df0 0x7f66b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b41094a0 0x7f66b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b4109df0 0x7f66b4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52352/0 (socket says 172.21.15.27:52352) 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b3400640 1 -- 172.21.15.27:0/2624449384 learned_addr learned my addr 172.21.15.27:0/2624449384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.242+0000 7f66b2a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b4103b60 0x7f66b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66b2000640 1 -- 172.21.15.27:0/2624449384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b4103b60 msgr2=0x7f66b4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66b2000640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b4103b60 0x7f66b4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66b2000640 1 -- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b4109df0 msgr2=0x7f66b4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66b2000640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b4109df0 0x7f66b4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66b2000640 1 -- 172.21.15.27:0/2624449384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b41a8740 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66b2a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b4103b60 0x7f66b4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66b2000640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b41094a0 0x7f66b4075c50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66b406c9f0 tx=0x7f66a000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.243+0000 7f66ab400640 1 -- 172.21.15.27:0/2624449384 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000cc40 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.244+0000 7f66ab400640 1 -- 172.21.15.27:0/2624449384 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a000cde0 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.244+0000 7f66ab400640 1 -- 172.21.15.27:0/2624449384 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004a00 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.244+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b41a8a30 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.244+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b41a8f70 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.245+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b4103f70 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.248+0000 7f66ab400640 1 -- 172.21.15.27:0/2624449384 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f66a0004ba0 con 0x7f66b41094a0 2024-09-21T07:11:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.250+0000 7f66ab400640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66980b6670 0x7f66980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.250+0000 7f66ab400640 1 -- 172.21.15.27:0/2624449384 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f66a00e7970 con 0x7f66b41094a0 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.250+0000 7f66b2a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66980b6670 0x7f66980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.250+0000 7f66ab400640 1 -- 172.21.15.27:0/2624449384 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66a00adca0 con 0x7f66b41094a0 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.251+0000 7f66b2a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66980b6670 0x7f66980b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009830 tx=0x7f66a40058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.460+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f66b4076a10 con 0x7f66b41094a0 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.461+0000 7f66ab400640 1 -- 172.21.15.27:0/2624449384 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66b4076a10 con 0x7f66b41094a0 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stdout:150323855429 2024-09-21T07:11:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66980b6670 msgr2=0x7f66980b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66980b6670 0x7f66980b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009830 tx=0x7f66a40058d0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b41094a0 msgr2=0x7f66b4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b41094a0 0x7f66b4075c50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66b406c9f0 tx=0x7f66a000ee50 comp rx=0 tx=0).stop 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 shutdown_connections 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66980b6670 0x7f66980b8b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b4109df0 0x7f66b4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b41094a0 0x7f66b4075c50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 --2- 172.21.15.27:0/2624449384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b4103b60 0x7f66b4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 >> 172.21.15.27:0/2624449384 conn(0x7f66b407b850 msgr2=0x7f66b4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 shutdown_connections 2024-09-21T07:11:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:50.463+0000 7f66b9a00640 1 -- 172.21.15.27:0/2624449384 wait complete. 2024-09-21T07:11:52.452 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855427 got 150323855429 for osd.5 2024-09-21T07:11:52.453 DEBUG:teuthology.parallel:result is None 2024-09-21T07:11:52.453 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T07:11:52.453 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T07:11:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:53.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:53 smithi027 ceph-mon[30779]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:53 smithi119 ceph-mon[34519]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:53 smithi153 ceph-mon[34747]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:11:53.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 -- 172.21.15.27:0/2893464043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34700760a0 msgr2=0x7f347010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:53.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 --2- 172.21.15.27:0/2893464043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34700760a0 0x7f347010fbe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f345c009860 tx=0x7f345c02a340 comp rx=0 tx=0).stop 2024-09-21T07:11:53.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 -- 172.21.15.27:0/2893464043 shutdown_connections 2024-09-21T07:11:53.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 --2- 172.21.15.27:0/2893464043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34700760a0 0x7f347010fbe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 --2- 172.21.15.27:0/2893464043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470110120 0x7f3470112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 --2- 172.21.15.27:0/2893464043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3470075750 0x7f3470075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 -- 172.21.15.27:0/2893464043 >> 172.21.15.27:0/2893464043 conn(0x7f347007b850 msgr2=0x7f347007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:53.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 -- 172.21.15.27:0/2893464043 shutdown_connections 2024-09-21T07:11:53.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.451+0000 7f3477c00640 1 -- 172.21.15.27:0/2893464043 wait complete. 2024-09-21T07:11:53.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.453+0000 7f3477c00640 1 Processor -- start 2024-09-21T07:11:53.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.453+0000 7f3477c00640 1 -- start start 2024-09-21T07:11:53.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3477c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3470075750 0x7f347019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:53.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3477c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34700760a0 0x7f347019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3477c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3470110120 0x7f34701a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3477c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3470115c90 con 0x7f3470110120 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3477c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3470115b10 con 0x7f34700760a0 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3477c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3470115e10 con 0x7f3470075750 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3475200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3470075750 0x7f347019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3475200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3470075750 0x7f347019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35688/0 (socket says 172.21.15.27:35688) 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3475200640 1 -- 172.21.15.27:0/1916165234 learned_addr learned my addr 172.21.15.27:0/1916165234 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f3475c00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3470110120 0x7f34701a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.454+0000 7f346fe00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34700760a0 0x7f347019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:53.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.455+0000 7f3475200640 1 -- 172.21.15.27:0/1916165234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34700760a0 msgr2=0x7f347019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:53.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.455+0000 7f3475200640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34700760a0 0x7f347019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.455+0000 7f3475200640 1 -- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3470110120 msgr2=0x7f34701a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:53.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.455+0000 7f3475200640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3470110120 0x7f34701a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.455+0000 7f3475200640 1 -- 172.21.15.27:0/1916165234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34701a2630 con 0x7f3470075750 2024-09-21T07:11:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.455+0000 7f3475200640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3470075750 0x7f347019c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f346400e9c0 tx=0x7f346400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.456+0000 7f346d600640 1 -- 172.21.15.27:0/1916165234 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400ccc0 con 0x7f3470075750 2024-09-21T07:11:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.456+0000 7f346d600640 1 -- 172.21.15.27:0/1916165234 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34640054d0 con 0x7f3470075750 2024-09-21T07:11:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.456+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34701a2920 con 0x7f3470075750 2024-09-21T07:11:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.456+0000 7f346d600640 1 -- 172.21.15.27:0/1916165234 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464010430 con 0x7f3470075750 2024-09-21T07:11:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.456+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34701a9080 con 0x7f3470075750 2024-09-21T07:11:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.457+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3470075b60 con 0x7f3470075750 2024-09-21T07:11:53.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.460+0000 7f346d600640 1 -- 172.21.15.27:0/1916165234 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3464004f80 con 0x7f3470075750 2024-09-21T07:11:53.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.462+0000 7f346d600640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f34580b65f0 0x7f34580b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.462+0000 7f346d600640 1 -- 172.21.15.27:0/1916165234 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9307+0+0 (secure 0 0 0) 0x7f34640e8060 con 0x7f3470075750 2024-09-21T07:11:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.462+0000 7f346fe00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f34580b65f0 0x7f34580b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:53.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.463+0000 7f346d600640 1 -- 172.21.15.27:0/1916165234 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34640ae300 con 0x7f3470075750 2024-09-21T07:11:53.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.463+0000 7f346fe00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f34580b65f0 0x7f34580b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f345c005940 tx=0x7f345c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:53.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.666+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3470063800 con 0x7f34580b65f0 2024-09-21T07:11:53.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.673+0000 7f346d600640 1 -- 172.21.15.27:0/1916165234 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340350 (secure 0 0 0) 0x7f3470063800 con 0x7f34580b65f0 2024-09-21T07:11:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:53.681 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f34580b65f0 msgr2=0x7f34580b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f34580b65f0 0x7f34580b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f345c005940 tx=0x7f345c0058d0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3470075750 msgr2=0x7f347019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3470075750 0x7f347019c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f346400e9c0 tx=0x7f346400ee80 comp rx=0 tx=0).stop 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 shutdown_connections 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f34580b65f0 0x7f34580b8b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3470110120 0x7f34701a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34700760a0 0x7f347019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 --2- 172.21.15.27:0/1916165234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3470075750 0x7f347019c6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.678+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 >> 172.21.15.27:0/1916165234 conn(0x7f347007b850 msgr2=0x7f3470101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.679+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 shutdown_connections 2024-09-21T07:11:53.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:53.679+0000 7f3477c00640 1 -- 172.21.15.27:0/1916165234 wait complete. 2024-09-21T07:11:54.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:11:54.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:54.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:54.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10, 5, 1]} : dispatch 2024-09-21T07:11:54.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]} : dispatch 2024-09-21T07:11:54.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:54 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]} : dispatch 2024-09-21T07:11:54.274 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:11:54.274 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:54.274 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:54.274 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10, 5, 1]} : dispatch 2024-09-21T07:11:54.274 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]} : dispatch 2024-09-21T07:11:54.274 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:54 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]} : dispatch 2024-09-21T07:11:54.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:11:54.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:54.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:54.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10, 5, 1]} : dispatch 2024-09-21T07:11:54.290 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]} : dispatch 2024-09-21T07:11:54.291 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:54 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]} : dispatch 2024-09-21T07:11:54.382 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-21T07:11:53.397949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660815,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":462,"num_read_kb":333,"num_write":291,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387652,"kb_used_data":13436,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554172,"statfs":{"total":1151940427776,"available":1151543472128,"internally_reserved":0,"allocated":13758464,"data_stored":7917251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003649"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844106+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:10:54.844106+0000","last_peered":"2024-09-21T07:10:54.844106+0000","last_clean":"2024-09-21T07:10:54.844106+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:10:54.844106+0000","last_undegraded":"2024-09-21T07:10:54.844106+0000","last_fullsized":"2024-09-21T07:10:54.844106+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857694+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:10:54.857694+0000","last_peered":"2024-09-21T07:10:54.857694+0000","last_clean":"2024-09-21T07:10:54.857694+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:10:54.857694+0000","last_undegraded":"2024-09-21T07:10:54.857694+0000","last_fullsized":"2024-09-21T07:10:54.857694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850712+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:10:54.850712+0000","last_peered":"2024-09-21T07:10:54.850712+0000","last_clean":"2024-09-21T07:10:54.850712+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:10:54.850712+0000","last_undegraded":"2024-09-21T07:10:54.850712+0000","last_fullsized":"2024-09-21T07:10:54.850712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858644+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:10:54.858644+0000","last_peered":"2024-09-21T07:10:54.858644+0000","last_clean":"2024-09-21T07:10:54.858644+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:10:54.858644+0000","last_undegraded":"2024-09-21T07:10:54.858644+0000","last_fullsized":"2024-09-21T07:10:54.858644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847175+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:10:54.847175+0000","last_peered":"2024-09-21T07:10:54.847175+0000","last_clean":"2024-09-21T07:10:54.847175+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:10:54.847175+0000","last_undegraded":"2024-09-21T07:10:54.847175+0000","last_fullsized":"2024-09-21T07:10:54.847175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854448+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:10:54.854448+0000","last_peered":"2024-09-21T07:10:54.854448+0000","last_clean":"2024-09-21T07:10:54.854448+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:10:54.854448+0000","last_undegraded":"2024-09-21T07:10:54.854448+0000","last_fullsized":"2024-09-21T07:10:54.854448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858193+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:10:54.858193+0000","last_peered":"2024-09-21T07:10:54.858193+0000","last_clean":"2024-09-21T07:10:54.858193+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:10:54.858193+0000","last_undegraded":"2024-09-21T07:10:54.858193+0000","last_fullsized":"2024-09-21T07:10:54.858193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848421+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:10:54.848421+0000","last_peered":"2024-09-21T07:10:54.848421+0000","last_clean":"2024-09-21T07:10:54.848421+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:10:54.848421+0000","last_undegraded":"2024-09-21T07:10:54.848421+0000","last_fullsized":"2024-09-21T07:10:54.848421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854828+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:10:54.854828+0000","last_peered":"2024-09-21T07:10:54.854828+0000","last_clean":"2024-09-21T07:10:54.854828+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:10:54.854828+0000","last_undegraded":"2024-09-21T07:10:54.854828+0000","last_fullsized":"2024-09-21T07:10:54.854828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857732+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:10:54.857732+0000","last_peered":"2024-09-21T07:10:54.857732+0000","last_clean":"2024-09-21T07:10:54.857732+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:10:54.857732+0000","last_undegraded":"2024-09-21T07:10:54.857732+0000","last_fullsized":"2024-09-21T07:10:54.857732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848591+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:10:54.848591+0000","last_peered":"2024-09-21T07:10:54.848591+0000","last_clean":"2024-09-21T07:10:54.848591+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:10:54.848591+0000","last_undegraded":"2024-09-21T07:10:54.848591+0000","last_fullsized":"2024-09-21T07:10:54.848591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854459+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:10:54.854459+0000","last_peered":"2024-09-21T07:10:54.854459+0000","last_clean":"2024-09-21T07:10:54.854459+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:10:54.854459+0000","last_undegraded":"2024-09-21T07:10:54.854459+0000","last_fullsized":"2024-09-21T07:10:54.854459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844977+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:10:54.844977+0000","last_peered":"2024-09-21T07:10:54.844977+0000","last_clean":"2024-09-21T07:10:54.844977+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:10:54.844977+0000","last_undegraded":"2024-09-21T07:10:54.844977+0000","last_fullsized":"2024-09-21T07:10:54.844977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857487+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:10:54.857487+0000","last_peered":"2024-09-21T07:10:54.857487+0000","last_clean":"2024-09-21T07:10:54.857487+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:10:54.857487+0000","last_undegraded":"2024-09-21T07:10:54.857487+0000","last_fullsized":"2024-09-21T07:10:54.857487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855146+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:10:54.855146+0000","last_peered":"2024-09-21T07:10:54.855146+0000","last_clean":"2024-09-21T07:10:54.855146+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:10:54.855146+0000","last_undegraded":"2024-09-21T07:10:54.855146+0000","last_fullsized":"2024-09-21T07:10:54.855146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856487+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:10:54.856487+0000","last_peered":"2024-09-21T07:10:54.856487+0000","last_clean":"2024-09-21T07:10:54.856487+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:10:54.856487+0000","last_undegraded":"2024-09-21T07:10:54.856487+0000","last_fullsized":"2024-09-21T07:10:54.856487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.867034+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:10:54.867034+0000","last_peered":"2024-09-21T07:10:54.867034+0000","last_clean":"2024-09-21T07:10:54.867034+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:10:54.867034+0000","last_undegraded":"2024-09-21T07:10:54.867034+0000","last_fullsized":"2024-09-21T07:10:54.867034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854799+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:10:54.854799+0000","last_peered":"2024-09-21T07:10:54.854799+0000","last_clean":"2024-09-21T07:10:54.854799+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:10:54.854799+0000","last_undegraded":"2024-09-21T07:10:54.854799+0000","last_fullsized":"2024-09-21T07:10:54.854799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851783+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:10:54.851783+0000","last_peered":"2024-09-21T07:10:54.851783+0000","last_clean":"2024-09-21T07:10:54.851783+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:10:54.851783+0000","last_undegraded":"2024-09-21T07:10:54.851783+0000","last_fullsized":"2024-09-21T07:10:54.851783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856047+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:10:54.856047+0000","last_peered":"2024-09-21T07:10:54.856047+0000","last_clean":"2024-09-21T07:10:54.856047+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:10:54.856047+0000","last_undegraded":"2024-09-21T07:10:54.856047+0000","last_fullsized":"2024-09-21T07:10:54.856047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843895+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:10:54.843895+0000","last_peered":"2024-09-21T07:10:54.843895+0000","last_clean":"2024-09-21T07:10:54.843895+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:10:54.843895+0000","last_undegraded":"2024-09-21T07:10:54.843895+0000","last_fullsized":"2024-09-21T07:10:54.843895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857546+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:10:54.857546+0000","last_peered":"2024-09-21T07:10:54.857546+0000","last_clean":"2024-09-21T07:10:54.857546+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:10:54.857546+0000","last_undegraded":"2024-09-21T07:10:54.857546+0000","last_fullsized":"2024-09-21T07:10:54.857546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847552+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:10:54.847552+0000","last_peered":"2024-09-21T07:10:54.847552+0000","last_clean":"2024-09-21T07:10:54.847552+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:10:54.847552+0000","last_undegraded":"2024-09-21T07:10:54.847552+0000","last_fullsized":"2024-09-21T07:10:54.847552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855277+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:10:54.855277+0000","last_peered":"2024-09-21T07:10:54.855277+0000","last_clean":"2024-09-21T07:10:54.855277+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:10:54.855277+0000","last_undegraded":"2024-09-21T07:10:54.855277+0000","last_fullsized":"2024-09-21T07:10:54.855277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847198+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:10:54.847198+0000","last_peered":"2024-09-21T07:10:54.847198+0000","last_clean":"2024-09-21T07:10:54.847198+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:10:54.847198+0000","last_undegraded":"2024-09-21T07:10:54.847198+0000","last_fullsized":"2024-09-21T07:10:54.847198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855459+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:10:54.855459+0000","last_peered":"2024-09-21T07:10:54.855459+0000","last_clean":"2024-09-21T07:10:54.855459+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:10:54.855459+0000","last_undegraded":"2024-09-21T07:10:54.855459+0000","last_fullsized":"2024-09-21T07:10:54.855459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852078+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:10:54.852078+0000","last_peered":"2024-09-21T07:10:54.852078+0000","last_clean":"2024-09-21T07:10:54.852078+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:10:54.852078+0000","last_undegraded":"2024-09-21T07:10:54.852078+0000","last_fullsized":"2024-09-21T07:10:54.852078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858105+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:10:54.858105+0000","last_peered":"2024-09-21T07:10:54.858105+0000","last_clean":"2024-09-21T07:10:54.858105+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:10:54.858105+0000","last_undegraded":"2024-09-21T07:10:54.858105+0000","last_fullsized":"2024-09-21T07:10:54.858105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'10","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:07.358581+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:11:07.358581+0000","last_peered":"2024-09-21T07:11:07.358581+0000","last_clean":"2024-09-21T07:11:07.358581+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:11:07.358581+0000","last_undegraded":"2024-09-21T07:11:07.358581+0000","last_fullsized":"2024-09-21T07:11:07.358581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855905+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:10:54.855905+0000","last_peered":"2024-09-21T07:10:54.855905+0000","last_clean":"2024-09-21T07:10:54.855905+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:10:54.855905+0000","last_undegraded":"2024-09-21T07:10:54.855905+0000","last_fullsized":"2024-09-21T07:10:54.855905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043363+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:10:55.043363+0000","last_peered":"2024-09-21T07:10:55.043363+0000","last_clean":"2024-09-21T07:10:55.043363+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:10:55.043363+0000","last_undegraded":"2024-09-21T07:10:55.043363+0000","last_fullsized":"2024-09-21T07:10:55.043363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858281+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:10:54.858281+0000","last_peered":"2024-09-21T07:10:54.858281+0000","last_clean":"2024-09-21T07:10:54.858281+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:10:54.858281+0000","last_undegraded":"2024-09-21T07:10:54.858281+0000","last_fullsized":"2024-09-21T07:10:54.858281+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851612+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:10:54.851612+0000","last_peered":"2024-09-21T07:10:54.851612+0000","last_clean":"2024-09-21T07:10:54.851612+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:10:54.851612+0000","last_undegraded":"2024-09-21T07:10:54.851612+0000","last_fullsized":"2024-09-21T07:10:54.851612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846903+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:10:54.846903+0000","last_peered":"2024-09-21T07:10:54.846903+0000","last_clean":"2024-09-21T07:10:54.846903+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:10:54.846903+0000","last_undegraded":"2024-09-21T07:10:54.846903+0000","last_fullsized":"2024-09-21T07:10:54.846903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843652+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:10:54.843652+0000","last_peered":"2024-09-21T07:10:54.843652+0000","last_clean":"2024-09-21T07:10:54.843652+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:10:54.843652+0000","last_undegraded":"2024-09-21T07:10:54.843652+0000","last_fullsized":"2024-09-21T07:10:54.843652+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857264+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:10:54.857264+0000","last_peered":"2024-09-21T07:10:54.857264+0000","last_clean":"2024-09-21T07:10:54.857264+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:10:54.857264+0000","last_undegraded":"2024-09-21T07:10:54.857264+0000","last_fullsized":"2024-09-21T07:10:54.857264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847609+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:10:54.847609+0000","last_peered":"2024-09-21T07:10:54.847609+0000","last_clean":"2024-09-21T07:10:54.847609+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:10:54.847609+0000","last_undegraded":"2024-09-21T07:10:54.847609+0000","last_fullsized":"2024-09-21T07:10:54.847609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845174+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:10:54.845174+0000","last_peered":"2024-09-21T07:10:54.845174+0000","last_clean":"2024-09-21T07:10:54.845174+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:10:54.845174+0000","last_undegraded":"2024-09-21T07:10:54.845174+0000","last_fullsized":"2024-09-21T07:10:54.845174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043157+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:10:55.043157+0000","last_peered":"2024-09-21T07:10:55.043157+0000","last_clean":"2024-09-21T07:10:55.043157+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:10:55.043157+0000","last_undegraded":"2024-09-21T07:10:55.043157+0000","last_fullsized":"2024-09-21T07:10:55.043157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847646+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:10:54.847646+0000","last_peered":"2024-09-21T07:10:54.847646+0000","last_clean":"2024-09-21T07:10:54.847646+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:10:54.847646+0000","last_undegraded":"2024-09-21T07:10:54.847646+0000","last_fullsized":"2024-09-21T07:10:54.847646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":105,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:49.653199+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:11:49.653199+0000","last_peered":"2024-09-21T07:11:49.653199+0000","last_clean":"2024-09-21T07:11:49.653199+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:11:49.653199+0000","last_undegraded":"2024-09-21T07:11:49.653199+0000","last_fullsized":"2024-09-21T07:11:49.653199+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043137+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:10:55.043137+0000","last_peered":"2024-09-21T07:10:55.043137+0000","last_clean":"2024-09-21T07:10:55.043137+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:10:55.043137+0000","last_undegraded":"2024-09-21T07:10:55.043137+0000","last_fullsized":"2024-09-21T07:10:55.043137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042862+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:10:55.042862+0000","last_peered":"2024-09-21T07:10:55.042862+0000","last_clean":"2024-09-21T07:10:55.042862+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:10:55.042862+0000","last_undegraded":"2024-09-21T07:10:55.042862+0000","last_fullsized":"2024-09-21T07:10:55.042862+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845586+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:10:54.845586+0000","last_peered":"2024-09-21T07:10:54.845586+0000","last_clean":"2024-09-21T07:10:54.845586+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:10:54.845586+0000","last_undegraded":"2024-09-21T07:10:54.845586+0000","last_fullsized":"2024-09-21T07:10:54.845586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848102+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:10:54.848102+0000","last_peered":"2024-09-21T07:10:54.848102+0000","last_clean":"2024-09-21T07:10:54.848102+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:10:54.848102+0000","last_undegraded":"2024-09-21T07:10:54.848102+0000","last_fullsized":"2024-09-21T07:10:54.848102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856634+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:10:54.856634+0000","last_peered":"2024-09-21T07:10:54.856634+0000","last_clean":"2024-09-21T07:10:54.856634+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:10:54.856634+0000","last_undegraded":"2024-09-21T07:10:54.856634+0000","last_fullsized":"2024-09-21T07:10:54.856634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855230+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:10:54.855230+0000","last_peered":"2024-09-21T07:10:54.855230+0000","last_clean":"2024-09-21T07:10:54.855230+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:10:54.855230+0000","last_undegraded":"2024-09-21T07:10:54.855230+0000","last_fullsized":"2024-09-21T07:10:54.855230+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846463+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:10:54.846463+0000","last_peered":"2024-09-21T07:10:54.846463+0000","last_clean":"2024-09-21T07:10:54.846463+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:10:54.846463+0000","last_undegraded":"2024-09-21T07:10:54.846463+0000","last_fullsized":"2024-09-21T07:10:54.846463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845250+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:10:54.845250+0000","last_peered":"2024-09-21T07:10:54.845250+0000","last_clean":"2024-09-21T07:10:54.845250+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:10:54.845250+0000","last_undegraded":"2024-09-21T07:10:54.845250+0000","last_fullsized":"2024-09-21T07:10:54.845250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846201+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:10:54.846201+0000","last_peered":"2024-09-21T07:10:54.846201+0000","last_clean":"2024-09-21T07:10:54.846201+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:10:54.846201+0000","last_undegraded":"2024-09-21T07:10:54.846201+0000","last_fullsized":"2024-09-21T07:10:54.846201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852850+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:10:54.852850+0000","last_peered":"2024-09-21T07:10:54.852850+0000","last_clean":"2024-09-21T07:10:54.852850+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:10:54.852850+0000","last_undegraded":"2024-09-21T07:10:54.852850+0000","last_fullsized":"2024-09-21T07:10:54.852850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847470+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:10:54.847470+0000","last_peered":"2024-09-21T07:10:54.847470+0000","last_clean":"2024-09-21T07:10:54.847470+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:10:54.847470+0000","last_undegraded":"2024-09-21T07:10:54.847470+0000","last_fullsized":"2024-09-21T07:10:54.847470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845395+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:10:54.845395+0000","last_peered":"2024-09-21T07:10:54.845395+0000","last_clean":"2024-09-21T07:10:54.845395+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:10:54.845395+0000","last_undegraded":"2024-09-21T07:10:54.845395+0000","last_fullsized":"2024-09-21T07:10:54.845395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848748+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:10:54.848748+0000","last_peered":"2024-09-21T07:10:54.848748+0000","last_clean":"2024-09-21T07:10:54.848748+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:10:54.848748+0000","last_undegraded":"2024-09-21T07:10:54.848748+0000","last_fullsized":"2024-09-21T07:10:54.848748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847352+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:10:54.847352+0000","last_peered":"2024-09-21T07:10:54.847352+0000","last_clean":"2024-09-21T07:10:54.847352+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:10:54.847352+0000","last_undegraded":"2024-09-21T07:10:54.847352+0000","last_fullsized":"2024-09-21T07:10:54.847352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845402+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:10:54.845402+0000","last_peered":"2024-09-21T07:10:54.845402+0000","last_clean":"2024-09-21T07:10:54.845402+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:10:54.845402+0000","last_undegraded":"2024-09-21T07:10:54.845402+0000","last_fullsized":"2024-09-21T07:10:54.845402+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043503+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:10:55.043503+0000","last_peered":"2024-09-21T07:10:55.043503+0000","last_clean":"2024-09-21T07:10:55.043503+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:10:55.043503+0000","last_undegraded":"2024-09-21T07:10:55.043503+0000","last_fullsized":"2024-09-21T07:10:55.043503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847367+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:10:54.847367+0000","last_peered":"2024-09-21T07:10:54.847367+0000","last_clean":"2024-09-21T07:10:54.847367+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:10:54.847367+0000","last_undegraded":"2024-09-21T07:10:54.847367+0000","last_fullsized":"2024-09-21T07:10:54.847367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856776+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:10:54.856776+0000","last_peered":"2024-09-21T07:10:54.856776+0000","last_clean":"2024-09-21T07:10:54.856776+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:10:54.856776+0000","last_undegraded":"2024-09-21T07:10:54.856776+0000","last_fullsized":"2024-09-21T07:10:54.856776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.864304+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:10:54.864304+0000","last_peered":"2024-09-21T07:10:54.864304+0000","last_clean":"2024-09-21T07:10:54.864304+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:10:54.864304+0000","last_undegraded":"2024-09-21T07:10:54.864304+0000","last_fullsized":"2024-09-21T07:10:54.864304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847539+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:10:54.847539+0000","last_peered":"2024-09-21T07:10:54.847539+0000","last_clean":"2024-09-21T07:10:54.847539+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:10:54.847539+0000","last_undegraded":"2024-09-21T07:10:54.847539+0000","last_fullsized":"2024-09-21T07:10:54.847539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.861825+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:10:54.861825+0000","last_peered":"2024-09-21T07:10:54.861825+0000","last_clean":"2024-09-21T07:10:54.861825+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:10:54.861825+0000","last_undegraded":"2024-09-21T07:10:54.861825+0000","last_fullsized":"2024-09-21T07:10:54.861825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":56,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856848+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:10:54.856848+0000","last_peered":"2024-09-21T07:10:54.856848+0000","last_clean":"2024-09-21T07:10:54.856848+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:10:54.856848+0000","last_undegraded":"2024-09-21T07:10:54.856848+0000","last_fullsized":"2024-09-21T07:10:54.856848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043501+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:10:55.043501+0000","last_peered":"2024-09-21T07:10:55.043501+0000","last_clean":"2024-09-21T07:10:55.043501+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:10:55.043501+0000","last_undegraded":"2024-09-21T07:10:55.043501+0000","last_fullsized":"2024-09-21T07:10:55.043501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845481+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:10:54.845481+0000","last_peered":"2024-09-21T07:10:54.845481+0000","last_clean":"2024-09-21T07:10:54.845481+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:10:54.845481+0000","last_undegraded":"2024-09-21T07:10:54.845481+0000","last_fullsized":"2024-09-21T07:10:54.845481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:48.652305+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:11:48.652305+0000","last_peered":"2024-09-21T07:11:48.652305+0000","last_clean":"2024-09-21T07:11:48.652305+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:11:48.652305+0000","last_undegraded":"2024-09-21T07:11:48.652305+0000","last_fullsized":"2024-09-21T07:11:48.652305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843898+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:10:54.843898+0000","last_peered":"2024-09-21T07:10:54.843898+0000","last_clean":"2024-09-21T07:10:54.843898+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:10:54.843898+0000","last_undegraded":"2024-09-21T07:10:54.843898+0000","last_fullsized":"2024-09-21T07:10:54.843898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845127+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:10:54.845127+0000","last_peered":"2024-09-21T07:10:54.845127+0000","last_clean":"2024-09-21T07:10:54.845127+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:10:54.845127+0000","last_undegraded":"2024-09-21T07:10:54.845127+0000","last_fullsized":"2024-09-21T07:10:54.845127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848528+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:10:54.848528+0000","last_peered":"2024-09-21T07:10:54.848528+0000","last_clean":"2024-09-21T07:10:54.848528+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:10:54.848528+0000","last_undegraded":"2024-09-21T07:10:54.848528+0000","last_fullsized":"2024-09-21T07:10:54.848528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852002+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:10:54.852002+0000","last_peered":"2024-09-21T07:10:54.852002+0000","last_clean":"2024-09-21T07:10:54.852002+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:10:54.852002+0000","last_undegraded":"2024-09-21T07:10:54.852002+0000","last_fullsized":"2024-09-21T07:10:54.852002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042934+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:10:55.042934+0000","last_peered":"2024-09-21T07:10:55.042934+0000","last_clean":"2024-09-21T07:10:55.042934+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:10:55.042934+0000","last_undegraded":"2024-09-21T07:10:55.042934+0000","last_fullsized":"2024-09-21T07:10:55.042934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852516+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:10:54.852516+0000","last_peered":"2024-09-21T07:10:54.852516+0000","last_clean":"2024-09-21T07:10:54.852516+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:10:54.852516+0000","last_undegraded":"2024-09-21T07:10:54.852516+0000","last_fullsized":"2024-09-21T07:10:54.852516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848450+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:10:54.848450+0000","last_peered":"2024-09-21T07:10:54.848450+0000","last_clean":"2024-09-21T07:10:54.848450+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:10:54.848450+0000","last_undegraded":"2024-09-21T07:10:54.848450+0000","last_fullsized":"2024-09-21T07:10:54.848450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857933+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:10:54.857933+0000","last_peered":"2024-09-21T07:10:54.857933+0000","last_clean":"2024-09-21T07:10:54.857933+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:10:54.857933+0000","last_undegraded":"2024-09-21T07:10:54.857933+0000","last_fullsized":"2024-09-21T07:10:54.857933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":197,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:47.650389+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:11:47.650389+0000","last_peered":"2024-09-21T07:11:47.650389+0000","last_clean":"2024-09-21T07:11:47.650389+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:11:47.650389+0000","last_undegraded":"2024-09-21T07:11:47.650389+0000","last_fullsized":"2024-09-21T07:11:47.650389+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":124,"num_read_kb":127,"num_write":34,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.862517+0000","last_change":"2024-09-21T07:10:54.861948+0000","last_active":"2024-09-21T07:10:54.862517+0000","last_peered":"2024-09-21T07:10:54.862517+0000","last_clean":"2024-09-21T07:10:54.862517+0000","last_became_active":"2024-09-21T07:10:54.861488+0000","last_became_peered":"2024-09-21T07:10:54.861488+0000","last_unstale":"2024-09-21T07:10:54.862517+0000","last_undegraded":"2024-09-21T07:10:54.862517+0000","last_fullsized":"2024-09-21T07:10:54.862517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:04.439180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848454+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:10:54.848454+0000","last_peered":"2024-09-21T07:10:54.848454+0000","last_clean":"2024-09-21T07:10:54.848454+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:10:54.848454+0000","last_undegraded":"2024-09-21T07:10:54.848454+0000","last_fullsized":"2024-09-21T07:10:54.848454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848684+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:10:54.848684+0000","last_peered":"2024-09-21T07:10:54.848684+0000","last_clean":"2024-09-21T07:10:54.848684+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:10:54.848684+0000","last_undegraded":"2024-09-21T07:10:54.848684+0000","last_fullsized":"2024-09-21T07:10:54.848684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855058+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:10:54.855058+0000","last_peered":"2024-09-21T07:10:54.855058+0000","last_clean":"2024-09-21T07:10:54.855058+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:10:54.855058+0000","last_undegraded":"2024-09-21T07:10:54.855058+0000","last_fullsized":"2024-09-21T07:10:54.855058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847228+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:10:54.847228+0000","last_peered":"2024-09-21T07:10:54.847228+0000","last_clean":"2024-09-21T07:10:54.847228+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:10:54.847228+0000","last_undegraded":"2024-09-21T07:10:54.847228+0000","last_fullsized":"2024-09-21T07:10:54.847228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845213+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:10:54.845213+0000","last_peered":"2024-09-21T07:10:54.845213+0000","last_clean":"2024-09-21T07:10:54.845213+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:10:54.845213+0000","last_undegraded":"2024-09-21T07:10:54.845213+0000","last_fullsized":"2024-09-21T07:10:54.845213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842687+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:10:54.842687+0000","last_peered":"2024-09-21T07:10:54.842687+0000","last_clean":"2024-09-21T07:10:54.842687+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:10:54.842687+0000","last_undegraded":"2024-09-21T07:10:54.842687+0000","last_fullsized":"2024-09-21T07:10:54.842687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848487+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:10:54.848487+0000","last_peered":"2024-09-21T07:10:54.848487+0000","last_clean":"2024-09-21T07:10:54.848487+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:10:54.848487+0000","last_undegraded":"2024-09-21T07:10:54.848487+0000","last_fullsized":"2024-09-21T07:10:54.848487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:46.648796+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:11:46.648796+0000","last_peered":"2024-09-21T07:11:46.648796+0000","last_clean":"2024-09-21T07:11:46.648796+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:11:46.648796+0000","last_undegraded":"2024-09-21T07:11:46.648796+0000","last_fullsized":"2024-09-21T07:11:46.648796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855623+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:10:54.855623+0000","last_peered":"2024-09-21T07:10:54.855623+0000","last_clean":"2024-09-21T07:10:54.855623+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:10:54.855623+0000","last_undegraded":"2024-09-21T07:10:54.855623+0000","last_fullsized":"2024-09-21T07:10:54.855623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846947+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:10:54.846947+0000","last_peered":"2024-09-21T07:10:54.846947+0000","last_clean":"2024-09-21T07:10:54.846947+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:10:54.846947+0000","last_undegraded":"2024-09-21T07:10:54.846947+0000","last_fullsized":"2024-09-21T07:10:54.846947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842939+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:10:54.842939+0000","last_peered":"2024-09-21T07:10:54.842939+0000","last_clean":"2024-09-21T07:10:54.842939+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:10:54.842939+0000","last_undegraded":"2024-09-21T07:10:54.842939+0000","last_fullsized":"2024-09-21T07:10:54.842939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848215+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:10:54.848215+0000","last_peered":"2024-09-21T07:10:54.848215+0000","last_clean":"2024-09-21T07:10:54.848215+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:10:54.848215+0000","last_undegraded":"2024-09-21T07:10:54.848215+0000","last_fullsized":"2024-09-21T07:10:54.848215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852789+0000","last_change":"2024-09-21T07:10:05.407223+0000","last_active":"2024-09-21T07:10:54.852789+0000","last_peered":"2024-09-21T07:10:54.852789+0000","last_clean":"2024-09-21T07:10:54.852789+0000","last_became_active":"2024-09-21T07:10:05.406718+0000","last_became_peered":"2024-09-21T07:10:05.406718+0000","last_unstale":"2024-09-21T07:10:54.852789+0000","last_undegraded":"2024-09-21T07:10:54.852789+0000","last_fullsized":"2024-09-21T07:10:54.852789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:41.827806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856134+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:10:54.856134+0000","last_peered":"2024-09-21T07:10:54.856134+0000","last_clean":"2024-09-21T07:10:54.856134+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:10:54.856134+0000","last_undegraded":"2024-09-21T07:10:54.856134+0000","last_fullsized":"2024-09-21T07:10:54.856134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043016+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:10:55.043016+0000","last_peered":"2024-09-21T07:10:55.043016+0000","last_clean":"2024-09-21T07:10:55.043016+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:10:55.043016+0000","last_undegraded":"2024-09-21T07:10:55.043016+0000","last_fullsized":"2024-09-21T07:10:55.043016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847711+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:10:54.847711+0000","last_peered":"2024-09-21T07:10:54.847711+0000","last_clean":"2024-09-21T07:10:54.847711+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:10:54.847711+0000","last_undegraded":"2024-09-21T07:10:54.847711+0000","last_fullsized":"2024-09-21T07:10:54.847711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848010+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:10:54.848010+0000","last_peered":"2024-09-21T07:10:54.848010+0000","last_clean":"2024-09-21T07:10:54.848010+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:10:54.848010+0000","last_undegraded":"2024-09-21T07:10:54.848010+0000","last_fullsized":"2024-09-21T07:10:54.848010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850541+0000","last_change":"2024-09-21T07:10:05.426469+0000","last_active":"2024-09-21T07:10:54.850541+0000","last_peered":"2024-09-21T07:10:54.850541+0000","last_clean":"2024-09-21T07:10:54.850541+0000","last_became_active":"2024-09-21T07:10:05.426039+0000","last_became_peered":"2024-09-21T07:10:05.426039+0000","last_unstale":"2024-09-21T07:10:54.850541+0000","last_undegraded":"2024-09-21T07:10:54.850541+0000","last_fullsized":"2024-09-21T07:10:54.850541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:48.652956+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:11:48.652956+0000","last_peered":"2024-09-21T07:11:48.652956+0000","last_clean":"2024-09-21T07:11:48.652956+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:11:48.652956+0000","last_undegraded":"2024-09-21T07:11:48.652956+0000","last_fullsized":"2024-09-21T07:11:48.652956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857504+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:10:54.857504+0000","last_peered":"2024-09-21T07:10:54.857504+0000","last_clean":"2024-09-21T07:10:54.857504+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:10:54.857504+0000","last_undegraded":"2024-09-21T07:10:54.857504+0000","last_fullsized":"2024-09-21T07:10:54.857504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851084+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:10:54.851084+0000","last_peered":"2024-09-21T07:10:54.851084+0000","last_clean":"2024-09-21T07:10:54.851084+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:10:54.851084+0000","last_undegraded":"2024-09-21T07:10:54.851084+0000","last_fullsized":"2024-09-21T07:10:54.851084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851654+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:10:54.851654+0000","last_peered":"2024-09-21T07:10:54.851654+0000","last_clean":"2024-09-21T07:10:54.851654+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:10:54.851654+0000","last_undegraded":"2024-09-21T07:10:54.851654+0000","last_fullsized":"2024-09-21T07:10:54.851654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.044649+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:10:55.044649+0000","last_peered":"2024-09-21T07:10:55.044649+0000","last_clean":"2024-09-21T07:10:55.044649+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:10:55.044649+0000","last_undegraded":"2024-09-21T07:10:55.044649+0000","last_fullsized":"2024-09-21T07:10:55.044649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850408+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:10:54.850408+0000","last_peered":"2024-09-21T07:10:54.850408+0000","last_clean":"2024-09-21T07:10:54.850408+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:10:54.850408+0000","last_undegraded":"2024-09-21T07:10:54.850408+0000","last_fullsized":"2024-09-21T07:10:54.850408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847360+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:10:54.847360+0000","last_peered":"2024-09-21T07:10:54.847360+0000","last_clean":"2024-09-21T07:10:54.847360+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:10:54.847360+0000","last_undegraded":"2024-09-21T07:10:54.847360+0000","last_fullsized":"2024-09-21T07:10:54.847360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043281+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:10:55.043281+0000","last_peered":"2024-09-21T07:10:55.043281+0000","last_clean":"2024-09-21T07:10:55.043281+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:10:55.043281+0000","last_undegraded":"2024-09-21T07:10:55.043281+0000","last_fullsized":"2024-09-21T07:10:55.043281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852709+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:10:54.852709+0000","last_peered":"2024-09-21T07:10:54.852709+0000","last_clean":"2024-09-21T07:10:54.852709+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:10:54.852709+0000","last_undegraded":"2024-09-21T07:10:54.852709+0000","last_fullsized":"2024-09-21T07:10:54.852709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851607+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:10:54.851607+0000","last_peered":"2024-09-21T07:10:54.851607+0000","last_clean":"2024-09-21T07:10:54.851607+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:10:54.851607+0000","last_undegraded":"2024-09-21T07:10:54.851607+0000","last_fullsized":"2024-09-21T07:10:54.851607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857988+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:10:54.857988+0000","last_peered":"2024-09-21T07:10:54.857988+0000","last_clean":"2024-09-21T07:10:54.857988+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:10:54.857988+0000","last_undegraded":"2024-09-21T07:10:54.857988+0000","last_fullsized":"2024-09-21T07:10:54.857988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858404+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:10:54.858404+0000","last_peered":"2024-09-21T07:10:54.858404+0000","last_clean":"2024-09-21T07:10:54.858404+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:10:54.858404+0000","last_undegraded":"2024-09-21T07:10:54.858404+0000","last_fullsized":"2024-09-21T07:10:54.858404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855648+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:10:54.855648+0000","last_peered":"2024-09-21T07:10:54.855648+0000","last_clean":"2024-09-21T07:10:54.855648+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:10:54.855648+0000","last_undegraded":"2024-09-21T07:10:54.855648+0000","last_fullsized":"2024-09-21T07:10:54.855648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847712+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:10:54.847712+0000","last_peered":"2024-09-21T07:10:54.847712+0000","last_clean":"2024-09-21T07:10:54.847712+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:10:54.847712+0000","last_undegraded":"2024-09-21T07:10:54.847712+0000","last_fullsized":"2024-09-21T07:10:54.847712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043249+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:10:55.043249+0000","last_peered":"2024-09-21T07:10:55.043249+0000","last_clean":"2024-09-21T07:10:55.043249+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:10:55.043249+0000","last_undegraded":"2024-09-21T07:10:55.043249+0000","last_fullsized":"2024-09-21T07:10:55.043249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852135+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:10:54.852135+0000","last_peered":"2024-09-21T07:10:54.852135+0000","last_clean":"2024-09-21T07:10:54.852135+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:10:54.852135+0000","last_undegraded":"2024-09-21T07:10:54.852135+0000","last_fullsized":"2024-09-21T07:10:54.852135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851861+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:10:54.851861+0000","last_peered":"2024-09-21T07:10:54.851861+0000","last_clean":"2024-09-21T07:10:54.851861+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:10:54.851861+0000","last_undegraded":"2024-09-21T07:10:54.851861+0000","last_fullsized":"2024-09-21T07:10:54.851861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843362+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:10:54.843362+0000","last_peered":"2024-09-21T07:10:54.843362+0000","last_clean":"2024-09-21T07:10:54.843362+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:10:54.843362+0000","last_undegraded":"2024-09-21T07:10:54.843362+0000","last_fullsized":"2024-09-21T07:10:54.843362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853323+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:10:54.853323+0000","last_peered":"2024-09-21T07:10:54.853323+0000","last_clean":"2024-09-21T07:10:54.853323+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:10:54.853323+0000","last_undegraded":"2024-09-21T07:10:54.853323+0000","last_fullsized":"2024-09-21T07:10:54.853323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848242+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:10:54.848242+0000","last_peered":"2024-09-21T07:10:54.848242+0000","last_clean":"2024-09-21T07:10:54.848242+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:10:54.848242+0000","last_undegraded":"2024-09-21T07:10:54.848242+0000","last_fullsized":"2024-09-21T07:10:54.848242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.861506+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:10:54.861506+0000","last_peered":"2024-09-21T07:10:54.861506+0000","last_clean":"2024-09-21T07:10:54.861506+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:10:54.861506+0000","last_undegraded":"2024-09-21T07:10:54.861506+0000","last_fullsized":"2024-09-21T07:10:54.861506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043140+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:10:55.043140+0000","last_peered":"2024-09-21T07:10:55.043140+0000","last_clean":"2024-09-21T07:10:55.043140+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:10:55.043140+0000","last_undegraded":"2024-09-21T07:10:55.043140+0000","last_fullsized":"2024-09-21T07:10:55.043140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.860207+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:10:54.860207+0000","last_peered":"2024-09-21T07:10:54.860207+0000","last_clean":"2024-09-21T07:10:54.860207+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:10:54.860207+0000","last_undegraded":"2024-09-21T07:10:54.860207+0000","last_fullsized":"2024-09-21T07:10:54.860207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.845743+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:10:54.845743+0000","last_peered":"2024-09-21T07:10:54.845743+0000","last_clean":"2024-09-21T07:10:54.845743+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:10:54.845743+0000","last_undegraded":"2024-09-21T07:10:54.845743+0000","last_fullsized":"2024-09-21T07:10:54.845743+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847513+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:10:54.847513+0000","last_peered":"2024-09-21T07:10:54.847513+0000","last_clean":"2024-09-21T07:10:54.847513+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:10:54.847513+0000","last_undegraded":"2024-09-21T07:10:54.847513+0000","last_fullsized":"2024-09-21T07:10:54.847513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848613+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:10:54.848613+0000","last_peered":"2024-09-21T07:10:54.848613+0000","last_clean":"2024-09-21T07:10:54.848613+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:10:54.848613+0000","last_undegraded":"2024-09-21T07:10:54.848613+0000","last_fullsized":"2024-09-21T07:10:54.848613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856045+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:10:54.856045+0000","last_peered":"2024-09-21T07:10:54.856045+0000","last_clean":"2024-09-21T07:10:54.856045+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:10:54.856045+0000","last_undegraded":"2024-09-21T07:10:54.856045+0000","last_fullsized":"2024-09-21T07:10:54.856045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844872+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:10:54.844872+0000","last_peered":"2024-09-21T07:10:54.844872+0000","last_clean":"2024-09-21T07:10:54.844872+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:10:54.844872+0000","last_undegraded":"2024-09-21T07:10:54.844872+0000","last_fullsized":"2024-09-21T07:10:54.844872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847825+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:10:54.847825+0000","last_peered":"2024-09-21T07:10:54.847825+0000","last_clean":"2024-09-21T07:10:54.847825+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:10:54.847825+0000","last_undegraded":"2024-09-21T07:10:54.847825+0000","last_fullsized":"2024-09-21T07:10:54.847825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857433+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:10:54.857433+0000","last_peered":"2024-09-21T07:10:54.857433+0000","last_clean":"2024-09-21T07:10:54.857433+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:10:54.857433+0000","last_undegraded":"2024-09-21T07:10:54.857433+0000","last_fullsized":"2024-09-21T07:10:54.857433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855117+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:10:54.855117+0000","last_peered":"2024-09-21T07:10:54.855117+0000","last_clean":"2024-09-21T07:10:54.855117+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:10:54.855117+0000","last_undegraded":"2024-09-21T07:10:54.855117+0000","last_fullsized":"2024-09-21T07:10:54.855117+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847084+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:10:54.847084+0000","last_peered":"2024-09-21T07:10:54.847084+0000","last_clean":"2024-09-21T07:10:54.847084+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:10:54.847084+0000","last_undegraded":"2024-09-21T07:10:54.847084+0000","last_fullsized":"2024-09-21T07:10:54.847084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853198+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:10:54.853198+0000","last_peered":"2024-09-21T07:10:54.853198+0000","last_clean":"2024-09-21T07:10:54.853198+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:10:54.853198+0000","last_undegraded":"2024-09-21T07:10:54.853198+0000","last_fullsized":"2024-09-21T07:10:54.853198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70213,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":381,"num_read_kb":264,"num_write":166,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":210639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":496595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972393,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1662976,"data_stored":1150444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135920,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":500905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495673,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1092638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":491985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1087835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":497026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":492179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":559457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":558107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705760,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":493373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:11:54.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T07:11:55.057 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:11:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='client.25162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10, 5, 1]}]': finished 2024-09-21T07:11:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-21T07:11:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-21T07:11:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T07:11:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:55 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:11:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='client.25162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10, 5, 1]}]': finished 2024-09-21T07:11:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-21T07:11:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-21T07:11:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T07:11:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:55 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:11:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='client.25162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10, 5, 1]}]': finished 2024-09-21T07:11:55.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-21T07:11:55.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-21T07:11:55.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T07:11:55.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:55.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:55 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 -- 172.21.15.27:0/3812887669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c105d80 msgr2=0x7fa96c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 --2- 172.21.15.27:0/3812887669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c105d80 0x7fa96c106190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa960009900 tx=0x7fa96002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 -- 172.21.15.27:0/3812887669 shutdown_connections 2024-09-21T07:11:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 --2- 172.21.15.27:0/3812887669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa96c10b6c0 0x7fa96c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 --2- 172.21.15.27:0/3812887669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa96c10c010 0x7fa96c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 --2- 172.21.15.27:0/3812887669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c105d80 0x7fa96c106190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 -- 172.21.15.27:0/3812887669 >> 172.21.15.27:0/3812887669 conn(0x7fa96c101810 msgr2=0x7fa96c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 -- 172.21.15.27:0/3812887669 shutdown_connections 2024-09-21T07:11:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.030+0000 7fa974800640 1 -- 172.21.15.27:0/3812887669 wait complete. 2024-09-21T07:11:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.031+0000 7fa974800640 1 Processor -- start 2024-09-21T07:11:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.032+0000 7fa974800640 1 -- start start 2024-09-21T07:11:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.032+0000 7fa974800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa96c105d80 0x7fa96c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.032+0000 7fa974800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa96c10b6c0 0x7fa96c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.032+0000 7fa974800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c10c010 0x7fa96c1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.032+0000 7fa974800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c115cd0 con 0x7fa96c10c010 2024-09-21T07:11:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.032+0000 7fa974800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c115b50 con 0x7fa96c105d80 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.032+0000 7fa974800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c115e50 con 0x7fa96c10b6c0 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa96c105d80 0x7fa96c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa96c10b6c0 0x7fa96c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa96c105d80 0x7fa96c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34990/0 (socket says 172.21.15.27:34990) 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971e00640 1 -- 172.21.15.27:0/970189504 learned_addr learned my addr 172.21.15.27:0/970189504 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa972800640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c10c010 0x7fa96c1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971400640 1 -- 172.21.15.27:0/970189504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa96c105d80 msgr2=0x7fa96c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971400640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa96c105d80 0x7fa96c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971400640 1 -- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c10c010 msgr2=0x7fa96c1a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971400640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c10c010 0x7fa96c1a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.033+0000 7fa971400640 1 -- 172.21.15.27:0/970189504 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c1a2670 con 0x7fa96c10b6c0 2024-09-21T07:11:56.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.034+0000 7fa971400640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa96c10b6c0 0x7fa96c19cca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa96c06caa0 tx=0x7fa95c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:56.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.034+0000 7fa966a00640 1 -- 172.21.15.27:0/970189504 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c009e60 con 0x7fa96c10b6c0 2024-09-21T07:11:56.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.034+0000 7fa966a00640 1 -- 172.21.15.27:0/970189504 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa95c00ccc0 con 0x7fa96c10b6c0 2024-09-21T07:11:56.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.034+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c1a2960 con 0x7fa96c10b6c0 2024-09-21T07:11:56.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.034+0000 7fa966a00640 1 -- 172.21.15.27:0/970189504 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0187a0 con 0x7fa96c10b6c0 2024-09-21T07:11:56.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.034+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c1a90c0 con 0x7fa96c10b6c0 2024-09-21T07:11:56.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.035+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c106190 con 0x7fa96c10b6c0 2024-09-21T07:11:56.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.037+0000 7fa966a00640 1 -- 172.21.15.27:0/970189504 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa95c010430 con 0x7fa96c10b6c0 2024-09-21T07:11:56.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.040+0000 7fa966a00640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9540b6670 0x7fa9540b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:56.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.040+0000 7fa966a00640 1 -- 172.21.15.27:0/970189504 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa95c0e7b70 con 0x7fa96c10b6c0 2024-09-21T07:11:56.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.040+0000 7fa971e00640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9540b6670 0x7fa9540b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:56.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.041+0000 7fa971e00640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9540b6670 0x7fa9540b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa960031440 tx=0x7fa9600058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:56.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.043+0000 7fa966a00640 1 -- 172.21.15.27:0/970189504 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa95c0addd0 con 0x7fa96c10b6c0 2024-09-21T07:11:56.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.299+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa96c063800 con 0x7fa9540b6670 2024-09-21T07:11:56.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.308+0000 7fa966a00640 1 -- 172.21.15.27:0/970189504 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340349 (secure 0 0 0) 0x7fa96c063800 con 0x7fa9540b6670 2024-09-21T07:11:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:56.317 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:11:56.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9540b6670 msgr2=0x7fa9540b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:56.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9540b6670 0x7fa9540b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa960031440 tx=0x7fa9600058d0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa96c10b6c0 msgr2=0x7fa96c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:56.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa96c10b6c0 0x7fa96c19cca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa96c06caa0 tx=0x7fa95c00ed90 comp rx=0 tx=0).stop 2024-09-21T07:11:56.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 shutdown_connections 2024-09-21T07:11:56.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa96c10b6c0 0x7fa96c19cca0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9540b6670 0x7fa9540b8b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa96c10c010 0x7fa96c1a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 --2- 172.21.15.27:0/970189504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa96c105d80 0x7fa96c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:56.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 >> 172.21.15.27:0/970189504 conn(0x7fa96c101810 msgr2=0x7fa96c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:56.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 shutdown_connections 2024-09-21T07:11:56.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:56.315+0000 7fa974800640 1 -- 172.21.15.27:0/970189504 wait complete. 2024-09-21T07:11:57.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:56 smithi027 ceph-mon[30779]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:11:57.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:56 smithi027 ceph-mon[30779]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T07:11:57.083 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-21T07:11:55.828668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660815,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":487,"num_read_kb":343,"num_write":291,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387692,"kb_used_data":13468,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554132,"statfs":{"total":1151940427776,"available":1151543431168,"internally_reserved":0,"allocated":13791232,"data_stored":7936587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":25,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.432531"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.844106+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:10:54.844106+0000","last_peered":"2024-09-21T07:10:54.844106+0000","last_clean":"2024-09-21T07:10:54.844106+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:10:54.844106+0000","last_undegraded":"2024-09-21T07:10:54.844106+0000","last_fullsized":"2024-09-21T07:10:54.844106+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857694+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:10:54.857694+0000","last_peered":"2024-09-21T07:10:54.857694+0000","last_clean":"2024-09-21T07:10:54.857694+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:10:54.857694+0000","last_undegraded":"2024-09-21T07:10:54.857694+0000","last_fullsized":"2024-09-21T07:10:54.857694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850712+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:10:54.850712+0000","last_peered":"2024-09-21T07:10:54.850712+0000","last_clean":"2024-09-21T07:10:54.850712+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:10:54.850712+0000","last_undegraded":"2024-09-21T07:10:54.850712+0000","last_fullsized":"2024-09-21T07:10:54.850712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858644+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:10:54.858644+0000","last_peered":"2024-09-21T07:10:54.858644+0000","last_clean":"2024-09-21T07:10:54.858644+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:10:54.858644+0000","last_undegraded":"2024-09-21T07:10:54.858644+0000","last_fullsized":"2024-09-21T07:10:54.858644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847175+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:10:54.847175+0000","last_peered":"2024-09-21T07:10:54.847175+0000","last_clean":"2024-09-21T07:10:54.847175+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:10:54.847175+0000","last_undegraded":"2024-09-21T07:10:54.847175+0000","last_fullsized":"2024-09-21T07:10:54.847175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854448+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:10:54.854448+0000","last_peered":"2024-09-21T07:10:54.854448+0000","last_clean":"2024-09-21T07:10:54.854448+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:10:54.854448+0000","last_undegraded":"2024-09-21T07:10:54.854448+0000","last_fullsized":"2024-09-21T07:10:54.854448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858193+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:10:54.858193+0000","last_peered":"2024-09-21T07:10:54.858193+0000","last_clean":"2024-09-21T07:10:54.858193+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:10:54.858193+0000","last_undegraded":"2024-09-21T07:10:54.858193+0000","last_fullsized":"2024-09-21T07:10:54.858193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848421+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:10:54.848421+0000","last_peered":"2024-09-21T07:10:54.848421+0000","last_clean":"2024-09-21T07:10:54.848421+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:10:54.848421+0000","last_undegraded":"2024-09-21T07:10:54.848421+0000","last_fullsized":"2024-09-21T07:10:54.848421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854828+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:10:54.854828+0000","last_peered":"2024-09-21T07:10:54.854828+0000","last_clean":"2024-09-21T07:10:54.854828+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:10:54.854828+0000","last_undegraded":"2024-09-21T07:10:54.854828+0000","last_fullsized":"2024-09-21T07:10:54.854828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857732+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:10:54.857732+0000","last_peered":"2024-09-21T07:10:54.857732+0000","last_clean":"2024-09-21T07:10:54.857732+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:10:54.857732+0000","last_undegraded":"2024-09-21T07:10:54.857732+0000","last_fullsized":"2024-09-21T07:10:54.857732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.834145+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:11:54.834145+0000","last_peered":"2024-09-21T07:11:54.834145+0000","last_clean":"2024-09-21T07:11:54.834145+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:11:54.834145+0000","last_undegraded":"2024-09-21T07:11:54.834145+0000","last_fullsized":"2024-09-21T07:11:54.834145+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854459+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:10:54.854459+0000","last_peered":"2024-09-21T07:10:54.854459+0000","last_clean":"2024-09-21T07:10:54.854459+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:10:54.854459+0000","last_undegraded":"2024-09-21T07:10:54.854459+0000","last_fullsized":"2024-09-21T07:10:54.854459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.880909+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:11:54.880909+0000","last_peered":"2024-09-21T07:11:54.880909+0000","last_clean":"2024-09-21T07:11:54.880909+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:11:54.880909+0000","last_undegraded":"2024-09-21T07:11:54.880909+0000","last_fullsized":"2024-09-21T07:11:54.880909+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857487+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:10:54.857487+0000","last_peered":"2024-09-21T07:10:54.857487+0000","last_clean":"2024-09-21T07:10:54.857487+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:10:54.857487+0000","last_undegraded":"2024-09-21T07:10:54.857487+0000","last_fullsized":"2024-09-21T07:10:54.857487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855146+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:10:54.855146+0000","last_peered":"2024-09-21T07:10:54.855146+0000","last_clean":"2024-09-21T07:10:54.855146+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:10:54.855146+0000","last_undegraded":"2024-09-21T07:10:54.855146+0000","last_fullsized":"2024-09-21T07:10:54.855146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856487+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:10:54.856487+0000","last_peered":"2024-09-21T07:10:54.856487+0000","last_clean":"2024-09-21T07:10:54.856487+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:10:54.856487+0000","last_undegraded":"2024-09-21T07:10:54.856487+0000","last_fullsized":"2024-09-21T07:10:54.856487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.867034+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:10:54.867034+0000","last_peered":"2024-09-21T07:10:54.867034+0000","last_clean":"2024-09-21T07:10:54.867034+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:10:54.867034+0000","last_undegraded":"2024-09-21T07:10:54.867034+0000","last_fullsized":"2024-09-21T07:10:54.867034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.854799+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:10:54.854799+0000","last_peered":"2024-09-21T07:10:54.854799+0000","last_clean":"2024-09-21T07:10:54.854799+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:10:54.854799+0000","last_undegraded":"2024-09-21T07:10:54.854799+0000","last_fullsized":"2024-09-21T07:10:54.854799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851783+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:10:54.851783+0000","last_peered":"2024-09-21T07:10:54.851783+0000","last_clean":"2024-09-21T07:10:54.851783+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:10:54.851783+0000","last_undegraded":"2024-09-21T07:10:54.851783+0000","last_fullsized":"2024-09-21T07:10:54.851783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856047+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:10:54.856047+0000","last_peered":"2024-09-21T07:10:54.856047+0000","last_clean":"2024-09-21T07:10:54.856047+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:10:54.856047+0000","last_undegraded":"2024-09-21T07:10:54.856047+0000","last_fullsized":"2024-09-21T07:10:54.856047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843895+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:10:54.843895+0000","last_peered":"2024-09-21T07:10:54.843895+0000","last_clean":"2024-09-21T07:10:54.843895+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:10:54.843895+0000","last_undegraded":"2024-09-21T07:10:54.843895+0000","last_fullsized":"2024-09-21T07:10:54.843895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857546+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:10:54.857546+0000","last_peered":"2024-09-21T07:10:54.857546+0000","last_clean":"2024-09-21T07:10:54.857546+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:10:54.857546+0000","last_undegraded":"2024-09-21T07:10:54.857546+0000","last_fullsized":"2024-09-21T07:10:54.857546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847552+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:10:54.847552+0000","last_peered":"2024-09-21T07:10:54.847552+0000","last_clean":"2024-09-21T07:10:54.847552+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:10:54.847552+0000","last_undegraded":"2024-09-21T07:10:54.847552+0000","last_fullsized":"2024-09-21T07:10:54.847552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855277+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:10:54.855277+0000","last_peered":"2024-09-21T07:10:54.855277+0000","last_clean":"2024-09-21T07:10:54.855277+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:10:54.855277+0000","last_undegraded":"2024-09-21T07:10:54.855277+0000","last_fullsized":"2024-09-21T07:10:54.855277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847198+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:10:54.847198+0000","last_peered":"2024-09-21T07:10:54.847198+0000","last_clean":"2024-09-21T07:10:54.847198+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:10:54.847198+0000","last_undegraded":"2024-09-21T07:10:54.847198+0000","last_fullsized":"2024-09-21T07:10:54.847198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855459+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:10:54.855459+0000","last_peered":"2024-09-21T07:10:54.855459+0000","last_clean":"2024-09-21T07:10:54.855459+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:10:54.855459+0000","last_undegraded":"2024-09-21T07:10:54.855459+0000","last_fullsized":"2024-09-21T07:10:54.855459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852078+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:10:54.852078+0000","last_peered":"2024-09-21T07:10:54.852078+0000","last_clean":"2024-09-21T07:10:54.852078+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:10:54.852078+0000","last_undegraded":"2024-09-21T07:10:54.852078+0000","last_fullsized":"2024-09-21T07:10:54.852078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858105+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:10:54.858105+0000","last_peered":"2024-09-21T07:10:54.858105+0000","last_clean":"2024-09-21T07:10:54.858105+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:10:54.858105+0000","last_undegraded":"2024-09-21T07:10:54.858105+0000","last_fullsized":"2024-09-21T07:10:54.858105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'10","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:07.358581+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:11:07.358581+0000","last_peered":"2024-09-21T07:11:07.358581+0000","last_clean":"2024-09-21T07:11:07.358581+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:11:07.358581+0000","last_undegraded":"2024-09-21T07:11:07.358581+0000","last_fullsized":"2024-09-21T07:11:07.358581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855905+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:10:54.855905+0000","last_peered":"2024-09-21T07:10:54.855905+0000","last_clean":"2024-09-21T07:10:54.855905+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:10:54.855905+0000","last_undegraded":"2024-09-21T07:10:54.855905+0000","last_fullsized":"2024-09-21T07:10:54.855905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043363+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:10:55.043363+0000","last_peered":"2024-09-21T07:10:55.043363+0000","last_clean":"2024-09-21T07:10:55.043363+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:10:55.043363+0000","last_undegraded":"2024-09-21T07:10:55.043363+0000","last_fullsized":"2024-09-21T07:10:55.043363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.858281+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:10:54.858281+0000","last_peered":"2024-09-21T07:10:54.858281+0000","last_clean":"2024-09-21T07:10:54.858281+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:10:54.858281+0000","last_undegraded":"2024-09-21T07:10:54.858281+0000","last_fullsized":"2024-09-21T07:10:54.858281+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851612+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:10:54.851612+0000","last_peered":"2024-09-21T07:10:54.851612+0000","last_clean":"2024-09-21T07:10:54.851612+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:10:54.851612+0000","last_undegraded":"2024-09-21T07:10:54.851612+0000","last_fullsized":"2024-09-21T07:10:54.851612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846903+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:10:54.846903+0000","last_peered":"2024-09-21T07:10:54.846903+0000","last_clean":"2024-09-21T07:10:54.846903+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:10:54.846903+0000","last_undegraded":"2024-09-21T07:10:54.846903+0000","last_fullsized":"2024-09-21T07:10:54.846903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843652+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:10:54.843652+0000","last_peered":"2024-09-21T07:10:54.843652+0000","last_clean":"2024-09-21T07:10:54.843652+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:10:54.843652+0000","last_undegraded":"2024-09-21T07:10:54.843652+0000","last_fullsized":"2024-09-21T07:10:54.843652+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857264+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:10:54.857264+0000","last_peered":"2024-09-21T07:10:54.857264+0000","last_clean":"2024-09-21T07:10:54.857264+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:10:54.857264+0000","last_undegraded":"2024-09-21T07:10:54.857264+0000","last_fullsized":"2024-09-21T07:10:54.857264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847609+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:10:54.847609+0000","last_peered":"2024-09-21T07:10:54.847609+0000","last_clean":"2024-09-21T07:10:54.847609+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:10:54.847609+0000","last_undegraded":"2024-09-21T07:10:54.847609+0000","last_fullsized":"2024-09-21T07:10:54.847609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881009+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:11:54.881009+0000","last_peered":"2024-09-21T07:11:54.881009+0000","last_clean":"2024-09-21T07:11:54.881009+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:11:54.881009+0000","last_undegraded":"2024-09-21T07:11:54.881009+0000","last_fullsized":"2024-09-21T07:11:54.881009+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043157+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:10:55.043157+0000","last_peered":"2024-09-21T07:10:55.043157+0000","last_clean":"2024-09-21T07:10:55.043157+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:10:55.043157+0000","last_undegraded":"2024-09-21T07:10:55.043157+0000","last_fullsized":"2024-09-21T07:10:55.043157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.833552+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:11:54.833552+0000","last_peered":"2024-09-21T07:11:54.833552+0000","last_clean":"2024-09-21T07:11:54.833552+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:11:54.833552+0000","last_undegraded":"2024-09-21T07:11:54.833552+0000","last_fullsized":"2024-09-21T07:11:54.833552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":105,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:49.653199+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:11:49.653199+0000","last_peered":"2024-09-21T07:11:49.653199+0000","last_clean":"2024-09-21T07:11:49.653199+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:11:49.653199+0000","last_undegraded":"2024-09-21T07:11:49.653199+0000","last_fullsized":"2024-09-21T07:11:49.653199+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043137+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:10:55.043137+0000","last_peered":"2024-09-21T07:10:55.043137+0000","last_clean":"2024-09-21T07:10:55.043137+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:10:55.043137+0000","last_undegraded":"2024-09-21T07:10:55.043137+0000","last_fullsized":"2024-09-21T07:10:55.043137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042862+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:10:55.042862+0000","last_peered":"2024-09-21T07:10:55.042862+0000","last_clean":"2024-09-21T07:10:55.042862+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:10:55.042862+0000","last_undegraded":"2024-09-21T07:10:55.042862+0000","last_fullsized":"2024-09-21T07:10:55.042862+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.880966+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:11:54.880966+0000","last_peered":"2024-09-21T07:11:54.880966+0000","last_clean":"2024-09-21T07:11:54.880966+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:11:54.880966+0000","last_undegraded":"2024-09-21T07:11:54.880966+0000","last_fullsized":"2024-09-21T07:11:54.880966+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.834052+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:11:54.834052+0000","last_peered":"2024-09-21T07:11:54.834052+0000","last_clean":"2024-09-21T07:11:54.834052+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:11:54.834052+0000","last_undegraded":"2024-09-21T07:11:54.834052+0000","last_fullsized":"2024-09-21T07:11:54.834052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856634+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:10:54.856634+0000","last_peered":"2024-09-21T07:10:54.856634+0000","last_clean":"2024-09-21T07:10:54.856634+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:10:54.856634+0000","last_undegraded":"2024-09-21T07:10:54.856634+0000","last_fullsized":"2024-09-21T07:10:54.856634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855230+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:10:54.855230+0000","last_peered":"2024-09-21T07:10:54.855230+0000","last_clean":"2024-09-21T07:10:54.855230+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:10:54.855230+0000","last_undegraded":"2024-09-21T07:10:54.855230+0000","last_fullsized":"2024-09-21T07:10:54.855230+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881032+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:11:54.881032+0000","last_peered":"2024-09-21T07:11:54.881032+0000","last_clean":"2024-09-21T07:11:54.881032+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:11:54.881032+0000","last_undegraded":"2024-09-21T07:11:54.881032+0000","last_fullsized":"2024-09-21T07:11:54.881032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881155+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:11:54.881155+0000","last_peered":"2024-09-21T07:11:54.881155+0000","last_clean":"2024-09-21T07:11:54.881155+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:11:54.881155+0000","last_undegraded":"2024-09-21T07:11:54.881155+0000","last_fullsized":"2024-09-21T07:11:54.881155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881272+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:11:54.881272+0000","last_peered":"2024-09-21T07:11:54.881272+0000","last_clean":"2024-09-21T07:11:54.881272+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:11:54.881272+0000","last_undegraded":"2024-09-21T07:11:54.881272+0000","last_fullsized":"2024-09-21T07:11:54.881272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852850+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:10:54.852850+0000","last_peered":"2024-09-21T07:10:54.852850+0000","last_clean":"2024-09-21T07:10:54.852850+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:10:54.852850+0000","last_undegraded":"2024-09-21T07:10:54.852850+0000","last_fullsized":"2024-09-21T07:10:54.852850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847470+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:10:54.847470+0000","last_peered":"2024-09-21T07:10:54.847470+0000","last_clean":"2024-09-21T07:10:54.847470+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:10:54.847470+0000","last_undegraded":"2024-09-21T07:10:54.847470+0000","last_fullsized":"2024-09-21T07:10:54.847470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881123+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:11:54.881123+0000","last_peered":"2024-09-21T07:11:54.881123+0000","last_clean":"2024-09-21T07:11:54.881123+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:11:54.881123+0000","last_undegraded":"2024-09-21T07:11:54.881123+0000","last_fullsized":"2024-09-21T07:11:54.881123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.834545+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:11:54.834545+0000","last_peered":"2024-09-21T07:11:54.834545+0000","last_clean":"2024-09-21T07:11:54.834545+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:11:54.834545+0000","last_undegraded":"2024-09-21T07:11:54.834545+0000","last_fullsized":"2024-09-21T07:11:54.834545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847352+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:10:54.847352+0000","last_peered":"2024-09-21T07:10:54.847352+0000","last_clean":"2024-09-21T07:10:54.847352+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:10:54.847352+0000","last_undegraded":"2024-09-21T07:10:54.847352+0000","last_fullsized":"2024-09-21T07:10:54.847352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881249+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:11:54.881249+0000","last_peered":"2024-09-21T07:11:54.881249+0000","last_clean":"2024-09-21T07:11:54.881249+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:11:54.881249+0000","last_undegraded":"2024-09-21T07:11:54.881249+0000","last_fullsized":"2024-09-21T07:11:54.881249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043503+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:10:55.043503+0000","last_peered":"2024-09-21T07:10:55.043503+0000","last_clean":"2024-09-21T07:10:55.043503+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:10:55.043503+0000","last_undegraded":"2024-09-21T07:10:55.043503+0000","last_fullsized":"2024-09-21T07:10:55.043503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.832078+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:11:54.832078+0000","last_peered":"2024-09-21T07:11:54.832078+0000","last_clean":"2024-09-21T07:11:54.832078+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:11:54.832078+0000","last_undegraded":"2024-09-21T07:11:54.832078+0000","last_fullsized":"2024-09-21T07:11:54.832078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856776+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:10:54.856776+0000","last_peered":"2024-09-21T07:10:54.856776+0000","last_clean":"2024-09-21T07:10:54.856776+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:10:54.856776+0000","last_undegraded":"2024-09-21T07:10:54.856776+0000","last_fullsized":"2024-09-21T07:10:54.856776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.864304+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:10:54.864304+0000","last_peered":"2024-09-21T07:10:54.864304+0000","last_clean":"2024-09-21T07:10:54.864304+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:10:54.864304+0000","last_undegraded":"2024-09-21T07:10:54.864304+0000","last_fullsized":"2024-09-21T07:10:54.864304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.833079+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:11:54.833079+0000","last_peered":"2024-09-21T07:11:54.833079+0000","last_clean":"2024-09-21T07:11:54.833079+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:11:54.833079+0000","last_undegraded":"2024-09-21T07:11:54.833079+0000","last_fullsized":"2024-09-21T07:11:54.833079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.833460+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:11:54.833460+0000","last_peered":"2024-09-21T07:11:54.833460+0000","last_clean":"2024-09-21T07:11:54.833460+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:11:54.833460+0000","last_undegraded":"2024-09-21T07:11:54.833460+0000","last_fullsized":"2024-09-21T07:11:54.833460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":56,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856848+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:10:54.856848+0000","last_peered":"2024-09-21T07:10:54.856848+0000","last_clean":"2024-09-21T07:10:54.856848+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:10:54.856848+0000","last_undegraded":"2024-09-21T07:10:54.856848+0000","last_fullsized":"2024-09-21T07:10:54.856848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043501+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:10:55.043501+0000","last_peered":"2024-09-21T07:10:55.043501+0000","last_clean":"2024-09-21T07:10:55.043501+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:10:55.043501+0000","last_undegraded":"2024-09-21T07:10:55.043501+0000","last_fullsized":"2024-09-21T07:10:55.043501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.880786+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:11:54.880786+0000","last_peered":"2024-09-21T07:11:54.880786+0000","last_clean":"2024-09-21T07:11:54.880786+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:11:54.880786+0000","last_undegraded":"2024-09-21T07:11:54.880786+0000","last_fullsized":"2024-09-21T07:11:54.880786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":112,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:53.660930+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:11:53.660930+0000","last_peered":"2024-09-21T07:11:53.660930+0000","last_clean":"2024-09-21T07:11:53.660930+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:11:53.660930+0000","last_undegraded":"2024-09-21T07:11:53.660930+0000","last_fullsized":"2024-09-21T07:11:53.660930+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843898+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:10:54.843898+0000","last_peered":"2024-09-21T07:10:54.843898+0000","last_clean":"2024-09-21T07:10:54.843898+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:10:54.843898+0000","last_undegraded":"2024-09-21T07:10:54.843898+0000","last_fullsized":"2024-09-21T07:10:54.843898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.880974+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:11:54.880974+0000","last_peered":"2024-09-21T07:11:54.880974+0000","last_clean":"2024-09-21T07:11:54.880974+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:11:54.880974+0000","last_undegraded":"2024-09-21T07:11:54.880974+0000","last_fullsized":"2024-09-21T07:11:54.880974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.833709+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:11:54.833709+0000","last_peered":"2024-09-21T07:11:54.833709+0000","last_clean":"2024-09-21T07:11:54.833709+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:11:54.833709+0000","last_undegraded":"2024-09-21T07:11:54.833709+0000","last_fullsized":"2024-09-21T07:11:54.833709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852002+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:10:54.852002+0000","last_peered":"2024-09-21T07:10:54.852002+0000","last_clean":"2024-09-21T07:10:54.852002+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:10:54.852002+0000","last_undegraded":"2024-09-21T07:10:54.852002+0000","last_fullsized":"2024-09-21T07:10:54.852002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.042934+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:10:55.042934+0000","last_peered":"2024-09-21T07:10:55.042934+0000","last_clean":"2024-09-21T07:10:55.042934+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:10:55.042934+0000","last_undegraded":"2024-09-21T07:10:55.042934+0000","last_fullsized":"2024-09-21T07:10:55.042934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852516+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:10:54.852516+0000","last_peered":"2024-09-21T07:10:54.852516+0000","last_clean":"2024-09-21T07:10:54.852516+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:10:54.852516+0000","last_undegraded":"2024-09-21T07:10:54.852516+0000","last_fullsized":"2024-09-21T07:10:54.852516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.834016+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:11:54.834016+0000","last_peered":"2024-09-21T07:11:54.834016+0000","last_clean":"2024-09-21T07:11:54.834016+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:11:54.834016+0000","last_undegraded":"2024-09-21T07:11:54.834016+0000","last_fullsized":"2024-09-21T07:11:54.834016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857933+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:10:54.857933+0000","last_peered":"2024-09-21T07:10:54.857933+0000","last_clean":"2024-09-21T07:10:54.857933+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:10:54.857933+0000","last_undegraded":"2024-09-21T07:10:54.857933+0000","last_fullsized":"2024-09-21T07:10:54.857933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":207,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:52.659704+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:11:52.659704+0000","last_peered":"2024-09-21T07:11:52.659704+0000","last_clean":"2024-09-21T07:11:52.659704+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:11:52.659704+0000","last_undegraded":"2024-09-21T07:11:52.659704+0000","last_fullsized":"2024-09-21T07:11:52.659704+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":34,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.862517+0000","last_change":"2024-09-21T07:10:54.861948+0000","last_active":"2024-09-21T07:10:54.862517+0000","last_peered":"2024-09-21T07:10:54.862517+0000","last_clean":"2024-09-21T07:10:54.862517+0000","last_became_active":"2024-09-21T07:10:54.861488+0000","last_became_peered":"2024-09-21T07:10:54.861488+0000","last_unstale":"2024-09-21T07:10:54.862517+0000","last_undegraded":"2024-09-21T07:10:54.862517+0000","last_fullsized":"2024-09-21T07:10:54.862517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:04.439180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848454+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:10:54.848454+0000","last_peered":"2024-09-21T07:10:54.848454+0000","last_clean":"2024-09-21T07:10:54.848454+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:10:54.848454+0000","last_undegraded":"2024-09-21T07:10:54.848454+0000","last_fullsized":"2024-09-21T07:10:54.848454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.834744+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:11:54.834744+0000","last_peered":"2024-09-21T07:11:54.834744+0000","last_clean":"2024-09-21T07:11:54.834744+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:11:54.834744+0000","last_undegraded":"2024-09-21T07:11:54.834744+0000","last_fullsized":"2024-09-21T07:11:54.834744+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855058+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:10:54.855058+0000","last_peered":"2024-09-21T07:10:54.855058+0000","last_clean":"2024-09-21T07:10:54.855058+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:10:54.855058+0000","last_undegraded":"2024-09-21T07:10:54.855058+0000","last_fullsized":"2024-09-21T07:10:54.855058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847228+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:10:54.847228+0000","last_peered":"2024-09-21T07:10:54.847228+0000","last_clean":"2024-09-21T07:10:54.847228+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:10:54.847228+0000","last_undegraded":"2024-09-21T07:10:54.847228+0000","last_fullsized":"2024-09-21T07:10:54.847228+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881170+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:11:54.881170+0000","last_peered":"2024-09-21T07:11:54.881170+0000","last_clean":"2024-09-21T07:11:54.881170+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:11:54.881170+0000","last_undegraded":"2024-09-21T07:11:54.881170+0000","last_fullsized":"2024-09-21T07:11:54.881170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842687+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:10:54.842687+0000","last_peered":"2024-09-21T07:10:54.842687+0000","last_clean":"2024-09-21T07:10:54.842687+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:10:54.842687+0000","last_undegraded":"2024-09-21T07:10:54.842687+0000","last_fullsized":"2024-09-21T07:10:54.842687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848487+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:10:54.848487+0000","last_peered":"2024-09-21T07:10:54.848487+0000","last_clean":"2024-09-21T07:10:54.848487+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:10:54.848487+0000","last_undegraded":"2024-09-21T07:10:54.848487+0000","last_fullsized":"2024-09-21T07:10:54.848487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":106,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:11:51.657874+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:11:51.657874+0000","last_peered":"2024-09-21T07:11:51.657874+0000","last_clean":"2024-09-21T07:11:51.657874+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:11:51.657874+0000","last_undegraded":"2024-09-21T07:11:51.657874+0000","last_fullsized":"2024-09-21T07:11:51.657874+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855623+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:10:54.855623+0000","last_peered":"2024-09-21T07:10:54.855623+0000","last_clean":"2024-09-21T07:10:54.855623+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:10:54.855623+0000","last_undegraded":"2024-09-21T07:10:54.855623+0000","last_fullsized":"2024-09-21T07:10:54.855623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846947+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:10:54.846947+0000","last_peered":"2024-09-21T07:10:54.846947+0000","last_clean":"2024-09-21T07:10:54.846947+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:10:54.846947+0000","last_undegraded":"2024-09-21T07:10:54.846947+0000","last_fullsized":"2024-09-21T07:10:54.846947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.842939+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:10:54.842939+0000","last_peered":"2024-09-21T07:10:54.842939+0000","last_clean":"2024-09-21T07:10:54.842939+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:10:54.842939+0000","last_undegraded":"2024-09-21T07:10:54.842939+0000","last_fullsized":"2024-09-21T07:10:54.842939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.832517+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:11:54.832517+0000","last_peered":"2024-09-21T07:11:54.832517+0000","last_clean":"2024-09-21T07:11:54.832517+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:11:54.832517+0000","last_undegraded":"2024-09-21T07:11:54.832517+0000","last_fullsized":"2024-09-21T07:11:54.832517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852789+0000","last_change":"2024-09-21T07:10:05.407223+0000","last_active":"2024-09-21T07:10:54.852789+0000","last_peered":"2024-09-21T07:10:54.852789+0000","last_clean":"2024-09-21T07:10:54.852789+0000","last_became_active":"2024-09-21T07:10:05.406718+0000","last_became_peered":"2024-09-21T07:10:05.406718+0000","last_unstale":"2024-09-21T07:10:54.852789+0000","last_undegraded":"2024-09-21T07:10:54.852789+0000","last_fullsized":"2024-09-21T07:10:54.852789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:41.827806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856134+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:10:54.856134+0000","last_peered":"2024-09-21T07:10:54.856134+0000","last_clean":"2024-09-21T07:10:54.856134+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:10:54.856134+0000","last_undegraded":"2024-09-21T07:10:54.856134+0000","last_fullsized":"2024-09-21T07:10:54.856134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043016+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:10:55.043016+0000","last_peered":"2024-09-21T07:10:55.043016+0000","last_clean":"2024-09-21T07:10:55.043016+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:10:55.043016+0000","last_undegraded":"2024-09-21T07:10:55.043016+0000","last_fullsized":"2024-09-21T07:10:55.043016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847711+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:10:54.847711+0000","last_peered":"2024-09-21T07:10:54.847711+0000","last_clean":"2024-09-21T07:10:54.847711+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:10:54.847711+0000","last_undegraded":"2024-09-21T07:10:54.847711+0000","last_fullsized":"2024-09-21T07:10:54.847711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.833727+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:11:54.833727+0000","last_peered":"2024-09-21T07:11:54.833727+0000","last_clean":"2024-09-21T07:11:54.833727+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:11:54.833727+0000","last_undegraded":"2024-09-21T07:11:54.833727+0000","last_fullsized":"2024-09-21T07:11:54.833727+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850541+0000","last_change":"2024-09-21T07:10:05.426469+0000","last_active":"2024-09-21T07:10:54.850541+0000","last_peered":"2024-09-21T07:10:54.850541+0000","last_clean":"2024-09-21T07:10:54.850541+0000","last_became_active":"2024-09-21T07:10:05.426039+0000","last_became_peered":"2024-09-21T07:10:05.426039+0000","last_unstale":"2024-09-21T07:10:54.850541+0000","last_undegraded":"2024-09-21T07:10:54.850541+0000","last_fullsized":"2024-09-21T07:10:54.850541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":114,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.833868+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:11:54.833868+0000","last_peered":"2024-09-21T07:11:54.833868+0000","last_clean":"2024-09-21T07:11:54.833868+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:11:54.833868+0000","last_undegraded":"2024-09-21T07:11:54.833868+0000","last_fullsized":"2024-09-21T07:11:54.833868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857504+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:10:54.857504+0000","last_peered":"2024-09-21T07:10:54.857504+0000","last_clean":"2024-09-21T07:10:54.857504+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:10:54.857504+0000","last_undegraded":"2024-09-21T07:10:54.857504+0000","last_fullsized":"2024-09-21T07:10:54.857504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851084+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:10:54.851084+0000","last_peered":"2024-09-21T07:10:54.851084+0000","last_clean":"2024-09-21T07:10:54.851084+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:10:54.851084+0000","last_undegraded":"2024-09-21T07:10:54.851084+0000","last_fullsized":"2024-09-21T07:10:54.851084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851654+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:10:54.851654+0000","last_peered":"2024-09-21T07:10:54.851654+0000","last_clean":"2024-09-21T07:10:54.851654+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:10:54.851654+0000","last_undegraded":"2024-09-21T07:10:54.851654+0000","last_fullsized":"2024-09-21T07:10:54.851654+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.044649+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:10:55.044649+0000","last_peered":"2024-09-21T07:10:55.044649+0000","last_clean":"2024-09-21T07:10:55.044649+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:10:55.044649+0000","last_undegraded":"2024-09-21T07:10:55.044649+0000","last_fullsized":"2024-09-21T07:10:55.044649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.850408+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:10:54.850408+0000","last_peered":"2024-09-21T07:10:54.850408+0000","last_clean":"2024-09-21T07:10:54.850408+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:10:54.850408+0000","last_undegraded":"2024-09-21T07:10:54.850408+0000","last_fullsized":"2024-09-21T07:10:54.850408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847360+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:10:54.847360+0000","last_peered":"2024-09-21T07:10:54.847360+0000","last_clean":"2024-09-21T07:10:54.847360+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:10:54.847360+0000","last_undegraded":"2024-09-21T07:10:54.847360+0000","last_fullsized":"2024-09-21T07:10:54.847360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043281+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:10:55.043281+0000","last_peered":"2024-09-21T07:10:55.043281+0000","last_clean":"2024-09-21T07:10:55.043281+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:10:55.043281+0000","last_undegraded":"2024-09-21T07:10:55.043281+0000","last_fullsized":"2024-09-21T07:10:55.043281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852709+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:10:54.852709+0000","last_peered":"2024-09-21T07:10:54.852709+0000","last_clean":"2024-09-21T07:10:54.852709+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:10:54.852709+0000","last_undegraded":"2024-09-21T07:10:54.852709+0000","last_fullsized":"2024-09-21T07:10:54.852709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851607+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:10:54.851607+0000","last_peered":"2024-09-21T07:10:54.851607+0000","last_clean":"2024-09-21T07:10:54.851607+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:10:54.851607+0000","last_undegraded":"2024-09-21T07:10:54.851607+0000","last_fullsized":"2024-09-21T07:10:54.851607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857988+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:10:54.857988+0000","last_peered":"2024-09-21T07:10:54.857988+0000","last_clean":"2024-09-21T07:10:54.857988+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:10:54.857988+0000","last_undegraded":"2024-09-21T07:10:54.857988+0000","last_fullsized":"2024-09-21T07:10:54.857988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.880673+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:11:54.880673+0000","last_peered":"2024-09-21T07:11:54.880673+0000","last_clean":"2024-09-21T07:11:54.880673+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:11:54.880673+0000","last_undegraded":"2024-09-21T07:11:54.880673+0000","last_fullsized":"2024-09-21T07:11:54.880673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855648+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:10:54.855648+0000","last_peered":"2024-09-21T07:10:54.855648+0000","last_clean":"2024-09-21T07:10:54.855648+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:10:54.855648+0000","last_undegraded":"2024-09-21T07:10:54.855648+0000","last_fullsized":"2024-09-21T07:10:54.855648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847712+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:10:54.847712+0000","last_peered":"2024-09-21T07:10:54.847712+0000","last_clean":"2024-09-21T07:10:54.847712+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:10:54.847712+0000","last_undegraded":"2024-09-21T07:10:54.847712+0000","last_fullsized":"2024-09-21T07:10:54.847712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043249+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:10:55.043249+0000","last_peered":"2024-09-21T07:10:55.043249+0000","last_clean":"2024-09-21T07:10:55.043249+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:10:55.043249+0000","last_undegraded":"2024-09-21T07:10:55.043249+0000","last_fullsized":"2024-09-21T07:10:55.043249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.852135+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:10:54.852135+0000","last_peered":"2024-09-21T07:10:54.852135+0000","last_clean":"2024-09-21T07:10:54.852135+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:10:54.852135+0000","last_undegraded":"2024-09-21T07:10:54.852135+0000","last_fullsized":"2024-09-21T07:10:54.852135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.851861+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:10:54.851861+0000","last_peered":"2024-09-21T07:10:54.851861+0000","last_clean":"2024-09-21T07:10:54.851861+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:10:54.851861+0000","last_undegraded":"2024-09-21T07:10:54.851861+0000","last_fullsized":"2024-09-21T07:10:54.851861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.843362+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:10:54.843362+0000","last_peered":"2024-09-21T07:10:54.843362+0000","last_clean":"2024-09-21T07:10:54.843362+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:10:54.843362+0000","last_undegraded":"2024-09-21T07:10:54.843362+0000","last_fullsized":"2024-09-21T07:10:54.843362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853323+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:10:54.853323+0000","last_peered":"2024-09-21T07:10:54.853323+0000","last_clean":"2024-09-21T07:10:54.853323+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:10:54.853323+0000","last_undegraded":"2024-09-21T07:10:54.853323+0000","last_fullsized":"2024-09-21T07:10:54.853323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848242+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:10:54.848242+0000","last_peered":"2024-09-21T07:10:54.848242+0000","last_clean":"2024-09-21T07:10:54.848242+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:10:54.848242+0000","last_undegraded":"2024-09-21T07:10:54.848242+0000","last_fullsized":"2024-09-21T07:10:54.848242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.861506+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:10:54.861506+0000","last_peered":"2024-09-21T07:10:54.861506+0000","last_clean":"2024-09-21T07:10:54.861506+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:10:54.861506+0000","last_undegraded":"2024-09-21T07:10:54.861506+0000","last_fullsized":"2024-09-21T07:10:54.861506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:55.043140+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:10:55.043140+0000","last_peered":"2024-09-21T07:10:55.043140+0000","last_clean":"2024-09-21T07:10:55.043140+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:10:55.043140+0000","last_undegraded":"2024-09-21T07:10:55.043140+0000","last_fullsized":"2024-09-21T07:10:55.043140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.860207+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:10:54.860207+0000","last_peered":"2024-09-21T07:10:54.860207+0000","last_clean":"2024-09-21T07:10:54.860207+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:10:54.860207+0000","last_undegraded":"2024-09-21T07:10:54.860207+0000","last_fullsized":"2024-09-21T07:10:54.860207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.881268+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:11:54.881268+0000","last_peered":"2024-09-21T07:11:54.881268+0000","last_clean":"2024-09-21T07:11:54.881268+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:11:54.881268+0000","last_undegraded":"2024-09-21T07:11:54.881268+0000","last_fullsized":"2024-09-21T07:11:54.881268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847513+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:10:54.847513+0000","last_peered":"2024-09-21T07:10:54.847513+0000","last_clean":"2024-09-21T07:10:54.847513+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:10:54.847513+0000","last_undegraded":"2024-09-21T07:10:54.847513+0000","last_fullsized":"2024-09-21T07:10:54.847513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.848613+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:10:54.848613+0000","last_peered":"2024-09-21T07:10:54.848613+0000","last_clean":"2024-09-21T07:10:54.848613+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:10:54.848613+0000","last_undegraded":"2024-09-21T07:10:54.848613+0000","last_fullsized":"2024-09-21T07:10:54.848613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.856045+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:10:54.856045+0000","last_peered":"2024-09-21T07:10:54.856045+0000","last_clean":"2024-09-21T07:10:54.856045+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:10:54.856045+0000","last_undegraded":"2024-09-21T07:10:54.856045+0000","last_fullsized":"2024-09-21T07:10:54.856045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-21T07:11:54.880816+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:11:54.880816+0000","last_peered":"2024-09-21T07:11:54.880816+0000","last_clean":"2024-09-21T07:11:54.880816+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:11:54.880816+0000","last_undegraded":"2024-09-21T07:11:54.880816+0000","last_fullsized":"2024-09-21T07:11:54.880816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847825+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:10:54.847825+0000","last_peered":"2024-09-21T07:10:54.847825+0000","last_clean":"2024-09-21T07:10:54.847825+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:10:54.847825+0000","last_undegraded":"2024-09-21T07:10:54.847825+0000","last_fullsized":"2024-09-21T07:10:54.847825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.857433+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:10:54.857433+0000","last_peered":"2024-09-21T07:10:54.857433+0000","last_clean":"2024-09-21T07:10:54.857433+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:10:54.857433+0000","last_undegraded":"2024-09-21T07:10:54.857433+0000","last_fullsized":"2024-09-21T07:10:54.857433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.855172+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:10:54.855172+0000","last_peered":"2024-09-21T07:10:54.855172+0000","last_clean":"2024-09-21T07:10:54.855172+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:10:54.855172+0000","last_undegraded":"2024-09-21T07:10:54.855172+0000","last_fullsized":"2024-09-21T07:10:54.855172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.846704+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:10:54.846704+0000","last_peered":"2024-09-21T07:10:54.846704+0000","last_clean":"2024-09-21T07:10:54.846704+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:10:54.846704+0000","last_undegraded":"2024-09-21T07:10:54.846704+0000","last_fullsized":"2024-09-21T07:10:54.846704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.847084+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:10:54.847084+0000","last_peered":"2024-09-21T07:10:54.847084+0000","last_clean":"2024-09-21T07:10:54.847084+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:10:54.847084+0000","last_undegraded":"2024-09-21T07:10:54.847084+0000","last_fullsized":"2024-09-21T07:10:54.847084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T07:10:54.853198+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:10:54.853198+0000","last_peered":"2024-09-21T07:10:54.853198+0000","last_clean":"2024-09-21T07:10:54.853198+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:10:54.853198+0000","last_undegraded":"2024-09-21T07:10:54.853198+0000","last_fullsized":"2024-09-21T07:10:54.853198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70213,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":406,"num_read_kb":274,"num_write":166,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":210639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":496595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972394,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1662976,"data_stored":1150444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135920,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":500905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":496707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495674,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1593344,"data_stored":1102306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659199,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":491985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855431,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1601536,"data_stored":1097503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018955,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":497026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":492179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":559457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":558107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705761,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":493373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:11:57.088 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T07:11:57.089 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T07:11:57.089 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T07:11:57.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T07:11:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:56 smithi153 ceph-mon[34747]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:11:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:56 smithi153 ceph-mon[34747]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T07:11:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:56 smithi119 ceph-mon[34519]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:11:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:56 smithi119 ceph-mon[34519]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T07:11:57.544 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:11:58.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:57 smithi027 ceph-mon[30779]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:58.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:58.013 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:57 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:58.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:57 smithi153 ceph-mon[34747]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:58.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:58.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:57 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:58.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 -- 172.21.15.27:0/2222027618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81092f0 msgr2=0x7f0fb8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:58.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 --2- 172.21.15.27:0/2222027618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81092f0 0x7f0fb8109700 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009900 tx=0x7f0fa402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 -- 172.21.15.27:0/2222027618 shutdown_connections 2024-09-21T07:11:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 --2- 172.21.15.27:0/2222027618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81092f0 0x7f0fb8109700 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 --2- 172.21.15.27:0/2222027618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb8109c40 0x7f0fb8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 --2- 172.21.15.27:0/2222027618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb81039b0 0x7f0fb8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 -- 172.21.15.27:0/2222027618 >> 172.21.15.27:0/2222027618 conn(0x7f0fb80ff440 msgr2=0x7f0fb8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 -- 172.21.15.27:0/2222027618 shutdown_connections 2024-09-21T07:11:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.197+0000 7f0fbde00640 1 -- 172.21.15.27:0/2222027618 wait complete. 2024-09-21T07:11:58.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.198+0000 7f0fbde00640 1 Processor -- start 2024-09-21T07:11:58.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.198+0000 7f0fbde00640 1 -- start start 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.198+0000 7f0fbde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81039b0 0x7f0fb819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb81092f0 0x7f0fb819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb8109c40 0x7f0fb81a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbde00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8113950 con 0x7f0fb81092f0 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbde00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81137d0 con 0x7f0fb8109c40 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbde00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8113ad0 con 0x7f0fb81039b0 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fb7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb81092f0 0x7f0fb819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fb7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb81092f0 0x7f0fb819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46560/0 (socket says 172.21.15.27:46560) 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fb7e00640 1 -- 172.21.15.27:0/3536213446 learned_addr learned my addr 172.21.15.27:0/3536213446 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbd400640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb8109c40 0x7f0fb81a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbca00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81039b0 0x7f0fb819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbca00640 1 -- 172.21.15.27:0/3536213446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb8109c40 msgr2=0x7f0fb81a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbca00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb8109c40 0x7f0fb81a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.199+0000 7f0fbca00640 1 -- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb81092f0 msgr2=0x7f0fb819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fbca00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb81092f0 0x7f0fb819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fbca00640 1 -- 172.21.15.27:0/3536213446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb81a2490 con 0x7f0fb81039b0 2024-09-21T07:11:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fb7e00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb81092f0 0x7f0fb819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:11:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fbca00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81039b0 0x7f0fb819c520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e9f0 tx=0x7f0fac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fb5600640 1 -- 172.21.15.27:0/3536213446 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00ccc0 con 0x7f0fb81039b0 2024-09-21T07:11:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fb5600640 1 -- 172.21.15.27:0/3536213446 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fac005020 con 0x7f0fb81039b0 2024-09-21T07:11:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb81a2780 con 0x7f0fb81039b0 2024-09-21T07:11:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fb5600640 1 -- 172.21.15.27:0/3536213446 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac004a00 con 0x7f0fb81039b0 2024-09-21T07:11:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.200+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb81a8ee0 con 0x7f0fb81039b0 2024-09-21T07:11:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.201+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb8103dc0 con 0x7f0fb81039b0 2024-09-21T07:11:58.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.204+0000 7f0fb5600640 1 -- 172.21.15.27:0/3536213446 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0fac004ba0 con 0x7f0fb81039b0 2024-09-21T07:11:58.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.206+0000 7f0fb5600640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fa00b6670 0x7f0fa00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:11:58.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.206+0000 7f0fb5600640 1 -- 172.21.15.27:0/3536213446 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0fac0e7950 con 0x7f0fb81039b0 2024-09-21T07:11:58.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.206+0000 7f0fb7e00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fa00b6670 0x7f0fa00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:11:58.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.207+0000 7f0fb7e00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fa00b6670 0x7f0fa00b8b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4006010 tx=0x7f0fa4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:11:58.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.207+0000 7f0fb5600640 1 -- 172.21.15.27:0/3536213446 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0fac0adc40 con 0x7f0fb81039b0 2024-09-21T07:11:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:57 smithi119 ceph-mon[34519]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:11:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:57 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:58.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.447+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0fb819d650 con 0x7f0fb81039b0 2024-09-21T07:11:58.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.447+0000 7f0fb5600640 1 -- 172.21.15.27:0/3536213446 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0fb819d650 con 0x7f0fb81039b0 2024-09-21T07:11:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:11:58.449 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T07:11:58.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fa00b6670 msgr2=0x7f0fa00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:58.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fa00b6670 0x7f0fa00b8b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4006010 tx=0x7f0fa4005a20 comp rx=0 tx=0).stop 2024-09-21T07:11:58.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81039b0 msgr2=0x7f0fb819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:11:58.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81039b0 0x7f0fb819c520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e9f0 tx=0x7f0fac00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 shutdown_connections 2024-09-21T07:11:58.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fa00b6670 0x7f0fa00b8b80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb8109c40 0x7f0fb81a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fb81092f0 0x7f0fb819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 --2- 172.21.15.27:0/3536213446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fb81039b0 0x7f0fb819c520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:11:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 >> 172.21.15.27:0/3536213446 conn(0x7f0fb80ff440 msgr2=0x7f0fb8106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:11:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 shutdown_connections 2024-09-21T07:11:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:11:58.450+0000 7f0fbde00640 1 -- 172.21.15.27:0/3536213446 wait complete. 2024-09-21T07:11:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:58 smithi027 ceph-mon[30779]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T07:11:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:11:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:11:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:11:58 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3536213446' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T07:11:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:58 smithi153 ceph-mon[34747]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T07:11:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:11:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:11:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:59.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:11:58 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3536213446' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T07:11:59.185 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T07:11:59.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:11:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:58 smithi119 ceph-mon[34519]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T07:11:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:11:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:11:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:11:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:11:58 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3536213446' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T07:11:59.570 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 -- 172.21.15.27:0/2548012940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810c040 msgr2=0x7fcb78112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/2548012940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810c040 0x7fcb78112870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009ca0 tx=0x7fcb7002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 -- 172.21.15.27:0/2548012940 shutdown_connections 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/2548012940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb7810b6f0 0x7fcb7810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/2548012940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810c040 0x7fcb78112870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/2548012940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb78105db0 0x7fcb781061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 -- 172.21.15.27:0/2548012940 >> 172.21.15.27:0/2548012940 conn(0x7fcb781017e0 msgr2=0x7fcb78103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 -- 172.21.15.27:0/2548012940 shutdown_connections 2024-09-21T07:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.179+0000 7fcb7fc00640 1 -- 172.21.15.27:0/2548012940 wait complete. 2024-09-21T07:12:00.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.180+0000 7fcb7fc00640 1 Processor -- start 2024-09-21T07:12:00.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7fc00640 1 -- start start 2024-09-21T07:12:00.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb78105db0 0x7fcb7819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:00.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7fc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810b6f0 0x7fcb7819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:00.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7fc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb7810c040 0x7fcb781a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7fc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78115d50 con 0x7fcb7810b6f0 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7fc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78115bd0 con 0x7fcb78105db0 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7fc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78115ed0 con 0x7fcb7810c040 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810b6f0 0x7fcb7819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810b6f0 0x7fcb7819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46576/0 (socket says 172.21.15.27:46576) 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb77e00640 1 -- 172.21.15.27:0/1267765837 learned_addr learned my addr 172.21.15.27:0/1267765837 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7dc00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb7810c040 0x7fcb781a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.181+0000 7fcb7d200640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb78105db0 0x7fcb7819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.182+0000 7fcb77e00640 1 -- 172.21.15.27:0/1267765837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb7810c040 msgr2=0x7fcb781a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.182+0000 7fcb77e00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb7810c040 0x7fcb781a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.182+0000 7fcb77e00640 1 -- 172.21.15.27:0/1267765837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb78105db0 msgr2=0x7fcb7819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.182+0000 7fcb77e00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb78105db0 0x7fcb7819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.182+0000 7fcb77e00640 1 -- 172.21.15.27:0/1267765837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb781a2610 con 0x7fcb7810b6f0 2024-09-21T07:12:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.182+0000 7fcb77e00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810b6f0 0x7fcb7819cc40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e9c0 tx=0x7fcb6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.182+0000 7fcb75600640 1 -- 172.21.15.27:0/1267765837 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400ccc0 con 0x7fcb7810b6f0 2024-09-21T07:12:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.183+0000 7fcb75600640 1 -- 172.21.15.27:0/1267765837 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb640054d0 con 0x7fcb7810b6f0 2024-09-21T07:12:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.183+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb781a2900 con 0x7fcb7810b6f0 2024-09-21T07:12:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.183+0000 7fcb75600640 1 -- 172.21.15.27:0/1267765837 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64010430 con 0x7fcb7810b6f0 2024-09-21T07:12:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.183+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb781a9060 con 0x7fcb7810b6f0 2024-09-21T07:12:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.184+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb781061c0 con 0x7fcb7810b6f0 2024-09-21T07:12:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.187+0000 7fcb75600640 1 -- 172.21.15.27:0/1267765837 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fcb64005020 con 0x7fcb7810b6f0 2024-09-21T07:12:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.188+0000 7fcb75600640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcb600b65f0 0x7fcb600b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.188+0000 7fcb75600640 1 -- 172.21.15.27:0/1267765837 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fcb640e7f20 con 0x7fcb7810b6f0 2024-09-21T07:12:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.189+0000 7fcb7d200640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcb600b65f0 0x7fcb600b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.189+0000 7fcb7d200640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcb600b65f0 0x7fcb600b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009bd0 tx=0x7fcb6c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:00.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.190+0000 7fcb75600640 1 -- 172.21.15.27:0/1267765837 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb64056710 con 0x7fcb7810b6f0 2024-09-21T07:12:00.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.430+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb7819d7d0 con 0x7fcb7810b6f0 2024-09-21T07:12:00.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.431+0000 7fcb75600640 1 -- 172.21.15.27:0/1267765837 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7fcb7819d7d0 con 0x7fcb7810b6f0 2024-09-21T07:12:00.433 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:00.433 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T07:10:53:642840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:52.789806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24773,24748,24740]},"id":1}]} 2024-09-21T07:12:00.433 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T07:12:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.433+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcb600b65f0 msgr2=0x7fcb600b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.433+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcb600b65f0 0x7fcb600b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009bd0 tx=0x7fcb6c005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.433+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810b6f0 msgr2=0x7fcb7819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.433+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810b6f0 0x7fcb7819cc40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e9c0 tx=0x7fcb6400ee80 comp rx=0 tx=0).stop 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 shutdown_connections 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb7810c040 0x7fcb781a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcb600b65f0 0x7fcb600b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb7810b6f0 0x7fcb7819cc40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 --2- 172.21.15.27:0/1267765837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb78105db0 0x7fcb7819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 >> 172.21.15.27:0/1267765837 conn(0x7fcb781017e0 msgr2=0x7fcb78108c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 shutdown_connections 2024-09-21T07:12:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:00.434+0000 7fcb7fc00640 1 -- 172.21.15.27:0/1267765837 wait complete. 2024-09-21T07:12:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:00 smithi027 ceph-mon[30779]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-21T07:12:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:00 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1267765837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:01.143 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:12:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:00 smithi153 ceph-mon[34747]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-21T07:12:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:00 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1267765837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:00 smithi119 ceph-mon[34519]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-21T07:12:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:00 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1267765837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:01.528 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:02.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 -- 172.21.15.27:0/1445201484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 msgr2=0x7f73c0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:02.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 --2- 172.21.15.27:0/1445201484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 0x7f73c0110620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009ca0 tx=0x7f73b802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 -- 172.21.15.27:0/1445201484 shutdown_connections 2024-09-21T07:12:02.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 --2- 172.21.15.27:0/1445201484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 0x7f73c0110620 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 --2- 172.21.15.27:0/1445201484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73c01094a0 0x7f73c01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 --2- 172.21.15.27:0/1445201484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 0x7f73c0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 -- 172.21.15.27:0/1445201484 >> 172.21.15.27:0/1445201484 conn(0x7f73c007b850 msgr2=0x7f73c007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:02.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 -- 172.21.15.27:0/1445201484 shutdown_connections 2024-09-21T07:12:02.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.104+0000 7f73c7e00640 1 -- 172.21.15.27:0/1445201484 wait complete. 2024-09-21T07:12:02.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.105+0000 7f73c7e00640 1 Processor -- start 2024-09-21T07:12:02.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.105+0000 7f73c7e00640 1 -- start start 2024-09-21T07:12:02.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 0x7f73c019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:02.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73c01094a0 0x7f73c019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:02.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 0x7f73c019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:02.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c7e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0113ab0 con 0x7f73c0109df0 2024-09-21T07:12:02.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0113930 con 0x7f73c01094a0 2024-09-21T07:12:02.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c7e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0113c30 con 0x7f73c0103b60 2024-09-21T07:12:02.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 0x7f73c019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:02.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 0x7f73c019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:02.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 0x7f73c019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41988/0 (socket says 172.21.15.27:41988) 2024-09-21T07:12:02.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73c01094a0 0x7f73c019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:02.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.106+0000 7f73c5400640 1 -- 172.21.15.27:0/952085844 learned_addr learned my addr 172.21.15.27:0/952085844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:02.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c5400640 1 -- 172.21.15.27:0/952085844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73c01094a0 msgr2=0x7f73c019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:02.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c5400640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73c01094a0 0x7f73c019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c5400640 1 -- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 msgr2=0x7f73c019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c5400640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 0x7f73c019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c5400640 1 -- 172.21.15.27:0/952085844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c01a03f0 con 0x7f73c0103b60 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c5400640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 0x7f73c019a480 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e9f0 tx=0x7f73b400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73be000640 1 -- 172.21.15.27:0/952085844 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b400cd80 con 0x7f73c0103b60 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73be000640 1 -- 172.21.15.27:0/952085844 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73b40054d0 con 0x7f73c0103b60 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c01a06e0 con 0x7f73c0103b60 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73be000640 1 -- 172.21.15.27:0/952085844 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b4010690 con 0x7f73c0103b60 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.107+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c01a6e40 con 0x7f73c0103b60 2024-09-21T07:12:02.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.108+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c0103f70 con 0x7f73c0103b60 2024-09-21T07:12:02.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.111+0000 7f73be000640 1 -- 172.21.15.27:0/952085844 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f73b4004e30 con 0x7f73c0103b60 2024-09-21T07:12:02.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.113+0000 7f73be000640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73a80b65f0 0x7f73a80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:02.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.113+0000 7f73be000640 1 -- 172.21.15.27:0/952085844 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f73b40e7f80 con 0x7f73c0103b60 2024-09-21T07:12:02.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.113+0000 7f73c4a00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73a80b65f0 0x7f73a80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:02.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.114+0000 7f73c4a00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73a80b65f0 0x7f73a80b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73c0106f40 tx=0x7f73ac005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.114+0000 7f73be000640 1 -- 172.21.15.27:0/952085844 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73b40ae1e0 con 0x7f73c0103b60 2024-09-21T07:12:02.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.354+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73c019b5b0 con 0x7f73c0103b60 2024-09-21T07:12:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.355+0000 7f73be000640 1 -- 172.21.15.27:0/952085844 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7f73c019b5b0 con 0x7f73c0103b60 2024-09-21T07:12:02.358 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:02.358 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T07:10:53:642840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:52.789806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24773,24748,24740]},"id":1}]} 2024-09-21T07:12:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T07:12:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73a80b65f0 msgr2=0x7f73a80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73a80b65f0 0x7f73a80b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73c0106f40 tx=0x7f73ac005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 msgr2=0x7f73c019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 0x7f73c019a480 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e9f0 tx=0x7f73b400edf0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 shutdown_connections 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73a80b65f0 0x7f73a80b8b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c0109df0 0x7f73c019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73c01094a0 0x7f73c019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 --2- 172.21.15.27:0/952085844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73c0103b60 0x7f73c019a480 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 >> 172.21.15.27:0/952085844 conn(0x7f73c007b850 msgr2=0x7f73c0105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 shutdown_connections 2024-09-21T07:12:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:02.358+0000 7f73c7e00640 1 -- 172.21.15.27:0/952085844 wait complete. 2024-09-21T07:12:03.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:02 smithi027 ceph-mon[30779]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T07:12:03.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:02 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/952085844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:03.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:12:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:02 smithi153 ceph-mon[34747]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T07:12:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:02 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/952085844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:02 smithi119 ceph-mon[34519]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T07:12:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:02 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/952085844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:03.444 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:04.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.120+0000 7efd73a00640 1 -- 172.21.15.27:0/2176847400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 msgr2=0x7efd6c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:04.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.120+0000 7efd73a00640 1 --2- 172.21.15.27:0/2176847400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c077d50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efd60009900 tx=0x7efd6002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.121+0000 7efd73a00640 1 -- 172.21.15.27:0/2176847400 shutdown_connections 2024-09-21T07:12:04.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.121+0000 7efd73a00640 1 --2- 172.21.15.27:0/2176847400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd6c078290 0x7efd6c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.121+0000 7efd73a00640 1 --2- 172.21.15.27:0/2176847400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c114740 0x7efd6c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.121+0000 7efd73a00640 1 --2- 172.21.15.27:0/2176847400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c077d50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.121+0000 7efd73a00640 1 -- 172.21.15.27:0/2176847400 >> 172.21.15.27:0/2176847400 conn(0x7efd6c101810 msgr2=0x7efd6c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:04.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.121+0000 7efd73a00640 1 -- 172.21.15.27:0/2176847400 shutdown_connections 2024-09-21T07:12:04.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.121+0000 7efd73a00640 1 -- 172.21.15.27:0/2176847400 wait complete. 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.122+0000 7efd73a00640 1 Processor -- start 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.122+0000 7efd73a00640 1 -- start start 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd73a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd73a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd6c078290 0x7efd6c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd73a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c114740 0x7efd6c1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd73a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c11a120 con 0x7efd6c077940 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd73a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c119fa0 con 0x7efd6c114740 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd73a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c11a2a0 con 0x7efd6c078290 2024-09-21T07:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd71000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:04.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd71000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46600/0 (socket says 172.21.15.27:46600) 2024-09-21T07:12:04.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd71000640 1 -- 172.21.15.27:0/3152389750 learned_addr learned my addr 172.21.15.27:0/3152389750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:04.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd6be00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd6c078290 0x7efd6c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:04.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd71a00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c114740 0x7efd6c1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd71000640 1 -- 172.21.15.27:0/3152389750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd6c078290 msgr2=0x7efd6c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.123+0000 7efd71000640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd6c078290 0x7efd6c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd71000640 1 -- 172.21.15.27:0/3152389750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c114740 msgr2=0x7efd6c1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd71000640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c114740 0x7efd6c1a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd71000640 1 -- 172.21.15.27:0/3152389750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c1a6aa0 con 0x7efd6c077940 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd71000640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c1a0b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efd60005ab0 tx=0x7efd60003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd69600640 1 -- 172.21.15.27:0/3152389750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6003f070 con 0x7efd6c077940 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd69600640 1 -- 172.21.15.27:0/3152389750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd60031d60 con 0x7efd6c077940 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd69600640 1 -- 172.21.15.27:0/3152389750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60035030 con 0x7efd6c077940 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c1a6d30 con 0x7efd6c077940 2024-09-21T07:12:04.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.124+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1ad4f0 con 0x7efd6c077940 2024-09-21T07:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.125+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c077d50 con 0x7efd6c077940 2024-09-21T07:12:04.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.129+0000 7efd69600640 1 -- 172.21.15.27:0/3152389750 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7efd60042070 con 0x7efd6c077940 2024-09-21T07:12:04.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.130+0000 7efd69600640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efd540b6670 0x7efd540b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:04.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.130+0000 7efd69600640 1 -- 172.21.15.27:0/3152389750 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7efd60108370 con 0x7efd6c077940 2024-09-21T07:12:04.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.131+0000 7efd6be00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efd540b6670 0x7efd540b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:04.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.131+0000 7efd6be00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efd540b6670 0x7efd540b8b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd6c07a250 tx=0x7efd58005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:04.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.131+0000 7efd69600640 1 -- 172.21.15.27:0/3152389750 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efd600d29b0 con 0x7efd6c077940 2024-09-21T07:12:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.331+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd6c02e640 con 0x7efd6c077940 2024-09-21T07:12:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.333+0000 7efd69600640 1 -- 172.21.15.27:0/3152389750 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25234 (secure 0 0 0) 0x7efd600d78f0 con 0x7efd6c077940 2024-09-21T07:12:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:04.336 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:11:55.826198+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10},{"from":5,"to":1}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:12:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efd540b6670 msgr2=0x7efd540b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efd540b6670 0x7efd540b8b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd6c07a250 tx=0x7efd58005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 msgr2=0x7efd6c1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c1a0b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efd60005ab0 tx=0x7efd60003c80 comp rx=0 tx=0).stop 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 shutdown_connections 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd6c078290 0x7efd6c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efd540b6670 0x7efd540b8b80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c114740 0x7efd6c1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 --2- 172.21.15.27:0/3152389750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd6c077940 0x7efd6c1a0b30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.337+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 >> 172.21.15.27:0/3152389750 conn(0x7efd6c101810 msgr2=0x7efd6c1167b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.338+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 shutdown_connections 2024-09-21T07:12:04.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:04.338+0000 7efd73a00640 1 -- 172.21.15.27:0/3152389750 wait complete. 2024-09-21T07:12:05.029 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:04 smithi027 ceph-mon[30779]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T07:12:05.030 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:04 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3152389750' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:05.071 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:12:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:04 smithi153 ceph-mon[34747]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T07:12:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:04 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3152389750' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:04 smithi119 ceph-mon[34519]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T07:12:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:04 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3152389750' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:05.459 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 -- 172.21.15.27:0/2206723993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c077940 msgr2=0x7f5e8c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 --2- 172.21.15.27:0/2206723993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c077940 0x7f5e8c077d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009900 tx=0x7f5e8002a410 comp rx=0 tx=0).stop 2024-09-21T07:12:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 -- 172.21.15.27:0/2206723993 shutdown_connections 2024-09-21T07:12:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 --2- 172.21.15.27:0/2206723993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c078290 0x7f5e8c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 --2- 172.21.15.27:0/2206723993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e8c114740 0x7f5e8c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 --2- 172.21.15.27:0/2206723993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c077940 0x7f5e8c077d50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 -- 172.21.15.27:0/2206723993 >> 172.21.15.27:0/2206723993 conn(0x7f5e8c101810 msgr2=0x7f5e8c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 -- 172.21.15.27:0/2206723993 shutdown_connections 2024-09-21T07:12:06.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.075+0000 7f5e93400640 1 -- 172.21.15.27:0/2206723993 wait complete. 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.076+0000 7f5e93400640 1 Processor -- start 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.076+0000 7f5e93400640 1 -- start start 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.076+0000 7f5e93400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c077940 0x7f5e8c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.076+0000 7f5e93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c078290 0x7f5e8c1a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e93400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e8c114740 0x7f5e8c1a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e93400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c11a0d0 con 0x7f5e8c114740 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e93400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c119f50 con 0x7f5e8c078290 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e93400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c11a250 con 0x7f5e8c077940 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c077940 0x7f5e8c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:06.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c077940 0x7f5e8c1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42022/0 (socket says 172.21.15.27:42022) 2024-09-21T07:12:06.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 -- 172.21.15.27:0/3134164951 learned_addr learned my addr 172.21.15.27:0/3134164951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:06.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e8be00640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c078290 0x7f5e8c1a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:06.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e91400640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e8c114740 0x7f5e8c1a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 -- 172.21.15.27:0/3134164951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c078290 msgr2=0x7f5e8c1a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c078290 0x7f5e8c1a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 -- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e8c114740 msgr2=0x7f5e8c1a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e8c114740 0x7f5e8c1a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.077+0000 7f5e90a00640 1 -- 172.21.15.27:0/3134164951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c1a6a90 con 0x7f5e8c077940 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.078+0000 7f5e90a00640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c077940 0x7f5e8c1a0b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005940 tx=0x7f5e800040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.078+0000 7f5e89600640 1 -- 172.21.15.27:0/3134164951 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8003f030 con 0x7f5e8c077940 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.078+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c1a6d20 con 0x7f5e8c077940 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.078+0000 7f5e89600640 1 -- 172.21.15.27:0/3134164951 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e800044c0 con 0x7f5e8c077940 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.078+0000 7f5e89600640 1 -- 172.21.15.27:0/3134164951 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80035070 con 0x7f5e8c077940 2024-09-21T07:12:06.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.078+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1ad4e0 con 0x7f5e8c077940 2024-09-21T07:12:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.079+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c077d50 con 0x7f5e8c077940 2024-09-21T07:12:06.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.082+0000 7f5e89600640 1 -- 172.21.15.27:0/3134164951 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5e80042030 con 0x7f5e8c077940 2024-09-21T07:12:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.084+0000 7f5e89600640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e740b65f0 0x7f5e740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.084+0000 7f5e89600640 1 -- 172.21.15.27:0/3134164951 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5e80107f90 con 0x7f5e8c077940 2024-09-21T07:12:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.084+0000 7f5e8be00640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e740b65f0 0x7f5e740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.085+0000 7f5e8be00640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e740b65f0 0x7f5e740b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c07a250 tx=0x7f5e78005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.085+0000 7f5e89600640 1 -- 172.21.15.27:0/3134164951 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5e800d25d0 con 0x7f5e8c077940 2024-09-21T07:12:06.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.322+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e8c10f8d0 con 0x7f5e8c077940 2024-09-21T07:12:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.323+0000 7f5e89600640 1 -- 172.21.15.27:0/3134164951 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7f5e800d7510 con 0x7f5e8c077940 2024-09-21T07:12:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:06.325 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T07:10:53:642840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:52.789806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24773,24748,24740]},"id":1}]} 2024-09-21T07:12:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e740b65f0 msgr2=0x7f5e740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e740b65f0 0x7f5e740b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c07a250 tx=0x7f5e78005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c077940 msgr2=0x7f5e8c1a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c077940 0x7f5e8c1a0b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005940 tx=0x7f5e800040c0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 shutdown_connections 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e740b65f0 0x7f5e740b8b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e8c114740 0x7f5e8c1a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e8c078290 0x7f5e8c1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 --2- 172.21.15.27:0/3134164951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e8c077940 0x7f5e8c1a0b20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.326+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 >> 172.21.15.27:0/3134164951 conn(0x7f5e8c101810 msgr2=0x7f5e8c1167f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.327+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 shutdown_connections 2024-09-21T07:12:06.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:06.327+0000 7f5e93400640 1 -- 172.21.15.27:0/3134164951 wait complete. 2024-09-21T07:12:06.998 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:06 smithi027 ceph-mon[30779]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:12:06.998 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:06 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3134164951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:07.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T07:09:56.473755+0000', 'modified': '2024-09-21T07:10:52.789806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24773, 'mds_1': 24748, 'mds_2': 24740}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24713': {'gid': 24713, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/3894045960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 3894045960}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3894045960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24721': {'gid': 24721, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/1555665457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 1555665457}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 1555665457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24736': {'gid': 24736, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6839/2081315981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 2081315981}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 2081315981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24740': {'gid': 24740, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.119:6837/4207522386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 4207522386}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 4207522386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24748': {'gid': 24748, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.27:6839/609554839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 609554839}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 609554839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24773': {'gid': 24773, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.119:6839/1565967470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1565967470}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1565967470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24773, 'qdb_cluster': [24773, 24748, 24740]} 2024-09-21T07:12:07.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:12:07.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T07:12:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:06 smithi153 ceph-mon[34747]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:12:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:06 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3134164951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:06 smithi119 ceph-mon[34519]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:12:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:06 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3134164951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:07.515 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.144+0000 7fc247200640 1 -- 172.21.15.27:0/614046999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 msgr2=0x7fc248109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.144+0000 7fc247200640 1 --2- 172.21.15.27:0/614046999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 0x7fc248109700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc234009900 tx=0x7fc23402a410 comp rx=0 tx=0).stop 2024-09-21T07:12:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.145+0000 7fc247200640 1 -- 172.21.15.27:0/614046999 shutdown_connections 2024-09-21T07:12:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.145+0000 7fc247200640 1 --2- 172.21.15.27:0/614046999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc248109c40 0x7fc248110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.145+0000 7fc247200640 1 --2- 172.21.15.27:0/614046999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 0x7fc248109700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.145+0000 7fc247200640 1 --2- 172.21.15.27:0/614046999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2481039b0 0x7fc248103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.145+0000 7fc247200640 1 -- 172.21.15.27:0/614046999 >> 172.21.15.27:0/614046999 conn(0x7fc2480ff400 msgr2=0x7fc248101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.145+0000 7fc247200640 1 -- 172.21.15.27:0/614046999 shutdown_connections 2024-09-21T07:12:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.145+0000 7fc247200640 1 -- 172.21.15.27:0/614046999 wait complete. 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.146+0000 7fc247200640 1 Processor -- start 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.146+0000 7fc247200640 1 -- start start 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.146+0000 7fc247200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2481039b0 0x7fc24819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.146+0000 7fc247200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 0x7fc24819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc247200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc248109c40 0x7fc24819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc247200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc248113900 con 0x7fc2481092f0 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc247200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc248113780 con 0x7fc2481039b0 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc247200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc248113a80 con 0x7fc248109c40 2024-09-21T07:12:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 0x7fc24819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2481039b0 0x7fc24819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2481039b0 0x7fc24819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33022/0 (socket says 172.21.15.27:33022) 2024-09-21T07:12:08.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245e00640 1 -- 172.21.15.27:0/1598289693 learned_addr learned my addr 172.21.15.27:0/1598289693 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:08.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc246800640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc248109c40 0x7fc24819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245400640 1 -- 172.21.15.27:0/1598289693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc248109c40 msgr2=0x7fc24819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245400640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc248109c40 0x7fc24819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245400640 1 -- 172.21.15.27:0/1598289693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2481039b0 msgr2=0x7fc24819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245400640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2481039b0 0x7fc24819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.147+0000 7fc245400640 1 -- 172.21.15.27:0/1598289693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2481a0220 con 0x7fc2481092f0 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.148+0000 7fc245400640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 0x7fc24819a850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc23402a930 tx=0x7fc234031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.148+0000 7fc23ea00640 1 -- 172.21.15.27:0/1598289693 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23403f070 con 0x7fc2481092f0 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.148+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2481a04b0 con 0x7fc2481092f0 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.148+0000 7fc23ea00640 1 -- 172.21.15.27:0/1598289693 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc234004490 con 0x7fc2481092f0 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.148+0000 7fc23ea00640 1 -- 172.21.15.27:0/1598289693 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234035070 con 0x7fc2481092f0 2024-09-21T07:12:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.148+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2481a6c70 con 0x7fc2481092f0 2024-09-21T07:12:08.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.152+0000 7fc253c4b740 1 Processor -- start 2024-09-21T07:12:08.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.152+0000 7fc253c4b740 1 -- start start 2024-09-21T07:12:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf976f60 0x55ffdfd1d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd1dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd1e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc253c4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfc94d50 con 0x55ffdf976f60 2024-09-21T07:12:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc253c4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfc84180 con 0x55ffdf956280 2024-09-21T07:12:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc253c4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfc87f90 con 0x55ffdf94e0a0 2024-09-21T07:12:08.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc246800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf976f60 0x55ffdfd1d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc246800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf976f60 0x55ffdfd1d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50782/0 (socket says 172.21.15.27:50782) 2024-09-21T07:12:08.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc246800640 1 -- 172.21.15.27:0/3595165628 learned_addr learned my addr 172.21.15.27:0/3595165628 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:08.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.153+0000 7fc245e00640 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd1dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.154+0000 7fc23ea00640 1 -- 172.21.15.27:0/1598289693 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc23403b460 con 0x7fc2481092f0 2024-09-21T07:12:08.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.154+0000 7fc245e00640 1 -- 172.21.15.27:0/3595165628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 msgr2=0x55ffdfd1e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.154+0000 7fc245e00640 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd1e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.154+0000 7fc245e00640 1 -- 172.21.15.27:0/3595165628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf976f60 msgr2=0x55ffdfd1d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.154+0000 7fc245e00640 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf976f60 0x55ffdfd1d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.154+0000 7fc245e00640 1 -- 172.21.15.27:0/3595165628 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffdfc4ea70 con 0x55ffdf94e0a0 2024-09-21T07:12:08.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.154+0000 7fc246800640 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf976f60 0x55ffdfd1d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:08.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.155+0000 7fc245e00640 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd1dbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc238009c80 tx=0x7fc238011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c2ce4063dc1026 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:08.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.155+0000 7fc233400640 1 -- 172.21.15.27:0/3595165628 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238004250 con 0x55ffdf94e0a0 2024-09-21T07:12:08.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.155+0000 7fc233400640 1 -- 172.21.15.27:0/3595165628 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc238009e30 con 0x55ffdf94e0a0 2024-09-21T07:12:08.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.155+0000 7fc233400640 1 -- 172.21.15.27:0/3595165628 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238004ca0 con 0x55ffdf94e0a0 2024-09-21T07:12:08.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.155+0000 7fc23ea00640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc22c0b6670 0x7fc22c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc23ea00640 1 -- 172.21.15.27:0/1598289693 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc234107d90 con 0x7fc2481092f0 2024-09-21T07:12:08.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc246800640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc22c0b6670 0x7fc22c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 -- 172.21.15.27:0/3595165628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf94e0a0 msgr2=0x55ffdfd1dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd1dbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc238009c80 tx=0x7fc238011810 comp rx=0 tx=0).stop 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 -- 172.21.15.27:0/3595165628 shutdown_connections 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd1dbb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd1e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3595165628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf976f60 0x55ffdfd1d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 -- 172.21.15.27:0/3595165628 >> 172.21.15.27:0/3595165628 conn(0x55ffdf96b350 msgr2=0x55ffdfd1b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc246800640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc22c0b6670 0x7fc22c0b8b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc240011890 tx=0x7fc240009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 -- 172.21.15.27:0/3595165628 shutdown_connections 2024-09-21T07:12:08.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.156+0000 7fc253c4b740 1 -- 172.21.15.27:0/3595165628 wait complete. 2024-09-21T07:12:08.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.157+0000 7fc253c4b740 1 -- start start 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.157+0000 7fc253c4b740 1 Processor -- start 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd2d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc253c4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfae55f0 con 0x55ffdf94e0a0 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc253c4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfc1f460 con 0x55ffdf956280 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc253c4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfd301b0 con 0x55ffdf976f60 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc245e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd2d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc253c4b740 10 client.0 _ll_register_callbacks cb 0x55ffdfc2f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc245e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd2d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50784/0 (socket says 172.21.15.27:50784) 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc245e00640 1 -- 172.21.15.27:0/3579914701 learned_addr learned my addr 172.21.15.27:0/3579914701 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc246800640 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.158+0000 7fc245400640 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc245e00640 1 -- 172.21.15.27:0/3579914701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 msgr2=0x55ffdfd394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc245e00640 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc245e00640 1 -- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 msgr2=0x55ffdfd2dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc245e00640 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc245e00640 1 -- 172.21.15.27:0/3579914701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffdf9e2540 con 0x55ffdf94e0a0 2024-09-21T07:12:08.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc245e00640 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd2d710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc238000c00 tx=0x7fc2380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de4e2d7c43e7ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:08.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc227e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:12:08.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc227e00640 1 -- 172.21.15.27:0/3579914701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238029070 con 0x55ffdf94e0a0 2024-09-21T07:12:08.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc227e00640 1 -- 172.21.15.27:0/3579914701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc238003a00 con 0x55ffdf94e0a0 2024-09-21T07:12:08.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.159+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffdfc21a70 con 0x55ffdf94e0a0 2024-09-21T07:12:08.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.160+0000 7fc227e00640 1 -- 172.21.15.27:0/3579914701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc238035050 con 0x55ffdf94e0a0 2024-09-21T07:12:08.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.160+0000 7fc253c4b740 10 client.15189 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:08.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.160+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ffdfc3c6f0 con 0x55ffdf94e0a0 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.160+0000 7fc227e00640 1 -- 172.21.15.27:0/3579914701 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ffdfc21a70 con 0x55ffdf94e0a0 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.160+0000 7fc227e00640 1 -- 172.21.15.27:0/3579914701 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc238023cb0 con 0x55ffdf94e0a0 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.161+0000 7fc253c4b740 10 client.15189 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.161+0000 7fc253c4b740 10 client.15189 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.161+0000 7fc253c4b740 20 client.15189 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.161+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf9358e0 0x55ffdfd3f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.161+0000 7fc253c4b740 4 client.15189 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.161+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffdfc96ca0 con 0x55ffdf9358e0 2024-09-21T07:12:08.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.161+0000 7fc245400640 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf9358e0 0x55ffdfd3f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.162+0000 7fc245400640 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf9358e0 0x55ffdfd3f830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e756474a058b2c8 server_cookie=929e9aab0b00985e in_seq=0 out_seq=0 2024-09-21T07:12:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.162+0000 7fc227e00640 10 client.15189 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc227e00640 1 -- 172.21.15.27:0/3579914701 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffdfc96ca0 con 0x55ffdf9358e0 2024-09-21T07:12:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc227e00640 10 client.15189 handle_command_reply: tid=0 2024-09-21T07:12:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 client.15189 shutdown 2024-09-21T07:12:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 msgr2=0x55ffdfd2d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd2d710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc238000c00 tx=0x7fc2380046c0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 shutdown_connections 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf9358e0 0x55ffdfd3f830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3579914701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd2d710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 >> 172.21.15.27:0/3579914701 conn(0x55ffdf96b350 msgr2=0x55ffdfd19ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc227e00640 0 client.15189 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 shutdown_connections 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 1 -- 172.21.15.27:0/3579914701 wait complete. 2024-09-21T07:12:08.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.163+0000 7fc253c4b740 20 client.15189 trim_cache size 0 max 16384 2024-09-21T07:12:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.185+0000 7fc253c4b740 1 Processor -- start 2024-09-21T07:12:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.185+0000 7fc253c4b740 1 -- start start 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.185+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd3ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.185+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.185+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd2dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc253c4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfd3db70 con 0x55ffdf94e0a0 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc253c4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfd3e240 con 0x55ffdf956280 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc253c4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfd3e910 con 0x55ffdf976f60 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd3ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd3ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50796/0 (socket says 172.21.15.27:50796) 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 -- 172.21.15.27:0/3748993658 learned_addr learned my addr 172.21.15.27:0/3748993658 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245400640 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc246800640 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd2dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 -- 172.21.15.27:0/3748993658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 msgr2=0x55ffdfd2dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd2dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 -- 172.21.15.27:0/3748993658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 msgr2=0x55ffdfd2d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245e00640 1 -- 172.21.15.27:0/3748993658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffdfd1be50 con 0x55ffdf94e0a0 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.186+0000 7fc245400640 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.187+0000 7fc245e00640 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd3ea90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc238009b80 tx=0x7fc238011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c41963cbb02de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.187+0000 7fc232a00640 1 -- 172.21.15.27:0/3748993658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238029070 con 0x55ffdf94e0a0 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.187+0000 7fc232a00640 1 -- 172.21.15.27:0/3748993658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc238019840 con 0x55ffdf94e0a0 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.187+0000 7fc232a00640 1 -- 172.21.15.27:0/3748993658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23802e6a0 con 0x55ffdf94e0a0 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 -- 172.21.15.27:0/3748993658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 msgr2=0x55ffdfd3ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd3ea90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc238009b80 tx=0x7fc238011dc0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 -- 172.21.15.27:0/3748993658 shutdown_connections 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf976f60 0x55ffdfd2dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf956280 0x55ffdfd2d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 --2- 172.21.15.27:0/3748993658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd3ea90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 -- 172.21.15.27:0/3748993658 >> 172.21.15.27:0/3748993658 conn(0x55ffdf96b350 msgr2=0x55ffdfd1aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 -- 172.21.15.27:0/3748993658 shutdown_connections 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 -- 172.21.15.27:0/3748993658 wait complete. 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.188+0000 7fc253c4b740 1 -- start start 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 1 Processor -- start 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf956280 0x55ffdfd47700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf976f60 0x55ffdfd47c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfd3f6b0 con 0x55ffdf94e0a0 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfd3efe0 con 0x55ffdf976f60 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ffdfd3fd80 con 0x55ffdf956280 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc253c4b740 10 client.0 _ll_register_callbacks cb 0x7fc2380043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc245e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.189+0000 7fc245400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf956280 0x55ffdfd47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf956280 0x55ffdfd47700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38182/0 (socket says 172.21.15.27:38182) 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50808/0 (socket says 172.21.15.27:50808) 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 -- 172.21.15.27:0/1654724317 learned_addr learned my addr 172.21.15.27:0/1654724317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc246800640 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf976f60 0x55ffdfd47c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 -- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf976f60 msgr2=0x55ffdfd47c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf976f60 0x55ffdfd47c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 -- 172.21.15.27:0/1654724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 msgr2=0x55ffdfd471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 -- 172.21.15.27:0/1654724317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffdfd45040 con 0x55ffdf956280 2024-09-21T07:12:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245e00640 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc245400640 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf956280 0x55ffdfd47700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2340cd6a0 tx=0x7fc234003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc1b3f805184896 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.190+0000 7fc227e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc227e00640 1 -- 172.21.15.27:0/1654724317 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234048040 con 0x55ffdf956280 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc227e00640 1 -- 172.21.15.27:0/1654724317 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc2340d7080 con 0x55ffdf956280 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc238019840 con 0x55ffdf956280 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc227e00640 1 -- 172.21.15.27:0/1654724317 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2340c9ff0 con 0x55ffdf956280 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 10 client.25216 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ffdfdaaf10 con 0x55ffdf956280 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc227e00640 1 -- 172.21.15.27:0/1654724317 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc238019840 con 0x55ffdf956280 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc227e00640 1 -- 172.21.15.27:0/1654724317 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc23404b060 con 0x55ffdf956280 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 10 client.25216 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 10 client.25216 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 20 client.25216 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf90ac90 0x55ffdfdae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 4 client.25216 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.191+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "status"}) -- 0x55ffdfc92750 con 0x55ffdf90ac90 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.192+0000 7fc245e00640 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf90ac90 0x55ffdfdae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.192+0000 7fc245e00640 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf90ac90 0x55ffdfdae7f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=938f18a860fdbd1c server_cookie=d8a0ff2e8121fda3 in_seq=0 out_seq=0 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.192+0000 7fc227e00640 10 client.25216 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc227e00640 1 -- 172.21.15.27:0/1654724317 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ffdfc92750 con 0x55ffdf90ac90 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc227e00640 10 client.25216 handle_command_reply: tid=0 2024-09-21T07:12:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 client.25216 shutdown 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf956280 msgr2=0x55ffdfd47700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf956280 0x55ffdfd47700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2340cd6a0 tx=0x7fc234003f50 comp rx=0 tx=0).stop 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 shutdown_connections 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ffdf90ac90 0x55ffdfdae7f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffdf956280 0x55ffdfd47700 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffdf976f60 0x55ffdfd47c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 --2- 172.21.15.27:0/1654724317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffdf94e0a0 0x55ffdfd471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc227e00640 0 client.25216 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 >> 172.21.15.27:0/1654724317 conn(0x55ffdf96b350 msgr2=0x55ffdfd2d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 shutdown_connections 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.193+0000 7fc253c4b740 1 -- 172.21.15.27:0/1654724317 wait complete. 2024-09-21T07:12:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc253c4b740 20 client.25216 trim_cache size 0 max 16384 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24740, 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 79.660168721000005, 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 156.841316611 2024-09-21T07:12:08.199 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc22c0b6670 msgr2=0x7fc22c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc22c0b6670 0x7fc22c0b8b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc240011890 tx=0x7fc240009500 comp rx=0 tx=0).stop 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 msgr2=0x7fc24819a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 0x7fc24819a850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc23402a930 tx=0x7fc234031a20 comp rx=0 tx=0).stop 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 shutdown_connections 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc248109c40 0x7fc24819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc22c0b6670 0x7fc22c0b8b80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2481092f0 0x7fc24819a850 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 --2- 172.21.15.27:0/1598289693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2481039b0 0x7fc24819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 >> 172.21.15.27:0/1598289693 conn(0x7fc2480ff400 msgr2=0x7fc248101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.194+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 shutdown_connections 2024-09-21T07:12:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:08.195+0000 7fc247200640 1 -- 172.21.15.27:0/1598289693 wait complete. 2024-09-21T07:12:08.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.g status 2024-09-21T07:12:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:08 smithi153 ceph-mon[34747]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:12:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:08 smithi119 ceph-mon[34519]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:12:09.288 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:08 smithi027 ceph-mon[30779]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.934+0000 7f39c0a00640 1 -- 172.21.15.27:0/4175986922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 msgr2=0x7f39bc112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.934+0000 7f39c0a00640 1 --2- 172.21.15.27:0/4175986922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc112670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009ca0 tx=0x7f39b402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.935+0000 7f39c0a00640 1 -- 172.21.15.27:0/4175986922 shutdown_connections 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.935+0000 7f39c0a00640 1 --2- 172.21.15.27:0/4175986922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc112670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.935+0000 7f39c0a00640 1 --2- 172.21.15.27:0/4175986922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39bc10b4f0 0x7f39bc10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.935+0000 7f39c0a00640 1 --2- 172.21.15.27:0/4175986922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39bc105bb0 0x7f39bc105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.935+0000 7f39c0a00640 1 -- 172.21.15.27:0/4175986922 >> 172.21.15.27:0/4175986922 conn(0x7f39bc101680 msgr2=0x7f39bc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.935+0000 7f39c0a00640 1 -- 172.21.15.27:0/4175986922 shutdown_connections 2024-09-21T07:12:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.935+0000 7f39c0a00640 1 -- 172.21.15.27:0/4175986922 wait complete. 2024-09-21T07:12:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.936+0000 7f39c0a00640 1 Processor -- start 2024-09-21T07:12:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.936+0000 7f39c0a00640 1 -- start start 2024-09-21T07:12:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39c0a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39bc105bb0 0x7f39bc19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39c0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39bc10b4f0 0x7f39bc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39c0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39c0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115b50 con 0x7f39bc10b4f0 2024-09-21T07:12:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39c0a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1159d0 con 0x7f39bc10be40 2024-09-21T07:12:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39c0a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115cd0 con 0x7f39bc105bb0 2024-09-21T07:12:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39bc10b4f0 0x7f39bc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc1a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33084/0 (socket says 172.21.15.27:33084) 2024-09-21T07:12:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39bbe00640 1 -- 172.21.15.27:0/2802436269 learned_addr learned my addr 172.21.15.27:0/2802436269 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.937+0000 7f39bb400640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39bc105bb0 0x7f39bc19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39bbe00640 1 -- 172.21.15.27:0/2802436269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39bc105bb0 msgr2=0x7f39bc19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39bc105bb0 0x7f39bc19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39bbe00640 1 -- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39bc10b4f0 msgr2=0x7f39bc19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39bc10b4f0 0x7f39bc19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39bbe00640 1 -- 172.21.15.27:0/2802436269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc1a2410 con 0x7f39bc10be40 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc1a1ce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009430 tx=0x7f39b40317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39b3e00640 1 -- 172.21.15.27:0/2802436269 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003b70 con 0x7f39bc10be40 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39b3e00640 1 -- 172.21.15.27:0/2802436269 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39b4003d10 con 0x7f39bc10be40 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.938+0000 7f39b3e00640 1 -- 172.21.15.27:0/2802436269 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b402fa30 con 0x7f39bc10be40 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.939+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1a26a0 con 0x7f39bc10be40 2024-09-21T07:12:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.939+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a8e60 con 0x7f39bc10be40 2024-09-21T07:12:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.942+0000 7f39c8bde740 1 Processor -- start 2024-09-21T07:12:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.942+0000 7f39b3e00640 1 -- 172.21.15.27:0/2802436269 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f39b402fbd0 con 0x7f39bc10be40 2024-09-21T07:12:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.942+0000 7f39c8bde740 1 -- start start 2024-09-21T07:12:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a032670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a032bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a0330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39c8bde740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564e89fa9d50 con 0x564e89c8bf60 2024-09-21T07:12:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39c8bde740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564e89f99180 con 0x564e89c6b280 2024-09-21T07:12:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39c8bde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e89f9cf90 con 0x564e89c630a0 2024-09-21T07:12:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a032670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a032670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50824/0 (socket says 172.21.15.27:50824) 2024-09-21T07:12:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39baa00640 1 -- 172.21.15.27:0/1232457508 learned_addr learned my addr 172.21.15.27:0/1232457508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39baa00640 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a0330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.943+0000 7f39bb400640 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a032bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39baa00640 1 -- 172.21.15.27:0/1232457508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 msgr2=0x564e8a032bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39baa00640 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a032bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39baa00640 1 -- 172.21.15.27:0/1232457508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 msgr2=0x564e8a0330f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39baa00640 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a0330f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39baa00640 1 -- 172.21.15.27:0/1232457508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e89f63a70 con 0x564e89c8bf60 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39baa00640 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a032670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39a8013720 tx=0x7f39a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b52e6d152144abd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39b3e00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39a00b6670 0x7f39a00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39a7e00640 1 -- 172.21.15.27:0/1232457508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8013ca0 con 0x564e89c8bf60 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39a7e00640 1 -- 172.21.15.27:0/1232457508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a8013e40 con 0x564e89c8bf60 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39a7e00640 1 -- 172.21.15.27:0/1232457508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8004a00 con 0x564e89c8bf60 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.944+0000 7f39b3e00640 1 -- 172.21.15.27:0/2802436269 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f39b4108b40 con 0x7f39bc10be40 2024-09-21T07:12:09.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.945+0000 7f39bb400640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39a00b6670 0x7f39a00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.945+0000 7f39c8bde740 1 -- 172.21.15.27:0/1232457508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 msgr2=0x564e8a032670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.945+0000 7f39bb400640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39a00b6670 0x7f39a00b8b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x564e8a0312f0 tx=0x7f39ac00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.945+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a032670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39a8013720 tx=0x7f39a8005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 -- 172.21.15.27:0/1232457508 shutdown_connections 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a032bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a0330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1232457508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a032670 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 -- 172.21.15.27:0/1232457508 >> 172.21.15.27:0/1232457508 conn(0x564e89c80350 msgr2=0x564e8a030560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 -- 172.21.15.27:0/1232457508 shutdown_connections 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 -- 172.21.15.27:0/1232457508 wait complete. 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.946+0000 7f39c8bde740 1 -- start start 2024-09-21T07:12:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 1 Processor -- start 2024-09-21T07:12:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a03e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c6b280 0x564e8a03e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a04a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564e89dfa5f0 con 0x564e89c6b280 2024-09-21T07:12:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564e89f34460 con 0x564e89c8bf60 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e8a040df0 con 0x564e89c630a0 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.947+0000 7f39c8bde740 10 client.0 _ll_register_callbacks cb 0x564e89f44210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a04a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a04a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33108/0 (socket says 172.21.15.27:33108) 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 -- 172.21.15.27:0/2083362613 learned_addr learned my addr 172.21.15.27:0/2083362613 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a03e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39baa00640 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c6b280 0x564e8a03e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 -- 172.21.15.27:0/2083362613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 msgr2=0x564e8a03e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a03e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 -- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c6b280 msgr2=0x564e8a03e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c6b280 0x564e8a03e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 -- 172.21.15.27:0/2083362613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e89cf7540 con 0x564e89c8bf60 2024-09-21T07:12:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39baa00640 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c6b280 0x564e8a03e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a04a140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ce3a0 tx=0x7f39b40ce470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1670eba7453b2d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.948+0000 7f39a4c00640 1 -- 172.21.15.27:0/2083362613 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b40ce1b0 con 0x564e89c8bf60 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39a4c00640 1 -- 172.21.15.27:0/2083362613 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f39b40cd900 con 0x564e89c8bf60 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e89f36a70 con 0x564e89c8bf60 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39a4c00640 1 -- 172.21.15.27:0/2083362613 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39b40ceda0 con 0x564e89c8bf60 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 10 client.25232 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e89f516f0 con 0x564e89c8bf60 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39a4c00640 1 -- 172.21.15.27:0/2083362613 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564e89f36a70 con 0x564e89c8bf60 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39a4c00640 1 -- 172.21.15.27:0/2083362613 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f39b40d71b0 con 0x564e89c8bf60 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 10 client.25232 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 10 client.25232 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 20 client.25232 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c4a8e0 0x564e8a0507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 4 client.25232 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.949+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e89fabca0 con 0x564e89c4a8e0 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.950+0000 7f39bb400640 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c4a8e0 0x564e8a0507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.950+0000 7f39a4c00640 10 client.25232 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.950+0000 7f39bb400640 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c4a8e0 0x564e8a0507b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee3a8c6652c8cfa1 server_cookie=303c7693622875ea in_seq=0 out_seq=0 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.951+0000 7f39a4c00640 1 -- 172.21.15.27:0/2083362613 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e89fabca0 con 0x564e89c4a8e0 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.951+0000 7f39a4c00640 10 client.25232 handle_command_reply: tid=0 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 client.25232 shutdown 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 msgr2=0x564e8a04a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a04a140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ce3a0 tx=0x7f39b40ce470 comp rx=0 tx=0).stop 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 shutdown_connections 2024-09-21T07:12:09.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c4a8e0 0x564e8a0507b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a04a140 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c6b280 0x564e8a03e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2083362613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a03e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39a4c00640 0 client.25232 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 >> 172.21.15.27:0/2083362613 conn(0x564e89c80350 msgr2=0x564e8a035850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 shutdown_connections 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 1 -- 172.21.15.27:0/2083362613 wait complete. 2024-09-21T07:12:09.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.952+0000 7f39c8bde740 20 client.25232 trim_cache size 0 max 16384 2024-09-21T07:12:09.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.972+0000 7f39c8bde740 1 Processor -- start 2024-09-21T07:12:09.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.972+0000 7f39c8bde740 1 -- start start 2024-09-21T07:12:09.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a04dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a04e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a039240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39c8bde740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564e8a04eaf0 con 0x564e89c8bf60 2024-09-21T07:12:09.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39c8bde740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564e8a04f1c0 con 0x564e89c6b280 2024-09-21T07:12:09.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39c8bde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e8a04f890 con 0x564e89c630a0 2024-09-21T07:12:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a04e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a04e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33122/0 (socket says 172.21.15.27:33122) 2024-09-21T07:12:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 -- 172.21.15.27:0/1856069410 learned_addr learned my addr 172.21.15.27:0/1856069410 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39bbe00640 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a039240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39bb400640 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a04dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 -- 172.21.15.27:0/1856069410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 msgr2=0x564e8a04dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a04dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 -- 172.21.15.27:0/1856069410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 msgr2=0x564e8a039240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a039240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.973+0000 7f39baa00640 1 -- 172.21.15.27:0/1856069410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e8a03f350 con 0x564e89c6b280 2024-09-21T07:12:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.974+0000 7f39baa00640 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a04e530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39a8016040 tx=0x7f39a801c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83ac11e11d19af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.974+0000 7f39a7400640 1 -- 172.21.15.27:0/1856069410 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a80050b0 con 0x564e89c6b280 2024-09-21T07:12:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.974+0000 7f39a7400640 1 -- 172.21.15.27:0/1856069410 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a8017040 con 0x564e89c6b280 2024-09-21T07:12:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.974+0000 7f39a7400640 1 -- 172.21.15.27:0/1856069410 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8003900 con 0x564e89c6b280 2024-09-21T07:12:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 -- 172.21.15.27:0/1856069410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 msgr2=0x564e8a04e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a04e530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39a8016040 tx=0x7f39a801c040 comp rx=0 tx=0).stop 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 -- 172.21.15.27:0/1856069410 shutdown_connections 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c8bf60 0x564e8a039240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c6b280 0x564e8a04e530 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 --2- 172.21.15.27:0/1856069410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c630a0 0x564e8a04dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 -- 172.21.15.27:0/1856069410 >> 172.21.15.27:0/1856069410 conn(0x564e89c80350 msgr2=0x564e8a04a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 -- 172.21.15.27:0/1856069410 shutdown_connections 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.975+0000 7f39c8bde740 1 -- 172.21.15.27:0/1856069410 wait complete. 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 -- start start 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 Processor -- start 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c630a0 0x564e8a059350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c6b280 0x564e8a052f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a0535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564e8a050630 con 0x564e89c630a0 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564e8a04ff60 con 0x564e89c8bf60 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e8a050d00 con 0x564e89c6b280 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.976+0000 7f39c8bde740 10 client.0 _ll_register_callbacks cb 0x7f39b40ca5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a0535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c630a0 0x564e8a059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39baa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c6b280 0x564e8a052f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a0535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33128/0 (socket says 172.21.15.27:33128) 2024-09-21T07:12:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 -- 172.21.15.27:0/2631344783 learned_addr learned my addr 172.21.15.27:0/2631344783 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 -- 172.21.15.27:0/2631344783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c6b280 msgr2=0x564e8a052f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c6b280 0x564e8a052f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 -- 172.21.15.27:0/2631344783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c630a0 msgr2=0x564e8a059350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c630a0 0x564e8a059350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 -- 172.21.15.27:0/2631344783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e8a053d50 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39bbe00640 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a0535e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ce630 tx=0x7f39b4105f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d002dc191777d015 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.977+0000 7f39a4c00640 1 -- 172.21.15.27:0/2631344783 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b40e9070 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a8017040 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39a4c00640 1 -- 172.21.15.27:0/2631344783 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f39b4004b40 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39a4c00640 1 -- 172.21.15.27:0/2631344783 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39b40cdff0 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 10 client.25238 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e8a0bbc80 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39a4c00640 1 -- 172.21.15.27:0/2631344783 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a8017040 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39a4c00640 1 -- 172.21.15.27:0/2631344783 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f39b40d4390 con 0x564e89c8bf60 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 10 client.25238 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 10 client.25238 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 20 client.25238 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c1fc90 0x564e8a0bf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 4 client.25238 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.978+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "status"}) -- 0x564e89fa7750 con 0x564e89c1fc90 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.979+0000 7f39bb400640 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c1fc90 0x564e8a0bf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.979+0000 7f39bb400640 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c1fc90 0x564e8a0bf3d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88033d01f929ffa3 server_cookie=1b865a63b517aacb in_seq=0 out_seq=0 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.979+0000 7f39a4c00640 10 client.25238 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.979+0000 7f39a4c00640 1 -- 172.21.15.27:0/2631344783 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x564e89fa7750 con 0x564e89c1fc90 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.979+0000 7f39a4c00640 10 client.25238 handle_command_reply: tid=0 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 client.25238 shutdown 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 msgr2=0x564e8a0535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a0535e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ce630 tx=0x7f39b4105f60 comp rx=0 tx=0).stop 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 shutdown_connections 2024-09-21T07:12:09.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x564e89c1fc90 0x564e8a0bf3d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39a4c00640 0 client.25238 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e89c6b280 0x564e8a052f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e89c8bf60 0x564e8a0535e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 --2- 172.21.15.27:0/2631344783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564e89c630a0 0x564e8a059350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 >> 172.21.15.27:0/2631344783 conn(0x564e89c80350 msgr2=0x564e8a04a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 shutdown_connections 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 1 -- 172.21.15.27:0/2631344783 wait complete. 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.980+0000 7f39c8bde740 20 client.25238 trim_cache size 0 max 16384 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:09.985 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24748, 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 82.454619944000001, 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 152.51265411 2024-09-21T07:12:09.986 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39a00b6670 msgr2=0x7f39a00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39a00b6670 0x7f39a00b8b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x564e8a0312f0 tx=0x7f39ac00b410 comp rx=0 tx=0).stop 2024-09-21T07:12:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 msgr2=0x7f39bc1a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc1a1ce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009430 tx=0x7f39b40317c0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 shutdown_connections 2024-09-21T07:12:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39a00b6670 0x7f39a00b8b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39bc10be40 0x7f39bc1a1ce0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39bc10b4f0 0x7f39bc19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 --2- 172.21.15.27:0/2802436269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39bc105bb0 0x7f39bc19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 >> 172.21.15.27:0/2802436269 conn(0x7f39bc101680 msgr2=0x7f39bc108a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 shutdown_connections 2024-09-21T07:12:09.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:09.981+0000 7f39c0a00640 1 -- 172.21.15.27:0/2802436269 wait complete. 2024-09-21T07:12:10.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T07:12:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:10 smithi027 ceph-mon[30779]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:10 smithi153 ceph-mon[34747]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:10 smithi119 ceph-mon[34519]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 -- 172.21.15.27:0/1108831111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 msgr2=0x7fd7441096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 --2- 172.21.15.27:0/1108831111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 0x7fd7441096e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd73002a410 comp rx=0 tx=0).stop 2024-09-21T07:12:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 -- 172.21.15.27:0/1108831111 shutdown_connections 2024-09-21T07:12:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 --2- 172.21.15.27:0/1108831111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd744109c20 0x7fd744110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 --2- 172.21.15.27:0/1108831111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 0x7fd7441096e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 --2- 172.21.15.27:0/1108831111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd744103990 0x7fd744103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 -- 172.21.15.27:0/1108831111 >> 172.21.15.27:0/1108831111 conn(0x7fd7440ff3e0 msgr2=0x7fd744101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 -- 172.21.15.27:0/1108831111 shutdown_connections 2024-09-21T07:12:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.737+0000 7fd749400640 1 -- 172.21.15.27:0/1108831111 wait complete. 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.738+0000 7fd749400640 1 Processor -- start 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.738+0000 7fd749400640 1 -- start start 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.738+0000 7fd749400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd744103990 0x7fd74419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.738+0000 7fd749400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 0x7fd74419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd749400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd744109c20 0x7fd74419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd749400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441138e0 con 0x7fd7441092d0 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd749400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd744113760 con 0x7fd744103990 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd749400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd744113a60 con 0x7fd744109c20 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 0x7fd74419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd744103990 0x7fd74419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd744103990 0x7fd74419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33142/0 (socket says 172.21.15.27:33142) 2024-09-21T07:12:11.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743e00640 1 -- 172.21.15.27:0/3383087998 learned_addr learned my addr 172.21.15.27:0/3383087998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:11.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd748a00640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd744109c20 0x7fd74419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743400640 1 -- 172.21.15.27:0/3383087998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd744109c20 msgr2=0x7fd74419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd744109c20 0x7fd74419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743400640 1 -- 172.21.15.27:0/3383087998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd744103990 msgr2=0x7fd74419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd744103990 0x7fd74419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.739+0000 7fd743400640 1 -- 172.21.15.27:0/3383087998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441a0260 con 0x7fd7441092d0 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.740+0000 7fd743400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 0x7fd74419a890 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd73002a930 tx=0x7fd730004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.740+0000 7fd740c00640 1 -- 172.21.15.27:0/3383087998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73003f070 con 0x7fd7441092d0 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.740+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7441a04f0 con 0x7fd7441092d0 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.740+0000 7fd740c00640 1 -- 172.21.15.27:0/3383087998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd730003ca0 con 0x7fd7441092d0 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.740+0000 7fd740c00640 1 -- 172.21.15.27:0/3383087998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730035070 con 0x7fd7441092d0 2024-09-21T07:12:11.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.740+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441a6cb0 con 0x7fd7441092d0 2024-09-21T07:12:11.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.744+0000 7fd7516b9740 1 Processor -- start 2024-09-21T07:12:11.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.744+0000 7fd7516b9740 1 -- start start 2024-09-21T07:12:11.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.744+0000 7fd740c00640 1 -- 172.21.15.27:0/3383087998 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd73003b460 con 0x7fd7441092d0 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.744+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973066df60 0x559730a14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.744+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd7516b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55973098bd50 con 0x55973064d280 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd7516b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55973097b180 con 0x5597306450a0 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd7516b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55973097ef90 con 0x55973066df60 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a14bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33146/0 (socket says 172.21.15.27:33146) 2024-09-21T07:12:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 -- 172.21.15.27:0/2313199423 learned_addr learned my addr 172.21.15.27:0/2313199423 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743400640 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd748a00640 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973066df60 0x559730a14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 -- 172.21.15.27:0/2313199423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973066df60 msgr2=0x559730a14670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973066df60 0x559730a14670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 -- 172.21.15.27:0/2313199423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 msgr2=0x559730a150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.745+0000 7fd743e00640 1 -- 172.21.15.27:0/2313199423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559730945a70 con 0x5597306450a0 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.746+0000 7fd743e00640 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a14bb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd734009dc0 tx=0x7fd734011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c717367e6d9bd8c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.746+0000 7fd739200640 1 -- 172.21.15.27:0/2313199423 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73401e070 con 0x5597306450a0 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.746+0000 7fd739200640 1 -- 172.21.15.27:0/2313199423 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd734019ae0 con 0x5597306450a0 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.746+0000 7fd739200640 1 -- 172.21.15.27:0/2313199423 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400ea90 con 0x5597306450a0 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.746+0000 7fd740c00640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7280b65f0 0x7fd7280b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.746+0000 7fd740c00640 1 -- 172.21.15.27:0/3383087998 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd730107f40 con 0x7fd7441092d0 2024-09-21T07:12:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd748a00640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7280b65f0 0x7fd7280b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 -- 172.21.15.27:0/2313199423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 msgr2=0x559730a14bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a14bb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd734009dc0 tx=0x7fd734011810 comp rx=0 tx=0).stop 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 -- 172.21.15.27:0/2313199423 shutdown_connections 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a14bb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2313199423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973066df60 0x559730a14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 -- 172.21.15.27:0/2313199423 >> 172.21.15.27:0/2313199423 conn(0x559730662350 msgr2=0x559730a12560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 -- 172.21.15.27:0/2313199423 shutdown_connections 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd748a00640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7280b65f0 0x7fd7280b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005fd0 tx=0x7fd73c009740 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.747+0000 7fd7516b9740 1 -- 172.21.15.27:0/2313199423 wait complete. 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.748+0000 7fd7516b9740 1 -- start start 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.748+0000 7fd7516b9740 1 Processor -- start 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597306450a0 0x559730a20200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a20740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a30550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd7516b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5597307dc5f0 con 0x55973064d280 2024-09-21T07:12:11.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd7516b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559730916460 con 0x55973066df60 2024-09-21T07:12:11.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd7516b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559730a22e50 con 0x5597306450a0 2024-09-21T07:12:11.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd7516b9740 10 client.0 _ll_register_callbacks cb 0x559730926210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd743400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a20740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd743e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597306450a0 0x559730a20200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd743400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a20740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50878/0 (socket says 172.21.15.27:50878) 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd743400640 1 -- 172.21.15.27:0/2541815964 learned_addr learned my addr 172.21.15.27:0/2541815964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.749+0000 7fd748a00640 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a30550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd743400640 1 -- 172.21.15.27:0/2541815964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597306450a0 msgr2=0x559730a20200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd743400640 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597306450a0 0x559730a20200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd743400640 1 -- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 msgr2=0x559730a30550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd743400640 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a30550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd743400640 1 -- 172.21.15.27:0/2541815964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597306d9540 con 0x55973064d280 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd743400640 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a20740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd7300cad10 tx=0x7fd7300045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e1ccd695175093 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd72d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd72d600640 1 -- 172.21.15.27:0/2541815964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730048070 con 0x55973064d280 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd72d600640 1 -- 172.21.15.27:0/2541815964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd7300d7230 con 0x55973064d280 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559730918a70 con 0x55973064d280 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd72d600640 1 -- 172.21.15.27:0/2541815964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7300c9940 con 0x55973064d280 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd7516b9740 10 client.15225 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.750+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597309336f0 con 0x55973064d280 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd72d600640 1 -- 172.21.15.27:0/2541815964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559730918a70 con 0x55973064d280 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd72d600640 1 -- 172.21.15.27:0/2541815964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd73004b060 con 0x55973064d280 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd7516b9740 10 client.15225 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd7516b9740 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:12:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd7516b9740 20 client.15225 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55973062c8e0 0x559730a36b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd7516b9740 4 client.15225 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.751+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55973098dca0 con 0x55973062c8e0 2024-09-21T07:12:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.752+0000 7fd743e00640 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55973062c8e0 0x559730a36b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.752+0000 7fd743e00640 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55973062c8e0 0x559730a36b80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11bc0dabdb5b4b90 server_cookie=8fde6c9467ddbf8b in_seq=0 out_seq=0 2024-09-21T07:12:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.752+0000 7fd72d600640 10 client.15225 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd72d600640 1 -- 172.21.15.27:0/2541815964 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55973098dca0 con 0x55973062c8e0 2024-09-21T07:12:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd72d600640 10 client.15225 handle_command_reply: tid=0 2024-09-21T07:12:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 client.15225 shutdown 2024-09-21T07:12:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 msgr2=0x559730a20740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a20740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd7300cad10 tx=0x7fd7300045a0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 shutdown_connections 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55973062c8e0 0x559730a36b80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a30550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973064d280 0x559730a20740 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 --2- 172.21.15.27:0/2541815964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597306450a0 0x559730a20200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 >> 172.21.15.27:0/2541815964 conn(0x559730662350 msgr2=0x559730a17800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.753+0000 7fd72d600640 0 client.15225 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.754+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 shutdown_connections 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.754+0000 7fd7516b9740 1 -- 172.21.15.27:0/2541815964 wait complete. 2024-09-21T07:12:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.754+0000 7fd7516b9740 20 client.15225 trim_cache size 0 max 16384 2024-09-21T07:12:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 Processor -- start 2024-09-21T07:12:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 -- start start 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a30540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a35040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973066df60 0x559730a1a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559730a34ec0 con 0x55973066df60 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559730a35590 con 0x5597306450a0 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.773+0000 7fd7516b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559730a35c60 con 0x55973064d280 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973066df60 0x559730a1a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973066df60 0x559730a1a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50888/0 (socket says 172.21.15.27:50888) 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 -- 172.21.15.27:0/369115966 learned_addr learned my addr 172.21.15.27:0/369115966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd743400640 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a35040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd743e00640 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a30540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 -- 172.21.15.27:0/369115966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 msgr2=0x559730a35040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a35040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 -- 172.21.15.27:0/369115966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 msgr2=0x559730a30540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a30540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.774+0000 7fd748a00640 1 -- 172.21.15.27:0/369115966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559730a1e500 con 0x55973066df60 2024-09-21T07:12:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.775+0000 7fd748a00640 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973066df60 0x559730a1a790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd73c01e7f0 tx=0x7fd73c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70508e65302893a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.775+0000 7fd72fe00640 1 -- 172.21.15.27:0/369115966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c01ed70 con 0x55973066df60 2024-09-21T07:12:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.775+0000 7fd72fe00640 1 -- 172.21.15.27:0/369115966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73c01fdd0 con 0x55973066df60 2024-09-21T07:12:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.775+0000 7fd72fe00640 1 -- 172.21.15.27:0/369115966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c01ce10 con 0x55973066df60 2024-09-21T07:12:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 -- 172.21.15.27:0/369115966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973066df60 msgr2=0x559730a1a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973066df60 0x559730a1a790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd73c01e7f0 tx=0x7fd73c015000 comp rx=0 tx=0).stop 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 -- 172.21.15.27:0/369115966 shutdown_connections 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a35040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55973066df60 0x559730a1a790 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 --2- 172.21.15.27:0/369115966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597306450a0 0x559730a30540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 -- 172.21.15.27:0/369115966 >> 172.21.15.27:0/369115966 conn(0x559730662350 msgr2=0x559730a1b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 -- 172.21.15.27:0/369115966 shutdown_connections 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 -- 172.21.15.27:0/369115966 wait complete. 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.776+0000 7fd7516b9740 1 -- start start 2024-09-21T07:12:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.777+0000 7fd7516b9740 1 Processor -- start 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.777+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597306450a0 0x559730a3f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a38b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd7516b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a39050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd7516b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559730a36a00 con 0x5597306450a0 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd7516b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559730a36330 con 0x55973066df60 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd7516b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559730a370d0 con 0x55973064d280 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd7516b9740 10 client.0 _ll_register_callbacks cb 0x7fd7300c9f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597306450a0 0x559730a3f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597306450a0 0x559730a3f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50898/0 (socket says 172.21.15.27:50898) 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 -- 172.21.15.27:0/3471005807 learned_addr learned my addr 172.21.15.27:0/3471005807 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743400640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a38b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd748a00640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a39050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 -- 172.21.15.27:0/3471005807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 msgr2=0x559730a38b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a38b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 -- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 msgr2=0x559730a39050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a39050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd743e00640 1 -- 172.21.15.27:0/3471005807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559730a397e0 con 0x5597306450a0 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.778+0000 7fd748a00640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a39050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd743e00640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597306450a0 0x559730a3f610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e720 tx=0x7fd734011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e483ca5c7ea990c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd72d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:12:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd72d600640 1 -- 172.21.15.27:0/3471005807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73401e040 con 0x5597306450a0 2024-09-21T07:12:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd72d600640 1 -- 172.21.15.27:0/3471005807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd734005160 con 0x5597306450a0 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73c01fdd0 con 0x5597306450a0 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd72d600640 1 -- 172.21.15.27:0/3471005807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd73403a050 con 0x5597306450a0 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd7516b9740 10 client.15231 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.779+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559730aa2050 con 0x5597306450a0 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd72d600640 1 -- 172.21.15.27:0/3471005807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73c01fdd0 con 0x5597306450a0 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd72d600640 1 -- 172.21.15.27:0/3471005807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd734005360 con 0x5597306450a0 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd7516b9740 10 client.15231 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd7516b9740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd7516b9740 20 client.15231 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd7516b9740 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559730601c90 0x559730aa5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd7516b9740 4 client.15231 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.780+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "status"}) -- 0x559730989750 con 0x559730601c90 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.781+0000 7fd743400640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559730601c90 0x559730aa5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.781+0000 7fd743400640 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559730601c90 0x559730aa5650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412d67c4133a941c server_cookie=f1dd70893a9cae7a in_seq=0 out_seq=0 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.781+0000 7fd72d600640 10 client.15231 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.781+0000 7fd72d600640 1 -- 172.21.15.27:0/3471005807 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559730989750 con 0x559730601c90 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.781+0000 7fd72d600640 10 client.15231 handle_command_reply: tid=0 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.781+0000 7fd7516b9740 1 client.15231 shutdown 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597306450a0 msgr2=0x559730a3f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597306450a0 0x559730a3f610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e720 tx=0x7fd734011e00 comp rx=0 tx=0).stop 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 shutdown_connections 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559730601c90 0x559730aa5650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973064d280 0x559730a38b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55973066df60 0x559730a39050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 --2- 172.21.15.27:0/3471005807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597306450a0 0x559730a3f610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 >> 172.21.15.27:0/3471005807 conn(0x559730662350 msgr2=0x559730a1e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd72d600640 0 client.15231 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 shutdown_connections 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 1 -- 172.21.15.27:0/3471005807 wait complete. 2024-09-21T07:12:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.782+0000 7fd7516b9740 20 client.15231 trim_cache size 0 max 16384 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24773, 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:12:11.786 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 134.440469396, 2024-09-21T07:12:11.787 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T07:12:11.787 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:12:11.787 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T07:12:11.787 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 148.597780102 2024-09-21T07:12:11.787 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7280b65f0 msgr2=0x7fd7280b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7280b65f0 0x7fd7280b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005fd0 tx=0x7fd73c009740 comp rx=0 tx=0).stop 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 msgr2=0x7fd74419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 0x7fd74419a890 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd73002a930 tx=0x7fd730004060 comp rx=0 tx=0).stop 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 shutdown_connections 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd744109c20 0x7fd74419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7280b65f0 0x7fd7280b8b00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7441092d0 0x7fd74419a890 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 --2- 172.21.15.27:0/3383087998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd744103990 0x7fd74419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 >> 172.21.15.27:0/3383087998 conn(0x7fd7440ff3e0 msgr2=0x7fd744101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 shutdown_connections 2024-09-21T07:12:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:11.783+0000 7fd749400640 1 -- 172.21.15.27:0/3383087998 wait complete. 2024-09-21T07:12:12.446 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T07:12:12.446 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T07:12:12.456 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:12:12.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T07:12:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:13.121 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:12 smithi027 ceph-mon[30779]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:12 smithi153 ceph-mon[34747]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:13.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:12 smithi119 ceph-mon[34519]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:13.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 -- 172.21.15.27:0/4270914830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392010b6d0 msgr2=0x7f392010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 --2- 172.21.15.27:0/4270914830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392010b6d0 0x7f392010bae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f390c009900 tx=0x7f390c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 -- 172.21.15.27:0/4270914830 shutdown_connections 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 --2- 172.21.15.27:0/4270914830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392010c020 0x7f3920112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 --2- 172.21.15.27:0/4270914830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f392010b6d0 0x7f392010bae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 --2- 172.21.15.27:0/4270914830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3920105d90 0x7f39201061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 -- 172.21.15.27:0/4270914830 >> 172.21.15.27:0/4270914830 conn(0x7f39201017c0 msgr2=0x7f3920103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 -- 172.21.15.27:0/4270914830 shutdown_connections 2024-09-21T07:12:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.459+0000 7f3928400640 1 -- 172.21.15.27:0/4270914830 wait complete. 2024-09-21T07:12:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.460+0000 7f3928400640 1 Processor -- start 2024-09-21T07:12:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.460+0000 7f3928400640 1 -- start start 2024-09-21T07:12:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3928400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3920105d90 0x7f392019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3928400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392010b6d0 0x7f392019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3928400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392010c020 0x7f39201a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3928400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3920115ce0 con 0x7f3920105d90 2024-09-21T07:12:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3928400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3920115b60 con 0x7f392010c020 2024-09-21T07:12:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3928400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3920115e60 con 0x7f392010b6d0 2024-09-21T07:12:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3925a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3920105d90 0x7f392019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3925a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3920105d90 0x7f392019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50924/0 (socket says 172.21.15.27:50924) 2024-09-21T07:12:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3925a00640 1 -- 172.21.15.27:0/1360736578 learned_addr learned my addr 172.21.15.27:0/1360736578 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3926400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392010c020 0x7f39201a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.461+0000 7f3925000640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392010b6d0 0x7f392019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3926400640 1 -- 172.21.15.27:0/1360736578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392010b6d0 msgr2=0x7f392019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3926400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392010b6d0 0x7f392019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3926400640 1 -- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3920105d90 msgr2=0x7f392019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3926400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3920105d90 0x7f392019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3926400640 1 -- 172.21.15.27:0/1360736578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39201a2640 con 0x7f392010c020 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3925a00640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3920105d90 0x7f392019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3926400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392010c020 0x7f39201a1f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3918009fd0 tx=0x7f391800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f391e000640 1 -- 172.21.15.27:0/1360736578 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391800ce20 con 0x7f392010c020 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.462+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39201a2930 con 0x7f392010c020 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.463+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39201a9090 con 0x7f392010c020 2024-09-21T07:12:13.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.463+0000 7f391e000640 1 -- 172.21.15.27:0/1360736578 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3918010430 con 0x7f392010c020 2024-09-21T07:12:13.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.463+0000 7f391e000640 1 -- 172.21.15.27:0/1360736578 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391800f620 con 0x7f392010c020 2024-09-21T07:12:13.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.463+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39000052c0 con 0x7f392010c020 2024-09-21T07:12:13.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.466+0000 7f391e000640 1 -- 172.21.15.27:0/1360736578 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3918004a40 con 0x7f392010c020 2024-09-21T07:12:13.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.468+0000 7f391e000640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39080b65f0 0x7f39080b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.468+0000 7f391e000640 1 -- 172.21.15.27:0/1360736578 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f39180e7fe0 con 0x7f392010c020 2024-09-21T07:12:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.468+0000 7f3925a00640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39080b65f0 0x7f39080b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:13.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.469+0000 7f3925a00640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39080b65f0 0x7f39080b8b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3914011550 tx=0x7f3914005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.470+0000 7f391e000640 1 -- 172.21.15.27:0/1360736578 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39180ae260 con 0x7f392010c020 2024-09-21T07:12:13.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.686+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3900000bf0 con 0x7f39080b65f0 2024-09-21T07:12:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.687+0000 7f391e000640 1 -- 172.21.15.27:0/1360736578 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3900000bf0 con 0x7f39080b65f0 2024-09-21T07:12:13.689 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2024-09-21T07:12:13.689 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2024-09-21T07:12:13.689 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.689+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39080b65f0 msgr2=0x7f39080b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.689+0000 7f3928400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39080b65f0 0x7f39080b8b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3914011550 tx=0x7f3914005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392010c020 msgr2=0x7f39201a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392010c020 0x7f39201a1f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3918009fd0 tx=0x7f391800ef10 comp rx=0 tx=0).stop 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 shutdown_connections 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392010b6d0 0x7f392019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f39080b65f0 0x7f39080b8b00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f392010c020 0x7f39201a1f10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 --2- 172.21.15.27:0/1360736578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3920105d90 0x7f392019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:13.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 >> 172.21.15.27:0/1360736578 conn(0x7f39201017c0 msgr2=0x7f392010ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:13.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 shutdown_connections 2024-09-21T07:12:13.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:13.690+0000 7f3928400640 1 -- 172.21.15.27:0/1360736578 wait complete. 2024-09-21T07:12:14.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T07:12:14.774 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:15.121 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:14 smithi027 ceph-mon[30779]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:15.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:14 smithi027 ceph-mon[30779]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:14 smithi153 ceph-mon[34747]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:14 smithi153 ceph-mon[34747]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:14 smithi119 ceph-mon[34519]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:15.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:14 smithi119 ceph-mon[34519]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 -- 172.21.15.27:0/3963884606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc000760a0 msgr2=0x7fdc0010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 --2- 172.21.15.27:0/3963884606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc000760a0 0x7fdc0010fc00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009900 tx=0x7fdbec02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 -- 172.21.15.27:0/3963884606 shutdown_connections 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 --2- 172.21.15.27:0/3963884606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc00110140 0x7fdc00112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 --2- 172.21.15.27:0/3963884606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc000760a0 0x7fdc0010fc00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 --2- 172.21.15.27:0/3963884606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 0x7fdc00075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 -- 172.21.15.27:0/3963884606 >> 172.21.15.27:0/3963884606 conn(0x7fdc000ff5d0 msgr2=0x7fdc00101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 -- 172.21.15.27:0/3963884606 shutdown_connections 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.417+0000 7fdc08000640 1 -- 172.21.15.27:0/3963884606 wait complete. 2024-09-21T07:12:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.418+0000 7fdc08000640 1 Processor -- start 2024-09-21T07:12:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.418+0000 7fdc08000640 1 -- start start 2024-09-21T07:12:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc08000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 0x7fdc0019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc08000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc000760a0 0x7fdc0019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc08000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc00110140 0x7fdc001a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc08000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00115cb0 con 0x7fdc00110140 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc08000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00115b30 con 0x7fdc00075750 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc08000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00115e30 con 0x7fdc000760a0 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc05600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 0x7fdc0019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc06000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc00110140 0x7fdc001a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc05600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 0x7fdc0019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33218/0 (socket says 172.21.15.27:33218) 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc05600640 1 -- 172.21.15.27:0/3899228499 learned_addr learned my addr 172.21.15.27:0/3899228499 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.419+0000 7fdc04c00640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc000760a0 0x7fdc0019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdc05600640 1 -- 172.21.15.27:0/3899228499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc000760a0 msgr2=0x7fdc0019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdc05600640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc000760a0 0x7fdc0019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdc05600640 1 -- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc00110140 msgr2=0x7fdc001a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdc05600640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc00110140 0x7fdc001a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdc05600640 1 -- 172.21.15.27:0/3899228499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc001a2630 con 0x7fdc00075750 2024-09-21T07:12:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdc04c00640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc000760a0 0x7fdc0019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdc05600640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 0x7fdc0019c6c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400ea60 tx=0x7fdbf400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdbfe000640 1 -- 172.21.15.27:0/3899228499 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400cde0 con 0x7fdc00075750 2024-09-21T07:12:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.420+0000 7fdbfe000640 1 -- 172.21.15.27:0/3899228499 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf40035f0 con 0x7fdc00075750 2024-09-21T07:12:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.421+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc001a2920 con 0x7fdc00075750 2024-09-21T07:12:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.421+0000 7fdbfe000640 1 -- 172.21.15.27:0/3899228499 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4010690 con 0x7fdc00075750 2024-09-21T07:12:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.421+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001a9080 con 0x7fdc00075750 2024-09-21T07:12:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.422+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc00075b60 con 0x7fdc00075750 2024-09-21T07:12:15.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.424+0000 7fdbfe000640 1 -- 172.21.15.27:0/3899228499 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fdbf40037a0 con 0x7fdc00075750 2024-09-21T07:12:15.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.426+0000 7fdbfe000640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdbe80b65f0 0x7fdbe80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:15.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.426+0000 7fdbfe000640 1 -- 172.21.15.27:0/3899228499 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fdbf40e7f20 con 0x7fdc00075750 2024-09-21T07:12:15.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.426+0000 7fdc04c00640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdbe80b65f0 0x7fdbe80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:15.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.427+0000 7fdc04c00640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdbe80b65f0 0x7fdbe80b8b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009230 tx=0x7fdbec000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:15.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.427+0000 7fdbfe000640 1 -- 172.21.15.27:0/3899228499 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdbf4056710 con 0x7fdc00075750 2024-09-21T07:12:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.634+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdc0010a3f0 con 0x7fdbe80b65f0 2024-09-21T07:12:15.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.647+0000 7fdbfe000640 1 -- 172.21.15.27:0/3899228499 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fdc0010a3f0 con 0x7fdbe80b65f0 2024-09-21T07:12:15.649 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T07:12:15.649 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (2m) 18s ago 2m 80.7M - 19.3.0-5120-g18432218 ddce307e1f32 51d98d36c1fd 2024-09-21T07:12:15.649 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi119 running (2m) 20s ago 2m 96.5M - 19.3.0-5120-g18432218 ddce307e1f32 88eb88a42e9a 2024-09-21T07:12:15.649 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi153 running (2m) 20s ago 2m 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 bc8dcfc1d30f 2024-09-21T07:12:15.649 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (2m) 18s ago 2m 83.0M - 19.3.0-5120-g18432218 ddce307e1f32 ac72738df547 2024-09-21T07:12:15.649 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi119 running (2m) 20s ago 2m 85.1M - 19.3.0-5120-g18432218 ddce307e1f32 0e099bd5a490 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi153 running (2m) 20s ago 2m 82.0M - 19.3.0-5120-g18432218 ddce307e1f32 69145071ef92 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (2m) 18s ago 2m 85.4M - 19.3.0-5120-g18432218 ddce307e1f32 a58697b2e4a9 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi119 running (2m) 20s ago 2m 86.6M - 19.3.0-5120-g18432218 ddce307e1f32 36c9de9089c1 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi153 running (2m) 20s ago 2m 87.1M - 19.3.0-5120-g18432218 ddce307e1f32 407106754a06 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (2m) 18s ago 2m 80.8M - 19.3.0-5120-g18432218 ddce307e1f32 71965bfaefa8 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi119 running (3m) 20s ago 3m 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 ad65fc685cca 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi153 running (3m) 20s ago 3m 82.6M - 19.3.0-5120-g18432218 ddce307e1f32 226bdb383465 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (11m) 18s ago 11m 588M - 19.3.0-5120-g18432218 ddce307e1f32 56905a59dc48 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi119 *:8443,8765 running (9m) 20s ago 9m 526M - 19.3.0-5120-g18432218 ddce307e1f32 b4e81ec188d4 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi153 *:8443,8765 running (9m) 20s ago 9m 524M - 19.3.0-5120-g18432218 ddce307e1f32 f6eae6f2140a 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (11m) 18s ago 11m 117M 2048M 19.3.0-5120-g18432218 ddce307e1f32 cd5196f7b24b 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi119 running (9m) 20s ago 9m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 fcfb2e88df90 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi153 running (9m) 20s ago 9m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 51e0fe7fc2e4 2024-09-21T07:12:15.650 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 18s ago 8m 113M 4272M 19.3.0-5120-g18432218 ddce307e1f32 bd1cad9b3836 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi119 running (7m) 20s ago 7m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 f43cc6c1e219 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi153 running (7m) 20s ago 7m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ef9399ef0a23 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (6m) 18s ago 6m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 0cda1597c0d7 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi119 running (6m) 20s ago 6m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 f65d1eb11326 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi153 running (6m) 20s ago 6m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c88407a2a5aa 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (5m) 18s ago 5m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 8d4ae64416e3 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi119 running (5m) 20s ago 5m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 75a442b25dad 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi153 running (4m) 20s ago 4m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 61710cf454f4 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (4m) 18s ago 4m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 517a63f36d0b 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi119 running (3m) 20s ago 3m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5e6172f192af 2024-09-21T07:12:15.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi153 running (3m) 20s ago 3m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 7b389e6c17dd 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdbe80b65f0 msgr2=0x7fdbe80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdbe80b65f0 0x7fdbe80b8b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009230 tx=0x7fdbec000bc0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 msgr2=0x7fdc0019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 0x7fdc0019c6c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400ea60 tx=0x7fdbf400ef20 comp rx=0 tx=0).stop 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 shutdown_connections 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc000760a0 0x7fdc0019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdbe80b65f0 0x7fdbe80b8b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc00110140 0x7fdc001a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 --2- 172.21.15.27:0/3899228499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc00075750 0x7fdc0019c6c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 >> 172.21.15.27:0/3899228499 conn(0x7fdc000ff5d0 msgr2=0x7fdc00109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 shutdown_connections 2024-09-21T07:12:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:15.649+0000 7fdc08000640 1 -- 172.21.15.27:0/3899228499 wait complete. 2024-09-21T07:12:16.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T07:12:16.821 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:16 smithi153 ceph-mon[34747]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:16 smithi153 ceph-mon[34747]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:17.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:16 smithi027 ceph-mon[30779]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:17.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:16 smithi027 ceph-mon[30779]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:16 smithi119 ceph-mon[34519]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:16 smithi119 ceph-mon[34519]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:17.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 -- 172.21.15.27:0/4270511139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c105db0 msgr2=0x7fb72c1061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 --2- 172.21.15.27:0/4270511139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c105db0 0x7fb72c1061c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009900 tx=0x7fb71c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 -- 172.21.15.27:0/4270511139 shutdown_connections 2024-09-21T07:12:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 --2- 172.21.15.27:0/4270511139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c10b6f0 0x7fb72c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 --2- 172.21.15.27:0/4270511139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb72c10c040 0x7fb72c112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 --2- 172.21.15.27:0/4270511139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c105db0 0x7fb72c1061c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 -- 172.21.15.27:0/4270511139 >> 172.21.15.27:0/4270511139 conn(0x7fb72c1017e0 msgr2=0x7fb72c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:17.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 -- 172.21.15.27:0/4270511139 shutdown_connections 2024-09-21T07:12:17.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.516+0000 7fb732000640 1 -- 172.21.15.27:0/4270511139 wait complete. 2024-09-21T07:12:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.517+0000 7fb732000640 1 Processor -- start 2024-09-21T07:12:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.517+0000 7fb732000640 1 -- start start 2024-09-21T07:12:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb732000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c105db0 0x7fb72c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb732000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c10b6f0 0x7fb72c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb732000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb72c10c040 0x7fb72c1a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb732000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c115d50 con 0x7fb72c10b6f0 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb732000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c115bd0 con 0x7fb72c10c040 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb732000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c115ed0 con 0x7fb72c105db0 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb72b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c105db0 0x7fb72c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb72b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c105db0 0x7fb72c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38366/0 (socket says 172.21.15.27:38366) 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb72b400640 1 -- 172.21.15.27:0/4018838763 learned_addr learned my addr 172.21.15.27:0/4018838763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb72aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c10b6f0 0x7fb72c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.518+0000 7fb72be00640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb72c10c040 0x7fb72c1a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.519+0000 7fb72b400640 1 -- 172.21.15.27:0/4018838763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb72c10c040 msgr2=0x7fb72c1a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.519+0000 7fb72b400640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb72c10c040 0x7fb72c1a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.519+0000 7fb72b400640 1 -- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c10b6f0 msgr2=0x7fb72c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.519+0000 7fb72b400640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c10b6f0 0x7fb72c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.519+0000 7fb72b400640 1 -- 172.21.15.27:0/4018838763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1a2730 con 0x7fb72c105db0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.519+0000 7fb72be00640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb72c10c040 0x7fb72c1a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.519+0000 7fb72b400640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c105db0 0x7fb72c19c7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005ab0 tx=0x7fb71c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.520+0000 7fb723e00640 1 -- 172.21.15.27:0/4018838763 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c03f070 con 0x7fb72c105db0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.520+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a29c0 con 0x7fb72c105db0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.520+0000 7fb723e00640 1 -- 172.21.15.27:0/4018838763 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c031b60 con 0x7fb72c105db0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.520+0000 7fb723e00640 1 -- 172.21.15.27:0/4018838763 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c035030 con 0x7fb72c105db0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.520+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a9180 con 0x7fb72c105db0 2024-09-21T07:12:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.521+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c1061c0 con 0x7fb72c105db0 2024-09-21T07:12:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.524+0000 7fb723e00640 1 -- 172.21.15.27:0/4018838763 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fb71c042070 con 0x7fb72c105db0 2024-09-21T07:12:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.525+0000 7fb723e00640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7100b6670 0x7fb7100b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.525+0000 7fb723e00640 1 -- 172.21.15.27:0/4018838763 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb71c1088a0 con 0x7fb72c105db0 2024-09-21T07:12:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.525+0000 7fb72aa00640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7100b6670 0x7fb7100b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.526+0000 7fb72aa00640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7100b6670 0x7fb7100b8b80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb72c104ee0 tx=0x7fb718005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.527+0000 7fb723e00640 1 -- 172.21.15.27:0/4018838763 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb71c0ceb00 con 0x7fb72c105db0 2024-09-21T07:12:17.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.732+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb72c063800 con 0x7fb7100b6670 2024-09-21T07:12:17.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.738+0000 7fb723e00640 1 -- 172.21.15.27:0/4018838763 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb72c063800 con 0x7fb7100b6670 2024-09-21T07:12:17.740 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T07:12:17.740 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 22s ago 3m smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi153=c;smithi153=f;smithi153=i;smithi153=l;count:12 2024-09-21T07:12:17.740 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 22s ago 9m smithi027=x;smithi119=y;smithi153=z;count:3 2024-09-21T07:12:17.740 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 22s ago 9m smithi027:172.21.15.27=a;smithi119:172.21.15.119=b;smithi153:172.21.15.153=c;count:3 2024-09-21T07:12:17.740 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 22s ago - 2024-09-21T07:12:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7100b6670 msgr2=0x7fb7100b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7100b6670 0x7fb7100b8b80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb72c104ee0 tx=0x7fb718005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c105db0 msgr2=0x7fb72c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c105db0 0x7fb72c19c7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005ab0 tx=0x7fb71c003c10 comp rx=0 tx=0).stop 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 shutdown_connections 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7100b6670 0x7fb7100b8b80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb72c10c040 0x7fb72c1a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72c10b6f0 0x7fb72c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 --2- 172.21.15.27:0/4018838763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb72c105db0 0x7fb72c19c7c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.740+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 >> 172.21.15.27:0/4018838763 conn(0x7fb72c1017e0 msgr2=0x7fb72c10ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.741+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 shutdown_connections 2024-09-21T07:12:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:17.741+0000 7fb732000640 1 -- 172.21.15.27:0/4018838763 wait complete. 2024-09-21T07:12:18.470 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T07:12:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:19.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:18 smithi027 ceph-mon[30779]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:12:19.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:18 smithi027 ceph-mon[30779]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:18 smithi153 ceph-mon[34747]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:12:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:18 smithi153 ceph-mon[34747]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:18 smithi119 ceph-mon[34519]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T07:12:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:18 smithi119 ceph-mon[34519]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:19.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.496+0000 7f8059200640 1 -- 172.21.15.27:0/2458225492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 msgr2=0x7f8054106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:19.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.496+0000 7f8059200640 1 --2- 172.21.15.27:0/2458225492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f8054106190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8044009900 tx=0x7f804402a410 comp rx=0 tx=0).stop 2024-09-21T07:12:19.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.497+0000 7f8059200640 1 -- 172.21.15.27:0/2458225492 shutdown_connections 2024-09-21T07:12:19.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.497+0000 7f8059200640 1 --2- 172.21.15.27:0/2458225492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f805410c010 0x7f8054112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.497+0000 7f8059200640 1 --2- 172.21.15.27:0/2458225492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f805410b6c0 0x7f805410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.497+0000 7f8059200640 1 --2- 172.21.15.27:0/2458225492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f8054106190 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.497+0000 7f8059200640 1 -- 172.21.15.27:0/2458225492 >> 172.21.15.27:0/2458225492 conn(0x7f8054101810 msgr2=0x7f8054103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:19.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.497+0000 7f8059200640 1 -- 172.21.15.27:0/2458225492 shutdown_connections 2024-09-21T07:12:19.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.497+0000 7f8059200640 1 -- 172.21.15.27:0/2458225492 wait complete. 2024-09-21T07:12:19.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.498+0000 7f8059200640 1 Processor -- start 2024-09-21T07:12:19.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.498+0000 7f8059200640 1 -- start start 2024-09-21T07:12:19.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8059200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f805419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:19.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8059200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f805410b6c0 0x7f805419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:19.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8059200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f805410c010 0x7f80541a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:19.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8059200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8054115cd0 con 0x7f8054105d80 2024-09-21T07:12:19.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8059200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8054115b50 con 0x7f805410c010 2024-09-21T07:12:19.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8059200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8054115e50 con 0x7f805410b6c0 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8051e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f805419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8051e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f805419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40500/0 (socket says 172.21.15.27:40500) 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8051e00640 1 -- 172.21.15.27:0/1286440491 learned_addr learned my addr 172.21.15.27:0/1286440491 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f804be00640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f805410b6c0 0x7f805419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8052800640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f805410c010 0x7f80541a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8051e00640 1 -- 172.21.15.27:0/1286440491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f805410b6c0 msgr2=0x7f805419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8051e00640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f805410b6c0 0x7f805419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8051e00640 1 -- 172.21.15.27:0/1286440491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f805410c010 msgr2=0x7f80541a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:19.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.499+0000 7f8051e00640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f805410c010 0x7f80541a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.500+0000 7f8051e00640 1 -- 172.21.15.27:0/1286440491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80541a2610 con 0x7f8054105d80 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.500+0000 7f8051e00640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f805419c6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8044005940 tx=0x7f80440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.500+0000 7f804aa00640 1 -- 172.21.15.27:0/1286440491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804403f030 con 0x7f8054105d80 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.500+0000 7f804aa00640 1 -- 172.21.15.27:0/1286440491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8044003cb0 con 0x7f8054105d80 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.500+0000 7f804aa00640 1 -- 172.21.15.27:0/1286440491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044035070 con 0x7f8054105d80 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.500+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80541a28a0 con 0x7f8054105d80 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.500+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80541a9060 con 0x7f8054105d80 2024-09-21T07:12:19.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.501+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8054107000 con 0x7f8054105d80 2024-09-21T07:12:19.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.504+0000 7f804aa00640 1 -- 172.21.15.27:0/1286440491 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8044042030 con 0x7f8054105d80 2024-09-21T07:12:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.505+0000 7f804aa00640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80380b6670 0x7f80380b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.506+0000 7f804aa00640 1 -- 172.21.15.27:0/1286440491 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f80441083b0 con 0x7f8054105d80 2024-09-21T07:12:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.506+0000 7f804be00640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80380b6670 0x7f80380b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.506+0000 7f804be00640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80380b6670 0x7f80380b8b80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f805410b1e0 tx=0x7f803c00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.508+0000 7f804aa00640 1 -- 172.21.15.27:0/1286440491 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80440d29f0 con 0x7f8054105d80 2024-09-21T07:12:19.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.714+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8054063800 con 0x7f80380b6670 2024-09-21T07:12:19.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.716+0000 7f804aa00640 1 -- 172.21.15.27:0/1286440491 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8054063800 con 0x7f80380b6670 2024-09-21T07:12:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2024-09-21T07:12:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2024-09-21T07:12:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:smithi119 172.21.15.119 2024-09-21T07:12:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:smithi153 172.21.15.153 2024-09-21T07:12:19.718 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2024-09-21T07:12:19.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80380b6670 msgr2=0x7f80380b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:19.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80380b6670 0x7f80380b8b80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f805410b1e0 tx=0x7f803c00b410 comp rx=0 tx=0).stop 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 msgr2=0x7f805419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f805419c6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8044005940 tx=0x7f80440042f0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 shutdown_connections 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f805410b6c0 0x7f805419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80380b6670 0x7f80380b8b80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f805410c010 0x7f80541a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 --2- 172.21.15.27:0/1286440491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8054105d80 0x7f805419c6a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.718+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 >> 172.21.15.27:0/1286440491 conn(0x7f8054101810 msgr2=0x7f8054103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.719+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 shutdown_connections 2024-09-21T07:12:19.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:19.719+0000 7f8059200640 1 -- 172.21.15.27:0/1286440491 wait complete. 2024-09-21T07:12:20.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T07:12:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:21.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:20 smithi027 ceph-mon[30779]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:21.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:20 smithi027 ceph-mon[30779]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:20 smithi153 ceph-mon[34747]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:20 smithi153 ceph-mon[34747]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:20 smithi119 ceph-mon[34519]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T07:12:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:20 smithi119 ceph-mon[34519]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.436+0000 7feda8e00640 1 -- 172.21.15.27:0/1114610116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 msgr2=0x7feda4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.436+0000 7feda8e00640 1 --2- 172.21.15.27:0/1114610116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 0x7feda4110470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009ca0 tx=0x7fed9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.436+0000 7feda8e00640 1 -- 172.21.15.27:0/1114610116 shutdown_connections 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.436+0000 7feda8e00640 1 --2- 172.21.15.27:0/1114610116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 0x7feda4110470 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.436+0000 7feda8e00640 1 --2- 172.21.15.27:0/1114610116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41092f0 0x7feda4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.436+0000 7feda8e00640 1 --2- 172.21.15.27:0/1114610116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41039b0 0x7feda4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.436+0000 7feda8e00640 1 -- 172.21.15.27:0/1114610116 >> 172.21.15.27:0/1114610116 conn(0x7feda40ff440 msgr2=0x7feda4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.437+0000 7feda8e00640 1 -- 172.21.15.27:0/1114610116 shutdown_connections 2024-09-21T07:12:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.437+0000 7feda8e00640 1 -- 172.21.15.27:0/1114610116 wait complete. 2024-09-21T07:12:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 Processor -- start 2024-09-21T07:12:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 -- start start 2024-09-21T07:12:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41039b0 0x7feda419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41092f0 0x7feda419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 0x7feda419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feda4113900 con 0x7feda41092f0 2024-09-21T07:12:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feda4113780 con 0x7feda41039b0 2024-09-21T07:12:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.438+0000 7feda8e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feda4113a80 con 0x7feda4109c40 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7fed9be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41092f0 0x7feda419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41039b0 0x7feda419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41039b0 0x7feda419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44366/0 (socket says 172.21.15.27:44366) 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7fed9be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41092f0 0x7feda419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40518/0 (socket says 172.21.15.27:40518) 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 -- 172.21.15.27:0/1210336424 learned_addr learned my addr 172.21.15.27:0/1210336424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3e00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 0x7feda419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 -- 172.21.15.27:0/1210336424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 msgr2=0x7feda419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 0x7feda419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 -- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41092f0 msgr2=0x7feda419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41092f0 0x7feda419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.439+0000 7feda3400640 1 -- 172.21.15.27:0/1210336424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda41a0260 con 0x7feda41039b0 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.440+0000 7feda3400640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41039b0 0x7feda419a2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed9400ea60 tx=0x7fed9400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.440+0000 7feda0c00640 1 -- 172.21.15.27:0/1210336424 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9400cde0 con 0x7feda41039b0 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.440+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda41a0550 con 0x7feda41039b0 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.440+0000 7feda0c00640 1 -- 172.21.15.27:0/1210336424 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed940035f0 con 0x7feda41039b0 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.440+0000 7feda0c00640 1 -- 172.21.15.27:0/1210336424 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94010690 con 0x7feda41039b0 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.440+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda41a6cb0 con 0x7feda41039b0 2024-09-21T07:12:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.441+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda4103dc0 con 0x7feda41039b0 2024-09-21T07:12:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.444+0000 7feda0c00640 1 -- 172.21.15.27:0/1210336424 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fed940037a0 con 0x7feda41039b0 2024-09-21T07:12:21.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.445+0000 7feda0c00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fed880b6670 0x7fed880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:21.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.445+0000 7feda0c00640 1 -- 172.21.15.27:0/1210336424 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fed940e7e10 con 0x7feda41039b0 2024-09-21T07:12:21.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.445+0000 7fed9be00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fed880b6670 0x7fed880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:21.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.446+0000 7fed9be00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fed880b6670 0x7fed880b8b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feda4108d60 tx=0x7fed8c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:21.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.447+0000 7feda0c00640 1 -- 172.21.15.27:0/1210336424 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed94056710 con 0x7feda41039b0 2024-09-21T07:12:21.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.656+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7feda4063800 con 0x7fed880b6670 2024-09-21T07:12:21.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.658+0000 7feda0c00640 1 -- 172.21.15.27:0/1210336424 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7feda4063800 con 0x7fed880b6670 2024-09-21T07:12:21.660 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T07:12:21.660 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-21T07:12:21.660 INFO:teuthology.orchestra.run.smithi027.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T07:12:21.660 INFO:teuthology.orchestra.run.smithi027.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T07:12:21.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fed880b6670 msgr2=0x7fed880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:21.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fed880b6670 0x7fed880b8b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feda4108d60 tx=0x7fed8c005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:21.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41039b0 msgr2=0x7feda419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:21.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41039b0 0x7feda419a2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed9400ea60 tx=0x7fed9400ef20 comp rx=0 tx=0).stop 2024-09-21T07:12:21.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 shutdown_connections 2024-09-21T07:12:21.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feda4109c40 0x7feda419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fed880b6670 0x7fed880b8b80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41092f0 0x7feda419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 --2- 172.21.15.27:0/1210336424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feda41039b0 0x7feda419a2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:21.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 >> 172.21.15.27:0/1210336424 conn(0x7feda40ff440 msgr2=0x7feda4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:21.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 shutdown_connections 2024-09-21T07:12:21.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:21.660+0000 7feda8e00640 1 -- 172.21.15.27:0/1210336424 wait complete. 2024-09-21T07:12:22.402 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T07:12:22.414 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:12:22.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T07:12:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:23.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:22 smithi027 ceph-mon[30779]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:23.151 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:22 smithi027 ceph-mon[30779]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:22 smithi153 ceph-mon[34747]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:22 smithi153 ceph-mon[34747]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:22 smithi119 ceph-mon[34519]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:22 smithi119 ceph-mon[34519]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.450+0000 7f88a0a00640 1 -- 172.21.15.27:0/3328616171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c077940 msgr2=0x7f889c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.450+0000 7f88a0a00640 1 --2- 172.21.15.27:0/3328616171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c077940 0x7f889c077d50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f888c009900 tx=0x7f888c02a410 comp rx=0 tx=0).stop 2024-09-21T07:12:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.450+0000 7f88a0a00640 1 -- 172.21.15.27:0/3328616171 shutdown_connections 2024-09-21T07:12:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.450+0000 7f88a0a00640 1 --2- 172.21.15.27:0/3328616171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c078290 0x7f889c114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.450+0000 7f88a0a00640 1 --2- 172.21.15.27:0/3328616171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c114570 0x7f889c116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.450+0000 7f88a0a00640 1 --2- 172.21.15.27:0/3328616171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c077940 0x7f889c077d50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.450+0000 7f88a0a00640 1 -- 172.21.15.27:0/3328616171 >> 172.21.15.27:0/3328616171 conn(0x7f889c101680 msgr2=0x7f889c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.451+0000 7f88a0a00640 1 -- 172.21.15.27:0/3328616171 shutdown_connections 2024-09-21T07:12:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.451+0000 7f88a0a00640 1 -- 172.21.15.27:0/3328616171 wait complete. 2024-09-21T07:12:23.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 Processor -- start 2024-09-21T07:12:23.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 -- start start 2024-09-21T07:12:23.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c077940 0x7f889c1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:23.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c078290 0x7f889c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:23.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c114570 0x7f889c1a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f889c119ef0 con 0x7f889c114570 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f889c119d70 con 0x7f889c078290 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.452+0000 7f88a0a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f889c11a070 con 0x7f889c077940 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f889a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c077940 0x7f889c1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f889b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c114570 0x7f889c1a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f889a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c077940 0x7f889c1a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52598/0 (socket says 172.21.15.27:52598) 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f8893e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c078290 0x7f889c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f889a800640 1 -- 172.21.15.27:0/1709396126 learned_addr learned my addr 172.21.15.27:0/1709396126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f8893e00640 1 -- 172.21.15.27:0/1709396126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c077940 msgr2=0x7f889c1a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:23.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f8893e00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c077940 0x7f889c1a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f8893e00640 1 -- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c114570 msgr2=0x7f889c1a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:23.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f8893e00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c114570 0x7f889c1a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f8893e00640 1 -- 172.21.15.27:0/1709396126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c1a6970 con 0x7f889c078290 2024-09-21T07:12:23.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f889a800640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c077940 0x7f889c1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:23.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.453+0000 7f8893e00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c078290 0x7f889c1a0fa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f889c06c630 tx=0x7f888400daf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:23.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.454+0000 7f8893400640 1 -- 172.21.15.27:0/1709396126 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884017070 con 0x7f889c078290 2024-09-21T07:12:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.454+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c1a6c60 con 0x7f889c078290 2024-09-21T07:12:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.454+0000 7f8893400640 1 -- 172.21.15.27:0/1709396126 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88840054d0 con 0x7f889c078290 2024-09-21T07:12:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.454+0000 7f8893400640 1 -- 172.21.15.27:0/1709396126 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840096f0 con 0x7f889c078290 2024-09-21T07:12:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.454+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c1ad3c0 con 0x7f889c078290 2024-09-21T07:12:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.455+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c077d50 con 0x7f889c078290 2024-09-21T07:12:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.458+0000 7f8893400640 1 -- 172.21.15.27:0/1709396126 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8884005020 con 0x7f889c078290 2024-09-21T07:12:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.459+0000 7f8893400640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88800b65f0 0x7f88800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.459+0000 7f8893400640 1 -- 172.21.15.27:0/1709396126 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f88840e7710 con 0x7f889c078290 2024-09-21T07:12:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.459+0000 7f889a800640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88800b65f0 0x7f88800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.460+0000 7f889a800640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88800b65f0 0x7f88800b8b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f888c031440 tx=0x7f888c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:23.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.461+0000 7f8893400640 1 -- 172.21.15.27:0/1709396126 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8884020020 con 0x7f889c078290 2024-09-21T07:12:23.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.700+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f889c1a1b30 con 0x7f889c078290 2024-09-21T07:12:23.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.701+0000 7f8893400640 1 -- 172.21.15.27:0/1709396126 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3096 (secure 0 0 0) 0x7f889c1a1b30 con 0x7f889c078290 2024-09-21T07:12:23.704 INFO:teuthology.orchestra.run.smithi027.stdout:e26 2024-09-21T07:12:23.704 INFO:teuthology.orchestra.run.smithi027.stdout:btime 2024-09-21T07:10:53:642840+0000 2024-09-21T07:12:23.704 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T07:12:23.704 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T07:12:23.704 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2024-09-21T07:12:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:23.704 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 26 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-21T07:09:56.473755+0000 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-21T07:10:52.789806+0000 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2024-09-21T07:12:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 3 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24773,1=24748,2=24740} 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3,4] 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2024-09-21T07:12:23.706 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:qdb_cluster leader: 24773 members: 24773,24748,24740 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:24773} state up:active seq 5 addr [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{0:24736} state up:standby-replay seq 1 addr [v2:172.21.15.153:6838/2081315981,v1:172.21.15.153:6839/2081315981] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{1:24748} state up:active seq 19 addr [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{1:24721} state up:standby-replay seq 1 addr [v2:172.21.15.153:6836/1555665457,v1:172.21.15.153:6837/1555665457] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{2:24740} state up:active seq 21 addr [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{2:24713} state up:standby-replay seq 1 addr [v2:172.21.15.119:6834/3894045960,v1:172.21.15.119:6835/3894045960] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{-1:14733} state up:standby seq 1 addr [v2:172.21.15.27:6840/2637528669,v1:172.21.15.27:6841/2637528669] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{-1:24667} state up:standby seq 1 addr [v2:172.21.15.153:6832/4121519709,v1:172.21.15.153:6833/4121519709] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{-1:24677} state up:standby seq 1 addr [v2:172.21.15.119:6832/1660738486,v1:172.21.15.119:6833/1660738486] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{-1:24689} state up:standby seq 1 addr [v2:172.21.15.27:6834/2994040514,v1:172.21.15.27:6835/2994040514] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.708 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{-1:24701} state up:standby seq 1 addr [v2:172.21.15.27:6836/2900670620,v1:172.21.15.27:6837/2900670620] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.708 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{-1:24709} state up:standby seq 1 addr [v2:172.21.15.153:6834/717203459,v1:172.21.15.153:6835/717203459] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T07:12:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T07:12:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.703+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88800b65f0 msgr2=0x7f88800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.703+0000 7f88a0a00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88800b65f0 0x7f88800b8b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f888c031440 tx=0x7f888c0058d0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.703+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c078290 msgr2=0x7f889c1a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.703+0000 7f88a0a00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c078290 0x7f889c1a0fa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f889c06c630 tx=0x7f888400daf0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 shutdown_connections 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88800b65f0 0x7f88800b8b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c114570 0x7f889c1a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c078290 0x7f889c1a0fa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 --2- 172.21.15.27:0/1709396126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f889c077940 0x7f889c1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 >> 172.21.15.27:0/1709396126 conn(0x7f889c101680 msgr2=0x7f889c116670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 shutdown_connections 2024-09-21T07:12:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:23.704+0000 7f88a0a00640 1 -- 172.21.15.27:0/1709396126 wait complete. 2024-09-21T07:12:24.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:23 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1709396126' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T07:12:24.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:23 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1709396126' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T07:12:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:23 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1709396126' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T07:12:24.572 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T07:12:24.959 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:24 smithi153 ceph-mon[34747]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:25.214 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:24 smithi027 ceph-mon[30779]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:24 smithi119 ceph-mon[34519]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 -- 172.21.15.27:0/1232803491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c10c020 msgr2=0x7f978c112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 --2- 172.21.15.27:0/1232803491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c10c020 0x7f978c112850 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9784009ca0 tx=0x7f978402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 -- 172.21.15.27:0/1232803491 shutdown_connections 2024-09-21T07:12:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 --2- 172.21.15.27:0/1232803491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c10b6d0 0x7f978c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 --2- 172.21.15.27:0/1232803491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c10c020 0x7f978c112850 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 --2- 172.21.15.27:0/1232803491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978c105d90 0x7f978c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 -- 172.21.15.27:0/1232803491 >> 172.21.15.27:0/1232803491 conn(0x7f978c101860 msgr2=0x7f978c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:25.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.567+0000 7f9790a00640 1 -- 172.21.15.27:0/1232803491 shutdown_connections 2024-09-21T07:12:25.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.568+0000 7f9790a00640 1 -- 172.21.15.27:0/1232803491 wait complete. 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.568+0000 7f9790a00640 1 Processor -- start 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9790a00640 1 -- start start 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9790a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c105d90 0x7f978c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9790a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c10b6d0 0x7f978c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9790a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978c10c020 0x7f978c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9790a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f978c115ce0 con 0x7f978c105d90 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9790a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f978c115b60 con 0x7f978c10c020 2024-09-21T07:12:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9790a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f978c115e60 con 0x7f978c10b6d0 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9789e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c105d90 0x7f978c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9789e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c105d90 0x7f978c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40546/0 (socket says 172.21.15.27:40546) 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.569+0000 7f9789e00640 1 -- 172.21.15.27:0/1044932932 learned_addr learned my addr 172.21.15.27:0/1044932932 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f9783e00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c10b6d0 0x7f978c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f978a800640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978c10c020 0x7f978c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f9789e00640 1 -- 172.21.15.27:0/1044932932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c10b6d0 msgr2=0x7f978c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f9789e00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c10b6d0 0x7f978c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f9789e00640 1 -- 172.21.15.27:0/1044932932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978c10c020 msgr2=0x7f978c1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f9789e00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978c10c020 0x7f978c1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f9789e00640 1 -- 172.21.15.27:0/1044932932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c1a25c0 con 0x7f978c105d90 2024-09-21T07:12:25.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.570+0000 7f9789e00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c105d90 0x7f978c19c650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f977c00e9f0 tx=0x7f977c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:25.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.571+0000 7f9782a00640 1 -- 172.21.15.27:0/1044932932 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c00cd80 con 0x7f978c105d90 2024-09-21T07:12:25.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.571+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c1a28b0 con 0x7f978c105d90 2024-09-21T07:12:25.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.571+0000 7f9782a00640 1 -- 172.21.15.27:0/1044932932 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f977c0108e0 con 0x7f978c105d90 2024-09-21T07:12:25.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.571+0000 7f9782a00640 1 -- 172.21.15.27:0/1044932932 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c018980 con 0x7f978c105d90 2024-09-21T07:12:25.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.571+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c1a9010 con 0x7f978c105d90 2024-09-21T07:12:25.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.572+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c1061a0 con 0x7f978c105d90 2024-09-21T07:12:25.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.574+0000 7f9782a00640 1 -- 172.21.15.27:0/1044932932 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f977c010430 con 0x7f978c105d90 2024-09-21T07:12:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.577+0000 7f9782a00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97700b6670 0x7f97700b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.577+0000 7f9782a00640 1 -- 172.21.15.27:0/1044932932 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f977c0e7c80 con 0x7f978c105d90 2024-09-21T07:12:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.577+0000 7f9783e00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97700b6670 0x7f97700b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:25.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.578+0000 7f9783e00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97700b6670 0x7f97700b8b80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f978c1050b0 tx=0x7f9774005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:25.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.581+0000 7f9782a00640 1 -- 172.21.15.27:0/1044932932 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f977c0b22c0 con 0x7f978c105d90 2024-09-21T07:12:25.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.785+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f978c02e640 con 0x7f978c105d90 2024-09-21T07:12:25.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.787+0000 7f9782a00640 1 -- 172.21.15.27:0/1044932932 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) ==== 56+0+6043 (secure 0 0 0) 0x7f977c0b7200 con 0x7f978c105d90 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 86 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-21T07:00:45.001187+0000 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-21T07:11:55.826198+0000 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 27 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-09-21T07:12:25.790 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.69 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 9 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/1967162431,v1:172.21.15.27:6803/1967162431] [v2:172.21.15.27:6804/1967162431,v1:172.21.15.27:6805/1967162431] exists,up d999c06a-d17f-4840-9bec-67d0325b7546 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 14 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/3407674054,v1:172.21.15.119:6801/3407674054] [v2:172.21.15.119:6802/3407674054,v1:172.21.15.119:6803/3407674054] exists,up 4063fc7c-3ba2-4b37-b188-45faf89d4127 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6800/206718354,v1:172.21.15.153:6801/206718354] [v2:172.21.15.153:6802/206718354,v1:172.21.15.153:6803/206718354] exists,up f871d34f-c35e-4ce0-9c20-09911568fc95 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/984637291,v1:172.21.15.27:6811/984637291] [v2:172.21.15.27:6812/984637291,v1:172.21.15.27:6813/984637291] exists,up db7d7254-c55f-4b12-8c53-9292a254ffcb 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 30 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/2014804508,v1:172.21.15.119:6809/2014804508] [v2:172.21.15.119:6810/2014804508,v1:172.21.15.119:6811/2014804508] exists,up fa23aaaf-55c1-4c48-bf7b-c2c2039624ab 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 35 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6808/2063779127,v1:172.21.15.153:6809/2063779127] [v2:172.21.15.153:6810/2063779127,v1:172.21.15.153:6811/2063779127] exists,up 2ab0114f-b98e-4b90-b112-aa1dec91830c 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 41 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/3534557792,v1:172.21.15.27:6819/3534557792] [v2:172.21.15.27:6820/3534557792,v1:172.21.15.27:6821/3534557792] exists,up d7bb5b9d-337b-41e4-ae7a-93db10aa151e 2024-09-21T07:12:25.791 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/1162667989,v1:172.21.15.119:6817/1162667989] [v2:172.21.15.119:6818/1162667989,v1:172.21.15.119:6819/1162667989] exists,up 2cb69c8e-353c-49e9-87f9-802de8728313 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 51 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6816/2174139924,v1:172.21.15.153:6817/2174139924] [v2:172.21.15.153:6818/2174139924,v1:172.21.15.153:6819/2174139924] exists,up 13844110-df7e-451b-b574-38419af2df5d 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 57 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/611699615,v1:172.21.15.27:6827/611699615] [v2:172.21.15.27:6828/611699615,v1:172.21.15.27:6829/611699615] exists,up 5c254d35-aa04-43e4-844e-5d6d2adb11ed 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 62 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/3847228474,v1:172.21.15.119:6825/3847228474] [v2:172.21.15.119:6826/3847228474,v1:172.21.15.119:6827/3847228474] exists,up 87219b10-05f8-49ae-8be3-3e7d79454a39 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 67 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6824/1280738577,v1:172.21.15.153:6825/1280738577] [v2:172.21.15.153:6826/1280738577,v1:172.21.15.153:6827/1280738577] exists,up 8ee8dc56-e77b-4969-8e32-4380b8b12b21 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.a [2,11] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.f [8,11] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.e [5,9] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.11 [4,8] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.21 [4,6] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.28 [5,7] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.29 [3,9] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.31 [4,8] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.34 [4,9] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.12 [9,10,5,1] 2024-09-21T07:12:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.1c [9,8] 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.1f [9,8] 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/785463945 expires 2024-09-22T07:02:53.267293+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2240730116 expires 2024-09-22T07:01:14.297512+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/2151540061 expires 2024-09-22T07:01:14.297512+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1830236521 expires 2024-09-22T07:01:14.297512+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/2513510737 expires 2024-09-22T07:01:45.325213+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/727966245 expires 2024-09-22T07:02:53.267293+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2811364463 expires 2024-09-22T07:01:14.297512+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/2513510737 expires 2024-09-22T07:01:45.325213+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/2151540061 expires 2024-09-22T07:01:14.297512+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3503043760 expires 2024-09-22T07:01:45.325213+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2989055970 expires 2024-09-22T07:01:45.325213+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1823929888 expires 2024-09-22T07:02:53.267293+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/727966245 expires 2024-09-22T07:02:53.267293+0000 2024-09-21T07:12:25.793 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/4193484002 expires 2024-09-22T07:01:45.325213+0000 2024-09-21T07:12:25.794 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/542737158 expires 2024-09-22T07:02:53.267293+0000 2024-09-21T07:12:25.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.789+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97700b6670 msgr2=0x7f97700b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:25.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.789+0000 7f9790a00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97700b6670 0x7f97700b8b80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f978c1050b0 tx=0x7f9774005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:25.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c105d90 msgr2=0x7f978c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c105d90 0x7f978c19c650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f977c00e9f0 tx=0x7f977c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 shutdown_connections 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f978c10b6d0 0x7f978c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97700b6670 0x7f97700b8b80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978c10c020 0x7f978c1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 --2- 172.21.15.27:0/1044932932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f978c105d90 0x7f978c19c650 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 >> 172.21.15.27:0/1044932932 conn(0x7f978c101860 msgr2=0x7f978c10ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 shutdown_connections 2024-09-21T07:12:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:25.790+0000 7f9790a00640 1 -- 172.21.15.27:0/1044932932 wait complete. 2024-09-21T07:12:26.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:25 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1044932932' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T07:12:26.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:25 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1044932932' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T07:12:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:25 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1044932932' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T07:12:26.481 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T07:12:26.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:12:26.878 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:26 smithi153 ceph-mon[34747]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T07:12:27.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:26 smithi153 ceph-mon[34747]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-09-21T07:12:27.191 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:26 smithi027 ceph-mon[30779]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T07:12:27.191 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:26 smithi027 ceph-mon[30779]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-09-21T07:12:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:26 smithi119 ceph-mon[34519]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T07:12:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:26 smithi119 ceph-mon[34519]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 -- 172.21.15.27:0/4245441041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac0757b0 msgr2=0x7f49ac075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 --2- 172.21.15.27:0/4245441041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac0757b0 0x7f49ac075bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f499c009900 tx=0x7f499c02a410 comp rx=0 tx=0).stop 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 -- 172.21.15.27:0/4245441041 shutdown_connections 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 --2- 172.21.15.27:0/4245441041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac076100 0x7f49ac10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 --2- 172.21.15.27:0/4245441041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac1100c0 0x7f49ac112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 --2- 172.21.15.27:0/4245441041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac0757b0 0x7f49ac075bc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 -- 172.21.15.27:0/4245441041 >> 172.21.15.27:0/4245441041 conn(0x7f49ac07b850 msgr2=0x7f49ac07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 -- 172.21.15.27:0/4245441041 shutdown_connections 2024-09-21T07:12:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.528+0000 7f49b2200640 1 -- 172.21.15.27:0/4245441041 wait complete. 2024-09-21T07:12:27.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.529+0000 7f49b2200640 1 Processor -- start 2024-09-21T07:12:27.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49b2200640 1 -- start start 2024-09-21T07:12:27.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49b2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac0757b0 0x7f49ac19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49b2200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac076100 0x7f49ac19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49b2200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac1100c0 0x7f49ac1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49b2200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac115c70 con 0x7f49ac076100 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49b2200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac115af0 con 0x7f49ac0757b0 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49b2200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac115df0 con 0x7f49ac1100c0 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49ab400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac0757b0 0x7f49ac19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac076100 0x7f49ac19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49ab400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac0757b0 0x7f49ac19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44424/0 (socket says 172.21.15.27:44424) 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49ab400640 1 -- 172.21.15.27:0/3484079588 learned_addr learned my addr 172.21.15.27:0/3484079588 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.530+0000 7f49abe00640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac1100c0 0x7f49ac1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.531+0000 7f49a3e00640 1 -- 172.21.15.27:0/3484079588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac1100c0 msgr2=0x7f49ac1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.531+0000 7f49a3e00640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac1100c0 0x7f49ac1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.531+0000 7f49a3e00640 1 -- 172.21.15.27:0/3484079588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac0757b0 msgr2=0x7f49ac19c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.531+0000 7f49a3e00640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac0757b0 0x7f49ac19c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.531+0000 7f49a3e00640 1 -- 172.21.15.27:0/3484079588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac1a2570 con 0x7f49ac076100 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.531+0000 7f49ab400640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac0757b0 0x7f49ac19c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.531+0000 7f49a3e00640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac076100 0x7f49ac19cba0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f499400d6a0 tx=0x7f499400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.532+0000 7f49a8c00640 1 -- 172.21.15.27:0/3484079588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994017070 con 0x7f49ac076100 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.532+0000 7f49a8c00640 1 -- 172.21.15.27:0/3484079588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4994009cd0 con 0x7f49ac076100 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.532+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac1a2860 con 0x7f49ac076100 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.532+0000 7f49a8c00640 1 -- 172.21.15.27:0/3484079588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994003b50 con 0x7f49ac076100 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.532+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac1a8fc0 con 0x7f49ac076100 2024-09-21T07:12:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.533+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac075bc0 con 0x7f49ac076100 2024-09-21T07:12:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.535+0000 7f49a8c00640 1 -- 172.21.15.27:0/3484079588 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f49940042b0 con 0x7f49ac076100 2024-09-21T07:12:27.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.537+0000 7f49a8c00640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f49900b6560 0x7f49900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:27.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.537+0000 7f49a8c00640 1 -- 172.21.15.27:0/3484079588 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f49940e71a0 con 0x7f49ac076100 2024-09-21T07:12:27.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.537+0000 7f49ab400640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f49900b6560 0x7f49900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:27.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.538+0000 7f49ab400640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f49900b6560 0x7f49900b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f499c031440 tx=0x7f499c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:27.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.541+0000 7f49a8c00640 1 -- 172.21.15.27:0/3484079588 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49940b1940 con 0x7f49ac076100 2024-09-21T07:12:27.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.779+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49ac19d730 con 0x7f49ac076100 2024-09-21T07:12:27.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.779+0000 7f49a8c00640 1 -- 172.21.15.27:0/3484079588 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7f49940b6880 con 0x7f49ac076100 2024-09-21T07:12:27.782 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:27.782 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T07:10:53:642840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:52.789806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24773,24748,24740]},"id":1}]} 2024-09-21T07:12:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T07:12:27.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f49900b6560 msgr2=0x7f49900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:27.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f49900b6560 0x7f49900b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f499c031440 tx=0x7f499c0058d0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac076100 msgr2=0x7f49ac19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac076100 0x7f49ac19cba0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f499400d6a0 tx=0x7f499400db60 comp rx=0 tx=0).stop 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 shutdown_connections 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac1100c0 0x7f49ac1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f49900b6560 0x7f49900b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49ac076100 0x7f49ac19cba0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 --2- 172.21.15.27:0/3484079588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac0757b0 0x7f49ac19c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.782+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 >> 172.21.15.27:0/3484079588 conn(0x7f49ac07b850 msgr2=0x7f49ac1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.783+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 shutdown_connections 2024-09-21T07:12:27.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:27.783+0000 7f49b2200640 1 -- 172.21.15.27:0/3484079588 wait complete. 2024-09-21T07:12:28.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:27 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3484079588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:28.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:27 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3484079588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:27 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3484079588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:28.431 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:12:28.816 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:28 smithi153 ceph-mon[34747]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:12:29.262 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:28 smithi027 ceph-mon[30779]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:12:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:28 smithi119 ceph-mon[34519]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:12:29.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 -- 172.21.15.27:0/1723559601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363810ffa0 msgr2=0x7f36381123c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:29.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 --2- 172.21.15.27:0/1723559601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363810ffa0 0x7f36381123c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3630009ca0 tx=0x7f363002a950 comp rx=0 tx=0).stop 2024-09-21T07:12:29.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 -- 172.21.15.27:0/1723559601 shutdown_connections 2024-09-21T07:12:29.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 --2- 172.21.15.27:0/1723559601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363810ffa0 0x7f36381123c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 --2- 172.21.15.27:0/1723559601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3638102d40 0x7f363810fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 --2- 172.21.15.27:0/1723559601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36381023f0 0x7f3638102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 -- 172.21.15.27:0/1723559601 >> 172.21.15.27:0/1723559601 conn(0x7f36380fe2d0 msgr2=0x7f3638100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:29.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 -- 172.21.15.27:0/1723559601 shutdown_connections 2024-09-21T07:12:29.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.445+0000 7f363da00640 1 -- 172.21.15.27:0/1723559601 wait complete. 2024-09-21T07:12:29.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.446+0000 7f363da00640 1 Processor -- start 2024-09-21T07:12:29.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.446+0000 7f363da00640 1 -- start start 2024-09-21T07:12:29.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f363da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36381023f0 0x7f363819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:29.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f363da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3638102d40 0x7f363819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f363da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f363810ffa0 0x7f36381a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f363da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3638115ae0 con 0x7f3638102d40 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f363da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3638115960 con 0x7f36381023f0 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f363da00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3638115c60 con 0x7f363810ffa0 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f362fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3638102d40 0x7f363819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f3637e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36381023f0 0x7f363819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f3637e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36381023f0 0x7f363819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48178/0 (socket says 172.21.15.27:48178) 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f362fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3638102d40 0x7f363819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46092/0 (socket says 172.21.15.27:46092) 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f3637e00640 1 -- 172.21.15.27:0/123423197 learned_addr learned my addr 172.21.15.27:0/123423197 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.447+0000 7f363d000640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f363810ffa0 0x7f36381a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3637e00640 1 -- 172.21.15.27:0/123423197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f363810ffa0 msgr2=0x7f36381a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3637e00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f363810ffa0 0x7f36381a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3637e00640 1 -- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3638102d40 msgr2=0x7f363819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3637e00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3638102d40 0x7f363819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3637e00640 1 -- 172.21.15.27:0/123423197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36381a23a0 con 0x7f36381023f0 2024-09-21T07:12:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3637e00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36381023f0 0x7f363819c430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f362800e9f0 tx=0x7f362800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3635600640 1 -- 172.21.15.27:0/123423197 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800cd80 con 0x7f36381023f0 2024-09-21T07:12:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36381a2690 con 0x7f36381023f0 2024-09-21T07:12:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3635600640 1 -- 172.21.15.27:0/123423197 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36280054d0 con 0x7f36381023f0 2024-09-21T07:12:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f3635600640 1 -- 172.21.15.27:0/123423197 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628010690 con 0x7f36381023f0 2024-09-21T07:12:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.448+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36381a8df0 con 0x7f36381023f0 2024-09-21T07:12:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.449+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3638102800 con 0x7f36381023f0 2024-09-21T07:12:29.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.452+0000 7f3635600640 1 -- 172.21.15.27:0/123423197 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f3628004e30 con 0x7f36381023f0 2024-09-21T07:12:29.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.454+0000 7f3635600640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f361c0b6560 0x7f361c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.454+0000 7f3635600640 1 -- 172.21.15.27:0/123423197 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f36280e7df0 con 0x7f36381023f0 2024-09-21T07:12:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.454+0000 7f362fe00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f361c0b6560 0x7f361c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.455+0000 7f362fe00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f361c0b6560 0x7f361c0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f36381113d0 tx=0x7f362000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:29.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.455+0000 7f3635600640 1 -- 172.21.15.27:0/123423197 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36280ae1b0 con 0x7f36381023f0 2024-09-21T07:12:29.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.696+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f363819d560 con 0x7f36381023f0 2024-09-21T07:12:29.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.698+0000 7f3635600640 1 -- 172.21.15.27:0/123423197 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7f363819d560 con 0x7f36381023f0 2024-09-21T07:12:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:29.700 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"btime":"2024-09-21T07:10:53:642840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:10:52.789806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24773,24748,24740]},"id":1}]} 2024-09-21T07:12:29.700 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-09-21T07:12:29.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f361c0b6560 msgr2=0x7f361c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:29.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f361c0b6560 0x7f361c0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f36381113d0 tx=0x7f362000b410 comp rx=0 tx=0).stop 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36381023f0 msgr2=0x7f363819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36381023f0 0x7f363819c430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f362800e9f0 tx=0x7f362800edf0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 shutdown_connections 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f363810ffa0 0x7f36381a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f361c0b6560 0x7f361c0b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3638102d40 0x7f363819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 --2- 172.21.15.27:0/123423197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36381023f0 0x7f363819c430 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.700+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 >> 172.21.15.27:0/123423197 conn(0x7f36380fe2d0 msgr2=0x7f3638111910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.701+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 shutdown_connections 2024-09-21T07:12:29.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:29.701+0000 7f363da00640 1 -- 172.21.15.27:0/123423197 wait complete. 2024-09-21T07:12:30.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:29 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/123423197' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:30.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:29 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/123423197' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:29 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/123423197' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:30.399 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:12:30.782 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:30 smithi153 ceph-mon[34747]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:12:31.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:30 smithi027 ceph-mon[30779]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:12:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:30 smithi119 ceph-mon[34519]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:12:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 -- 172.21.15.27:0/2828744489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 msgr2=0x7fa9fc10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 --2- 172.21.15.27:0/2828744489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc10bad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009900 tx=0x7fa9e402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 -- 172.21.15.27:0/2828744489 shutdown_connections 2024-09-21T07:12:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 --2- 172.21.15.27:0/2828744489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc10c010 0x7fa9fc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 --2- 172.21.15.27:0/2828744489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc10bad0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 --2- 172.21.15.27:0/2828744489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9fc105d80 0x7fa9fc106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 -- 172.21.15.27:0/2828744489 >> 172.21.15.27:0/2828744489 conn(0x7fa9fc101810 msgr2=0x7fa9fc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 -- 172.21.15.27:0/2828744489 shutdown_connections 2024-09-21T07:12:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.502+0000 7faa02200640 1 -- 172.21.15.27:0/2828744489 wait complete. 2024-09-21T07:12:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.503+0000 7faa02200640 1 Processor -- start 2024-09-21T07:12:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.503+0000 7faa02200640 1 -- start start 2024-09-21T07:12:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7faa02200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc105d80 0x7fa9fc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7faa02200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7faa02200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9fc10c010 0x7fa9fc1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7faa02200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc115d20 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7faa02200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc115ba0 con 0x7fa9fc105d80 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7faa02200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc115ea0 con 0x7fa9fc10c010 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7fa9f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7fa9f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46100/0 (socket says 172.21.15.27:46100) 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7fa9f3e00640 1 -- 172.21.15.27:0/944452830 learned_addr learned my addr 172.21.15.27:0/944452830 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7fa9fb400640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc105d80 0x7fa9fc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.504+0000 7fa9fbe00640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9fc10c010 0x7fa9fc1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f3e00640 1 -- 172.21.15.27:0/944452830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9fc10c010 msgr2=0x7fa9fc1a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f3e00640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9fc10c010 0x7fa9fc1a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f3e00640 1 -- 172.21.15.27:0/944452830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc105d80 msgr2=0x7fa9fc19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f3e00640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc105d80 0x7fa9fc19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f3e00640 1 -- 172.21.15.27:0/944452830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc1a2620 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9fb400640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc105d80 0x7fa9fc19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f3e00640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc19cc50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005ab0 tx=0x7fa9e4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f8c00640 1 -- 172.21.15.27:0/944452830 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e403f070 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f8c00640 1 -- 172.21.15.27:0/944452830 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e4031b50 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7fa9f8c00640 1 -- 172.21.15.27:0/944452830 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4035030 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.505+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc1a28b0 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.506+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc1a9070 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.507+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc106190 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.509+0000 7fa9f8c00640 1 -- 172.21.15.27:0/944452830 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fa9e4042070 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.511+0000 7fa9f8c00640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9e00b65e0 0x7fa9e00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.511+0000 7fa9f8c00640 1 -- 172.21.15.27:0/944452830 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa9e41081d0 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.511+0000 7fa9fb400640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9e00b65e0 0x7fa9e00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.512+0000 7fa9fb400640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9e00b65e0 0x7fa9e00b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00bfd0 tx=0x7fa9ec00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.513+0000 7fa9f8c00640 1 -- 172.21.15.27:0/944452830 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9e40d2970 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.715+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9fc10de00 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.717+0000 7fa9f8c00640 1 -- 172.21.15.27:0/944452830 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25234 (secure 0 0 0) 0x7fa9e40d78b0 con 0x7fa9fc10b6c0 2024-09-21T07:12:31.719 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:31.720 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:11:55.826198+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10},{"from":5,"to":1}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:12:31.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9e00b65e0 msgr2=0x7fa9e00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:31.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9e00b65e0 0x7fa9e00b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00bfd0 tx=0x7fa9ec00b390 comp rx=0 tx=0).stop 2024-09-21T07:12:31.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 msgr2=0x7fa9fc19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:31.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc19cc50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005ab0 tx=0x7fa9e4003b90 comp rx=0 tx=0).stop 2024-09-21T07:12:31.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 shutdown_connections 2024-09-21T07:12:31.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9fc10c010 0x7fa9fc1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa9e00b65e0 0x7fa9e00b8af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9fc10b6c0 0x7fa9fc19cc50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 --2- 172.21.15.27:0/944452830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9fc105d80 0x7fa9fc19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:31.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.719+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 >> 172.21.15.27:0/944452830 conn(0x7fa9fc101810 msgr2=0x7fa9fc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:31.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.720+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 shutdown_connections 2024-09-21T07:12:31.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:31.720+0000 7faa02200640 1 -- 172.21.15.27:0/944452830 wait complete. 2024-09-21T07:12:32.117 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:31 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/944452830' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:32.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:31 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/944452830' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:31 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/944452830' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:32.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T07:09:56.473755+0000', 'modified': '2024-09-21T07:10:52.789806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24773, 'mds_1': 24748, 'mds_2': 24740}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24713': {'gid': 24713, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/3894045960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 3894045960}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3894045960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24721': {'gid': 24721, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/1555665457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 1555665457}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 1555665457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24736': {'gid': 24736, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6839/2081315981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 2081315981}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 2081315981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24740': {'gid': 24740, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.119:6837/4207522386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 4207522386}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 4207522386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24748': {'gid': 24748, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.27:6839/609554839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 609554839}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 609554839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24773': {'gid': 24773, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.119:6839/1565967470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1565967470}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1565967470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24773, 'qdb_cluster': [24773, 24748, 24740]} 2024-09-21T07:12:32.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:12:32.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T07:12:32.883 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:32 smithi027 ceph-mon[30779]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:12:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:32 smithi153 ceph-mon[34747]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:12:33.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:32 smithi119 ceph-mon[34519]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:12:33.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.646+0000 7fa037200640 1 -- 172.21.15.27:0/331141650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa038105bc0 msgr2=0x7fa038105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.646+0000 7fa037200640 1 --2- 172.21.15.27:0/331141650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa038105bc0 0x7fa038105fd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa028009900 tx=0x7fa02802a410 comp rx=0 tx=0).stop 2024-09-21T07:12:33.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.647+0000 7fa037200640 1 -- 172.21.15.27:0/331141650 shutdown_connections 2024-09-21T07:12:33.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.647+0000 7fa037200640 1 --2- 172.21.15.27:0/331141650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa03810be50 0x7fa038112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.647+0000 7fa037200640 1 --2- 172.21.15.27:0/331141650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 0x7fa03810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.647+0000 7fa037200640 1 --2- 172.21.15.27:0/331141650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa038105bc0 0x7fa038105fd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.647+0000 7fa037200640 1 -- 172.21.15.27:0/331141650 >> 172.21.15.27:0/331141650 conn(0x7fa038101670 msgr2=0x7fa038103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.647+0000 7fa037200640 1 -- 172.21.15.27:0/331141650 shutdown_connections 2024-09-21T07:12:33.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.647+0000 7fa037200640 1 -- 172.21.15.27:0/331141650 wait complete. 2024-09-21T07:12:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 Processor -- start 2024-09-21T07:12:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 -- start start 2024-09-21T07:12:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa038105bc0 0x7fa03819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 0x7fa03819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa03810be50 0x7fa0381a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa038115b10 con 0x7fa03810be50 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa038115990 con 0x7fa03810b500 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.648+0000 7fa037200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa038115c90 con 0x7fa038105bc0 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 0x7fa03819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 0x7fa03819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48212/0 (socket says 172.21.15.27:48212) 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 -- 172.21.15.27:0/685886772 learned_addr learned my addr 172.21.15.27:0/685886772 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa036800640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa03810be50 0x7fa0381a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035e00640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa038105bc0 0x7fa03819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 -- 172.21.15.27:0/685886772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa038105bc0 msgr2=0x7fa03819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa038105bc0 0x7fa03819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 -- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa03810be50 msgr2=0x7fa0381a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa03810be50 0x7fa0381a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.649+0000 7fa035400640 1 -- 172.21.15.27:0/685886772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0381a2490 con 0x7fa03810b500 2024-09-21T07:12:33.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.650+0000 7fa035400640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 0x7fa03819cac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e9f0 tx=0x7fa02400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:33.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.650+0000 7fa02ea00640 1 -- 172.21.15.27:0/685886772 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02400ccc0 con 0x7fa03810b500 2024-09-21T07:12:33.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.650+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0381a2780 con 0x7fa03810b500 2024-09-21T07:12:33.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.650+0000 7fa02ea00640 1 -- 172.21.15.27:0/685886772 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0240108e0 con 0x7fa03810b500 2024-09-21T07:12:33.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.650+0000 7fa02ea00640 1 -- 172.21.15.27:0/685886772 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024018860 con 0x7fa03810b500 2024-09-21T07:12:33.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.650+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0381a8ee0 con 0x7fa03810b500 2024-09-21T07:12:33.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.653+0000 7fa0435c2740 1 Processor -- start 2024-09-21T07:12:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.653+0000 7fa0435c2740 1 -- start start 2024-09-21T07:12:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8f7f60 0x55e60fc9e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8cf0a0 0x55e60fc9ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fc9f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa0435c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e60fc15d50 con 0x55e60f8d7280 2024-09-21T07:12:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa0435c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e60fc05180 con 0x55e60f8cf0a0 2024-09-21T07:12:33.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa0435c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e60fc08f90 con 0x55e60f8f7f60 2024-09-21T07:12:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa02ea00640 1 -- 172.21.15.27:0/685886772 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fa024010430 con 0x7fa03810b500 2024-09-21T07:12:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa036800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8f7f60 0x55e60fc9e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa035e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8cf0a0 0x55e60fc9ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa036800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8f7f60 0x55e60fc9e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49752/0 (socket says 172.21.15.27:49752) 2024-09-21T07:12:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.654+0000 7fa036800640 1 -- 172.21.15.27:0/4043961645 learned_addr learned my addr 172.21.15.27:0/4043961645 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035400640 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fc9f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035e00640 1 -- 172.21.15.27:0/4043961645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8f7f60 msgr2=0x55e60fc9e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035e00640 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8f7f60 0x55e60fc9e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035e00640 1 -- 172.21.15.27:0/4043961645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 msgr2=0x55e60fc9f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035e00640 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fc9f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035e00640 1 -- 172.21.15.27:0/4043961645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60fbcfa70 con 0x55e60f8cf0a0 2024-09-21T07:12:33.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035400640 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fc9f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:12:33.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa035e00640 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8cf0a0 0x55e60fc9ebb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa028009230 tx=0x7fa028032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f72e50362675f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:33.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa023000640 1 -- 172.21.15.27:0/4043961645 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028040070 con 0x55e60f8cf0a0 2024-09-21T07:12:33.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa023000640 1 -- 172.21.15.27:0/4043961645 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028004030 con 0x55e60f8cf0a0 2024-09-21T07:12:33.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.655+0000 7fa023000640 1 -- 172.21.15.27:0/4043961645 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028035040 con 0x55e60f8cf0a0 2024-09-21T07:12:33.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.656+0000 7fa02ea00640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa01c0b6560 0x7fa01c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.656+0000 7fa0435c2740 1 -- 172.21.15.27:0/4043961645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8cf0a0 msgr2=0x55e60fc9ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.656+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8cf0a0 0x55e60fc9ebb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa028009230 tx=0x7fa028032b90 comp rx=0 tx=0).stop 2024-09-21T07:12:33.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.656+0000 7fa02ea00640 1 -- 172.21.15.27:0/685886772 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa0240e7ac0 con 0x7fa03810b500 2024-09-21T07:12:33.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.656+0000 7fa036800640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa01c0b6560 0x7fa01c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 -- 172.21.15.27:0/4043961645 shutdown_connections 2024-09-21T07:12:33.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fc9f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8cf0a0 0x55e60fc9ebb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4043961645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8f7f60 0x55e60fc9e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 -- 172.21.15.27:0/4043961645 >> 172.21.15.27:0/4043961645 conn(0x55e60f8ec350 msgr2=0x55e60fc9c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:33.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 -- 172.21.15.27:0/4043961645 shutdown_connections 2024-09-21T07:12:33.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 -- 172.21.15.27:0/4043961645 wait complete. 2024-09-21T07:12:33.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa036800640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa01c0b6560 0x7fa01c0b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa030005fd0 tx=0x7fa030009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:33.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.657+0000 7fa0435c2740 1 -- start start 2024-09-21T07:12:33.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 1 Processor -- start 2024-09-21T07:12:33.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcaa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8d7280 0x55e60fcaa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8f7f60 0x55e60fcb60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e60fa665f0 con 0x55e60f8f7f60 2024-09-21T07:12:33.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e60fba0460 con 0x55e60f8d7280 2024-09-21T07:12:33.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e60fcacdf0 con 0x55e60f8cf0a0 2024-09-21T07:12:33.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa0435c2740 10 client.0 _ll_register_callbacks cb 0x55e60fbb0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:33.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.658+0000 7fa035400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8d7280 0x55e60fcaa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8d7280 0x55e60fcaa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48226/0 (socket says 172.21.15.27:48226) 2024-09-21T07:12:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 -- 172.21.15.27:0/865270636 learned_addr learned my addr 172.21.15.27:0/865270636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035e00640 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcaa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa036800640 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8f7f60 0x55e60fcb60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 -- 172.21.15.27:0/865270636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 msgr2=0x55e60fcaa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcaa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 -- 172.21.15.27:0/865270636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8f7f60 msgr2=0x55e60fcb60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8f7f60 0x55e60fcb60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 -- 172.21.15.27:0/865270636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60f963540 con 0x55e60f8d7280 2024-09-21T07:12:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa035400640 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8d7280 0x55e60fcaa7e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa0240ad530 tx=0x7fa0240096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fb6c072cf503514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.659+0000 7fa017400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:12:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.660+0000 7fa017400640 1 -- 172.21.15.27:0/865270636 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240ad340 con 0x55e60f8d7280 2024-09-21T07:12:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.660+0000 7fa017400640 1 -- 172.21.15.27:0/865270636 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa0240aeab0 con 0x55e60f8d7280 2024-09-21T07:12:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.660+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e60fba2a70 con 0x55e60f8d7280 2024-09-21T07:12:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.660+0000 7fa017400640 1 -- 172.21.15.27:0/865270636 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa0240b2260 con 0x55e60f8d7280 2024-09-21T07:12:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.660+0000 7fa0435c2740 10 client.25328 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.660+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e60fbbd6f0 con 0x55e60f8d7280 2024-09-21T07:12:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.660+0000 7fa017400640 1 -- 172.21.15.27:0/865270636 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e60fba2a70 con 0x55e60f8d7280 2024-09-21T07:12:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa017400640 1 -- 172.21.15.27:0/865270636 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa0240b34f0 con 0x55e60f8d7280 2024-09-21T07:12:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa0435c2740 10 client.25328 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa0435c2740 10 client.25328 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T07:12:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa0435c2740 20 client.25328 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa0435c2740 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f8b68e0 0x55e60fcbc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa0435c2740 4 client.25328 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:33.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60fc17ca0 con 0x55e60f8b68e0 2024-09-21T07:12:33.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.661+0000 7fa035e00640 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f8b68e0 0x55e60fcbc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.662+0000 7fa035e00640 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f8b68e0 0x55e60fcbc5f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7dc32458b603d5f9 server_cookie=d72c2015c72ff590 in_seq=0 out_seq=0 2024-09-21T07:12:33.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.662+0000 7fa017400640 10 client.25328 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:33.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa017400640 1 -- 172.21.15.27:0/865270636 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e60fc17ca0 con 0x55e60f8b68e0 2024-09-21T07:12:33.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa017400640 10 client.25328 handle_command_reply: tid=0 2024-09-21T07:12:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 client.25328 shutdown 2024-09-21T07:12:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8d7280 msgr2=0x55e60fcaa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8d7280 0x55e60fcaa7e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa0240ad530 tx=0x7fa0240096a0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 shutdown_connections 2024-09-21T07:12:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f8b68e0 0x55e60fcbc5f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8f7f60 0x55e60fcb60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8d7280 0x55e60fcaa7e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 --2- 172.21.15.27:0/865270636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcaa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa017400640 0 client.25328 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 >> 172.21.15.27:0/865270636 conn(0x55e60f8ec350 msgr2=0x55e60fca1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 shutdown_connections 2024-09-21T07:12:33.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 1 -- 172.21.15.27:0/865270636 wait complete. 2024-09-21T07:12:33.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.663+0000 7fa0435c2740 20 client.25328 trim_cache size 0 max 16384 2024-09-21T07:12:33.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.682+0000 7fa0435c2740 1 Processor -- start 2024-09-21T07:12:33.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.682+0000 7fa0435c2740 1 -- start start 2024-09-21T07:12:33.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8cf0a0 0x55e60fcbb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8d7280 0x55e60fca44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fca5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa0435c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e60fcba930 con 0x55e60f8cf0a0 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa0435c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e60fcbb000 con 0x55e60f8f7f60 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa0435c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e60fcbb6d0 con 0x55e60f8d7280 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa035e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8cf0a0 0x55e60fcbb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa035e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8cf0a0 0x55e60fcbb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46138/0 (socket says 172.21.15.27:46138) 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa035e00640 1 -- 172.21.15.27:0/4128869554 learned_addr learned my addr 172.21.15.27:0/4128869554 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa036800640 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fca5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.683+0000 7fa035400640 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8d7280 0x55e60fca44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa035e00640 1 -- 172.21.15.27:0/4128869554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8d7280 msgr2=0x55e60fca44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa035e00640 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8d7280 0x55e60fca44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa035e00640 1 -- 172.21.15.27:0/4128869554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 msgr2=0x55e60fca5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa035e00640 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fca5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa035e00640 1 -- 172.21.15.27:0/4128869554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60fca14d0 con 0x55e60f8cf0a0 2024-09-21T07:12:33.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa035e00640 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8cf0a0 0x55e60fcbb180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa028005150 tx=0x7fa028005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a518be85e2a819a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:33.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa022600640 1 -- 172.21.15.27:0/4128869554 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028035040 con 0x55e60f8cf0a0 2024-09-21T07:12:33.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa022600640 1 -- 172.21.15.27:0/4128869554 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028031dd0 con 0x55e60f8cf0a0 2024-09-21T07:12:33.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.684+0000 7fa022600640 1 -- 172.21.15.27:0/4128869554 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028040040 con 0x55e60f8cf0a0 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 -- 172.21.15.27:0/4128869554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8cf0a0 msgr2=0x55e60fcbb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8cf0a0 0x55e60fcbb180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa028005150 tx=0x7fa028005180 comp rx=0 tx=0).stop 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 -- 172.21.15.27:0/4128869554 shutdown_connections 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8d7280 0x55e60fca44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fca5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 --2- 172.21.15.27:0/4128869554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8cf0a0 0x55e60fcbb180 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 -- 172.21.15.27:0/4128869554 >> 172.21.15.27:0/4128869554 conn(0x55e60f8ec350 msgr2=0x55e60fcaa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 -- 172.21.15.27:0/4128869554 shutdown_connections 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.685+0000 7fa0435c2740 1 -- 172.21.15.27:0/4128869554 wait complete. 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.686+0000 7fa0435c2740 1 -- start start 2024-09-21T07:12:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.686+0000 7fa0435c2740 1 Processor -- start 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcc5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fcc3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa0435c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fcc45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa0435c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e60fcbc470 con 0x55e60f8d7280 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa0435c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e60fcbbda0 con 0x55e60f8f7f60 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa0435c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e60fcbcb40 con 0x55e60f8cf0a0 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa0435c2740 10 client.0 _ll_register_callbacks cb 0x7fa0240ad620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:33.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa035400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fcc3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa036800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fcc45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa035e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcc5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa035400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fcc3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46146/0 (socket says 172.21.15.27:46146) 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.687+0000 7fa035400640 1 -- 172.21.15.27:0/681511051 learned_addr learned my addr 172.21.15.27:0/681511051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa035400640 1 -- 172.21.15.27:0/681511051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 msgr2=0x55e60fcc5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa035400640 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcc5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa035400640 1 -- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 msgr2=0x55e60fcc45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa035400640 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fcc45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa035400640 1 -- 172.21.15.27:0/681511051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60fcc2e10 con 0x55e60f8d7280 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa036800640 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fcc45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa035400640 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fcc3ed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa0240a9db0 tx=0x7fa02400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9578b913ffa966 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa017400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa017400640 1 -- 172.21.15.27:0/681511051 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240b6890 con 0x55e60f8d7280 2024-09-21T07:12:33.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa017400640 1 -- 172.21.15.27:0/681511051 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa0240ac710 con 0x55e60f8d7280 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa028031dd0 con 0x55e60f8d7280 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa017400640 1 -- 172.21.15.27:0/681511051 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa0240ad340 con 0x55e60f8d7280 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa0435c2740 10 client.15306 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.688+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e60fd27c40 con 0x55e60f8d7280 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa017400640 1 -- 172.21.15.27:0/681511051 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028031dd0 con 0x55e60f8d7280 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa017400640 1 -- 172.21.15.27:0/681511051 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa0240ad830 con 0x55e60f8d7280 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa0435c2740 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa0435c2740 10 client.15306 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa0435c2740 20 client.15306 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa0435c2740 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f88bc90 0x55e60fd2b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa0435c2740 4 client.15306 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:12:33.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "status"}) -- 0x55e60fc13750 con 0x55e60f88bc90 2024-09-21T07:12:33.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.689+0000 7fa035e00640 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f88bc90 0x55e60fd2b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:33.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.690+0000 7fa035e00640 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f88bc90 0x55e60fd2b3a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f536d18fef2187b server_cookie=ffdc3696a59ae53c in_seq=0 out_seq=0 2024-09-21T07:12:33.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.690+0000 7fa017400640 10 client.15306 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:33.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.690+0000 7fa017400640 1 -- 172.21.15.27:0/681511051 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e60fc13750 con 0x55e60f88bc90 2024-09-21T07:12:33.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.690+0000 7fa017400640 10 client.15306 handle_command_reply: tid=0 2024-09-21T07:12:33.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.690+0000 7fa0435c2740 1 client.15306 shutdown 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.690+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 msgr2=0x55e60fcc3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.690+0000 7fa0435c2740 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fcc3ed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa0240a9db0 tx=0x7fa02400cef0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 shutdown_connections 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e60f88bc90 0x55e60fd2b3a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e60f8f7f60 0x55e60fcc45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e60f8d7280 0x55e60fcc3ed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 --2- 172.21.15.27:0/681511051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e60f8cf0a0 0x55e60fcc5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa017400640 0 client.15306 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 >> 172.21.15.27:0/681511051 conn(0x55e60f8ec350 msgr2=0x55e60fcaa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 shutdown_connections 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 1 -- 172.21.15.27:0/681511051 wait complete. 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.691+0000 7fa0435c2740 20 client.15306 trim_cache size 0 max 16384 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24740, 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:12:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:12:33.698 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 105.157742564, 2024-09-21T07:12:33.698 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T07:12:33.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:12:33.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T07:12:33.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 182.33889107300001 2024-09-21T07:12:33.698 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa01c0b6560 msgr2=0x7fa01c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa01c0b6560 0x7fa01c0b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa030005fd0 tx=0x7fa030009500 comp rx=0 tx=0).stop 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 msgr2=0x7fa03819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 0x7fa03819cac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e9f0 tx=0x7fa02400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 shutdown_connections 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa01c0b6560 0x7fa01c0b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa03810be50 0x7fa0381a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa03810b500 0x7fa03819cac0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 --2- 172.21.15.27:0/685886772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa038105bc0 0x7fa03819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 >> 172.21.15.27:0/685886772 conn(0x7fa038101670 msgr2=0x7fa038103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 shutdown_connections 2024-09-21T07:12:33.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:33.692+0000 7fa037200640 1 -- 172.21.15.27:0/685886772 wait complete. 2024-09-21T07:12:34.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.g status 2024-09-21T07:12:34.878 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:34 smithi153 ceph-mon[34747]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T07:12:35.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:34 smithi027 ceph-mon[30779]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T07:12:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:34 smithi119 ceph-mon[34519]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 -- 172.21.15.27:0/3018821218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 msgr2=0x7f4fc410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 --2- 172.21.15.27:0/3018821218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 0x7f4fc410bae0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009900 tx=0x7f4fb002a410 comp rx=0 tx=0).stop 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 -- 172.21.15.27:0/3018821218 shutdown_connections 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 --2- 172.21.15.27:0/3018821218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc410c020 0x7f4fc4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 --2- 172.21.15.27:0/3018821218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 0x7f4fc410bae0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 --2- 172.21.15.27:0/3018821218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc4105d90 0x7f4fc41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 -- 172.21.15.27:0/3018821218 >> 172.21.15.27:0/3018821218 conn(0x7f4fc41017c0 msgr2=0x7f4fc4103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.499+0000 7f4fcb800640 1 -- 172.21.15.27:0/3018821218 shutdown_connections 2024-09-21T07:12:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.500+0000 7f4fcb800640 1 -- 172.21.15.27:0/3018821218 wait complete. 2024-09-21T07:12:35.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 Processor -- start 2024-09-21T07:12:35.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 -- start start 2024-09-21T07:12:35.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc4105d90 0x7f4fc419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 0x7f4fc419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc410c020 0x7f4fc41a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4115d30 con 0x7f4fc410b6d0 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4115bb0 con 0x7f4fc410c020 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.501+0000 7f4fcb800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4115eb0 con 0x7f4fc4105d90 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc410c020 0x7f4fc41a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc4105d90 0x7f4fc419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 0x7f4fc419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc410c020 0x7f4fc41a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48240/0 (socket says 172.21.15.27:48240) 2024-09-21T07:12:35.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 -- 172.21.15.27:0/1068626802 learned_addr learned my addr 172.21.15.27:0/1068626802 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:35.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc4105d90 0x7f4fc419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49804/0 (socket says 172.21.15.27:49804) 2024-09-21T07:12:35.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 -- 172.21.15.27:0/1068626802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc4105d90 msgr2=0x7f4fc419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc4105d90 0x7f4fc419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 -- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 msgr2=0x7f4fc419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 0x7f4fc419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.502+0000 7f4fc9800640 1 -- 172.21.15.27:0/1068626802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc41a26d0 con 0x7f4fc410c020 2024-09-21T07:12:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.503+0000 7f4fc9800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc410c020 0x7f4fc41a1fa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00eb10 tx=0x7f4fbc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.503+0000 7f4fc1600640 1 -- 172.21.15.27:0/1068626802 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc004b20 con 0x7f4fc410c020 2024-09-21T07:12:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.503+0000 7f4fc1600640 1 -- 172.21.15.27:0/1068626802 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc004cc0 con 0x7f4fc410c020 2024-09-21T07:12:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.503+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc41a29c0 con 0x7f4fc410c020 2024-09-21T07:12:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.503+0000 7f4fc1600640 1 -- 172.21.15.27:0/1068626802 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc010670 con 0x7f4fc410c020 2024-09-21T07:12:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.503+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc41a9120 con 0x7f4fc410c020 2024-09-21T07:12:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.506+0000 7f4fd25f3740 1 Processor -- start 2024-09-21T07:12:35.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.507+0000 7f4fc1600640 1 -- 172.21.15.27:0/1068626802 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f4fbc005020 con 0x7f4fc410c020 2024-09-21T07:12:35.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.507+0000 7f4fd25f3740 1 -- start start 2024-09-21T07:12:35.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a930490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a9309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a569280 0x557e1a930f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557e1a8a7d50 con 0x557e1a569280 2024-09-21T07:12:35.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557e1a897180 con 0x557e1a5610a0 2024-09-21T07:12:35.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e1a89af90 con 0x557e1a589f60 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a9309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a9309d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48242/0 (socket says 172.21.15.27:48242) 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fc8e00640 1 -- 172.21.15.27:0/3648575709 learned_addr learned my addr 172.21.15.27:0/3648575709 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a930490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.508+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a569280 0x557e1a930f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc8e00640 1 -- 172.21.15.27:0/3648575709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 msgr2=0x557e1a930490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a930490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc8e00640 1 -- 172.21.15.27:0/3648575709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a569280 msgr2=0x557e1a930f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a569280 0x557e1a930f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc8e00640 1 -- 172.21.15.27:0/3648575709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1a861a70 con 0x557e1a5610a0 2024-09-21T07:12:35.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a9309d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009cf0 tx=0x7f4fb8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92bb92c6668177e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:35.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc1600640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6560 0x7f4fac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fb6000640 1 -- 172.21.15.27:0/3648575709 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8004230 con 0x557e1a5610a0 2024-09-21T07:12:35.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fc1600640 1 -- 172.21.15.27:0/1068626802 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4fbc0e7dc0 con 0x7f4fc410c020 2024-09-21T07:12:35.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fb6000640 1 -- 172.21.15.27:0/3648575709 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb80043d0 con 0x557e1a5610a0 2024-09-21T07:12:35.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.509+0000 7f4fb6000640 1 -- 172.21.15.27:0/3648575709 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8019880 con 0x557e1a5610a0 2024-09-21T07:12:35.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.510+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6560 0x7f4fac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.510+0000 7f4fd25f3740 1 -- 172.21.15.27:0/3648575709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 msgr2=0x557e1a9309d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.510+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a9309d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009cf0 tx=0x7f4fb8011810 comp rx=0 tx=0).stop 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.510+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6560 0x7f4fac0b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002aa00 tx=0x7f4fb0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.510+0000 7f4fd25f3740 1 -- 172.21.15.27:0/3648575709 shutdown_connections 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.510+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a569280 0x557e1a930f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.510+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a9309d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.511+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/3648575709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a930490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.511+0000 7f4fd25f3740 1 -- 172.21.15.27:0/3648575709 >> 172.21.15.27:0/3648575709 conn(0x557e1a57e350 msgr2=0x557e1a92e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.511+0000 7f4fd25f3740 1 -- 172.21.15.27:0/3648575709 shutdown_connections 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.511+0000 7f4fd25f3740 1 -- 172.21.15.27:0/3648575709 wait complete. 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.511+0000 7f4fd25f3740 1 -- start start 2024-09-21T07:12:35.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.511+0000 7f4fd25f3740 1 Processor -- start 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a93c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a93c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a589f60 0x557e1a947f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557e1a6f85f0 con 0x557e1a5610a0 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557e1a832460 con 0x557e1a589f60 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e1a93ec70 con 0x557e1a569280 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fd25f3740 10 client.0 _ll_register_callbacks cb 0x557e1a842210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a93c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a93c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a93c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46174/0 (socket says 172.21.15.27:46174) 2024-09-21T07:12:35.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fc8e00640 1 -- 172.21.15.27:0/2281125078 learned_addr learned my addr 172.21.15.27:0/2281125078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.512+0000 7f4fc9800640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a589f60 0x557e1a947f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc3e00640 1 -- 172.21.15.27:0/2281125078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a589f60 msgr2=0x557e1a947f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a589f60 0x557e1a947f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc3e00640 1 -- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 msgr2=0x557e1a93c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a93c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc3e00640 1 -- 172.21.15.27:0/2281125078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1a5f5540 con 0x557e1a569280 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a93c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc9800640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a589f60 0x557e1a947f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a93c540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009230 tx=0x7f4fb0005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d58b7f3e1efab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.513+0000 7f4faaa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4faaa00640 1 -- 172.21.15.27:0/2281125078 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003d80 con 0x557e1a569280 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e1a834a70 con 0x557e1a569280 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4faaa00640 1 -- 172.21.15.27:0/2281125078 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4fb00314a0 con 0x557e1a569280 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4faaa00640 1 -- 172.21.15.27:0/2281125078 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fb00316a0 con 0x557e1a569280 2024-09-21T07:12:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4fd25f3740 10 client.25342 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:35.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e1a84f6f0 con 0x557e1a569280 2024-09-21T07:12:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4faaa00640 1 -- 172.21.15.27:0/2281125078 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e1a834a70 con 0x557e1a569280 2024-09-21T07:12:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.514+0000 7f4faaa00640 1 -- 172.21.15.27:0/2281125078 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4fb0044030 con 0x557e1a569280 2024-09-21T07:12:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.515+0000 7f4fd25f3740 10 client.25342 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.515+0000 7f4fd25f3740 10 client.25342 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T07:12:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.515+0000 7f4fd25f3740 20 client.25342 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.515+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a5488e0 0x557e1a94e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.515+0000 7f4fd25f3740 4 client.25342 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.515+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e1a8a9ca0 con 0x557e1a5488e0 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.515+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a5488e0 0x557e1a94e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.516+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a5488e0 0x557e1a94e4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=338a75f1c061a61f server_cookie=12be4f724daae37a in_seq=0 out_seq=0 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.516+0000 7f4faaa00640 10 client.25342 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.516+0000 7f4faaa00640 1 -- 172.21.15.27:0/2281125078 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e1a8a9ca0 con 0x557e1a5488e0 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4faaa00640 10 client.25342 handle_command_reply: tid=0 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 client.25342 shutdown 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 msgr2=0x557e1a93c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a93c540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009230 tx=0x7f4fb0005780 comp rx=0 tx=0).stop 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 shutdown_connections 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a5488e0 0x557e1a94e4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a93c540 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a589f60 0x557e1a947f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2281125078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a93c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4faaa00640 0 client.25342 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:35.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 >> 172.21.15.27:0/2281125078 conn(0x557e1a57e350 msgr2=0x557e1a933620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:35.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 shutdown_connections 2024-09-21T07:12:35.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2281125078 wait complete. 2024-09-21T07:12:35.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.517+0000 7f4fd25f3740 20 client.25342 trim_cache size 0 max 16384 2024-09-21T07:12:35.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.536+0000 7f4fd25f3740 1 Processor -- start 2024-09-21T07:12:35.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fd25f3740 1 -- start start 2024-09-21T07:12:35.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a94e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a569280 0x557e1a929010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a93a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557e1a94c810 con 0x557e1a5610a0 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557e1a94cee0 con 0x557e1a569280 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e1a94d5b0 con 0x557e1a589f60 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a569280 0x557e1a929010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a94e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a569280 0x557e1a929010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48260/0 (socket says 172.21.15.27:48260) 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fc3e00640 1 -- 172.21.15.27:0/2138310403 learned_addr learned my addr 172.21.15.27:0/2138310403 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.537+0000 7f4fc9800640 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a93a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fc3e00640 1 -- 172.21.15.27:0/2138310403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 msgr2=0x557e1a93a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a93a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fc3e00640 1 -- 172.21.15.27:0/2138310403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 msgr2=0x557e1a94e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a94e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fc3e00640 1 -- 172.21.15.27:0/2138310403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1a93d0c0 con 0x557e1a569280 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a569280 0x557e1a929010 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005ab0 tx=0x7f4fb002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42637fcd1b2a22f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fb5600640 1 -- 172.21.15.27:0/2138310403 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb003db30 con 0x557e1a569280 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fb5600640 1 -- 172.21.15.27:0/2138310403 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb004c070 con 0x557e1a569280 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.538+0000 7f4fb5600640 1 -- 172.21.15.27:0/2138310403 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003dc0 con 0x557e1a569280 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2138310403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a569280 msgr2=0x557e1a929010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a569280 0x557e1a929010 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005ab0 tx=0x7f4fb002a410 comp rx=0 tx=0).stop 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2138310403 shutdown_connections 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a589f60 0x557e1a93a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a569280 0x557e1a929010 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2138310403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a5610a0 0x557e1a94e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2138310403 >> 172.21.15.27:0/2138310403 conn(0x557e1a57e350 msgr2=0x557e1a936fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:35.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2138310403 shutdown_connections 2024-09-21T07:12:35.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.539+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2138310403 wait complete. 2024-09-21T07:12:35.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.540+0000 7f4fd25f3740 1 -- start start 2024-09-21T07:12:35.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.540+0000 7f4fd25f3740 1 Processor -- start 2024-09-21T07:12:35.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.540+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a956cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.540+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a950180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.540+0000 7f4fd25f3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a589f60 0x557e1a9506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557e1a94e350 con 0x557e1a589f60 2024-09-21T07:12:35.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557e1a94dc80 con 0x557e1a5610a0 2024-09-21T07:12:35.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fd25f3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e1a94ea20 con 0x557e1a569280 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fd25f3740 10 client.0 _ll_register_callbacks cb 0x7f4fb0004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a589f60 0x557e1a9506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a956cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a956cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48268/0 (socket says 172.21.15.27:48268) 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc8e00640 1 -- 172.21.15.27:0/2457802947 learned_addr learned my addr 172.21.15.27:0/2457802947 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a589f60 0x557e1a9506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46196/0 (socket says 172.21.15.27:46196) 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc3e00640 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a950180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc9800640 1 -- 172.21.15.27:0/2457802947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 msgr2=0x557e1a950180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc9800640 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a950180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc9800640 1 -- 172.21.15.27:0/2457802947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 msgr2=0x557e1a956cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:35.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc9800640 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a956cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc9800640 1 -- 172.21.15.27:0/2457802947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1a950c00 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.541+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a956cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4fc9800640 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a589f60 0x557e1a9506c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0aacf0 tx=0x7f4fbc0ad910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6cee267f5c5387 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4faaa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4faaa00640 1 -- 172.21.15.27:0/2457802947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0b2360 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4faaa00640 1 -- 172.21.15.27:0/2457802947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4fbc0acee0 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb004c070 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4faaa00640 1 -- 172.21.15.27:0/2457802947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fbc0ad590 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4fd25f3740 10 client.15324 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e1a956620 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.542+0000 7f4faaa00640 1 -- 172.21.15.27:0/2457802947 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb004c070 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4faaa00640 1 -- 172.21.15.27:0/2457802947 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4fbc0ac7e0 con 0x557e1a589f60 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4fd25f3740 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:35.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4fd25f3740 10 client.15324 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T07:12:35.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4fd25f3740 20 client.15324 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:35.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a51dc90 0x557e1a9bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:35.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4fd25f3740 4 client.15324 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:12:35.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "status"}) -- 0x557e1a8a5750 con 0x557e1a51dc90 2024-09-21T07:12:35.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.543+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a51dc90 0x557e1a9bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fc8e00640 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a51dc90 0x557e1a9bcf60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce1eba9c4e636aa8 server_cookie=ba113d709b6f61b7 in_seq=0 out_seq=0 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4faaa00640 10 client.15324 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4faaa00640 1 -- 172.21.15.27:0/2457802947 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557e1a8a5750 con 0x557e1a51dc90 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4faaa00640 10 client.15324 handle_command_reply: tid=0 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 client.15324 shutdown 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a589f60 msgr2=0x557e1a9506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a589f60 0x557e1a9506c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0aacf0 tx=0x7f4fbc0ad910 comp rx=0 tx=0).stop 2024-09-21T07:12:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 shutdown_connections 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557e1a51dc90 0x557e1a9bcf60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e1a569280 0x557e1a950180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557e1a589f60 0x557e1a9506c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 --2- 172.21.15.27:0/2457802947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1a5610a0 0x557e1a956cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 >> 172.21.15.27:0/2457802947 conn(0x557e1a57e350 msgr2=0x557e1a93bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.544+0000 7f4faaa00640 0 client.15324 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.545+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 shutdown_connections 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.545+0000 7f4fd25f3740 1 -- 172.21.15.27:0/2457802947 wait complete. 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.545+0000 7f4fd25f3740 20 client.15324 trim_cache size 0 max 16384 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24748, 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 108.019120883, 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 178.077155127 2024-09-21T07:12:35.551 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.545+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6560 msgr2=0x7f4fac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.545+0000 7f4fcb800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6560 0x7f4fac0b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002aa00 tx=0x7f4fb0005a20 comp rx=0 tx=0).stop 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.545+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc410c020 msgr2=0x7f4fc41a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.545+0000 7f4fcb800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc410c020 0x7f4fc41a1fa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00eb10 tx=0x7f4fbc00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 shutdown_connections 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6560 0x7f4fac0b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc410c020 0x7f4fc41a1fa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc410b6d0 0x7f4fc419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 --2- 172.21.15.27:0/1068626802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc4105d90 0x7f4fc419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 >> 172.21.15.27:0/1068626802 conn(0x7f4fc41017c0 msgr2=0x7f4fc410ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 shutdown_connections 2024-09-21T07:12:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:35.546+0000 7f4fcb800640 1 -- 172.21.15.27:0/1068626802 wait complete. 2024-09-21T07:12:36.376 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T07:12:36.762 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:36 smithi153 ceph-mon[34747]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T07:12:37.241 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:36 smithi027 ceph-mon[30779]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T07:12:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:36 smithi119 ceph-mon[34519]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T07:12:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 -- 172.21.15.27:0/1977899448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 msgr2=0x7f88d410fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 --2- 172.21.15.27:0/1977899448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 0x7f88d410fa30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009900 tx=0x7f88c802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 -- 172.21.15.27:0/1977899448 shutdown_connections 2024-09-21T07:12:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 --2- 172.21.15.27:0/1977899448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d410ff70 0x7f88d4112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 --2- 172.21.15.27:0/1977899448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 0x7f88d410fa30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 --2- 172.21.15.27:0/1977899448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d4075750 0x7f88d4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 -- 172.21.15.27:0/1977899448 >> 172.21.15.27:0/1977899448 conn(0x7f88d40ff400 msgr2=0x7f88d4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:37.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 -- 172.21.15.27:0/1977899448 shutdown_connections 2024-09-21T07:12:37.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.428+0000 7f88da200640 1 -- 172.21.15.27:0/1977899448 wait complete. 2024-09-21T07:12:37.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.429+0000 7f88da200640 1 Processor -- start 2024-09-21T07:12:37.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.429+0000 7f88da200640 1 -- start start 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88da200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d4075750 0x7f88d419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88da200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 0x7f88d419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88da200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d410ff70 0x7f88d41a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88da200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4115ae0 con 0x7f88d4075750 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88da200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4115960 con 0x7f88d40760a0 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88da200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4115c60 con 0x7f88d410ff70 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88d8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d4075750 0x7f88d419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88d8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d4075750 0x7f88d419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46222/0 (socket says 172.21.15.27:46222) 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88d8e00640 1 -- 172.21.15.27:0/304484061 learned_addr learned my addr 172.21.15.27:0/304484061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88d9800640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d410ff70 0x7f88d41a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88d3e00640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 0x7f88d419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88d8e00640 1 -- 172.21.15.27:0/304484061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d410ff70 msgr2=0x7f88d41a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.430+0000 7f88d8e00640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d410ff70 0x7f88d41a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88d8e00640 1 -- 172.21.15.27:0/304484061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 msgr2=0x7f88d419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88d8e00640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 0x7f88d419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88d8e00640 1 -- 172.21.15.27:0/304484061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d41a2500 con 0x7f88d4075750 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88d8e00640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d4075750 0x7f88d419c590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e9f0 tx=0x7f88c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88d1600640 1 -- 172.21.15.27:0/304484061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c400ccc0 con 0x7f88d4075750 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d41a27f0 con 0x7f88d4075750 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88d1600640 1 -- 172.21.15.27:0/304484061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c400ce60 con 0x7f88d4075750 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88d1600640 1 -- 172.21.15.27:0/304484061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4010430 con 0x7f88d4075750 2024-09-21T07:12:37.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.431+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d41a8f50 con 0x7f88d4075750 2024-09-21T07:12:37.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.436+0000 7f88d1600640 1 -- 172.21.15.27:0/304484061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f88c40046c0 con 0x7f88d4075750 2024-09-21T07:12:37.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.436+0000 7f88e24d6740 1 Processor -- start 2024-09-21T07:12:37.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.436+0000 7f88e24d6740 1 -- start start 2024-09-21T07:12:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae24f60 0x55d8db1cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88e24d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d8db142d50 con 0x55d8dae04280 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88e24d6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d8db132180 con 0x55d8dae24f60 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88e24d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8db135f90 con 0x55d8dadfc0a0 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88d9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae24f60 0x55d8db1cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88d8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88d9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae24f60 0x55d8db1cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48300/0 (socket says 172.21.15.27:48300) 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.437+0000 7f88d9800640 1 -- 172.21.15.27:0/2675536475 learned_addr learned my addr 172.21.15.27:0/2675536475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:37.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d3e00640 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d1600640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88bc0b6560 0x7f88bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d9800640 1 -- 172.21.15.27:0/2675536475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 msgr2=0x55d8db1cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d9800640 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d9800640 1 -- 172.21.15.27:0/2675536475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 msgr2=0x55d8db1cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d9800640 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d9800640 1 -- 172.21.15.27:0/2675536475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8db0fca70 con 0x55d8dae24f60 2024-09-21T07:12:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d1600640 1 -- 172.21.15.27:0/304484061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f88c40e7c50 con 0x7f88d4075750 2024-09-21T07:12:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.438+0000 7f88d9800640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88bc0b6560 0x7f88bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.439+0000 7f88d9800640 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae24f60 0x55d8db1cb670 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009f60 tx=0x7f88cc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03fac8aa7b61539 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.439+0000 7f88c2000640 1 -- 172.21.15.27:0/2675536475 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00f680 con 0x55d8dae24f60 2024-09-21T07:12:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.439+0000 7f88c2000640 1 -- 172.21.15.27:0/2675536475 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc00fcd0 con 0x55d8dae24f60 2024-09-21T07:12:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.439+0000 7f88c2000640 1 -- 172.21.15.27:0/2675536475 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc01bae0 con 0x55d8dae24f60 2024-09-21T07:12:37.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.439+0000 7f88d9800640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88bc0b6560 0x7f88bc0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88cc020000 tx=0x7f88cc021000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:37.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 -- 172.21.15.27:0/2675536475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae24f60 msgr2=0x55d8db1cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae24f60 0x55d8db1cb670 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009f60 tx=0x7f88cc009f90 comp rx=0 tx=0).stop 2024-09-21T07:12:37.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 -- 172.21.15.27:0/2675536475 shutdown_connections 2024-09-21T07:12:37.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2675536475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae24f60 0x55d8db1cb670 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 -- 172.21.15.27:0/2675536475 >> 172.21.15.27:0/2675536475 conn(0x55d8dae19350 msgr2=0x55d8db1c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:37.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 -- 172.21.15.27:0/2675536475 shutdown_connections 2024-09-21T07:12:37.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.440+0000 7f88e24d6740 1 -- 172.21.15.27:0/2675536475 wait complete. 2024-09-21T07:12:37.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.441+0000 7f88e24d6740 1 -- start start 2024-09-21T07:12:37.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.441+0000 7f88e24d6740 1 Processor -- start 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dadfc0a0 0x55d8db1d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dae24f60 0x55d8db1e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88e24d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d8daf935f0 con 0x55d8dae04280 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88e24d6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d8db0cd460 con 0x55d8dadfc0a0 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88e24d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1d9e50 con 0x55d8dae24f60 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88e24d6740 10 client.0 _ll_register_callbacks cb 0x55d8db0dd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:37.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88d8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dadfc0a0 0x55d8db1d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1d7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46238/0 (socket says 172.21.15.27:46238) 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88d3e00640 1 -- 172.21.15.27:0/1938874153 learned_addr learned my addr 172.21.15.27:0/1938874153 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.442+0000 7f88d9800640 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dae24f60 0x55d8db1e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88d8e00640 1 -- 172.21.15.27:0/1938874153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dae24f60 msgr2=0x55d8db1e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88d8e00640 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dae24f60 0x55d8db1e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88d8e00640 1 -- 172.21.15.27:0/1938874153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 msgr2=0x55d8db1d7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88d8e00640 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1d7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88d8e00640 1 -- 172.21.15.27:0/1938874153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8dae90540 con 0x55d8dadfc0a0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88d8e00640 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dadfc0a0 0x55d8db1d71f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55d8db1ca2f0 tx=0x7f88c40b7210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f7ddb3c01927be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88b6a00640 1 -- 172.21.15.27:0/1938874153 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40a97b0 con 0x55d8dadfc0a0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.443+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8db0cfa70 con 0x55d8dadfc0a0 2024-09-21T07:12:37.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.444+0000 7f88b6a00640 1 -- 172.21.15.27:0/1938874153 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f88c40ac5e0 con 0x55d8dadfc0a0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.444+0000 7f88b6a00640 1 -- 172.21.15.27:0/1938874153 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88c40ad080 con 0x55d8dadfc0a0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.444+0000 7f88e24d6740 10 client.25364 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.444+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8db0ea6f0 con 0x55d8dadfc0a0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.444+0000 7f88b6a00640 1 -- 172.21.15.27:0/1938874153 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d8db0cfa70 con 0x55d8dadfc0a0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.444+0000 7f88b6a00640 1 -- 172.21.15.27:0/1938874153 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88c40d3070 con 0x55d8dadfc0a0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.445+0000 7f88e24d6740 10 client.25364 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.445+0000 7f88e24d6740 10 client.25364 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.445+0000 7f88e24d6740 20 client.25364 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.445+0000 7f88e24d6740 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dade38e0 0x55d8db1e9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.445+0000 7f88e24d6740 4 client.25364 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.445+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8db144ca0 con 0x55d8dade38e0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.445+0000 7f88d3e00640 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dade38e0 0x55d8db1e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.446+0000 7f88d3e00640 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dade38e0 0x55d8db1e9820 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909b2ab333b6509d server_cookie=56fae4cf879db24c in_seq=0 out_seq=0 2024-09-21T07:12:37.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.446+0000 7f88b6a00640 10 client.25364 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.446+0000 7f88b6a00640 1 -- 172.21.15.27:0/1938874153 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8db144ca0 con 0x55d8dade38e0 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.446+0000 7f88b6a00640 10 client.25364 handle_command_reply: tid=0 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.446+0000 7f88e24d6740 1 client.25364 shutdown 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dadfc0a0 msgr2=0x55d8db1d71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dadfc0a0 0x55d8db1d71f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55d8db1ca2f0 tx=0x7f88c40b7210 comp rx=0 tx=0).stop 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 shutdown_connections 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dade38e0 0x55d8db1e9820 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dae24f60 0x55d8db1e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae04280 0x55d8db1d7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 --2- 172.21.15.27:0/1938874153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dadfc0a0 0x55d8db1d71f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 >> 172.21.15.27:0/1938874153 conn(0x55d8dae19350 msgr2=0x55d8db1ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88b6a00640 0 client.25364 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 shutdown_connections 2024-09-21T07:12:37.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 1 -- 172.21.15.27:0/1938874153 wait complete. 2024-09-21T07:12:37.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.447+0000 7f88e24d6740 20 client.25364 trim_cache size 0 max 16384 2024-09-21T07:12:37.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 Processor -- start 2024-09-21T07:12:37.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 -- start start 2024-09-21T07:12:37.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1e9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1e7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1e7b60 con 0x55d8dae24f60 2024-09-21T07:12:37.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1e8230 con 0x55d8dae04280 2024-09-21T07:12:37.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.467+0000 7f88e24d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1e8900 con 0x55d8dadfc0a0 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1e7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46246/0 (socket says 172.21.15.27:46246) 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d9800640 1 -- 172.21.15.27:0/585703080 learned_addr learned my addr 172.21.15.27:0/585703080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d3e00640 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d8e00640 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d8e00640 1 -- 172.21.15.27:0/585703080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 msgr2=0x55d8db1d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d8e00640 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d8e00640 1 -- 172.21.15.27:0/585703080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 msgr2=0x55d8db1e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d8e00640 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1e7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d8e00640 1 -- 172.21.15.27:0/585703080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8db1d8090 con 0x55d8dadfc0a0 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d3e00640 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.468+0000 7f88d9800640 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.469+0000 7f88d8e00640 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1e9820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c6a0 tx=0x7f88c4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4762b9812777f21c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.469+0000 7f88c1600640 1 -- 172.21.15.27:0/585703080 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40c7070 con 0x55d8dadfc0a0 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.469+0000 7f88c1600640 1 -- 172.21.15.27:0/585703080 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c40ad420 con 0x55d8dadfc0a0 2024-09-21T07:12:37.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.469+0000 7f88c1600640 1 -- 172.21.15.27:0/585703080 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40ad260 con 0x55d8dadfc0a0 2024-09-21T07:12:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 -- 172.21.15.27:0/585703080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 msgr2=0x55d8db1e9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1e9820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c6a0 tx=0x7f88c4009310 comp rx=0 tx=0).stop 2024-09-21T07:12:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 -- 172.21.15.27:0/585703080 shutdown_connections 2024-09-21T07:12:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 --2- 172.21.15.27:0/585703080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1e9820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 -- 172.21.15.27:0/585703080 >> 172.21.15.27:0/585703080 conn(0x55d8dae19350 msgr2=0x55d8db1d5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 -- 172.21.15.27:0/585703080 shutdown_connections 2024-09-21T07:12:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.470+0000 7f88e24d6740 1 -- 172.21.15.27:0/585703080 wait complete. 2024-09-21T07:12:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.471+0000 7f88e24d6740 1 -- start start 2024-09-21T07:12:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.471+0000 7f88e24d6740 1 Processor -- start 2024-09-21T07:12:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88e24d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88e24d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1e96a0 con 0x55d8dae24f60 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88e24d6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1e8fd0 con 0x55d8dae04280 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88e24d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1e9d70 con 0x55d8dadfc0a0 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88e24d6740 10 client.0 _ll_register_callbacks cb 0x7f88c40d1050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88d8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88d9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88d8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49912/0 (socket says 172.21.15.27:49912) 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88d8e00640 1 -- 172.21.15.27:0/2067624490 learned_addr learned my addr 172.21.15.27:0/2067624490 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88d3e00640 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88d8e00640 1 -- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 msgr2=0x55d8db1f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.472+0000 7f88d8e00640 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88d8e00640 1 -- 172.21.15.27:0/2067624490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 msgr2=0x55d8db1eb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88d8e00640 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1eb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88d8e00640 1 -- 172.21.15.27:0/2067624490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8db1ebc60 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88d8e00640 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1f1ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88c40ad8a0 tx=0x7f88c4005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13cde7909704373c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88b6a00640 1 -- 172.21.15.27:0/2067624490 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40c7070 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88b6a00640 1 -- 172.21.15.27:0/2067624490 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f88c40ac7e0 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c40ad420 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88b6a00640 1 -- 172.21.15.27:0/2067624490 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88c40cd050 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88e24d6740 10 client.25366 fetch_fsmap learned FSMap version 26 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.473+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8db1f1660 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88b6a00640 1 -- 172.21.15.27:0/2067624490 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c40ad420 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88b6a00640 1 -- 172.21.15.27:0/2067624490 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88c40d3070 con 0x55d8dadfc0a0 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88e24d6740 10 client.25366 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88e24d6740 10 client.25366 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:12:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88e24d6740 20 client.25366 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dadb8c90 0x55d8db257bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88e24d6740 4 client.25366 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "status"}) -- 0x55d8db140750 con 0x55d8dadb8c90 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.474+0000 7f88d3e00640 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dadb8c90 0x55d8db257bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.475+0000 7f88d3e00640 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dadb8c90 0x55d8db257bb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5255d867bdfb21 server_cookie=afe433559e3e58d3 in_seq=0 out_seq=0 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.475+0000 7f88b6a00640 10 client.25366 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.475+0000 7f88b6a00640 1 -- 172.21.15.27:0/2067624490 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d8db140750 con 0x55d8dadb8c90 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.475+0000 7f88b6a00640 10 client.25366 handle_command_reply: tid=0 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.475+0000 7f88e24d6740 1 client.25366 shutdown 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.475+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 msgr2=0x55d8db1f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.475+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1f1ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88c40ad8a0 tx=0x7f88c4005760 comp rx=0 tx=0).stop 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 shutdown_connections 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8dadb8c90 0x55d8db257bb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8dae24f60 0x55d8db1eb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8dae04280 0x55d8db1f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88b6a00640 0 client.25366 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 --2- 172.21.15.27:0/2067624490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8dadfc0a0 0x55d8db1f1ec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 >> 172.21.15.27:0/2067624490 conn(0x55d8dae19350 msgr2=0x55d8db1c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 shutdown_connections 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 1 -- 172.21.15.27:0/2067624490 wait complete. 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.476+0000 7f88e24d6740 20 client.25366 trim_cache size 0 max 16384 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24773, 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 160.13538477500001, 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 26, 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 174.29269597800001 2024-09-21T07:12:37.484 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88bc0b6560 msgr2=0x7f88bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88bc0b6560 0x7f88bc0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88cc020000 tx=0x7f88cc021000 comp rx=0 tx=0).stop 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d4075750 msgr2=0x7f88d419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d4075750 0x7f88d419c590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e9f0 tx=0x7f88c400edf0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 shutdown_connections 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d410ff70 0x7f88d41a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88bc0b6560 0x7f88bc0b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d40760a0 0x7f88d419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 --2- 172.21.15.27:0/304484061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88d4075750 0x7f88d419c590 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:37.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 >> 172.21.15.27:0/304484061 conn(0x7f88d40ff400 msgr2=0x7f88d4109b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:37.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 shutdown_connections 2024-09-21T07:12:37.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:37.477+0000 7f88da200640 1 -- 172.21.15.27:0/304484061 wait complete. 2024-09-21T07:12:38.382 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T07:12:38.399 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T07:12:38.399 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T07:12:38.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:38.399 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T07:12:38.436 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T07:12:38.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:38.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:12:38.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:38.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-21T07:12:38.819 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-21T07:12:38.819 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:12:38.820 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:12:38.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T07:12:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:38 smithi027 ceph-mon[30779]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-21T07:12:39.208 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:38 smithi119 ceph-mon[34519]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-21T07:12:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:38 smithi153 ceph-mon[34747]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-21T07:12:39.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 -- 172.21.15.27:0/4178467803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 msgr2=0x7f603010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:39.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 --2- 172.21.15.27:0/4178467803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 0x7f603010bad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f601c009900 tx=0x7f601c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:39.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 -- 172.21.15.27:0/4178467803 shutdown_connections 2024-09-21T07:12:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 --2- 172.21.15.27:0/4178467803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f603010c010 0x7f6030112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 --2- 172.21.15.27:0/4178467803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 0x7f603010bad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 --2- 172.21.15.27:0/4178467803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6030105d80 0x7f6030106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 -- 172.21.15.27:0/4178467803 >> 172.21.15.27:0/4178467803 conn(0x7f6030101810 msgr2=0x7f6030103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.817+0000 7f6038000640 1 -- 172.21.15.27:0/4178467803 shutdown_connections 2024-09-21T07:12:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.818+0000 7f6038000640 1 -- 172.21.15.27:0/4178467803 wait complete. 2024-09-21T07:12:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 Processor -- start 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 -- start start 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6030105d80 0x7f603019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 0x7f603019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f603010c010 0x7f60301a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6030115cd0 con 0x7f6030105d80 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6030115b50 con 0x7f603010b6c0 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.819+0000 7f6038000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6030115e50 con 0x7f603010c010 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6035600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6030105d80 0x7f603019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6035600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6030105d80 0x7f603019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41506/0 (socket says 172.21.15.27:41506) 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6035600640 1 -- 172.21.15.27:0/1346041206 learned_addr learned my addr 172.21.15.27:0/1346041206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6036000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f603010c010 0x7f60301a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6034c00640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 0x7f603019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6036000640 1 -- 172.21.15.27:0/1346041206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 msgr2=0x7f603019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6036000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 0x7f603019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6036000640 1 -- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6030105d80 msgr2=0x7f603019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6036000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6030105d80 0x7f603019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6036000640 1 -- 172.21.15.27:0/1346041206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301a2630 con 0x7f603010c010 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6035600640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6030105d80 0x7f603019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.820+0000 7f6034c00640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 0x7f603019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.821+0000 7f6036000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f603010c010 0x7f60301a1f00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6028009fd0 tx=0x7f602800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.821+0000 7f602e000640 1 -- 172.21.15.27:0/1346041206 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602800ce20 con 0x7f603010c010 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.821+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60301a2920 con 0x7f603010c010 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.821+0000 7f602e000640 1 -- 172.21.15.27:0/1346041206 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6028005020 con 0x7f603010c010 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.821+0000 7f602e000640 1 -- 172.21.15.27:0/1346041206 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028010690 con 0x7f603010c010 2024-09-21T07:12:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.821+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a9080 con 0x7f603010c010 2024-09-21T07:12:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.822+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6030106190 con 0x7f603010c010 2024-09-21T07:12:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.825+0000 7f602e000640 1 -- 172.21.15.27:0/1346041206 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f60280054d0 con 0x7f603010c010 2024-09-21T07:12:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.826+0000 7f602e000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60180b6560 0x7f60180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.827+0000 7f602e000640 1 -- 172.21.15.27:0/1346041206 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f60280e7ee0 con 0x7f603010c010 2024-09-21T07:12:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.827+0000 7f6035600640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60180b6560 0x7f60180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.827+0000 7f6035600640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60180b6560 0x7f60180b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6024011500 tx=0x7f6024005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:39.828+0000 7f602e000640 1 -- 172.21.15.27:0/1346041206 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f60280ae2a0 con 0x7f603010c010 2024-09-21T07:12:40.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.063+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f603019d7f0 con 0x7f603010c010 2024-09-21T07:12:40.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.064+0000 7f602e000640 1 -- 172.21.15.27:0/1346041206 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7f603019d7f0 con 0x7f603010c010 2024-09-21T07:12:40.066 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T07:12:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60180b6560 msgr2=0x7f60180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60180b6560 0x7f60180b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6024011500 tx=0x7f6024005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f603010c010 msgr2=0x7f60301a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f603010c010 0x7f60301a1f00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6028009fd0 tx=0x7f602800ef10 comp rx=0 tx=0).stop 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 shutdown_connections 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f603010c010 0x7f60301a1f00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60180b6560 0x7f60180b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f603010b6c0 0x7f603019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 --2- 172.21.15.27:0/1346041206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6030105d80 0x7f603019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.067+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 >> 172.21.15.27:0/1346041206 conn(0x7f6030101810 msgr2=0x7f6030103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.068+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 shutdown_connections 2024-09-21T07:12:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:40.068+0000 7f6038000640 1 -- 172.21.15.27:0/1346041206 wait complete. 2024-09-21T07:12:40.878 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:12:40.879 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:12:40.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:40.879 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-21T07:12:40.916 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:12:40.916 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:12:40.916 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:12:40.916 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:12:40.916 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-21T07:12:40.916 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-21T07:12:40.917 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:12:40.918 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41987sec preferred_lft 41987sec 2024-09-21T07:12:40.918 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T07:12:40.918 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:12:40.918 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:12:40.918 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-21T07:12:40.918 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-21T07:12:40.919 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:12:40.919 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:12:40.919 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:12:40.919 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:12:40.919 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-09-21T07:12:40.919 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-09-21T07:12:40.919 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:12:40.919 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:12:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:12:41.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:40 smithi027 ceph-mon[30779]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T07:12:41.095 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:40 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1346041206' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T07:12:41.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:41.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:41.153 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:12:41.210 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-21T07:12:41.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:41.213 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-21T07:12:41.251 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-21T07:12:41.251 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:12:41.251 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:12:41.251 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:12:41.251 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:12:41.251 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:12:41.251 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:12:41.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:12:41.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:12:41.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:12:41.252 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:12:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:40 smithi119 ceph-mon[34519]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T07:12:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:40 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1346041206' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T07:12:41.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:12:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:40 smithi153 ceph-mon[34747]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T07:12:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:40 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1346041206' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T07:12:41.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:41.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:41.455 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T07:12:41.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:41.492 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-21T07:12:41.568 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:12:41.568 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:12:41.568 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:12:41.568 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T07:12:41.568 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:12:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:12:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:41.751 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T07:12:41.751 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:12:41.751 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:12:41.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T07:12:41.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:12:41.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T07:12:41.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T07:12:41.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T07:12:41.752 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:12:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:12:41.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:41.975 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:12:41.975 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T07:12:41.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-21T07:12:41.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:12:41.975 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T07:12:42.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:12:42.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:42.144 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:12:42.144 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:12:42.180 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:12:42.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:12:42.181 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:12:42.258 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T07:12:42.258 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T07:12:42.258 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/c9dc85fb-de11-40b4-8b5e-6af2fcaf6fa0 2024-09-21T07:12:42.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:42.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/c9dc85fb-de11-40b4-8b5e-6af2fcaf6fa0 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-21T07:12:42.463 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-21T07:12:42.463 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-21T07:12:42.463 INFO:teuthology.orchestra.run.smithi027.stdout:invalid new device string format 2024-09-21T07:12:43.071 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-21T07:12:43.071 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-21T07:12:43.071 INFO:teuthology.orchestra.run.smithi027.stdout:invalid new device string format 2024-09-21T07:12:43.071 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:6789,172.21.15.119:6789,172.21.15.153:6789" 2024-09-21T07:12:43.071 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.27:6789,172.21.15.119:6789,172.21.15.153:6789:/volumes/qa/sv_1/c9dc85fb-de11-40b4-8b5e-6af2fcaf6fa0 2024-09-21T07:12:43.071 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=0ff1e276-77e7-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T07:12:43.073 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:12:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:12:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:12:43.073 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:12:43.074 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:12:43.074 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:42 smithi027 ceph-mon[30779]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:43.076 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:12:43.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:12:43.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:12:43.137 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:12:43.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:12:43.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:43.231 INFO:teuthology.orchestra.run.smithi027.stdout:client25375 2024-09-21T07:12:43.231 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:12:43.231 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:12:43.268 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:12:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:42 smithi119 ceph-mon[34519]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:43.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:12:43.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:43.405 INFO:teuthology.orchestra.run.smithi027.stdout:client25375 2024-09-21T07:12:43.405 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:12:43.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:42 smithi153 ceph-mon[34747]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:12:43.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:12:43.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:43.497 INFO:teuthology.orchestra.run.smithi027.stdout:client25375 2024-09-21T07:12:43.498 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490.client25375/status') 2024-09-21T07:12:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:12:43.601 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.25375 (3)172.21.15.27:0/3196375262 2024-09-21T07:12:43.601 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-09-21T07:12:43.601 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-09-21T07:12:43.602 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-09-21T07:12:43.602 INFO:teuthology.orchestra.run.smithi027.stderr:71 bytes copied, 6.1193e-05 s, 1.2 MB/s 2024-09-21T07:12:43.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:12:43.612 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T07:12:43.622 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T07:12:43.622 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:12:43.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 -- 172.21.15.27:0/3094419916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 msgr2=0x7fe700109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 --2- 172.21.15.27:0/3094419916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 0x7fe700109640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009900 tx=0x7fe6e802a410 comp rx=0 tx=0).stop 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 -- 172.21.15.27:0/3094419916 shutdown_connections 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 --2- 172.21.15.27:0/3094419916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700109b80 0x7fe7001103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 --2- 172.21.15.27:0/3094419916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 0x7fe700109640 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 --2- 172.21.15.27:0/3094419916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 0x7fe700103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 -- 172.21.15.27:0/3094419916 >> 172.21.15.27:0/3094419916 conn(0x7fe7000ff360 msgr2=0x7fe7001017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:43.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 -- 172.21.15.27:0/3094419916 shutdown_connections 2024-09-21T07:12:43.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.848+0000 7fe705c00640 1 -- 172.21.15.27:0/3094419916 wait complete. 2024-09-21T07:12:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.849+0000 7fe705c00640 1 Processor -- start 2024-09-21T07:12:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.849+0000 7fe705c00640 1 -- start start 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe705c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 0x7fe70019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe705c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 0x7fe70019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe705c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700109b80 0x7fe7001a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe705c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe700113840 con 0x7fe700109230 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe705c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001136c0 con 0x7fe700109b80 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe705c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001139c0 con 0x7fe700103900 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 0x7fe70019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 0x7fe70019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 0x7fe70019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36274/0 (socket says 172.21.15.27:36274) 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 -- 172.21.15.27:0/2795653973 learned_addr learned my addr 172.21.15.27:0/2795653973 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe705200640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700109b80 0x7fe7001a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 -- 172.21.15.27:0/2795653973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700109b80 msgr2=0x7fe7001a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700109b80 0x7fe7001a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 -- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 msgr2=0x7fe70019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 0x7fe70019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6ffe00640 1 -- 172.21.15.27:0/2795653973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7001a2410 con 0x7fe700103900 2024-09-21T07:12:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.850+0000 7fe6f7e00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 0x7fe70019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.851+0000 7fe6ffe00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 0x7fe70019c4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e9f0 tx=0x7fe6f000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.851+0000 7fe6fd600640 1 -- 172.21.15.27:0/2795653973 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000ccc0 con 0x7fe700103900 2024-09-21T07:12:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.851+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7001a2700 con 0x7fe700103900 2024-09-21T07:12:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.851+0000 7fe6fd600640 1 -- 172.21.15.27:0/2795653973 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f000ce60 con 0x7fe700103900 2024-09-21T07:12:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.851+0000 7fe6fd600640 1 -- 172.21.15.27:0/2795653973 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0010430 con 0x7fe700103900 2024-09-21T07:12:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.851+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7001a8e90 con 0x7fe700103900 2024-09-21T07:12:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.852+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe700103d10 con 0x7fe700103900 2024-09-21T07:12:43.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.855+0000 7fe6fd600640 1 -- 172.21.15.27:0/2795653973 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fe6f00046c0 con 0x7fe700103900 2024-09-21T07:12:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.856+0000 7fe6fd600640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6e40b6560 0x7fe6e40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.857+0000 7fe6fd600640 1 -- 172.21.15.27:0/2795653973 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe6f00e7e30 con 0x7fe700103900 2024-09-21T07:12:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.857+0000 7fe6f7e00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6e40b6560 0x7fe6e40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.857+0000 7fe6f7e00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6e40b6560 0x7fe6e40b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009230 tx=0x7fe6e8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:43.858+0000 7fe6fd600640 1 -- 172.21.15.27:0/2795653973 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6f0056710 con 0x7fe700103900 2024-09-21T07:12:44.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.096+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fe70019d600 con 0x7fe700103900 2024-09-21T07:12:44.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.941+0000 7fe6fd600640 1 -- 172.21.15.27:0/2795653973 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7fe70019d600 con 0x7fe700103900 2024-09-21T07:12:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6e40b6560 msgr2=0x7fe6e40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6e40b6560 0x7fe6e40b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009230 tx=0x7fe6e8031400 comp rx=0 tx=0).stop 2024-09-21T07:12:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 msgr2=0x7fe70019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 0x7fe70019c4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e9f0 tx=0x7fe6f000edf0 comp rx=0 tx=0).stop 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 shutdown_connections 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6e40b6560 0x7fe6e40b8a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700109b80 0x7fe7001a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe700109230 0x7fe70019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 --2- 172.21.15.27:0/2795653973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe700103900 0x7fe70019c4e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 >> 172.21.15.27:0/2795653973 conn(0x7fe7000ff360 msgr2=0x7fe70010e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 shutdown_connections 2024-09-21T07:12:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:44.945+0000 7fe705c00640 1 -- 172.21.15.27:0/2795653973 wait complete. 2024-09-21T07:12:44.978 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T07:12:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 -- 172.21.15.27:0/83184615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce410b620 msgr2=0x7f9ce410ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 --2- 172.21.15.27:0/83184615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce410b620 0x7f9ce410ba30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009860 tx=0x7f9ccc02a340 comp rx=0 tx=0).stop 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 -- 172.21.15.27:0/83184615 shutdown_connections 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 --2- 172.21.15.27:0/83184615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce410b620 0x7f9ce410ba30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 --2- 172.21.15.27:0/83184615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce410bf70 0x7f9ce41127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 --2- 172.21.15.27:0/83184615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce4105cf0 0x7f9ce4106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 -- 172.21.15.27:0/83184615 >> 172.21.15.27:0/83184615 conn(0x7f9ce41016d0 msgr2=0x7f9ce4103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.208+0000 7f9ce9200640 1 -- 172.21.15.27:0/83184615 shutdown_connections 2024-09-21T07:12:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.209+0000 7f9ce9200640 1 -- 172.21.15.27:0/83184615 wait complete. 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 Processor -- start 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 -- start start 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce4105cf0 0x7f9ce419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce410b620 0x7f9ce419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce410bf70 0x7f9ce41a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4115c30 con 0x7f9ce410bf70 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4115ab0 con 0x7f9ce410b620 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce9200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4115db0 con 0x7f9ce4105cf0 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce4105cf0 0x7f9ce419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce410bf70 0x7f9ce41a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce4105cf0 0x7f9ce419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36292/0 (socket says 172.21.15.27:36292) 2024-09-21T07:12:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.210+0000 7f9ce1e00640 1 -- 172.21.15.27:0/2711609517 learned_addr learned my addr 172.21.15.27:0/2711609517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9cdbe00640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce410b620 0x7f9ce419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9ce1e00640 1 -- 172.21.15.27:0/2711609517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce410b620 msgr2=0x7f9ce419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9ce1e00640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce410b620 0x7f9ce419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9ce1e00640 1 -- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce410bf70 msgr2=0x7f9ce41a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9ce1e00640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce410bf70 0x7f9ce41a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9ce1e00640 1 -- 172.21.15.27:0/2711609517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce41a25c0 con 0x7f9ce4105cf0 2024-09-21T07:12:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9ce1e00640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce4105cf0 0x7f9ce419c690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9f0 tx=0x7f9cd400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.211+0000 7f9cdaa00640 1 -- 172.21.15.27:0/2711609517 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400cd80 con 0x7f9ce4105cf0 2024-09-21T07:12:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.212+0000 7f9cdaa00640 1 -- 172.21.15.27:0/2711609517 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd40054d0 con 0x7f9ce4105cf0 2024-09-21T07:12:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.212+0000 7f9cdaa00640 1 -- 172.21.15.27:0/2711609517 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4010690 con 0x7f9ce4105cf0 2024-09-21T07:12:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.212+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce41a28b0 con 0x7f9ce4105cf0 2024-09-21T07:12:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.212+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce41a9040 con 0x7f9ce4105cf0 2024-09-21T07:12:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.213+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce4106100 con 0x7f9ce4105cf0 2024-09-21T07:12:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.215+0000 7f9cdaa00640 1 -- 172.21.15.27:0/2711609517 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f9cd4004e30 con 0x7f9ce4105cf0 2024-09-21T07:12:45.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.217+0000 7f9cdaa00640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cc80b65e0 0x7f9cc80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.217+0000 7f9cdaa00640 1 -- 172.21.15.27:0/2711609517 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9cd40e7db0 con 0x7f9ce4105cf0 2024-09-21T07:12:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.217+0000 7f9cdbe00640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cc80b65e0 0x7f9cc80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.218+0000 7f9cdbe00640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cc80b65e0 0x7f9cc80b8af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc035040 tx=0x7f9ccc030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.218+0000 7f9cdaa00640 1 -- 172.21.15.27:0/2711609517 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9cd40ae170 con 0x7f9ce4105cf0 2024-09-21T07:12:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:44 smithi119 ceph-mon[34519]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:44 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2795653973' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T07:12:45.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:44 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:44 smithi027 ceph-mon[30779]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:44 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2795653973' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:44 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T07:12:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:44 smithi153 ceph-mon[34747]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T07:12:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:44 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2795653973' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T07:12:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:44 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T07:12:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.451+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f9ce4108480 con 0x7f9cc80b65e0 2024-09-21T07:12:45.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.474+0000 7f9cdaa00640 1 -- 172.21.15.27:0/2711609517 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f9ce4108480 con 0x7f9cc80b65e0 2024-09-21T07:12:45.477 INFO:teuthology.orchestra.run.smithi027.stdout:{} 2024-09-21T07:12:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cc80b65e0 msgr2=0x7f9cc80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cc80b65e0 0x7f9cc80b8af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc035040 tx=0x7f9ccc030340 comp rx=0 tx=0).stop 2024-09-21T07:12:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce4105cf0 msgr2=0x7f9ce419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce4105cf0 0x7f9ce419c690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9f0 tx=0x7f9cd400edf0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 shutdown_connections 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cc80b65e0 0x7f9cc80b8af0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce410bf70 0x7f9ce41a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ce410b620 0x7f9ce419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 --2- 172.21.15.27:0/2711609517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce4105cf0 0x7f9ce419c690 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 >> 172.21.15.27:0/2711609517 conn(0x7f9ce41016d0 msgr2=0x7f9ce4107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 shutdown_connections 2024-09-21T07:12:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.477+0000 7f9ce9200640 1 -- 172.21.15.27:0/2711609517 wait complete. 2024-09-21T07:12:45.502 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T07:12:45.513 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T07:12:45.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:12:45.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 -- 172.21.15.27:0/1158942577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109d50 msgr2=0x7f07c0110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 --2- 172.21.15.27:0/1158942577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109d50 0x7f07c0110580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009ca0 tx=0x7f07b802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 -- 172.21.15.27:0/1158942577 shutdown_connections 2024-09-21T07:12:45.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 --2- 172.21.15.27:0/1158942577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109d50 0x7f07c0110580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 --2- 172.21.15.27:0/1158942577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0109400 0x7f07c0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 --2- 172.21.15.27:0/1158942577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0103ad0 0x7f07c0103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 -- 172.21.15.27:0/1158942577 >> 172.21.15.27:0/1158942577 conn(0x7f07c00ff510 msgr2=0x7f07c0101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:45.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 -- 172.21.15.27:0/1158942577 shutdown_connections 2024-09-21T07:12:45.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.753+0000 7f07c4c00640 1 -- 172.21.15.27:0/1158942577 wait complete. 2024-09-21T07:12:45.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.754+0000 7f07c4c00640 1 Processor -- start 2024-09-21T07:12:45.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07c4c00640 1 -- start start 2024-09-21T07:12:45.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07c4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0103ad0 0x7f07c019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07c4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109400 0x7f07c019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07c4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0109d50 0x7f07c01a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07c4c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0113a10 con 0x7f07c0109d50 2024-09-21T07:12:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07c4c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0113890 con 0x7f07c0103ad0 2024-09-21T07:12:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07c4c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0113b90 con 0x7f07c0109400 2024-09-21T07:12:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07be800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0109d50 0x7f07c01a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07b7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109400 0x7f07c019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07be800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0109d50 0x7f07c01a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41562/0 (socket says 172.21.15.27:41562) 2024-09-21T07:12:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07be800640 1 -- 172.21.15.27:0/3696147061 learned_addr learned my addr 172.21.15.27:0/3696147061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.755+0000 7f07bde00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0103ad0 0x7f07c019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07be800640 1 -- 172.21.15.27:0/3696147061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109400 msgr2=0x7f07c019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07be800640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109400 0x7f07c019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07be800640 1 -- 172.21.15.27:0/3696147061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0103ad0 msgr2=0x7f07c019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07be800640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0103ad0 0x7f07c019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07be800640 1 -- 172.21.15.27:0/3696147061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c01a2500 con 0x7f07c0109d50 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07be800640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0109d50 0x7f07c01a1e00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f07b80098d0 tx=0x7f07b80317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003b70 con 0x7f07c0109d50 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8003d10 con 0x7f07c0109d50 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c01a2790 con 0x7f07c0109d50 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.756+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b802fa20 con 0x7f07c0109d50 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.757+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c01a8f80 con 0x7f07c0109d50 2024-09-21T07:12:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.758+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c0104d40 con 0x7f07c0109d50 2024-09-21T07:12:45.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.760+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f07b802fc40 con 0x7f07c0109d50 2024-09-21T07:12:45.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.761+0000 7f07b6a00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07a40b65e0 0x7f07a40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:45.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.761+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f07b8108a10 con 0x7f07c0109d50 2024-09-21T07:12:45.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.762+0000 7f07bde00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07a40b65e0 0x7f07a40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.762+0000 7f07bde00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07a40b65e0 0x7f07a40b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009bd0 tx=0x7f07b0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:45.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.766+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07b80cedd0 con 0x7f07c0109d50 2024-09-21T07:12:45.841 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:12:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:12:45.457+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:12:45.841 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:12:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:12:45.457+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:12:45.841 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:12:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:12:45.457+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:12:45.841 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:12:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:12:45.457+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:12:45.841 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:12:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:12:45.457+0000 7f7623c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:12:45.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.960+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f07b80ca940 con 0x7f07c0109d50 2024-09-21T07:12:45.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.972+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07c01a91e0 con 0x7f07c0109d50 2024-09-21T07:12:45.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.974+0000 7f07b6a00640 1 -- 172.21.15.27:0/3696147061 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25234 (secure 0 0 0) 0x7f07b80812b0 con 0x7f07c0109d50 2024-09-21T07:12:45.976 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:45.976 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:11:55.826198+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10},{"from":5,"to":1}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:12:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07a40b65e0 msgr2=0x7f07a40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07a40b65e0 0x7f07a40b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009bd0 tx=0x7f07b0005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0109d50 msgr2=0x7f07c01a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0109d50 0x7f07c01a1e00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f07b80098d0 tx=0x7f07b80317b0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 shutdown_connections 2024-09-21T07:12:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0109400 0x7f07c019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07a40b65e0 0x7f07a40b8af0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c0109d50 0x7f07c01a1e00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 --2- 172.21.15.27:0/3696147061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0103ad0 0x7f07c019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 >> 172.21.15.27:0/3696147061 conn(0x7f07c00ff510 msgr2=0x7f07c0106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 shutdown_connections 2024-09-21T07:12:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:45.976+0000 7f07c4c00640 1 -- 172.21.15.27:0/3696147061 wait complete. 2024-09-21T07:12:46.004 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:04:59.755438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T07:09:50.469679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T07:09:53.416727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T07:10:03.398488+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T07:12:46.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:12:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.238+0000 7fab87000640 1 -- 172.21.15.27:0/122535501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801038b0 msgr2=0x7fab80103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.238+0000 7fab87000640 1 --2- 172.21.15.27:0/122535501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801038b0 0x7fab80103cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fab74009900 tx=0x7fab7402a410 comp rx=0 tx=0).stop 2024-09-21T07:12:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.238+0000 7fab87000640 1 -- 172.21.15.27:0/122535501 shutdown_connections 2024-09-21T07:12:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.238+0000 7fab87000640 1 --2- 172.21.15.27:0/122535501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab801091e0 0x7fab801095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.238+0000 7fab87000640 1 --2- 172.21.15.27:0/122535501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80109b30 0x7fab80110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.238+0000 7fab87000640 1 --2- 172.21.15.27:0/122535501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801038b0 0x7fab80103cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.238+0000 7fab87000640 1 -- 172.21.15.27:0/122535501 >> 172.21.15.27:0/122535501 conn(0x7fab800ff2d0 msgr2=0x7fab80101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.239+0000 7fab87000640 1 -- 172.21.15.27:0/122535501 shutdown_connections 2024-09-21T07:12:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.239+0000 7fab87000640 1 -- 172.21.15.27:0/122535501 wait complete. 2024-09-21T07:12:46.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 Processor -- start 2024-09-21T07:12:46.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 -- start start 2024-09-21T07:12:46.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab801038b0 0x7fab80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801091e0 0x7fab80075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab80109b30 0x7fab80076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab80113840 con 0x7fab801091e0 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab801136c0 con 0x7fab801038b0 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.240+0000 7fab87000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab801139c0 con 0x7fab80109b30 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab801038b0 0x7fab80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801091e0 0x7fab80075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab801038b0 0x7fab80075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58634/0 (socket says 172.21.15.27:58634) 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 -- 172.21.15.27:0/289825055 learned_addr learned my addr 172.21.15.27:0/289825055 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab86600640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab80109b30 0x7fab80076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 -- 172.21.15.27:0/289825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab80109b30 msgr2=0x7fab80076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab80109b30 0x7fab80076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 -- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801091e0 msgr2=0x7fab80075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801091e0 0x7fab80075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 -- 172.21.15.27:0/289825055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab801a8530 con 0x7fab801038b0 2024-09-21T07:12:46.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.241+0000 7fab85c00640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab801038b0 0x7fab80075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab74005ab0 tx=0x7fab74003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.242+0000 7fab7e000640 1 -- 172.21.15.27:0/289825055 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7403f070 con 0x7fab801038b0 2024-09-21T07:12:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.242+0000 7fab7e000640 1 -- 172.21.15.27:0/289825055 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74031b60 con 0x7fab801038b0 2024-09-21T07:12:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.242+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab801a87c0 con 0x7fab801038b0 2024-09-21T07:12:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.242+0000 7fab7e000640 1 -- 172.21.15.27:0/289825055 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74035030 con 0x7fab801038b0 2024-09-21T07:12:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.242+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab801a8cb0 con 0x7fab801038b0 2024-09-21T07:12:46.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.243+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab80103cc0 con 0x7fab801038b0 2024-09-21T07:12:46.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.245+0000 7fab7e000640 1 -- 172.21.15.27:0/289825055 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fab74042070 con 0x7fab801038b0 2024-09-21T07:12:46.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.247+0000 7fab7e000640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab680b6620 0x7fab680b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.247+0000 7fab7e000640 1 -- 172.21.15.27:0/289825055 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab74108470 con 0x7fab801038b0 2024-09-21T07:12:46.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.247+0000 7fab85200640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab680b6620 0x7fab680b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.248+0000 7fab85200640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab680b6620 0x7fab680b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fab80108d40 tx=0x7fab6c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:46.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.249+0000 7fab7e000640 1 -- 172.21.15.27:0/289825055 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab740d2ab0 con 0x7fab801038b0 2024-09-21T07:12:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:45 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T07:12:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:45 smithi119 ceph-mon[34519]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:12:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:45 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T07:12:46.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:45 smithi027 ceph-mon[30779]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:12:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:45 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T07:12:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:45 smithi153 ceph-mon[34747]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T07:12:46.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.447+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fab8002e640 con 0x7fab801038b0 2024-09-21T07:12:46.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.448+0000 7fab7e000640 1 -- 172.21.15.27:0/289825055 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7fab740d79f0 con 0x7fab801038b0 2024-09-21T07:12:46.450 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab680b6620 msgr2=0x7fab680b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab680b6620 0x7fab680b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fab80108d40 tx=0x7fab6c005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab801038b0 msgr2=0x7fab80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab801038b0 0x7fab80075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab74005ab0 tx=0x7fab74003c10 comp rx=0 tx=0).stop 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 shutdown_connections 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab80109b30 0x7fab80076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab680b6620 0x7fab680b8b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab801091e0 0x7fab80075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 --2- 172.21.15.27:0/289825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab801038b0 0x7fab80075710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 >> 172.21.15.27:0/289825055 conn(0x7fab800ff2d0 msgr2=0x7fab801016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 shutdown_connections 2024-09-21T07:12:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.451+0000 7fab87000640 1 -- 172.21.15.27:0/289825055 wait complete. 2024-09-21T07:12:46.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T07:12:46.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 -- 172.21.15.27:0/691834814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4105cd0 msgr2=0x7fb8a41060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 --2- 172.21.15.27:0/691834814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4105cd0 0x7fb8a41060e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb894009900 tx=0x7fb89402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 -- 172.21.15.27:0/691834814 shutdown_connections 2024-09-21T07:12:46.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 --2- 172.21.15.27:0/691834814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a410b600 0x7fb8a410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 --2- 172.21.15.27:0/691834814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a410bf50 0x7fb8a4112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 --2- 172.21.15.27:0/691834814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4105cd0 0x7fb8a41060e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 -- 172.21.15.27:0/691834814 >> 172.21.15.27:0/691834814 conn(0x7fb8a4101710 msgr2=0x7fb8a4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:46.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 -- 172.21.15.27:0/691834814 shutdown_connections 2024-09-21T07:12:46.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.711+0000 7fb8aa400640 1 -- 172.21.15.27:0/691834814 wait complete. 2024-09-21T07:12:46.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.712+0000 7fb8aa400640 1 Processor -- start 2024-09-21T07:12:46.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.712+0000 7fb8aa400640 1 -- start start 2024-09-21T07:12:46.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8aa400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a4105cd0 0x7fb8a419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8aa400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a410b600 0x7fb8a419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8aa400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a410bf50 0x7fb8a41a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8aa400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4115c60 con 0x7fb8a410bf50 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8aa400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4115ae0 con 0x7fb8a4105cd0 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8aa400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4115de0 con 0x7fb8a410b600 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a410bf50 0x7fb8a41a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8a3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a4105cd0 0x7fb8a419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a410bf50 0x7fb8a41a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41618/0 (socket says 172.21.15.27:41618) 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8a3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a4105cd0 0x7fb8a419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58656/0 (socket says 172.21.15.27:58656) 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb8a3e00640 1 -- 172.21.15.27:0/534666811 learned_addr learned my addr 172.21.15.27:0/534666811 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.713+0000 7fb89aa00640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a410b600 0x7fb8a419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a3e00640 1 -- 172.21.15.27:0/534666811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a410b600 msgr2=0x7fb8a419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a3e00640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a410b600 0x7fb8a419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a3e00640 1 -- 172.21.15.27:0/534666811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a4105cd0 msgr2=0x7fb8a419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a3e00640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a4105cd0 0x7fb8a419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a3e00640 1 -- 172.21.15.27:0/534666811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a41a2590 con 0x7fb8a410bf50 2024-09-21T07:12:46.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a3400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a4105cd0 0x7fb8a419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:46.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a3e00640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a410bf50 0x7fb8a41a1e90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00efd0 tx=0x7fb89c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:46.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a0c00640 1 -- 172.21.15.27:0/534666811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c003b70 con 0x7fb8a410bf50 2024-09-21T07:12:46.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a41a2880 con 0x7fb8a410bf50 2024-09-21T07:12:46.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a0c00640 1 -- 172.21.15.27:0/534666811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb89c003d10 con 0x7fb8a410bf50 2024-09-21T07:12:46.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8a0c00640 1 -- 172.21.15.27:0/534666811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c0108a0 con 0x7fb8a410bf50 2024-09-21T07:12:46.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.714+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a41a9010 con 0x7fb8a410bf50 2024-09-21T07:12:46.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.715+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a41060e0 con 0x7fb8a410bf50 2024-09-21T07:12:46.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.718+0000 7fb8a0c00640 1 -- 172.21.15.27:0/534666811 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fb89c004b40 con 0x7fb8a410bf50 2024-09-21T07:12:46.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.720+0000 7fb8a0c00640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb8880b65f0 0x7fb8880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:46.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.720+0000 7fb8a0c00640 1 -- 172.21.15.27:0/534666811 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb89c0e81f0 con 0x7fb8a410bf50 2024-09-21T07:12:46.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.720+0000 7fb8a3400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb8880b65f0 0x7fb8880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:46.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.721+0000 7fb8a3400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb8880b65f0 0x7fb8880b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb894005ab0 tx=0x7fb894005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:46.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.722+0000 7fb8a0c00640 1 -- 172.21.15.27:0/534666811 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb89c0ae450 con 0x7fb8a410bf50 2024-09-21T07:12:46.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.924+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8a402e640 con 0x7fb8a410bf50 2024-09-21T07:12:46.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.925+0000 7fb8a0c00640 1 -- 172.21.15.27:0/534666811 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7fb89c0b2830 con 0x7fb8a410bf50 2024-09-21T07:12:46.927 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb8880b65f0 msgr2=0x7fb8880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb8880b65f0 0x7fb8880b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb894005ab0 tx=0x7fb894005a20 comp rx=0 tx=0).stop 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a410bf50 msgr2=0x7fb8a41a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a410bf50 0x7fb8a41a1e90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00efd0 tx=0x7fb89c00c6a0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 shutdown_connections 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a410b600 0x7fb8a419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb8880b65f0 0x7fb8880b8b00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a410bf50 0x7fb8a41a1e90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 --2- 172.21.15.27:0/534666811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a4105cd0 0x7fb8a419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.927+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 >> 172.21.15.27:0/534666811 conn(0x7fb8a4101710 msgr2=0x7fb8a4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.928+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 shutdown_connections 2024-09-21T07:12:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:46.928+0000 7fb8aa400640 1 -- 172.21.15.27:0/534666811 wait complete. 2024-09-21T07:12:46.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T07:12:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 -- 172.21.15.27:0/2352423033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f010b410 msgr2=0x7f02f010b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 --2- 172.21.15.27:0/2352423033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f010b410 0x7f02f010b820 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009860 tx=0x7f02d802a340 comp rx=0 tx=0).stop 2024-09-21T07:12:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 -- 172.21.15.27:0/2352423033 shutdown_connections 2024-09-21T07:12:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 --2- 172.21.15.27:0/2352423033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f010bd60 0x7f02f0112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 --2- 172.21.15.27:0/2352423033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f010b410 0x7f02f010b820 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 --2- 172.21.15.27:0/2352423033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f0105ae0 0x7f02f0105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 -- 172.21.15.27:0/2352423033 >> 172.21.15.27:0/2352423033 conn(0x7f02f0101560 msgr2=0x7f02f01039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:47.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 -- 172.21.15.27:0/2352423033 shutdown_connections 2024-09-21T07:12:47.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.187+0000 7f02f5400640 1 -- 172.21.15.27:0/2352423033 wait complete. 2024-09-21T07:12:47.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.188+0000 7f02f5400640 1 Processor -- start 2024-09-21T07:12:47.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.188+0000 7f02f5400640 1 -- start start 2024-09-21T07:12:47.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02f5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f0105ae0 0x7f02f019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02f5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f010b410 0x7f02f019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02f5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f010bd60 0x7f02f01a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02f5400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0115a20 con 0x7f02f010bd60 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02f5400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01158a0 con 0x7f02f0105ae0 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02f5400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0115ba0 con 0x7f02f010b410 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f0105ae0 0x7f02f019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f0105ae0 0x7f02f019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58676/0 (socket says 172.21.15.27:58676) 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 -- 172.21.15.27:0/3745511497 learned_addr learned my addr 172.21.15.27:0/3745511497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02f4a00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f010bd60 0x7f02f01a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02e7e00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f010b410 0x7f02f019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 -- 172.21.15.27:0/3745511497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f010b410 msgr2=0x7f02f019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f010b410 0x7f02f019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 -- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f010bd60 msgr2=0x7f02f01a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f010bd60 0x7f02f01a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02efe00640 1 -- 172.21.15.27:0/3745511497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f01a2370 con 0x7f02f0105ae0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.189+0000 7f02e7e00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f010b410 0x7f02f019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.190+0000 7f02efe00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f0105ae0 0x7f02f019c440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02e000e9c0 tx=0x7f02e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.190+0000 7f02ed600640 1 -- 172.21.15.27:0/3745511497 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e000cde0 con 0x7f02f0105ae0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.190+0000 7f02ed600640 1 -- 172.21.15.27:0/3745511497 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e00035f0 con 0x7f02f0105ae0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.190+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f01a2660 con 0x7f02f0105ae0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.190+0000 7f02ed600640 1 -- 172.21.15.27:0/3745511497 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0010690 con 0x7f02f0105ae0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.190+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f01a8df0 con 0x7f02f0105ae0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.191+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f0105ef0 con 0x7f02f0105ae0 2024-09-21T07:12:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.194+0000 7f02ed600640 1 -- 172.21.15.27:0/3745511497 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f02e0004000 con 0x7f02f0105ae0 2024-09-21T07:12:47.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.195+0000 7f02ed600640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02d40b6670 0x7f02d40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.195+0000 7f02ed600640 1 -- 172.21.15.27:0/3745511497 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f02e00e7dc0 con 0x7f02f0105ae0 2024-09-21T07:12:47.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.195+0000 7f02e7e00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02d40b6670 0x7f02d40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.196+0000 7f02e7e00640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02d40b6670 0x7f02d40b8b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009830 tx=0x7f02d80058b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:47.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.197+0000 7f02ed600640 1 -- 172.21.15.27:0/3745511497 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02e00ae0b0 con 0x7f02f0105ae0 2024-09-21T07:12:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:46 smithi119 ceph-mon[34519]: pgmap v370: 129 pgs: 129 active+clean; 651 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T07:12:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:46 smithi119 ceph-mon[34519]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:46 smithi119 ceph-mon[34519]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T07:12:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:46 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3696147061' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:46 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/289825055' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:12:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:46 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/534666811' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:12:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:46 smithi027 ceph-mon[30779]: pgmap v370: 129 pgs: 129 active+clean; 651 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T07:12:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:46 smithi027 ceph-mon[30779]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:46 smithi027 ceph-mon[30779]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T07:12:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:46 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3696147061' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:46 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/289825055' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:12:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:46 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/534666811' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:12:47.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.392+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f02f002e640 con 0x7f02f0105ae0 2024-09-21T07:12:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.393+0000 7f02ed600640 1 -- 172.21.15.27:0/3745511497 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f02e00b2490 con 0x7f02f0105ae0 2024-09-21T07:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.395+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02d40b6670 msgr2=0x7f02d40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.395+0000 7f02f5400640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02d40b6670 0x7f02d40b8b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009830 tx=0x7f02d80058b0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f0105ae0 msgr2=0x7f02f019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f0105ae0 0x7f02f019c440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02e000e9c0 tx=0x7f02e000ee80 comp rx=0 tx=0).stop 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 shutdown_connections 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f010b410 0x7f02f019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02d40b6670 0x7f02d40b8b80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f010bd60 0x7f02f01a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 --2- 172.21.15.27:0/3745511497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02f0105ae0 0x7f02f019c440 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 >> 172.21.15.27:0/3745511497 conn(0x7f02f0101560 msgr2=0x7f02f0103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:47.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 shutdown_connections 2024-09-21T07:12:47.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.396+0000 7f02f5400640 1 -- 172.21.15.27:0/3745511497 wait complete. 2024-09-21T07:12:47.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T07:12:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:46 smithi153 ceph-mon[34747]: pgmap v370: 129 pgs: 129 active+clean; 651 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T07:12:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:46 smithi153 ceph-mon[34747]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:46 smithi153 ceph-mon[34747]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T07:12:47.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:46 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3696147061' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:47.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:46 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/289825055' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:12:47.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:46 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/534666811' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:12:47.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.654+0000 7fc7fcc00640 1 -- 172.21.15.27:0/3451776806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 msgr2=0x7fc7f81060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.654+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/3451776806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f81060e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009900 tx=0x7fc7e802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.655+0000 7fc7fcc00640 1 -- 172.21.15.27:0/3451776806 shutdown_connections 2024-09-21T07:12:47.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.655+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/3451776806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7f810bf50 0x7fc7f8112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.655+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/3451776806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f810b600 0x7fc7f810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.655+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/3451776806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f81060e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.655+0000 7fc7fcc00640 1 -- 172.21.15.27:0/3451776806 >> 172.21.15.27:0/3451776806 conn(0x7fc7f8101710 msgr2=0x7fc7f8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:47.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.655+0000 7fc7fcc00640 1 -- 172.21.15.27:0/3451776806 shutdown_connections 2024-09-21T07:12:47.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.655+0000 7fc7fcc00640 1 -- 172.21.15.27:0/3451776806 wait complete. 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 Processor -- start 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 -- start start 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f810b600 0x7fc7f819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7f810bf50 0x7fc7f81a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8115c60 con 0x7fc7f810b600 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8115ae0 con 0x7fc7f8105cd0 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.656+0000 7fc7fcc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8115de0 con 0x7fc7f810bf50 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f810b600 0x7fc7f819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58682/0 (socket says 172.21.15.27:58682) 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 -- 172.21.15.27:0/224439137 learned_addr learned my addr 172.21.15.27:0/224439137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:47.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f6800640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7f810bf50 0x7fc7f81a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 -- 172.21.15.27:0/224439137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7f810bf50 msgr2=0x7fc7f81a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7f810bf50 0x7fc7f81a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 -- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f810b600 msgr2=0x7fc7f819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f810b600 0x7fc7f819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 -- 172.21.15.27:0/224439137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81a2570 con 0x7fc7f8105cd0 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.657+0000 7fc7f5e00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f819c640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8005ab0 tx=0x7fc7e8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.658+0000 7fc7eea00640 1 -- 172.21.15.27:0/224439137 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e803f070 con 0x7fc7f8105cd0 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.658+0000 7fc7eea00640 1 -- 172.21.15.27:0/224439137 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e8031b60 con 0x7fc7f8105cd0 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.658+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81a2800 con 0x7fc7f8105cd0 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.658+0000 7fc7eea00640 1 -- 172.21.15.27:0/224439137 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8035030 con 0x7fc7f8105cd0 2024-09-21T07:12:47.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.658+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81a8ff0 con 0x7fc7f8105cd0 2024-09-21T07:12:47.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.659+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f81060e0 con 0x7fc7f8105cd0 2024-09-21T07:12:47.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.661+0000 7fc7eea00640 1 -- 172.21.15.27:0/224439137 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fc7e8042070 con 0x7fc7f8105cd0 2024-09-21T07:12:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.663+0000 7fc7eea00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7dc0b6670 0x7fc7dc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.663+0000 7fc7eea00640 1 -- 172.21.15.27:0/224439137 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc7e8108820 con 0x7fc7f8105cd0 2024-09-21T07:12:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.663+0000 7fc7efe00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7dc0b6670 0x7fc7dc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:47.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.664+0000 7fc7efe00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7dc0b6670 0x7fc7dc0b8b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00095f0 tx=0x7fc7e0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:47.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.665+0000 7fc7eea00640 1 -- 172.21.15.27:0/224439137 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7e80d2e60 con 0x7fc7f8105cd0 2024-09-21T07:12:47.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.863+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc7f810dd80 con 0x7fc7f8105cd0 2024-09-21T07:12:47.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.864+0000 7fc7eea00640 1 -- 172.21.15.27:0/224439137 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) ==== 103+0+11 (secure 0 0 0) 0x7fc7e80d7da0 con 0x7fc7f8105cd0 2024-09-21T07:12:47.866 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7dc0b6670 msgr2=0x7fc7dc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7dc0b6670 0x7fc7dc0b8b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00095f0 tx=0x7fc7e0005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 msgr2=0x7fc7f819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f819c640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8005ab0 tx=0x7fc7e8003c10 comp rx=0 tx=0).stop 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 shutdown_connections 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7f810bf50 0x7fc7f81a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7dc0b6670 0x7fc7dc0b8b80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f810b600 0x7fc7f819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 --2- 172.21.15.27:0/224439137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7f8105cd0 0x7fc7f819c640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 >> 172.21.15.27:0/224439137 conn(0x7fc7f8101710 msgr2=0x7fc7f810ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 shutdown_connections 2024-09-21T07:12:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:47.867+0000 7fc7fcc00640 1 -- 172.21.15.27:0/224439137 wait complete. 2024-09-21T07:12:47.891 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T07:12:47.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:12:48.277 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:47 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3745511497' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:12:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:47 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/224439137' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:12:48.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:47 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3745511497' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:12:48.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:47 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/224439137' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:12:48.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:47 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3745511497' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:12:48.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:47 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/224439137' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:12:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 -- 172.21.15.27:0/1823402007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810b4f0 msgr2=0x7f62b810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 --2- 172.21.15.27:0/1823402007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810b4f0 0x7f62b810b900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009860 tx=0x7f62a402a340 comp rx=0 tx=0).stop 2024-09-21T07:12:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 -- 172.21.15.27:0/1823402007 shutdown_connections 2024-09-21T07:12:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 --2- 172.21.15.27:0/1823402007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b810be40 0x7f62b8112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 --2- 172.21.15.27:0/1823402007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810b4f0 0x7f62b810b900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 --2- 172.21.15.27:0/1823402007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 0x7f62b8105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 -- 172.21.15.27:0/1823402007 >> 172.21.15.27:0/1823402007 conn(0x7f62b8101680 msgr2=0x7f62b8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 -- 172.21.15.27:0/1823402007 shutdown_connections 2024-09-21T07:12:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.931+0000 7f62bcc00640 1 -- 172.21.15.27:0/1823402007 wait complete. 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.932+0000 7f62bcc00640 1 Processor -- start 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.932+0000 7f62bcc00640 1 -- start start 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.932+0000 7f62bcc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 0x7f62b819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.932+0000 7f62bcc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b810b4f0 0x7f62b819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.932+0000 7f62bcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810be40 0x7f62b81a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62bcc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8115b00 con 0x7f62b8105bb0 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62bcc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8115980 con 0x7f62b810be40 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62bcc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8115c80 con 0x7f62b810b4f0 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 0x7f62b819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 0x7f62b819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45634/0 (socket says 172.21.15.27:45634) 2024-09-21T07:12:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 -- 172.21.15.27:0/4070899656 learned_addr learned my addr 172.21.15.27:0/4070899656 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7e00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810be40 0x7f62b81a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b6a00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b810b4f0 0x7f62b819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 -- 172.21.15.27:0/4070899656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b810b4f0 msgr2=0x7f62b819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b810b4f0 0x7f62b819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 -- 172.21.15.27:0/4070899656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810be40 msgr2=0x7f62b81a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810be40 0x7f62b81a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.933+0000 7f62b7400640 1 -- 172.21.15.27:0/4070899656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b81a24a0 con 0x7f62b8105bb0 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.934+0000 7f62b7400640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 0x7f62b819c530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9c0 tx=0x7f62a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.934+0000 7f62afe00640 1 -- 172.21.15.27:0/4070899656 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a800ccc0 con 0x7f62b8105bb0 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.934+0000 7f62afe00640 1 -- 172.21.15.27:0/4070899656 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a80054d0 con 0x7f62b8105bb0 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.934+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b81a2790 con 0x7f62b8105bb0 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.934+0000 7f62afe00640 1 -- 172.21.15.27:0/4070899656 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8010430 con 0x7f62b8105bb0 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.934+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b81a8ef0 con 0x7f62b8105bb0 2024-09-21T07:12:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.935+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b8105fc0 con 0x7f62b8105bb0 2024-09-21T07:12:48.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.938+0000 7f62afe00640 1 -- 172.21.15.27:0/4070899656 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f62a8004f80 con 0x7f62b8105bb0 2024-09-21T07:12:48.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.939+0000 7f62afe00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f629c0b6670 0x7f629c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:48.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.939+0000 7f62afe00640 1 -- 172.21.15.27:0/4070899656 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f62a80e7d20 con 0x7f62b8105bb0 2024-09-21T07:12:48.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.939+0000 7f62b6a00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f629c0b6670 0x7f629c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:48.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.940+0000 7f62b6a00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f629c0b6670 0x7f629c0b8b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62a4005940 tx=0x7f62a40058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:48.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:48.943+0000 7f62afe00640 1 -- 172.21.15.27:0/4070899656 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62a80adfe0 con 0x7f62b8105bb0 2024-09-21T07:12:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:48 smithi027 ceph-mon[30779]: pgmap v371: 129 pgs: 129 active+clean; 652 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-21T07:12:49.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.184+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b802e640 con 0x7f62b8105bb0 2024-09-21T07:12:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.185+0000 7f62afe00640 1 -- 172.21.15.27:0/4070899656 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f62a80b23c0 con 0x7f62b8105bb0 2024-09-21T07:12:49.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:49.187 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:12:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f629c0b6670 msgr2=0x7f629c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f629c0b6670 0x7f629c0b8b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62a4005940 tx=0x7f62a40058d0 comp rx=0 tx=0).stop 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 msgr2=0x7f62b819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 0x7f62b819c530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9c0 tx=0x7f62a800ee80 comp rx=0 tx=0).stop 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 shutdown_connections 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b810b4f0 0x7f62b819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f629c0b6670 0x7f629c0b8b80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b810be40 0x7f62b81a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 --2- 172.21.15.27:0/4070899656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b8105bb0 0x7f62b819c530 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 >> 172.21.15.27:0/4070899656 conn(0x7f62b8101680 msgr2=0x7f62b8103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 shutdown_connections 2024-09-21T07:12:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:49.188+0000 7f62bcc00640 1 -- 172.21.15.27:0/4070899656 wait complete. 2024-09-21T07:12:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:48 smithi119 ceph-mon[34519]: pgmap v371: 129 pgs: 129 active+clean; 652 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-21T07:12:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:48 smithi153 ceph-mon[34747]: pgmap v371: 129 pgs: 129 active+clean; 652 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-21T07:12:49.978 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T07:12:49.978 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T07:12:49.978 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:12:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 -- 172.21.15.27:0/3904817232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 msgr2=0x7fd794105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 --2- 172.21.15.27:0/3904817232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd794105ef0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd784009900 tx=0x7fd78402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 -- 172.21.15.27:0/3904817232 shutdown_connections 2024-09-21T07:12:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 --2- 172.21.15.27:0/3904817232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd79410b410 0x7fd79410b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 --2- 172.21.15.27:0/3904817232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd79410bd60 0x7fd794112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 --2- 172.21.15.27:0/3904817232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd794105ef0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 -- 172.21.15.27:0/3904817232 >> 172.21.15.27:0/3904817232 conn(0x7fd794101560 msgr2=0x7fd7941039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 -- 172.21.15.27:0/3904817232 shutdown_connections 2024-09-21T07:12:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.211+0000 7fd799e00640 1 -- 172.21.15.27:0/3904817232 wait complete. 2024-09-21T07:12:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.212+0000 7fd799e00640 1 Processor -- start 2024-09-21T07:12:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.212+0000 7fd799e00640 1 -- start start 2024-09-21T07:12:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd799e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd79419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd799e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd79410b410 0x7fd79419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd799e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd79410bd60 0x7fd7941a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd799e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd794115a70 con 0x7fd794105ae0 2024-09-21T07:12:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd799e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941158f0 con 0x7fd79410bd60 2024-09-21T07:12:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd799e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd794115bf0 con 0x7fd79410b410 2024-09-21T07:12:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd798a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd79419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd798a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd79419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45648/0 (socket says 172.21.15.27:45648) 2024-09-21T07:12:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd798a00640 1 -- 172.21.15.27:0/1450961341 learned_addr learned my addr 172.21.15.27:0/1450961341 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd78be00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd79410b410 0x7fd79419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd799400640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd79410bd60 0x7fd7941a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.575 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:49 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4070899656' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:50.575 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:49 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4070899656' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd798a00640 1 -- 172.21.15.27:0/1450961341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd79410b410 msgr2=0x7fd79419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd798a00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd79410b410 0x7fd79419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd798a00640 1 -- 172.21.15.27:0/1450961341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd79410bd60 msgr2=0x7fd7941a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.213+0000 7fd798a00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd79410bd60 0x7fd7941a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd798a00640 1 -- 172.21.15.27:0/1450961341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7941a23c0 con 0x7fd794105ae0 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd799400640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd79410bd60 0x7fd7941a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd798a00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd79419c490 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd784005ab0 tx=0x7fd784003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd792000640 1 -- 172.21.15.27:0/1450961341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78403f070 con 0x7fd794105ae0 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd792000640 1 -- 172.21.15.27:0/1450961341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd784031b60 con 0x7fd794105ae0 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7941a2650 con 0x7fd794105ae0 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd792000640 1 -- 172.21.15.27:0/1450961341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784035030 con 0x7fd794105ae0 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.214+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7941a8e40 con 0x7fd794105ae0 2024-09-21T07:12:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.216+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd794105ef0 con 0x7fd794105ae0 2024-09-21T07:12:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.218+0000 7fd792000640 1 -- 172.21.15.27:0/1450961341 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fd784042070 con 0x7fd794105ae0 2024-09-21T07:12:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.219+0000 7fd792000640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7780b6670 0x7fd7780b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.220+0000 7fd792000640 1 -- 172.21.15.27:0/1450961341 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd7841080e0 con 0x7fd794105ae0 2024-09-21T07:12:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.220+0000 7fd78be00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7780b6670 0x7fd7780b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.220+0000 7fd78be00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7780b6670 0x7fd7780b8b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd794104db0 tx=0x7fd77c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.223+0000 7fd792000640 1 -- 172.21.15.27:0/1450961341 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7840ce340 con 0x7fd794105ae0 2024-09-21T07:12:50.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:49 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4070899656' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.420+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd794063800 con 0x7fd7780b6670 2024-09-21T07:12:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.427+0000 7fd792000640 1 -- 172.21.15.27:0/1450961341 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340476 (secure 0 0 0) 0x7fd794063800 con 0x7fd7780b6670 2024-09-21T07:12:50.583 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7780b6670 msgr2=0x7fd7780b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7780b6670 0x7fd7780b8b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd794104db0 tx=0x7fd77c005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 msgr2=0x7fd79419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd79419c490 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd784005ab0 tx=0x7fd784003c10 comp rx=0 tx=0).stop 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 shutdown_connections 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd79410b410 0x7fd79419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd7780b6670 0x7fd7780b8b80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd79410bd60 0x7fd7941a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 --2- 172.21.15.27:0/1450961341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd794105ae0 0x7fd79419c490 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 >> 172.21.15.27:0/1450961341 conn(0x7fd794101560 msgr2=0x7fd79410aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 shutdown_connections 2024-09-21T07:12:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.432+0000 7fd799e00640 1 -- 172.21.15.27:0/1450961341 wait complete. 2024-09-21T07:12:50.587 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-21T07:12:49.414487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670572,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":812,"num_read_kb":518,"num_write":313,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392208,"kb_used_data":13840,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549616,"statfs":{"total":1151940427776,"available":1151538806784,"internally_reserved":0,"allocated":14172160,"data_stored":8174400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9367,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":48,"num_write":15,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003468"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.831114+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:11:55.831114+0000","last_peered":"2024-09-21T07:11:55.831114+0000","last_clean":"2024-09-21T07:11:55.831114+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:11:55.831114+0000","last_undegraded":"2024-09-21T07:11:55.831114+0000","last_fullsized":"2024-09-21T07:11:55.831114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805255+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:12:23.805255+0000","last_peered":"2024-09-21T07:12:23.805255+0000","last_clean":"2024-09-21T07:12:23.805255+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:12:23.805255+0000","last_undegraded":"2024-09-21T07:12:23.805255+0000","last_fullsized":"2024-09-21T07:12:23.805255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832485+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:11:55.832485+0000","last_peered":"2024-09-21T07:11:55.832485+0000","last_clean":"2024-09-21T07:11:55.832485+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:11:55.832485+0000","last_undegraded":"2024-09-21T07:11:55.832485+0000","last_fullsized":"2024-09-21T07:11:55.832485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833749+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:11:55.833749+0000","last_peered":"2024-09-21T07:11:55.833749+0000","last_clean":"2024-09-21T07:11:55.833749+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:11:55.833749+0000","last_undegraded":"2024-09-21T07:11:55.833749+0000","last_fullsized":"2024-09-21T07:11:55.833749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811697+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:12:23.811697+0000","last_peered":"2024-09-21T07:12:23.811697+0000","last_clean":"2024-09-21T07:12:23.811697+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:12:23.811697+0000","last_undegraded":"2024-09-21T07:12:23.811697+0000","last_fullsized":"2024-09-21T07:12:23.811697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844771+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:11:55.844771+0000","last_peered":"2024-09-21T07:11:55.844771+0000","last_clean":"2024-09-21T07:11:55.844771+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:11:55.844771+0000","last_undegraded":"2024-09-21T07:11:55.844771+0000","last_fullsized":"2024-09-21T07:11:55.844771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847516+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:11:55.847516+0000","last_peered":"2024-09-21T07:11:55.847516+0000","last_clean":"2024-09-21T07:11:55.847516+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:11:55.847516+0000","last_undegraded":"2024-09-21T07:11:55.847516+0000","last_fullsized":"2024-09-21T07:11:55.847516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814377+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:12:23.814377+0000","last_peered":"2024-09-21T07:12:23.814377+0000","last_clean":"2024-09-21T07:12:23.814377+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:12:23.814377+0000","last_undegraded":"2024-09-21T07:12:23.814377+0000","last_fullsized":"2024-09-21T07:12:23.814377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845066+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:11:55.845066+0000","last_peered":"2024-09-21T07:11:55.845066+0000","last_clean":"2024-09-21T07:11:55.845066+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:11:55.845066+0000","last_undegraded":"2024-09-21T07:11:55.845066+0000","last_fullsized":"2024-09-21T07:11:55.845066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847357+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:11:55.847357+0000","last_peered":"2024-09-21T07:11:55.847357+0000","last_clean":"2024-09-21T07:11:55.847357+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:11:55.847357+0000","last_undegraded":"2024-09-21T07:11:55.847357+0000","last_fullsized":"2024-09-21T07:11:55.847357+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803507+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:12:23.803507+0000","last_peered":"2024-09-21T07:12:23.803507+0000","last_clean":"2024-09-21T07:12:23.803507+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:12:23.803507+0000","last_undegraded":"2024-09-21T07:12:23.803507+0000","last_fullsized":"2024-09-21T07:12:23.803507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844870+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:11:55.844870+0000","last_peered":"2024-09-21T07:11:55.844870+0000","last_clean":"2024-09-21T07:11:55.844870+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:11:55.844870+0000","last_undegraded":"2024-09-21T07:11:55.844870+0000","last_fullsized":"2024-09-21T07:11:55.844870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813396+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:12:23.813396+0000","last_peered":"2024-09-21T07:12:23.813396+0000","last_clean":"2024-09-21T07:12:23.813396+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:12:23.813396+0000","last_undegraded":"2024-09-21T07:12:23.813396+0000","last_fullsized":"2024-09-21T07:12:23.813396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846961+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:11:55.846961+0000","last_peered":"2024-09-21T07:11:55.846961+0000","last_clean":"2024-09-21T07:11:55.846961+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:11:55.846961+0000","last_undegraded":"2024-09-21T07:11:55.846961+0000","last_fullsized":"2024-09-21T07:11:55.846961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845137+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:11:55.845137+0000","last_peered":"2024-09-21T07:11:55.845137+0000","last_clean":"2024-09-21T07:11:55.845137+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:11:55.845137+0000","last_undegraded":"2024-09-21T07:11:55.845137+0000","last_fullsized":"2024-09-21T07:11:55.845137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807789+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:12:23.807789+0000","last_peered":"2024-09-21T07:12:23.807789+0000","last_clean":"2024-09-21T07:12:23.807789+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:12:23.807789+0000","last_undegraded":"2024-09-21T07:12:23.807789+0000","last_fullsized":"2024-09-21T07:12:23.807789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832340+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:11:55.832340+0000","last_peered":"2024-09-21T07:11:55.832340+0000","last_clean":"2024-09-21T07:11:55.832340+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:11:55.832340+0000","last_undegraded":"2024-09-21T07:11:55.832340+0000","last_fullsized":"2024-09-21T07:11:55.832340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845070+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:11:55.845070+0000","last_peered":"2024-09-21T07:11:55.845070+0000","last_clean":"2024-09-21T07:11:55.845070+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:11:55.845070+0000","last_undegraded":"2024-09-21T07:11:55.845070+0000","last_fullsized":"2024-09-21T07:11:55.845070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798453+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:12:23.798453+0000","last_peered":"2024-09-21T07:12:23.798453+0000","last_clean":"2024-09-21T07:12:23.798453+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:12:23.798453+0000","last_undegraded":"2024-09-21T07:12:23.798453+0000","last_fullsized":"2024-09-21T07:12:23.798453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833325+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:11:55.833325+0000","last_peered":"2024-09-21T07:11:55.833325+0000","last_clean":"2024-09-21T07:11:55.833325+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:11:55.833325+0000","last_undegraded":"2024-09-21T07:11:55.833325+0000","last_fullsized":"2024-09-21T07:11:55.833325+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830920+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:11:55.830920+0000","last_peered":"2024-09-21T07:11:55.830920+0000","last_clean":"2024-09-21T07:11:55.830920+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:11:55.830920+0000","last_undegraded":"2024-09-21T07:11:55.830920+0000","last_fullsized":"2024-09-21T07:11:55.830920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.780871+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:12:23.780871+0000","last_peered":"2024-09-21T07:12:23.780871+0000","last_clean":"2024-09-21T07:12:23.780871+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:12:23.780871+0000","last_undegraded":"2024-09-21T07:12:23.780871+0000","last_fullsized":"2024-09-21T07:12:23.780871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833051+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:11:55.833051+0000","last_peered":"2024-09-21T07:11:55.833051+0000","last_clean":"2024-09-21T07:11:55.833051+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:11:55.833051+0000","last_undegraded":"2024-09-21T07:11:55.833051+0000","last_fullsized":"2024-09-21T07:11:55.833051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845144+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:11:55.845144+0000","last_peered":"2024-09-21T07:11:55.845144+0000","last_clean":"2024-09-21T07:11:55.845144+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:11:55.845144+0000","last_undegraded":"2024-09-21T07:11:55.845144+0000","last_fullsized":"2024-09-21T07:11:55.845144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.800049+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:12:23.800049+0000","last_peered":"2024-09-21T07:12:23.800049+0000","last_clean":"2024-09-21T07:12:23.800049+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:12:23.800049+0000","last_undegraded":"2024-09-21T07:12:23.800049+0000","last_fullsized":"2024-09-21T07:12:23.800049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845369+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:11:55.845369+0000","last_peered":"2024-09-21T07:11:55.845369+0000","last_clean":"2024-09-21T07:11:55.845369+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:11:55.845369+0000","last_undegraded":"2024-09-21T07:11:55.845369+0000","last_fullsized":"2024-09-21T07:11:55.845369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808689+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:12:23.808689+0000","last_peered":"2024-09-21T07:12:23.808689+0000","last_clean":"2024-09-21T07:12:23.808689+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:12:23.808689+0000","last_undegraded":"2024-09-21T07:12:23.808689+0000","last_fullsized":"2024-09-21T07:12:23.808689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'13","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:43.372588+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:12:43.372588+0000","last_peered":"2024-09-21T07:12:43.372588+0000","last_clean":"2024-09-21T07:12:43.372588+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:12:43.372588+0000","last_undegraded":"2024-09-21T07:12:43.372588+0000","last_fullsized":"2024-09-21T07:12:43.372588+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832993+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:11:55.832993+0000","last_peered":"2024-09-21T07:11:55.832993+0000","last_clean":"2024-09-21T07:11:55.832993+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:11:55.832993+0000","last_undegraded":"2024-09-21T07:11:55.832993+0000","last_fullsized":"2024-09-21T07:11:55.832993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809315+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:12:23.809315+0000","last_peered":"2024-09-21T07:12:23.809315+0000","last_clean":"2024-09-21T07:12:23.809315+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:12:23.809315+0000","last_undegraded":"2024-09-21T07:12:23.809315+0000","last_fullsized":"2024-09-21T07:12:23.809315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847542+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:11:55.847542+0000","last_peered":"2024-09-21T07:11:55.847542+0000","last_clean":"2024-09-21T07:11:55.847542+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:11:55.847542+0000","last_undegraded":"2024-09-21T07:11:55.847542+0000","last_fullsized":"2024-09-21T07:11:55.847542+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790825+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:12:23.790825+0000","last_peered":"2024-09-21T07:12:23.790825+0000","last_clean":"2024-09-21T07:12:23.790825+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:12:23.790825+0000","last_undegraded":"2024-09-21T07:12:23.790825+0000","last_fullsized":"2024-09-21T07:12:23.790825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830831+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:11:55.830831+0000","last_peered":"2024-09-21T07:11:55.830831+0000","last_clean":"2024-09-21T07:11:55.830831+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:11:55.830831+0000","last_undegraded":"2024-09-21T07:11:55.830831+0000","last_fullsized":"2024-09-21T07:11:55.830831+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846866+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:11:55.846866+0000","last_peered":"2024-09-21T07:11:55.846866+0000","last_clean":"2024-09-21T07:11:55.846866+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:11:55.846866+0000","last_undegraded":"2024-09-21T07:11:55.846866+0000","last_fullsized":"2024-09-21T07:11:55.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.794745+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:12:23.794745+0000","last_peered":"2024-09-21T07:12:23.794745+0000","last_clean":"2024-09-21T07:12:23.794745+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:12:23.794745+0000","last_undegraded":"2024-09-21T07:12:23.794745+0000","last_fullsized":"2024-09-21T07:12:23.794745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981709+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:11:55.981709+0000","last_peered":"2024-09-21T07:11:55.981709+0000","last_clean":"2024-09-21T07:11:55.981709+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:11:55.981709+0000","last_undegraded":"2024-09-21T07:11:55.981709+0000","last_fullsized":"2024-09-21T07:11:55.981709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845966+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:11:55.845966+0000","last_peered":"2024-09-21T07:11:55.845966+0000","last_clean":"2024-09-21T07:11:55.845966+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:11:55.845966+0000","last_undegraded":"2024-09-21T07:11:55.845966+0000","last_fullsized":"2024-09-21T07:11:55.845966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795850+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:12:23.795850+0000","last_peered":"2024-09-21T07:12:23.795850+0000","last_clean":"2024-09-21T07:12:23.795850+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:12:23.795850+0000","last_undegraded":"2024-09-21T07:12:23.795850+0000","last_fullsized":"2024-09-21T07:12:23.795850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:45.470360+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:12:45.470360+0000","last_peered":"2024-09-21T07:12:45.470360+0000","last_clean":"2024-09-21T07:12:45.470360+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:12:45.470360+0000","last_undegraded":"2024-09-21T07:12:45.470360+0000","last_fullsized":"2024-09-21T07:12:45.470360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846521+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:11:55.846521+0000","last_peered":"2024-09-21T07:11:55.846521+0000","last_clean":"2024-09-21T07:11:55.846521+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:11:55.846521+0000","last_undegraded":"2024-09-21T07:11:55.846521+0000","last_fullsized":"2024-09-21T07:11:55.846521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807529+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:12:23.807529+0000","last_peered":"2024-09-21T07:12:23.807529+0000","last_clean":"2024-09-21T07:12:23.807529+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:12:23.807529+0000","last_undegraded":"2024-09-21T07:12:23.807529+0000","last_fullsized":"2024-09-21T07:12:23.807529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981500+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:11:55.981500+0000","last_peered":"2024-09-21T07:11:55.981500+0000","last_clean":"2024-09-21T07:11:55.981500+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:11:55.981500+0000","last_undegraded":"2024-09-21T07:11:55.981500+0000","last_fullsized":"2024-09-21T07:11:55.981500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845592+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:11:55.845592+0000","last_peered":"2024-09-21T07:11:55.845592+0000","last_clean":"2024-09-21T07:11:55.845592+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:11:55.845592+0000","last_undegraded":"2024-09-21T07:11:55.845592+0000","last_fullsized":"2024-09-21T07:11:55.845592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811077+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:12:23.811077+0000","last_peered":"2024-09-21T07:12:23.811077+0000","last_clean":"2024-09-21T07:12:23.811077+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:12:23.811077+0000","last_undegraded":"2024-09-21T07:12:23.811077+0000","last_fullsized":"2024-09-21T07:12:23.811077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845392+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:11:55.845392+0000","last_peered":"2024-09-21T07:11:55.845392+0000","last_clean":"2024-09-21T07:11:55.845392+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:11:55.845392+0000","last_undegraded":"2024-09-21T07:11:55.845392+0000","last_fullsized":"2024-09-21T07:11:55.845392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981606+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:11:55.981606+0000","last_peered":"2024-09-21T07:11:55.981606+0000","last_clean":"2024-09-21T07:11:55.981606+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:11:55.981606+0000","last_undegraded":"2024-09-21T07:11:55.981606+0000","last_fullsized":"2024-09-21T07:11:55.981606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981807+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:11:55.981807+0000","last_peered":"2024-09-21T07:11:55.981807+0000","last_clean":"2024-09-21T07:11:55.981807+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:11:55.981807+0000","last_undegraded":"2024-09-21T07:11:55.981807+0000","last_fullsized":"2024-09-21T07:11:55.981807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814663+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:12:23.814663+0000","last_peered":"2024-09-21T07:12:23.814663+0000","last_clean":"2024-09-21T07:12:23.814663+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:12:23.814663+0000","last_undegraded":"2024-09-21T07:12:23.814663+0000","last_fullsized":"2024-09-21T07:12:23.814663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.834191+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:11:55.834191+0000","last_peered":"2024-09-21T07:11:55.834191+0000","last_clean":"2024-09-21T07:11:55.834191+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:11:55.834191+0000","last_undegraded":"2024-09-21T07:11:55.834191+0000","last_fullsized":"2024-09-21T07:11:55.834191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833166+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:11:55.833166+0000","last_peered":"2024-09-21T07:11:55.833166+0000","last_clean":"2024-09-21T07:11:55.833166+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:11:55.833166+0000","last_undegraded":"2024-09-21T07:11:55.833166+0000","last_fullsized":"2024-09-21T07:11:55.833166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807409+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:12:23.807409+0000","last_peered":"2024-09-21T07:12:23.807409+0000","last_clean":"2024-09-21T07:12:23.807409+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:12:23.807409+0000","last_undegraded":"2024-09-21T07:12:23.807409+0000","last_fullsized":"2024-09-21T07:12:23.807409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846421+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:11:55.846421+0000","last_peered":"2024-09-21T07:11:55.846421+0000","last_clean":"2024-09-21T07:11:55.846421+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:11:55.846421+0000","last_undegraded":"2024-09-21T07:11:55.846421+0000","last_fullsized":"2024-09-21T07:11:55.846421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846714+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:11:55.846714+0000","last_peered":"2024-09-21T07:11:55.846714+0000","last_clean":"2024-09-21T07:11:55.846714+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:11:55.846714+0000","last_undegraded":"2024-09-21T07:11:55.846714+0000","last_fullsized":"2024-09-21T07:11:55.846714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814102+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:12:23.814102+0000","last_peered":"2024-09-21T07:12:23.814102+0000","last_clean":"2024-09-21T07:12:23.814102+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:12:23.814102+0000","last_undegraded":"2024-09-21T07:12:23.814102+0000","last_fullsized":"2024-09-21T07:12:23.814102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846385+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:11:55.846385+0000","last_peered":"2024-09-21T07:11:55.846385+0000","last_clean":"2024-09-21T07:11:55.846385+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:11:55.846385+0000","last_undegraded":"2024-09-21T07:11:55.846385+0000","last_fullsized":"2024-09-21T07:11:55.846385+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845182+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:11:55.845182+0000","last_peered":"2024-09-21T07:11:55.845182+0000","last_clean":"2024-09-21T07:11:55.845182+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:11:55.845182+0000","last_undegraded":"2024-09-21T07:11:55.845182+0000","last_fullsized":"2024-09-21T07:11:55.845182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801647+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:12:23.801647+0000","last_peered":"2024-09-21T07:12:23.801647+0000","last_clean":"2024-09-21T07:12:23.801647+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:12:23.801647+0000","last_undegraded":"2024-09-21T07:12:23.801647+0000","last_fullsized":"2024-09-21T07:12:23.801647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832950+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:11:55.832950+0000","last_peered":"2024-09-21T07:11:55.832950+0000","last_clean":"2024-09-21T07:11:55.832950+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:11:55.832950+0000","last_undegraded":"2024-09-21T07:11:55.832950+0000","last_fullsized":"2024-09-21T07:11:55.832950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845860+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:11:55.845860+0000","last_peered":"2024-09-21T07:11:55.845860+0000","last_clean":"2024-09-21T07:11:55.845860+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:11:55.845860+0000","last_undegraded":"2024-09-21T07:11:55.845860+0000","last_fullsized":"2024-09-21T07:11:55.845860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802831+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:12:23.802831+0000","last_peered":"2024-09-21T07:12:23.802831+0000","last_clean":"2024-09-21T07:12:23.802831+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:12:23.802831+0000","last_undegraded":"2024-09-21T07:12:23.802831+0000","last_fullsized":"2024-09-21T07:12:23.802831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846785+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:11:55.846785+0000","last_peered":"2024-09-21T07:11:55.846785+0000","last_clean":"2024-09-21T07:11:55.846785+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:11:55.846785+0000","last_undegraded":"2024-09-21T07:11:55.846785+0000","last_fullsized":"2024-09-21T07:11:55.846785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846243+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:11:55.846243+0000","last_peered":"2024-09-21T07:11:55.846243+0000","last_clean":"2024-09-21T07:11:55.846243+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:11:55.846243+0000","last_undegraded":"2024-09-21T07:11:55.846243+0000","last_fullsized":"2024-09-21T07:11:55.846243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805668+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:12:23.805668+0000","last_peered":"2024-09-21T07:12:23.805668+0000","last_clean":"2024-09-21T07:12:23.805668+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:12:23.805668+0000","last_undegraded":"2024-09-21T07:12:23.805668+0000","last_fullsized":"2024-09-21T07:12:23.805668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":169,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:43.744886+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:12:43.744886+0000","last_peered":"2024-09-21T07:12:43.744886+0000","last_clean":"2024-09-21T07:12:43.744886+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:12:43.744886+0000","last_undegraded":"2024-09-21T07:12:43.744886+0000","last_fullsized":"2024-09-21T07:12:43.744886+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830985+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:11:55.830985+0000","last_peered":"2024-09-21T07:11:55.830985+0000","last_clean":"2024-09-21T07:11:55.830985+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:11:55.830985+0000","last_undegraded":"2024-09-21T07:11:55.830985+0000","last_fullsized":"2024-09-21T07:11:55.830985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813592+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:12:23.813592+0000","last_peered":"2024-09-21T07:12:23.813592+0000","last_clean":"2024-09-21T07:12:23.813592+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:12:23.813592+0000","last_undegraded":"2024-09-21T07:12:23.813592+0000","last_fullsized":"2024-09-21T07:12:23.813592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846256+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:11:55.846256+0000","last_peered":"2024-09-21T07:11:55.846256+0000","last_clean":"2024-09-21T07:11:55.846256+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:11:55.846256+0000","last_undegraded":"2024-09-21T07:11:55.846256+0000","last_fullsized":"2024-09-21T07:11:55.846256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832414+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:11:55.832414+0000","last_peered":"2024-09-21T07:11:55.832414+0000","last_clean":"2024-09-21T07:11:55.832414+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:11:55.832414+0000","last_undegraded":"2024-09-21T07:11:55.832414+0000","last_fullsized":"2024-09-21T07:11:55.832414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808494+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:12:23.808494+0000","last_peered":"2024-09-21T07:12:23.808494+0000","last_clean":"2024-09-21T07:12:23.808494+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:12:23.808494+0000","last_undegraded":"2024-09-21T07:12:23.808494+0000","last_fullsized":"2024-09-21T07:12:23.808494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833515+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:11:55.833515+0000","last_peered":"2024-09-21T07:11:55.833515+0000","last_clean":"2024-09-21T07:11:55.833515+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:11:55.833515+0000","last_undegraded":"2024-09-21T07:11:55.833515+0000","last_fullsized":"2024-09-21T07:11:55.833515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846046+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:11:55.846046+0000","last_peered":"2024-09-21T07:11:55.846046+0000","last_clean":"2024-09-21T07:11:55.846046+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:11:55.846046+0000","last_undegraded":"2024-09-21T07:11:55.846046+0000","last_fullsized":"2024-09-21T07:11:55.846046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798135+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:12:23.798135+0000","last_peered":"2024-09-21T07:12:23.798135+0000","last_clean":"2024-09-21T07:12:23.798135+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:12:23.798135+0000","last_undegraded":"2024-09-21T07:12:23.798135+0000","last_fullsized":"2024-09-21T07:12:23.798135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":333,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:47.753953+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:12:47.753953+0000","last_peered":"2024-09-21T07:12:47.753953+0000","last_clean":"2024-09-21T07:12:47.753953+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:12:47.753953+0000","last_undegraded":"2024-09-21T07:12:47.753953+0000","last_fullsized":"2024-09-21T07:12:47.753953+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":195,"num_write":42,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.848049+0000","last_change":"2024-09-21T07:11:55.847916+0000","last_active":"2024-09-21T07:11:55.848049+0000","last_peered":"2024-09-21T07:11:55.848049+0000","last_clean":"2024-09-21T07:11:55.848049+0000","last_became_active":"2024-09-21T07:11:55.847491+0000","last_became_peered":"2024-09-21T07:11:55.847491+0000","last_unstale":"2024-09-21T07:11:55.848049+0000","last_undegraded":"2024-09-21T07:11:55.848049+0000","last_fullsized":"2024-09-21T07:11:55.848049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:25.702259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,1],"acting":[10,8,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813257+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:12:23.813257+0000","last_peered":"2024-09-21T07:12:23.813257+0000","last_clean":"2024-09-21T07:12:23.813257+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:12:23.813257+0000","last_undegraded":"2024-09-21T07:12:23.813257+0000","last_fullsized":"2024-09-21T07:12:23.813257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846367+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:11:55.846367+0000","last_peered":"2024-09-21T07:11:55.846367+0000","last_clean":"2024-09-21T07:11:55.846367+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:11:55.846367+0000","last_undegraded":"2024-09-21T07:11:55.846367+0000","last_fullsized":"2024-09-21T07:11:55.846367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845245+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:11:55.845245+0000","last_peered":"2024-09-21T07:11:55.845245+0000","last_clean":"2024-09-21T07:11:55.845245+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:11:55.845245+0000","last_undegraded":"2024-09-21T07:11:55.845245+0000","last_fullsized":"2024-09-21T07:11:55.845245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.793771+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:12:23.793771+0000","last_peered":"2024-09-21T07:12:23.793771+0000","last_clean":"2024-09-21T07:12:23.793771+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:12:23.793771+0000","last_undegraded":"2024-09-21T07:12:23.793771+0000","last_fullsized":"2024-09-21T07:12:23.793771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981878+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:11:55.981878+0000","last_peered":"2024-09-21T07:11:55.981878+0000","last_clean":"2024-09-21T07:11:55.981878+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:11:55.981878+0000","last_undegraded":"2024-09-21T07:11:55.981878+0000","last_fullsized":"2024-09-21T07:11:55.981878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830120+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:11:55.830120+0000","last_peered":"2024-09-21T07:11:55.830120+0000","last_clean":"2024-09-21T07:11:55.830120+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:11:55.830120+0000","last_undegraded":"2024-09-21T07:11:55.830120+0000","last_fullsized":"2024-09-21T07:11:55.830120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813258+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:12:23.813258+0000","last_peered":"2024-09-21T07:12:23.813258+0000","last_clean":"2024-09-21T07:12:23.813258+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:12:23.813258+0000","last_undegraded":"2024-09-21T07:12:23.813258+0000","last_fullsized":"2024-09-21T07:12:23.813258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:46.751398+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:12:46.751398+0000","last_peered":"2024-09-21T07:12:46.751398+0000","last_clean":"2024-09-21T07:12:46.751398+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:12:46.751398+0000","last_undegraded":"2024-09-21T07:12:46.751398+0000","last_fullsized":"2024-09-21T07:12:46.751398+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833187+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:11:55.833187+0000","last_peered":"2024-09-21T07:11:55.833187+0000","last_clean":"2024-09-21T07:11:55.833187+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:11:55.833187+0000","last_undegraded":"2024-09-21T07:11:55.833187+0000","last_fullsized":"2024-09-21T07:11:55.833187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847006+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:11:55.847006+0000","last_peered":"2024-09-21T07:11:55.847006+0000","last_clean":"2024-09-21T07:11:55.847006+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:11:55.847006+0000","last_undegraded":"2024-09-21T07:11:55.847006+0000","last_fullsized":"2024-09-21T07:11:55.847006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802309+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:12:23.802309+0000","last_peered":"2024-09-21T07:12:23.802309+0000","last_clean":"2024-09-21T07:12:23.802309+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:12:23.802309+0000","last_undegraded":"2024-09-21T07:12:23.802309+0000","last_fullsized":"2024-09-21T07:12:23.802309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845739+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:11:55.845739+0000","last_peered":"2024-09-21T07:11:55.845739+0000","last_clean":"2024-09-21T07:11:55.845739+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:11:55.845739+0000","last_undegraded":"2024-09-21T07:11:55.845739+0000","last_fullsized":"2024-09-21T07:11:55.845739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849176+0000","last_change":"2024-09-21T07:11:55.848588+0000","last_active":"2024-09-21T07:11:55.849176+0000","last_peered":"2024-09-21T07:11:55.849176+0000","last_clean":"2024-09-21T07:11:55.849176+0000","last_became_active":"2024-09-21T07:11:55.848025+0000","last_became_peered":"2024-09-21T07:11:55.848025+0000","last_unstale":"2024-09-21T07:11:55.849176+0000","last_undegraded":"2024-09-21T07:11:55.849176+0000","last_fullsized":"2024-09-21T07:11:55.849176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:37.413873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810233+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:12:23.810233+0000","last_peered":"2024-09-21T07:12:23.810233+0000","last_clean":"2024-09-21T07:12:23.810233+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:12:23.810233+0000","last_undegraded":"2024-09-21T07:12:23.810233+0000","last_fullsized":"2024-09-21T07:12:23.810233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845457+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:11:55.845457+0000","last_peered":"2024-09-21T07:11:55.845457+0000","last_clean":"2024-09-21T07:11:55.845457+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:11:55.845457+0000","last_undegraded":"2024-09-21T07:11:55.845457+0000","last_fullsized":"2024-09-21T07:11:55.845457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.835562+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:11:55.835562+0000","last_peered":"2024-09-21T07:11:55.835562+0000","last_clean":"2024-09-21T07:11:55.835562+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:11:55.835562+0000","last_undegraded":"2024-09-21T07:11:55.835562+0000","last_fullsized":"2024-09-21T07:11:55.835562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807495+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:12:23.807495+0000","last_peered":"2024-09-21T07:12:23.807495+0000","last_clean":"2024-09-21T07:12:23.807495+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:12:23.807495+0000","last_undegraded":"2024-09-21T07:12:23.807495+0000","last_fullsized":"2024-09-21T07:12:23.807495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849703+0000","last_change":"2024-09-21T07:11:55.848551+0000","last_active":"2024-09-21T07:11:55.849703+0000","last_peered":"2024-09-21T07:11:55.849703+0000","last_clean":"2024-09-21T07:11:55.849703+0000","last_became_active":"2024-09-21T07:11:55.848056+0000","last_became_peered":"2024-09-21T07:11:55.848056+0000","last_unstale":"2024-09-21T07:11:55.849703+0000","last_undegraded":"2024-09-21T07:11:55.849703+0000","last_fullsized":"2024-09-21T07:11:55.849703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:07.779820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":170,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:44.747830+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:12:44.747830+0000","last_peered":"2024-09-21T07:12:44.747830+0000","last_clean":"2024-09-21T07:12:44.747830+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:12:44.747830+0000","last_undegraded":"2024-09-21T07:12:44.747830+0000","last_fullsized":"2024-09-21T07:12:44.747830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788293+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:12:23.788293+0000","last_peered":"2024-09-21T07:12:23.788293+0000","last_clean":"2024-09-21T07:12:23.788293+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:12:23.788293+0000","last_undegraded":"2024-09-21T07:12:23.788293+0000","last_fullsized":"2024-09-21T07:12:23.788293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833541+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:11:55.833541+0000","last_peered":"2024-09-21T07:11:55.833541+0000","last_clean":"2024-09-21T07:11:55.833541+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:11:55.833541+0000","last_undegraded":"2024-09-21T07:11:55.833541+0000","last_fullsized":"2024-09-21T07:11:55.833541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803098+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:12:23.803098+0000","last_peered":"2024-09-21T07:12:23.803098+0000","last_clean":"2024-09-21T07:12:23.803098+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:12:23.803098+0000","last_undegraded":"2024-09-21T07:12:23.803098+0000","last_fullsized":"2024-09-21T07:12:23.803098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807453+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:12:23.807453+0000","last_peered":"2024-09-21T07:12:23.807453+0000","last_clean":"2024-09-21T07:12:23.807453+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:12:23.807453+0000","last_undegraded":"2024-09-21T07:12:23.807453+0000","last_fullsized":"2024-09-21T07:12:23.807453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808124+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:12:23.808124+0000","last_peered":"2024-09-21T07:12:23.808124+0000","last_clean":"2024-09-21T07:12:23.808124+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:12:23.808124+0000","last_undegraded":"2024-09-21T07:12:23.808124+0000","last_fullsized":"2024-09-21T07:12:23.808124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795490+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:12:23.795490+0000","last_peered":"2024-09-21T07:12:23.795490+0000","last_clean":"2024-09-21T07:12:23.795490+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:12:23.795490+0000","last_undegraded":"2024-09-21T07:12:23.795490+0000","last_fullsized":"2024-09-21T07:12:23.795490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805863+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:12:23.805863+0000","last_peered":"2024-09-21T07:12:23.805863+0000","last_clean":"2024-09-21T07:12:23.805863+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:12:23.805863+0000","last_undegraded":"2024-09-21T07:12:23.805863+0000","last_fullsized":"2024-09-21T07:12:23.805863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801102+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:12:23.801102+0000","last_peered":"2024-09-21T07:12:23.801102+0000","last_clean":"2024-09-21T07:12:23.801102+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:12:23.801102+0000","last_undegraded":"2024-09-21T07:12:23.801102+0000","last_fullsized":"2024-09-21T07:12:23.801102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797345+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:12:23.797345+0000","last_peered":"2024-09-21T07:12:23.797345+0000","last_clean":"2024-09-21T07:12:23.797345+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:12:23.797345+0000","last_undegraded":"2024-09-21T07:12:23.797345+0000","last_fullsized":"2024-09-21T07:12:23.797345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806243+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:12:23.806243+0000","last_peered":"2024-09-21T07:12:23.806243+0000","last_clean":"2024-09-21T07:12:23.806243+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:12:23.806243+0000","last_undegraded":"2024-09-21T07:12:23.806243+0000","last_fullsized":"2024-09-21T07:12:23.806243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810287+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:12:23.810287+0000","last_peered":"2024-09-21T07:12:23.810287+0000","last_clean":"2024-09-21T07:12:23.810287+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:12:23.810287+0000","last_undegraded":"2024-09-21T07:12:23.810287+0000","last_fullsized":"2024-09-21T07:12:23.810287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797647+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:12:23.797647+0000","last_peered":"2024-09-21T07:12:23.797647+0000","last_clean":"2024-09-21T07:12:23.797647+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:12:23.797647+0000","last_undegraded":"2024-09-21T07:12:23.797647+0000","last_fullsized":"2024-09-21T07:12:23.797647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.815349+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:12:23.815349+0000","last_peered":"2024-09-21T07:12:23.815349+0000","last_clean":"2024-09-21T07:12:23.815349+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:12:23.815349+0000","last_undegraded":"2024-09-21T07:12:23.815349+0000","last_fullsized":"2024-09-21T07:12:23.815349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809317+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:12:23.809317+0000","last_peered":"2024-09-21T07:12:23.809317+0000","last_clean":"2024-09-21T07:12:23.809317+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:12:23.809317+0000","last_undegraded":"2024-09-21T07:12:23.809317+0000","last_fullsized":"2024-09-21T07:12:23.809317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795846+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:12:23.795846+0000","last_peered":"2024-09-21T07:12:23.795846+0000","last_clean":"2024-09-21T07:12:23.795846+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:12:23.795846+0000","last_undegraded":"2024-09-21T07:12:23.795846+0000","last_fullsized":"2024-09-21T07:12:23.795846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808162+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:12:23.808162+0000","last_peered":"2024-09-21T07:12:23.808162+0000","last_clean":"2024-09-21T07:12:23.808162+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:12:23.808162+0000","last_undegraded":"2024-09-21T07:12:23.808162+0000","last_fullsized":"2024-09-21T07:12:23.808162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797650+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:12:23.797650+0000","last_peered":"2024-09-21T07:12:23.797650+0000","last_clean":"2024-09-21T07:12:23.797650+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:12:23.797650+0000","last_undegraded":"2024-09-21T07:12:23.797650+0000","last_fullsized":"2024-09-21T07:12:23.797650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811678+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:12:23.811678+0000","last_peered":"2024-09-21T07:12:23.811678+0000","last_clean":"2024-09-21T07:12:23.811678+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:12:23.811678+0000","last_undegraded":"2024-09-21T07:12:23.811678+0000","last_fullsized":"2024-09-21T07:12:23.811678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814831+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:12:23.814831+0000","last_peered":"2024-09-21T07:12:23.814831+0000","last_clean":"2024-09-21T07:12:23.814831+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:12:23.814831+0000","last_undegraded":"2024-09-21T07:12:23.814831+0000","last_fullsized":"2024-09-21T07:12:23.814831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798081+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:12:23.798081+0000","last_peered":"2024-09-21T07:12:23.798081+0000","last_clean":"2024-09-21T07:12:23.798081+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:12:23.798081+0000","last_undegraded":"2024-09-21T07:12:23.798081+0000","last_fullsized":"2024-09-21T07:12:23.798081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797741+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:12:23.797741+0000","last_peered":"2024-09-21T07:12:23.797741+0000","last_clean":"2024-09-21T07:12:23.797741+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:12:23.797741+0000","last_undegraded":"2024-09-21T07:12:23.797741+0000","last_fullsized":"2024-09-21T07:12:23.797741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797093+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:12:23.797093+0000","last_peered":"2024-09-21T07:12:23.797093+0000","last_clean":"2024-09-21T07:12:23.797093+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:12:23.797093+0000","last_undegraded":"2024-09-21T07:12:23.797093+0000","last_fullsized":"2024-09-21T07:12:23.797093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797350+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:12:23.797350+0000","last_peered":"2024-09-21T07:12:23.797350+0000","last_clean":"2024-09-21T07:12:23.797350+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:12:23.797350+0000","last_undegraded":"2024-09-21T07:12:23.797350+0000","last_fullsized":"2024-09-21T07:12:23.797350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814242+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:12:23.814242+0000","last_peered":"2024-09-21T07:12:23.814242+0000","last_clean":"2024-09-21T07:12:23.814242+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:12:23.814242+0000","last_undegraded":"2024-09-21T07:12:23.814242+0000","last_fullsized":"2024-09-21T07:12:23.814242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790030+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:12:23.790030+0000","last_peered":"2024-09-21T07:12:23.790030+0000","last_clean":"2024-09-21T07:12:23.790030+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:12:23.790030+0000","last_undegraded":"2024-09-21T07:12:23.790030+0000","last_fullsized":"2024-09-21T07:12:23.790030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810378+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:12:23.810378+0000","last_peered":"2024-09-21T07:12:23.810378+0000","last_clean":"2024-09-21T07:12:23.810378+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:12:23.810378+0000","last_undegraded":"2024-09-21T07:12:23.810378+0000","last_fullsized":"2024-09-21T07:12:23.810378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809256+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:12:23.809256+0000","last_peered":"2024-09-21T07:12:23.809256+0000","last_clean":"2024-09-21T07:12:23.809256+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:12:23.809256+0000","last_undegraded":"2024-09-21T07:12:23.809256+0000","last_fullsized":"2024-09-21T07:12:23.809256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806255+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:12:23.806255+0000","last_peered":"2024-09-21T07:12:23.806255+0000","last_clean":"2024-09-21T07:12:23.806255+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:12:23.806255+0000","last_undegraded":"2024-09-21T07:12:23.806255+0000","last_fullsized":"2024-09-21T07:12:23.806255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788650+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:12:23.788650+0000","last_peered":"2024-09-21T07:12:23.788650+0000","last_clean":"2024-09-21T07:12:23.788650+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:12:23.788650+0000","last_undegraded":"2024-09-21T07:12:23.788650+0000","last_fullsized":"2024-09-21T07:12:23.788650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809845+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:12:23.809845+0000","last_peered":"2024-09-21T07:12:23.809845+0000","last_clean":"2024-09-21T07:12:23.809845+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:12:23.809845+0000","last_undegraded":"2024-09-21T07:12:23.809845+0000","last_fullsized":"2024-09-21T07:12:23.809845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.789398+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:12:23.789398+0000","last_peered":"2024-09-21T07:12:23.789398+0000","last_clean":"2024-09-21T07:12:23.789398+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:12:23.789398+0000","last_undegraded":"2024-09-21T07:12:23.789398+0000","last_fullsized":"2024-09-21T07:12:23.789398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811791+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:12:23.811791+0000","last_peered":"2024-09-21T07:12:23.811791+0000","last_clean":"2024-09-21T07:12:23.811791+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:12:23.811791+0000","last_undegraded":"2024-09-21T07:12:23.811791+0000","last_fullsized":"2024-09-21T07:12:23.811791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.799236+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:12:23.799236+0000","last_peered":"2024-09-21T07:12:23.799236+0000","last_clean":"2024-09-21T07:12:23.799236+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:12:23.799236+0000","last_undegraded":"2024-09-21T07:12:23.799236+0000","last_fullsized":"2024-09-21T07:12:23.799236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811604+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:12:23.811604+0000","last_peered":"2024-09-21T07:12:23.811604+0000","last_clean":"2024-09-21T07:12:23.811604+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:12:23.811604+0000","last_undegraded":"2024-09-21T07:12:23.811604+0000","last_fullsized":"2024-09-21T07:12:23.811604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79970,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":731,"num_read_kb":449,"num_write":188,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":237781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":515860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972405,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1703936,"data_stored":1179155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135931,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":520170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":515972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495684,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1112355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659210,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":511250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855441,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1107411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215194,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":511444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":588338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542245,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":583375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705772,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":77178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":77627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:12:50.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:12:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.821+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3128284823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 msgr2=0x7f0f28109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.821+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3128284823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f28109730 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009900 tx=0x7f0f1402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.821+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3128284823 shutdown_connections 2024-09-21T07:12:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.821+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3128284823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f28109c70 0x7f0f281104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.821+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3128284823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f28109730 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.821+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3128284823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f281038f0 0x7f0f28103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.821+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3128284823 >> 172.21.15.27:0/3128284823 conn(0x7f0f280ff330 msgr2=0x7f0f28101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.822+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3128284823 shutdown_connections 2024-09-21T07:12:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.822+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3128284823 wait complete. 2024-09-21T07:12:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 Processor -- start 2024-09-21T07:12:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 -- start start 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f281038f0 0x7f0f2819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f2819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f28109c70 0x7f0f281a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28113930 con 0x7f0f28109320 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f281137b0 con 0x7f0f28109c70 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2ea00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28113ab0 con 0x7f0f281038f0 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f2819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f2819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45664/0 (socket says 172.21.15.27:45664) 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.823+0000 7f0f2cc00640 1 -- 172.21.15.27:0/3293131009 learned_addr learned my addr 172.21.15.27:0/3293131009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2e000640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f28109c70 0x7f0f281a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2d600640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f281038f0 0x7f0f2819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2cc00640 1 -- 172.21.15.27:0/3293131009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f281038f0 msgr2=0x7f0f2819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2cc00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f281038f0 0x7f0f2819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2cc00640 1 -- 172.21.15.27:0/3293131009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f28109c70 msgr2=0x7f0f281a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2cc00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f28109c70 0x7f0f281a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2cc00640 1 -- 172.21.15.27:0/3293131009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f281a2540 con 0x7f0f28109320 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2e000640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f28109c70 0x7f0f281a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2d600640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f281038f0 0x7f0f2819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.824+0000 7f0f2cc00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f2819cbb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f1402a910 tx=0x7f0f14031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.825+0000 7f0f26000640 1 -- 172.21.15.27:0/3293131009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1403f030 con 0x7f0f28109320 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.825+0000 7f0f26000640 1 -- 172.21.15.27:0/3293131009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f140043d0 con 0x7f0f28109320 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.825+0000 7f0f26000640 1 -- 172.21.15.27:0/3293131009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14035070 con 0x7f0f28109320 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.825+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f281a27d0 con 0x7f0f28109320 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.825+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f281a8fc0 con 0x7f0f28109320 2024-09-21T07:12:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.826+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f28104b60 con 0x7f0f28109320 2024-09-21T07:12:50.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.828+0000 7f0f26000640 1 -- 172.21.15.27:0/3293131009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f0f14042030 con 0x7f0f28109320 2024-09-21T07:12:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.830+0000 7f0f26000640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f100b65f0 0x7f0f100b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.830+0000 7f0f26000640 1 -- 172.21.15.27:0/3293131009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0f14108050 con 0x7f0f28109320 2024-09-21T07:12:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.830+0000 7f0f2d600640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f100b65f0 0x7f0f100b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:50.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.831+0000 7f0f2d600640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f100b65f0 0x7f0f100b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009fc0 tx=0x7f0f1c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:50.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:50.832+0000 7f0f26000640 1 -- 172.21.15.27:0/3293131009 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f140ce310 con 0x7f0f28109320 2024-09-21T07:12:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.029+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f2810f750 con 0x7f0f100b65f0 2024-09-21T07:12:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.036+0000 7f0f26000640 1 -- 172.21.15.27:0/3293131009 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340476 (secure 0 0 0) 0x7f0f2810f750 con 0x7f0f100b65f0 2024-09-21T07:12:51.041 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:51.042 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f100b65f0 msgr2=0x7f0f100b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f100b65f0 0x7f0f100b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009fc0 tx=0x7f0f1c005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 msgr2=0x7f0f2819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f2819cbb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f1402a910 tx=0x7f0f14031a20 comp rx=0 tx=0).stop 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 shutdown_connections 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f100b65f0 0x7f0f100b8b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f28109c70 0x7f0f281a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f28109320 0x7f0f2819cbb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 --2- 172.21.15.27:0/3293131009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f281038f0 0x7f0f2819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 >> 172.21.15.27:0/3293131009 conn(0x7f0f280ff330 msgr2=0x7f0f28105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 shutdown_connections 2024-09-21T07:12:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.041+0000 7f0f2ea00640 1 -- 172.21.15.27:0/3293131009 wait complete. 2024-09-21T07:12:51.066 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-21T07:12:49.414487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670572,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":812,"num_read_kb":518,"num_write":313,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392208,"kb_used_data":13840,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549616,"statfs":{"total":1151940427776,"available":1151538806784,"internally_reserved":0,"allocated":14172160,"data_stored":8174400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9367,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":48,"num_write":15,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003468"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.831114+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:11:55.831114+0000","last_peered":"2024-09-21T07:11:55.831114+0000","last_clean":"2024-09-21T07:11:55.831114+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:11:55.831114+0000","last_undegraded":"2024-09-21T07:11:55.831114+0000","last_fullsized":"2024-09-21T07:11:55.831114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805255+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:12:23.805255+0000","last_peered":"2024-09-21T07:12:23.805255+0000","last_clean":"2024-09-21T07:12:23.805255+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:12:23.805255+0000","last_undegraded":"2024-09-21T07:12:23.805255+0000","last_fullsized":"2024-09-21T07:12:23.805255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832485+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:11:55.832485+0000","last_peered":"2024-09-21T07:11:55.832485+0000","last_clean":"2024-09-21T07:11:55.832485+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:11:55.832485+0000","last_undegraded":"2024-09-21T07:11:55.832485+0000","last_fullsized":"2024-09-21T07:11:55.832485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833749+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:11:55.833749+0000","last_peered":"2024-09-21T07:11:55.833749+0000","last_clean":"2024-09-21T07:11:55.833749+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:11:55.833749+0000","last_undegraded":"2024-09-21T07:11:55.833749+0000","last_fullsized":"2024-09-21T07:11:55.833749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811697+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:12:23.811697+0000","last_peered":"2024-09-21T07:12:23.811697+0000","last_clean":"2024-09-21T07:12:23.811697+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:12:23.811697+0000","last_undegraded":"2024-09-21T07:12:23.811697+0000","last_fullsized":"2024-09-21T07:12:23.811697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844771+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:11:55.844771+0000","last_peered":"2024-09-21T07:11:55.844771+0000","last_clean":"2024-09-21T07:11:55.844771+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:11:55.844771+0000","last_undegraded":"2024-09-21T07:11:55.844771+0000","last_fullsized":"2024-09-21T07:11:55.844771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847516+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:11:55.847516+0000","last_peered":"2024-09-21T07:11:55.847516+0000","last_clean":"2024-09-21T07:11:55.847516+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:11:55.847516+0000","last_undegraded":"2024-09-21T07:11:55.847516+0000","last_fullsized":"2024-09-21T07:11:55.847516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814377+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:12:23.814377+0000","last_peered":"2024-09-21T07:12:23.814377+0000","last_clean":"2024-09-21T07:12:23.814377+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:12:23.814377+0000","last_undegraded":"2024-09-21T07:12:23.814377+0000","last_fullsized":"2024-09-21T07:12:23.814377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845066+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:11:55.845066+0000","last_peered":"2024-09-21T07:11:55.845066+0000","last_clean":"2024-09-21T07:11:55.845066+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:11:55.845066+0000","last_undegraded":"2024-09-21T07:11:55.845066+0000","last_fullsized":"2024-09-21T07:11:55.845066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847357+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:11:55.847357+0000","last_peered":"2024-09-21T07:11:55.847357+0000","last_clean":"2024-09-21T07:11:55.847357+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:11:55.847357+0000","last_undegraded":"2024-09-21T07:11:55.847357+0000","last_fullsized":"2024-09-21T07:11:55.847357+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803507+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:12:23.803507+0000","last_peered":"2024-09-21T07:12:23.803507+0000","last_clean":"2024-09-21T07:12:23.803507+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:12:23.803507+0000","last_undegraded":"2024-09-21T07:12:23.803507+0000","last_fullsized":"2024-09-21T07:12:23.803507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844870+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:11:55.844870+0000","last_peered":"2024-09-21T07:11:55.844870+0000","last_clean":"2024-09-21T07:11:55.844870+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:11:55.844870+0000","last_undegraded":"2024-09-21T07:11:55.844870+0000","last_fullsized":"2024-09-21T07:11:55.844870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813396+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:12:23.813396+0000","last_peered":"2024-09-21T07:12:23.813396+0000","last_clean":"2024-09-21T07:12:23.813396+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:12:23.813396+0000","last_undegraded":"2024-09-21T07:12:23.813396+0000","last_fullsized":"2024-09-21T07:12:23.813396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846961+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:11:55.846961+0000","last_peered":"2024-09-21T07:11:55.846961+0000","last_clean":"2024-09-21T07:11:55.846961+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:11:55.846961+0000","last_undegraded":"2024-09-21T07:11:55.846961+0000","last_fullsized":"2024-09-21T07:11:55.846961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845137+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:11:55.845137+0000","last_peered":"2024-09-21T07:11:55.845137+0000","last_clean":"2024-09-21T07:11:55.845137+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:11:55.845137+0000","last_undegraded":"2024-09-21T07:11:55.845137+0000","last_fullsized":"2024-09-21T07:11:55.845137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807789+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:12:23.807789+0000","last_peered":"2024-09-21T07:12:23.807789+0000","last_clean":"2024-09-21T07:12:23.807789+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:12:23.807789+0000","last_undegraded":"2024-09-21T07:12:23.807789+0000","last_fullsized":"2024-09-21T07:12:23.807789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832340+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:11:55.832340+0000","last_peered":"2024-09-21T07:11:55.832340+0000","last_clean":"2024-09-21T07:11:55.832340+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:11:55.832340+0000","last_undegraded":"2024-09-21T07:11:55.832340+0000","last_fullsized":"2024-09-21T07:11:55.832340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845070+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:11:55.845070+0000","last_peered":"2024-09-21T07:11:55.845070+0000","last_clean":"2024-09-21T07:11:55.845070+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:11:55.845070+0000","last_undegraded":"2024-09-21T07:11:55.845070+0000","last_fullsized":"2024-09-21T07:11:55.845070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798453+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:12:23.798453+0000","last_peered":"2024-09-21T07:12:23.798453+0000","last_clean":"2024-09-21T07:12:23.798453+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:12:23.798453+0000","last_undegraded":"2024-09-21T07:12:23.798453+0000","last_fullsized":"2024-09-21T07:12:23.798453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833325+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:11:55.833325+0000","last_peered":"2024-09-21T07:11:55.833325+0000","last_clean":"2024-09-21T07:11:55.833325+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:11:55.833325+0000","last_undegraded":"2024-09-21T07:11:55.833325+0000","last_fullsized":"2024-09-21T07:11:55.833325+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830920+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:11:55.830920+0000","last_peered":"2024-09-21T07:11:55.830920+0000","last_clean":"2024-09-21T07:11:55.830920+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:11:55.830920+0000","last_undegraded":"2024-09-21T07:11:55.830920+0000","last_fullsized":"2024-09-21T07:11:55.830920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.780871+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:12:23.780871+0000","last_peered":"2024-09-21T07:12:23.780871+0000","last_clean":"2024-09-21T07:12:23.780871+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:12:23.780871+0000","last_undegraded":"2024-09-21T07:12:23.780871+0000","last_fullsized":"2024-09-21T07:12:23.780871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833051+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:11:55.833051+0000","last_peered":"2024-09-21T07:11:55.833051+0000","last_clean":"2024-09-21T07:11:55.833051+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:11:55.833051+0000","last_undegraded":"2024-09-21T07:11:55.833051+0000","last_fullsized":"2024-09-21T07:11:55.833051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845144+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:11:55.845144+0000","last_peered":"2024-09-21T07:11:55.845144+0000","last_clean":"2024-09-21T07:11:55.845144+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:11:55.845144+0000","last_undegraded":"2024-09-21T07:11:55.845144+0000","last_fullsized":"2024-09-21T07:11:55.845144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.800049+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:12:23.800049+0000","last_peered":"2024-09-21T07:12:23.800049+0000","last_clean":"2024-09-21T07:12:23.800049+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:12:23.800049+0000","last_undegraded":"2024-09-21T07:12:23.800049+0000","last_fullsized":"2024-09-21T07:12:23.800049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845369+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:11:55.845369+0000","last_peered":"2024-09-21T07:11:55.845369+0000","last_clean":"2024-09-21T07:11:55.845369+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:11:55.845369+0000","last_undegraded":"2024-09-21T07:11:55.845369+0000","last_fullsized":"2024-09-21T07:11:55.845369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808689+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:12:23.808689+0000","last_peered":"2024-09-21T07:12:23.808689+0000","last_clean":"2024-09-21T07:12:23.808689+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:12:23.808689+0000","last_undegraded":"2024-09-21T07:12:23.808689+0000","last_fullsized":"2024-09-21T07:12:23.808689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'13","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:43.372588+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:12:43.372588+0000","last_peered":"2024-09-21T07:12:43.372588+0000","last_clean":"2024-09-21T07:12:43.372588+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:12:43.372588+0000","last_undegraded":"2024-09-21T07:12:43.372588+0000","last_fullsized":"2024-09-21T07:12:43.372588+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832993+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:11:55.832993+0000","last_peered":"2024-09-21T07:11:55.832993+0000","last_clean":"2024-09-21T07:11:55.832993+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:11:55.832993+0000","last_undegraded":"2024-09-21T07:11:55.832993+0000","last_fullsized":"2024-09-21T07:11:55.832993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809315+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:12:23.809315+0000","last_peered":"2024-09-21T07:12:23.809315+0000","last_clean":"2024-09-21T07:12:23.809315+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:12:23.809315+0000","last_undegraded":"2024-09-21T07:12:23.809315+0000","last_fullsized":"2024-09-21T07:12:23.809315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847542+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:11:55.847542+0000","last_peered":"2024-09-21T07:11:55.847542+0000","last_clean":"2024-09-21T07:11:55.847542+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:11:55.847542+0000","last_undegraded":"2024-09-21T07:11:55.847542+0000","last_fullsized":"2024-09-21T07:11:55.847542+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790825+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:12:23.790825+0000","last_peered":"2024-09-21T07:12:23.790825+0000","last_clean":"2024-09-21T07:12:23.790825+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:12:23.790825+0000","last_undegraded":"2024-09-21T07:12:23.790825+0000","last_fullsized":"2024-09-21T07:12:23.790825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830831+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:11:55.830831+0000","last_peered":"2024-09-21T07:11:55.830831+0000","last_clean":"2024-09-21T07:11:55.830831+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:11:55.830831+0000","last_undegraded":"2024-09-21T07:11:55.830831+0000","last_fullsized":"2024-09-21T07:11:55.830831+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846866+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:11:55.846866+0000","last_peered":"2024-09-21T07:11:55.846866+0000","last_clean":"2024-09-21T07:11:55.846866+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:11:55.846866+0000","last_undegraded":"2024-09-21T07:11:55.846866+0000","last_fullsized":"2024-09-21T07:11:55.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.794745+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:12:23.794745+0000","last_peered":"2024-09-21T07:12:23.794745+0000","last_clean":"2024-09-21T07:12:23.794745+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:12:23.794745+0000","last_undegraded":"2024-09-21T07:12:23.794745+0000","last_fullsized":"2024-09-21T07:12:23.794745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981709+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:11:55.981709+0000","last_peered":"2024-09-21T07:11:55.981709+0000","last_clean":"2024-09-21T07:11:55.981709+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:11:55.981709+0000","last_undegraded":"2024-09-21T07:11:55.981709+0000","last_fullsized":"2024-09-21T07:11:55.981709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845966+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:11:55.845966+0000","last_peered":"2024-09-21T07:11:55.845966+0000","last_clean":"2024-09-21T07:11:55.845966+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:11:55.845966+0000","last_undegraded":"2024-09-21T07:11:55.845966+0000","last_fullsized":"2024-09-21T07:11:55.845966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795850+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:12:23.795850+0000","last_peered":"2024-09-21T07:12:23.795850+0000","last_clean":"2024-09-21T07:12:23.795850+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:12:23.795850+0000","last_undegraded":"2024-09-21T07:12:23.795850+0000","last_fullsized":"2024-09-21T07:12:23.795850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:45.470360+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:12:45.470360+0000","last_peered":"2024-09-21T07:12:45.470360+0000","last_clean":"2024-09-21T07:12:45.470360+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:12:45.470360+0000","last_undegraded":"2024-09-21T07:12:45.470360+0000","last_fullsized":"2024-09-21T07:12:45.470360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846521+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:11:55.846521+0000","last_peered":"2024-09-21T07:11:55.846521+0000","last_clean":"2024-09-21T07:11:55.846521+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:11:55.846521+0000","last_undegraded":"2024-09-21T07:11:55.846521+0000","last_fullsized":"2024-09-21T07:11:55.846521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807529+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:12:23.807529+0000","last_peered":"2024-09-21T07:12:23.807529+0000","last_clean":"2024-09-21T07:12:23.807529+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:12:23.807529+0000","last_undegraded":"2024-09-21T07:12:23.807529+0000","last_fullsized":"2024-09-21T07:12:23.807529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981500+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:11:55.981500+0000","last_peered":"2024-09-21T07:11:55.981500+0000","last_clean":"2024-09-21T07:11:55.981500+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:11:55.981500+0000","last_undegraded":"2024-09-21T07:11:55.981500+0000","last_fullsized":"2024-09-21T07:11:55.981500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845592+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:11:55.845592+0000","last_peered":"2024-09-21T07:11:55.845592+0000","last_clean":"2024-09-21T07:11:55.845592+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:11:55.845592+0000","last_undegraded":"2024-09-21T07:11:55.845592+0000","last_fullsized":"2024-09-21T07:11:55.845592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811077+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:12:23.811077+0000","last_peered":"2024-09-21T07:12:23.811077+0000","last_clean":"2024-09-21T07:12:23.811077+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:12:23.811077+0000","last_undegraded":"2024-09-21T07:12:23.811077+0000","last_fullsized":"2024-09-21T07:12:23.811077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845392+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:11:55.845392+0000","last_peered":"2024-09-21T07:11:55.845392+0000","last_clean":"2024-09-21T07:11:55.845392+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:11:55.845392+0000","last_undegraded":"2024-09-21T07:11:55.845392+0000","last_fullsized":"2024-09-21T07:11:55.845392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981606+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:11:55.981606+0000","last_peered":"2024-09-21T07:11:55.981606+0000","last_clean":"2024-09-21T07:11:55.981606+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:11:55.981606+0000","last_undegraded":"2024-09-21T07:11:55.981606+0000","last_fullsized":"2024-09-21T07:11:55.981606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981807+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:11:55.981807+0000","last_peered":"2024-09-21T07:11:55.981807+0000","last_clean":"2024-09-21T07:11:55.981807+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:11:55.981807+0000","last_undegraded":"2024-09-21T07:11:55.981807+0000","last_fullsized":"2024-09-21T07:11:55.981807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814663+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:12:23.814663+0000","last_peered":"2024-09-21T07:12:23.814663+0000","last_clean":"2024-09-21T07:12:23.814663+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:12:23.814663+0000","last_undegraded":"2024-09-21T07:12:23.814663+0000","last_fullsized":"2024-09-21T07:12:23.814663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.834191+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:11:55.834191+0000","last_peered":"2024-09-21T07:11:55.834191+0000","last_clean":"2024-09-21T07:11:55.834191+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:11:55.834191+0000","last_undegraded":"2024-09-21T07:11:55.834191+0000","last_fullsized":"2024-09-21T07:11:55.834191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833166+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:11:55.833166+0000","last_peered":"2024-09-21T07:11:55.833166+0000","last_clean":"2024-09-21T07:11:55.833166+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:11:55.833166+0000","last_undegraded":"2024-09-21T07:11:55.833166+0000","last_fullsized":"2024-09-21T07:11:55.833166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807409+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:12:23.807409+0000","last_peered":"2024-09-21T07:12:23.807409+0000","last_clean":"2024-09-21T07:12:23.807409+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:12:23.807409+0000","last_undegraded":"2024-09-21T07:12:23.807409+0000","last_fullsized":"2024-09-21T07:12:23.807409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846421+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:11:55.846421+0000","last_peered":"2024-09-21T07:11:55.846421+0000","last_clean":"2024-09-21T07:11:55.846421+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:11:55.846421+0000","last_undegraded":"2024-09-21T07:11:55.846421+0000","last_fullsized":"2024-09-21T07:11:55.846421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846714+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:11:55.846714+0000","last_peered":"2024-09-21T07:11:55.846714+0000","last_clean":"2024-09-21T07:11:55.846714+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:11:55.846714+0000","last_undegraded":"2024-09-21T07:11:55.846714+0000","last_fullsized":"2024-09-21T07:11:55.846714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814102+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:12:23.814102+0000","last_peered":"2024-09-21T07:12:23.814102+0000","last_clean":"2024-09-21T07:12:23.814102+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:12:23.814102+0000","last_undegraded":"2024-09-21T07:12:23.814102+0000","last_fullsized":"2024-09-21T07:12:23.814102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846385+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:11:55.846385+0000","last_peered":"2024-09-21T07:11:55.846385+0000","last_clean":"2024-09-21T07:11:55.846385+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:11:55.846385+0000","last_undegraded":"2024-09-21T07:11:55.846385+0000","last_fullsized":"2024-09-21T07:11:55.846385+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845182+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:11:55.845182+0000","last_peered":"2024-09-21T07:11:55.845182+0000","last_clean":"2024-09-21T07:11:55.845182+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:11:55.845182+0000","last_undegraded":"2024-09-21T07:11:55.845182+0000","last_fullsized":"2024-09-21T07:11:55.845182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801647+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:12:23.801647+0000","last_peered":"2024-09-21T07:12:23.801647+0000","last_clean":"2024-09-21T07:12:23.801647+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:12:23.801647+0000","last_undegraded":"2024-09-21T07:12:23.801647+0000","last_fullsized":"2024-09-21T07:12:23.801647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832950+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:11:55.832950+0000","last_peered":"2024-09-21T07:11:55.832950+0000","last_clean":"2024-09-21T07:11:55.832950+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:11:55.832950+0000","last_undegraded":"2024-09-21T07:11:55.832950+0000","last_fullsized":"2024-09-21T07:11:55.832950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845860+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:11:55.845860+0000","last_peered":"2024-09-21T07:11:55.845860+0000","last_clean":"2024-09-21T07:11:55.845860+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:11:55.845860+0000","last_undegraded":"2024-09-21T07:11:55.845860+0000","last_fullsized":"2024-09-21T07:11:55.845860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802831+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:12:23.802831+0000","last_peered":"2024-09-21T07:12:23.802831+0000","last_clean":"2024-09-21T07:12:23.802831+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:12:23.802831+0000","last_undegraded":"2024-09-21T07:12:23.802831+0000","last_fullsized":"2024-09-21T07:12:23.802831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846785+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:11:55.846785+0000","last_peered":"2024-09-21T07:11:55.846785+0000","last_clean":"2024-09-21T07:11:55.846785+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:11:55.846785+0000","last_undegraded":"2024-09-21T07:11:55.846785+0000","last_fullsized":"2024-09-21T07:11:55.846785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846243+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:11:55.846243+0000","last_peered":"2024-09-21T07:11:55.846243+0000","last_clean":"2024-09-21T07:11:55.846243+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:11:55.846243+0000","last_undegraded":"2024-09-21T07:11:55.846243+0000","last_fullsized":"2024-09-21T07:11:55.846243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805668+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:12:23.805668+0000","last_peered":"2024-09-21T07:12:23.805668+0000","last_clean":"2024-09-21T07:12:23.805668+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:12:23.805668+0000","last_undegraded":"2024-09-21T07:12:23.805668+0000","last_fullsized":"2024-09-21T07:12:23.805668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":169,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:43.744886+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:12:43.744886+0000","last_peered":"2024-09-21T07:12:43.744886+0000","last_clean":"2024-09-21T07:12:43.744886+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:12:43.744886+0000","last_undegraded":"2024-09-21T07:12:43.744886+0000","last_fullsized":"2024-09-21T07:12:43.744886+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830985+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:11:55.830985+0000","last_peered":"2024-09-21T07:11:55.830985+0000","last_clean":"2024-09-21T07:11:55.830985+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:11:55.830985+0000","last_undegraded":"2024-09-21T07:11:55.830985+0000","last_fullsized":"2024-09-21T07:11:55.830985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813592+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:12:23.813592+0000","last_peered":"2024-09-21T07:12:23.813592+0000","last_clean":"2024-09-21T07:12:23.813592+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:12:23.813592+0000","last_undegraded":"2024-09-21T07:12:23.813592+0000","last_fullsized":"2024-09-21T07:12:23.813592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846256+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:11:55.846256+0000","last_peered":"2024-09-21T07:11:55.846256+0000","last_clean":"2024-09-21T07:11:55.846256+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:11:55.846256+0000","last_undegraded":"2024-09-21T07:11:55.846256+0000","last_fullsized":"2024-09-21T07:11:55.846256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832414+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:11:55.832414+0000","last_peered":"2024-09-21T07:11:55.832414+0000","last_clean":"2024-09-21T07:11:55.832414+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:11:55.832414+0000","last_undegraded":"2024-09-21T07:11:55.832414+0000","last_fullsized":"2024-09-21T07:11:55.832414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808494+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:12:23.808494+0000","last_peered":"2024-09-21T07:12:23.808494+0000","last_clean":"2024-09-21T07:12:23.808494+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:12:23.808494+0000","last_undegraded":"2024-09-21T07:12:23.808494+0000","last_fullsized":"2024-09-21T07:12:23.808494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833515+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:11:55.833515+0000","last_peered":"2024-09-21T07:11:55.833515+0000","last_clean":"2024-09-21T07:11:55.833515+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:11:55.833515+0000","last_undegraded":"2024-09-21T07:11:55.833515+0000","last_fullsized":"2024-09-21T07:11:55.833515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846046+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:11:55.846046+0000","last_peered":"2024-09-21T07:11:55.846046+0000","last_clean":"2024-09-21T07:11:55.846046+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:11:55.846046+0000","last_undegraded":"2024-09-21T07:11:55.846046+0000","last_fullsized":"2024-09-21T07:11:55.846046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798135+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:12:23.798135+0000","last_peered":"2024-09-21T07:12:23.798135+0000","last_clean":"2024-09-21T07:12:23.798135+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:12:23.798135+0000","last_undegraded":"2024-09-21T07:12:23.798135+0000","last_fullsized":"2024-09-21T07:12:23.798135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":333,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:47.753953+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:12:47.753953+0000","last_peered":"2024-09-21T07:12:47.753953+0000","last_clean":"2024-09-21T07:12:47.753953+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:12:47.753953+0000","last_undegraded":"2024-09-21T07:12:47.753953+0000","last_fullsized":"2024-09-21T07:12:47.753953+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":195,"num_write":42,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.848049+0000","last_change":"2024-09-21T07:11:55.847916+0000","last_active":"2024-09-21T07:11:55.848049+0000","last_peered":"2024-09-21T07:11:55.848049+0000","last_clean":"2024-09-21T07:11:55.848049+0000","last_became_active":"2024-09-21T07:11:55.847491+0000","last_became_peered":"2024-09-21T07:11:55.847491+0000","last_unstale":"2024-09-21T07:11:55.848049+0000","last_undegraded":"2024-09-21T07:11:55.848049+0000","last_fullsized":"2024-09-21T07:11:55.848049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:25.702259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,1],"acting":[10,8,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813257+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:12:23.813257+0000","last_peered":"2024-09-21T07:12:23.813257+0000","last_clean":"2024-09-21T07:12:23.813257+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:12:23.813257+0000","last_undegraded":"2024-09-21T07:12:23.813257+0000","last_fullsized":"2024-09-21T07:12:23.813257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846367+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:11:55.846367+0000","last_peered":"2024-09-21T07:11:55.846367+0000","last_clean":"2024-09-21T07:11:55.846367+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:11:55.846367+0000","last_undegraded":"2024-09-21T07:11:55.846367+0000","last_fullsized":"2024-09-21T07:11:55.846367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845245+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:11:55.845245+0000","last_peered":"2024-09-21T07:11:55.845245+0000","last_clean":"2024-09-21T07:11:55.845245+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:11:55.845245+0000","last_undegraded":"2024-09-21T07:11:55.845245+0000","last_fullsized":"2024-09-21T07:11:55.845245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.793771+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:12:23.793771+0000","last_peered":"2024-09-21T07:12:23.793771+0000","last_clean":"2024-09-21T07:12:23.793771+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:12:23.793771+0000","last_undegraded":"2024-09-21T07:12:23.793771+0000","last_fullsized":"2024-09-21T07:12:23.793771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981878+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:11:55.981878+0000","last_peered":"2024-09-21T07:11:55.981878+0000","last_clean":"2024-09-21T07:11:55.981878+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:11:55.981878+0000","last_undegraded":"2024-09-21T07:11:55.981878+0000","last_fullsized":"2024-09-21T07:11:55.981878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830120+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:11:55.830120+0000","last_peered":"2024-09-21T07:11:55.830120+0000","last_clean":"2024-09-21T07:11:55.830120+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:11:55.830120+0000","last_undegraded":"2024-09-21T07:11:55.830120+0000","last_fullsized":"2024-09-21T07:11:55.830120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813258+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:12:23.813258+0000","last_peered":"2024-09-21T07:12:23.813258+0000","last_clean":"2024-09-21T07:12:23.813258+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:12:23.813258+0000","last_undegraded":"2024-09-21T07:12:23.813258+0000","last_fullsized":"2024-09-21T07:12:23.813258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:46.751398+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:12:46.751398+0000","last_peered":"2024-09-21T07:12:46.751398+0000","last_clean":"2024-09-21T07:12:46.751398+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:12:46.751398+0000","last_undegraded":"2024-09-21T07:12:46.751398+0000","last_fullsized":"2024-09-21T07:12:46.751398+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833187+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:11:55.833187+0000","last_peered":"2024-09-21T07:11:55.833187+0000","last_clean":"2024-09-21T07:11:55.833187+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:11:55.833187+0000","last_undegraded":"2024-09-21T07:11:55.833187+0000","last_fullsized":"2024-09-21T07:11:55.833187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847006+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:11:55.847006+0000","last_peered":"2024-09-21T07:11:55.847006+0000","last_clean":"2024-09-21T07:11:55.847006+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:11:55.847006+0000","last_undegraded":"2024-09-21T07:11:55.847006+0000","last_fullsized":"2024-09-21T07:11:55.847006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802309+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:12:23.802309+0000","last_peered":"2024-09-21T07:12:23.802309+0000","last_clean":"2024-09-21T07:12:23.802309+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:12:23.802309+0000","last_undegraded":"2024-09-21T07:12:23.802309+0000","last_fullsized":"2024-09-21T07:12:23.802309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845739+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:11:55.845739+0000","last_peered":"2024-09-21T07:11:55.845739+0000","last_clean":"2024-09-21T07:11:55.845739+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:11:55.845739+0000","last_undegraded":"2024-09-21T07:11:55.845739+0000","last_fullsized":"2024-09-21T07:11:55.845739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849176+0000","last_change":"2024-09-21T07:11:55.848588+0000","last_active":"2024-09-21T07:11:55.849176+0000","last_peered":"2024-09-21T07:11:55.849176+0000","last_clean":"2024-09-21T07:11:55.849176+0000","last_became_active":"2024-09-21T07:11:55.848025+0000","last_became_peered":"2024-09-21T07:11:55.848025+0000","last_unstale":"2024-09-21T07:11:55.849176+0000","last_undegraded":"2024-09-21T07:11:55.849176+0000","last_fullsized":"2024-09-21T07:11:55.849176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:37.413873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810233+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:12:23.810233+0000","last_peered":"2024-09-21T07:12:23.810233+0000","last_clean":"2024-09-21T07:12:23.810233+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:12:23.810233+0000","last_undegraded":"2024-09-21T07:12:23.810233+0000","last_fullsized":"2024-09-21T07:12:23.810233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845457+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:11:55.845457+0000","last_peered":"2024-09-21T07:11:55.845457+0000","last_clean":"2024-09-21T07:11:55.845457+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:11:55.845457+0000","last_undegraded":"2024-09-21T07:11:55.845457+0000","last_fullsized":"2024-09-21T07:11:55.845457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.835562+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:11:55.835562+0000","last_peered":"2024-09-21T07:11:55.835562+0000","last_clean":"2024-09-21T07:11:55.835562+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:11:55.835562+0000","last_undegraded":"2024-09-21T07:11:55.835562+0000","last_fullsized":"2024-09-21T07:11:55.835562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807495+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:12:23.807495+0000","last_peered":"2024-09-21T07:12:23.807495+0000","last_clean":"2024-09-21T07:12:23.807495+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:12:23.807495+0000","last_undegraded":"2024-09-21T07:12:23.807495+0000","last_fullsized":"2024-09-21T07:12:23.807495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849703+0000","last_change":"2024-09-21T07:11:55.848551+0000","last_active":"2024-09-21T07:11:55.849703+0000","last_peered":"2024-09-21T07:11:55.849703+0000","last_clean":"2024-09-21T07:11:55.849703+0000","last_became_active":"2024-09-21T07:11:55.848056+0000","last_became_peered":"2024-09-21T07:11:55.848056+0000","last_unstale":"2024-09-21T07:11:55.849703+0000","last_undegraded":"2024-09-21T07:11:55.849703+0000","last_fullsized":"2024-09-21T07:11:55.849703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:07.779820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":170,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:44.747830+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:12:44.747830+0000","last_peered":"2024-09-21T07:12:44.747830+0000","last_clean":"2024-09-21T07:12:44.747830+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:12:44.747830+0000","last_undegraded":"2024-09-21T07:12:44.747830+0000","last_fullsized":"2024-09-21T07:12:44.747830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788293+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:12:23.788293+0000","last_peered":"2024-09-21T07:12:23.788293+0000","last_clean":"2024-09-21T07:12:23.788293+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:12:23.788293+0000","last_undegraded":"2024-09-21T07:12:23.788293+0000","last_fullsized":"2024-09-21T07:12:23.788293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833541+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:11:55.833541+0000","last_peered":"2024-09-21T07:11:55.833541+0000","last_clean":"2024-09-21T07:11:55.833541+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:11:55.833541+0000","last_undegraded":"2024-09-21T07:11:55.833541+0000","last_fullsized":"2024-09-21T07:11:55.833541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803098+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:12:23.803098+0000","last_peered":"2024-09-21T07:12:23.803098+0000","last_clean":"2024-09-21T07:12:23.803098+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:12:23.803098+0000","last_undegraded":"2024-09-21T07:12:23.803098+0000","last_fullsized":"2024-09-21T07:12:23.803098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807453+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:12:23.807453+0000","last_peered":"2024-09-21T07:12:23.807453+0000","last_clean":"2024-09-21T07:12:23.807453+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:12:23.807453+0000","last_undegraded":"2024-09-21T07:12:23.807453+0000","last_fullsized":"2024-09-21T07:12:23.807453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808124+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:12:23.808124+0000","last_peered":"2024-09-21T07:12:23.808124+0000","last_clean":"2024-09-21T07:12:23.808124+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:12:23.808124+0000","last_undegraded":"2024-09-21T07:12:23.808124+0000","last_fullsized":"2024-09-21T07:12:23.808124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795490+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:12:23.795490+0000","last_peered":"2024-09-21T07:12:23.795490+0000","last_clean":"2024-09-21T07:12:23.795490+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:12:23.795490+0000","last_undegraded":"2024-09-21T07:12:23.795490+0000","last_fullsized":"2024-09-21T07:12:23.795490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805863+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:12:23.805863+0000","last_peered":"2024-09-21T07:12:23.805863+0000","last_clean":"2024-09-21T07:12:23.805863+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:12:23.805863+0000","last_undegraded":"2024-09-21T07:12:23.805863+0000","last_fullsized":"2024-09-21T07:12:23.805863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801102+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:12:23.801102+0000","last_peered":"2024-09-21T07:12:23.801102+0000","last_clean":"2024-09-21T07:12:23.801102+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:12:23.801102+0000","last_undegraded":"2024-09-21T07:12:23.801102+0000","last_fullsized":"2024-09-21T07:12:23.801102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797345+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:12:23.797345+0000","last_peered":"2024-09-21T07:12:23.797345+0000","last_clean":"2024-09-21T07:12:23.797345+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:12:23.797345+0000","last_undegraded":"2024-09-21T07:12:23.797345+0000","last_fullsized":"2024-09-21T07:12:23.797345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806243+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:12:23.806243+0000","last_peered":"2024-09-21T07:12:23.806243+0000","last_clean":"2024-09-21T07:12:23.806243+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:12:23.806243+0000","last_undegraded":"2024-09-21T07:12:23.806243+0000","last_fullsized":"2024-09-21T07:12:23.806243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810287+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:12:23.810287+0000","last_peered":"2024-09-21T07:12:23.810287+0000","last_clean":"2024-09-21T07:12:23.810287+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:12:23.810287+0000","last_undegraded":"2024-09-21T07:12:23.810287+0000","last_fullsized":"2024-09-21T07:12:23.810287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797647+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:12:23.797647+0000","last_peered":"2024-09-21T07:12:23.797647+0000","last_clean":"2024-09-21T07:12:23.797647+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:12:23.797647+0000","last_undegraded":"2024-09-21T07:12:23.797647+0000","last_fullsized":"2024-09-21T07:12:23.797647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.815349+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:12:23.815349+0000","last_peered":"2024-09-21T07:12:23.815349+0000","last_clean":"2024-09-21T07:12:23.815349+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:12:23.815349+0000","last_undegraded":"2024-09-21T07:12:23.815349+0000","last_fullsized":"2024-09-21T07:12:23.815349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809317+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:12:23.809317+0000","last_peered":"2024-09-21T07:12:23.809317+0000","last_clean":"2024-09-21T07:12:23.809317+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:12:23.809317+0000","last_undegraded":"2024-09-21T07:12:23.809317+0000","last_fullsized":"2024-09-21T07:12:23.809317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795846+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:12:23.795846+0000","last_peered":"2024-09-21T07:12:23.795846+0000","last_clean":"2024-09-21T07:12:23.795846+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:12:23.795846+0000","last_undegraded":"2024-09-21T07:12:23.795846+0000","last_fullsized":"2024-09-21T07:12:23.795846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808162+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:12:23.808162+0000","last_peered":"2024-09-21T07:12:23.808162+0000","last_clean":"2024-09-21T07:12:23.808162+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:12:23.808162+0000","last_undegraded":"2024-09-21T07:12:23.808162+0000","last_fullsized":"2024-09-21T07:12:23.808162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797650+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:12:23.797650+0000","last_peered":"2024-09-21T07:12:23.797650+0000","last_clean":"2024-09-21T07:12:23.797650+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:12:23.797650+0000","last_undegraded":"2024-09-21T07:12:23.797650+0000","last_fullsized":"2024-09-21T07:12:23.797650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811678+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:12:23.811678+0000","last_peered":"2024-09-21T07:12:23.811678+0000","last_clean":"2024-09-21T07:12:23.811678+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:12:23.811678+0000","last_undegraded":"2024-09-21T07:12:23.811678+0000","last_fullsized":"2024-09-21T07:12:23.811678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814831+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:12:23.814831+0000","last_peered":"2024-09-21T07:12:23.814831+0000","last_clean":"2024-09-21T07:12:23.814831+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:12:23.814831+0000","last_undegraded":"2024-09-21T07:12:23.814831+0000","last_fullsized":"2024-09-21T07:12:23.814831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798081+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:12:23.798081+0000","last_peered":"2024-09-21T07:12:23.798081+0000","last_clean":"2024-09-21T07:12:23.798081+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:12:23.798081+0000","last_undegraded":"2024-09-21T07:12:23.798081+0000","last_fullsized":"2024-09-21T07:12:23.798081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797741+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:12:23.797741+0000","last_peered":"2024-09-21T07:12:23.797741+0000","last_clean":"2024-09-21T07:12:23.797741+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:12:23.797741+0000","last_undegraded":"2024-09-21T07:12:23.797741+0000","last_fullsized":"2024-09-21T07:12:23.797741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797093+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:12:23.797093+0000","last_peered":"2024-09-21T07:12:23.797093+0000","last_clean":"2024-09-21T07:12:23.797093+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:12:23.797093+0000","last_undegraded":"2024-09-21T07:12:23.797093+0000","last_fullsized":"2024-09-21T07:12:23.797093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797350+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:12:23.797350+0000","last_peered":"2024-09-21T07:12:23.797350+0000","last_clean":"2024-09-21T07:12:23.797350+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:12:23.797350+0000","last_undegraded":"2024-09-21T07:12:23.797350+0000","last_fullsized":"2024-09-21T07:12:23.797350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814242+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:12:23.814242+0000","last_peered":"2024-09-21T07:12:23.814242+0000","last_clean":"2024-09-21T07:12:23.814242+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:12:23.814242+0000","last_undegraded":"2024-09-21T07:12:23.814242+0000","last_fullsized":"2024-09-21T07:12:23.814242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790030+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:12:23.790030+0000","last_peered":"2024-09-21T07:12:23.790030+0000","last_clean":"2024-09-21T07:12:23.790030+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:12:23.790030+0000","last_undegraded":"2024-09-21T07:12:23.790030+0000","last_fullsized":"2024-09-21T07:12:23.790030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810378+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:12:23.810378+0000","last_peered":"2024-09-21T07:12:23.810378+0000","last_clean":"2024-09-21T07:12:23.810378+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:12:23.810378+0000","last_undegraded":"2024-09-21T07:12:23.810378+0000","last_fullsized":"2024-09-21T07:12:23.810378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809256+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:12:23.809256+0000","last_peered":"2024-09-21T07:12:23.809256+0000","last_clean":"2024-09-21T07:12:23.809256+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:12:23.809256+0000","last_undegraded":"2024-09-21T07:12:23.809256+0000","last_fullsized":"2024-09-21T07:12:23.809256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806255+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:12:23.806255+0000","last_peered":"2024-09-21T07:12:23.806255+0000","last_clean":"2024-09-21T07:12:23.806255+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:12:23.806255+0000","last_undegraded":"2024-09-21T07:12:23.806255+0000","last_fullsized":"2024-09-21T07:12:23.806255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788650+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:12:23.788650+0000","last_peered":"2024-09-21T07:12:23.788650+0000","last_clean":"2024-09-21T07:12:23.788650+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:12:23.788650+0000","last_undegraded":"2024-09-21T07:12:23.788650+0000","last_fullsized":"2024-09-21T07:12:23.788650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809845+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:12:23.809845+0000","last_peered":"2024-09-21T07:12:23.809845+0000","last_clean":"2024-09-21T07:12:23.809845+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:12:23.809845+0000","last_undegraded":"2024-09-21T07:12:23.809845+0000","last_fullsized":"2024-09-21T07:12:23.809845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.789398+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:12:23.789398+0000","last_peered":"2024-09-21T07:12:23.789398+0000","last_clean":"2024-09-21T07:12:23.789398+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:12:23.789398+0000","last_undegraded":"2024-09-21T07:12:23.789398+0000","last_fullsized":"2024-09-21T07:12:23.789398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811791+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:12:23.811791+0000","last_peered":"2024-09-21T07:12:23.811791+0000","last_clean":"2024-09-21T07:12:23.811791+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:12:23.811791+0000","last_undegraded":"2024-09-21T07:12:23.811791+0000","last_fullsized":"2024-09-21T07:12:23.811791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.799236+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:12:23.799236+0000","last_peered":"2024-09-21T07:12:23.799236+0000","last_clean":"2024-09-21T07:12:23.799236+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:12:23.799236+0000","last_undegraded":"2024-09-21T07:12:23.799236+0000","last_fullsized":"2024-09-21T07:12:23.799236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811604+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:12:23.811604+0000","last_peered":"2024-09-21T07:12:23.811604+0000","last_clean":"2024-09-21T07:12:23.811604+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:12:23.811604+0000","last_undegraded":"2024-09-21T07:12:23.811604+0000","last_fullsized":"2024-09-21T07:12:23.811604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79970,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":731,"num_read_kb":449,"num_write":188,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":237781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":515860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972405,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1703936,"data_stored":1179155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135931,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":520170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":515972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495684,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1112355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659210,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":511250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855441,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1107411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215194,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":511444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":588338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542245,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":583375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705772,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":77178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":77627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:12:51.072 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T07:12:51.072 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:12:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:50 smithi119 ceph-mon[34519]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:12:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 -- 172.21.15.27:0/2045470655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109320 msgr2=0x7fcebc109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 --2- 172.21.15.27:0/2045470655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109320 0x7fcebc109730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009900 tx=0x7fcea802a410 comp rx=0 tx=0).stop 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 -- 172.21.15.27:0/2045470655 shutdown_connections 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 --2- 172.21.15.27:0/2045470655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcebc109c70 0x7fcebc1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 --2- 172.21.15.27:0/2045470655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109320 0x7fcebc109730 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 --2- 172.21.15.27:0/2045470655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc1038f0 0x7fcebc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 -- 172.21.15.27:0/2045470655 >> 172.21.15.27:0/2045470655 conn(0x7fcebc0ff330 msgr2=0x7fcebc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 -- 172.21.15.27:0/2045470655 shutdown_connections 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.336+0000 7fcec1200640 1 -- 172.21.15.27:0/2045470655 wait complete. 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 Processor -- start 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 -- start start 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcebc1038f0 0x7fcebc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc109320 0x7fcebc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109c70 0x7fcebc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc113980 con 0x7fcebc109c70 2024-09-21T07:12:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc113800 con 0x7fcebc109320 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.337+0000 7fcec1200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc113b00 con 0x7fcebc1038f0 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109c70 0x7fcebc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcebc1038f0 0x7fcebc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109c70 0x7fcebc076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45676/0 (socket says 172.21.15.27:45676) 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcebc1038f0 0x7fcebc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34412/0 (socket says 172.21.15.27:34412) 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 -- 172.21.15.27:0/219279 learned_addr learned my addr 172.21.15.27:0/219279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebaa00640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc109320 0x7fcebc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 -- 172.21.15.27:0/219279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcebc1038f0 msgr2=0x7fcebc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcebc1038f0 0x7fcebc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 -- 172.21.15.27:0/219279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc109320 msgr2=0x7fcebc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc109320 0x7fcebc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 -- 172.21.15.27:0/219279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc1a85b0 con 0x7fcebc109c70 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.338+0000 7fcebbe00640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109c70 0x7fcebc076190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fceb400eb10 tx=0x7fceb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.339+0000 7fceb3e00640 1 -- 172.21.15.27:0/219279 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4004b20 con 0x7fcebc109c70 2024-09-21T07:12:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.339+0000 7fceb3e00640 1 -- 172.21.15.27:0/219279 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb4004cc0 con 0x7fcebc109c70 2024-09-21T07:12:51.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.339+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcebc1a88a0 con 0x7fcebc109c70 2024-09-21T07:12:51.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.339+0000 7fceb3e00640 1 -- 172.21.15.27:0/219279 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4010670 con 0x7fcebc109c70 2024-09-21T07:12:51.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.339+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcebc1a8e10 con 0x7fcebc109c70 2024-09-21T07:12:51.351 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:50 smithi027 ceph-mon[30779]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:12:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.340+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcebc103d00 con 0x7fcebc109c70 2024-09-21T07:12:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.342+0000 7fceb3e00640 1 -- 172.21.15.27:0/219279 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fceb4005020 con 0x7fcebc109c70 2024-09-21T07:12:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.344+0000 7fceb3e00640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcea00b66a0 0x7fcea00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.344+0000 7fceb3e00640 1 -- 172.21.15.27:0/219279 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fceb40e7eb0 con 0x7fcebc109c70 2024-09-21T07:12:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.344+0000 7fcebb400640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcea00b66a0 0x7fcea00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.345+0000 7fcebb400640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcea00b66a0 0x7fcea00b8bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcebc077160 tx=0x7fceac005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:51.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.348+0000 7fceb3e00640 1 -- 172.21.15.27:0/219279 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fceb40ae110 con 0x7fcebc109c70 2024-09-21T07:12:51.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:50 smithi153 ceph-mon[34747]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:12:51.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.550+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcebc063800 con 0x7fcea00b66a0 2024-09-21T07:12:51.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.556+0000 7fceb3e00640 1 -- 172.21.15.27:0/219279 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340476 (secure 0 0 0) 0x7fcebc063800 con 0x7fcea00b66a0 2024-09-21T07:12:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:12:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.560+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcea00b66a0 msgr2=0x7fcea00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.560+0000 7fcec1200640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcea00b66a0 0x7fcea00b8bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcebc077160 tx=0x7fceac005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.560+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109c70 msgr2=0x7fcebc076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.560+0000 7fcec1200640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109c70 0x7fcebc076190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fceb400eb10 tx=0x7fceb400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.560+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 shutdown_connections 2024-09-21T07:12:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.560+0000 7fcec1200640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcea00b66a0 0x7fcea00b8bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.561+0000 7fcec1200640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcebc109c70 0x7fcebc076190 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.561+0000 7fcec1200640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc109320 0x7fcebc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.561+0000 7fcec1200640 1 --2- 172.21.15.27:0/219279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcebc1038f0 0x7fcebc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.561+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 >> 172.21.15.27:0/219279 conn(0x7fcebc0ff330 msgr2=0x7fcebc108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.561+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 shutdown_connections 2024-09-21T07:12:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:51.561+0000 7fcec1200640 1 -- 172.21.15.27:0/219279 wait complete. 2024-09-21T07:12:51.587 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-21T07:12:51.415216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670963,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":829,"num_read_kb":525,"num_write":314,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":755,"ondisk_log_size":755,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392212,"kb_used_data":13844,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549612,"statfs":{"total":1151940427776,"available":1151538802688,"internally_reserved":0,"allocated":14176256,"data_stored":8178875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9758,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":16,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003666"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.831114+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:11:55.831114+0000","last_peered":"2024-09-21T07:11:55.831114+0000","last_clean":"2024-09-21T07:11:55.831114+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:11:55.831114+0000","last_undegraded":"2024-09-21T07:11:55.831114+0000","last_fullsized":"2024-09-21T07:11:55.831114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805255+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:12:23.805255+0000","last_peered":"2024-09-21T07:12:23.805255+0000","last_clean":"2024-09-21T07:12:23.805255+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:12:23.805255+0000","last_undegraded":"2024-09-21T07:12:23.805255+0000","last_fullsized":"2024-09-21T07:12:23.805255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832485+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:11:55.832485+0000","last_peered":"2024-09-21T07:11:55.832485+0000","last_clean":"2024-09-21T07:11:55.832485+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:11:55.832485+0000","last_undegraded":"2024-09-21T07:11:55.832485+0000","last_fullsized":"2024-09-21T07:11:55.832485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833749+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:11:55.833749+0000","last_peered":"2024-09-21T07:11:55.833749+0000","last_clean":"2024-09-21T07:11:55.833749+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:11:55.833749+0000","last_undegraded":"2024-09-21T07:11:55.833749+0000","last_fullsized":"2024-09-21T07:11:55.833749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811697+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:12:23.811697+0000","last_peered":"2024-09-21T07:12:23.811697+0000","last_clean":"2024-09-21T07:12:23.811697+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:12:23.811697+0000","last_undegraded":"2024-09-21T07:12:23.811697+0000","last_fullsized":"2024-09-21T07:12:23.811697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844771+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:11:55.844771+0000","last_peered":"2024-09-21T07:11:55.844771+0000","last_clean":"2024-09-21T07:11:55.844771+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:11:55.844771+0000","last_undegraded":"2024-09-21T07:11:55.844771+0000","last_fullsized":"2024-09-21T07:11:55.844771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847516+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:11:55.847516+0000","last_peered":"2024-09-21T07:11:55.847516+0000","last_clean":"2024-09-21T07:11:55.847516+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:11:55.847516+0000","last_undegraded":"2024-09-21T07:11:55.847516+0000","last_fullsized":"2024-09-21T07:11:55.847516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814377+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:12:23.814377+0000","last_peered":"2024-09-21T07:12:23.814377+0000","last_clean":"2024-09-21T07:12:23.814377+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:12:23.814377+0000","last_undegraded":"2024-09-21T07:12:23.814377+0000","last_fullsized":"2024-09-21T07:12:23.814377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845066+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:11:55.845066+0000","last_peered":"2024-09-21T07:11:55.845066+0000","last_clean":"2024-09-21T07:11:55.845066+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:11:55.845066+0000","last_undegraded":"2024-09-21T07:11:55.845066+0000","last_fullsized":"2024-09-21T07:11:55.845066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847357+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:11:55.847357+0000","last_peered":"2024-09-21T07:11:55.847357+0000","last_clean":"2024-09-21T07:11:55.847357+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:11:55.847357+0000","last_undegraded":"2024-09-21T07:11:55.847357+0000","last_fullsized":"2024-09-21T07:11:55.847357+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803507+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:12:23.803507+0000","last_peered":"2024-09-21T07:12:23.803507+0000","last_clean":"2024-09-21T07:12:23.803507+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:12:23.803507+0000","last_undegraded":"2024-09-21T07:12:23.803507+0000","last_fullsized":"2024-09-21T07:12:23.803507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844870+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:11:55.844870+0000","last_peered":"2024-09-21T07:11:55.844870+0000","last_clean":"2024-09-21T07:11:55.844870+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:11:55.844870+0000","last_undegraded":"2024-09-21T07:11:55.844870+0000","last_fullsized":"2024-09-21T07:11:55.844870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813396+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:12:23.813396+0000","last_peered":"2024-09-21T07:12:23.813396+0000","last_clean":"2024-09-21T07:12:23.813396+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:12:23.813396+0000","last_undegraded":"2024-09-21T07:12:23.813396+0000","last_fullsized":"2024-09-21T07:12:23.813396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846961+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:11:55.846961+0000","last_peered":"2024-09-21T07:11:55.846961+0000","last_clean":"2024-09-21T07:11:55.846961+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:11:55.846961+0000","last_undegraded":"2024-09-21T07:11:55.846961+0000","last_fullsized":"2024-09-21T07:11:55.846961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845137+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:11:55.845137+0000","last_peered":"2024-09-21T07:11:55.845137+0000","last_clean":"2024-09-21T07:11:55.845137+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:11:55.845137+0000","last_undegraded":"2024-09-21T07:11:55.845137+0000","last_fullsized":"2024-09-21T07:11:55.845137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807789+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:12:23.807789+0000","last_peered":"2024-09-21T07:12:23.807789+0000","last_clean":"2024-09-21T07:12:23.807789+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:12:23.807789+0000","last_undegraded":"2024-09-21T07:12:23.807789+0000","last_fullsized":"2024-09-21T07:12:23.807789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832340+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:11:55.832340+0000","last_peered":"2024-09-21T07:11:55.832340+0000","last_clean":"2024-09-21T07:11:55.832340+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:11:55.832340+0000","last_undegraded":"2024-09-21T07:11:55.832340+0000","last_fullsized":"2024-09-21T07:11:55.832340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845070+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:11:55.845070+0000","last_peered":"2024-09-21T07:11:55.845070+0000","last_clean":"2024-09-21T07:11:55.845070+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:11:55.845070+0000","last_undegraded":"2024-09-21T07:11:55.845070+0000","last_fullsized":"2024-09-21T07:11:55.845070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798453+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:12:23.798453+0000","last_peered":"2024-09-21T07:12:23.798453+0000","last_clean":"2024-09-21T07:12:23.798453+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:12:23.798453+0000","last_undegraded":"2024-09-21T07:12:23.798453+0000","last_fullsized":"2024-09-21T07:12:23.798453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833325+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:11:55.833325+0000","last_peered":"2024-09-21T07:11:55.833325+0000","last_clean":"2024-09-21T07:11:55.833325+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:11:55.833325+0000","last_undegraded":"2024-09-21T07:11:55.833325+0000","last_fullsized":"2024-09-21T07:11:55.833325+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830920+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:11:55.830920+0000","last_peered":"2024-09-21T07:11:55.830920+0000","last_clean":"2024-09-21T07:11:55.830920+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:11:55.830920+0000","last_undegraded":"2024-09-21T07:11:55.830920+0000","last_fullsized":"2024-09-21T07:11:55.830920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.780871+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:12:23.780871+0000","last_peered":"2024-09-21T07:12:23.780871+0000","last_clean":"2024-09-21T07:12:23.780871+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:12:23.780871+0000","last_undegraded":"2024-09-21T07:12:23.780871+0000","last_fullsized":"2024-09-21T07:12:23.780871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833051+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:11:55.833051+0000","last_peered":"2024-09-21T07:11:55.833051+0000","last_clean":"2024-09-21T07:11:55.833051+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:11:55.833051+0000","last_undegraded":"2024-09-21T07:11:55.833051+0000","last_fullsized":"2024-09-21T07:11:55.833051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845144+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:11:55.845144+0000","last_peered":"2024-09-21T07:11:55.845144+0000","last_clean":"2024-09-21T07:11:55.845144+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:11:55.845144+0000","last_undegraded":"2024-09-21T07:11:55.845144+0000","last_fullsized":"2024-09-21T07:11:55.845144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.800049+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:12:23.800049+0000","last_peered":"2024-09-21T07:12:23.800049+0000","last_clean":"2024-09-21T07:12:23.800049+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:12:23.800049+0000","last_undegraded":"2024-09-21T07:12:23.800049+0000","last_fullsized":"2024-09-21T07:12:23.800049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845369+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:11:55.845369+0000","last_peered":"2024-09-21T07:11:55.845369+0000","last_clean":"2024-09-21T07:11:55.845369+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:11:55.845369+0000","last_undegraded":"2024-09-21T07:11:55.845369+0000","last_fullsized":"2024-09-21T07:11:55.845369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808689+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:12:23.808689+0000","last_peered":"2024-09-21T07:12:23.808689+0000","last_clean":"2024-09-21T07:12:23.808689+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:12:23.808689+0000","last_undegraded":"2024-09-21T07:12:23.808689+0000","last_fullsized":"2024-09-21T07:12:23.808689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'13","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:43.372588+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:12:43.372588+0000","last_peered":"2024-09-21T07:12:43.372588+0000","last_clean":"2024-09-21T07:12:43.372588+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:12:43.372588+0000","last_undegraded":"2024-09-21T07:12:43.372588+0000","last_fullsized":"2024-09-21T07:12:43.372588+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832993+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:11:55.832993+0000","last_peered":"2024-09-21T07:11:55.832993+0000","last_clean":"2024-09-21T07:11:55.832993+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:11:55.832993+0000","last_undegraded":"2024-09-21T07:11:55.832993+0000","last_fullsized":"2024-09-21T07:11:55.832993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809315+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:12:23.809315+0000","last_peered":"2024-09-21T07:12:23.809315+0000","last_clean":"2024-09-21T07:12:23.809315+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:12:23.809315+0000","last_undegraded":"2024-09-21T07:12:23.809315+0000","last_fullsized":"2024-09-21T07:12:23.809315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847542+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:11:55.847542+0000","last_peered":"2024-09-21T07:11:55.847542+0000","last_clean":"2024-09-21T07:11:55.847542+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:11:55.847542+0000","last_undegraded":"2024-09-21T07:11:55.847542+0000","last_fullsized":"2024-09-21T07:11:55.847542+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790825+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:12:23.790825+0000","last_peered":"2024-09-21T07:12:23.790825+0000","last_clean":"2024-09-21T07:12:23.790825+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:12:23.790825+0000","last_undegraded":"2024-09-21T07:12:23.790825+0000","last_fullsized":"2024-09-21T07:12:23.790825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830831+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:11:55.830831+0000","last_peered":"2024-09-21T07:11:55.830831+0000","last_clean":"2024-09-21T07:11:55.830831+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:11:55.830831+0000","last_undegraded":"2024-09-21T07:11:55.830831+0000","last_fullsized":"2024-09-21T07:11:55.830831+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846866+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:11:55.846866+0000","last_peered":"2024-09-21T07:11:55.846866+0000","last_clean":"2024-09-21T07:11:55.846866+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:11:55.846866+0000","last_undegraded":"2024-09-21T07:11:55.846866+0000","last_fullsized":"2024-09-21T07:11:55.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.794745+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:12:23.794745+0000","last_peered":"2024-09-21T07:12:23.794745+0000","last_clean":"2024-09-21T07:12:23.794745+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:12:23.794745+0000","last_undegraded":"2024-09-21T07:12:23.794745+0000","last_fullsized":"2024-09-21T07:12:23.794745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981709+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:11:55.981709+0000","last_peered":"2024-09-21T07:11:55.981709+0000","last_clean":"2024-09-21T07:11:55.981709+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:11:55.981709+0000","last_undegraded":"2024-09-21T07:11:55.981709+0000","last_fullsized":"2024-09-21T07:11:55.981709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845966+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:11:55.845966+0000","last_peered":"2024-09-21T07:11:55.845966+0000","last_clean":"2024-09-21T07:11:55.845966+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:11:55.845966+0000","last_undegraded":"2024-09-21T07:11:55.845966+0000","last_fullsized":"2024-09-21T07:11:55.845966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795850+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:12:23.795850+0000","last_peered":"2024-09-21T07:12:23.795850+0000","last_clean":"2024-09-21T07:12:23.795850+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:12:23.795850+0000","last_undegraded":"2024-09-21T07:12:23.795850+0000","last_fullsized":"2024-09-21T07:12:23.795850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":172,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:49.755378+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:12:49.755378+0000","last_peered":"2024-09-21T07:12:49.755378+0000","last_clean":"2024-09-21T07:12:49.755378+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:12:49.755378+0000","last_undegraded":"2024-09-21T07:12:49.755378+0000","last_fullsized":"2024-09-21T07:12:49.755378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846521+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:11:55.846521+0000","last_peered":"2024-09-21T07:11:55.846521+0000","last_clean":"2024-09-21T07:11:55.846521+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:11:55.846521+0000","last_undegraded":"2024-09-21T07:11:55.846521+0000","last_fullsized":"2024-09-21T07:11:55.846521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807529+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:12:23.807529+0000","last_peered":"2024-09-21T07:12:23.807529+0000","last_clean":"2024-09-21T07:12:23.807529+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:12:23.807529+0000","last_undegraded":"2024-09-21T07:12:23.807529+0000","last_fullsized":"2024-09-21T07:12:23.807529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981500+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:11:55.981500+0000","last_peered":"2024-09-21T07:11:55.981500+0000","last_clean":"2024-09-21T07:11:55.981500+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:11:55.981500+0000","last_undegraded":"2024-09-21T07:11:55.981500+0000","last_fullsized":"2024-09-21T07:11:55.981500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845592+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:11:55.845592+0000","last_peered":"2024-09-21T07:11:55.845592+0000","last_clean":"2024-09-21T07:11:55.845592+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:11:55.845592+0000","last_undegraded":"2024-09-21T07:11:55.845592+0000","last_fullsized":"2024-09-21T07:11:55.845592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811077+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:12:23.811077+0000","last_peered":"2024-09-21T07:12:23.811077+0000","last_clean":"2024-09-21T07:12:23.811077+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:12:23.811077+0000","last_undegraded":"2024-09-21T07:12:23.811077+0000","last_fullsized":"2024-09-21T07:12:23.811077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845392+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:11:55.845392+0000","last_peered":"2024-09-21T07:11:55.845392+0000","last_clean":"2024-09-21T07:11:55.845392+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:11:55.845392+0000","last_undegraded":"2024-09-21T07:11:55.845392+0000","last_fullsized":"2024-09-21T07:11:55.845392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981606+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:11:55.981606+0000","last_peered":"2024-09-21T07:11:55.981606+0000","last_clean":"2024-09-21T07:11:55.981606+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:11:55.981606+0000","last_undegraded":"2024-09-21T07:11:55.981606+0000","last_fullsized":"2024-09-21T07:11:55.981606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981807+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:11:55.981807+0000","last_peered":"2024-09-21T07:11:55.981807+0000","last_clean":"2024-09-21T07:11:55.981807+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:11:55.981807+0000","last_undegraded":"2024-09-21T07:11:55.981807+0000","last_fullsized":"2024-09-21T07:11:55.981807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814663+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:12:23.814663+0000","last_peered":"2024-09-21T07:12:23.814663+0000","last_clean":"2024-09-21T07:12:23.814663+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:12:23.814663+0000","last_undegraded":"2024-09-21T07:12:23.814663+0000","last_fullsized":"2024-09-21T07:12:23.814663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.834191+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:11:55.834191+0000","last_peered":"2024-09-21T07:11:55.834191+0000","last_clean":"2024-09-21T07:11:55.834191+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:11:55.834191+0000","last_undegraded":"2024-09-21T07:11:55.834191+0000","last_fullsized":"2024-09-21T07:11:55.834191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833166+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:11:55.833166+0000","last_peered":"2024-09-21T07:11:55.833166+0000","last_clean":"2024-09-21T07:11:55.833166+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:11:55.833166+0000","last_undegraded":"2024-09-21T07:11:55.833166+0000","last_fullsized":"2024-09-21T07:11:55.833166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807409+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:12:23.807409+0000","last_peered":"2024-09-21T07:12:23.807409+0000","last_clean":"2024-09-21T07:12:23.807409+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:12:23.807409+0000","last_undegraded":"2024-09-21T07:12:23.807409+0000","last_fullsized":"2024-09-21T07:12:23.807409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846421+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:11:55.846421+0000","last_peered":"2024-09-21T07:11:55.846421+0000","last_clean":"2024-09-21T07:11:55.846421+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:11:55.846421+0000","last_undegraded":"2024-09-21T07:11:55.846421+0000","last_fullsized":"2024-09-21T07:11:55.846421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846714+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:11:55.846714+0000","last_peered":"2024-09-21T07:11:55.846714+0000","last_clean":"2024-09-21T07:11:55.846714+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:11:55.846714+0000","last_undegraded":"2024-09-21T07:11:55.846714+0000","last_fullsized":"2024-09-21T07:11:55.846714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814102+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:12:23.814102+0000","last_peered":"2024-09-21T07:12:23.814102+0000","last_clean":"2024-09-21T07:12:23.814102+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:12:23.814102+0000","last_undegraded":"2024-09-21T07:12:23.814102+0000","last_fullsized":"2024-09-21T07:12:23.814102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846385+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:11:55.846385+0000","last_peered":"2024-09-21T07:11:55.846385+0000","last_clean":"2024-09-21T07:11:55.846385+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:11:55.846385+0000","last_undegraded":"2024-09-21T07:11:55.846385+0000","last_fullsized":"2024-09-21T07:11:55.846385+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845182+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:11:55.845182+0000","last_peered":"2024-09-21T07:11:55.845182+0000","last_clean":"2024-09-21T07:11:55.845182+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:11:55.845182+0000","last_undegraded":"2024-09-21T07:11:55.845182+0000","last_fullsized":"2024-09-21T07:11:55.845182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801647+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:12:23.801647+0000","last_peered":"2024-09-21T07:12:23.801647+0000","last_clean":"2024-09-21T07:12:23.801647+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:12:23.801647+0000","last_undegraded":"2024-09-21T07:12:23.801647+0000","last_fullsized":"2024-09-21T07:12:23.801647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832950+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:11:55.832950+0000","last_peered":"2024-09-21T07:11:55.832950+0000","last_clean":"2024-09-21T07:11:55.832950+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:11:55.832950+0000","last_undegraded":"2024-09-21T07:11:55.832950+0000","last_fullsized":"2024-09-21T07:11:55.832950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845860+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:11:55.845860+0000","last_peered":"2024-09-21T07:11:55.845860+0000","last_clean":"2024-09-21T07:11:55.845860+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:11:55.845860+0000","last_undegraded":"2024-09-21T07:11:55.845860+0000","last_fullsized":"2024-09-21T07:11:55.845860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802831+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:12:23.802831+0000","last_peered":"2024-09-21T07:12:23.802831+0000","last_clean":"2024-09-21T07:12:23.802831+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:12:23.802831+0000","last_undegraded":"2024-09-21T07:12:23.802831+0000","last_fullsized":"2024-09-21T07:12:23.802831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846785+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:11:55.846785+0000","last_peered":"2024-09-21T07:11:55.846785+0000","last_clean":"2024-09-21T07:11:55.846785+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:11:55.846785+0000","last_undegraded":"2024-09-21T07:11:55.846785+0000","last_fullsized":"2024-09-21T07:11:55.846785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846243+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:11:55.846243+0000","last_peered":"2024-09-21T07:11:55.846243+0000","last_clean":"2024-09-21T07:11:55.846243+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:11:55.846243+0000","last_undegraded":"2024-09-21T07:11:55.846243+0000","last_fullsized":"2024-09-21T07:11:55.846243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805668+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:12:23.805668+0000","last_peered":"2024-09-21T07:12:23.805668+0000","last_clean":"2024-09-21T07:12:23.805668+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:12:23.805668+0000","last_undegraded":"2024-09-21T07:12:23.805668+0000","last_fullsized":"2024-09-21T07:12:23.805668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":176,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:48.754380+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:12:48.754380+0000","last_peered":"2024-09-21T07:12:48.754380+0000","last_clean":"2024-09-21T07:12:48.754380+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:12:48.754380+0000","last_undegraded":"2024-09-21T07:12:48.754380+0000","last_fullsized":"2024-09-21T07:12:48.754380+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830985+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:11:55.830985+0000","last_peered":"2024-09-21T07:11:55.830985+0000","last_clean":"2024-09-21T07:11:55.830985+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:11:55.830985+0000","last_undegraded":"2024-09-21T07:11:55.830985+0000","last_fullsized":"2024-09-21T07:11:55.830985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813592+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:12:23.813592+0000","last_peered":"2024-09-21T07:12:23.813592+0000","last_clean":"2024-09-21T07:12:23.813592+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:12:23.813592+0000","last_undegraded":"2024-09-21T07:12:23.813592+0000","last_fullsized":"2024-09-21T07:12:23.813592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846256+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:11:55.846256+0000","last_peered":"2024-09-21T07:11:55.846256+0000","last_clean":"2024-09-21T07:11:55.846256+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:11:55.846256+0000","last_undegraded":"2024-09-21T07:11:55.846256+0000","last_fullsized":"2024-09-21T07:11:55.846256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832414+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:11:55.832414+0000","last_peered":"2024-09-21T07:11:55.832414+0000","last_clean":"2024-09-21T07:11:55.832414+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:11:55.832414+0000","last_undegraded":"2024-09-21T07:11:55.832414+0000","last_fullsized":"2024-09-21T07:11:55.832414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808494+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:12:23.808494+0000","last_peered":"2024-09-21T07:12:23.808494+0000","last_clean":"2024-09-21T07:12:23.808494+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:12:23.808494+0000","last_undegraded":"2024-09-21T07:12:23.808494+0000","last_fullsized":"2024-09-21T07:12:23.808494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833515+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:11:55.833515+0000","last_peered":"2024-09-21T07:11:55.833515+0000","last_clean":"2024-09-21T07:11:55.833515+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:11:55.833515+0000","last_undegraded":"2024-09-21T07:11:55.833515+0000","last_fullsized":"2024-09-21T07:11:55.833515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846046+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:11:55.846046+0000","last_peered":"2024-09-21T07:11:55.846046+0000","last_clean":"2024-09-21T07:11:55.846046+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:11:55.846046+0000","last_undegraded":"2024-09-21T07:11:55.846046+0000","last_fullsized":"2024-09-21T07:11:55.846046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798135+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:12:23.798135+0000","last_peered":"2024-09-21T07:12:23.798135+0000","last_clean":"2024-09-21T07:12:23.798135+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:12:23.798135+0000","last_undegraded":"2024-09-21T07:12:23.798135+0000","last_fullsized":"2024-09-21T07:12:23.798135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":333,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:47.753953+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:12:47.753953+0000","last_peered":"2024-09-21T07:12:47.753953+0000","last_clean":"2024-09-21T07:12:47.753953+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:12:47.753953+0000","last_undegraded":"2024-09-21T07:12:47.753953+0000","last_fullsized":"2024-09-21T07:12:47.753953+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":195,"num_write":42,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.848049+0000","last_change":"2024-09-21T07:11:55.847916+0000","last_active":"2024-09-21T07:11:55.848049+0000","last_peered":"2024-09-21T07:11:55.848049+0000","last_clean":"2024-09-21T07:11:55.848049+0000","last_became_active":"2024-09-21T07:11:55.847491+0000","last_became_peered":"2024-09-21T07:11:55.847491+0000","last_unstale":"2024-09-21T07:11:55.848049+0000","last_undegraded":"2024-09-21T07:11:55.848049+0000","last_fullsized":"2024-09-21T07:11:55.848049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:25.702259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,1],"acting":[10,8,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813257+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:12:23.813257+0000","last_peered":"2024-09-21T07:12:23.813257+0000","last_clean":"2024-09-21T07:12:23.813257+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:12:23.813257+0000","last_undegraded":"2024-09-21T07:12:23.813257+0000","last_fullsized":"2024-09-21T07:12:23.813257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846367+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:11:55.846367+0000","last_peered":"2024-09-21T07:11:55.846367+0000","last_clean":"2024-09-21T07:11:55.846367+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:11:55.846367+0000","last_undegraded":"2024-09-21T07:11:55.846367+0000","last_fullsized":"2024-09-21T07:11:55.846367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845245+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:11:55.845245+0000","last_peered":"2024-09-21T07:11:55.845245+0000","last_clean":"2024-09-21T07:11:55.845245+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:11:55.845245+0000","last_undegraded":"2024-09-21T07:11:55.845245+0000","last_fullsized":"2024-09-21T07:11:55.845245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.793771+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:12:23.793771+0000","last_peered":"2024-09-21T07:12:23.793771+0000","last_clean":"2024-09-21T07:12:23.793771+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:12:23.793771+0000","last_undegraded":"2024-09-21T07:12:23.793771+0000","last_fullsized":"2024-09-21T07:12:23.793771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981878+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:11:55.981878+0000","last_peered":"2024-09-21T07:11:55.981878+0000","last_clean":"2024-09-21T07:11:55.981878+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:11:55.981878+0000","last_undegraded":"2024-09-21T07:11:55.981878+0000","last_fullsized":"2024-09-21T07:11:55.981878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830120+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:11:55.830120+0000","last_peered":"2024-09-21T07:11:55.830120+0000","last_clean":"2024-09-21T07:11:55.830120+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:11:55.830120+0000","last_undegraded":"2024-09-21T07:11:55.830120+0000","last_fullsized":"2024-09-21T07:11:55.830120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813258+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:12:23.813258+0000","last_peered":"2024-09-21T07:12:23.813258+0000","last_clean":"2024-09-21T07:12:23.813258+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:12:23.813258+0000","last_undegraded":"2024-09-21T07:12:23.813258+0000","last_fullsized":"2024-09-21T07:12:23.813258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:46.751398+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:12:46.751398+0000","last_peered":"2024-09-21T07:12:46.751398+0000","last_clean":"2024-09-21T07:12:46.751398+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:12:46.751398+0000","last_undegraded":"2024-09-21T07:12:46.751398+0000","last_fullsized":"2024-09-21T07:12:46.751398+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833187+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:11:55.833187+0000","last_peered":"2024-09-21T07:11:55.833187+0000","last_clean":"2024-09-21T07:11:55.833187+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:11:55.833187+0000","last_undegraded":"2024-09-21T07:11:55.833187+0000","last_fullsized":"2024-09-21T07:11:55.833187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847006+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:11:55.847006+0000","last_peered":"2024-09-21T07:11:55.847006+0000","last_clean":"2024-09-21T07:11:55.847006+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:11:55.847006+0000","last_undegraded":"2024-09-21T07:11:55.847006+0000","last_fullsized":"2024-09-21T07:11:55.847006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802309+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:12:23.802309+0000","last_peered":"2024-09-21T07:12:23.802309+0000","last_clean":"2024-09-21T07:12:23.802309+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:12:23.802309+0000","last_undegraded":"2024-09-21T07:12:23.802309+0000","last_fullsized":"2024-09-21T07:12:23.802309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845739+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:11:55.845739+0000","last_peered":"2024-09-21T07:11:55.845739+0000","last_clean":"2024-09-21T07:11:55.845739+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:11:55.845739+0000","last_undegraded":"2024-09-21T07:11:55.845739+0000","last_fullsized":"2024-09-21T07:11:55.845739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849176+0000","last_change":"2024-09-21T07:11:55.848588+0000","last_active":"2024-09-21T07:11:55.849176+0000","last_peered":"2024-09-21T07:11:55.849176+0000","last_clean":"2024-09-21T07:11:55.849176+0000","last_became_active":"2024-09-21T07:11:55.848025+0000","last_became_peered":"2024-09-21T07:11:55.848025+0000","last_unstale":"2024-09-21T07:11:55.849176+0000","last_undegraded":"2024-09-21T07:11:55.849176+0000","last_fullsized":"2024-09-21T07:11:55.849176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:37.413873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810233+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:12:23.810233+0000","last_peered":"2024-09-21T07:12:23.810233+0000","last_clean":"2024-09-21T07:12:23.810233+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:12:23.810233+0000","last_undegraded":"2024-09-21T07:12:23.810233+0000","last_fullsized":"2024-09-21T07:12:23.810233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845457+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:11:55.845457+0000","last_peered":"2024-09-21T07:11:55.845457+0000","last_clean":"2024-09-21T07:11:55.845457+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:11:55.845457+0000","last_undegraded":"2024-09-21T07:11:55.845457+0000","last_fullsized":"2024-09-21T07:11:55.845457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.835562+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:11:55.835562+0000","last_peered":"2024-09-21T07:11:55.835562+0000","last_clean":"2024-09-21T07:11:55.835562+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:11:55.835562+0000","last_undegraded":"2024-09-21T07:11:55.835562+0000","last_fullsized":"2024-09-21T07:11:55.835562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807495+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:12:23.807495+0000","last_peered":"2024-09-21T07:12:23.807495+0000","last_clean":"2024-09-21T07:12:23.807495+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:12:23.807495+0000","last_undegraded":"2024-09-21T07:12:23.807495+0000","last_fullsized":"2024-09-21T07:12:23.807495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849703+0000","last_change":"2024-09-21T07:11:55.848551+0000","last_active":"2024-09-21T07:11:55.849703+0000","last_peered":"2024-09-21T07:11:55.849703+0000","last_clean":"2024-09-21T07:11:55.849703+0000","last_became_active":"2024-09-21T07:11:55.848056+0000","last_became_peered":"2024-09-21T07:11:55.848056+0000","last_unstale":"2024-09-21T07:11:55.849703+0000","last_undegraded":"2024-09-21T07:11:55.849703+0000","last_fullsized":"2024-09-21T07:11:55.849703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:07.779820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":176,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:48.755513+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:12:48.755513+0000","last_peered":"2024-09-21T07:12:48.755513+0000","last_clean":"2024-09-21T07:12:48.755513+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:12:48.755513+0000","last_undegraded":"2024-09-21T07:12:48.755513+0000","last_fullsized":"2024-09-21T07:12:48.755513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788293+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:12:23.788293+0000","last_peered":"2024-09-21T07:12:23.788293+0000","last_clean":"2024-09-21T07:12:23.788293+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:12:23.788293+0000","last_undegraded":"2024-09-21T07:12:23.788293+0000","last_fullsized":"2024-09-21T07:12:23.788293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833541+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:11:55.833541+0000","last_peered":"2024-09-21T07:11:55.833541+0000","last_clean":"2024-09-21T07:11:55.833541+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:11:55.833541+0000","last_undegraded":"2024-09-21T07:11:55.833541+0000","last_fullsized":"2024-09-21T07:11:55.833541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803098+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:12:23.803098+0000","last_peered":"2024-09-21T07:12:23.803098+0000","last_clean":"2024-09-21T07:12:23.803098+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:12:23.803098+0000","last_undegraded":"2024-09-21T07:12:23.803098+0000","last_fullsized":"2024-09-21T07:12:23.803098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807453+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:12:23.807453+0000","last_peered":"2024-09-21T07:12:23.807453+0000","last_clean":"2024-09-21T07:12:23.807453+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:12:23.807453+0000","last_undegraded":"2024-09-21T07:12:23.807453+0000","last_fullsized":"2024-09-21T07:12:23.807453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808124+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:12:23.808124+0000","last_peered":"2024-09-21T07:12:23.808124+0000","last_clean":"2024-09-21T07:12:23.808124+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:12:23.808124+0000","last_undegraded":"2024-09-21T07:12:23.808124+0000","last_fullsized":"2024-09-21T07:12:23.808124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795490+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:12:23.795490+0000","last_peered":"2024-09-21T07:12:23.795490+0000","last_clean":"2024-09-21T07:12:23.795490+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:12:23.795490+0000","last_undegraded":"2024-09-21T07:12:23.795490+0000","last_fullsized":"2024-09-21T07:12:23.795490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805863+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:12:23.805863+0000","last_peered":"2024-09-21T07:12:23.805863+0000","last_clean":"2024-09-21T07:12:23.805863+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:12:23.805863+0000","last_undegraded":"2024-09-21T07:12:23.805863+0000","last_fullsized":"2024-09-21T07:12:23.805863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801102+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:12:23.801102+0000","last_peered":"2024-09-21T07:12:23.801102+0000","last_clean":"2024-09-21T07:12:23.801102+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:12:23.801102+0000","last_undegraded":"2024-09-21T07:12:23.801102+0000","last_fullsized":"2024-09-21T07:12:23.801102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797345+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:12:23.797345+0000","last_peered":"2024-09-21T07:12:23.797345+0000","last_clean":"2024-09-21T07:12:23.797345+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:12:23.797345+0000","last_undegraded":"2024-09-21T07:12:23.797345+0000","last_fullsized":"2024-09-21T07:12:23.797345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806243+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:12:23.806243+0000","last_peered":"2024-09-21T07:12:23.806243+0000","last_clean":"2024-09-21T07:12:23.806243+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:12:23.806243+0000","last_undegraded":"2024-09-21T07:12:23.806243+0000","last_fullsized":"2024-09-21T07:12:23.806243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810287+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:12:23.810287+0000","last_peered":"2024-09-21T07:12:23.810287+0000","last_clean":"2024-09-21T07:12:23.810287+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:12:23.810287+0000","last_undegraded":"2024-09-21T07:12:23.810287+0000","last_fullsized":"2024-09-21T07:12:23.810287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797647+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:12:23.797647+0000","last_peered":"2024-09-21T07:12:23.797647+0000","last_clean":"2024-09-21T07:12:23.797647+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:12:23.797647+0000","last_undegraded":"2024-09-21T07:12:23.797647+0000","last_fullsized":"2024-09-21T07:12:23.797647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.815349+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:12:23.815349+0000","last_peered":"2024-09-21T07:12:23.815349+0000","last_clean":"2024-09-21T07:12:23.815349+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:12:23.815349+0000","last_undegraded":"2024-09-21T07:12:23.815349+0000","last_fullsized":"2024-09-21T07:12:23.815349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809317+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:12:23.809317+0000","last_peered":"2024-09-21T07:12:23.809317+0000","last_clean":"2024-09-21T07:12:23.809317+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:12:23.809317+0000","last_undegraded":"2024-09-21T07:12:23.809317+0000","last_fullsized":"2024-09-21T07:12:23.809317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795846+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:12:23.795846+0000","last_peered":"2024-09-21T07:12:23.795846+0000","last_clean":"2024-09-21T07:12:23.795846+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:12:23.795846+0000","last_undegraded":"2024-09-21T07:12:23.795846+0000","last_fullsized":"2024-09-21T07:12:23.795846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808162+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:12:23.808162+0000","last_peered":"2024-09-21T07:12:23.808162+0000","last_clean":"2024-09-21T07:12:23.808162+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:12:23.808162+0000","last_undegraded":"2024-09-21T07:12:23.808162+0000","last_fullsized":"2024-09-21T07:12:23.808162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797650+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:12:23.797650+0000","last_peered":"2024-09-21T07:12:23.797650+0000","last_clean":"2024-09-21T07:12:23.797650+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:12:23.797650+0000","last_undegraded":"2024-09-21T07:12:23.797650+0000","last_fullsized":"2024-09-21T07:12:23.797650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811678+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:12:23.811678+0000","last_peered":"2024-09-21T07:12:23.811678+0000","last_clean":"2024-09-21T07:12:23.811678+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:12:23.811678+0000","last_undegraded":"2024-09-21T07:12:23.811678+0000","last_fullsized":"2024-09-21T07:12:23.811678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814831+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:12:23.814831+0000","last_peered":"2024-09-21T07:12:23.814831+0000","last_clean":"2024-09-21T07:12:23.814831+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:12:23.814831+0000","last_undegraded":"2024-09-21T07:12:23.814831+0000","last_fullsized":"2024-09-21T07:12:23.814831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798081+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:12:23.798081+0000","last_peered":"2024-09-21T07:12:23.798081+0000","last_clean":"2024-09-21T07:12:23.798081+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:12:23.798081+0000","last_undegraded":"2024-09-21T07:12:23.798081+0000","last_fullsized":"2024-09-21T07:12:23.798081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797741+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:12:23.797741+0000","last_peered":"2024-09-21T07:12:23.797741+0000","last_clean":"2024-09-21T07:12:23.797741+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:12:23.797741+0000","last_undegraded":"2024-09-21T07:12:23.797741+0000","last_fullsized":"2024-09-21T07:12:23.797741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797093+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:12:23.797093+0000","last_peered":"2024-09-21T07:12:23.797093+0000","last_clean":"2024-09-21T07:12:23.797093+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:12:23.797093+0000","last_undegraded":"2024-09-21T07:12:23.797093+0000","last_fullsized":"2024-09-21T07:12:23.797093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797350+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:12:23.797350+0000","last_peered":"2024-09-21T07:12:23.797350+0000","last_clean":"2024-09-21T07:12:23.797350+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:12:23.797350+0000","last_undegraded":"2024-09-21T07:12:23.797350+0000","last_fullsized":"2024-09-21T07:12:23.797350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814242+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:12:23.814242+0000","last_peered":"2024-09-21T07:12:23.814242+0000","last_clean":"2024-09-21T07:12:23.814242+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:12:23.814242+0000","last_undegraded":"2024-09-21T07:12:23.814242+0000","last_fullsized":"2024-09-21T07:12:23.814242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790030+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:12:23.790030+0000","last_peered":"2024-09-21T07:12:23.790030+0000","last_clean":"2024-09-21T07:12:23.790030+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:12:23.790030+0000","last_undegraded":"2024-09-21T07:12:23.790030+0000","last_fullsized":"2024-09-21T07:12:23.790030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810378+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:12:23.810378+0000","last_peered":"2024-09-21T07:12:23.810378+0000","last_clean":"2024-09-21T07:12:23.810378+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:12:23.810378+0000","last_undegraded":"2024-09-21T07:12:23.810378+0000","last_fullsized":"2024-09-21T07:12:23.810378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809256+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:12:23.809256+0000","last_peered":"2024-09-21T07:12:23.809256+0000","last_clean":"2024-09-21T07:12:23.809256+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:12:23.809256+0000","last_undegraded":"2024-09-21T07:12:23.809256+0000","last_fullsized":"2024-09-21T07:12:23.809256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806255+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:12:23.806255+0000","last_peered":"2024-09-21T07:12:23.806255+0000","last_clean":"2024-09-21T07:12:23.806255+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:12:23.806255+0000","last_undegraded":"2024-09-21T07:12:23.806255+0000","last_fullsized":"2024-09-21T07:12:23.806255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788650+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:12:23.788650+0000","last_peered":"2024-09-21T07:12:23.788650+0000","last_clean":"2024-09-21T07:12:23.788650+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:12:23.788650+0000","last_undegraded":"2024-09-21T07:12:23.788650+0000","last_fullsized":"2024-09-21T07:12:23.788650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809845+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:12:23.809845+0000","last_peered":"2024-09-21T07:12:23.809845+0000","last_clean":"2024-09-21T07:12:23.809845+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:12:23.809845+0000","last_undegraded":"2024-09-21T07:12:23.809845+0000","last_fullsized":"2024-09-21T07:12:23.809845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.789398+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:12:23.789398+0000","last_peered":"2024-09-21T07:12:23.789398+0000","last_clean":"2024-09-21T07:12:23.789398+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:12:23.789398+0000","last_undegraded":"2024-09-21T07:12:23.789398+0000","last_fullsized":"2024-09-21T07:12:23.789398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811791+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:12:23.811791+0000","last_peered":"2024-09-21T07:12:23.811791+0000","last_clean":"2024-09-21T07:12:23.811791+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:12:23.811791+0000","last_undegraded":"2024-09-21T07:12:23.811791+0000","last_fullsized":"2024-09-21T07:12:23.811791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.799236+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:12:23.799236+0000","last_peered":"2024-09-21T07:12:23.799236+0000","last_clean":"2024-09-21T07:12:23.799236+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:12:23.799236+0000","last_undegraded":"2024-09-21T07:12:23.799236+0000","last_fullsized":"2024-09-21T07:12:23.799236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811604+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:12:23.811604+0000","last_peered":"2024-09-21T07:12:23.811604+0000","last_clean":"2024-09-21T07:12:23.811604+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:12:23.811604+0000","last_undegraded":"2024-09-21T07:12:23.811604+0000","last_fullsized":"2024-09-21T07:12:23.811604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80361,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":748,"num_read_kb":456,"num_write":189,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":241083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":515860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972405,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1703936,"data_stored":1179155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135931,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":520170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":515972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495685,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1113137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659211,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":511250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855442,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1107802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018966,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215194,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":511444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":588338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":586286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705772,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":77178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":77627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:12:51.593 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:12:52.018 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:52.284 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:51 smithi027 ceph-mon[30779]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:52.284 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:51 smithi027 ceph-mon[30779]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:51 smithi119 ceph-mon[34519]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:51 smithi119 ceph-mon[34519]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:52.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:51 smithi153 ceph-mon[34747]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:52.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:51 smithi153 ceph-mon[34747]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 -- 172.21.15.27:0/12101419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f7810faf0 msgr2=0x7f8f7810ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 --2- 172.21.15.27:0/12101419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f7810faf0 0x7f8f7810ff00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009900 tx=0x7f8f6002a410 comp rx=0 tx=0).stop 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 -- 172.21.15.27:0/12101419 shutdown_connections 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 --2- 172.21.15.27:0/12101419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f78110440 0x7f8f78116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 --2- 172.21.15.27:0/12101419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f7810faf0 0x7f8f7810ff00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 --2- 172.21.15.27:0/12101419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7810a1b0 0x7f8f7810a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 -- 172.21.15.27:0/12101419 >> 172.21.15.27:0/12101419 conn(0x7f8f780780e0 msgr2=0x7f8f781081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 -- 172.21.15.27:0/12101419 shutdown_connections 2024-09-21T07:12:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.599+0000 7f8f7de00640 1 -- 172.21.15.27:0/12101419 wait complete. 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.600+0000 7f8f7de00640 1 Processor -- start 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f7de00640 1 -- start start 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f7de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f7810a1b0 0x7f8f781a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f7de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7810faf0 0x7f8f781a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f7de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78110440 0x7f8f781a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f7de00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7811a0d0 con 0x7f8f78110440 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f7de00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78119f50 con 0x7f8f7810faf0 2024-09-21T07:12:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f7de00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7811a250 con 0x7f8f7810a1b0 2024-09-21T07:12:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.601+0000 7f8f77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78110440 0x7f8f781a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f7810a1b0 0x7f8f781a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f6e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7810faf0 0x7f8f781a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f7810a1b0 0x7f8f781a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34430/0 (socket says 172.21.15.27:34430) 2024-09-21T07:12:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78110440 0x7f8f781a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45690/0 (socket says 172.21.15.27:45690) 2024-09-21T07:12:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 -- 172.21.15.27:0/2385115207 learned_addr learned my addr 172.21.15.27:0/2385115207 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:52.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 -- 172.21.15.27:0/2385115207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7810faf0 msgr2=0x7f8f781a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:52.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7810faf0 0x7f8f781a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 -- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78110440 msgr2=0x7f8f781a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:52.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78110440 0x7f8f781a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 -- 172.21.15.27:0/2385115207 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f781a6a30 con 0x7f8f7810a1b0 2024-09-21T07:12:52.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.602+0000 7f8f76a00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f7810a1b0 0x7f8f781a0ac0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800e9f0 tx=0x7f8f6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.603+0000 7f8f7ca00640 1 -- 172.21.15.27:0/2385115207 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6800cd80 con 0x7f8f7810a1b0 2024-09-21T07:12:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.603+0000 7f8f7ca00640 1 -- 172.21.15.27:0/2385115207 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f680108e0 con 0x7f8f7810a1b0 2024-09-21T07:12:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.603+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f781a6d20 con 0x7f8f7810a1b0 2024-09-21T07:12:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.603+0000 7f8f7ca00640 1 -- 172.21.15.27:0/2385115207 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68018980 con 0x7f8f7810a1b0 2024-09-21T07:12:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.603+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f781ad480 con 0x7f8f7810a1b0 2024-09-21T07:12:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.604+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7810a5c0 con 0x7f8f7810a1b0 2024-09-21T07:12:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.607+0000 7f8f7ca00640 1 -- 172.21.15.27:0/2385115207 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f8f68010430 con 0x7f8f7810a1b0 2024-09-21T07:12:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.608+0000 7f8f7ca00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f5c0b65f0 0x7f8f5c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.608+0000 7f8f7ca00640 1 -- 172.21.15.27:0/2385115207 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f8f680e8120 con 0x7f8f7810a1b0 2024-09-21T07:12:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.608+0000 7f8f6e000640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f5c0b65f0 0x7f8f5c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.609+0000 7f8f6e000640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f5c0b65f0 0x7f8f5c0b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009230 tx=0x7f8f60031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.610+0000 7f8f7ca00640 1 -- 172.21.15.27:0/2385115207 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f680b27c0 con 0x7f8f7810a1b0 2024-09-21T07:12:52.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.845+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f781a1bf0 con 0x7f8f7810a1b0 2024-09-21T07:12:52.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.846+0000 7f8f7ca00640 1 -- 172.21.15.27:0/2385115207 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8f680b7700 con 0x7f8f7810a1b0 2024-09-21T07:12:52.849 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:52.849 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:12:52.849 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f5c0b65f0 msgr2=0x7f8f5c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f5c0b65f0 0x7f8f5c0b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009230 tx=0x7f8f60031400 comp rx=0 tx=0).stop 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f7810a1b0 msgr2=0x7f8f781a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f7810a1b0 0x7f8f781a0ac0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800e9f0 tx=0x7f8f6800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 shutdown_connections 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8f5c0b65f0 0x7f8f5c0b8b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78110440 0x7f8f781a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7810faf0 0x7f8f781a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 --2- 172.21.15.27:0/2385115207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f7810a1b0 0x7f8f781a0ac0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:52.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.849+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 >> 172.21.15.27:0/2385115207 conn(0x7f8f780780e0 msgr2=0x7f8f78078920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:52.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.850+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 shutdown_connections 2024-09-21T07:12:52.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:52.850+0000 7f8f7de00640 1 -- 172.21.15.27:0/2385115207 wait complete. 2024-09-21T07:12:53.203 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:52 smithi027 ceph-mon[30779]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-21T07:12:53.203 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:52 smithi027 ceph-mon[30779]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:53.203 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:52 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2385115207' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:52 smithi119 ceph-mon[34519]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-21T07:12:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:52 smithi119 ceph-mon[34519]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:52 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2385115207' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:52 smithi153 ceph-mon[34747]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-21T07:12:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:52 smithi153 ceph-mon[34747]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:12:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:52 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2385115207' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:12:53.627 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:12:54.016 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:54.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 -- 172.21.15.27:0/163329946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c40757b0 msgr2=0x7f76c4075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:54.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 --2- 172.21.15.27:0/163329946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c40757b0 0x7f76c4075bc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009900 tx=0x7f76b402a410 comp rx=0 tx=0).stop 2024-09-21T07:12:54.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 -- 172.21.15.27:0/163329946 shutdown_connections 2024-09-21T07:12:54.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 --2- 172.21.15.27:0/163329946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76c41100c0 0x7f76c4112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 --2- 172.21.15.27:0/163329946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c4076100 0x7f76c410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 --2- 172.21.15.27:0/163329946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c40757b0 0x7f76c4075bc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 -- 172.21.15.27:0/163329946 >> 172.21.15.27:0/163329946 conn(0x7f76c407b850 msgr2=0x7f76c407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:54.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 -- 172.21.15.27:0/163329946 shutdown_connections 2024-09-21T07:12:54.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.608+0000 7f76ca600640 1 -- 172.21.15.27:0/163329946 wait complete. 2024-09-21T07:12:54.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.609+0000 7f76ca600640 1 Processor -- start 2024-09-21T07:12:54.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.609+0000 7f76ca600640 1 -- start start 2024-09-21T07:12:54.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.609+0000 7f76ca600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c40757b0 0x7f76c419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:54.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76ca600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76c4076100 0x7f76c419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:54.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76ca600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c41100c0 0x7f76c41a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:54.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76ca600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4115c70 con 0x7f76c41100c0 2024-09-21T07:12:54.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76ca600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4115af0 con 0x7f76c40757b0 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76ca600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4115df0 con 0x7f76c4076100 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c40757b0 0x7f76c419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c40757b0 0x7f76c419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36336/0 (socket says 172.21.15.27:36336) 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3400640 1 -- 172.21.15.27:0/1537216174 learned_addr learned my addr 172.21.15.27:0/1537216174 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3e00640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c41100c0 0x7f76c41a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76baa00640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76c4076100 0x7f76c419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3e00640 1 -- 172.21.15.27:0/1537216174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76c4076100 msgr2=0x7f76c419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3e00640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76c4076100 0x7f76c419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3e00640 1 -- 172.21.15.27:0/1537216174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c40757b0 msgr2=0x7f76c419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3e00640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c40757b0 0x7f76c419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.610+0000 7f76c3e00640 1 -- 172.21.15.27:0/1537216174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c41a2570 con 0x7f76c41100c0 2024-09-21T07:12:54.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.611+0000 7f76c3e00640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c41100c0 0x7f76c41a1e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e940 tx=0x7f76bc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:54.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.611+0000 7f76c0c00640 1 -- 172.21.15.27:0/1537216174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc004d20 con 0x7f76c41100c0 2024-09-21T07:12:54.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.611+0000 7f76c0c00640 1 -- 172.21.15.27:0/1537216174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76bc005020 con 0x7f76c41100c0 2024-09-21T07:12:54.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.611+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c41a2860 con 0x7f76c41100c0 2024-09-21T07:12:54.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.611+0000 7f76c0c00640 1 -- 172.21.15.27:0/1537216174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0116d0 con 0x7f76c41100c0 2024-09-21T07:12:54.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.611+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c41a8fc0 con 0x7f76c41100c0 2024-09-21T07:12:54.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.612+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c4075bc0 con 0x7f76c41100c0 2024-09-21T07:12:54.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.615+0000 7f76c0c00640 1 -- 172.21.15.27:0/1537216174 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f76bc005460 con 0x7f76c41100c0 2024-09-21T07:12:54.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.616+0000 7f76c0c00640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76a80b65f0 0x7f76a80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:54.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.616+0000 7f76c0c00640 1 -- 172.21.15.27:0/1537216174 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f76bc0e7a50 con 0x7f76c41100c0 2024-09-21T07:12:54.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.617+0000 7f76c3400640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76a80b65f0 0x7f76a80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:54.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.617+0000 7f76c3400640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76a80b65f0 0x7f76a80b8b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76b4031440 tx=0x7f76b40058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:54.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.620+0000 7f76c0c00640 1 -- 172.21.15.27:0/1537216174 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f76bc016030 con 0x7f76c41100c0 2024-09-21T07:12:54.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.822+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76c419d730 con 0x7f76c41100c0 2024-09-21T07:12:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.823+0000 7f76c0c00640 1 -- 172.21.15.27:0/1537216174 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25234 (secure 0 0 0) 0x7f76bc0b2090 con 0x7f76c41100c0 2024-09-21T07:12:54.826 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:12:54.826 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:11:55.826198+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10},{"from":5,"to":1}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:12:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76a80b65f0 msgr2=0x7f76a80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76a80b65f0 0x7f76a80b8b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76b4031440 tx=0x7f76b40058d0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c41100c0 msgr2=0x7f76c41a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c41100c0 0x7f76c41a1e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e940 tx=0x7f76bc00ee00 comp rx=0 tx=0).stop 2024-09-21T07:12:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 shutdown_connections 2024-09-21T07:12:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76c4076100 0x7f76c419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76a80b65f0 0x7f76a80b8b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c41100c0 0x7f76c41a1e40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 --2- 172.21.15.27:0/1537216174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c40757b0 0x7f76c419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 >> 172.21.15.27:0/1537216174 conn(0x7f76c407b850 msgr2=0x7f76c4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 shutdown_connections 2024-09-21T07:12:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:54.827+0000 7f76ca600640 1 -- 172.21.15.27:0/1537216174 wait complete. 2024-09-21T07:12:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:54 smithi027 ceph-mon[30779]: pgmap v374: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-21T07:12:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:54 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1537216174' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:54 smithi119 ceph-mon[34519]: pgmap v374: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-21T07:12:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:54 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1537216174' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:54 smithi153 ceph-mon[34747]: pgmap v374: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-21T07:12:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:54 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1537216174' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:12:55.652 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T07:12:55.652 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T07:12:55.662 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T07:12:55.662 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:12:55.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T07:12:55.663 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T07:12:55.663 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:12:55.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:12:55.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 -- 172.21.15.27:0/1103947226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f4076100 msgr2=0x7fb5f410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:55.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 --2- 172.21.15.27:0/1103947226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f4076100 0x7fb5f410f920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009860 tx=0x7fb5e002a340 comp rx=0 tx=0).stop 2024-09-21T07:12:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 -- 172.21.15.27:0/1103947226 shutdown_connections 2024-09-21T07:12:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 --2- 172.21.15.27:0/1103947226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5f410fe60 0x7fb5f41122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 --2- 172.21.15.27:0/1103947226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f4076100 0x7fb5f410f920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 --2- 172.21.15.27:0/1103947226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 0x7fb5f4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 -- 172.21.15.27:0/1103947226 >> 172.21.15.27:0/1103947226 conn(0x7fb5f40ff330 msgr2=0x7fb5f4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 -- 172.21.15.27:0/1103947226 shutdown_connections 2024-09-21T07:12:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.893+0000 7fb5fa600640 1 -- 172.21.15.27:0/1103947226 wait complete. 2024-09-21T07:12:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.894+0000 7fb5fa600640 1 Processor -- start 2024-09-21T07:12:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5fa600640 1 -- start start 2024-09-21T07:12:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5fa600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 0x7fb5f419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5fa600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5f4076100 0x7fb5f419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5fa600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f410fe60 0x7fb5f41a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5fa600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4115a30 con 0x7fb5f40757b0 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5fa600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41158b0 con 0x7fb5f410fe60 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5fa600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4115bb0 con 0x7fb5f4076100 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5f9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 0x7fb5f419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5f9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 0x7fb5f419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45718/0 (socket says 172.21.15.27:45718) 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5f9200640 1 -- 172.21.15.27:0/3361185519 learned_addr learned my addr 172.21.15.27:0/3361185519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.895+0000 7fb5f3e00640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5f4076100 0x7fb5f419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9c00640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f410fe60 0x7fb5f41a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9200640 1 -- 172.21.15.27:0/3361185519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5f4076100 msgr2=0x7fb5f419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9200640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5f4076100 0x7fb5f419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9200640 1 -- 172.21.15.27:0/3361185519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f410fe60 msgr2=0x7fb5f41a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9200640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f410fe60 0x7fb5f41a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9200640 1 -- 172.21.15.27:0/3361185519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f41a2440 con 0x7fb5f40757b0 2024-09-21T07:12:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f3e00640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5f4076100 0x7fb5f419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9c00640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f410fe60 0x7fb5f41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.896+0000 7fb5f9200640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 0x7fb5f419c510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9c0 tx=0x7fb5e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.897+0000 7fb5f1600640 1 -- 172.21.15.27:0/3361185519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800cde0 con 0x7fb5f40757b0 2024-09-21T07:12:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.897+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f41a2730 con 0x7fb5f40757b0 2024-09-21T07:12:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.897+0000 7fb5f1600640 1 -- 172.21.15.27:0/3361185519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e80035f0 con 0x7fb5f40757b0 2024-09-21T07:12:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.897+0000 7fb5f1600640 1 -- 172.21.15.27:0/3361185519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8010690 con 0x7fb5f40757b0 2024-09-21T07:12:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.897+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f41a8ec0 con 0x7fb5f40757b0 2024-09-21T07:12:55.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.898+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f4075bc0 con 0x7fb5f40757b0 2024-09-21T07:12:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.901+0000 7fb5f1600640 1 -- 172.21.15.27:0/3361185519 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fb5e8004000 con 0x7fb5f40757b0 2024-09-21T07:12:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.902+0000 7fb5f1600640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5dc0b65f0 0x7fb5dc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.902+0000 7fb5f1600640 1 -- 172.21.15.27:0/3361185519 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb5e80e7dc0 con 0x7fb5f40757b0 2024-09-21T07:12:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.903+0000 7fb5f3e00640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5dc0b65f0 0x7fb5dc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.904+0000 7fb5f3e00640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5dc0b65f0 0x7fb5dc0b8b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0035040 tx=0x7fb5e0030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:55.904+0000 7fb5f1600640 1 -- 172.21.15.27:0/3361185519 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5e80ae0b0 con 0x7fb5f40757b0 2024-09-21T07:12:56.083 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:56.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.165+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb5f410a110 con 0x7fb5dc0b65f0 2024-09-21T07:12:56.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.353+0000 7fb5f1600640 1 -- 172.21.15.27:0/3361185519 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb5f410a110 con 0x7fb5dc0b65f0 2024-09-21T07:12:56.356 INFO:teuthology.orchestra.run.smithi027.stdout:Schedule set for path / 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5dc0b65f0 msgr2=0x7fb5dc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5dc0b65f0 0x7fb5dc0b8b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0035040 tx=0x7fb5e0030340 comp rx=0 tx=0).stop 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 msgr2=0x7fb5f419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 0x7fb5f419c510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9c0 tx=0x7fb5e800ee80 comp rx=0 tx=0).stop 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 shutdown_connections 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5f4076100 0x7fb5f419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5dc0b65f0 0x7fb5dc0b8b00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f410fe60 0x7fb5f41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 --2- 172.21.15.27:0/3361185519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f40757b0 0x7fb5f419c510 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 >> 172.21.15.27:0/3361185519 conn(0x7fb5f40ff330 msgr2=0x7fb5f4109a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 shutdown_connections 2024-09-21T07:12:56.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.356+0000 7fb5fa600640 1 -- 172.21.15.27:0/3361185519 wait complete. 2024-09-21T07:12:56.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.620+0000 7f6ef5400640 1 -- 172.21.15.27:0/2175067924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 msgr2=0x7f6ef0075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.620+0000 7f6ef5400640 1 --2- 172.21.15.27:0/2175067924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 0x7f6ef0075bc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009900 tx=0x7f6ee002a410 comp rx=0 tx=0).stop 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.621+0000 7f6ef5400640 1 -- 172.21.15.27:0/2175067924 shutdown_connections 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.621+0000 7f6ef5400640 1 --2- 172.21.15.27:0/2175067924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef0110030 0x7f6ef0112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.621+0000 7f6ef5400640 1 --2- 172.21.15.27:0/2175067924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ef0076100 0x7f6ef010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.621+0000 7f6ef5400640 1 --2- 172.21.15.27:0/2175067924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 0x7f6ef0075bc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.621+0000 7f6ef5400640 1 -- 172.21.15.27:0/2175067924 >> 172.21.15.27:0/2175067924 conn(0x7f6ef00ff520 msgr2=0x7f6ef0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.621+0000 7f6ef5400640 1 -- 172.21.15.27:0/2175067924 shutdown_connections 2024-09-21T07:12:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.621+0000 7f6ef5400640 1 -- 172.21.15.27:0/2175067924 wait complete. 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 Processor -- start 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 -- start start 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 0x7f6ef019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ef0076100 0x7f6ef019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef0110030 0x7f6ef01a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0115c70 con 0x7f6ef0110030 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0115af0 con 0x7f6ef0076100 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.622+0000 7f6ef5400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0115df0 con 0x7f6ef00757b0 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eee800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 0x7f6ef019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eee800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 0x7f6ef019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34500/0 (socket says 172.21.15.27:34500) 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ef0076100 0x7f6ef019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eee800640 1 -- 172.21.15.27:0/378004832 learned_addr learned my addr 172.21.15.27:0/378004832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eef200640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef0110030 0x7f6ef01a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eede00640 1 -- 172.21.15.27:0/378004832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 msgr2=0x7f6ef019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eede00640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 0x7f6ef019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eede00640 1 -- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef0110030 msgr2=0x7f6ef01a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eede00640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef0110030 0x7f6ef01a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eede00640 1 -- 172.21.15.27:0/378004832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef01a25e0 con 0x7f6ef0076100 2024-09-21T07:12:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.623+0000 7f6eede00640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ef0076100 0x7f6ef019cc50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00777d0 tx=0x7f6edc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.624+0000 7f6ee7400640 1 -- 172.21.15.27:0/378004832 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00cc40 con 0x7f6ef0076100 2024-09-21T07:12:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.624+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef01a28d0 con 0x7f6ef0076100 2024-09-21T07:12:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.624+0000 7f6ee7400640 1 -- 172.21.15.27:0/378004832 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6edc00cde0 con 0x7f6ef0076100 2024-09-21T07:12:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.624+0000 7f6ee7400640 1 -- 172.21.15.27:0/378004832 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc004a00 con 0x7f6ef0076100 2024-09-21T07:12:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.624+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a9060 con 0x7f6ef0076100 2024-09-21T07:12:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.625+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef0075bc0 con 0x7f6ef0076100 2024-09-21T07:12:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.628+0000 7f6ee7400640 1 -- 172.21.15.27:0/378004832 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f6edc004ba0 con 0x7f6ef0076100 2024-09-21T07:12:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.629+0000 7f6ee7400640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ed40b65f0 0x7f6ed40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.629+0000 7f6ee7400640 1 -- 172.21.15.27:0/378004832 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f6edc0e7a30 con 0x7f6ef0076100 2024-09-21T07:12:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.630+0000 7f6eee800640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ed40b65f0 0x7f6ed40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.630+0000 7f6eee800640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ed40b65f0 0x7f6ed40b8b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00096c0 tx=0x7f6ee0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.631+0000 7f6ee7400640 1 -- 172.21.15.27:0/378004832 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6edc0add20 con 0x7f6ef0076100 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 -- 172.21.15.27:0/3130826056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4110140 msgr2=0x7feac4112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 --2- 172.21.15.27:0/3130826056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4110140 0x7feac4112560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feabc009ca0 tx=0x7feabc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 -- 172.21.15.27:0/3130826056 shutdown_connections 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 --2- 172.21.15.27:0/3130826056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4110140 0x7feac4112560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 --2- 172.21.15.27:0/3130826056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac4076080 0x7feac410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 --2- 172.21.15.27:0/3130826056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 0x7feac4075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 -- 172.21.15.27:0/3130826056 >> 172.21.15.27:0/3130826056 conn(0x7feac407b850 msgr2=0x7feac407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 -- 172.21.15.27:0/3130826056 shutdown_connections 2024-09-21T07:12:56.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.711+0000 7feacb800640 1 -- 172.21.15.27:0/3130826056 wait complete. 2024-09-21T07:12:56.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 Processor -- start 2024-09-21T07:12:56.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 -- start start 2024-09-21T07:12:56.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 0x7feac419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4076080 0x7feac419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac4110140 0x7feac41a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feac4115cd0 con 0x7feac4110140 2024-09-21T07:12:56.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feac4115b50 con 0x7feac4075730 2024-09-21T07:12:56.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.713+0000 7feacb800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feac4115e50 con 0x7feac4076080 2024-09-21T07:12:56.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac4110140 0x7feac41a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 0x7feac419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4076080 0x7feac419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4076080 0x7feac419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34520/0 (socket says 172.21.15.27:34520) 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac4110140 0x7feac41a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45770/0 (socket says 172.21.15.27:45770) 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac9800640 1 -- 172.21.15.27:0/3106131750 learned_addr learned my addr 172.21.15.27:0/3106131750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 0x7feac419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36396/0 (socket says 172.21.15.27:36396) 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac3e00640 1 -- 172.21.15.27:0/3106131750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 msgr2=0x7feac419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac3e00640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 0x7feac419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac3e00640 1 -- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac4110140 msgr2=0x7feac41a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac3e00640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac4110140 0x7feac41a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac3e00640 1 -- 172.21.15.27:0/3106131750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac41a25f0 con 0x7feac4076080 2024-09-21T07:12:56.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.714+0000 7feac8e00640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 0x7feac419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:56.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.715+0000 7feac3e00640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4076080 0x7feac419cc20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feac406c9f0 tx=0x7feab000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.715+0000 7feac1600640 1 -- 172.21.15.27:0/3106131750 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab000cc40 con 0x7feac4076080 2024-09-21T07:12:56.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.715+0000 7feac1600640 1 -- 172.21.15.27:0/3106131750 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab000cde0 con 0x7feac4076080 2024-09-21T07:12:56.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.715+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac41a28e0 con 0x7feac4076080 2024-09-21T07:12:56.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.715+0000 7feac1600640 1 -- 172.21.15.27:0/3106131750 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0018780 con 0x7feac4076080 2024-09-21T07:12:56.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.715+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41a9040 con 0x7feac4076080 2024-09-21T07:12:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.720+0000 7feac1600640 1 -- 172.21.15.27:0/3106131750 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7feab0010a90 con 0x7feac4076080 2024-09-21T07:12:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.720+0000 7fead24a4740 1 Processor -- start 2024-09-21T07:12:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.720+0000 7fead24a4740 1 -- start start 2024-09-21T07:12:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7fead24a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b123d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7fead24a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b124250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7fead24a4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b124790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7fead24a4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b06b099e90 con 0x55b06ad5b670 2024-09-21T07:12:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7fead24a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b06b0892c0 con 0x55b06ad53490 2024-09-21T07:12:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7fead24a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b06b08d0d0 con 0x55b06ad7c350 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7feac9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b123d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7feac3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b124790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7feac8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b124250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7feac9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b123d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34528/0 (socket says 172.21.15.27:34528) 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.721+0000 7feac9800640 1 -- 172.21.15.27:0/744917926 learned_addr learned my addr 172.21.15.27:0/744917926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.722+0000 7feac9800640 1 -- 172.21.15.27:0/744917926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 msgr2=0x55b06b124250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.722+0000 7feac9800640 1 --2- 172.21.15.27:0/744917926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b124250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.722+0000 7feac9800640 1 -- 172.21.15.27:0/744917926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 msgr2=0x55b06b124790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.722+0000 7feac9800640 1 --2- 172.21.15.27:0/744917926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b124790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.722+0000 7feac9800640 1 -- 172.21.15.27:0/744917926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06ade7930 con 0x55b06ad7c350 2024-09-21T07:12:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.722+0000 7feac9800640 1 --2- 172.21.15.27:0/744917926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b123d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feabc0094c0 tx=0x7feabc009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da93fd3ae964f5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.722+0000 7feac1600640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feaac0b6670 0x7feaac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.723+0000 7feab6000640 1 -- 172.21.15.27:0/744917926 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc031db0 con 0x55b06ad7c350 2024-09-21T07:12:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.723+0000 7feac1600640 1 -- 172.21.15.27:0/3106131750 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7feab00e7b30 con 0x7feac4076080 2024-09-21T07:12:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.723+0000 7feab6000640 1 -- 172.21.15.27:0/744917926 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feabc036030 con 0x55b06ad7c350 2024-09-21T07:12:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.723+0000 7feab6000640 1 -- 172.21.15.27:0/744917926 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0042a0 con 0x55b06ad7c350 2024-09-21T07:12:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.723+0000 7feac8e00640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feaac0b6670 0x7feaac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 -- 172.21.15.27:0/744917926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 msgr2=0x55b06b123d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 --2- 172.21.15.27:0/744917926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b123d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feabc0094c0 tx=0x7feabc009a10 comp rx=0 tx=0).stop 2024-09-21T07:12:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7feac8e00640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feaac0b6670 0x7feaac0b8b80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55b06b122990 tx=0x7feab8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 -- 172.21.15.27:0/744917926 shutdown_connections 2024-09-21T07:12:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 --2- 172.21.15.27:0/744917926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b124790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 --2- 172.21.15.27:0/744917926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b124250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 --2- 172.21.15.27:0/744917926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b123d10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 -- 172.21.15.27:0/744917926 >> 172.21.15.27:0/744917926 conn(0x55b06ad70740 msgr2=0x55b06b121c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 -- 172.21.15.27:0/744917926 shutdown_connections 2024-09-21T07:12:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.724+0000 7fead24a4740 1 -- 172.21.15.27:0/744917926 wait complete. 2024-09-21T07:12:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.725+0000 7fead24a4740 1 -- start start 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.725+0000 7fead24a4740 1 Processor -- start 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7fead24a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b12f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7fead24a4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b12fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7fead24a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b13b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7fead24a4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b06aeea890 con 0x55b06ad5b670 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7fead24a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b06b024160 con 0x55b06ad53490 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7fead24a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b06b132310 con 0x55b06ad7c350 2024-09-21T07:12:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7fead24a4740 10 client.0 _ll_register_callbacks cb 0x55b06b040a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:56.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7feac8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b12f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7feac8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b12f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36410/0 (socket says 172.21.15.27:36410) 2024-09-21T07:12:56.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7feac8e00640 1 -- 172.21.15.27:0/1419952129 learned_addr learned my addr 172.21.15.27:0/1419952129 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7feac3e00640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b12fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.726+0000 7feac9800640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b13b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac8e00640 1 -- 172.21.15.27:0/1419952129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 msgr2=0x55b06b13b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac8e00640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b13b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac8e00640 1 -- 172.21.15.27:0/1419952129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 msgr2=0x55b06b12fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac8e00640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b12fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac8e00640 1 -- 172.21.15.27:0/1419952129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06b13bc10 con 0x55b06ad53490 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac9800640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac3e00640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b12fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feac8e00640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b12f920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feab8003620 tx=0x7feab8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd469203d3221717 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feaaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7feaaaa00640 1 -- 172.21.15.27:0/1419952129 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab800f8b0 con 0x55b06ad53490 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.727+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc036030 con 0x55b06ad53490 2024-09-21T07:12:56.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.728+0000 7feaaaa00640 1 -- 172.21.15.27:0/1419952129 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7feab800fa50 con 0x55b06ad53490 2024-09-21T07:12:56.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.728+0000 7feaaaa00640 1 -- 172.21.15.27:0/1419952129 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feab800fc50 con 0x55b06ad53490 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.728+0000 7fead24a4740 10 client.25490 fetch_fsmap learned FSMap version 27 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.728+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b06b13e5f0 con 0x55b06ad53490 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.728+0000 7feaaaa00640 1 -- 172.21.15.27:0/1419952129 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feabc036030 con 0x55b06ad53490 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7feaaaa00640 1 -- 172.21.15.27:0/1419952129 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feab8027030 con 0x55b06ad53490 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7fead24a4740 10 client.25490 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7fead24a4740 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7fead24a4740 20 client.25490 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7fead24a4740 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad3acd0 0x55b06b141a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7fead24a4740 4 client.25490 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b06b09bde0 con 0x55b06ad3acd0 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.729+0000 7feac3e00640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad3acd0 0x55b06b141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.730+0000 7feac3e00640 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad3acd0 0x55b06b141a50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dcd2a4559f191e0 server_cookie=d86f7178974ae4d4 in_seq=0 out_seq=0 2024-09-21T07:12:56.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.730+0000 7feaaaa00640 10 client.25490 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:56.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.730+0000 7feaaaa00640 1 -- 172.21.15.27:0/1419952129 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b06b09bde0 con 0x55b06ad3acd0 2024-09-21T07:12:56.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.730+0000 7feaaaa00640 10 client.25490 handle_command_reply: tid=0 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 client.25490 shutdown 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 msgr2=0x55b06b12f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b12f920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feab8003620 tx=0x7feab8004730 comp rx=0 tx=0).stop 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 shutdown_connections 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad3acd0 0x55b06b141a50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad7c350 0x55b06b13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b12fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 --2- 172.21.15.27:0/1419952129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad53490 0x55b06b12f920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7feaaaa00640 0 client.25490 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 >> 172.21.15.27:0/1419952129 conn(0x55b06ad70740 msgr2=0x55b06b120280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 shutdown_connections 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 1 -- 172.21.15.27:0/1419952129 wait complete. 2024-09-21T07:12:56.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.731+0000 7fead24a4740 20 client.25490 trim_cache size 0 max 16384 2024-09-21T07:12:56.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.750+0000 7fead24a4740 1 Processor -- start 2024-09-21T07:12:56.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.750+0000 7fead24a4740 1 -- start start 2024-09-21T07:12:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7fead24a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad53490 0x55b06b120280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7fead24a4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b1207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7fead24a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b128e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7fead24a4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feac41b5810 con 0x55b06ad5b670 2024-09-21T07:12:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7fead24a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feac41b30e0 con 0x55b06ad7c350 2024-09-21T07:12:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7fead24a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feac41b0f30 con 0x55b06ad53490 2024-09-21T07:12:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7feac3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b1207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7feac3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b1207c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45792/0 (socket says 172.21.15.27:45792) 2024-09-21T07:12:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7feac3e00640 1 -- 172.21.15.27:0/212351892 learned_addr learned my addr 172.21.15.27:0/212351892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7feac8e00640 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad53490 0x55b06b120280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7feac9800640 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b128e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7feac3e00640 1 -- 172.21.15.27:0/212351892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad53490 msgr2=0x55b06b120280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.751+0000 7feac3e00640 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad53490 0x55b06b120280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.752+0000 7feac3e00640 1 -- 172.21.15.27:0/212351892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 msgr2=0x55b06b128e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.752+0000 7feac3e00640 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b128e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.752+0000 7feac3e00640 1 -- 172.21.15.27:0/212351892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06b12afb0 con 0x55b06ad5b670 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.752+0000 7feac3e00640 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b1207c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feab00e55f0 tx=0x7feab00a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3785f62299ad89b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.752+0000 7feab5600640 1 -- 172.21.15.27:0/212351892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00ad0f0 con 0x55b06ad5b670 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.752+0000 7feab5600640 1 -- 172.21.15.27:0/212351892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab00ad2b0 con 0x55b06ad5b670 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.752+0000 7feab5600640 1 -- 172.21.15.27:0/212351892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00acf10 con 0x55b06ad5b670 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 -- 172.21.15.27:0/212351892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 msgr2=0x55b06b1207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b1207c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feab00e55f0 tx=0x7feab00a9040 comp rx=0 tx=0).stop 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 -- 172.21.15.27:0/212351892 shutdown_connections 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b128e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad5b670 0x55b06b1207c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 --2- 172.21.15.27:0/212351892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad53490 0x55b06b120280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 -- 172.21.15.27:0/212351892 >> 172.21.15.27:0/212351892 conn(0x55b06ad70740 msgr2=0x55b06b12dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 -- 172.21.15.27:0/212351892 shutdown_connections 2024-09-21T07:12:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.753+0000 7fead24a4740 1 -- 172.21.15.27:0/212351892 wait complete. 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.754+0000 7fead24a4740 1 -- start start 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.754+0000 7fead24a4740 1 Processor -- start 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.754+0000 7fead24a4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad53490 0x55b06b137570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.754+0000 7fead24a4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad5b670 0x55b06b137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7fead24a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b132520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7fead24a4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feac41b15a0 con 0x55b06ad53490 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7fead24a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feac41b2a10 con 0x55b06ad7c350 2024-09-21T07:12:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7fead24a4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feac41b1c70 con 0x55b06ad5b670 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7fead24a4740 10 client.0 _ll_register_callbacks cb 0x7feab8009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad5b670 0x55b06b137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad53490 0x55b06b137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad53490 0x55b06b137570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45800/0 (socket says 172.21.15.27:45800) 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad5b670 0x55b06b137ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34546/0 (socket says 172.21.15.27:34546) 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 -- 172.21.15.27:0/2566050533 learned_addr learned my addr 172.21.15.27:0/2566050533 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac9800640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b132520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 -- 172.21.15.27:0/2566050533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad5b670 msgr2=0x55b06b137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad5b670 0x55b06b137ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 -- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 msgr2=0x55b06b132520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b132520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac8e00640 1 -- 172.21.15.27:0/2566050533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06b132a60 con 0x55b06ad53490 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac3e00640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad5b670 0x55b06b137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.755+0000 7feac9800640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b132520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:12:56.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.756+0000 7feac8e00640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad53490 0x55b06b137570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feab8012f40 tx=0x7feab8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ccdae3e13aac64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:56.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.756+0000 7feaaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:12:56.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.756+0000 7feaaaa00640 1 -- 172.21.15.27:0/2566050533 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab801d950 con 0x55b06ad53490 2024-09-21T07:12:56.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.756+0000 7feaaaa00640 1 -- 172.21.15.27:0/2566050533 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7feab801dd10 con 0x55b06ad53490 2024-09-21T07:12:56.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.756+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab00ad2b0 con 0x55b06ad53490 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7feaaaa00640 1 -- 172.21.15.27:0/2566050533 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feab8042050 con 0x55b06ad53490 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b06b1aca70 con 0x55b06ad53490 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7feaaaa00640 1 -- 172.21.15.27:0/2566050533 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab00ad2b0 con 0x55b06ad53490 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7feaaaa00640 1 -- 172.21.15.27:0/2566050533 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feab803d030 con 0x55b06ad53490 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 20 client.15468 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad10080 0x55b06b1af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 4 client.15468 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.757+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b06b097890 con 0x55b06ad10080 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.758+0000 7feac3e00640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad10080 0x55b06b1af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.758+0000 7feac3e00640 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad10080 0x55b06b1af190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73fcdbcfebbe861 server_cookie=6af82e166b548bf9 in_seq=0 out_seq=0 2024-09-21T07:12:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.758+0000 7feaaaa00640 10 client.15468 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7feaaaa00640 1 -- 172.21.15.27:0/2566050533 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b06b097890 con 0x55b06ad10080 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7feaaaa00640 10 client.15468 handle_command_reply: tid=0 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 client.15468 shutdown 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad53490 msgr2=0x55b06b137570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad53490 0x55b06b137570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feab8012f40 tx=0x7feab8003f40 comp rx=0 tx=0).stop 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 shutdown_connections 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b06ad10080 0x55b06b1af190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b06ad5b670 0x55b06b137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b06ad7c350 0x55b06b132520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.760+0000 7fead24a4740 1 --2- 172.21.15.27:0/2566050533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b06ad53490 0x55b06b137570 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.761+0000 7feaaaa00640 0 client.15468 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.761+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 >> 172.21.15.27:0/2566050533 conn(0x55b06ad70740 msgr2=0x55b06b12df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.761+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 shutdown_connections 2024-09-21T07:12:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.761+0000 7fead24a4740 1 -- 172.21.15.27:0/2566050533 wait complete. 2024-09-21T07:12:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.761+0000 7fead24a4740 20 client.15468 trim_cache size 0 max 16384 2024-09-21T07:12:56.764 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:56.764 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:12:56.764 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b", 2024-09-21T07:12:56.764 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:12:56.764 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feaac0b6670 msgr2=0x7feaac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feaac0b6670 0x7feaac0b8b80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55b06b122990 tx=0x7feab8005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4076080 msgr2=0x7feac419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4076080 0x7feac419cc20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feac406c9f0 tx=0x7feab000ee50 comp rx=0 tx=0).stop 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 shutdown_connections 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feac4076080 0x7feac419cc20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feaac0b6670 0x7feaac0b8b80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac4110140 0x7feac41a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 --2- 172.21.15.27:0/3106131750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac4075730 0x7feac419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 >> 172.21.15.27:0/3106131750 conn(0x7feac407b850 msgr2=0x7feac4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 shutdown_connections 2024-09-21T07:12:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.762+0000 7feacb800640 1 -- 172.21.15.27:0/3106131750 wait complete. 2024-09-21T07:12:56.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.871+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f6ef010a250 con 0x7f6ed40b65f0 2024-09-21T07:12:56.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.946+0000 7f6ee7400640 1 -- 172.21.15.27:0/378004832 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f6ef010a250 con 0x7f6ed40b65f0 2024-09-21T07:12:56.948 INFO:teuthology.orchestra.run.smithi027.stdout:Retention added to path / 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ed40b65f0 msgr2=0x7f6ed40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ed40b65f0 0x7f6ed40b8b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00096c0 tx=0x7f6ee0005a20 comp rx=0 tx=0).stop 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ef0076100 msgr2=0x7f6ef019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ef0076100 0x7f6ef019cc50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00777d0 tx=0x7f6edc00ee50 comp rx=0 tx=0).stop 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 shutdown_connections 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ed40b65f0 0x7f6ed40b8b00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef0110030 0x7f6ef01a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ef0076100 0x7f6ef019cc50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 --2- 172.21.15.27:0/378004832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ef00757b0 0x7f6ef019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 >> 172.21.15.27:0/378004832 conn(0x7f6ef00ff520 msgr2=0x7f6ef0109b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 shutdown_connections 2024-09-21T07:12:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:56.949+0000 7f6ef5400640 1 -- 172.21.15.27:0/378004832 wait complete. 2024-09-21T07:12:56.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T07:12:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:56 smithi027 ceph-mon[30779]: pgmap v375: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T07:12:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 -- 172.21.15.27:0/4287131945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d410bd90 msgr2=0x7fb0d41125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/4287131945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d410bd90 0x7fb0d41125c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009ca0 tx=0x7fb0cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 -- 172.21.15.27:0/4287131945 shutdown_connections 2024-09-21T07:12:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/4287131945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d410bd90 0x7fb0d41125c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/4287131945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d410b440 0x7fb0d410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/4287131945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d4105b10 0x7fb0d4105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 -- 172.21.15.27:0/4287131945 >> 172.21.15.27:0/4287131945 conn(0x7fb0d4101550 msgr2=0x7fb0d41039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:57.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 -- 172.21.15.27:0/4287131945 shutdown_connections 2024-09-21T07:12:57.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.205+0000 7fb0d3e00640 1 -- 172.21.15.27:0/4287131945 wait complete. 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.206+0000 7fb0d3e00640 1 Processor -- start 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.206+0000 7fb0d3e00640 1 -- start start 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d4105b10 0x7fb0d419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d410b440 0x7fb0d419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d410bd90 0x7fb0d41a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4115aa0 con 0x7fb0d4105b10 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4115920 con 0x7fb0d410b440 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4115c20 con 0x7fb0d410bd90 2024-09-21T07:12:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d4105b10 0x7fb0d419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d4105b10 0x7fb0d419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45828/0 (socket says 172.21.15.27:45828) 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 -- 172.21.15.27:0/785770600 learned_addr learned my addr 172.21.15.27:0/785770600 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d3400640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d410bd90 0x7fb0d41a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d410b440 0x7fb0d419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 -- 172.21.15.27:0/785770600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d410bd90 msgr2=0x7fb0d41a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d410bd90 0x7fb0d41a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 -- 172.21.15.27:0/785770600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d410b440 msgr2=0x7fb0d419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d410b440 0x7fb0d419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.207+0000 7fb0d2a00640 1 -- 172.21.15.27:0/785770600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d41a23b0 con 0x7fb0d4105b10 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.208+0000 7fb0d2a00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d4105b10 0x7fb0d419c480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9f0 tx=0x7fb0c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.208+0000 7fb0cb400640 1 -- 172.21.15.27:0/785770600 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400cd80 con 0x7fb0d4105b10 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.208+0000 7fb0cb400640 1 -- 172.21.15.27:0/785770600 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c40108e0 con 0x7fb0d4105b10 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.208+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d41a26a0 con 0x7fb0d4105b10 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.208+0000 7fb0cb400640 1 -- 172.21.15.27:0/785770600 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4018980 con 0x7fb0d4105b10 2024-09-21T07:12:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.208+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d41a8e30 con 0x7fb0d4105b10 2024-09-21T07:12:57.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.209+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d4105f20 con 0x7fb0d4105b10 2024-09-21T07:12:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.211+0000 7fb0cb400640 1 -- 172.21.15.27:0/785770600 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fb0c4010430 con 0x7fb0d4105b10 2024-09-21T07:12:57.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.213+0000 7fb0cb400640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0b80b65f0 0x7fb0b80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.213+0000 7fb0cb400640 1 -- 172.21.15.27:0/785770600 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb0c40e7d00 con 0x7fb0d4105b10 2024-09-21T07:12:57.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.213+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0b80b65f0 0x7fb0b80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.214+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0b80b65f0 0x7fb0b80b8b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4104d30 tx=0x7fb0bc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:57.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.217+0000 7fb0cb400640 1 -- 172.21.15.27:0/785770600 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0c40b2340 con 0x7fb0d4105b10 2024-09-21T07:12:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:56 smithi119 ceph-mon[34519]: pgmap v375: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T07:12:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:56 smithi153 ceph-mon[34747]: pgmap v375: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T07:12:57.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.441+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb0d4063800 con 0x7fb0b80b65f0 2024-09-21T07:12:57.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.450+0000 7fb0cb400640 1 -- 172.21.15.27:0/785770600 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb0d4063800 con 0x7fb0b80b65f0 2024-09-21T07:12:57.451 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T07:12:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T07:12:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0b80b65f0 msgr2=0x7fb0b80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0b80b65f0 0x7fb0b80b8b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4104d30 tx=0x7fb0bc005f90 comp rx=0 tx=0).stop 2024-09-21T07:12:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d4105b10 msgr2=0x7fb0d419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d4105b10 0x7fb0d419c480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9f0 tx=0x7fb0c400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 shutdown_connections 2024-09-21T07:12:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0d410bd90 0x7fb0d41a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0b80b65f0 0x7fb0b80b8b00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d410b440 0x7fb0d419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 --2- 172.21.15.27:0/785770600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d4105b10 0x7fb0d419c480 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 >> 172.21.15.27:0/785770600 conn(0x7fb0d4101550 msgr2=0x7fb0d410aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:57.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 shutdown_connections 2024-09-21T07:12:57.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.452+0000 7fb0d3e00640 1 -- 172.21.15.27:0/785770600 wait complete. 2024-09-21T07:12:57.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T07:12:57.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 -- 172.21.15.27:0/3114631961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84103910 msgr2=0x7f5d84103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 --2- 172.21.15.27:0/3114631961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84103910 0x7f5d84103d20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009860 tx=0x7f5d7402a340 comp rx=0 tx=0).stop 2024-09-21T07:12:57.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 -- 172.21.15.27:0/3114631961 shutdown_connections 2024-09-21T07:12:57.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 --2- 172.21.15.27:0/3114631961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84109b90 0x7f5d841103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 --2- 172.21.15.27:0/3114631961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d84109240 0x7f5d84109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 --2- 172.21.15.27:0/3114631961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84103910 0x7f5d84103d20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 -- 172.21.15.27:0/3114631961 >> 172.21.15.27:0/3114631961 conn(0x7f5d840ff330 msgr2=0x7f5d84101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:57.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 -- 172.21.15.27:0/3114631961 shutdown_connections 2024-09-21T07:12:57.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.707+0000 7f5d89c00640 1 -- 172.21.15.27:0/3114631961 wait complete. 2024-09-21T07:12:57.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.708+0000 7f5d89c00640 1 Processor -- start 2024-09-21T07:12:57.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.708+0000 7f5d89c00640 1 -- start start 2024-09-21T07:12:57.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d89c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84103910 0x7f5d84075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d89c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84109240 0x7f5d84075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d89c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d84109b90 0x7f5d84076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d89c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84113850 con 0x7f5d84109b90 2024-09-21T07:12:57.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d89c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841136d0 con 0x7f5d84109240 2024-09-21T07:12:57.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d89c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841139d0 con 0x7f5d84103910 2024-09-21T07:12:57.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d83e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84103910 0x7f5d84075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d89200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d84109b90 0x7f5d84076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d83e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84103910 0x7f5d84075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34604/0 (socket says 172.21.15.27:34604) 2024-09-21T07:12:57.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d83e00640 1 -- 172.21.15.27:0/2407578084 learned_addr learned my addr 172.21.15.27:0/2407578084 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:57.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.709+0000 7f5d83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84109240 0x7f5d84075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.710+0000 7f5d83e00640 1 -- 172.21.15.27:0/2407578084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84109240 msgr2=0x7f5d84075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.710+0000 7f5d83e00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84109240 0x7f5d84075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.710+0000 7f5d83e00640 1 -- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d84109b90 msgr2=0x7f5d84076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.710+0000 7f5d83e00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d84109b90 0x7f5d84076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.710+0000 7f5d83e00640 1 -- 172.21.15.27:0/2407578084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d841a8520 con 0x7f5d84103910 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.710+0000 7f5d83e00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84103910 0x7f5d84075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5d740303f0 tx=0x7f5d74032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.710+0000 7f5d80c00640 1 -- 172.21.15.27:0/2407578084 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74040020 con 0x7f5d84103910 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.711+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d841a87b0 con 0x7f5d84103910 2024-09-21T07:12:57.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.711+0000 7f5d80c00640 1 -- 172.21.15.27:0/2407578084 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d74030dd0 con 0x7f5d84103910 2024-09-21T07:12:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.711+0000 7f5d80c00640 1 -- 172.21.15.27:0/2407578084 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74037040 con 0x7f5d84103910 2024-09-21T07:12:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.711+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a8cf0 con 0x7f5d84103910 2024-09-21T07:12:57.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.712+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d84103d20 con 0x7f5d84103910 2024-09-21T07:12:57.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.715+0000 7f5d80c00640 1 -- 172.21.15.27:0/2407578084 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f5d74043020 con 0x7f5d84103910 2024-09-21T07:12:57.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.716+0000 7f5d80c00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d680b6620 0x7f5d680b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:57.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.716+0000 7f5d80c00640 1 -- 172.21.15.27:0/2407578084 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5d74107d10 con 0x7f5d84103910 2024-09-21T07:12:57.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.716+0000 7f5d83400640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d680b6620 0x7f5d680b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:57.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.717+0000 7f5d83400640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d680b6620 0x7f5d680b8b30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d8406ab90 tx=0x7f5d70009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:57.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.718+0000 7f5d80c00640 1 -- 172.21.15.27:0/2407578084 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d7404a020 con 0x7f5d84103910 2024-09-21T07:12:57.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:12:57.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.942+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5d84063800 con 0x7f5d680b6620 2024-09-21T07:12:57.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.950+0000 7f5d80c00640 1 -- 172.21.15.27:0/2407578084 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5d84063800 con 0x7f5d680b6620 2024-09-21T07:12:57.952 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.952+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d680b6620 msgr2=0x7f5d680b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d680b6620 0x7f5d680b8b30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d8406ab90 tx=0x7f5d70009260 comp rx=0 tx=0).stop 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84103910 msgr2=0x7f5d84075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84103910 0x7f5d84075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5d740303f0 tx=0x7f5d74032da0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 shutdown_connections 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d680b6620 0x7f5d680b8b30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d84109b90 0x7f5d84076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d84109240 0x7f5d84075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 --2- 172.21.15.27:0/2407578084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d84103910 0x7f5d84075710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 >> 172.21.15.27:0/2407578084 conn(0x7f5d840ff330 msgr2=0x7f5d84101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 shutdown_connections 2024-09-21T07:12:57.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:57.953+0000 7f5d89c00640 1 -- 172.21.15.27:0/2407578084 wait complete. 2024-09-21T07:12:57.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T07:12:58.038 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:57 smithi027 ceph-mon[30779]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:58.038 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:57 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:12:58.038 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:57 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:12:58.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:57 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:12:58.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:57 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:12:58.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:57 smithi027 ceph-mon[30779]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:58.051 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T07:12:58.061 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T07:12:58.129 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:12:58.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T07:12:58.134 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T07:12:58.147 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T07:12:58.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T07:12:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:57 smithi119 ceph-mon[34519]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:57 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:12:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:57 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:12:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:57 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:12:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:57 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:12:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:57 smithi119 ceph-mon[34519]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:57 smithi153 ceph-mon[34747]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:57 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:12:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:57 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:12:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:57 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:12:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:57 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:12:58.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:57 smithi153 ceph-mon[34747]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:58.803 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:12:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.819+0000 7f14b2800640 1 -- 172.21.15.27:0/2636264333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac10de10 msgr2=0x7f14ac110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.819+0000 7f14b2800640 1 --2- 172.21.15.27:0/2636264333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac10de10 0x7f14ac110300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009be0 tx=0x7f14a402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.820+0000 7f14b2800640 1 -- 172.21.15.27:0/2636264333 shutdown_connections 2024-09-21T07:12:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.820+0000 7f14b2800640 1 --2- 172.21.15.27:0/2636264333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac10de10 0x7f14ac110300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.820+0000 7f14b2800640 1 --2- 172.21.15.27:0/2636264333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14ac105300 0x7f14ac10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.820+0000 7f14b2800640 1 --2- 172.21.15.27:0/2636264333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14ac1049b0 0x7f14ac104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.820+0000 7f14b2800640 1 -- 172.21.15.27:0/2636264333 >> 172.21.15.27:0/2636264333 conn(0x7f14ac079730 msgr2=0x7f14ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.820+0000 7f14b2800640 1 -- 172.21.15.27:0/2636264333 shutdown_connections 2024-09-21T07:12:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.820+0000 7f14b2800640 1 -- 172.21.15.27:0/2636264333 wait complete. 2024-09-21T07:12:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.822+0000 7f14b2800640 1 Processor -- start 2024-09-21T07:12:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.822+0000 7f14b2800640 1 -- start start 2024-09-21T07:12:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14b2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14ac1049b0 0x7f14ac1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14b2800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac105300 0x7f14ac102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14ac10de10 0x7f14ac103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14b2800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac113a90 con 0x7f14ac1049b0 2024-09-21T07:12:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14b2800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac113910 con 0x7f14ac10de10 2024-09-21T07:12:58.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14b2800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac113c10 con 0x7f14ac105300 2024-09-21T07:12:58.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac105300 0x7f14ac102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac105300 0x7f14ac102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53824/0 (socket says 172.21.15.27:53824) 2024-09-21T07:12:58.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14ac1049b0 0x7f14ac1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14aaa00640 1 -- 172.21.15.27:0/320543839 learned_addr learned my addr 172.21.15.27:0/320543839 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:58.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.823+0000 7f14abe00640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14ac10de10 0x7f14ac103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.824+0000 7f14abe00640 1 -- 172.21.15.27:0/320543839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac105300 msgr2=0x7f14ac102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.824+0000 7f14abe00640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac105300 0x7f14ac102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.824+0000 7f14abe00640 1 -- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14ac1049b0 msgr2=0x7f14ac1026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.824+0000 7f14abe00640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14ac1049b0 0x7f14ac1026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.824+0000 7f14abe00640 1 -- 172.21.15.27:0/320543839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac1a86e0 con 0x7f14ac10de10 2024-09-21T07:12:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.824+0000 7f14abe00640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14ac10de10 0x7f14ac103170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14a402abe0 tx=0x7f14a4003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.825+0000 7f14a3e00640 1 -- 172.21.15.27:0/320543839 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4041020 con 0x7f14ac10de10 2024-09-21T07:12:58.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.825+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac1a8970 con 0x7f14ac10de10 2024-09-21T07:12:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.825+0000 7f14a3e00640 1 -- 172.21.15.27:0/320543839 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a4003e10 con 0x7f14ac10de10 2024-09-21T07:12:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.825+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1a8e70 con 0x7f14ac10de10 2024-09-21T07:12:58.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.825+0000 7f14a3e00640 1 -- 172.21.15.27:0/320543839 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4036040 con 0x7f14ac10de10 2024-09-21T07:12:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.830+0000 7f14b942d740 1 Processor -- start 2024-09-21T07:12:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.831+0000 7f14a3e00640 1 -- 172.21.15.27:0/320543839 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f14a4044020 con 0x7f14ac10de10 2024-09-21T07:12:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.831+0000 7f14b942d740 1 -- start start 2024-09-21T07:12:58.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14b942d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2e7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14b942d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf0f1b0 0x561e8d2e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14b942d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 0x561e8d2e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14b942d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e8d255f70 con 0x561e8cf38070 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14b942d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561e8d2453a0 con 0x561e8cf17390 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14b942d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e8d2491b0 con 0x561e8cf0f1b0 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14ab400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf0f1b0 0x561e8d2e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14aaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14ab400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf0f1b0 0x561e8d2e7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53830/0 (socket says 172.21.15.27:53830) 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.832+0000 7f14ab400640 1 -- 172.21.15.27:0/3313318 learned_addr learned my addr 172.21.15.27:0/3313318 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14aaa00640 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 0x561e8d2e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14ab400640 1 -- 172.21.15.27:0/3313318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 msgr2=0x561e8d2e7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14ab400640 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 0x561e8d2e7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14ab400640 1 -- 172.21.15.27:0/3313318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 msgr2=0x561e8d2e7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14ab400640 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2e7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14ab400640 1 -- 172.21.15.27:0/3313318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8d20fc90 con 0x561e8cf0f1b0 2024-09-21T07:12:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14a3e00640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14900b6700 0x7f14900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.833+0000 7f14ab400640 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf0f1b0 0x561e8d2e7ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f149c009c60 tx=0x7f149c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d1ef781c39e84f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.834+0000 7f14a3e00640 1 -- 172.21.15.27:0/320543839 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f14a4107ec0 con 0x7f14ac10de10 2024-09-21T07:12:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.834+0000 7f14aaa00640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14900b6700 0x7f14900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.834+0000 7f14a0c00640 1 -- 172.21.15.27:0/3313318 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c004220 con 0x561e8cf0f1b0 2024-09-21T07:12:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.834+0000 7f14a0c00640 1 -- 172.21.15.27:0/3313318 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f149c0043c0 con 0x561e8cf0f1b0 2024-09-21T07:12:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.834+0000 7f14a0c00640 1 -- 172.21.15.27:0/3313318 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c0197a0 con 0x561e8cf0f1b0 2024-09-21T07:12:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.835+0000 7f14b942d740 1 -- 172.21.15.27:0/3313318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf0f1b0 msgr2=0x561e8d2e7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14aaa00640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14900b6700 0x7f14900b8c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1498009fd0 tx=0x7f1498015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.835+0000 7f14b942d740 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf0f1b0 0x561e8d2e7ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f149c009c60 tx=0x7f149c011810 comp rx=0 tx=0).stop 2024-09-21T07:12:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14b942d740 1 -- 172.21.15.27:0/3313318 shutdown_connections 2024-09-21T07:12:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14b942d740 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf0f1b0 0x561e8d2e7ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14b942d740 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 0x561e8d2e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14b942d740 1 --2- 172.21.15.27:0/3313318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2e7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14b942d740 1 -- 172.21.15.27:0/3313318 >> 172.21.15.27:0/3313318 conn(0x561e8cf2c460 msgr2=0x561e8d2e5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14b942d740 1 -- 172.21.15.27:0/3313318 shutdown_connections 2024-09-21T07:12:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.836+0000 7f14b942d740 1 -- 172.21.15.27:0/3313318 wait complete. 2024-09-21T07:12:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.837+0000 7f14b942d740 1 -- start start 2024-09-21T07:12:58.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.838+0000 7f14b942d740 1 Processor -- start 2024-09-21T07:12:58.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.838+0000 7f14b942d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf0f1b0 0x561e8d2f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.838+0000 7f14b942d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 0x561e8d2f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.838+0000 7f14b942d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf38070 0x561e8d2f3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.839+0000 7f14b942d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e8d0a6700 con 0x561e8cf0f1b0 2024-09-21T07:12:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.839+0000 7f14b942d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561e8d1e0540 con 0x561e8cf17390 2024-09-21T07:12:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.839+0000 7f14b942d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e8d2f5df0 con 0x561e8cf38070 2024-09-21T07:12:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.839+0000 7f14b942d740 10 client.0 _ll_register_callbacks cb 0x561e8d1f0430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.839+0000 7f14ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf0f1b0 0x561e8d2f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.839+0000 7f14ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf0f1b0 0x561e8d2f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56646/0 (socket says 172.21.15.27:56646) 2024-09-21T07:12:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.839+0000 7f14ab400640 1 -- 172.21.15.27:0/2039799460 learned_addr learned my addr 172.21.15.27:0/2039799460 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f14ab400640 1 -- 172.21.15.27:0/2039799460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf38070 msgr2=0x561e8d2f3690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:12:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f14ab400640 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf38070 0x561e8d2f3690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f14ab400640 1 -- 172.21.15.27:0/2039799460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 msgr2=0x561e8d2f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f14ab400640 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 0x561e8d2f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f14ab400640 1 -- 172.21.15.27:0/2039799460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8cfa3650 con 0x561e8cf0f1b0 2024-09-21T07:12:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f14ab400640 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf0f1b0 0x561e8d2f2bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f149c01b040 tx=0x7f149c003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e08ef4b410b3bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f1495600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:12:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.840+0000 7f1495600640 1 -- 172.21.15.27:0/2039799460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c028070 con 0x561e8cf0f1b0 2024-09-21T07:12:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.841+0000 7f1495600640 1 -- 172.21.15.27:0/2039799460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f149c004210 con 0x561e8cf0f1b0 2024-09-21T07:12:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.841+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e8d1e2b50 con 0x561e8cf0f1b0 2024-09-21T07:12:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.841+0000 7f1495600640 1 -- 172.21.15.27:0/2039799460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f149c034050 con 0x561e8cf0f1b0 2024-09-21T07:12:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.841+0000 7f14b942d740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-09-21T07:12:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.841+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e8d1fd910 con 0x561e8cf0f1b0 2024-09-21T07:12:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.841+0000 7f1495600640 1 -- 172.21.15.27:0/2039799460 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561e8d1e2b50 con 0x561e8cf0f1b0 2024-09-21T07:12:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.842+0000 7f1495600640 1 -- 172.21.15.27:0/2039799460 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f149c004800 con 0x561e8cf0f1b0 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.842+0000 7f14b942d740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.842+0000 7f14b942d740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.842+0000 7f14b942d740 20 client.15492 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.842+0000 7f14b942d740 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8cef69f0 0x561e8d304ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.842+0000 7f14b942d740 4 client.15492 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.842+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e8d257ec0 con 0x561e8cef69f0 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.843+0000 7f14aaa00640 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8cef69f0 0x561e8d304ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.844+0000 7f1495600640 10 client.15492 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:58.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.844+0000 7f14aaa00640 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8cef69f0 0x561e8d304ce0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77653dd551e9e2ee server_cookie=500a81068310f814 in_seq=0 out_seq=0 2024-09-21T07:12:58.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f1495600640 1 -- 172.21.15.27:0/2039799460 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e8d257ec0 con 0x561e8cef69f0 2024-09-21T07:12:58.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f1495600640 10 client.15492 handle_command_reply: tid=0 2024-09-21T07:12:58.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 client.15492 shutdown 2024-09-21T07:12:58.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf0f1b0 msgr2=0x561e8d2f2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf0f1b0 0x561e8d2f2bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f149c01b040 tx=0x7f149c003890 comp rx=0 tx=0).stop 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 shutdown_connections 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8cef69f0 0x561e8d304ce0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf38070 0x561e8d2f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf17390 0x561e8d2f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 --2- 172.21.15.27:0/2039799460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf0f1b0 0x561e8d2f2bf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 >> 172.21.15.27:0/2039799460 conn(0x561e8cf2c460 msgr2=0x561e8d2ea750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.845+0000 7f1495600640 0 client.15492 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.846+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 shutdown_connections 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.846+0000 7f14b942d740 1 -- 172.21.15.27:0/2039799460 wait complete. 2024-09-21T07:12:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.846+0000 7f14b942d740 20 client.15492 trim_cache size 0 max 16384 2024-09-21T07:12:58.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 Processor -- start 2024-09-21T07:12:58.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 -- start start 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2f3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cecbda0 0x561e8d2f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1b1380 con 0x561e8cf38070 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1aec50 con 0x561e8cecbda0 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.868+0000 7f14b942d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1acb00 con 0x561e8cf17390 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14ab400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14ab400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53850/0 (socket says 172.21.15.27:53850) 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14ab400640 1 -- 172.21.15.27:0/965645912 learned_addr learned my addr 172.21.15.27:0/965645912 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14abe00640 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cecbda0 0x561e8d2f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14aaa00640 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14abe00640 1 -- 172.21.15.27:0/965645912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 msgr2=0x561e8d2f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:58.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14abe00640 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14abe00640 1 -- 172.21.15.27:0/965645912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 msgr2=0x561e8d2f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:58.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14abe00640 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14abe00640 1 -- 172.21.15.27:0/965645912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8d2e9860 con 0x561e8cecbda0 2024-09-21T07:12:58.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14ab400640 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:12:58.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.869+0000 7f14abe00640 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cecbda0 0x561e8d2f4820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14a4105260 tx=0x7f14a40ca070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bfae9ce7c31c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:58.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.870+0000 7f1497e00640 1 -- 172.21.15.27:0/965645912 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a40d2020 con 0x561e8cecbda0 2024-09-21T07:12:58.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.870+0000 7f1497e00640 1 -- 172.21.15.27:0/965645912 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a40cd470 con 0x561e8cecbda0 2024-09-21T07:12:58.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.870+0000 7f1497e00640 1 -- 172.21.15.27:0/965645912 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a40d7380 con 0x561e8cecbda0 2024-09-21T07:12:58.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 -- 172.21.15.27:0/965645912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cecbda0 msgr2=0x561e8d2f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cecbda0 0x561e8d2f4820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14a4105260 tx=0x7f14a40ca070 comp rx=0 tx=0).stop 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 -- 172.21.15.27:0/965645912 shutdown_connections 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cecbda0 0x561e8d2f4820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cf38070 0x561e8d2f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 --2- 172.21.15.27:0/965645912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 -- 172.21.15.27:0/965645912 >> 172.21.15.27:0/965645912 conn(0x561e8cf2c460 msgr2=0x561e8d2f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 -- 172.21.15.27:0/965645912 shutdown_connections 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.871+0000 7f14b942d740 1 -- 172.21.15.27:0/965645912 wait complete. 2024-09-21T07:12:58.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.872+0000 7f14b942d740 1 -- start start 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.872+0000 7f14b942d740 1 Processor -- start 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.872+0000 7f14b942d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cecbda0 0x561e8d2fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14b942d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14b942d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf38070 0x561e8d2f8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14b942d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1ad150 con 0x561e8cecbda0 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14b942d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1ae580 con 0x561e8cf38070 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14b942d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1ad7e0 con 0x561e8cf17390 2024-09-21T07:12:58.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14b942d740 10 client.0 _ll_register_callbacks cb 0x7f149c009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:12:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cecbda0 0x561e8d2fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53862/0 (socket says 172.21.15.27:53862) 2024-09-21T07:12:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 -- 172.21.15.27:0/2424502330 learned_addr learned my addr 172.21.15.27:0/2424502330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14abe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf38070 0x561e8d2f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cecbda0 0x561e8d2fb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56664/0 (socket says 172.21.15.27:56664) 2024-09-21T07:12:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 -- 172.21.15.27:0/2424502330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf38070 msgr2=0x561e8d2f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf38070 0x561e8d2f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 -- 172.21.15.27:0/2424502330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cecbda0 msgr2=0x561e8d2fb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:12:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cecbda0 0x561e8d2fb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.873+0000 7f14aaa00640 1 -- 172.21.15.27:0/2424502330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8d2f8580 con 0x561e8cf17390 2024-09-21T07:12:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.874+0000 7f14aaa00640 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f7b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f149800f790 tx=0x7f1498009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e59e48f262ed8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.874+0000 7f1495600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.874+0000 7f1495600640 1 -- 172.21.15.27:0/2424502330 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498037060 con 0x561e8cf17390 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.874+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a40cd470 con 0x561e8cf17390 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.874+0000 7f1495600640 1 -- 172.21.15.27:0/2424502330 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1498022a40 con 0x561e8cf17390 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.874+0000 7f1495600640 1 -- 172.21.15.27:0/2424502330 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f149803e050 con 0x561e8cf17390 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 10 client.25522 fetch_fsmap learned FSMap version 27 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e8d3703f0 con 0x561e8cf17390 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f1495600640 1 -- 172.21.15.27:0/2424502330 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a40cd470 con 0x561e8cf17390 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f1495600640 1 -- 172.21.15.27:0/2424502330 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1498031070 con 0x561e8cf17390 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 10 client.25522 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 20 client.25522 populate_metadata read hostname 'smithi027' 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8d370b50 0x561e8d372f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 4 client.25522 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:12:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.875+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e8d253970 con 0x561e8d370b50 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.876+0000 7f14ab400640 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8d370b50 0x561e8d372f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.876+0000 7f14ab400640 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8d370b50 0x561e8d372f70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3538a6995fa16ec3 server_cookie=d127520f145ecc5d in_seq=0 out_seq=0 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.876+0000 7f1495600640 10 client.25522 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f1495600640 1 -- 172.21.15.27:0/2424502330 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x561e8d253970 con 0x561e8d370b50 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f1495600640 10 client.25522 handle_command_reply: tid=0 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 client.25522 shutdown 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 msgr2=0x561e8d2f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f7b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f149800f790 tx=0x7f1498009560 comp rx=0 tx=0).stop 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 shutdown_connections 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561e8d370b50 0x561e8d372f70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e8cf17390 0x561e8d2f7b00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8cf38070 0x561e8d2f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 --2- 172.21.15.27:0/2424502330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e8cecbda0 0x561e8d2fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 >> 172.21.15.27:0/2424502330 conn(0x561e8cf2c460 msgr2=0x561e8d2f1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f1495600640 0 client.25522 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 shutdown_connections 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 1 -- 172.21.15.27:0/2424502330 wait complete. 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.877+0000 7f14b942d740 20 client.25522 trim_cache size 0 max 16384 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:12:58.881 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b", 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:12:58.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:12:58.883 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:12:58.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14900b6700 msgr2=0x7f14900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14900b6700 0x7f14900b8c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1498009fd0 tx=0x7f1498015040 comp rx=0 tx=0).stop 2024-09-21T07:12:58.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14ac10de10 msgr2=0x7f14ac103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:58.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14ac10de10 0x7f14ac103170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14a402abe0 tx=0x7f14a4003c20 comp rx=0 tx=0).stop 2024-09-21T07:12:58.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 shutdown_connections 2024-09-21T07:12:58.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14ac105300 0x7f14ac102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14900b6700 0x7f14900b8c10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14ac10de10 0x7f14ac103170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 --2- 172.21.15.27:0/320543839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14ac1049b0 0x7f14ac1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:58.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.878+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 >> 172.21.15.27:0/320543839 conn(0x7f14ac079730 msgr2=0x7f14ac0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:58.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.879+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 shutdown_connections 2024-09-21T07:12:58.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:58.879+0000 7f14b2800640 1 -- 172.21.15.27:0/320543839 wait complete. 2024-09-21T07:12:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: pgmap v376: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='client.25507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:12:59.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:12:58 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: pgmap v376: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='client.25507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:12:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:12:58 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: pgmap v376: 129 pgs: 129 active+clean; 655 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T07:12:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='client.25507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:12:59.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:12:58 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.793+0000 7f3814c00640 1 -- 172.21.15.27:0/907401604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3810105d80 msgr2=0x7f3810106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.793+0000 7f3814c00640 1 --2- 172.21.15.27:0/907401604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3810105d80 0x7f3810106190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3800009900 tx=0x7f380002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.793+0000 7f3814c00640 1 -- 172.21.15.27:0/907401604 shutdown_connections 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.793+0000 7f3814c00640 1 --2- 172.21.15.27:0/907401604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f381010b6c0 0x7f381010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.793+0000 7f3814c00640 1 --2- 172.21.15.27:0/907401604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f381010c010 0x7f3810112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.793+0000 7f3814c00640 1 --2- 172.21.15.27:0/907401604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3810105d80 0x7f3810106190 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.793+0000 7f3814c00640 1 -- 172.21.15.27:0/907401604 >> 172.21.15.27:0/907401604 conn(0x7f3810101810 msgr2=0x7f3810103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:12:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.794+0000 7f3814c00640 1 -- 172.21.15.27:0/907401604 shutdown_connections 2024-09-21T07:12:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.794+0000 7f3814c00640 1 -- 172.21.15.27:0/907401604 wait complete. 2024-09-21T07:12:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 Processor -- start 2024-09-21T07:12:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 -- start start 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3810105d80 0x7f381019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f381010b6c0 0x7f381019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f381010c010 0x7f38101a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3810115d20 con 0x7f381010b6c0 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3810115ba0 con 0x7f3810105d80 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f3814c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3810115ea0 con 0x7f381010c010 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f380d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f381010b6c0 0x7f381019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f380d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f381010b6c0 0x7f381019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56680/0 (socket says 172.21.15.27:56680) 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.795+0000 7f380d400640 1 -- 172.21.15.27:0/1133529416 learned_addr learned my addr 172.21.15.27:0/1133529416 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380de00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3810105d80 0x7f381019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380e800640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f381010c010 0x7f38101a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380d400640 1 -- 172.21.15.27:0/1133529416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f381010c010 msgr2=0x7f38101a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380d400640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f381010c010 0x7f38101a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380d400640 1 -- 172.21.15.27:0/1133529416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3810105d80 msgr2=0x7f381019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380d400640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3810105d80 0x7f381019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380d400640 1 -- 172.21.15.27:0/1133529416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38101a2660 con 0x7f381010b6c0 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f380d400640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f381010b6c0 0x7f381019cc90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00e9c0 tx=0x7f37fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.796+0000 7f3806a00640 1 -- 172.21.15.27:0/1133529416 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00ccc0 con 0x7f381010b6c0 2024-09-21T07:12:59.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.797+0000 7f3806a00640 1 -- 172.21.15.27:0/1133529416 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37fc0054d0 con 0x7f381010b6c0 2024-09-21T07:12:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.797+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38101a2950 con 0x7f381010b6c0 2024-09-21T07:12:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.797+0000 7f3806a00640 1 -- 172.21.15.27:0/1133529416 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc010430 con 0x7f381010b6c0 2024-09-21T07:12:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.797+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38101a90b0 con 0x7f381010b6c0 2024-09-21T07:12:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.798+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3810106190 con 0x7f381010b6c0 2024-09-21T07:12:59.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.801+0000 7f3806a00640 1 -- 172.21.15.27:0/1133529416 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f37fc005020 con 0x7f381010b6c0 2024-09-21T07:12:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.802+0000 7f3806a00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37f40b6700 0x7f37f40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:12:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.803+0000 7f3806a00640 1 -- 172.21.15.27:0/1133529416 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f37fc0e7d40 con 0x7f381010b6c0 2024-09-21T07:12:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.803+0000 7f380de00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37f40b6700 0x7f37f40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:12:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.803+0000 7f380de00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37f40b6700 0x7f37f40b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3800005ab0 tx=0x7f3800005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:12:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:12:59.807+0000 7f3806a00640 1 -- 172.21.15.27:0/1133529416 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37fc0adf40 con 0x7f381010b6c0 2024-09-21T07:13:00.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.007+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3810063800 con 0x7f37f40b6700 2024-09-21T07:13:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.015+0000 7f3806a00640 1 -- 172.21.15.27:0/1133529416 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340520 (secure 0 0 0) 0x7f3810063800 con 0x7f37f40b6700 2024-09-21T07:13:00.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37f40b6700 msgr2=0x7f37f40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37f40b6700 0x7f37f40b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3800005ab0 tx=0x7f3800005a20 comp rx=0 tx=0).stop 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f381010b6c0 msgr2=0x7f381019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f381010b6c0 0x7f381019cc90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00e9c0 tx=0x7f37fc00ee80 comp rx=0 tx=0).stop 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 shutdown_connections 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f381010c010 0x7f38101a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37f40b6700 0x7f37f40b8c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f381010b6c0 0x7f381019cc90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 --2- 172.21.15.27:0/1133529416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3810105d80 0x7f381019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 >> 172.21.15.27:0/1133529416 conn(0x7f3810101810 msgr2=0x7f3810103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 shutdown_connections 2024-09-21T07:13:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:00.021+0000 7f3814c00640 1 -- 172.21.15.27:0/1133529416 wait complete. 2024-09-21T07:13:00.032 INFO:tasks.cephfs.filesystem:scrub status for tag:b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b - {'path': '/', 'tag': 'b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b', 'options': 'recursive,force'} 2024-09-21T07:13:00.776 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-21T07:12:59.417681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1064179,"num_objects":65,"num_object_clones":0,"num_object_copies":197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":1179,"num_read_kb":823,"num_write":468,"num_write_kb":3513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397976,"kb_used_data":15388,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543848,"statfs":{"total":1151940427776,"available":1151532900352,"internally_reserved":0,"allocated":15757312,"data_stored":9752859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":396518,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":384,"num_read_kb":322,"num_write":163,"num_write_kb":1224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003790"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:58.089462+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:11:58.089462+0000","last_peered":"2024-09-21T07:11:58.089462+0000","last_clean":"2024-09-21T07:11:58.089462+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:11:58.089462+0000","last_undegraded":"2024-09-21T07:11:58.089462+0000","last_fullsized":"2024-09-21T07:11:58.089462+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805255+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:12:23.805255+0000","last_peered":"2024-09-21T07:12:23.805255+0000","last_clean":"2024-09-21T07:12:23.805255+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:12:23.805255+0000","last_undegraded":"2024-09-21T07:12:23.805255+0000","last_fullsized":"2024-09-21T07:12:23.805255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832485+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:11:55.832485+0000","last_peered":"2024-09-21T07:11:55.832485+0000","last_clean":"2024-09-21T07:11:55.832485+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:11:55.832485+0000","last_undegraded":"2024-09-21T07:11:55.832485+0000","last_fullsized":"2024-09-21T07:11:55.832485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833749+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:11:55.833749+0000","last_peered":"2024-09-21T07:11:55.833749+0000","last_clean":"2024-09-21T07:11:55.833749+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:11:55.833749+0000","last_undegraded":"2024-09-21T07:11:55.833749+0000","last_fullsized":"2024-09-21T07:11:55.833749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811697+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:12:23.811697+0000","last_peered":"2024-09-21T07:12:23.811697+0000","last_clean":"2024-09-21T07:12:23.811697+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:12:23.811697+0000","last_undegraded":"2024-09-21T07:12:23.811697+0000","last_fullsized":"2024-09-21T07:12:23.811697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844771+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:11:55.844771+0000","last_peered":"2024-09-21T07:11:55.844771+0000","last_clean":"2024-09-21T07:11:55.844771+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:11:55.844771+0000","last_undegraded":"2024-09-21T07:11:55.844771+0000","last_fullsized":"2024-09-21T07:11:55.844771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847516+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:11:55.847516+0000","last_peered":"2024-09-21T07:11:55.847516+0000","last_clean":"2024-09-21T07:11:55.847516+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:11:55.847516+0000","last_undegraded":"2024-09-21T07:11:55.847516+0000","last_fullsized":"2024-09-21T07:11:55.847516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814377+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:12:23.814377+0000","last_peered":"2024-09-21T07:12:23.814377+0000","last_clean":"2024-09-21T07:12:23.814377+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:12:23.814377+0000","last_undegraded":"2024-09-21T07:12:23.814377+0000","last_fullsized":"2024-09-21T07:12:23.814377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845066+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:11:55.845066+0000","last_peered":"2024-09-21T07:11:55.845066+0000","last_clean":"2024-09-21T07:11:55.845066+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:11:55.845066+0000","last_undegraded":"2024-09-21T07:11:55.845066+0000","last_fullsized":"2024-09-21T07:11:55.845066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847357+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:11:55.847357+0000","last_peered":"2024-09-21T07:11:55.847357+0000","last_clean":"2024-09-21T07:11:55.847357+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:11:55.847357+0000","last_undegraded":"2024-09-21T07:11:55.847357+0000","last_fullsized":"2024-09-21T07:11:55.847357+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803507+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:12:23.803507+0000","last_peered":"2024-09-21T07:12:23.803507+0000","last_clean":"2024-09-21T07:12:23.803507+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:12:23.803507+0000","last_undegraded":"2024-09-21T07:12:23.803507+0000","last_fullsized":"2024-09-21T07:12:23.803507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844870+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:11:55.844870+0000","last_peered":"2024-09-21T07:11:55.844870+0000","last_clean":"2024-09-21T07:11:55.844870+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:11:55.844870+0000","last_undegraded":"2024-09-21T07:11:55.844870+0000","last_fullsized":"2024-09-21T07:11:55.844870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813396+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:12:23.813396+0000","last_peered":"2024-09-21T07:12:23.813396+0000","last_clean":"2024-09-21T07:12:23.813396+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:12:23.813396+0000","last_undegraded":"2024-09-21T07:12:23.813396+0000","last_fullsized":"2024-09-21T07:12:23.813396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846961+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:11:55.846961+0000","last_peered":"2024-09-21T07:11:55.846961+0000","last_clean":"2024-09-21T07:11:55.846961+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:11:55.846961+0000","last_undegraded":"2024-09-21T07:11:55.846961+0000","last_fullsized":"2024-09-21T07:11:55.846961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845137+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:11:55.845137+0000","last_peered":"2024-09-21T07:11:55.845137+0000","last_clean":"2024-09-21T07:11:55.845137+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:11:55.845137+0000","last_undegraded":"2024-09-21T07:11:55.845137+0000","last_fullsized":"2024-09-21T07:11:55.845137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807789+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:12:23.807789+0000","last_peered":"2024-09-21T07:12:23.807789+0000","last_clean":"2024-09-21T07:12:23.807789+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:12:23.807789+0000","last_undegraded":"2024-09-21T07:12:23.807789+0000","last_fullsized":"2024-09-21T07:12:23.807789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832340+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:11:55.832340+0000","last_peered":"2024-09-21T07:11:55.832340+0000","last_clean":"2024-09-21T07:11:55.832340+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:11:55.832340+0000","last_undegraded":"2024-09-21T07:11:55.832340+0000","last_fullsized":"2024-09-21T07:11:55.832340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845070+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:11:55.845070+0000","last_peered":"2024-09-21T07:11:55.845070+0000","last_clean":"2024-09-21T07:11:55.845070+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:11:55.845070+0000","last_undegraded":"2024-09-21T07:11:55.845070+0000","last_fullsized":"2024-09-21T07:11:55.845070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798453+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:12:23.798453+0000","last_peered":"2024-09-21T07:12:23.798453+0000","last_clean":"2024-09-21T07:12:23.798453+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:12:23.798453+0000","last_undegraded":"2024-09-21T07:12:23.798453+0000","last_fullsized":"2024-09-21T07:12:23.798453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833325+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:11:55.833325+0000","last_peered":"2024-09-21T07:11:55.833325+0000","last_clean":"2024-09-21T07:11:55.833325+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:11:55.833325+0000","last_undegraded":"2024-09-21T07:11:55.833325+0000","last_fullsized":"2024-09-21T07:11:55.833325+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830920+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:11:55.830920+0000","last_peered":"2024-09-21T07:11:55.830920+0000","last_clean":"2024-09-21T07:11:55.830920+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:11:55.830920+0000","last_undegraded":"2024-09-21T07:11:55.830920+0000","last_fullsized":"2024-09-21T07:11:55.830920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.780871+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:12:23.780871+0000","last_peered":"2024-09-21T07:12:23.780871+0000","last_clean":"2024-09-21T07:12:23.780871+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:12:23.780871+0000","last_undegraded":"2024-09-21T07:12:23.780871+0000","last_fullsized":"2024-09-21T07:12:23.780871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833051+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:11:55.833051+0000","last_peered":"2024-09-21T07:11:55.833051+0000","last_clean":"2024-09-21T07:11:55.833051+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:11:55.833051+0000","last_undegraded":"2024-09-21T07:11:55.833051+0000","last_fullsized":"2024-09-21T07:11:55.833051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845144+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:11:55.845144+0000","last_peered":"2024-09-21T07:11:55.845144+0000","last_clean":"2024-09-21T07:11:55.845144+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:11:55.845144+0000","last_undegraded":"2024-09-21T07:11:55.845144+0000","last_fullsized":"2024-09-21T07:11:55.845144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.800049+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:12:23.800049+0000","last_peered":"2024-09-21T07:12:23.800049+0000","last_clean":"2024-09-21T07:12:23.800049+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:12:23.800049+0000","last_undegraded":"2024-09-21T07:12:23.800049+0000","last_fullsized":"2024-09-21T07:12:23.800049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845369+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:11:55.845369+0000","last_peered":"2024-09-21T07:11:55.845369+0000","last_clean":"2024-09-21T07:11:55.845369+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:11:55.845369+0000","last_undegraded":"2024-09-21T07:11:55.845369+0000","last_fullsized":"2024-09-21T07:11:55.845369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808689+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:12:23.808689+0000","last_peered":"2024-09-21T07:12:23.808689+0000","last_clean":"2024-09-21T07:12:23.808689+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:12:23.808689+0000","last_undegraded":"2024-09-21T07:12:23.808689+0000","last_fullsized":"2024-09-21T07:12:23.808689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'97","reported_seq":232,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:57.948829+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:12:57.948829+0000","last_peered":"2024-09-21T07:12:57.948829+0000","last_clean":"2024-09-21T07:12:57.948829+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:12:57.948829+0000","last_undegraded":"2024-09-21T07:12:57.948829+0000","last_fullsized":"2024-09-21T07:12:57.948829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832993+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:11:55.832993+0000","last_peered":"2024-09-21T07:11:55.832993+0000","last_clean":"2024-09-21T07:11:55.832993+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:11:55.832993+0000","last_undegraded":"2024-09-21T07:11:55.832993+0000","last_fullsized":"2024-09-21T07:11:55.832993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809315+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:12:23.809315+0000","last_peered":"2024-09-21T07:12:23.809315+0000","last_clean":"2024-09-21T07:12:23.809315+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:12:23.809315+0000","last_undegraded":"2024-09-21T07:12:23.809315+0000","last_fullsized":"2024-09-21T07:12:23.809315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847542+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:11:55.847542+0000","last_peered":"2024-09-21T07:11:55.847542+0000","last_clean":"2024-09-21T07:11:55.847542+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:11:55.847542+0000","last_undegraded":"2024-09-21T07:11:55.847542+0000","last_fullsized":"2024-09-21T07:11:55.847542+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790825+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:12:23.790825+0000","last_peered":"2024-09-21T07:12:23.790825+0000","last_clean":"2024-09-21T07:12:23.790825+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:12:23.790825+0000","last_undegraded":"2024-09-21T07:12:23.790825+0000","last_fullsized":"2024-09-21T07:12:23.790825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830831+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:11:55.830831+0000","last_peered":"2024-09-21T07:11:55.830831+0000","last_clean":"2024-09-21T07:11:55.830831+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:11:55.830831+0000","last_undegraded":"2024-09-21T07:11:55.830831+0000","last_fullsized":"2024-09-21T07:11:55.830831+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846866+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:11:55.846866+0000","last_peered":"2024-09-21T07:11:55.846866+0000","last_clean":"2024-09-21T07:11:55.846866+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:11:55.846866+0000","last_undegraded":"2024-09-21T07:11:55.846866+0000","last_fullsized":"2024-09-21T07:11:55.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.794745+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:12:23.794745+0000","last_peered":"2024-09-21T07:12:23.794745+0000","last_clean":"2024-09-21T07:12:23.794745+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:12:23.794745+0000","last_undegraded":"2024-09-21T07:12:23.794745+0000","last_fullsized":"2024-09-21T07:12:23.794745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981709+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:11:55.981709+0000","last_peered":"2024-09-21T07:11:55.981709+0000","last_clean":"2024-09-21T07:11:55.981709+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:11:55.981709+0000","last_undegraded":"2024-09-21T07:11:55.981709+0000","last_fullsized":"2024-09-21T07:11:55.981709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845966+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:11:55.845966+0000","last_peered":"2024-09-21T07:11:55.845966+0000","last_clean":"2024-09-21T07:11:55.845966+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:11:55.845966+0000","last_undegraded":"2024-09-21T07:11:55.845966+0000","last_fullsized":"2024-09-21T07:11:55.845966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795850+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:12:23.795850+0000","last_peered":"2024-09-21T07:12:23.795850+0000","last_clean":"2024-09-21T07:12:23.795850+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:12:23.795850+0000","last_undegraded":"2024-09-21T07:12:23.795850+0000","last_fullsized":"2024-09-21T07:12:23.795850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":177,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:54.764107+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:12:54.764107+0000","last_peered":"2024-09-21T07:12:54.764107+0000","last_clean":"2024-09-21T07:12:54.764107+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:12:54.764107+0000","last_undegraded":"2024-09-21T07:12:54.764107+0000","last_fullsized":"2024-09-21T07:12:54.764107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846521+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:11:55.846521+0000","last_peered":"2024-09-21T07:11:55.846521+0000","last_clean":"2024-09-21T07:11:55.846521+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:11:55.846521+0000","last_undegraded":"2024-09-21T07:11:55.846521+0000","last_fullsized":"2024-09-21T07:11:55.846521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807529+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:12:23.807529+0000","last_peered":"2024-09-21T07:12:23.807529+0000","last_clean":"2024-09-21T07:12:23.807529+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:12:23.807529+0000","last_undegraded":"2024-09-21T07:12:23.807529+0000","last_fullsized":"2024-09-21T07:12:23.807529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981500+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:11:55.981500+0000","last_peered":"2024-09-21T07:11:55.981500+0000","last_clean":"2024-09-21T07:11:55.981500+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:11:55.981500+0000","last_undegraded":"2024-09-21T07:11:55.981500+0000","last_fullsized":"2024-09-21T07:11:55.981500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845592+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:11:55.845592+0000","last_peered":"2024-09-21T07:11:55.845592+0000","last_clean":"2024-09-21T07:11:55.845592+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:11:55.845592+0000","last_undegraded":"2024-09-21T07:11:55.845592+0000","last_fullsized":"2024-09-21T07:11:55.845592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811077+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:12:23.811077+0000","last_peered":"2024-09-21T07:12:23.811077+0000","last_clean":"2024-09-21T07:12:23.811077+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:12:23.811077+0000","last_undegraded":"2024-09-21T07:12:23.811077+0000","last_fullsized":"2024-09-21T07:12:23.811077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845392+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:11:55.845392+0000","last_peered":"2024-09-21T07:11:55.845392+0000","last_clean":"2024-09-21T07:11:55.845392+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:11:55.845392+0000","last_undegraded":"2024-09-21T07:11:55.845392+0000","last_fullsized":"2024-09-21T07:11:55.845392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981606+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:11:55.981606+0000","last_peered":"2024-09-21T07:11:55.981606+0000","last_clean":"2024-09-21T07:11:55.981606+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:11:55.981606+0000","last_undegraded":"2024-09-21T07:11:55.981606+0000","last_fullsized":"2024-09-21T07:11:55.981606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981807+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:11:55.981807+0000","last_peered":"2024-09-21T07:11:55.981807+0000","last_clean":"2024-09-21T07:11:55.981807+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:11:55.981807+0000","last_undegraded":"2024-09-21T07:11:55.981807+0000","last_fullsized":"2024-09-21T07:11:55.981807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814663+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:12:23.814663+0000","last_peered":"2024-09-21T07:12:23.814663+0000","last_clean":"2024-09-21T07:12:23.814663+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:12:23.814663+0000","last_undegraded":"2024-09-21T07:12:23.814663+0000","last_fullsized":"2024-09-21T07:12:23.814663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.834191+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:11:55.834191+0000","last_peered":"2024-09-21T07:11:55.834191+0000","last_clean":"2024-09-21T07:11:55.834191+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:11:55.834191+0000","last_undegraded":"2024-09-21T07:11:55.834191+0000","last_fullsized":"2024-09-21T07:11:55.834191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833166+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:11:55.833166+0000","last_peered":"2024-09-21T07:11:55.833166+0000","last_clean":"2024-09-21T07:11:55.833166+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:11:55.833166+0000","last_undegraded":"2024-09-21T07:11:55.833166+0000","last_fullsized":"2024-09-21T07:11:55.833166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807409+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:12:23.807409+0000","last_peered":"2024-09-21T07:12:23.807409+0000","last_clean":"2024-09-21T07:12:23.807409+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:12:23.807409+0000","last_undegraded":"2024-09-21T07:12:23.807409+0000","last_fullsized":"2024-09-21T07:12:23.807409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846421+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:11:55.846421+0000","last_peered":"2024-09-21T07:11:55.846421+0000","last_clean":"2024-09-21T07:11:55.846421+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:11:55.846421+0000","last_undegraded":"2024-09-21T07:11:55.846421+0000","last_fullsized":"2024-09-21T07:11:55.846421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846714+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:11:55.846714+0000","last_peered":"2024-09-21T07:11:55.846714+0000","last_clean":"2024-09-21T07:11:55.846714+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:11:55.846714+0000","last_undegraded":"2024-09-21T07:11:55.846714+0000","last_fullsized":"2024-09-21T07:11:55.846714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814102+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:12:23.814102+0000","last_peered":"2024-09-21T07:12:23.814102+0000","last_clean":"2024-09-21T07:12:23.814102+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:12:23.814102+0000","last_undegraded":"2024-09-21T07:12:23.814102+0000","last_fullsized":"2024-09-21T07:12:23.814102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846385+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:11:55.846385+0000","last_peered":"2024-09-21T07:11:55.846385+0000","last_clean":"2024-09-21T07:11:55.846385+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:11:55.846385+0000","last_undegraded":"2024-09-21T07:11:55.846385+0000","last_fullsized":"2024-09-21T07:11:55.846385+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845182+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:11:55.845182+0000","last_peered":"2024-09-21T07:11:55.845182+0000","last_clean":"2024-09-21T07:11:55.845182+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:11:55.845182+0000","last_undegraded":"2024-09-21T07:11:55.845182+0000","last_fullsized":"2024-09-21T07:11:55.845182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801647+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:12:23.801647+0000","last_peered":"2024-09-21T07:12:23.801647+0000","last_clean":"2024-09-21T07:12:23.801647+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:12:23.801647+0000","last_undegraded":"2024-09-21T07:12:23.801647+0000","last_fullsized":"2024-09-21T07:12:23.801647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832950+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:11:55.832950+0000","last_peered":"2024-09-21T07:11:55.832950+0000","last_clean":"2024-09-21T07:11:55.832950+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:11:55.832950+0000","last_undegraded":"2024-09-21T07:11:55.832950+0000","last_fullsized":"2024-09-21T07:11:55.832950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845860+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:11:55.845860+0000","last_peered":"2024-09-21T07:11:55.845860+0000","last_clean":"2024-09-21T07:11:55.845860+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:11:55.845860+0000","last_undegraded":"2024-09-21T07:11:55.845860+0000","last_fullsized":"2024-09-21T07:11:55.845860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802831+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:12:23.802831+0000","last_peered":"2024-09-21T07:12:23.802831+0000","last_clean":"2024-09-21T07:12:23.802831+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:12:23.802831+0000","last_undegraded":"2024-09-21T07:12:23.802831+0000","last_fullsized":"2024-09-21T07:12:23.802831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846785+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:11:55.846785+0000","last_peered":"2024-09-21T07:11:55.846785+0000","last_clean":"2024-09-21T07:11:55.846785+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:11:55.846785+0000","last_undegraded":"2024-09-21T07:11:55.846785+0000","last_fullsized":"2024-09-21T07:11:55.846785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846243+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:11:55.846243+0000","last_peered":"2024-09-21T07:11:55.846243+0000","last_clean":"2024-09-21T07:11:55.846243+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:11:55.846243+0000","last_undegraded":"2024-09-21T07:11:55.846243+0000","last_fullsized":"2024-09-21T07:11:55.846243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805668+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:12:23.805668+0000","last_peered":"2024-09-21T07:12:23.805668+0000","last_clean":"2024-09-21T07:12:23.805668+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:12:23.805668+0000","last_undegraded":"2024-09-21T07:12:23.805668+0000","last_fullsized":"2024-09-21T07:12:23.805668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":181,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:53.763181+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:12:53.763181+0000","last_peered":"2024-09-21T07:12:53.763181+0000","last_clean":"2024-09-21T07:12:53.763181+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:12:53.763181+0000","last_undegraded":"2024-09-21T07:12:53.763181+0000","last_fullsized":"2024-09-21T07:12:53.763181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830985+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:11:55.830985+0000","last_peered":"2024-09-21T07:11:55.830985+0000","last_clean":"2024-09-21T07:11:55.830985+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:11:55.830985+0000","last_undegraded":"2024-09-21T07:11:55.830985+0000","last_fullsized":"2024-09-21T07:11:55.830985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813592+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:12:23.813592+0000","last_peered":"2024-09-21T07:12:23.813592+0000","last_clean":"2024-09-21T07:12:23.813592+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:12:23.813592+0000","last_undegraded":"2024-09-21T07:12:23.813592+0000","last_fullsized":"2024-09-21T07:12:23.813592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846256+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:11:55.846256+0000","last_peered":"2024-09-21T07:11:55.846256+0000","last_clean":"2024-09-21T07:11:55.846256+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:11:55.846256+0000","last_undegraded":"2024-09-21T07:11:55.846256+0000","last_fullsized":"2024-09-21T07:11:55.846256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832414+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:11:55.832414+0000","last_peered":"2024-09-21T07:11:55.832414+0000","last_clean":"2024-09-21T07:11:55.832414+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:11:55.832414+0000","last_undegraded":"2024-09-21T07:11:55.832414+0000","last_fullsized":"2024-09-21T07:11:55.832414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808494+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:12:23.808494+0000","last_peered":"2024-09-21T07:12:23.808494+0000","last_clean":"2024-09-21T07:12:23.808494+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:12:23.808494+0000","last_undegraded":"2024-09-21T07:12:23.808494+0000","last_fullsized":"2024-09-21T07:12:23.808494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833515+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:11:55.833515+0000","last_peered":"2024-09-21T07:11:55.833515+0000","last_clean":"2024-09-21T07:11:55.833515+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:11:55.833515+0000","last_undegraded":"2024-09-21T07:11:55.833515+0000","last_fullsized":"2024-09-21T07:11:55.833515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846046+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:11:55.846046+0000","last_peered":"2024-09-21T07:11:55.846046+0000","last_clean":"2024-09-21T07:11:55.846046+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:11:55.846046+0000","last_undegraded":"2024-09-21T07:11:55.846046+0000","last_fullsized":"2024-09-21T07:11:55.846046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798135+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:12:23.798135+0000","last_peered":"2024-09-21T07:12:23.798135+0000","last_clean":"2024-09-21T07:12:23.798135+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:12:23.798135+0000","last_undegraded":"2024-09-21T07:12:23.798135+0000","last_fullsized":"2024-09-21T07:12:23.798135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":355,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:57.769530+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:12:57.769530+0000","last_peered":"2024-09-21T07:12:57.769530+0000","last_clean":"2024-09-21T07:12:57.769530+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:12:57.769530+0000","last_undegraded":"2024-09-21T07:12:57.769530+0000","last_fullsized":"2024-09-21T07:12:57.769530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":269,"num_read_kb":209,"num_write":42,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.848049+0000","last_change":"2024-09-21T07:11:55.847916+0000","last_active":"2024-09-21T07:11:55.848049+0000","last_peered":"2024-09-21T07:11:55.848049+0000","last_clean":"2024-09-21T07:11:55.848049+0000","last_became_active":"2024-09-21T07:11:55.847491+0000","last_became_peered":"2024-09-21T07:11:55.847491+0000","last_unstale":"2024-09-21T07:11:55.848049+0000","last_undegraded":"2024-09-21T07:11:55.848049+0000","last_fullsized":"2024-09-21T07:11:55.848049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:25.702259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,1],"acting":[10,8,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813257+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:12:23.813257+0000","last_peered":"2024-09-21T07:12:23.813257+0000","last_clean":"2024-09-21T07:12:23.813257+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:12:23.813257+0000","last_undegraded":"2024-09-21T07:12:23.813257+0000","last_fullsized":"2024-09-21T07:12:23.813257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846367+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:11:55.846367+0000","last_peered":"2024-09-21T07:11:55.846367+0000","last_clean":"2024-09-21T07:11:55.846367+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:11:55.846367+0000","last_undegraded":"2024-09-21T07:11:55.846367+0000","last_fullsized":"2024-09-21T07:11:55.846367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845245+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:11:55.845245+0000","last_peered":"2024-09-21T07:11:55.845245+0000","last_clean":"2024-09-21T07:11:55.845245+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:11:55.845245+0000","last_undegraded":"2024-09-21T07:11:55.845245+0000","last_fullsized":"2024-09-21T07:11:55.845245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.793771+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:12:23.793771+0000","last_peered":"2024-09-21T07:12:23.793771+0000","last_clean":"2024-09-21T07:12:23.793771+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:12:23.793771+0000","last_undegraded":"2024-09-21T07:12:23.793771+0000","last_fullsized":"2024-09-21T07:12:23.793771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981878+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:11:55.981878+0000","last_peered":"2024-09-21T07:11:55.981878+0000","last_clean":"2024-09-21T07:11:55.981878+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:11:55.981878+0000","last_undegraded":"2024-09-21T07:11:55.981878+0000","last_fullsized":"2024-09-21T07:11:55.981878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830120+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:11:55.830120+0000","last_peered":"2024-09-21T07:11:55.830120+0000","last_clean":"2024-09-21T07:11:55.830120+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:11:55.830120+0000","last_undegraded":"2024-09-21T07:11:55.830120+0000","last_fullsized":"2024-09-21T07:11:55.830120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813258+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:12:23.813258+0000","last_peered":"2024-09-21T07:12:23.813258+0000","last_clean":"2024-09-21T07:12:23.813258+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:12:23.813258+0000","last_undegraded":"2024-09-21T07:12:23.813258+0000","last_fullsized":"2024-09-21T07:12:23.813258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":177,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:56.768270+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:12:56.768270+0000","last_peered":"2024-09-21T07:12:56.768270+0000","last_clean":"2024-09-21T07:12:56.768270+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:12:56.768270+0000","last_undegraded":"2024-09-21T07:12:56.768270+0000","last_fullsized":"2024-09-21T07:12:56.768270+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833187+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:11:55.833187+0000","last_peered":"2024-09-21T07:11:55.833187+0000","last_clean":"2024-09-21T07:11:55.833187+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:11:55.833187+0000","last_undegraded":"2024-09-21T07:11:55.833187+0000","last_fullsized":"2024-09-21T07:11:55.833187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847006+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:11:55.847006+0000","last_peered":"2024-09-21T07:11:55.847006+0000","last_clean":"2024-09-21T07:11:55.847006+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:11:55.847006+0000","last_undegraded":"2024-09-21T07:11:55.847006+0000","last_fullsized":"2024-09-21T07:11:55.847006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802309+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:12:23.802309+0000","last_peered":"2024-09-21T07:12:23.802309+0000","last_clean":"2024-09-21T07:12:23.802309+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:12:23.802309+0000","last_undegraded":"2024-09-21T07:12:23.802309+0000","last_fullsized":"2024-09-21T07:12:23.802309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845739+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:11:55.845739+0000","last_peered":"2024-09-21T07:11:55.845739+0000","last_clean":"2024-09-21T07:11:55.845739+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:11:55.845739+0000","last_undegraded":"2024-09-21T07:11:55.845739+0000","last_fullsized":"2024-09-21T07:11:55.845739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849176+0000","last_change":"2024-09-21T07:11:55.848588+0000","last_active":"2024-09-21T07:11:55.849176+0000","last_peered":"2024-09-21T07:11:55.849176+0000","last_clean":"2024-09-21T07:11:55.849176+0000","last_became_active":"2024-09-21T07:11:55.848025+0000","last_became_peered":"2024-09-21T07:11:55.848025+0000","last_unstale":"2024-09-21T07:11:55.849176+0000","last_undegraded":"2024-09-21T07:11:55.849176+0000","last_fullsized":"2024-09-21T07:11:55.849176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:37.413873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810233+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:12:23.810233+0000","last_peered":"2024-09-21T07:12:23.810233+0000","last_clean":"2024-09-21T07:12:23.810233+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:12:23.810233+0000","last_undegraded":"2024-09-21T07:12:23.810233+0000","last_fullsized":"2024-09-21T07:12:23.810233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845457+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:11:55.845457+0000","last_peered":"2024-09-21T07:11:55.845457+0000","last_clean":"2024-09-21T07:11:55.845457+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:11:55.845457+0000","last_undegraded":"2024-09-21T07:11:55.845457+0000","last_fullsized":"2024-09-21T07:11:55.845457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.835562+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:11:55.835562+0000","last_peered":"2024-09-21T07:11:55.835562+0000","last_clean":"2024-09-21T07:11:55.835562+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:11:55.835562+0000","last_undegraded":"2024-09-21T07:11:55.835562+0000","last_fullsized":"2024-09-21T07:11:55.835562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807495+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:12:23.807495+0000","last_peered":"2024-09-21T07:12:23.807495+0000","last_clean":"2024-09-21T07:12:23.807495+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:12:23.807495+0000","last_undegraded":"2024-09-21T07:12:23.807495+0000","last_fullsized":"2024-09-21T07:12:23.807495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849703+0000","last_change":"2024-09-21T07:11:55.848551+0000","last_active":"2024-09-21T07:11:55.849703+0000","last_peered":"2024-09-21T07:11:55.849703+0000","last_clean":"2024-09-21T07:11:55.849703+0000","last_became_active":"2024-09-21T07:11:55.848056+0000","last_became_peered":"2024-09-21T07:11:55.848056+0000","last_unstale":"2024-09-21T07:11:55.849703+0000","last_undegraded":"2024-09-21T07:11:55.849703+0000","last_fullsized":"2024-09-21T07:11:55.849703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:07.779820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":181,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:53.763902+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:12:53.763902+0000","last_peered":"2024-09-21T07:12:53.763902+0000","last_clean":"2024-09-21T07:12:53.763902+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:12:53.763902+0000","last_undegraded":"2024-09-21T07:12:53.763902+0000","last_fullsized":"2024-09-21T07:12:53.763902+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788293+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:12:23.788293+0000","last_peered":"2024-09-21T07:12:23.788293+0000","last_clean":"2024-09-21T07:12:23.788293+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:12:23.788293+0000","last_undegraded":"2024-09-21T07:12:23.788293+0000","last_fullsized":"2024-09-21T07:12:23.788293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833541+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:11:55.833541+0000","last_peered":"2024-09-21T07:11:55.833541+0000","last_clean":"2024-09-21T07:11:55.833541+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:11:55.833541+0000","last_undegraded":"2024-09-21T07:11:55.833541+0000","last_fullsized":"2024-09-21T07:11:55.833541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803098+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:12:23.803098+0000","last_peered":"2024-09-21T07:12:23.803098+0000","last_clean":"2024-09-21T07:12:23.803098+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:12:23.803098+0000","last_undegraded":"2024-09-21T07:12:23.803098+0000","last_fullsized":"2024-09-21T07:12:23.803098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807453+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:12:23.807453+0000","last_peered":"2024-09-21T07:12:23.807453+0000","last_clean":"2024-09-21T07:12:23.807453+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:12:23.807453+0000","last_undegraded":"2024-09-21T07:12:23.807453+0000","last_fullsized":"2024-09-21T07:12:23.807453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808124+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:12:23.808124+0000","last_peered":"2024-09-21T07:12:23.808124+0000","last_clean":"2024-09-21T07:12:23.808124+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:12:23.808124+0000","last_undegraded":"2024-09-21T07:12:23.808124+0000","last_fullsized":"2024-09-21T07:12:23.808124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795490+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:12:23.795490+0000","last_peered":"2024-09-21T07:12:23.795490+0000","last_clean":"2024-09-21T07:12:23.795490+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:12:23.795490+0000","last_undegraded":"2024-09-21T07:12:23.795490+0000","last_fullsized":"2024-09-21T07:12:23.795490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805863+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:12:23.805863+0000","last_peered":"2024-09-21T07:12:23.805863+0000","last_clean":"2024-09-21T07:12:23.805863+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:12:23.805863+0000","last_undegraded":"2024-09-21T07:12:23.805863+0000","last_fullsized":"2024-09-21T07:12:23.805863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801102+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:12:23.801102+0000","last_peered":"2024-09-21T07:12:23.801102+0000","last_clean":"2024-09-21T07:12:23.801102+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:12:23.801102+0000","last_undegraded":"2024-09-21T07:12:23.801102+0000","last_fullsized":"2024-09-21T07:12:23.801102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797345+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:12:23.797345+0000","last_peered":"2024-09-21T07:12:23.797345+0000","last_clean":"2024-09-21T07:12:23.797345+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:12:23.797345+0000","last_undegraded":"2024-09-21T07:12:23.797345+0000","last_fullsized":"2024-09-21T07:12:23.797345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806243+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:12:23.806243+0000","last_peered":"2024-09-21T07:12:23.806243+0000","last_clean":"2024-09-21T07:12:23.806243+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:12:23.806243+0000","last_undegraded":"2024-09-21T07:12:23.806243+0000","last_fullsized":"2024-09-21T07:12:23.806243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810287+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:12:23.810287+0000","last_peered":"2024-09-21T07:12:23.810287+0000","last_clean":"2024-09-21T07:12:23.810287+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:12:23.810287+0000","last_undegraded":"2024-09-21T07:12:23.810287+0000","last_fullsized":"2024-09-21T07:12:23.810287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797647+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:12:23.797647+0000","last_peered":"2024-09-21T07:12:23.797647+0000","last_clean":"2024-09-21T07:12:23.797647+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:12:23.797647+0000","last_undegraded":"2024-09-21T07:12:23.797647+0000","last_fullsized":"2024-09-21T07:12:23.797647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.815349+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:12:23.815349+0000","last_peered":"2024-09-21T07:12:23.815349+0000","last_clean":"2024-09-21T07:12:23.815349+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:12:23.815349+0000","last_undegraded":"2024-09-21T07:12:23.815349+0000","last_fullsized":"2024-09-21T07:12:23.815349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809317+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:12:23.809317+0000","last_peered":"2024-09-21T07:12:23.809317+0000","last_clean":"2024-09-21T07:12:23.809317+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:12:23.809317+0000","last_undegraded":"2024-09-21T07:12:23.809317+0000","last_fullsized":"2024-09-21T07:12:23.809317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795846+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:12:23.795846+0000","last_peered":"2024-09-21T07:12:23.795846+0000","last_clean":"2024-09-21T07:12:23.795846+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:12:23.795846+0000","last_undegraded":"2024-09-21T07:12:23.795846+0000","last_fullsized":"2024-09-21T07:12:23.795846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808162+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:12:23.808162+0000","last_peered":"2024-09-21T07:12:23.808162+0000","last_clean":"2024-09-21T07:12:23.808162+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:12:23.808162+0000","last_undegraded":"2024-09-21T07:12:23.808162+0000","last_fullsized":"2024-09-21T07:12:23.808162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797650+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:12:23.797650+0000","last_peered":"2024-09-21T07:12:23.797650+0000","last_clean":"2024-09-21T07:12:23.797650+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:12:23.797650+0000","last_undegraded":"2024-09-21T07:12:23.797650+0000","last_fullsized":"2024-09-21T07:12:23.797650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811678+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:12:23.811678+0000","last_peered":"2024-09-21T07:12:23.811678+0000","last_clean":"2024-09-21T07:12:23.811678+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:12:23.811678+0000","last_undegraded":"2024-09-21T07:12:23.811678+0000","last_fullsized":"2024-09-21T07:12:23.811678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814831+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:12:23.814831+0000","last_peered":"2024-09-21T07:12:23.814831+0000","last_clean":"2024-09-21T07:12:23.814831+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:12:23.814831+0000","last_undegraded":"2024-09-21T07:12:23.814831+0000","last_fullsized":"2024-09-21T07:12:23.814831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798081+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:12:23.798081+0000","last_peered":"2024-09-21T07:12:23.798081+0000","last_clean":"2024-09-21T07:12:23.798081+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:12:23.798081+0000","last_undegraded":"2024-09-21T07:12:23.798081+0000","last_fullsized":"2024-09-21T07:12:23.798081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797741+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:12:23.797741+0000","last_peered":"2024-09-21T07:12:23.797741+0000","last_clean":"2024-09-21T07:12:23.797741+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:12:23.797741+0000","last_undegraded":"2024-09-21T07:12:23.797741+0000","last_fullsized":"2024-09-21T07:12:23.797741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797093+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:12:23.797093+0000","last_peered":"2024-09-21T07:12:23.797093+0000","last_clean":"2024-09-21T07:12:23.797093+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:12:23.797093+0000","last_undegraded":"2024-09-21T07:12:23.797093+0000","last_fullsized":"2024-09-21T07:12:23.797093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797350+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:12:23.797350+0000","last_peered":"2024-09-21T07:12:23.797350+0000","last_clean":"2024-09-21T07:12:23.797350+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:12:23.797350+0000","last_undegraded":"2024-09-21T07:12:23.797350+0000","last_fullsized":"2024-09-21T07:12:23.797350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814242+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:12:23.814242+0000","last_peered":"2024-09-21T07:12:23.814242+0000","last_clean":"2024-09-21T07:12:23.814242+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:12:23.814242+0000","last_undegraded":"2024-09-21T07:12:23.814242+0000","last_fullsized":"2024-09-21T07:12:23.814242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790030+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:12:23.790030+0000","last_peered":"2024-09-21T07:12:23.790030+0000","last_clean":"2024-09-21T07:12:23.790030+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:12:23.790030+0000","last_undegraded":"2024-09-21T07:12:23.790030+0000","last_fullsized":"2024-09-21T07:12:23.790030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810378+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:12:23.810378+0000","last_peered":"2024-09-21T07:12:23.810378+0000","last_clean":"2024-09-21T07:12:23.810378+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:12:23.810378+0000","last_undegraded":"2024-09-21T07:12:23.810378+0000","last_fullsized":"2024-09-21T07:12:23.810378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809256+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:12:23.809256+0000","last_peered":"2024-09-21T07:12:23.809256+0000","last_clean":"2024-09-21T07:12:23.809256+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:12:23.809256+0000","last_undegraded":"2024-09-21T07:12:23.809256+0000","last_fullsized":"2024-09-21T07:12:23.809256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806255+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:12:23.806255+0000","last_peered":"2024-09-21T07:12:23.806255+0000","last_clean":"2024-09-21T07:12:23.806255+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:12:23.806255+0000","last_undegraded":"2024-09-21T07:12:23.806255+0000","last_fullsized":"2024-09-21T07:12:23.806255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788650+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:12:23.788650+0000","last_peered":"2024-09-21T07:12:23.788650+0000","last_clean":"2024-09-21T07:12:23.788650+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:12:23.788650+0000","last_undegraded":"2024-09-21T07:12:23.788650+0000","last_fullsized":"2024-09-21T07:12:23.788650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809845+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:12:23.809845+0000","last_peered":"2024-09-21T07:12:23.809845+0000","last_clean":"2024-09-21T07:12:23.809845+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:12:23.809845+0000","last_undegraded":"2024-09-21T07:12:23.809845+0000","last_fullsized":"2024-09-21T07:12:23.809845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.789398+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:12:23.789398+0000","last_peered":"2024-09-21T07:12:23.789398+0000","last_clean":"2024-09-21T07:12:23.789398+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:12:23.789398+0000","last_undegraded":"2024-09-21T07:12:23.789398+0000","last_fullsized":"2024-09-21T07:12:23.789398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811791+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:12:23.811791+0000","last_peered":"2024-09-21T07:12:23.811791+0000","last_clean":"2024-09-21T07:12:23.811791+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:12:23.811791+0000","last_undegraded":"2024-09-21T07:12:23.811791+0000","last_fullsized":"2024-09-21T07:12:23.811791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.799236+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:12:23.799236+0000","last_peered":"2024-09-21T07:12:23.799236+0000","last_clean":"2024-09-21T07:12:23.799236+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:12:23.799236+0000","last_undegraded":"2024-09-21T07:12:23.799236+0000","last_fullsized":"2024-09-21T07:12:23.799236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811604+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:12:23.811604+0000","last_peered":"2024-09-21T07:12:23.811604+0000","last_clean":"2024-09-21T07:12:23.811604+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:12:23.811604+0000","last_undegraded":"2024-09-21T07:12:23.811604+0000","last_fullsized":"2024-09-21T07:12:23.811604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":473577,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1098,"num_read_kb":754,"num_write":343,"num_write_kb":1361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":2208283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":245,"ondisk_log_size":245,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":515860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972407,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37456,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707696,"statfs":{"total":95995035648,"available":95956680704,"internally_reserved":0,"allocated":2101248,"data_stored":1572931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135933,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1413120,"data_stored":913386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":515972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495686,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1113137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659212,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":511250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855443,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1107802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018967,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215196,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708052,"statfs":{"total":95995035648,"available":95957045248,"internally_reserved":0,"allocated":1798144,"data_stored":1298436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":588338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":586286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705774,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":77178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":471403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:13:00.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T07:13:01.002 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:13:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:13:00.958+0000 7f75fc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:13:01.002 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:13:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:13:00.958+0000 7f75fc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:13:01.002 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:13:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:13:00.958+0000 7f75fc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:13:01.003 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:13:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:13:00.958+0000 7f75fc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:13:01.003 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:13:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:13:00.958+0000 7f75fc000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:13:01.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:01 smithi027 ceph-mon[30779]: pgmap v377: 129 pgs: 129 active+clean; 1.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 102 KiB/s wr, 45 op/s 2024-09-21T07:13:01.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:01 smithi027 ceph-mon[30779]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:01.231 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:01 smithi119 ceph-mon[34519]: pgmap v377: 129 pgs: 129 active+clean; 1.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 102 KiB/s wr, 45 op/s 2024-09-21T07:13:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:01 smithi119 ceph-mon[34519]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:01 smithi153 ceph-mon[34747]: pgmap v377: 129 pgs: 129 active+clean; 1.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 102 KiB/s wr, 45 op/s 2024-09-21T07:13:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:01 smithi153 ceph-mon[34747]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:01.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 -- 172.21.15.27:0/33174929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f4103b60 msgr2=0x7f37f4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 --2- 172.21.15.27:0/33174929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f4103b60 0x7f37f4103f70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009900 tx=0x7f37e402a410 comp rx=0 tx=0).stop 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 -- 172.21.15.27:0/33174929 shutdown_connections 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 --2- 172.21.15.27:0/33174929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37f4109df0 0x7f37f4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 --2- 172.21.15.27:0/33174929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37f41094a0 0x7f37f41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 --2- 172.21.15.27:0/33174929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f4103b60 0x7f37f4103f70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 -- 172.21.15.27:0/33174929 >> 172.21.15.27:0/33174929 conn(0x7f37f407b850 msgr2=0x7f37f407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 -- 172.21.15.27:0/33174929 shutdown_connections 2024-09-21T07:13:01.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.822+0000 7f37fa800640 1 -- 172.21.15.27:0/33174929 wait complete. 2024-09-21T07:13:01.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.823+0000 7f37fa800640 1 Processor -- start 2024-09-21T07:13:01.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.823+0000 7f37fa800640 1 -- start start 2024-09-21T07:13:01.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37fa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37f4103b60 0x7f37f419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:01.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37fa800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f41094a0 0x7f37f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:01.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37fa800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37f4109df0 0x7f37f41a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:01.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37fa800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4113ab0 con 0x7f37f41094a0 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37fa800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4113930 con 0x7f37f4103b60 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37fa800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4113c30 con 0x7f37f4109df0 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37f3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37f4103b60 0x7f37f419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37ebe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f41094a0 0x7f37f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37f3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37f4103b60 0x7f37f419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47924/0 (socket says 172.21.15.27:47924) 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37f3400640 1 -- 172.21.15.27:0/724491930 learned_addr learned my addr 172.21.15.27:0/724491930 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.824+0000 7f37f3e00640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37f4109df0 0x7f37f41a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37ebe00640 1 -- 172.21.15.27:0/724491930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37f4109df0 msgr2=0x7f37f41a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37ebe00640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37f4109df0 0x7f37f41a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37ebe00640 1 -- 172.21.15.27:0/724491930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37f4103b60 msgr2=0x7f37f419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37ebe00640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37f4103b60 0x7f37f419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37ebe00640 1 -- 172.21.15.27:0/724491930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f41a25f0 con 0x7f37f41094a0 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37f3e00640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37f4109df0 0x7f37f41a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:01.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37ebe00640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f41094a0 0x7f37f419cc20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f37f406c9f0 tx=0x7f37dc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:01.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00cc40 con 0x7f37f41094a0 2024-09-21T07:13:01.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37dc00cde0 con 0x7f37f41094a0 2024-09-21T07:13:01.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.826+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc018780 con 0x7f37f41094a0 2024-09-21T07:13:01.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.825+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f41a28e0 con 0x7f37f41094a0 2024-09-21T07:13:01.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.826+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f41a9040 con 0x7f37f41094a0 2024-09-21T07:13:01.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.827+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f4103f70 con 0x7f37f41094a0 2024-09-21T07:13:01.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.830+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f37dc010a90 con 0x7f37f41094a0 2024-09-21T07:13:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.833+0000 7f37f0c00640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37d80b6680 0x7f37d80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:01.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.833+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f37dc0e8210 con 0x7f37f41094a0 2024-09-21T07:13:01.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.833+0000 7f37f3400640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37d80b6680 0x7f37d80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:01.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.834+0000 7f37f3400640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37d80b6680 0x7f37d80b8b90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f37e4031440 tx=0x7f37e40058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:01.834+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37dc0b2790 con 0x7f37f41094a0 2024-09-21T07:13:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.018+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f37dc0a9fd0 con 0x7f37f41094a0 2024-09-21T07:13:02.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.035+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37f41079c0 con 0x7f37d80b6680 2024-09-21T07:13:02.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.042+0000 7f37f0c00640 1 -- 172.21.15.27:0/724491930 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340537 (secure 0 0 0) 0x7f37f41079c0 con 0x7f37d80b6680 2024-09-21T07:13:02.049 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:02.050 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:13:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37d80b6680 msgr2=0x7f37d80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37d80b6680 0x7f37d80b8b90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f37e4031440 tx=0x7f37e40058d0 comp rx=0 tx=0).stop 2024-09-21T07:13:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f41094a0 msgr2=0x7f37f419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f41094a0 0x7f37f419cc20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f37f406c9f0 tx=0x7f37dc00ee50 comp rx=0 tx=0).stop 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 shutdown_connections 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37f4109df0 0x7f37f41a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37d80b6680 0x7f37d80b8b90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f41094a0 0x7f37f419cc20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 --2- 172.21.15.27:0/724491930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37f4103b60 0x7f37f419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 >> 172.21.15.27:0/724491930 conn(0x7f37f407b850 msgr2=0x7f37f41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 shutdown_connections 2024-09-21T07:13:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:02.047+0000 7f37fa800640 1 -- 172.21.15.27:0/724491930 wait complete. 2024-09-21T07:13:02.954 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-21T07:13:01.418318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1457955,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1226,"num_read_kb":859,"num_write":538,"num_write_kb":4385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":898,"ondisk_log_size":898,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398424,"kb_used_data":15776,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543400,"statfs":{"total":1151940427776,"available":1151532441600,"internally_reserved":0,"allocated":16154624,"data_stored":10146635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":787383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":414,"num_read_kb":341,"num_write":225,"num_write_kb":2089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003831"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:58.089462+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:11:58.089462+0000","last_peered":"2024-09-21T07:11:58.089462+0000","last_clean":"2024-09-21T07:11:58.089462+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:11:58.089462+0000","last_undegraded":"2024-09-21T07:11:58.089462+0000","last_fullsized":"2024-09-21T07:11:58.089462+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805255+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:12:23.805255+0000","last_peered":"2024-09-21T07:12:23.805255+0000","last_clean":"2024-09-21T07:12:23.805255+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:12:23.805255+0000","last_undegraded":"2024-09-21T07:12:23.805255+0000","last_fullsized":"2024-09-21T07:12:23.805255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832485+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:11:55.832485+0000","last_peered":"2024-09-21T07:11:55.832485+0000","last_clean":"2024-09-21T07:11:55.832485+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:11:55.832485+0000","last_undegraded":"2024-09-21T07:11:55.832485+0000","last_fullsized":"2024-09-21T07:11:55.832485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833749+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:11:55.833749+0000","last_peered":"2024-09-21T07:11:55.833749+0000","last_clean":"2024-09-21T07:11:55.833749+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:11:55.833749+0000","last_undegraded":"2024-09-21T07:11:55.833749+0000","last_fullsized":"2024-09-21T07:11:55.833749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811697+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:12:23.811697+0000","last_peered":"2024-09-21T07:12:23.811697+0000","last_clean":"2024-09-21T07:12:23.811697+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:12:23.811697+0000","last_undegraded":"2024-09-21T07:12:23.811697+0000","last_fullsized":"2024-09-21T07:12:23.811697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844771+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:11:55.844771+0000","last_peered":"2024-09-21T07:11:55.844771+0000","last_clean":"2024-09-21T07:11:55.844771+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:11:55.844771+0000","last_undegraded":"2024-09-21T07:11:55.844771+0000","last_fullsized":"2024-09-21T07:11:55.844771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847516+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:11:55.847516+0000","last_peered":"2024-09-21T07:11:55.847516+0000","last_clean":"2024-09-21T07:11:55.847516+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:11:55.847516+0000","last_undegraded":"2024-09-21T07:11:55.847516+0000","last_fullsized":"2024-09-21T07:11:55.847516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814377+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:12:23.814377+0000","last_peered":"2024-09-21T07:12:23.814377+0000","last_clean":"2024-09-21T07:12:23.814377+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:12:23.814377+0000","last_undegraded":"2024-09-21T07:12:23.814377+0000","last_fullsized":"2024-09-21T07:12:23.814377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845066+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:11:55.845066+0000","last_peered":"2024-09-21T07:11:55.845066+0000","last_clean":"2024-09-21T07:11:55.845066+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:11:55.845066+0000","last_undegraded":"2024-09-21T07:11:55.845066+0000","last_fullsized":"2024-09-21T07:11:55.845066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847357+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:11:55.847357+0000","last_peered":"2024-09-21T07:11:55.847357+0000","last_clean":"2024-09-21T07:11:55.847357+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:11:55.847357+0000","last_undegraded":"2024-09-21T07:11:55.847357+0000","last_fullsized":"2024-09-21T07:11:55.847357+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803507+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:12:23.803507+0000","last_peered":"2024-09-21T07:12:23.803507+0000","last_clean":"2024-09-21T07:12:23.803507+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:12:23.803507+0000","last_undegraded":"2024-09-21T07:12:23.803507+0000","last_fullsized":"2024-09-21T07:12:23.803507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844870+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:11:55.844870+0000","last_peered":"2024-09-21T07:11:55.844870+0000","last_clean":"2024-09-21T07:11:55.844870+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:11:55.844870+0000","last_undegraded":"2024-09-21T07:11:55.844870+0000","last_fullsized":"2024-09-21T07:11:55.844870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813396+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:12:23.813396+0000","last_peered":"2024-09-21T07:12:23.813396+0000","last_clean":"2024-09-21T07:12:23.813396+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:12:23.813396+0000","last_undegraded":"2024-09-21T07:12:23.813396+0000","last_fullsized":"2024-09-21T07:12:23.813396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846961+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:11:55.846961+0000","last_peered":"2024-09-21T07:11:55.846961+0000","last_clean":"2024-09-21T07:11:55.846961+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:11:55.846961+0000","last_undegraded":"2024-09-21T07:11:55.846961+0000","last_fullsized":"2024-09-21T07:11:55.846961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845137+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:11:55.845137+0000","last_peered":"2024-09-21T07:11:55.845137+0000","last_clean":"2024-09-21T07:11:55.845137+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:11:55.845137+0000","last_undegraded":"2024-09-21T07:11:55.845137+0000","last_fullsized":"2024-09-21T07:11:55.845137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807789+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:12:23.807789+0000","last_peered":"2024-09-21T07:12:23.807789+0000","last_clean":"2024-09-21T07:12:23.807789+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:12:23.807789+0000","last_undegraded":"2024-09-21T07:12:23.807789+0000","last_fullsized":"2024-09-21T07:12:23.807789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832340+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:11:55.832340+0000","last_peered":"2024-09-21T07:11:55.832340+0000","last_clean":"2024-09-21T07:11:55.832340+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:11:55.832340+0000","last_undegraded":"2024-09-21T07:11:55.832340+0000","last_fullsized":"2024-09-21T07:11:55.832340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845070+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:11:55.845070+0000","last_peered":"2024-09-21T07:11:55.845070+0000","last_clean":"2024-09-21T07:11:55.845070+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:11:55.845070+0000","last_undegraded":"2024-09-21T07:11:55.845070+0000","last_fullsized":"2024-09-21T07:11:55.845070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798453+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:12:23.798453+0000","last_peered":"2024-09-21T07:12:23.798453+0000","last_clean":"2024-09-21T07:12:23.798453+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:12:23.798453+0000","last_undegraded":"2024-09-21T07:12:23.798453+0000","last_fullsized":"2024-09-21T07:12:23.798453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833325+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:11:55.833325+0000","last_peered":"2024-09-21T07:11:55.833325+0000","last_clean":"2024-09-21T07:11:55.833325+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:11:55.833325+0000","last_undegraded":"2024-09-21T07:11:55.833325+0000","last_fullsized":"2024-09-21T07:11:55.833325+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830920+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:11:55.830920+0000","last_peered":"2024-09-21T07:11:55.830920+0000","last_clean":"2024-09-21T07:11:55.830920+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:11:55.830920+0000","last_undegraded":"2024-09-21T07:11:55.830920+0000","last_fullsized":"2024-09-21T07:11:55.830920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.780871+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:12:23.780871+0000","last_peered":"2024-09-21T07:12:23.780871+0000","last_clean":"2024-09-21T07:12:23.780871+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:12:23.780871+0000","last_undegraded":"2024-09-21T07:12:23.780871+0000","last_fullsized":"2024-09-21T07:12:23.780871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833051+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:11:55.833051+0000","last_peered":"2024-09-21T07:11:55.833051+0000","last_clean":"2024-09-21T07:11:55.833051+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:11:55.833051+0000","last_undegraded":"2024-09-21T07:11:55.833051+0000","last_fullsized":"2024-09-21T07:11:55.833051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845144+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:11:55.845144+0000","last_peered":"2024-09-21T07:11:55.845144+0000","last_clean":"2024-09-21T07:11:55.845144+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:11:55.845144+0000","last_undegraded":"2024-09-21T07:11:55.845144+0000","last_fullsized":"2024-09-21T07:11:55.845144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.800049+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:12:23.800049+0000","last_peered":"2024-09-21T07:12:23.800049+0000","last_clean":"2024-09-21T07:12:23.800049+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:12:23.800049+0000","last_undegraded":"2024-09-21T07:12:23.800049+0000","last_fullsized":"2024-09-21T07:12:23.800049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845369+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:11:55.845369+0000","last_peered":"2024-09-21T07:11:55.845369+0000","last_clean":"2024-09-21T07:11:55.845369+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:11:55.845369+0000","last_undegraded":"2024-09-21T07:11:55.845369+0000","last_fullsized":"2024-09-21T07:11:55.845369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808689+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:12:23.808689+0000","last_peered":"2024-09-21T07:12:23.808689+0000","last_clean":"2024-09-21T07:12:23.808689+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:12:23.808689+0000","last_undegraded":"2024-09-21T07:12:23.808689+0000","last_fullsized":"2024-09-21T07:12:23.808689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'97","reported_seq":232,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:57.948829+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:12:57.948829+0000","last_peered":"2024-09-21T07:12:57.948829+0000","last_clean":"2024-09-21T07:12:57.948829+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:12:57.948829+0000","last_undegraded":"2024-09-21T07:12:57.948829+0000","last_fullsized":"2024-09-21T07:12:57.948829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832993+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:11:55.832993+0000","last_peered":"2024-09-21T07:11:55.832993+0000","last_clean":"2024-09-21T07:11:55.832993+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:11:55.832993+0000","last_undegraded":"2024-09-21T07:11:55.832993+0000","last_fullsized":"2024-09-21T07:11:55.832993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809315+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:12:23.809315+0000","last_peered":"2024-09-21T07:12:23.809315+0000","last_clean":"2024-09-21T07:12:23.809315+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:12:23.809315+0000","last_undegraded":"2024-09-21T07:12:23.809315+0000","last_fullsized":"2024-09-21T07:12:23.809315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847542+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:11:55.847542+0000","last_peered":"2024-09-21T07:11:55.847542+0000","last_clean":"2024-09-21T07:11:55.847542+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:11:55.847542+0000","last_undegraded":"2024-09-21T07:11:55.847542+0000","last_fullsized":"2024-09-21T07:11:55.847542+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790825+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:12:23.790825+0000","last_peered":"2024-09-21T07:12:23.790825+0000","last_clean":"2024-09-21T07:12:23.790825+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:12:23.790825+0000","last_undegraded":"2024-09-21T07:12:23.790825+0000","last_fullsized":"2024-09-21T07:12:23.790825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830831+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:11:55.830831+0000","last_peered":"2024-09-21T07:11:55.830831+0000","last_clean":"2024-09-21T07:11:55.830831+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:11:55.830831+0000","last_undegraded":"2024-09-21T07:11:55.830831+0000","last_fullsized":"2024-09-21T07:11:55.830831+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846866+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:11:55.846866+0000","last_peered":"2024-09-21T07:11:55.846866+0000","last_clean":"2024-09-21T07:11:55.846866+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:11:55.846866+0000","last_undegraded":"2024-09-21T07:11:55.846866+0000","last_fullsized":"2024-09-21T07:11:55.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.794745+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:12:23.794745+0000","last_peered":"2024-09-21T07:12:23.794745+0000","last_clean":"2024-09-21T07:12:23.794745+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:12:23.794745+0000","last_undegraded":"2024-09-21T07:12:23.794745+0000","last_fullsized":"2024-09-21T07:12:23.794745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981709+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:11:55.981709+0000","last_peered":"2024-09-21T07:11:55.981709+0000","last_clean":"2024-09-21T07:11:55.981709+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:11:55.981709+0000","last_undegraded":"2024-09-21T07:11:55.981709+0000","last_fullsized":"2024-09-21T07:11:55.981709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845966+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:11:55.845966+0000","last_peered":"2024-09-21T07:11:55.845966+0000","last_clean":"2024-09-21T07:11:55.845966+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:11:55.845966+0000","last_undegraded":"2024-09-21T07:11:55.845966+0000","last_fullsized":"2024-09-21T07:11:55.845966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795850+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:12:23.795850+0000","last_peered":"2024-09-21T07:12:23.795850+0000","last_clean":"2024-09-21T07:12:23.795850+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:12:23.795850+0000","last_undegraded":"2024-09-21T07:12:23.795850+0000","last_fullsized":"2024-09-21T07:12:23.795850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":182,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:59.771521+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:12:59.771521+0000","last_peered":"2024-09-21T07:12:59.771521+0000","last_clean":"2024-09-21T07:12:59.771521+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:12:59.771521+0000","last_undegraded":"2024-09-21T07:12:59.771521+0000","last_fullsized":"2024-09-21T07:12:59.771521+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846521+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:11:55.846521+0000","last_peered":"2024-09-21T07:11:55.846521+0000","last_clean":"2024-09-21T07:11:55.846521+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:11:55.846521+0000","last_undegraded":"2024-09-21T07:11:55.846521+0000","last_fullsized":"2024-09-21T07:11:55.846521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807529+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:12:23.807529+0000","last_peered":"2024-09-21T07:12:23.807529+0000","last_clean":"2024-09-21T07:12:23.807529+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:12:23.807529+0000","last_undegraded":"2024-09-21T07:12:23.807529+0000","last_fullsized":"2024-09-21T07:12:23.807529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981500+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:11:55.981500+0000","last_peered":"2024-09-21T07:11:55.981500+0000","last_clean":"2024-09-21T07:11:55.981500+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:11:55.981500+0000","last_undegraded":"2024-09-21T07:11:55.981500+0000","last_fullsized":"2024-09-21T07:11:55.981500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845592+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:11:55.845592+0000","last_peered":"2024-09-21T07:11:55.845592+0000","last_clean":"2024-09-21T07:11:55.845592+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:11:55.845592+0000","last_undegraded":"2024-09-21T07:11:55.845592+0000","last_fullsized":"2024-09-21T07:11:55.845592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811077+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:12:23.811077+0000","last_peered":"2024-09-21T07:12:23.811077+0000","last_clean":"2024-09-21T07:12:23.811077+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:12:23.811077+0000","last_undegraded":"2024-09-21T07:12:23.811077+0000","last_fullsized":"2024-09-21T07:12:23.811077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845392+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:11:55.845392+0000","last_peered":"2024-09-21T07:11:55.845392+0000","last_clean":"2024-09-21T07:11:55.845392+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:11:55.845392+0000","last_undegraded":"2024-09-21T07:11:55.845392+0000","last_fullsized":"2024-09-21T07:11:55.845392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981606+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:11:55.981606+0000","last_peered":"2024-09-21T07:11:55.981606+0000","last_clean":"2024-09-21T07:11:55.981606+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:11:55.981606+0000","last_undegraded":"2024-09-21T07:11:55.981606+0000","last_fullsized":"2024-09-21T07:11:55.981606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981807+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:11:55.981807+0000","last_peered":"2024-09-21T07:11:55.981807+0000","last_clean":"2024-09-21T07:11:55.981807+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:11:55.981807+0000","last_undegraded":"2024-09-21T07:11:55.981807+0000","last_fullsized":"2024-09-21T07:11:55.981807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814663+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:12:23.814663+0000","last_peered":"2024-09-21T07:12:23.814663+0000","last_clean":"2024-09-21T07:12:23.814663+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:12:23.814663+0000","last_undegraded":"2024-09-21T07:12:23.814663+0000","last_fullsized":"2024-09-21T07:12:23.814663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.834191+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:11:55.834191+0000","last_peered":"2024-09-21T07:11:55.834191+0000","last_clean":"2024-09-21T07:11:55.834191+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:11:55.834191+0000","last_undegraded":"2024-09-21T07:11:55.834191+0000","last_fullsized":"2024-09-21T07:11:55.834191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833166+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:11:55.833166+0000","last_peered":"2024-09-21T07:11:55.833166+0000","last_clean":"2024-09-21T07:11:55.833166+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:11:55.833166+0000","last_undegraded":"2024-09-21T07:11:55.833166+0000","last_fullsized":"2024-09-21T07:11:55.833166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807409+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:12:23.807409+0000","last_peered":"2024-09-21T07:12:23.807409+0000","last_clean":"2024-09-21T07:12:23.807409+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:12:23.807409+0000","last_undegraded":"2024-09-21T07:12:23.807409+0000","last_fullsized":"2024-09-21T07:12:23.807409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'60","reported_seq":135,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:57.447818+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:12:57.447818+0000","last_peered":"2024-09-21T07:12:57.447818+0000","last_clean":"2024-09-21T07:12:57.447818+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:12:57.447818+0000","last_undegraded":"2024-09-21T07:12:57.447818+0000","last_fullsized":"2024-09-21T07:12:57.447818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846714+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:11:55.846714+0000","last_peered":"2024-09-21T07:11:55.846714+0000","last_clean":"2024-09-21T07:11:55.846714+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:11:55.846714+0000","last_undegraded":"2024-09-21T07:11:55.846714+0000","last_fullsized":"2024-09-21T07:11:55.846714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814102+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:12:23.814102+0000","last_peered":"2024-09-21T07:12:23.814102+0000","last_clean":"2024-09-21T07:12:23.814102+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:12:23.814102+0000","last_undegraded":"2024-09-21T07:12:23.814102+0000","last_fullsized":"2024-09-21T07:12:23.814102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846385+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:11:55.846385+0000","last_peered":"2024-09-21T07:11:55.846385+0000","last_clean":"2024-09-21T07:11:55.846385+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:11:55.846385+0000","last_undegraded":"2024-09-21T07:11:55.846385+0000","last_fullsized":"2024-09-21T07:11:55.846385+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845182+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:11:55.845182+0000","last_peered":"2024-09-21T07:11:55.845182+0000","last_clean":"2024-09-21T07:11:55.845182+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:11:55.845182+0000","last_undegraded":"2024-09-21T07:11:55.845182+0000","last_fullsized":"2024-09-21T07:11:55.845182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801647+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:12:23.801647+0000","last_peered":"2024-09-21T07:12:23.801647+0000","last_clean":"2024-09-21T07:12:23.801647+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:12:23.801647+0000","last_undegraded":"2024-09-21T07:12:23.801647+0000","last_fullsized":"2024-09-21T07:12:23.801647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832950+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:11:55.832950+0000","last_peered":"2024-09-21T07:11:55.832950+0000","last_clean":"2024-09-21T07:11:55.832950+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:11:55.832950+0000","last_undegraded":"2024-09-21T07:11:55.832950+0000","last_fullsized":"2024-09-21T07:11:55.832950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845860+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:11:55.845860+0000","last_peered":"2024-09-21T07:11:55.845860+0000","last_clean":"2024-09-21T07:11:55.845860+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:11:55.845860+0000","last_undegraded":"2024-09-21T07:11:55.845860+0000","last_fullsized":"2024-09-21T07:11:55.845860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802831+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:12:23.802831+0000","last_peered":"2024-09-21T07:12:23.802831+0000","last_clean":"2024-09-21T07:12:23.802831+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:12:23.802831+0000","last_undegraded":"2024-09-21T07:12:23.802831+0000","last_fullsized":"2024-09-21T07:12:23.802831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846785+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:11:55.846785+0000","last_peered":"2024-09-21T07:11:55.846785+0000","last_clean":"2024-09-21T07:11:55.846785+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:11:55.846785+0000","last_undegraded":"2024-09-21T07:11:55.846785+0000","last_fullsized":"2024-09-21T07:11:55.846785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846243+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:11:55.846243+0000","last_peered":"2024-09-21T07:11:55.846243+0000","last_clean":"2024-09-21T07:11:55.846243+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:11:55.846243+0000","last_undegraded":"2024-09-21T07:11:55.846243+0000","last_fullsized":"2024-09-21T07:11:55.846243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805668+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:12:23.805668+0000","last_peered":"2024-09-21T07:12:23.805668+0000","last_clean":"2024-09-21T07:12:23.805668+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:12:23.805668+0000","last_undegraded":"2024-09-21T07:12:23.805668+0000","last_fullsized":"2024-09-21T07:12:23.805668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":186,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:58.770752+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:12:58.770752+0000","last_peered":"2024-09-21T07:12:58.770752+0000","last_clean":"2024-09-21T07:12:58.770752+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:12:58.770752+0000","last_undegraded":"2024-09-21T07:12:58.770752+0000","last_fullsized":"2024-09-21T07:12:58.770752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830985+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:11:55.830985+0000","last_peered":"2024-09-21T07:11:55.830985+0000","last_clean":"2024-09-21T07:11:55.830985+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:11:55.830985+0000","last_undegraded":"2024-09-21T07:11:55.830985+0000","last_fullsized":"2024-09-21T07:11:55.830985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813592+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:12:23.813592+0000","last_peered":"2024-09-21T07:12:23.813592+0000","last_clean":"2024-09-21T07:12:23.813592+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:12:23.813592+0000","last_undegraded":"2024-09-21T07:12:23.813592+0000","last_fullsized":"2024-09-21T07:12:23.813592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846256+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:11:55.846256+0000","last_peered":"2024-09-21T07:11:55.846256+0000","last_clean":"2024-09-21T07:11:55.846256+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:11:55.846256+0000","last_undegraded":"2024-09-21T07:11:55.846256+0000","last_fullsized":"2024-09-21T07:11:55.846256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832414+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:11:55.832414+0000","last_peered":"2024-09-21T07:11:55.832414+0000","last_clean":"2024-09-21T07:11:55.832414+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:11:55.832414+0000","last_undegraded":"2024-09-21T07:11:55.832414+0000","last_fullsized":"2024-09-21T07:11:55.832414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808494+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:12:23.808494+0000","last_peered":"2024-09-21T07:12:23.808494+0000","last_clean":"2024-09-21T07:12:23.808494+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:12:23.808494+0000","last_undegraded":"2024-09-21T07:12:23.808494+0000","last_fullsized":"2024-09-21T07:12:23.808494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833515+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:11:55.833515+0000","last_peered":"2024-09-21T07:11:55.833515+0000","last_clean":"2024-09-21T07:11:55.833515+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:11:55.833515+0000","last_undegraded":"2024-09-21T07:11:55.833515+0000","last_fullsized":"2024-09-21T07:11:55.833515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846046+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:11:55.846046+0000","last_peered":"2024-09-21T07:11:55.846046+0000","last_clean":"2024-09-21T07:11:55.846046+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:11:55.846046+0000","last_undegraded":"2024-09-21T07:11:55.846046+0000","last_fullsized":"2024-09-21T07:11:55.846046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798135+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:12:23.798135+0000","last_peered":"2024-09-21T07:12:23.798135+0000","last_clean":"2024-09-21T07:12:23.798135+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:12:23.798135+0000","last_undegraded":"2024-09-21T07:12:23.798135+0000","last_fullsized":"2024-09-21T07:12:23.798135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":355,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:57.769530+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:12:57.769530+0000","last_peered":"2024-09-21T07:12:57.769530+0000","last_clean":"2024-09-21T07:12:57.769530+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:12:57.769530+0000","last_undegraded":"2024-09-21T07:12:57.769530+0000","last_fullsized":"2024-09-21T07:12:57.769530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":269,"num_read_kb":209,"num_write":42,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.848049+0000","last_change":"2024-09-21T07:11:55.847916+0000","last_active":"2024-09-21T07:11:55.848049+0000","last_peered":"2024-09-21T07:11:55.848049+0000","last_clean":"2024-09-21T07:11:55.848049+0000","last_became_active":"2024-09-21T07:11:55.847491+0000","last_became_peered":"2024-09-21T07:11:55.847491+0000","last_unstale":"2024-09-21T07:11:55.848049+0000","last_undegraded":"2024-09-21T07:11:55.848049+0000","last_fullsized":"2024-09-21T07:11:55.848049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:25.702259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,1],"acting":[10,8,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813257+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:12:23.813257+0000","last_peered":"2024-09-21T07:12:23.813257+0000","last_clean":"2024-09-21T07:12:23.813257+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:12:23.813257+0000","last_undegraded":"2024-09-21T07:12:23.813257+0000","last_fullsized":"2024-09-21T07:12:23.813257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846367+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:11:55.846367+0000","last_peered":"2024-09-21T07:11:55.846367+0000","last_clean":"2024-09-21T07:11:55.846367+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:11:55.846367+0000","last_undegraded":"2024-09-21T07:11:55.846367+0000","last_fullsized":"2024-09-21T07:11:55.846367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845245+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:11:55.845245+0000","last_peered":"2024-09-21T07:11:55.845245+0000","last_clean":"2024-09-21T07:11:55.845245+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:11:55.845245+0000","last_undegraded":"2024-09-21T07:11:55.845245+0000","last_fullsized":"2024-09-21T07:11:55.845245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.793771+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:12:23.793771+0000","last_peered":"2024-09-21T07:12:23.793771+0000","last_clean":"2024-09-21T07:12:23.793771+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:12:23.793771+0000","last_undegraded":"2024-09-21T07:12:23.793771+0000","last_fullsized":"2024-09-21T07:12:23.793771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981878+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:11:55.981878+0000","last_peered":"2024-09-21T07:11:55.981878+0000","last_clean":"2024-09-21T07:11:55.981878+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:11:55.981878+0000","last_undegraded":"2024-09-21T07:11:55.981878+0000","last_fullsized":"2024-09-21T07:11:55.981878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830120+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:11:55.830120+0000","last_peered":"2024-09-21T07:11:55.830120+0000","last_clean":"2024-09-21T07:11:55.830120+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:11:55.830120+0000","last_undegraded":"2024-09-21T07:11:55.830120+0000","last_fullsized":"2024-09-21T07:11:55.830120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813258+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:12:23.813258+0000","last_peered":"2024-09-21T07:12:23.813258+0000","last_clean":"2024-09-21T07:12:23.813258+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:12:23.813258+0000","last_undegraded":"2024-09-21T07:12:23.813258+0000","last_fullsized":"2024-09-21T07:12:23.813258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":177,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:56.768270+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:12:56.768270+0000","last_peered":"2024-09-21T07:12:56.768270+0000","last_clean":"2024-09-21T07:12:56.768270+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:12:56.768270+0000","last_undegraded":"2024-09-21T07:12:56.768270+0000","last_fullsized":"2024-09-21T07:12:56.768270+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833187+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:11:55.833187+0000","last_peered":"2024-09-21T07:11:55.833187+0000","last_clean":"2024-09-21T07:11:55.833187+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:11:55.833187+0000","last_undegraded":"2024-09-21T07:11:55.833187+0000","last_fullsized":"2024-09-21T07:11:55.833187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847006+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:11:55.847006+0000","last_peered":"2024-09-21T07:11:55.847006+0000","last_clean":"2024-09-21T07:11:55.847006+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:11:55.847006+0000","last_undegraded":"2024-09-21T07:11:55.847006+0000","last_fullsized":"2024-09-21T07:11:55.847006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802309+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:12:23.802309+0000","last_peered":"2024-09-21T07:12:23.802309+0000","last_clean":"2024-09-21T07:12:23.802309+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:12:23.802309+0000","last_undegraded":"2024-09-21T07:12:23.802309+0000","last_fullsized":"2024-09-21T07:12:23.802309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845739+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:11:55.845739+0000","last_peered":"2024-09-21T07:11:55.845739+0000","last_clean":"2024-09-21T07:11:55.845739+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:11:55.845739+0000","last_undegraded":"2024-09-21T07:11:55.845739+0000","last_fullsized":"2024-09-21T07:11:55.845739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849176+0000","last_change":"2024-09-21T07:11:55.848588+0000","last_active":"2024-09-21T07:11:55.849176+0000","last_peered":"2024-09-21T07:11:55.849176+0000","last_clean":"2024-09-21T07:11:55.849176+0000","last_became_active":"2024-09-21T07:11:55.848025+0000","last_became_peered":"2024-09-21T07:11:55.848025+0000","last_unstale":"2024-09-21T07:11:55.849176+0000","last_undegraded":"2024-09-21T07:11:55.849176+0000","last_fullsized":"2024-09-21T07:11:55.849176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:37.413873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810233+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:12:23.810233+0000","last_peered":"2024-09-21T07:12:23.810233+0000","last_clean":"2024-09-21T07:12:23.810233+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:12:23.810233+0000","last_undegraded":"2024-09-21T07:12:23.810233+0000","last_fullsized":"2024-09-21T07:12:23.810233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845457+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:11:55.845457+0000","last_peered":"2024-09-21T07:11:55.845457+0000","last_clean":"2024-09-21T07:11:55.845457+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:11:55.845457+0000","last_undegraded":"2024-09-21T07:11:55.845457+0000","last_fullsized":"2024-09-21T07:11:55.845457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.835562+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:11:55.835562+0000","last_peered":"2024-09-21T07:11:55.835562+0000","last_clean":"2024-09-21T07:11:55.835562+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:11:55.835562+0000","last_undegraded":"2024-09-21T07:11:55.835562+0000","last_fullsized":"2024-09-21T07:11:55.835562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807495+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:12:23.807495+0000","last_peered":"2024-09-21T07:12:23.807495+0000","last_clean":"2024-09-21T07:12:23.807495+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:12:23.807495+0000","last_undegraded":"2024-09-21T07:12:23.807495+0000","last_fullsized":"2024-09-21T07:12:23.807495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849703+0000","last_change":"2024-09-21T07:11:55.848551+0000","last_active":"2024-09-21T07:11:55.849703+0000","last_peered":"2024-09-21T07:11:55.849703+0000","last_clean":"2024-09-21T07:11:55.849703+0000","last_became_active":"2024-09-21T07:11:55.848056+0000","last_became_peered":"2024-09-21T07:11:55.848056+0000","last_unstale":"2024-09-21T07:11:55.849703+0000","last_undegraded":"2024-09-21T07:11:55.849703+0000","last_fullsized":"2024-09-21T07:11:55.849703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:07.779820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":186,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:58.771615+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:12:58.771615+0000","last_peered":"2024-09-21T07:12:58.771615+0000","last_clean":"2024-09-21T07:12:58.771615+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:12:58.771615+0000","last_undegraded":"2024-09-21T07:12:58.771615+0000","last_fullsized":"2024-09-21T07:12:58.771615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788293+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:12:23.788293+0000","last_peered":"2024-09-21T07:12:23.788293+0000","last_clean":"2024-09-21T07:12:23.788293+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:12:23.788293+0000","last_undegraded":"2024-09-21T07:12:23.788293+0000","last_fullsized":"2024-09-21T07:12:23.788293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833541+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:11:55.833541+0000","last_peered":"2024-09-21T07:11:55.833541+0000","last_clean":"2024-09-21T07:11:55.833541+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:11:55.833541+0000","last_undegraded":"2024-09-21T07:11:55.833541+0000","last_fullsized":"2024-09-21T07:11:55.833541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803098+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:12:23.803098+0000","last_peered":"2024-09-21T07:12:23.803098+0000","last_clean":"2024-09-21T07:12:23.803098+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:12:23.803098+0000","last_undegraded":"2024-09-21T07:12:23.803098+0000","last_fullsized":"2024-09-21T07:12:23.803098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807453+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:12:23.807453+0000","last_peered":"2024-09-21T07:12:23.807453+0000","last_clean":"2024-09-21T07:12:23.807453+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:12:23.807453+0000","last_undegraded":"2024-09-21T07:12:23.807453+0000","last_fullsized":"2024-09-21T07:12:23.807453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808124+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:12:23.808124+0000","last_peered":"2024-09-21T07:12:23.808124+0000","last_clean":"2024-09-21T07:12:23.808124+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:12:23.808124+0000","last_undegraded":"2024-09-21T07:12:23.808124+0000","last_fullsized":"2024-09-21T07:12:23.808124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795490+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:12:23.795490+0000","last_peered":"2024-09-21T07:12:23.795490+0000","last_clean":"2024-09-21T07:12:23.795490+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:12:23.795490+0000","last_undegraded":"2024-09-21T07:12:23.795490+0000","last_fullsized":"2024-09-21T07:12:23.795490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805863+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:12:23.805863+0000","last_peered":"2024-09-21T07:12:23.805863+0000","last_clean":"2024-09-21T07:12:23.805863+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:12:23.805863+0000","last_undegraded":"2024-09-21T07:12:23.805863+0000","last_fullsized":"2024-09-21T07:12:23.805863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801102+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:12:23.801102+0000","last_peered":"2024-09-21T07:12:23.801102+0000","last_clean":"2024-09-21T07:12:23.801102+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:12:23.801102+0000","last_undegraded":"2024-09-21T07:12:23.801102+0000","last_fullsized":"2024-09-21T07:12:23.801102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797345+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:12:23.797345+0000","last_peered":"2024-09-21T07:12:23.797345+0000","last_clean":"2024-09-21T07:12:23.797345+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:12:23.797345+0000","last_undegraded":"2024-09-21T07:12:23.797345+0000","last_fullsized":"2024-09-21T07:12:23.797345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806243+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:12:23.806243+0000","last_peered":"2024-09-21T07:12:23.806243+0000","last_clean":"2024-09-21T07:12:23.806243+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:12:23.806243+0000","last_undegraded":"2024-09-21T07:12:23.806243+0000","last_fullsized":"2024-09-21T07:12:23.806243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810287+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:12:23.810287+0000","last_peered":"2024-09-21T07:12:23.810287+0000","last_clean":"2024-09-21T07:12:23.810287+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:12:23.810287+0000","last_undegraded":"2024-09-21T07:12:23.810287+0000","last_fullsized":"2024-09-21T07:12:23.810287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797647+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:12:23.797647+0000","last_peered":"2024-09-21T07:12:23.797647+0000","last_clean":"2024-09-21T07:12:23.797647+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:12:23.797647+0000","last_undegraded":"2024-09-21T07:12:23.797647+0000","last_fullsized":"2024-09-21T07:12:23.797647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.815349+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:12:23.815349+0000","last_peered":"2024-09-21T07:12:23.815349+0000","last_clean":"2024-09-21T07:12:23.815349+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:12:23.815349+0000","last_undegraded":"2024-09-21T07:12:23.815349+0000","last_fullsized":"2024-09-21T07:12:23.815349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809317+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:12:23.809317+0000","last_peered":"2024-09-21T07:12:23.809317+0000","last_clean":"2024-09-21T07:12:23.809317+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:12:23.809317+0000","last_undegraded":"2024-09-21T07:12:23.809317+0000","last_fullsized":"2024-09-21T07:12:23.809317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795846+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:12:23.795846+0000","last_peered":"2024-09-21T07:12:23.795846+0000","last_clean":"2024-09-21T07:12:23.795846+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:12:23.795846+0000","last_undegraded":"2024-09-21T07:12:23.795846+0000","last_fullsized":"2024-09-21T07:12:23.795846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808162+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:12:23.808162+0000","last_peered":"2024-09-21T07:12:23.808162+0000","last_clean":"2024-09-21T07:12:23.808162+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:12:23.808162+0000","last_undegraded":"2024-09-21T07:12:23.808162+0000","last_fullsized":"2024-09-21T07:12:23.808162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797650+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:12:23.797650+0000","last_peered":"2024-09-21T07:12:23.797650+0000","last_clean":"2024-09-21T07:12:23.797650+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:12:23.797650+0000","last_undegraded":"2024-09-21T07:12:23.797650+0000","last_fullsized":"2024-09-21T07:12:23.797650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811678+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:12:23.811678+0000","last_peered":"2024-09-21T07:12:23.811678+0000","last_clean":"2024-09-21T07:12:23.811678+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:12:23.811678+0000","last_undegraded":"2024-09-21T07:12:23.811678+0000","last_fullsized":"2024-09-21T07:12:23.811678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814831+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:12:23.814831+0000","last_peered":"2024-09-21T07:12:23.814831+0000","last_clean":"2024-09-21T07:12:23.814831+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:12:23.814831+0000","last_undegraded":"2024-09-21T07:12:23.814831+0000","last_fullsized":"2024-09-21T07:12:23.814831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798081+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:12:23.798081+0000","last_peered":"2024-09-21T07:12:23.798081+0000","last_clean":"2024-09-21T07:12:23.798081+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:12:23.798081+0000","last_undegraded":"2024-09-21T07:12:23.798081+0000","last_fullsized":"2024-09-21T07:12:23.798081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797741+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:12:23.797741+0000","last_peered":"2024-09-21T07:12:23.797741+0000","last_clean":"2024-09-21T07:12:23.797741+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:12:23.797741+0000","last_undegraded":"2024-09-21T07:12:23.797741+0000","last_fullsized":"2024-09-21T07:12:23.797741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797093+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:12:23.797093+0000","last_peered":"2024-09-21T07:12:23.797093+0000","last_clean":"2024-09-21T07:12:23.797093+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:12:23.797093+0000","last_undegraded":"2024-09-21T07:12:23.797093+0000","last_fullsized":"2024-09-21T07:12:23.797093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797350+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:12:23.797350+0000","last_peered":"2024-09-21T07:12:23.797350+0000","last_clean":"2024-09-21T07:12:23.797350+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:12:23.797350+0000","last_undegraded":"2024-09-21T07:12:23.797350+0000","last_fullsized":"2024-09-21T07:12:23.797350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814242+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:12:23.814242+0000","last_peered":"2024-09-21T07:12:23.814242+0000","last_clean":"2024-09-21T07:12:23.814242+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:12:23.814242+0000","last_undegraded":"2024-09-21T07:12:23.814242+0000","last_fullsized":"2024-09-21T07:12:23.814242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790030+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:12:23.790030+0000","last_peered":"2024-09-21T07:12:23.790030+0000","last_clean":"2024-09-21T07:12:23.790030+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:12:23.790030+0000","last_undegraded":"2024-09-21T07:12:23.790030+0000","last_fullsized":"2024-09-21T07:12:23.790030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810378+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:12:23.810378+0000","last_peered":"2024-09-21T07:12:23.810378+0000","last_clean":"2024-09-21T07:12:23.810378+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:12:23.810378+0000","last_undegraded":"2024-09-21T07:12:23.810378+0000","last_fullsized":"2024-09-21T07:12:23.810378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809256+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:12:23.809256+0000","last_peered":"2024-09-21T07:12:23.809256+0000","last_clean":"2024-09-21T07:12:23.809256+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:12:23.809256+0000","last_undegraded":"2024-09-21T07:12:23.809256+0000","last_fullsized":"2024-09-21T07:12:23.809256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806255+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:12:23.806255+0000","last_peered":"2024-09-21T07:12:23.806255+0000","last_clean":"2024-09-21T07:12:23.806255+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:12:23.806255+0000","last_undegraded":"2024-09-21T07:12:23.806255+0000","last_fullsized":"2024-09-21T07:12:23.806255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788650+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:12:23.788650+0000","last_peered":"2024-09-21T07:12:23.788650+0000","last_clean":"2024-09-21T07:12:23.788650+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:12:23.788650+0000","last_undegraded":"2024-09-21T07:12:23.788650+0000","last_fullsized":"2024-09-21T07:12:23.788650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809845+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:12:23.809845+0000","last_peered":"2024-09-21T07:12:23.809845+0000","last_clean":"2024-09-21T07:12:23.809845+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:12:23.809845+0000","last_undegraded":"2024-09-21T07:12:23.809845+0000","last_fullsized":"2024-09-21T07:12:23.809845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.789398+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:12:23.789398+0000","last_peered":"2024-09-21T07:12:23.789398+0000","last_clean":"2024-09-21T07:12:23.789398+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:12:23.789398+0000","last_undegraded":"2024-09-21T07:12:23.789398+0000","last_fullsized":"2024-09-21T07:12:23.789398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811791+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:12:23.811791+0000","last_peered":"2024-09-21T07:12:23.811791+0000","last_clean":"2024-09-21T07:12:23.811791+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:12:23.811791+0000","last_undegraded":"2024-09-21T07:12:23.811791+0000","last_fullsized":"2024-09-21T07:12:23.811791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.799236+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:12:23.799236+0000","last_peered":"2024-09-21T07:12:23.799236+0000","last_clean":"2024-09-21T07:12:23.799236+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:12:23.799236+0000","last_undegraded":"2024-09-21T07:12:23.799236+0000","last_fullsized":"2024-09-21T07:12:23.799236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811604+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:12:23.811604+0000","last_peered":"2024-09-21T07:12:23.811604+0000","last_clean":"2024-09-21T07:12:23.811604+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:12:23.811604+0000","last_undegraded":"2024-09-21T07:12:23.811604+0000","last_fullsized":"2024-09-21T07:12:23.811604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":867353,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1145,"num_read_kb":790,"num_write":413,"num_write_kb":2233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2838528,"data_stored":2602059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":515860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972407,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37456,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707696,"statfs":{"total":95995035648,"available":95956680704,"internally_reserved":0,"allocated":2101248,"data_stored":1572931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135933,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1413120,"data_stored":913386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":515972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495687,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1113137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659213,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":511250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855444,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37372,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707780,"statfs":{"total":95995035648,"available":95956766720,"internally_reserved":0,"allocated":2015232,"data_stored":1501578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215196,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37100,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708052,"statfs":{"total":95995035648,"available":95957045248,"internally_reserved":0,"allocated":1798144,"data_stored":1298436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":588338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":586286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705774,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":77178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":395954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":471403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:13:02.959 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T07:13:02.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T07:13:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:03 smithi119 ceph-mon[34519]: pgmap v378: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-09-21T07:13:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:03 smithi119 ceph-mon[34519]: mgrmap e26: x(active, since 10m), standbys: z, y 2024-09-21T07:13:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:03 smithi119 ceph-mon[34519]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:03 smithi027 ceph-mon[30779]: pgmap v378: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-09-21T07:13:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:03 smithi027 ceph-mon[30779]: mgrmap e26: x(active, since 10m), standbys: z, y 2024-09-21T07:13:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:03 smithi027 ceph-mon[30779]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:03.345 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:03 smithi153 ceph-mon[34747]: pgmap v378: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-09-21T07:13:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:03 smithi153 ceph-mon[34747]: mgrmap e26: x(active, since 10m), standbys: z, y 2024-09-21T07:13:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:03 smithi153 ceph-mon[34747]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 -- 172.21.15.27:0/2215809946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1039b0 msgr2=0x7eff3c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 --2- 172.21.15.27:0/2215809946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1039b0 0x7eff3c103dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009900 tx=0x7eff2c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 -- 172.21.15.27:0/2215809946 shutdown_connections 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 --2- 172.21.15.27:0/2215809946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c1092f0 0x7eff3c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 --2- 172.21.15.27:0/2215809946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c109c40 0x7eff3c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 --2- 172.21.15.27:0/2215809946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1039b0 0x7eff3c103dc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 -- 172.21.15.27:0/2215809946 >> 172.21.15.27:0/2215809946 conn(0x7eff3c0ff400 msgr2=0x7eff3c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 -- 172.21.15.27:0/2215809946 shutdown_connections 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.957+0000 7eff41600640 1 -- 172.21.15.27:0/2215809946 wait complete. 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.958+0000 7eff41600640 1 Processor -- start 2024-09-21T07:13:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.959+0000 7eff41600640 1 -- start start 2024-09-21T07:13:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.959+0000 7eff41600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c1039b0 0x7eff3c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.959+0000 7eff41600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1092f0 0x7eff3c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.959+0000 7eff41600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c109c40 0x7eff3c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.959+0000 7eff41600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c113900 con 0x7eff3c109c40 2024-09-21T07:13:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.959+0000 7eff41600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c113780 con 0x7eff3c1092f0 2024-09-21T07:13:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.959+0000 7eff41600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c113a80 con 0x7eff3c1039b0 2024-09-21T07:13:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c1039b0 0x7eff3c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1092f0 0x7eff3c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c1039b0 0x7eff3c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53946/0 (socket says 172.21.15.27:53946) 2024-09-21T07:13:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3be00640 1 -- 172.21.15.27:0/3418395967 learned_addr learned my addr 172.21.15.27:0/3418395967 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1092f0 0x7eff3c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47940/0 (socket says 172.21.15.27:47940) 2024-09-21T07:13:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff40c00640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c109c40 0x7eff3c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 -- 172.21.15.27:0/3418395967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c1039b0 msgr2=0x7eff3c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c1039b0 0x7eff3c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 -- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c109c40 msgr2=0x7eff3c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c109c40 0x7eff3c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 -- 172.21.15.27:0/3418395967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c1a0240 con 0x7eff3c1092f0 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.960+0000 7eff3b400640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1092f0 0x7eff3c19a870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff3c06b930 tx=0x7eff2800ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.961+0000 7eff38c00640 1 -- 172.21.15.27:0/3418395967 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2800ccc0 con 0x7eff3c1092f0 2024-09-21T07:13:03.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.961+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c1a0530 con 0x7eff3c1092f0 2024-09-21T07:13:03.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.961+0000 7eff38c00640 1 -- 172.21.15.27:0/3418395967 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff28009e40 con 0x7eff3c1092f0 2024-09-21T07:13:03.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.961+0000 7eff38c00640 1 -- 172.21.15.27:0/3418395967 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28004a00 con 0x7eff3c1092f0 2024-09-21T07:13:03.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.961+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c1a6c90 con 0x7eff3c1092f0 2024-09-21T07:13:03.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.962+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3c103dc0 con 0x7eff3c1092f0 2024-09-21T07:13:03.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.966+0000 7eff38c00640 1 -- 172.21.15.27:0/3418395967 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7eff28005020 con 0x7eff3c1092f0 2024-09-21T07:13:03.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.967+0000 7eff38c00640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff200b6710 0x7eff200b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:03.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.967+0000 7eff38c00640 1 -- 172.21.15.27:0/3418395967 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7eff280e7ca0 con 0x7eff3c1092f0 2024-09-21T07:13:03.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.967+0000 7eff3be00640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff200b6710 0x7eff200b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:03.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.968+0000 7eff3be00640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff200b6710 0x7eff200b8c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff2c031440 tx=0x7eff2c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:03.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:03.968+0000 7eff38c00640 1 -- 172.21.15.27:0/3418395967 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff280addc0 con 0x7eff3c1092f0 2024-09-21T07:13:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.166+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff3c063800 con 0x7eff200b6710 2024-09-21T07:13:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.172+0000 7eff38c00640 1 -- 172.21.15.27:0/3418395967 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340541 (secure 0 0 0) 0x7eff3c063800 con 0x7eff200b6710 2024-09-21T07:13:04.181 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:04.183 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T07:13:04.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff200b6710 msgr2=0x7eff200b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:04.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff200b6710 0x7eff200b8c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff2c031440 tx=0x7eff2c0058d0 comp rx=0 tx=0).stop 2024-09-21T07:13:04.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1092f0 msgr2=0x7eff3c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:04.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1092f0 0x7eff3c19a870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff3c06b930 tx=0x7eff2800ed90 comp rx=0 tx=0).stop 2024-09-21T07:13:04.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 shutdown_connections 2024-09-21T07:13:04.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff200b6710 0x7eff200b8c20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:04.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff3c109c40 0x7eff3c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:04.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff3c1092f0 0x7eff3c19a870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:04.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 --2- 172.21.15.27:0/3418395967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c1039b0 0x7eff3c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:04.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 >> 172.21.15.27:0/3418395967 conn(0x7eff3c0ff400 msgr2=0x7eff3c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:04.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 shutdown_connections 2024-09-21T07:13:04.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:04.177+0000 7eff41600640 1 -- 172.21.15.27:0/3418395967 wait complete. 2024-09-21T07:13:05.024 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-21T07:13:03.418873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1457955,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1336,"num_read_kb":958,"num_write":584,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":922,"ondisk_log_size":922,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398928,"kb_used_data":16160,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124542896,"statfs":{"total":1151940427776,"available":1151531925504,"internally_reserved":0,"allocated":16547840,"data_stored":10539851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":507,"num_read_kb":433,"num_write":270,"num_write_kb":2216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003656"},"pg_stats":[{"pgid":"2.1f","version":"82'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:58.089462+0000","last_change":"2024-09-21T07:09:52.322762+0000","last_active":"2024-09-21T07:11:58.089462+0000","last_peered":"2024-09-21T07:11:58.089462+0000","last_clean":"2024-09-21T07:11:58.089462+0000","last_became_active":"2024-09-21T07:09:52.322223+0000","last_became_peered":"2024-09-21T07:09:52.322223+0000","last_unstale":"2024-09-21T07:11:58.089462+0000","last_undegraded":"2024-09-21T07:11:58.089462+0000","last_fullsized":"2024-09-21T07:11:58.089462+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:55.392908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805255+0000","last_change":"2024-09-21T07:09:55.349242+0000","last_active":"2024-09-21T07:12:23.805255+0000","last_peered":"2024-09-21T07:12:23.805255+0000","last_clean":"2024-09-21T07:12:23.805255+0000","last_became_active":"2024-09-21T07:09:55.346704+0000","last_became_peered":"2024-09-21T07:09:55.346704+0000","last_unstale":"2024-09-21T07:12:23.805255+0000","last_undegraded":"2024-09-21T07:12:23.805255+0000","last_fullsized":"2024-09-21T07:12:23.805255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:55.513809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832485+0000","last_change":"2024-09-21T07:10:05.417935+0000","last_active":"2024-09-21T07:11:55.832485+0000","last_peered":"2024-09-21T07:11:55.832485+0000","last_clean":"2024-09-21T07:11:55.832485+0000","last_became_active":"2024-09-21T07:10:05.417530+0000","last_became_peered":"2024-09-21T07:10:05.417530+0000","last_unstale":"2024-09-21T07:11:55.832485+0000","last_undegraded":"2024-09-21T07:11:55.832485+0000","last_fullsized":"2024-09-21T07:11:55.832485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:54.281067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833749+0000","last_change":"2024-09-21T07:09:52.359007+0000","last_active":"2024-09-21T07:11:55.833749+0000","last_peered":"2024-09-21T07:11:55.833749+0000","last_clean":"2024-09-21T07:11:55.833749+0000","last_became_active":"2024-09-21T07:09:52.358478+0000","last_became_peered":"2024-09-21T07:09:52.358478+0000","last_unstale":"2024-09-21T07:11:55.833749+0000","last_undegraded":"2024-09-21T07:11:55.833749+0000","last_fullsized":"2024-09-21T07:11:55.833749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:11.228909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811697+0000","last_change":"2024-09-21T07:09:55.359225+0000","last_active":"2024-09-21T07:12:23.811697+0000","last_peered":"2024-09-21T07:12:23.811697+0000","last_clean":"2024-09-21T07:12:23.811697+0000","last_became_active":"2024-09-21T07:09:55.357863+0000","last_became_peered":"2024-09-21T07:09:55.357863+0000","last_unstale":"2024-09-21T07:12:23.811697+0000","last_undegraded":"2024-09-21T07:12:23.811697+0000","last_fullsized":"2024-09-21T07:12:23.811697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:56.369673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844771+0000","last_change":"2024-09-21T07:10:05.418368+0000","last_active":"2024-09-21T07:11:55.844771+0000","last_peered":"2024-09-21T07:11:55.844771+0000","last_clean":"2024-09-21T07:11:55.844771+0000","last_became_active":"2024-09-21T07:10:05.417813+0000","last_became_peered":"2024-09-21T07:10:05.417813+0000","last_unstale":"2024-09-21T07:11:55.844771+0000","last_undegraded":"2024-09-21T07:11:55.844771+0000","last_fullsized":"2024-09-21T07:11:55.844771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:52.922863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847516+0000","last_change":"2024-09-21T07:09:52.321059+0000","last_active":"2024-09-21T07:11:55.847516+0000","last_peered":"2024-09-21T07:11:55.847516+0000","last_clean":"2024-09-21T07:11:55.847516+0000","last_became_active":"2024-09-21T07:09:52.320671+0000","last_became_peered":"2024-09-21T07:09:52.320671+0000","last_unstale":"2024-09-21T07:11:55.847516+0000","last_undegraded":"2024-09-21T07:11:55.847516+0000","last_fullsized":"2024-09-21T07:11:55.847516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:23.618173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814377+0000","last_change":"2024-09-21T07:09:55.347987+0000","last_active":"2024-09-21T07:12:23.814377+0000","last_peered":"2024-09-21T07:12:23.814377+0000","last_clean":"2024-09-21T07:12:23.814377+0000","last_became_active":"2024-09-21T07:09:55.347561+0000","last_became_peered":"2024-09-21T07:09:55.347561+0000","last_unstale":"2024-09-21T07:12:23.814377+0000","last_undegraded":"2024-09-21T07:12:23.814377+0000","last_fullsized":"2024-09-21T07:12:23.814377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:47.566164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845066+0000","last_change":"2024-09-21T07:10:05.418417+0000","last_active":"2024-09-21T07:11:55.845066+0000","last_peered":"2024-09-21T07:11:55.845066+0000","last_clean":"2024-09-21T07:11:55.845066+0000","last_became_active":"2024-09-21T07:10:05.417912+0000","last_became_peered":"2024-09-21T07:10:05.417912+0000","last_unstale":"2024-09-21T07:11:55.845066+0000","last_undegraded":"2024-09-21T07:11:55.845066+0000","last_fullsized":"2024-09-21T07:11:55.845066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.812071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847357+0000","last_change":"2024-09-21T07:09:52.320310+0000","last_active":"2024-09-21T07:11:55.847357+0000","last_peered":"2024-09-21T07:11:55.847357+0000","last_clean":"2024-09-21T07:11:55.847357+0000","last_became_active":"2024-09-21T07:09:52.319933+0000","last_became_peered":"2024-09-21T07:09:52.319933+0000","last_unstale":"2024-09-21T07:11:55.847357+0000","last_undegraded":"2024-09-21T07:11:55.847357+0000","last_fullsized":"2024-09-21T07:11:55.847357+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:47.710829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803507+0000","last_change":"2024-09-21T07:09:55.349244+0000","last_active":"2024-09-21T07:12:23.803507+0000","last_peered":"2024-09-21T07:12:23.803507+0000","last_clean":"2024-09-21T07:12:23.803507+0000","last_became_active":"2024-09-21T07:09:55.348850+0000","last_became_peered":"2024-09-21T07:09:55.348850+0000","last_unstale":"2024-09-21T07:12:23.803507+0000","last_undegraded":"2024-09-21T07:12:23.803507+0000","last_fullsized":"2024-09-21T07:12:23.803507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:20.672763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.844870+0000","last_change":"2024-09-21T07:10:05.403499+0000","last_active":"2024-09-21T07:11:55.844870+0000","last_peered":"2024-09-21T07:11:55.844870+0000","last_clean":"2024-09-21T07:11:55.844870+0000","last_became_active":"2024-09-21T07:10:05.402817+0000","last_became_peered":"2024-09-21T07:10:05.402817+0000","last_unstale":"2024-09-21T07:11:55.844870+0000","last_undegraded":"2024-09-21T07:11:55.844870+0000","last_fullsized":"2024-09-21T07:11:55.844870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:39.004525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813396+0000","last_change":"2024-09-21T07:09:55.353799+0000","last_active":"2024-09-21T07:12:23.813396+0000","last_peered":"2024-09-21T07:12:23.813396+0000","last_clean":"2024-09-21T07:12:23.813396+0000","last_became_active":"2024-09-21T07:09:55.353289+0000","last_became_peered":"2024-09-21T07:09:55.353289+0000","last_unstale":"2024-09-21T07:12:23.813396+0000","last_undegraded":"2024-09-21T07:12:23.813396+0000","last_fullsized":"2024-09-21T07:12:23.813396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:48.005660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846961+0000","last_change":"2024-09-21T07:09:52.322260+0000","last_active":"2024-09-21T07:11:55.846961+0000","last_peered":"2024-09-21T07:11:55.846961+0000","last_clean":"2024-09-21T07:11:55.846961+0000","last_became_active":"2024-09-21T07:09:52.321828+0000","last_became_peered":"2024-09-21T07:09:52.321828+0000","last_unstale":"2024-09-21T07:11:55.846961+0000","last_undegraded":"2024-09-21T07:11:55.846961+0000","last_fullsized":"2024-09-21T07:11:55.846961+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:35.312221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845137+0000","last_change":"2024-09-21T07:10:05.614396+0000","last_active":"2024-09-21T07:11:55.845137+0000","last_peered":"2024-09-21T07:11:55.845137+0000","last_clean":"2024-09-21T07:11:55.845137+0000","last_became_active":"2024-09-21T07:10:05.613871+0000","last_became_peered":"2024-09-21T07:10:05.613871+0000","last_unstale":"2024-09-21T07:11:55.845137+0000","last_undegraded":"2024-09-21T07:11:55.845137+0000","last_fullsized":"2024-09-21T07:11:55.845137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:57.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807789+0000","last_change":"2024-09-21T07:09:55.359442+0000","last_active":"2024-09-21T07:12:23.807789+0000","last_peered":"2024-09-21T07:12:23.807789+0000","last_clean":"2024-09-21T07:12:23.807789+0000","last_became_active":"2024-09-21T07:09:55.358830+0000","last_became_peered":"2024-09-21T07:09:55.358830+0000","last_unstale":"2024-09-21T07:12:23.807789+0000","last_undegraded":"2024-09-21T07:12:23.807789+0000","last_fullsized":"2024-09-21T07:12:23.807789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:25.603652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832340+0000","last_change":"2024-09-21T07:10:54.866491+0000","last_active":"2024-09-21T07:11:55.832340+0000","last_peered":"2024-09-21T07:11:55.832340+0000","last_clean":"2024-09-21T07:11:55.832340+0000","last_became_active":"2024-09-21T07:10:54.858731+0000","last_became_peered":"2024-09-21T07:10:54.858731+0000","last_unstale":"2024-09-21T07:11:55.832340+0000","last_undegraded":"2024-09-21T07:11:55.832340+0000","last_fullsized":"2024-09-21T07:11:55.832340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:27.766739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845070+0000","last_change":"2024-09-21T07:10:05.412177+0000","last_active":"2024-09-21T07:11:55.845070+0000","last_peered":"2024-09-21T07:11:55.845070+0000","last_clean":"2024-09-21T07:11:55.845070+0000","last_became_active":"2024-09-21T07:10:05.411669+0000","last_became_peered":"2024-09-21T07:10:05.411669+0000","last_unstale":"2024-09-21T07:11:55.845070+0000","last_undegraded":"2024-09-21T07:11:55.845070+0000","last_fullsized":"2024-09-21T07:11:55.845070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:19.801965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798453+0000","last_change":"2024-09-21T07:09:55.356390+0000","last_active":"2024-09-21T07:12:23.798453+0000","last_peered":"2024-09-21T07:12:23.798453+0000","last_clean":"2024-09-21T07:12:23.798453+0000","last_became_active":"2024-09-21T07:09:55.355970+0000","last_became_peered":"2024-09-21T07:09:55.355970+0000","last_unstale":"2024-09-21T07:12:23.798453+0000","last_undegraded":"2024-09-21T07:12:23.798453+0000","last_fullsized":"2024-09-21T07:12:23.798453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:03.512221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833325+0000","last_change":"2024-09-21T07:09:52.319990+0000","last_active":"2024-09-21T07:11:55.833325+0000","last_peered":"2024-09-21T07:11:55.833325+0000","last_clean":"2024-09-21T07:11:55.833325+0000","last_became_active":"2024-09-21T07:09:52.318931+0000","last_became_peered":"2024-09-21T07:09:52.318931+0000","last_unstale":"2024-09-21T07:11:55.833325+0000","last_undegraded":"2024-09-21T07:11:55.833325+0000","last_fullsized":"2024-09-21T07:11:55.833325+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:15.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830920+0000","last_change":"2024-09-21T07:10:05.406383+0000","last_active":"2024-09-21T07:11:55.830920+0000","last_peered":"2024-09-21T07:11:55.830920+0000","last_clean":"2024-09-21T07:11:55.830920+0000","last_became_active":"2024-09-21T07:10:05.405869+0000","last_became_peered":"2024-09-21T07:10:05.405869+0000","last_unstale":"2024-09-21T07:11:55.830920+0000","last_undegraded":"2024-09-21T07:11:55.830920+0000","last_fullsized":"2024-09-21T07:11:55.830920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:18.446302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.780871+0000","last_change":"2024-09-21T07:09:55.342374+0000","last_active":"2024-09-21T07:12:23.780871+0000","last_peered":"2024-09-21T07:12:23.780871+0000","last_clean":"2024-09-21T07:12:23.780871+0000","last_became_active":"2024-09-21T07:09:55.342000+0000","last_became_peered":"2024-09-21T07:09:55.342000+0000","last_unstale":"2024-09-21T07:12:23.780871+0000","last_undegraded":"2024-09-21T07:12:23.780871+0000","last_fullsized":"2024-09-21T07:12:23.780871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:07.560593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833051+0000","last_change":"2024-09-21T07:09:52.358685+0000","last_active":"2024-09-21T07:11:55.833051+0000","last_peered":"2024-09-21T07:11:55.833051+0000","last_clean":"2024-09-21T07:11:55.833051+0000","last_became_active":"2024-09-21T07:09:52.358275+0000","last_became_peered":"2024-09-21T07:09:52.358275+0000","last_unstale":"2024-09-21T07:11:55.833051+0000","last_undegraded":"2024-09-21T07:11:55.833051+0000","last_fullsized":"2024-09-21T07:11:55.833051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:04.154157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845144+0000","last_change":"2024-09-21T07:10:05.615701+0000","last_active":"2024-09-21T07:11:55.845144+0000","last_peered":"2024-09-21T07:11:55.845144+0000","last_clean":"2024-09-21T07:11:55.845144+0000","last_became_active":"2024-09-21T07:10:05.615269+0000","last_became_peered":"2024-09-21T07:10:05.615269+0000","last_unstale":"2024-09-21T07:11:55.845144+0000","last_undegraded":"2024-09-21T07:11:55.845144+0000","last_fullsized":"2024-09-21T07:11:55.845144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.072961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.800049+0000","last_change":"2024-09-21T07:09:55.350513+0000","last_active":"2024-09-21T07:12:23.800049+0000","last_peered":"2024-09-21T07:12:23.800049+0000","last_clean":"2024-09-21T07:12:23.800049+0000","last_became_active":"2024-09-21T07:09:55.349432+0000","last_became_peered":"2024-09-21T07:09:55.349432+0000","last_unstale":"2024-09-21T07:12:23.800049+0000","last_undegraded":"2024-09-21T07:12:23.800049+0000","last_fullsized":"2024-09-21T07:12:23.800049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:29.851634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845369+0000","last_change":"2024-09-21T07:09:52.318116+0000","last_active":"2024-09-21T07:11:55.845369+0000","last_peered":"2024-09-21T07:11:55.845369+0000","last_clean":"2024-09-21T07:11:55.845369+0000","last_became_active":"2024-09-21T07:09:52.317747+0000","last_became_peered":"2024-09-21T07:09:52.317747+0000","last_unstale":"2024-09-21T07:11:55.845369+0000","last_undegraded":"2024-09-21T07:11:55.845369+0000","last_fullsized":"2024-09-21T07:11:55.845369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:20.357687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.404709+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.404210+0000","last_became_peered":"2024-09-21T07:10:05.404210+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:24.735207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808689+0000","last_change":"2024-09-21T07:09:55.343813+0000","last_active":"2024-09-21T07:12:23.808689+0000","last_peered":"2024-09-21T07:12:23.808689+0000","last_clean":"2024-09-21T07:12:23.808689+0000","last_became_active":"2024-09-21T07:09:55.343392+0000","last_became_peered":"2024-09-21T07:09:55.343392+0000","last_unstale":"2024-09-21T07:12:23.808689+0000","last_undegraded":"2024-09-21T07:12:23.808689+0000","last_fullsized":"2024-09-21T07:12:23.808689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:10.935468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'121","reported_seq":289,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:13:01.040148+0000","last_change":"2024-09-21T07:09:52.318805+0000","last_active":"2024-09-21T07:13:01.040148+0000","last_peered":"2024-09-21T07:13:01.040148+0000","last_clean":"2024-09-21T07:13:01.040148+0000","last_became_active":"2024-09-21T07:09:52.318097+0000","last_became_peered":"2024-09-21T07:09:52.318097+0000","last_unstale":"2024-09-21T07:13:01.040148+0000","last_undegraded":"2024-09-21T07:13:01.040148+0000","last_fullsized":"2024-09-21T07:13:01.040148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:40.615213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":412,"num_read_kb":367,"num_write":225,"num_write_kb":1350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832993+0000","last_change":"2024-09-21T07:10:05.615712+0000","last_active":"2024-09-21T07:11:55.832993+0000","last_peered":"2024-09-21T07:11:55.832993+0000","last_clean":"2024-09-21T07:11:55.832993+0000","last_became_active":"2024-09-21T07:10:05.613908+0000","last_became_peered":"2024-09-21T07:10:05.613908+0000","last_unstale":"2024-09-21T07:11:55.832993+0000","last_undegraded":"2024-09-21T07:11:55.832993+0000","last_fullsized":"2024-09-21T07:11:55.832993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:06.143319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809315+0000","last_change":"2024-09-21T07:09:55.351819+0000","last_active":"2024-09-21T07:12:23.809315+0000","last_peered":"2024-09-21T07:12:23.809315+0000","last_clean":"2024-09-21T07:12:23.809315+0000","last_became_active":"2024-09-21T07:09:55.351077+0000","last_became_peered":"2024-09-21T07:09:55.351077+0000","last_unstale":"2024-09-21T07:12:23.809315+0000","last_undegraded":"2024-09-21T07:12:23.809315+0000","last_fullsized":"2024-09-21T07:12:23.809315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:00.115772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847542+0000","last_change":"2024-09-21T07:09:52.361458+0000","last_active":"2024-09-21T07:11:55.847542+0000","last_peered":"2024-09-21T07:11:55.847542+0000","last_clean":"2024-09-21T07:11:55.847542+0000","last_became_active":"2024-09-21T07:09:52.360994+0000","last_became_peered":"2024-09-21T07:09:52.360994+0000","last_unstale":"2024-09-21T07:11:55.847542+0000","last_undegraded":"2024-09-21T07:11:55.847542+0000","last_fullsized":"2024-09-21T07:11:55.847542+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:07.547795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832686+0000","last_change":"2024-09-21T07:10:05.421189+0000","last_active":"2024-09-21T07:11:55.832686+0000","last_peered":"2024-09-21T07:11:55.832686+0000","last_clean":"2024-09-21T07:11:55.832686+0000","last_became_active":"2024-09-21T07:10:05.420589+0000","last_became_peered":"2024-09-21T07:10:05.420589+0000","last_unstale":"2024-09-21T07:11:55.832686+0000","last_undegraded":"2024-09-21T07:11:55.832686+0000","last_fullsized":"2024-09-21T07:11:55.832686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:08.777049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790825+0000","last_change":"2024-09-21T07:09:55.354649+0000","last_active":"2024-09-21T07:12:23.790825+0000","last_peered":"2024-09-21T07:12:23.790825+0000","last_clean":"2024-09-21T07:12:23.790825+0000","last_became_active":"2024-09-21T07:09:55.354258+0000","last_became_peered":"2024-09-21T07:09:55.354258+0000","last_unstale":"2024-09-21T07:12:23.790825+0000","last_undegraded":"2024-09-21T07:12:23.790825+0000","last_fullsized":"2024-09-21T07:12:23.790825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:39.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830831+0000","last_change":"2024-09-21T07:09:52.359161+0000","last_active":"2024-09-21T07:11:55.830831+0000","last_peered":"2024-09-21T07:11:55.830831+0000","last_clean":"2024-09-21T07:11:55.830831+0000","last_became_active":"2024-09-21T07:09:52.358731+0000","last_became_peered":"2024-09-21T07:09:52.358731+0000","last_unstale":"2024-09-21T07:11:55.830831+0000","last_undegraded":"2024-09-21T07:11:55.830831+0000","last_fullsized":"2024-09-21T07:11:55.830831+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:16.338733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846866+0000","last_change":"2024-09-21T07:10:05.614597+0000","last_active":"2024-09-21T07:11:55.846866+0000","last_peered":"2024-09-21T07:11:55.846866+0000","last_clean":"2024-09-21T07:11:55.846866+0000","last_became_active":"2024-09-21T07:10:05.613912+0000","last_became_peered":"2024-09-21T07:10:05.613912+0000","last_unstale":"2024-09-21T07:11:55.846866+0000","last_undegraded":"2024-09-21T07:11:55.846866+0000","last_fullsized":"2024-09-21T07:11:55.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:53.248478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.794745+0000","last_change":"2024-09-21T07:09:55.355832+0000","last_active":"2024-09-21T07:12:23.794745+0000","last_peered":"2024-09-21T07:12:23.794745+0000","last_clean":"2024-09-21T07:12:23.794745+0000","last_became_active":"2024-09-21T07:09:55.355430+0000","last_became_peered":"2024-09-21T07:09:55.355430+0000","last_unstale":"2024-09-21T07:12:23.794745+0000","last_undegraded":"2024-09-21T07:12:23.794745+0000","last_fullsized":"2024-09-21T07:12:23.794745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:42.600806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981709+0000","last_change":"2024-09-21T07:09:52.322572+0000","last_active":"2024-09-21T07:11:55.981709+0000","last_peered":"2024-09-21T07:11:55.981709+0000","last_clean":"2024-09-21T07:11:55.981709+0000","last_became_active":"2024-09-21T07:09:52.322193+0000","last_became_peered":"2024-09-21T07:09:52.322193+0000","last_unstale":"2024-09-21T07:11:55.981709+0000","last_undegraded":"2024-09-21T07:11:55.981709+0000","last_fullsized":"2024-09-21T07:11:55.981709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:02.105090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845966+0000","last_change":"2024-09-21T07:10:05.422644+0000","last_active":"2024-09-21T07:11:55.845966+0000","last_peered":"2024-09-21T07:11:55.845966+0000","last_clean":"2024-09-21T07:11:55.845966+0000","last_became_active":"2024-09-21T07:10:05.409245+0000","last_became_peered":"2024-09-21T07:10:05.409245+0000","last_unstale":"2024-09-21T07:11:55.845966+0000","last_undegraded":"2024-09-21T07:11:55.845966+0000","last_fullsized":"2024-09-21T07:11:55.845966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:47.370803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795850+0000","last_change":"2024-09-21T07:09:55.342650+0000","last_active":"2024-09-21T07:12:23.795850+0000","last_peered":"2024-09-21T07:12:23.795850+0000","last_clean":"2024-09-21T07:12:23.795850+0000","last_became_active":"2024-09-21T07:09:55.342266+0000","last_became_peered":"2024-09-21T07:09:55.342266+0000","last_unstale":"2024-09-21T07:12:23.795850+0000","last_undegraded":"2024-09-21T07:12:23.795850+0000","last_fullsized":"2024-09-21T07:12:23.795850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:10.737326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":182,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:59.771521+0000","last_change":"2024-09-21T07:09:52.361116+0000","last_active":"2024-09-21T07:12:59.771521+0000","last_peered":"2024-09-21T07:12:59.771521+0000","last_clean":"2024-09-21T07:12:59.771521+0000","last_became_active":"2024-09-21T07:09:52.360105+0000","last_became_peered":"2024-09-21T07:09:52.360105+0000","last_unstale":"2024-09-21T07:12:59.771521+0000","last_undegraded":"2024-09-21T07:12:59.771521+0000","last_fullsized":"2024-09-21T07:12:59.771521+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:23.621374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846521+0000","last_change":"2024-09-21T07:10:05.614704+0000","last_active":"2024-09-21T07:11:55.846521+0000","last_peered":"2024-09-21T07:11:55.846521+0000","last_clean":"2024-09-21T07:11:55.846521+0000","last_became_active":"2024-09-21T07:10:05.614170+0000","last_became_peered":"2024-09-21T07:10:05.614170+0000","last_unstale":"2024-09-21T07:11:55.846521+0000","last_undegraded":"2024-09-21T07:11:55.846521+0000","last_fullsized":"2024-09-21T07:11:55.846521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:54.080373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807529+0000","last_change":"2024-09-21T07:09:55.357497+0000","last_active":"2024-09-21T07:12:23.807529+0000","last_peered":"2024-09-21T07:12:23.807529+0000","last_clean":"2024-09-21T07:12:23.807529+0000","last_became_active":"2024-09-21T07:09:55.355352+0000","last_became_peered":"2024-09-21T07:09:55.355352+0000","last_unstale":"2024-09-21T07:12:23.807529+0000","last_undegraded":"2024-09-21T07:12:23.807529+0000","last_fullsized":"2024-09-21T07:12:23.807529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:55.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981500+0000","last_change":"2024-09-21T07:09:52.322989+0000","last_active":"2024-09-21T07:11:55.981500+0000","last_peered":"2024-09-21T07:11:55.981500+0000","last_clean":"2024-09-21T07:11:55.981500+0000","last_became_active":"2024-09-21T07:09:52.322044+0000","last_became_peered":"2024-09-21T07:09:52.322044+0000","last_unstale":"2024-09-21T07:11:55.981500+0000","last_undegraded":"2024-09-21T07:11:55.981500+0000","last_fullsized":"2024-09-21T07:11:55.981500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:51.361585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845592+0000","last_change":"2024-09-21T07:10:05.615276+0000","last_active":"2024-09-21T07:11:55.845592+0000","last_peered":"2024-09-21T07:11:55.845592+0000","last_clean":"2024-09-21T07:11:55.845592+0000","last_became_active":"2024-09-21T07:10:05.614679+0000","last_became_peered":"2024-09-21T07:10:05.614679+0000","last_unstale":"2024-09-21T07:11:55.845592+0000","last_undegraded":"2024-09-21T07:11:55.845592+0000","last_fullsized":"2024-09-21T07:11:55.845592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:09.664726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811077+0000","last_change":"2024-09-21T07:09:55.357275+0000","last_active":"2024-09-21T07:12:23.811077+0000","last_peered":"2024-09-21T07:12:23.811077+0000","last_clean":"2024-09-21T07:12:23.811077+0000","last_became_active":"2024-09-21T07:09:55.352677+0000","last_became_peered":"2024-09-21T07:09:55.352677+0000","last_unstale":"2024-09-21T07:12:23.811077+0000","last_undegraded":"2024-09-21T07:12:23.811077+0000","last_fullsized":"2024-09-21T07:12:23.811077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:56.628693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845392+0000","last_change":"2024-09-21T07:09:52.316994+0000","last_active":"2024-09-21T07:11:55.845392+0000","last_peered":"2024-09-21T07:11:55.845392+0000","last_clean":"2024-09-21T07:11:55.845392+0000","last_became_active":"2024-09-21T07:09:52.315930+0000","last_became_peered":"2024-09-21T07:09:52.315930+0000","last_unstale":"2024-09-21T07:11:55.845392+0000","last_undegraded":"2024-09-21T07:11:55.845392+0000","last_fullsized":"2024-09-21T07:11:55.845392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.537914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981606+0000","last_change":"2024-09-21T07:08:40.214297+0000","last_active":"2024-09-21T07:11:55.981606+0000","last_peered":"2024-09-21T07:11:55.981606+0000","last_clean":"2024-09-21T07:11:55.981606+0000","last_became_active":"2024-09-21T07:08:40.208927+0000","last_became_peered":"2024-09-21T07:08:40.208927+0000","last_unstale":"2024-09-21T07:11:55.981606+0000","last_undegraded":"2024-09-21T07:11:55.981606+0000","last_fullsized":"2024-09-21T07:11:55.981606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:05:00.007582+0000","last_clean_scrub_stamp":"2024-09-21T07:05:00.007582+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:43.676479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981807+0000","last_change":"2024-09-21T07:10:05.418418+0000","last_active":"2024-09-21T07:11:55.981807+0000","last_peered":"2024-09-21T07:11:55.981807+0000","last_clean":"2024-09-21T07:11:55.981807+0000","last_became_active":"2024-09-21T07:10:05.417957+0000","last_became_peered":"2024-09-21T07:10:05.417957+0000","last_unstale":"2024-09-21T07:11:55.981807+0000","last_undegraded":"2024-09-21T07:11:55.981807+0000","last_fullsized":"2024-09-21T07:11:55.981807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:50.869281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814663+0000","last_change":"2024-09-21T07:09:55.348269+0000","last_active":"2024-09-21T07:12:23.814663+0000","last_peered":"2024-09-21T07:12:23.814663+0000","last_clean":"2024-09-21T07:12:23.814663+0000","last_became_active":"2024-09-21T07:09:55.347858+0000","last_became_peered":"2024-09-21T07:09:55.347858+0000","last_unstale":"2024-09-21T07:12:23.814663+0000","last_undegraded":"2024-09-21T07:12:23.814663+0000","last_fullsized":"2024-09-21T07:12:23.814663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:19.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.834191+0000","last_change":"2024-09-21T07:09:52.361738+0000","last_active":"2024-09-21T07:11:55.834191+0000","last_peered":"2024-09-21T07:11:55.834191+0000","last_clean":"2024-09-21T07:11:55.834191+0000","last_became_active":"2024-09-21T07:09:52.361343+0000","last_became_peered":"2024-09-21T07:09:52.361343+0000","last_unstale":"2024-09-21T07:11:55.834191+0000","last_undegraded":"2024-09-21T07:11:55.834191+0000","last_fullsized":"2024-09-21T07:11:55.834191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.864574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833166+0000","last_change":"2024-09-21T07:10:05.402410+0000","last_active":"2024-09-21T07:11:55.833166+0000","last_peered":"2024-09-21T07:11:55.833166+0000","last_clean":"2024-09-21T07:11:55.833166+0000","last_became_active":"2024-09-21T07:10:05.401689+0000","last_became_peered":"2024-09-21T07:10:05.401689+0000","last_unstale":"2024-09-21T07:11:55.833166+0000","last_undegraded":"2024-09-21T07:11:55.833166+0000","last_fullsized":"2024-09-21T07:11:55.833166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:41.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807409+0000","last_change":"2024-09-21T07:09:55.342655+0000","last_active":"2024-09-21T07:12:23.807409+0000","last_peered":"2024-09-21T07:12:23.807409+0000","last_clean":"2024-09-21T07:12:23.807409+0000","last_became_active":"2024-09-21T07:09:55.342285+0000","last_became_peered":"2024-09-21T07:09:55.342285+0000","last_unstale":"2024-09-21T07:12:23.807409+0000","last_undegraded":"2024-09-21T07:12:23.807409+0000","last_fullsized":"2024-09-21T07:12:23.807409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.794971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'60","reported_seq":135,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:57.447818+0000","last_change":"2024-09-21T07:09:52.318525+0000","last_active":"2024-09-21T07:12:57.447818+0000","last_peered":"2024-09-21T07:12:57.447818+0000","last_clean":"2024-09-21T07:12:57.447818+0000","last_became_active":"2024-09-21T07:09:52.318056+0000","last_became_peered":"2024-09-21T07:09:52.318056+0000","last_unstale":"2024-09-21T07:12:57.447818+0000","last_undegraded":"2024-09-21T07:12:57.447818+0000","last_fullsized":"2024-09-21T07:12:57.447818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:36.701284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846714+0000","last_change":"2024-09-21T07:10:05.420851+0000","last_active":"2024-09-21T07:11:55.846714+0000","last_peered":"2024-09-21T07:11:55.846714+0000","last_clean":"2024-09-21T07:11:55.846714+0000","last_became_active":"2024-09-21T07:10:05.420124+0000","last_became_peered":"2024-09-21T07:10:05.420124+0000","last_unstale":"2024-09-21T07:11:55.846714+0000","last_undegraded":"2024-09-21T07:11:55.846714+0000","last_fullsized":"2024-09-21T07:11:55.846714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:33.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814102+0000","last_change":"2024-09-21T07:09:55.344129+0000","last_active":"2024-09-21T07:12:23.814102+0000","last_peered":"2024-09-21T07:12:23.814102+0000","last_clean":"2024-09-21T07:12:23.814102+0000","last_became_active":"2024-09-21T07:09:55.343739+0000","last_became_peered":"2024-09-21T07:09:55.343739+0000","last_unstale":"2024-09-21T07:12:23.814102+0000","last_undegraded":"2024-09-21T07:12:23.814102+0000","last_fullsized":"2024-09-21T07:12:23.814102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:32.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846385+0000","last_change":"2024-09-21T07:09:52.320116+0000","last_active":"2024-09-21T07:11:55.846385+0000","last_peered":"2024-09-21T07:11:55.846385+0000","last_clean":"2024-09-21T07:11:55.846385+0000","last_became_active":"2024-09-21T07:09:52.319653+0000","last_became_peered":"2024-09-21T07:09:52.319653+0000","last_unstale":"2024-09-21T07:11:55.846385+0000","last_undegraded":"2024-09-21T07:11:55.846385+0000","last_fullsized":"2024-09-21T07:11:55.846385+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:07.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845182+0000","last_change":"2024-09-21T07:10:05.404620+0000","last_active":"2024-09-21T07:11:55.845182+0000","last_peered":"2024-09-21T07:11:55.845182+0000","last_clean":"2024-09-21T07:11:55.845182+0000","last_became_active":"2024-09-21T07:10:05.404072+0000","last_became_peered":"2024-09-21T07:10:05.404072+0000","last_unstale":"2024-09-21T07:11:55.845182+0000","last_undegraded":"2024-09-21T07:11:55.845182+0000","last_fullsized":"2024-09-21T07:11:55.845182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:10.707318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801647+0000","last_change":"2024-09-21T07:10:54.856655+0000","last_active":"2024-09-21T07:12:23.801647+0000","last_peered":"2024-09-21T07:12:23.801647+0000","last_clean":"2024-09-21T07:12:23.801647+0000","last_became_active":"2024-09-21T07:10:54.856220+0000","last_became_peered":"2024-09-21T07:10:54.856220+0000","last_unstale":"2024-09-21T07:12:23.801647+0000","last_undegraded":"2024-09-21T07:12:23.801647+0000","last_fullsized":"2024-09-21T07:12:23.801647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:06.366118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832950+0000","last_change":"2024-09-21T07:10:54.863735+0000","last_active":"2024-09-21T07:11:55.832950+0000","last_peered":"2024-09-21T07:11:55.832950+0000","last_clean":"2024-09-21T07:11:55.832950+0000","last_became_active":"2024-09-21T07:10:54.856362+0000","last_became_peered":"2024-09-21T07:10:54.856362+0000","last_unstale":"2024-09-21T07:11:55.832950+0000","last_undegraded":"2024-09-21T07:11:55.832950+0000","last_fullsized":"2024-09-21T07:11:55.832950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:31.004053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845860+0000","last_change":"2024-09-21T07:10:05.416940+0000","last_active":"2024-09-21T07:11:55.845860+0000","last_peered":"2024-09-21T07:11:55.845860+0000","last_clean":"2024-09-21T07:11:55.845860+0000","last_became_active":"2024-09-21T07:10:05.416519+0000","last_became_peered":"2024-09-21T07:10:05.416519+0000","last_unstale":"2024-09-21T07:11:55.845860+0000","last_undegraded":"2024-09-21T07:11:55.845860+0000","last_fullsized":"2024-09-21T07:11:55.845860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:08.082982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802831+0000","last_change":"2024-09-21T07:10:54.861606+0000","last_active":"2024-09-21T07:12:23.802831+0000","last_peered":"2024-09-21T07:12:23.802831+0000","last_clean":"2024-09-21T07:12:23.802831+0000","last_became_active":"2024-09-21T07:10:54.861225+0000","last_became_peered":"2024-09-21T07:10:54.861225+0000","last_unstale":"2024-09-21T07:12:23.802831+0000","last_undegraded":"2024-09-21T07:12:23.802831+0000","last_fullsized":"2024-09-21T07:12:23.802831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:41.135665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846785+0000","last_change":"2024-09-21T07:09:52.360540+0000","last_active":"2024-09-21T07:11:55.846785+0000","last_peered":"2024-09-21T07:11:55.846785+0000","last_clean":"2024-09-21T07:11:55.846785+0000","last_became_active":"2024-09-21T07:09:52.360060+0000","last_became_peered":"2024-09-21T07:09:52.360060+0000","last_unstale":"2024-09-21T07:11:55.846785+0000","last_undegraded":"2024-09-21T07:11:55.846785+0000","last_fullsized":"2024-09-21T07:11:55.846785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:29.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846243+0000","last_change":"2024-09-21T07:10:05.420930+0000","last_active":"2024-09-21T07:11:55.846243+0000","last_peered":"2024-09-21T07:11:55.846243+0000","last_clean":"2024-09-21T07:11:55.846243+0000","last_became_active":"2024-09-21T07:10:05.420527+0000","last_became_peered":"2024-09-21T07:10:05.420527+0000","last_unstale":"2024-09-21T07:11:55.846243+0000","last_undegraded":"2024-09-21T07:11:55.846243+0000","last_fullsized":"2024-09-21T07:11:55.846243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:33.941254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805668+0000","last_change":"2024-09-21T07:09:55.341455+0000","last_active":"2024-09-21T07:12:23.805668+0000","last_peered":"2024-09-21T07:12:23.805668+0000","last_clean":"2024-09-21T07:12:23.805668+0000","last_became_active":"2024-09-21T07:09:55.341049+0000","last_became_peered":"2024-09-21T07:09:55.341049+0000","last_unstale":"2024-09-21T07:12:23.805668+0000","last_undegraded":"2024-09-21T07:12:23.805668+0000","last_fullsized":"2024-09-21T07:12:23.805668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:00.430814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":186,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:58.770752+0000","last_change":"2024-09-21T07:09:52.317101+0000","last_active":"2024-09-21T07:12:58.770752+0000","last_peered":"2024-09-21T07:12:58.770752+0000","last_clean":"2024-09-21T07:12:58.770752+0000","last_became_active":"2024-09-21T07:09:52.315983+0000","last_became_peered":"2024-09-21T07:09:52.315983+0000","last_unstale":"2024-09-21T07:12:58.770752+0000","last_undegraded":"2024-09-21T07:12:58.770752+0000","last_fullsized":"2024-09-21T07:12:58.770752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:54.867005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830985+0000","last_change":"2024-09-21T07:10:05.411969+0000","last_active":"2024-09-21T07:11:55.830985+0000","last_peered":"2024-09-21T07:11:55.830985+0000","last_clean":"2024-09-21T07:11:55.830985+0000","last_became_active":"2024-09-21T07:10:05.411544+0000","last_became_peered":"2024-09-21T07:10:05.411544+0000","last_unstale":"2024-09-21T07:11:55.830985+0000","last_undegraded":"2024-09-21T07:11:55.830985+0000","last_fullsized":"2024-09-21T07:11:55.830985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:30.846408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813592+0000","last_change":"2024-09-21T07:09:55.351530+0000","last_active":"2024-09-21T07:12:23.813592+0000","last_peered":"2024-09-21T07:12:23.813592+0000","last_clean":"2024-09-21T07:12:23.813592+0000","last_became_active":"2024-09-21T07:09:55.351198+0000","last_became_peered":"2024-09-21T07:09:55.351198+0000","last_unstale":"2024-09-21T07:12:23.813592+0000","last_undegraded":"2024-09-21T07:12:23.813592+0000","last_fullsized":"2024-09-21T07:12:23.813592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:59.458166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846256+0000","last_change":"2024-09-21T07:09:52.322654+0000","last_active":"2024-09-21T07:11:55.846256+0000","last_peered":"2024-09-21T07:11:55.846256+0000","last_clean":"2024-09-21T07:11:55.846256+0000","last_became_active":"2024-09-21T07:09:52.322251+0000","last_became_peered":"2024-09-21T07:09:52.322251+0000","last_unstale":"2024-09-21T07:11:55.846256+0000","last_undegraded":"2024-09-21T07:11:55.846256+0000","last_fullsized":"2024-09-21T07:11:55.846256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:50.224589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.832414+0000","last_change":"2024-09-21T07:10:05.424754+0000","last_active":"2024-09-21T07:11:55.832414+0000","last_peered":"2024-09-21T07:11:55.832414+0000","last_clean":"2024-09-21T07:11:55.832414+0000","last_became_active":"2024-09-21T07:10:05.424203+0000","last_became_peered":"2024-09-21T07:10:05.424203+0000","last_unstale":"2024-09-21T07:11:55.832414+0000","last_undegraded":"2024-09-21T07:11:55.832414+0000","last_fullsized":"2024-09-21T07:11:55.832414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:39.309423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808494+0000","last_change":"2024-09-21T07:09:55.348968+0000","last_active":"2024-09-21T07:12:23.808494+0000","last_peered":"2024-09-21T07:12:23.808494+0000","last_clean":"2024-09-21T07:12:23.808494+0000","last_became_active":"2024-09-21T07:09:55.348498+0000","last_became_peered":"2024-09-21T07:09:55.348498+0000","last_unstale":"2024-09-21T07:12:23.808494+0000","last_undegraded":"2024-09-21T07:12:23.808494+0000","last_fullsized":"2024-09-21T07:12:23.808494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:51.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833515+0000","last_change":"2024-09-21T07:09:52.322466+0000","last_active":"2024-09-21T07:11:55.833515+0000","last_peered":"2024-09-21T07:11:55.833515+0000","last_clean":"2024-09-21T07:11:55.833515+0000","last_became_active":"2024-09-21T07:09:52.321204+0000","last_became_peered":"2024-09-21T07:09:52.321204+0000","last_unstale":"2024-09-21T07:11:55.833515+0000","last_undegraded":"2024-09-21T07:11:55.833515+0000","last_fullsized":"2024-09-21T07:11:55.833515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:49.466733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846046+0000","last_change":"2024-09-21T07:10:05.425103+0000","last_active":"2024-09-21T07:11:55.846046+0000","last_peered":"2024-09-21T07:11:55.846046+0000","last_clean":"2024-09-21T07:11:55.846046+0000","last_became_active":"2024-09-21T07:10:05.424592+0000","last_became_peered":"2024-09-21T07:10:05.424592+0000","last_unstale":"2024-09-21T07:11:55.846046+0000","last_undegraded":"2024-09-21T07:11:55.846046+0000","last_fullsized":"2024-09-21T07:11:55.846046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:16.358544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798135+0000","last_change":"2024-09-21T07:09:55.342676+0000","last_active":"2024-09-21T07:12:23.798135+0000","last_peered":"2024-09-21T07:12:23.798135+0000","last_clean":"2024-09-21T07:12:23.798135+0000","last_became_active":"2024-09-21T07:09:55.342022+0000","last_became_peered":"2024-09-21T07:09:55.342022+0000","last_unstale":"2024-09-21T07:12:23.798135+0000","last_undegraded":"2024-09-21T07:12:23.798135+0000","last_fullsized":"2024-09-21T07:12:23.798135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:31.259049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":355,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:57.769530+0000","last_change":"2024-09-21T07:09:52.321322+0000","last_active":"2024-09-21T07:12:57.769530+0000","last_peered":"2024-09-21T07:12:57.769530+0000","last_clean":"2024-09-21T07:12:57.769530+0000","last_became_active":"2024-09-21T07:09:52.320932+0000","last_became_peered":"2024-09-21T07:09:52.320932+0000","last_unstale":"2024-09-21T07:12:57.769530+0000","last_undegraded":"2024-09-21T07:12:57.769530+0000","last_fullsized":"2024-09-21T07:12:57.769530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.623079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":269,"num_read_kb":209,"num_write":42,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.848049+0000","last_change":"2024-09-21T07:11:55.847916+0000","last_active":"2024-09-21T07:11:55.848049+0000","last_peered":"2024-09-21T07:11:55.848049+0000","last_clean":"2024-09-21T07:11:55.848049+0000","last_became_active":"2024-09-21T07:11:55.847491+0000","last_became_peered":"2024-09-21T07:11:55.847491+0000","last_unstale":"2024-09-21T07:11:55.848049+0000","last_undegraded":"2024-09-21T07:11:55.848049+0000","last_fullsized":"2024-09-21T07:11:55.848049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:25.702259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,1],"acting":[10,8,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813257+0000","last_change":"2024-09-21T07:09:55.353381+0000","last_active":"2024-09-21T07:12:23.813257+0000","last_peered":"2024-09-21T07:12:23.813257+0000","last_clean":"2024-09-21T07:12:23.813257+0000","last_became_active":"2024-09-21T07:09:55.352943+0000","last_became_peered":"2024-09-21T07:09:55.352943+0000","last_unstale":"2024-09-21T07:12:23.813257+0000","last_undegraded":"2024-09-21T07:12:23.813257+0000","last_fullsized":"2024-09-21T07:12:23.813257+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:40.518275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.846367+0000","last_change":"2024-09-21T07:09:52.361421+0000","last_active":"2024-09-21T07:11:55.846367+0000","last_peered":"2024-09-21T07:11:55.846367+0000","last_clean":"2024-09-21T07:11:55.846367+0000","last_became_active":"2024-09-21T07:09:52.361000+0000","last_became_peered":"2024-09-21T07:09:52.361000+0000","last_unstale":"2024-09-21T07:11:55.846367+0000","last_undegraded":"2024-09-21T07:11:55.846367+0000","last_fullsized":"2024-09-21T07:11:55.846367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:50.233647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845245+0000","last_change":"2024-09-21T07:10:05.422303+0000","last_active":"2024-09-21T07:11:55.845245+0000","last_peered":"2024-09-21T07:11:55.845245+0000","last_clean":"2024-09-21T07:11:55.845245+0000","last_became_active":"2024-09-21T07:10:05.421779+0000","last_became_peered":"2024-09-21T07:10:05.421779+0000","last_unstale":"2024-09-21T07:11:55.845245+0000","last_undegraded":"2024-09-21T07:11:55.845245+0000","last_fullsized":"2024-09-21T07:11:55.845245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:54.824080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.793771+0000","last_change":"2024-09-21T07:09:55.353360+0000","last_active":"2024-09-21T07:12:23.793771+0000","last_peered":"2024-09-21T07:12:23.793771+0000","last_clean":"2024-09-21T07:12:23.793771+0000","last_became_active":"2024-09-21T07:09:55.352994+0000","last_became_peered":"2024-09-21T07:09:55.352994+0000","last_unstale":"2024-09-21T07:12:23.793771+0000","last_undegraded":"2024-09-21T07:12:23.793771+0000","last_fullsized":"2024-09-21T07:12:23.793771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:59.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.981878+0000","last_change":"2024-09-21T07:09:52.325851+0000","last_active":"2024-09-21T07:11:55.981878+0000","last_peered":"2024-09-21T07:11:55.981878+0000","last_clean":"2024-09-21T07:11:55.981878+0000","last_became_active":"2024-09-21T07:09:52.325449+0000","last_became_peered":"2024-09-21T07:09:52.325449+0000","last_unstale":"2024-09-21T07:11:55.981878+0000","last_undegraded":"2024-09-21T07:11:55.981878+0000","last_fullsized":"2024-09-21T07:11:55.981878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:31.171035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.830120+0000","last_change":"2024-09-21T07:10:05.408474+0000","last_active":"2024-09-21T07:11:55.830120+0000","last_peered":"2024-09-21T07:11:55.830120+0000","last_clean":"2024-09-21T07:11:55.830120+0000","last_became_active":"2024-09-21T07:10:05.408039+0000","last_became_peered":"2024-09-21T07:10:05.408039+0000","last_unstale":"2024-09-21T07:11:55.830120+0000","last_undegraded":"2024-09-21T07:11:55.830120+0000","last_fullsized":"2024-09-21T07:11:55.830120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:57.892279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.813258+0000","last_change":"2024-09-21T07:09:55.346066+0000","last_active":"2024-09-21T07:12:23.813258+0000","last_peered":"2024-09-21T07:12:23.813258+0000","last_clean":"2024-09-21T07:12:23.813258+0000","last_became_active":"2024-09-21T07:09:55.345668+0000","last_became_peered":"2024-09-21T07:09:55.345668+0000","last_unstale":"2024-09-21T07:12:23.813258+0000","last_undegraded":"2024-09-21T07:12:23.813258+0000","last_fullsized":"2024-09-21T07:12:23.813258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:30.158403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":177,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:56.768270+0000","last_change":"2024-09-21T07:09:52.324015+0000","last_active":"2024-09-21T07:12:56.768270+0000","last_peered":"2024-09-21T07:12:56.768270+0000","last_clean":"2024-09-21T07:12:56.768270+0000","last_became_active":"2024-09-21T07:09:52.323505+0000","last_became_peered":"2024-09-21T07:09:52.323505+0000","last_unstale":"2024-09-21T07:12:56.768270+0000","last_undegraded":"2024-09-21T07:12:56.768270+0000","last_fullsized":"2024-09-21T07:12:56.768270+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:15.915334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833187+0000","last_change":"2024-09-21T07:10:05.417337+0000","last_active":"2024-09-21T07:11:55.833187+0000","last_peered":"2024-09-21T07:11:55.833187+0000","last_clean":"2024-09-21T07:11:55.833187+0000","last_became_active":"2024-09-21T07:10:05.415794+0000","last_became_peered":"2024-09-21T07:10:05.415794+0000","last_unstale":"2024-09-21T07:11:55.833187+0000","last_undegraded":"2024-09-21T07:11:55.833187+0000","last_fullsized":"2024-09-21T07:11:55.833187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.774839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.847006+0000","last_change":"2024-09-21T07:10:05.424086+0000","last_active":"2024-09-21T07:11:55.847006+0000","last_peered":"2024-09-21T07:11:55.847006+0000","last_clean":"2024-09-21T07:11:55.847006+0000","last_became_active":"2024-09-21T07:10:05.423703+0000","last_became_peered":"2024-09-21T07:10:05.423703+0000","last_unstale":"2024-09-21T07:11:55.847006+0000","last_undegraded":"2024-09-21T07:11:55.847006+0000","last_fullsized":"2024-09-21T07:11:55.847006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:37.008895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.802309+0000","last_change":"2024-09-21T07:09:55.347952+0000","last_active":"2024-09-21T07:12:23.802309+0000","last_peered":"2024-09-21T07:12:23.802309+0000","last_clean":"2024-09-21T07:12:23.802309+0000","last_became_active":"2024-09-21T07:09:55.347614+0000","last_became_peered":"2024-09-21T07:09:55.347614+0000","last_unstale":"2024-09-21T07:12:23.802309+0000","last_undegraded":"2024-09-21T07:12:23.802309+0000","last_fullsized":"2024-09-21T07:12:23.802309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:10.401797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845739+0000","last_change":"2024-09-21T07:09:52.322782+0000","last_active":"2024-09-21T07:11:55.845739+0000","last_peered":"2024-09-21T07:11:55.845739+0000","last_clean":"2024-09-21T07:11:55.845739+0000","last_became_active":"2024-09-21T07:09:52.322402+0000","last_became_peered":"2024-09-21T07:09:52.322402+0000","last_unstale":"2024-09-21T07:11:55.845739+0000","last_undegraded":"2024-09-21T07:11:55.845739+0000","last_fullsized":"2024-09-21T07:11:55.845739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:13.025330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849176+0000","last_change":"2024-09-21T07:11:55.848588+0000","last_active":"2024-09-21T07:11:55.849176+0000","last_peered":"2024-09-21T07:11:55.849176+0000","last_clean":"2024-09-21T07:11:55.849176+0000","last_became_active":"2024-09-21T07:11:55.848025+0000","last_became_peered":"2024-09-21T07:11:55.848025+0000","last_unstale":"2024-09-21T07:11:55.849176+0000","last_undegraded":"2024-09-21T07:11:55.849176+0000","last_fullsized":"2024-09-21T07:11:55.849176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:37.413873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810233+0000","last_change":"2024-09-21T07:09:55.351436+0000","last_active":"2024-09-21T07:12:23.810233+0000","last_peered":"2024-09-21T07:12:23.810233+0000","last_clean":"2024-09-21T07:12:23.810233+0000","last_became_active":"2024-09-21T07:09:55.351034+0000","last_became_peered":"2024-09-21T07:09:55.351034+0000","last_unstale":"2024-09-21T07:12:23.810233+0000","last_undegraded":"2024-09-21T07:12:23.810233+0000","last_fullsized":"2024-09-21T07:12:23.810233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:08.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.845457+0000","last_change":"2024-09-21T07:09:52.321482+0000","last_active":"2024-09-21T07:11:55.845457+0000","last_peered":"2024-09-21T07:11:55.845457+0000","last_clean":"2024-09-21T07:11:55.845457+0000","last_became_active":"2024-09-21T07:09:52.321086+0000","last_became_peered":"2024-09-21T07:09:52.321086+0000","last_unstale":"2024-09-21T07:11:55.845457+0000","last_undegraded":"2024-09-21T07:11:55.845457+0000","last_fullsized":"2024-09-21T07:11:55.845457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:54.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.835562+0000","last_change":"2024-09-21T07:09:52.320540+0000","last_active":"2024-09-21T07:11:55.835562+0000","last_peered":"2024-09-21T07:11:55.835562+0000","last_clean":"2024-09-21T07:11:55.835562+0000","last_became_active":"2024-09-21T07:09:52.320030+0000","last_became_peered":"2024-09-21T07:09:52.320030+0000","last_unstale":"2024-09-21T07:11:55.835562+0000","last_undegraded":"2024-09-21T07:11:55.835562+0000","last_fullsized":"2024-09-21T07:11:55.835562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:18.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807495+0000","last_change":"2024-09-21T07:09:55.346271+0000","last_active":"2024-09-21T07:12:23.807495+0000","last_peered":"2024-09-21T07:12:23.807495+0000","last_clean":"2024-09-21T07:12:23.807495+0000","last_became_active":"2024-09-21T07:09:55.345904+0000","last_became_peered":"2024-09-21T07:09:55.345904+0000","last_unstale":"2024-09-21T07:12:23.807495+0000","last_undegraded":"2024-09-21T07:12:23.807495+0000","last_fullsized":"2024-09-21T07:12:23.807495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:49.020117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.849703+0000","last_change":"2024-09-21T07:11:55.848551+0000","last_active":"2024-09-21T07:11:55.849703+0000","last_peered":"2024-09-21T07:11:55.849703+0000","last_clean":"2024-09-21T07:11:55.849703+0000","last_became_active":"2024-09-21T07:11:55.848056+0000","last_became_peered":"2024-09-21T07:11:55.848056+0000","last_unstale":"2024-09-21T07:11:55.849703+0000","last_undegraded":"2024-09-21T07:11:55.849703+0000","last_fullsized":"2024-09-21T07:11:55.849703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:07.779820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'7","reported_seq":186,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:58.771615+0000","last_change":"2024-09-21T07:09:52.324933+0000","last_active":"2024-09-21T07:12:58.771615+0000","last_peered":"2024-09-21T07:12:58.771615+0000","last_clean":"2024-09-21T07:12:58.771615+0000","last_became_active":"2024-09-21T07:09:52.324513+0000","last_became_peered":"2024-09-21T07:09:52.324513+0000","last_unstale":"2024-09-21T07:12:58.771615+0000","last_undegraded":"2024-09-21T07:12:58.771615+0000","last_fullsized":"2024-09-21T07:12:58.771615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:51.287130+0000","last_clean_scrub_stamp":"2024-09-21T07:09:51.287130+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:59.937304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788293+0000","last_change":"2024-09-21T07:09:55.343428+0000","last_active":"2024-09-21T07:12:23.788293+0000","last_peered":"2024-09-21T07:12:23.788293+0000","last_clean":"2024-09-21T07:12:23.788293+0000","last_became_active":"2024-09-21T07:09:55.343019+0000","last_became_peered":"2024-09-21T07:09:55.343019+0000","last_unstale":"2024-09-21T07:12:23.788293+0000","last_undegraded":"2024-09-21T07:12:23.788293+0000","last_fullsized":"2024-09-21T07:12:23.788293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:23.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:11:55.833541+0000","last_change":"2024-09-21T07:10:05.404331+0000","last_active":"2024-09-21T07:11:55.833541+0000","last_peered":"2024-09-21T07:11:55.833541+0000","last_clean":"2024-09-21T07:11:55.833541+0000","last_became_active":"2024-09-21T07:10:05.403705+0000","last_became_peered":"2024-09-21T07:10:05.403705+0000","last_unstale":"2024-09-21T07:11:55.833541+0000","last_undegraded":"2024-09-21T07:11:55.833541+0000","last_fullsized":"2024-09-21T07:11:55.833541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:10:04.381376+0000","last_clean_scrub_stamp":"2024-09-21T07:10:04.381376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:32.621912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.803098+0000","last_change":"2024-09-21T07:09:55.342439+0000","last_active":"2024-09-21T07:12:23.803098+0000","last_peered":"2024-09-21T07:12:23.803098+0000","last_clean":"2024-09-21T07:12:23.803098+0000","last_became_active":"2024-09-21T07:09:55.342045+0000","last_became_peered":"2024-09-21T07:09:55.342045+0000","last_unstale":"2024-09-21T07:12:23.803098+0000","last_undegraded":"2024-09-21T07:12:23.803098+0000","last_fullsized":"2024-09-21T07:12:23.803098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:52.213623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.807453+0000","last_change":"2024-09-21T07:10:55.044531+0000","last_active":"2024-09-21T07:12:23.807453+0000","last_peered":"2024-09-21T07:12:23.807453+0000","last_clean":"2024-09-21T07:12:23.807453+0000","last_became_active":"2024-09-21T07:10:55.044099+0000","last_became_peered":"2024-09-21T07:10:55.044099+0000","last_unstale":"2024-09-21T07:12:23.807453+0000","last_undegraded":"2024-09-21T07:12:23.807453+0000","last_fullsized":"2024-09-21T07:12:23.807453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:14.723829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808124+0000","last_change":"2024-09-21T07:09:55.359597+0000","last_active":"2024-09-21T07:12:23.808124+0000","last_peered":"2024-09-21T07:12:23.808124+0000","last_clean":"2024-09-21T07:12:23.808124+0000","last_became_active":"2024-09-21T07:09:55.359171+0000","last_became_peered":"2024-09-21T07:09:55.359171+0000","last_unstale":"2024-09-21T07:12:23.808124+0000","last_undegraded":"2024-09-21T07:12:23.808124+0000","last_fullsized":"2024-09-21T07:12:23.808124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:44.698105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795490+0000","last_change":"2024-09-21T07:09:55.356914+0000","last_active":"2024-09-21T07:12:23.795490+0000","last_peered":"2024-09-21T07:12:23.795490+0000","last_clean":"2024-09-21T07:12:23.795490+0000","last_became_active":"2024-09-21T07:09:55.356220+0000","last_became_peered":"2024-09-21T07:09:55.356220+0000","last_unstale":"2024-09-21T07:12:23.795490+0000","last_undegraded":"2024-09-21T07:12:23.795490+0000","last_fullsized":"2024-09-21T07:12:23.795490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:28.530828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.805863+0000","last_change":"2024-09-21T07:09:55.358150+0000","last_active":"2024-09-21T07:12:23.805863+0000","last_peered":"2024-09-21T07:12:23.805863+0000","last_clean":"2024-09-21T07:12:23.805863+0000","last_became_active":"2024-09-21T07:09:55.357723+0000","last_became_peered":"2024-09-21T07:09:55.357723+0000","last_unstale":"2024-09-21T07:12:23.805863+0000","last_undegraded":"2024-09-21T07:12:23.805863+0000","last_fullsized":"2024-09-21T07:12:23.805863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:47.714556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.801102+0000","last_change":"2024-09-21T07:09:55.343600+0000","last_active":"2024-09-21T07:12:23.801102+0000","last_peered":"2024-09-21T07:12:23.801102+0000","last_clean":"2024-09-21T07:12:23.801102+0000","last_became_active":"2024-09-21T07:09:55.343227+0000","last_became_peered":"2024-09-21T07:09:55.343227+0000","last_unstale":"2024-09-21T07:12:23.801102+0000","last_undegraded":"2024-09-21T07:12:23.801102+0000","last_fullsized":"2024-09-21T07:12:23.801102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:27.364109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797345+0000","last_change":"2024-09-21T07:09:55.347844+0000","last_active":"2024-09-21T07:12:23.797345+0000","last_peered":"2024-09-21T07:12:23.797345+0000","last_clean":"2024-09-21T07:12:23.797345+0000","last_became_active":"2024-09-21T07:09:55.347452+0000","last_became_peered":"2024-09-21T07:09:55.347452+0000","last_unstale":"2024-09-21T07:12:23.797345+0000","last_undegraded":"2024-09-21T07:12:23.797345+0000","last_fullsized":"2024-09-21T07:12:23.797345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:00.108513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806243+0000","last_change":"2024-09-21T07:09:55.363602+0000","last_active":"2024-09-21T07:12:23.806243+0000","last_peered":"2024-09-21T07:12:23.806243+0000","last_clean":"2024-09-21T07:12:23.806243+0000","last_became_active":"2024-09-21T07:09:55.362341+0000","last_became_peered":"2024-09-21T07:09:55.362341+0000","last_unstale":"2024-09-21T07:12:23.806243+0000","last_undegraded":"2024-09-21T07:12:23.806243+0000","last_fullsized":"2024-09-21T07:12:23.806243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:57.503749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810287+0000","last_change":"2024-09-21T07:10:54.857953+0000","last_active":"2024-09-21T07:12:23.810287+0000","last_peered":"2024-09-21T07:12:23.810287+0000","last_clean":"2024-09-21T07:12:23.810287+0000","last_became_active":"2024-09-21T07:10:54.857555+0000","last_became_peered":"2024-09-21T07:10:54.857555+0000","last_unstale":"2024-09-21T07:12:23.810287+0000","last_undegraded":"2024-09-21T07:12:23.810287+0000","last_fullsized":"2024-09-21T07:12:23.810287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.747341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797647+0000","last_change":"2024-09-21T07:10:54.854723+0000","last_active":"2024-09-21T07:12:23.797647+0000","last_peered":"2024-09-21T07:12:23.797647+0000","last_clean":"2024-09-21T07:12:23.797647+0000","last_became_active":"2024-09-21T07:10:54.854281+0000","last_became_peered":"2024-09-21T07:10:54.854281+0000","last_unstale":"2024-09-21T07:12:23.797647+0000","last_undegraded":"2024-09-21T07:12:23.797647+0000","last_fullsized":"2024-09-21T07:12:23.797647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.942737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.815349+0000","last_change":"2024-09-21T07:09:55.346027+0000","last_active":"2024-09-21T07:12:23.815349+0000","last_peered":"2024-09-21T07:12:23.815349+0000","last_clean":"2024-09-21T07:12:23.815349+0000","last_became_active":"2024-09-21T07:09:55.345397+0000","last_became_peered":"2024-09-21T07:09:55.345397+0000","last_unstale":"2024-09-21T07:12:23.815349+0000","last_undegraded":"2024-09-21T07:12:23.815349+0000","last_fullsized":"2024-09-21T07:12:23.815349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:56.307621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809317+0000","last_change":"2024-09-21T07:09:55.348738+0000","last_active":"2024-09-21T07:12:23.809317+0000","last_peered":"2024-09-21T07:12:23.809317+0000","last_clean":"2024-09-21T07:12:23.809317+0000","last_became_active":"2024-09-21T07:09:55.347830+0000","last_became_peered":"2024-09-21T07:09:55.347830+0000","last_unstale":"2024-09-21T07:12:23.809317+0000","last_undegraded":"2024-09-21T07:12:23.809317+0000","last_fullsized":"2024-09-21T07:12:23.809317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:12.282800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.795846+0000","last_change":"2024-09-21T07:09:55.352792+0000","last_active":"2024-09-21T07:12:23.795846+0000","last_peered":"2024-09-21T07:12:23.795846+0000","last_clean":"2024-09-21T07:12:23.795846+0000","last_became_active":"2024-09-21T07:09:55.352472+0000","last_became_peered":"2024-09-21T07:09:55.352472+0000","last_unstale":"2024-09-21T07:12:23.795846+0000","last_undegraded":"2024-09-21T07:12:23.795846+0000","last_fullsized":"2024-09-21T07:12:23.795846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.332695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.808162+0000","last_change":"2024-09-21T07:09:55.359907+0000","last_active":"2024-09-21T07:12:23.808162+0000","last_peered":"2024-09-21T07:12:23.808162+0000","last_clean":"2024-09-21T07:12:23.808162+0000","last_became_active":"2024-09-21T07:09:55.359613+0000","last_became_peered":"2024-09-21T07:09:55.359613+0000","last_unstale":"2024-09-21T07:12:23.808162+0000","last_undegraded":"2024-09-21T07:12:23.808162+0000","last_fullsized":"2024-09-21T07:12:23.808162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:40.998774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797650+0000","last_change":"2024-09-21T07:09:55.350196+0000","last_active":"2024-09-21T07:12:23.797650+0000","last_peered":"2024-09-21T07:12:23.797650+0000","last_clean":"2024-09-21T07:12:23.797650+0000","last_became_active":"2024-09-21T07:09:55.349382+0000","last_became_peered":"2024-09-21T07:09:55.349382+0000","last_unstale":"2024-09-21T07:12:23.797650+0000","last_undegraded":"2024-09-21T07:12:23.797650+0000","last_fullsized":"2024-09-21T07:12:23.797650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:32.917191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811678+0000","last_change":"2024-09-21T07:09:55.356380+0000","last_active":"2024-09-21T07:12:23.811678+0000","last_peered":"2024-09-21T07:12:23.811678+0000","last_clean":"2024-09-21T07:12:23.811678+0000","last_became_active":"2024-09-21T07:09:55.355994+0000","last_became_peered":"2024-09-21T07:09:55.355994+0000","last_unstale":"2024-09-21T07:12:23.811678+0000","last_undegraded":"2024-09-21T07:12:23.811678+0000","last_fullsized":"2024-09-21T07:12:23.811678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:28.601601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814831+0000","last_change":"2024-09-21T07:09:55.360321+0000","last_active":"2024-09-21T07:12:23.814831+0000","last_peered":"2024-09-21T07:12:23.814831+0000","last_clean":"2024-09-21T07:12:23.814831+0000","last_became_active":"2024-09-21T07:09:55.359986+0000","last_became_peered":"2024-09-21T07:09:55.359986+0000","last_unstale":"2024-09-21T07:12:23.814831+0000","last_undegraded":"2024-09-21T07:12:23.814831+0000","last_fullsized":"2024-09-21T07:12:23.814831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:44.109594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.798081+0000","last_change":"2024-09-21T07:10:54.861396+0000","last_active":"2024-09-21T07:12:23.798081+0000","last_peered":"2024-09-21T07:12:23.798081+0000","last_clean":"2024-09-21T07:12:23.798081+0000","last_became_active":"2024-09-21T07:10:54.861023+0000","last_became_peered":"2024-09-21T07:10:54.861023+0000","last_unstale":"2024-09-21T07:12:23.798081+0000","last_undegraded":"2024-09-21T07:12:23.798081+0000","last_fullsized":"2024-09-21T07:12:23.798081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:36.040514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797741+0000","last_change":"2024-09-21T07:09:55.343528+0000","last_active":"2024-09-21T07:12:23.797741+0000","last_peered":"2024-09-21T07:12:23.797741+0000","last_clean":"2024-09-21T07:12:23.797741+0000","last_became_active":"2024-09-21T07:09:55.343175+0000","last_became_peered":"2024-09-21T07:09:55.343175+0000","last_unstale":"2024-09-21T07:12:23.797741+0000","last_undegraded":"2024-09-21T07:12:23.797741+0000","last_fullsized":"2024-09-21T07:12:23.797741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:02.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797093+0000","last_change":"2024-09-21T07:09:55.348307+0000","last_active":"2024-09-21T07:12:23.797093+0000","last_peered":"2024-09-21T07:12:23.797093+0000","last_clean":"2024-09-21T07:12:23.797093+0000","last_became_active":"2024-09-21T07:09:55.347080+0000","last_became_peered":"2024-09-21T07:09:55.347080+0000","last_unstale":"2024-09-21T07:12:23.797093+0000","last_undegraded":"2024-09-21T07:12:23.797093+0000","last_fullsized":"2024-09-21T07:12:23.797093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:03.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.797350+0000","last_change":"2024-09-21T07:10:54.859519+0000","last_active":"2024-09-21T07:12:23.797350+0000","last_peered":"2024-09-21T07:12:23.797350+0000","last_clean":"2024-09-21T07:12:23.797350+0000","last_became_active":"2024-09-21T07:10:54.859080+0000","last_became_peered":"2024-09-21T07:10:54.859080+0000","last_unstale":"2024-09-21T07:12:23.797350+0000","last_undegraded":"2024-09-21T07:12:23.797350+0000","last_fullsized":"2024-09-21T07:12:23.797350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:30.589642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.814242+0000","last_change":"2024-09-21T07:09:55.349233+0000","last_active":"2024-09-21T07:12:23.814242+0000","last_peered":"2024-09-21T07:12:23.814242+0000","last_clean":"2024-09-21T07:12:23.814242+0000","last_became_active":"2024-09-21T07:09:55.348875+0000","last_became_peered":"2024-09-21T07:09:55.348875+0000","last_unstale":"2024-09-21T07:12:23.814242+0000","last_undegraded":"2024-09-21T07:12:23.814242+0000","last_fullsized":"2024-09-21T07:12:23.814242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:04.365784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.790030+0000","last_change":"2024-09-21T07:09:55.350469+0000","last_active":"2024-09-21T07:12:23.790030+0000","last_peered":"2024-09-21T07:12:23.790030+0000","last_clean":"2024-09-21T07:12:23.790030+0000","last_became_active":"2024-09-21T07:09:55.350109+0000","last_became_peered":"2024-09-21T07:09:55.350109+0000","last_unstale":"2024-09-21T07:12:23.790030+0000","last_undegraded":"2024-09-21T07:12:23.790030+0000","last_fullsized":"2024-09-21T07:12:23.790030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.882041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.810378+0000","last_change":"2024-09-21T07:09:55.346026+0000","last_active":"2024-09-21T07:12:23.810378+0000","last_peered":"2024-09-21T07:12:23.810378+0000","last_clean":"2024-09-21T07:12:23.810378+0000","last_became_active":"2024-09-21T07:09:55.345644+0000","last_became_peered":"2024-09-21T07:09:55.345644+0000","last_unstale":"2024-09-21T07:12:23.810378+0000","last_undegraded":"2024-09-21T07:12:23.810378+0000","last_fullsized":"2024-09-21T07:12:23.810378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:48.447381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809256+0000","last_change":"2024-09-21T07:09:55.344896+0000","last_active":"2024-09-21T07:12:23.809256+0000","last_peered":"2024-09-21T07:12:23.809256+0000","last_clean":"2024-09-21T07:12:23.809256+0000","last_became_active":"2024-09-21T07:09:55.344290+0000","last_became_peered":"2024-09-21T07:09:55.344290+0000","last_unstale":"2024-09-21T07:12:23.809256+0000","last_undegraded":"2024-09-21T07:12:23.809256+0000","last_fullsized":"2024-09-21T07:12:23.809256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:31.659595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.806255+0000","last_change":"2024-09-21T07:09:55.344030+0000","last_active":"2024-09-21T07:12:23.806255+0000","last_peered":"2024-09-21T07:12:23.806255+0000","last_clean":"2024-09-21T07:12:23.806255+0000","last_became_active":"2024-09-21T07:09:55.343643+0000","last_became_peered":"2024-09-21T07:09:55.343643+0000","last_unstale":"2024-09-21T07:12:23.806255+0000","last_undegraded":"2024-09-21T07:12:23.806255+0000","last_fullsized":"2024-09-21T07:12:23.806255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:39.161994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.788650+0000","last_change":"2024-09-21T07:09:55.347962+0000","last_active":"2024-09-21T07:12:23.788650+0000","last_peered":"2024-09-21T07:12:23.788650+0000","last_clean":"2024-09-21T07:12:23.788650+0000","last_became_active":"2024-09-21T07:09:55.347586+0000","last_became_peered":"2024-09-21T07:09:55.347586+0000","last_unstale":"2024-09-21T07:12:23.788650+0000","last_undegraded":"2024-09-21T07:12:23.788650+0000","last_fullsized":"2024-09-21T07:12:23.788650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:18.698938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.809845+0000","last_change":"2024-09-21T07:09:55.351562+0000","last_active":"2024-09-21T07:12:23.809845+0000","last_peered":"2024-09-21T07:12:23.809845+0000","last_clean":"2024-09-21T07:12:23.809845+0000","last_became_active":"2024-09-21T07:09:55.351171+0000","last_became_peered":"2024-09-21T07:09:55.351171+0000","last_unstale":"2024-09-21T07:12:23.809845+0000","last_undegraded":"2024-09-21T07:12:23.809845+0000","last_fullsized":"2024-09-21T07:12:23.809845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:36.741502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.789398+0000","last_change":"2024-09-21T07:09:55.340115+0000","last_active":"2024-09-21T07:12:23.789398+0000","last_peered":"2024-09-21T07:12:23.789398+0000","last_clean":"2024-09-21T07:12:23.789398+0000","last_became_active":"2024-09-21T07:09:55.339735+0000","last_became_peered":"2024-09-21T07:09:55.339735+0000","last_unstale":"2024-09-21T07:12:23.789398+0000","last_undegraded":"2024-09-21T07:12:23.789398+0000","last_fullsized":"2024-09-21T07:12:23.789398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:16.675093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811791+0000","last_change":"2024-09-21T07:09:55.348702+0000","last_active":"2024-09-21T07:12:23.811791+0000","last_peered":"2024-09-21T07:12:23.811791+0000","last_clean":"2024-09-21T07:12:23.811791+0000","last_became_active":"2024-09-21T07:09:55.348367+0000","last_became_peered":"2024-09-21T07:09:55.348367+0000","last_unstale":"2024-09-21T07:12:23.811791+0000","last_undegraded":"2024-09-21T07:12:23.811791+0000","last_fullsized":"2024-09-21T07:12:23.811791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.792064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.799236+0000","last_change":"2024-09-21T07:09:55.343252+0000","last_active":"2024-09-21T07:12:23.799236+0000","last_peered":"2024-09-21T07:12:23.799236+0000","last_clean":"2024-09-21T07:12:23.799236+0000","last_became_active":"2024-09-21T07:09:55.342882+0000","last_became_peered":"2024-09-21T07:09:55.342882+0000","last_unstale":"2024-09-21T07:12:23.799236+0000","last_undegraded":"2024-09-21T07:12:23.799236+0000","last_fullsized":"2024-09-21T07:12:23.799236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.187495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:12:23.811604+0000","last_change":"2024-09-21T07:09:55.349515+0000","last_active":"2024-09-21T07:12:23.811604+0000","last_peered":"2024-09-21T07:12:23.811604+0000","last_clean":"2024-09-21T07:12:23.811604+0000","last_became_active":"2024-09-21T07:09:55.349109+0000","last_became_peered":"2024-09-21T07:09:55.349109+0000","last_unstale":"2024-09-21T07:12:23.811604+0000","last_undegraded":"2024-09-21T07:12:23.811604+0000","last_fullsized":"2024-09-21T07:12:23.811604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:09:54.313353+0000","last_clean_scrub_stamp":"2024-09-21T07:09:54.313353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:15.082497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":867353,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1255,"num_read_kb":889,"num_write":459,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2838528,"data_stored":2602059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":328,"ondisk_log_size":328,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":515860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972407,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37456,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707696,"statfs":{"total":95995035648,"available":95956680704,"internally_reserved":0,"allocated":2101248,"data_stored":1572931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135934,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1413120,"data_stored":913386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332164,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1404928,"data_stored":909188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495687,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1113137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659213,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":511250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855444,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37372,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707780,"statfs":{"total":95995035648,"available":95956766720,"internally_reserved":0,"allocated":2015232,"data_stored":1501578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215197,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1798144,"data_stored":1298436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":588338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":586286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705775,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":77178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":395954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":471403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:13:05.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:13:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:05 smithi119 ceph-mon[34519]: pgmap v379: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 185 KiB/s wr, 64 op/s 2024-09-21T07:13:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:05 smithi027 ceph-mon[30779]: pgmap v379: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 185 KiB/s wr, 64 op/s 2024-09-21T07:13:05.415 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:05 smithi153 ceph-mon[34747]: pgmap v379: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 185 KiB/s wr, 64 op/s 2024-09-21T07:13:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 -- 172.21.15.27:0/1167707006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84841094c0 msgr2=0x7f84841098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 --2- 172.21.15.27:0/1167707006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84841094c0 0x7f84841098d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8470009900 tx=0x7f847002a410 comp rx=0 tx=0).stop 2024-09-21T07:13:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 -- 172.21.15.27:0/1167707006 shutdown_connections 2024-09-21T07:13:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 --2- 172.21.15.27:0/1167707006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8484109e10 0x7f8484110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 --2- 172.21.15.27:0/1167707006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84841094c0 0x7f84841098d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 --2- 172.21.15.27:0/1167707006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484103b80 0x7f8484103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 -- 172.21.15.27:0/1167707006 >> 172.21.15.27:0/1167707006 conn(0x7f84840ff5d0 msgr2=0x7f8484101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:06.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 -- 172.21.15.27:0/1167707006 shutdown_connections 2024-09-21T07:13:06.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.034+0000 7f8489400640 1 -- 172.21.15.27:0/1167707006 wait complete. 2024-09-21T07:13:06.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 Processor -- start 2024-09-21T07:13:06.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 -- start start 2024-09-21T07:13:06.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484103b80 0x7f848419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:06.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84841094c0 0x7f848419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8484109e10 0x7f84841a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8484113ad0 con 0x7f8484109e10 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8484113950 con 0x7f8484103b80 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.036+0000 7f8489400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8484113c50 con 0x7f84841094c0 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8483200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8484109e10 0x7f84841a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8482800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484103b80 0x7f848419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8483200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8484109e10 0x7f84841a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56728/0 (socket says 172.21.15.27:56728) 2024-09-21T07:13:06.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8483200640 1 -- 172.21.15.27:0/2644006850 learned_addr learned my addr 172.21.15.27:0/2644006850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:06.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8481e00640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84841094c0 0x7f848419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:06.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8482800640 1 -- 172.21.15.27:0/2644006850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84841094c0 msgr2=0x7f848419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:06.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8482800640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84841094c0 0x7f848419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8482800640 1 -- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8484109e10 msgr2=0x7f84841a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:06.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8482800640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8484109e10 0x7f84841a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.037+0000 7f8482800640 1 -- 172.21.15.27:0/2644006850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84841a2670 con 0x7f8484103b80 2024-09-21T07:13:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.038+0000 7f8482800640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484103b80 0x7f848419c700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f847400e9f0 tx=0x7f847400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.038+0000 7f847b400640 1 -- 172.21.15.27:0/2644006850 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847400ccc0 con 0x7f8484103b80 2024-09-21T07:13:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.038+0000 7f847b400640 1 -- 172.21.15.27:0/2644006850 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84740108e0 con 0x7f8484103b80 2024-09-21T07:13:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.038+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84841a2960 con 0x7f8484103b80 2024-09-21T07:13:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.038+0000 7f847b400640 1 -- 172.21.15.27:0/2644006850 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474018880 con 0x7f8484103b80 2024-09-21T07:13:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.038+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84841a90c0 con 0x7f8484103b80 2024-09-21T07:13:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.039+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8484103f90 con 0x7f8484103b80 2024-09-21T07:13:06.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.041+0000 7f847b400640 1 -- 172.21.15.27:0/2644006850 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f8474010430 con 0x7f8484103b80 2024-09-21T07:13:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.043+0000 7f847b400640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84680b6790 0x7f84680b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.043+0000 7f847b400640 1 -- 172.21.15.27:0/2644006850 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f84740e7f80 con 0x7f8484103b80 2024-09-21T07:13:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.043+0000 7f8481e00640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84680b6790 0x7f84680b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.044+0000 7f8481e00640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84680b6790 0x7f84680b8ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8470009230 tx=0x7f8470031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.047+0000 7f847b400640 1 -- 172.21.15.27:0/2644006850 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84740ae010 con 0x7f8484103b80 2024-09-21T07:13:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:06 smithi119 ceph-mon[34519]: from='client.25547 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:06.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.286+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84841a9430 con 0x7f8484103b80 2024-09-21T07:13:06.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:06 smithi027 ceph-mon[30779]: from='client.25547 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:06.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.287+0000 7f847b400640 1 -- 172.21.15.27:0/2644006850 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f84740b23f0 con 0x7f8484103b80 2024-09-21T07:13:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:06.290 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:13:06.290 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:13:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84680b6790 msgr2=0x7f84680b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84680b6790 0x7f84680b8ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8470009230 tx=0x7f8470031400 comp rx=0 tx=0).stop 2024-09-21T07:13:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484103b80 msgr2=0x7f848419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484103b80 0x7f848419c700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f847400e9f0 tx=0x7f847400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 shutdown_connections 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84841094c0 0x7f848419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84680b6790 0x7f84680b8ca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8484109e10 0x7f84841a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 --2- 172.21.15.27:0/2644006850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484103b80 0x7f848419c700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.291+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 >> 172.21.15.27:0/2644006850 conn(0x7f84840ff5d0 msgr2=0x7f8484108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.292+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 shutdown_connections 2024-09-21T07:13:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:06.292+0000 7f8489400640 1 -- 172.21.15.27:0/2644006850 wait complete. 2024-09-21T07:13:06.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:06 smithi153 ceph-mon[34747]: from='client.25547 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T07:13:07.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:13:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:07 smithi119 ceph-mon[34519]: pgmap v380: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 185 KiB/s wr, 66 op/s 2024-09-21T07:13:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:07 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2644006850' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:07.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:07 smithi027 ceph-mon[30779]: pgmap v380: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 185 KiB/s wr, 66 op/s 2024-09-21T07:13:07.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:07 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2644006850' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:07 smithi153 ceph-mon[34747]: pgmap v380: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 185 KiB/s wr, 66 op/s 2024-09-21T07:13:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:07 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2644006850' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:07.679 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.336+0000 7fa1ece00640 1 -- 172.21.15.27:0/2272533493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8114450 msgr2=0x7fa1e8116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.336+0000 7fa1ece00640 1 --2- 172.21.15.27:0/2272533493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8114450 0x7fa1e8116920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009ca0 tx=0x7fa1e002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.337+0000 7fa1ece00640 1 -- 172.21.15.27:0/2272533493 shutdown_connections 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.337+0000 7fa1ece00640 1 --2- 172.21.15.27:0/2272533493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8114450 0x7fa1e8116920 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.337+0000 7fa1ece00640 1 --2- 172.21.15.27:0/2272533493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e8075dd0 0x7fa1e80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.337+0000 7fa1ece00640 1 --2- 172.21.15.27:0/2272533493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8077380 0x7fa1e80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.337+0000 7fa1ece00640 1 -- 172.21.15.27:0/2272533493 >> 172.21.15.27:0/2272533493 conn(0x7fa1e80ff5d0 msgr2=0x7fa1e8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.337+0000 7fa1ece00640 1 -- 172.21.15.27:0/2272533493 shutdown_connections 2024-09-21T07:13:08.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.337+0000 7fa1ece00640 1 -- 172.21.15.27:0/2272533493 wait complete. 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 Processor -- start 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 -- start start 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8075dd0 0x7fa1e81a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8077380 0x7fa1e81a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e8114450 0x7fa1e81a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e811a0d0 con 0x7fa1e8114450 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8119f50 con 0x7fa1e8077380 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.338+0000 7fa1ece00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e811a250 con 0x7fa1e8075dd0 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8075dd0 0x7fa1e81a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8075dd0 0x7fa1e81a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40614/0 (socket says 172.21.15.27:40614) 2024-09-21T07:13:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 -- 172.21.15.27:0/4006368577 learned_addr learned my addr 172.21.15.27:0/4006368577 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e6800640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e8114450 0x7fa1e81a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1dd400640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8077380 0x7fa1e81a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 -- 172.21.15.27:0/4006368577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8077380 msgr2=0x7fa1e81a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8077380 0x7fa1e81a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 -- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e8114450 msgr2=0x7fa1e81a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e8114450 0x7fa1e81a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 -- 172.21.15.27:0/4006368577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e81a6970 con 0x7fa1e8075dd0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e6800640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e8114450 0x7fa1e81a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.339+0000 7fa1e5e00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8075dd0 0x7fa1e81a0a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9f0 tx=0x7fa1d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.340+0000 7fa1df400640 1 -- 172.21.15.27:0/4006368577 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800cd80 con 0x7fa1e8075dd0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.340+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e81a6c60 con 0x7fa1e8075dd0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.340+0000 7fa1df400640 1 -- 172.21.15.27:0/4006368577 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1d80108e0 con 0x7fa1e8075dd0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.340+0000 7fa1df400640 1 -- 172.21.15.27:0/4006368577 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8018980 con 0x7fa1e8075dd0 2024-09-21T07:13:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.340+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81ad3c0 con 0x7fa1e8075dd0 2024-09-21T07:13:08.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.341+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e80761e0 con 0x7fa1e8075dd0 2024-09-21T07:13:08.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.344+0000 7fa1df400640 1 -- 172.21.15.27:0/4006368577 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fa1d8010430 con 0x7fa1e8075dd0 2024-09-21T07:13:08.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.346+0000 7fa1df400640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1cc0b6710 0x7fa1cc0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:08.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.346+0000 7fa1df400640 1 -- 172.21.15.27:0/4006368577 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa1d80e84d0 con 0x7fa1e8075dd0 2024-09-21T07:13:08.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.346+0000 7fa1dd400640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1cc0b6710 0x7fa1cc0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:08.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.347+0000 7fa1dd400640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1cc0b6710 0x7fa1cc0b8c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00095f0 tx=0x7fa1d0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:08.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.347+0000 7fa1df400640 1 -- 172.21.15.27:0/4006368577 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1d80ae500 con 0x7fa1e8075dd0 2024-09-21T07:13:08.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.583+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1e802e640 con 0x7fa1e8075dd0 2024-09-21T07:13:08.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.584+0000 7fa1df400640 1 -- 172.21.15.27:0/4006368577 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fa1d80b28e0 con 0x7fa1e8075dd0 2024-09-21T07:13:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:08.587 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:13:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:13:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1cc0b6710 msgr2=0x7fa1cc0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1cc0b6710 0x7fa1cc0b8c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00095f0 tx=0x7fa1d0005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8075dd0 msgr2=0x7fa1e81a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8075dd0 0x7fa1e81a0a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9f0 tx=0x7fa1d800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 shutdown_connections 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1cc0b6710 0x7fa1cc0b8c20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e8114450 0x7fa1e81a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e8077380 0x7fa1e81a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 --2- 172.21.15.27:0/4006368577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8075dd0 0x7fa1e81a0a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.587+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 >> 172.21.15.27:0/4006368577 conn(0x7fa1e80ff5d0 msgr2=0x7fa1e8110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.588+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 shutdown_connections 2024-09-21T07:13:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:08.588+0000 7fa1ece00640 1 -- 172.21.15.27:0/4006368577 wait complete. 2024-09-21T07:13:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:09 smithi119 ceph-mon[34519]: pgmap v381: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 196 KiB/s wr, 67 op/s 2024-09-21T07:13:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:09 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:13:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:09 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4006368577' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:09.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:09 smithi027 ceph-mon[30779]: pgmap v381: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 196 KiB/s wr, 67 op/s 2024-09-21T07:13:09.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:09 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:13:09.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:09 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4006368577' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:09.364 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:13:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:09 smithi153 ceph-mon[34747]: pgmap v381: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 196 KiB/s wr, 67 op/s 2024-09-21T07:13:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:09 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:13:09.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:09 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4006368577' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 -- 172.21.15.27:0/2638447466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 msgr2=0x7f53f410fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 --2- 172.21.15.27:0/2638447466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f410fbe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009900 tx=0x7f53dc02a410 comp rx=0 tx=0).stop 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 -- 172.21.15.27:0/2638447466 shutdown_connections 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 --2- 172.21.15.27:0/2638447466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f4110120 0x7f53f4112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 --2- 172.21.15.27:0/2638447466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f410fbe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 --2- 172.21.15.27:0/2638447466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f4075750 0x7f53f4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 -- 172.21.15.27:0/2638447466 >> 172.21.15.27:0/2638447466 conn(0x7f53f407b850 msgr2=0x7f53f407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 -- 172.21.15.27:0/2638447466 shutdown_connections 2024-09-21T07:13:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.377+0000 7f53f9c00640 1 -- 172.21.15.27:0/2638447466 wait complete. 2024-09-21T07:13:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.378+0000 7f53f9c00640 1 Processor -- start 2024-09-21T07:13:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53f9c00640 1 -- start start 2024-09-21T07:13:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53f9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f4075750 0x7f53f419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53f9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f4110120 0x7f53f41a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53f9c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4115c90 con 0x7f53f40760a0 2024-09-21T07:13:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53f9c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4115b10 con 0x7f53f4110120 2024-09-21T07:13:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53f9c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4115e10 con 0x7f53f4075750 2024-09-21T07:13:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.379+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43340/0 (socket says 172.21.15.27:43340) 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53f2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f4075750 0x7f53f419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53ebe00640 1 -- 172.21.15.27:0/1329673972 learned_addr learned my addr 172.21.15.27:0/1329673972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53f3400640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f4110120 0x7f53f41a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53ebe00640 1 -- 172.21.15.27:0/1329673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f4075750 msgr2=0x7f53f419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53ebe00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f4075750 0x7f53f419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53ebe00640 1 -- 172.21.15.27:0/1329673972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f4110120 msgr2=0x7f53f41a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53ebe00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f4110120 0x7f53f41a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53ebe00640 1 -- 172.21.15.27:0/1329673972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f41a2590 con 0x7f53f40760a0 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53f3400640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f4110120 0x7f53f41a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53f2a00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f4075750 0x7f53f419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.380+0000 7f53ebe00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f419cbc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f53dc02a930 tx=0x7f53dc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.381+0000 7f53eb400640 1 -- 172.21.15.27:0/1329673972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc03f030 con 0x7f53f40760a0 2024-09-21T07:13:10.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.381+0000 7f53eb400640 1 -- 172.21.15.27:0/1329673972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53dc004490 con 0x7f53f40760a0 2024-09-21T07:13:10.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.381+0000 7f53eb400640 1 -- 172.21.15.27:0/1329673972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc035070 con 0x7f53f40760a0 2024-09-21T07:13:10.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.381+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f41a2820 con 0x7f53f40760a0 2024-09-21T07:13:10.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.381+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f41a8fe0 con 0x7f53f40760a0 2024-09-21T07:13:10.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.382+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f4076c60 con 0x7f53f40760a0 2024-09-21T07:13:10.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.386+0000 7f53eb400640 1 -- 172.21.15.27:0/1329673972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f53dc042030 con 0x7f53f40760a0 2024-09-21T07:13:10.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.388+0000 7f53eb400640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53d80b6710 0x7f53d80b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:10.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.388+0000 7f53eb400640 1 -- 172.21.15.27:0/1329673972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f53dc108360 con 0x7f53f40760a0 2024-09-21T07:13:10.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.388+0000 7f53f2a00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53d80b6710 0x7f53d80b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:10.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.389+0000 7f53f2a00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53d80b6710 0x7f53d80b8c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f53e40095f0 tx=0x7f53e4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:10.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.391+0000 7f53eb400640 1 -- 172.21.15.27:0/1329673972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53dc0d2770 con 0x7f53f40760a0 2024-09-21T07:13:10.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.598+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53f410b2a0 con 0x7f53f40760a0 2024-09-21T07:13:10.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.599+0000 7f53eb400640 1 -- 172.21.15.27:0/1329673972 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25234 (secure 0 0 0) 0x7f53dc0d76b0 con 0x7f53f40760a0 2024-09-21T07:13:10.602 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:10.602 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:11:55.826198+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10},{"from":5,"to":1}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:13:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53d80b6710 msgr2=0x7f53d80b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53d80b6710 0x7f53d80b8c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f53e40095f0 tx=0x7f53e4005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 msgr2=0x7f53f419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f419cbc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f53dc02a930 tx=0x7f53dc031a20 comp rx=0 tx=0).stop 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 shutdown_connections 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53d80b6710 0x7f53d80b8c20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f4110120 0x7f53f41a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f40760a0 0x7f53f419cbc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 --2- 172.21.15.27:0/1329673972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f4075750 0x7f53f419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 >> 172.21.15.27:0/1329673972 conn(0x7f53f407b850 msgr2=0x7f53f4101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 shutdown_connections 2024-09-21T07:13:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:10.602+0000 7f53f9c00640 1 -- 172.21.15.27:0/1329673972 wait complete. 2024-09-21T07:13:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:11 smithi119 ceph-mon[34519]: pgmap v382: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 196 KiB/s wr, 68 op/s 2024-09-21T07:13:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:11 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1329673972' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:13:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:11 smithi027 ceph-mon[30779]: pgmap v382: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 196 KiB/s wr, 68 op/s 2024-09-21T07:13:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:11 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1329673972' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:13:11.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T07:13:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 -- 172.21.15.27:0/2157296451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8109240 msgr2=0x7fefc8109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 --2- 172.21.15.27:0/2157296451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8109240 0x7fefc8109650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009860 tx=0x7fefb402a340 comp rx=0 tx=0).stop 2024-09-21T07:13:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 -- 172.21.15.27:0/2157296451 shutdown_connections 2024-09-21T07:13:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 --2- 172.21.15.27:0/2157296451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc8109b90 0x7fefc81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 --2- 172.21.15.27:0/2157296451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8109240 0x7fefc8109650 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 --2- 172.21.15.27:0/2157296451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc8103910 0x7fefc8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 -- 172.21.15.27:0/2157296451 >> 172.21.15.27:0/2157296451 conn(0x7fefc80ff330 msgr2=0x7fefc8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 -- 172.21.15.27:0/2157296451 shutdown_connections 2024-09-21T07:13:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.412+0000 7fefcec00640 1 -- 172.21.15.27:0/2157296451 wait complete. 2024-09-21T07:13:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.413+0000 7fefcec00640 1 Processor -- start 2024-09-21T07:13:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.413+0000 7fefcec00640 1 -- start start 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcec00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8103910 0x7fefc819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcec00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc8109240 0x7fefc819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcec00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc8109b90 0x7fefc81a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcec00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fefc8113850 con 0x7fefc8109240 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcec00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81136d0 con 0x7fefc8103910 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcec00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81139d0 con 0x7fefc8109b90 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc8109240 0x7fefc819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcd800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8103910 0x7fefc819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc8109240 0x7fefc819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43366/0 (socket says 172.21.15.27:43366) 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefcce00640 1 -- 172.21.15.27:0/3519337985 learned_addr learned my addr 172.21.15.27:0/3519337985 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.414+0000 7fefce200640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc8109b90 0x7fefc81a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefcd800640 1 -- 172.21.15.27:0/3519337985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc8109b90 msgr2=0x7fefc81a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefcd800640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc8109b90 0x7fefc81a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefcd800640 1 -- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc8109240 msgr2=0x7fefc819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefcd800640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc8109240 0x7fefc819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefcd800640 1 -- 172.21.15.27:0/3519337985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc81a2460 con 0x7fefc8103910 2024-09-21T07:13:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefcd800640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8103910 0x7fefc819c530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00e9f0 tx=0x7fefbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefc6000640 1 -- 172.21.15.27:0/3519337985 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00ccc0 con 0x7fefc8103910 2024-09-21T07:13:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefc6000640 1 -- 172.21.15.27:0/3519337985 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefbc005020 con 0x7fefc8103910 2024-09-21T07:13:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc81a2750 con 0x7fefc8103910 2024-09-21T07:13:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.415+0000 7fefc6000640 1 -- 172.21.15.27:0/3519337985 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc004a00 con 0x7fefc8103910 2024-09-21T07:13:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.416+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc81a8ee0 con 0x7fefc8103910 2024-09-21T07:13:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.419+0000 7fefd73c7740 1 Processor -- start 2024-09-21T07:13:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.420+0000 7fefd73c7740 1 -- start start 2024-09-21T07:13:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.420+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.420+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e7452f0 0x55ed0eb13350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.420+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb13890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.420+0000 7fefd73c7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ed0ea886b0 con 0x55ed0e7452f0 2024-09-21T07:13:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.420+0000 7fefd73c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ed0ea77ae0 con 0x55ed0e74d4d0 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.420+0000 7fefd73c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed0ea7b8f0 con 0x55ed0e76e1b0 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb12e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40652/0 (socket says 172.21.15.27:40652) 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 -- 172.21.15.27:0/3628547019 learned_addr learned my addr 172.21.15.27:0/3628547019 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefcce00640 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb13890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 -- 172.21.15.27:0/3628547019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 msgr2=0x55ed0eb13890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb13890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 -- 172.21.15.27:0/3628547019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e7452f0 msgr2=0x55ed0eb13350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e7452f0 0x55ed0eb13350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefce200640 1 -- 172.21.15.27:0/3628547019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0ea44780 con 0x55ed0e76e1b0 2024-09-21T07:13:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.421+0000 7fefc6000640 1 -- 172.21.15.27:0/3519337985 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fefbc004ba0 con 0x7fefc8103910 2024-09-21T07:13:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.422+0000 7fefce200640 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb12e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009a20 tx=0x7fefc00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5298621dda81a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.423+0000 7fefc6000640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fefb00b6710 0x7fefb00b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.423+0000 7fefbaa00640 1 -- 172.21.15.27:0/3628547019 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc001d070 con 0x55ed0e76e1b0 2024-09-21T07:13:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.423+0000 7fefbaa00640 1 -- 172.21.15.27:0/3628547019 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefc0003e40 con 0x55ed0e76e1b0 2024-09-21T07:13:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.423+0000 7fefbaa00640 1 -- 172.21.15.27:0/3628547019 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0018a40 con 0x55ed0e76e1b0 2024-09-21T07:13:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.423+0000 7fefc6000640 1 -- 172.21.15.27:0/3519337985 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fefbc0e7c10 con 0x7fefc8103910 2024-09-21T07:13:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.423+0000 7fefcd800640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fefb00b6710 0x7fefb00b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 -- 172.21.15.27:0/3628547019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 msgr2=0x55ed0eb12e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb12e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009a20 tx=0x7fefc00095c0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefcd800640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fefb00b6710 0x7fefb00b8c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0e4f50 tx=0x7fefbc005460 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 -- 172.21.15.27:0/3628547019 shutdown_connections 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb13890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e7452f0 0x55ed0eb13350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 --2- 172.21.15.27:0/3628547019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb12e10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 -- 172.21.15.27:0/3628547019 >> 172.21.15.27:0/3628547019 conn(0x55ed0e762300 msgr2=0x55ed0eb10d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 -- 172.21.15.27:0/3628547019 shutdown_connections 2024-09-21T07:13:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.424+0000 7fefd73c7740 1 -- 172.21.15.27:0/3628547019 wait complete. 2024-09-21T07:13:11.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:11 smithi153 ceph-mon[34747]: pgmap v382: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 196 KiB/s wr, 68 op/s 2024-09-21T07:13:11.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:11 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1329673972' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.425+0000 7fefd73c7740 1 -- start start 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.425+0000 7fefd73c7740 1 Processor -- start 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb1e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb1e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb2a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefd73c7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ed0e9afab0 con 0x55ed0e74d4d0 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefd73c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ed0ea1e570 con 0x55ed0e7452f0 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefd73c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed0eb21590 con 0x55ed0e76e1b0 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefd73c7740 10 client.0 _ll_register_callbacks cb 0x55ed0ea2d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefcce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb1e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefcce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb1e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43382/0 (socket says 172.21.15.27:43382) 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefcce00640 1 -- 172.21.15.27:0/1337644486 learned_addr learned my addr 172.21.15.27:0/1337644486 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefce200640 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb1e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.426+0000 7fefce200640 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb2a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefcce00640 1 -- 172.21.15.27:0/1337644486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 msgr2=0x55ed0eb2a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefcce00640 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb2a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefcce00640 1 -- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 msgr2=0x55ed0eb1e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefcce00640 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb1e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefcce00640 1 -- 172.21.15.27:0/1337644486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0e7d9790 con 0x55ed0e74d4d0 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefcce00640 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb1e9d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fefb402a860 tx=0x7fefb4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb30885e11d2e615 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefaf400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefaf400640 1 -- 172.21.15.27:0/1337644486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4042070 con 0x55ed0e74d4d0 2024-09-21T07:13:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed0ea8a600 con 0x55ed0e74d4d0 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.427+0000 7fefaf400640 1 -- 172.21.15.27:0/1337644486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fefb4004240 con 0x55ed0e74d4d0 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefaf400640 1 -- 172.21.15.27:0/1337644486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefb404e050 con 0x55ed0e74d4d0 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 10 client.15549 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed0e852620 con 0x55ed0e74d4d0 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefaf400640 1 -- 172.21.15.27:0/1337644486 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ed0ea8a600 con 0x55ed0e74d4d0 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefaf400640 1 -- 172.21.15.27:0/1337644486 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fefb40039a0 con 0x55ed0e74d4d0 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 20 client.15549 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e72cd80 0x55ed0eb30600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 4 client.15549 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.428+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed0ea860b0 con 0x55ed0e72cd80 2024-09-21T07:13:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.429+0000 7fefce200640 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e72cd80 0x55ed0eb30600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.429+0000 7fefce200640 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e72cd80 0x55ed0eb30600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c24455ed1fd61a2 server_cookie=194ea2d58e0eadd3 in_seq=0 out_seq=0 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.429+0000 7fefaf400640 10 client.15549 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.430+0000 7fefaf400640 1 -- 172.21.15.27:0/1337644486 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed0ea860b0 con 0x55ed0e72cd80 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.430+0000 7fefaf400640 10 client.15549 handle_command_reply: tid=0 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.430+0000 7fefd73c7740 1 client.15549 shutdown 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.430+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 msgr2=0x55ed0eb1e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb1e9d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fefb402a860 tx=0x7fefb4030d80 comp rx=0 tx=0).stop 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 shutdown_connections 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e72cd80 0x55ed0eb30600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb2a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb1e9d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 --2- 172.21.15.27:0/1337644486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb1e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefaf400640 0 client.15549 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 >> 172.21.15.27:0/1337644486 conn(0x55ed0e762300 msgr2=0x55ed0eb13e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 shutdown_connections 2024-09-21T07:13:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 1 -- 172.21.15.27:0/1337644486 wait complete. 2024-09-21T07:13:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.431+0000 7fefd73c7740 20 client.15549 trim_cache size 0 max 16384 2024-09-21T07:13:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 Processor -- start 2024-09-21T07:13:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 -- start start 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e7452f0 0x55ed0eb0fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb1b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81b56d0 con 0x55ed0e76e1b0 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81b2fa0 con 0x55ed0e74d4d0 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.447+0000 7fefd73c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81b0df0 con 0x55ed0e7452f0 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefce200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb1b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:57070/0 (socket says 172.21.15.27:57070) 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefce200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb1b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43384/0 (socket says 172.21.15.27:43384) 2024-09-21T07:13:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 -- 172.21.15.27:0/628799706 learned_addr learned my addr 172.21.15.27:0/628799706 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefce200640 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e7452f0 0x55ed0eb0fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 -- 172.21.15.27:0/628799706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e7452f0 msgr2=0x55ed0eb0fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e7452f0 0x55ed0eb0fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 -- 172.21.15.27:0/628799706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e76e1b0 msgr2=0x55ed0eb1b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb1b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 -- 172.21.15.27:0/628799706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0eb1c970 con 0x55ed0e74d4d0 2024-09-21T07:13:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefce200640 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb1b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.448+0000 7fefcce00640 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb200c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fefb4005fd0 tx=0x7fefb402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b71fd8943b3113d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.449+0000 7fefba000640 1 -- 172.21.15.27:0/628799706 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4042030 con 0x55ed0e74d4d0 2024-09-21T07:13:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.449+0000 7fefba000640 1 -- 172.21.15.27:0/628799706 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb40096b0 con 0x55ed0e74d4d0 2024-09-21T07:13:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.449+0000 7fefba000640 1 -- 172.21.15.27:0/628799706 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4037040 con 0x55ed0e74d4d0 2024-09-21T07:13:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 -- 172.21.15.27:0/628799706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 msgr2=0x55ed0eb200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb200c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fefb4005fd0 tx=0x7fefb402a340 comp rx=0 tx=0).stop 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 -- 172.21.15.27:0/628799706 shutdown_connections 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb1b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb200c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 --2- 172.21.15.27:0/628799706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e7452f0 0x55ed0eb0fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 -- 172.21.15.27:0/628799706 >> 172.21.15.27:0/628799706 conn(0x55ed0e762300 msgr2=0x55ed0eb1f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 -- 172.21.15.27:0/628799706 shutdown_connections 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 -- 172.21.15.27:0/628799706 wait complete. 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 -- start start 2024-09-21T07:13:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.450+0000 7fefd73c7740 1 Processor -- start 2024-09-21T07:13:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb24220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefd73c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb91f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefd73c7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81b1460 con 0x55ed0e74d4d0 2024-09-21T07:13:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefd73c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81b28d0 con 0x55ed0e7452f0 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefd73c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81b1b30 con 0x55ed0e76e1b0 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefd73c7740 10 client.0 _ll_register_callbacks cb 0x7fefbc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefcce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefcce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb24800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43392/0 (socket says 172.21.15.27:43392) 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefce200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb24220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefcce00640 1 -- 172.21.15.27:0/4159807956 learned_addr learned my addr 172.21.15.27:0/4159807956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.451+0000 7fefce200640 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb91f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefce200640 1 -- 172.21.15.27:0/4159807956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 msgr2=0x55ed0eb91f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefce200640 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb91f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefce200640 1 -- 172.21.15.27:0/4159807956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 msgr2=0x55ed0eb24800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefce200640 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb24800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefce200640 1 -- 172.21.15.27:0/4159807956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0eb92620 con 0x55ed0e7452f0 2024-09-21T07:13:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefcce00640 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb24800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefce200640 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb24220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55ed0eb11a20 tx=0x7fefc00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e3ec3193bfc8dad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefaf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefaf400640 1 -- 172.21.15.27:0/4159807956 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc001d040 con 0x55ed0e7452f0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb40096b0 con 0x55ed0e7452f0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefaf400640 1 -- 172.21.15.27:0/4159807956 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fefc0004000 con 0x55ed0e7452f0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.452+0000 7fefaf400640 1 -- 172.21.15.27:0/4159807956 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefc003f050 con 0x55ed0e7452f0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 10 client.25583 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed0eb9aab0 con 0x55ed0e7452f0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefaf400640 1 -- 172.21.15.27:0/4159807956 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb40096b0 con 0x55ed0e7452f0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefaf400640 1 -- 172.21.15.27:0/4159807956 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fefc000b040 con 0x55ed0e7452f0 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 10 client.25583 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 20 client.25583 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e7021c0 0x55ed0eb9cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 4 client.25583 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:13:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.453+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed0ea834e0 con 0x55ed0e7021c0 2024-09-21T07:13:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.454+0000 7fefcce00640 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e7021c0 0x55ed0eb9cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.454+0000 7fefcce00640 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e7021c0 0x55ed0eb9cf50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ab434cd2f544de server_cookie=83291a1bf23ff087 in_seq=0 out_seq=0 2024-09-21T07:13:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.454+0000 7fefaf400640 10 client.25583 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.468+0000 7fefaf400640 1 -- 172.21.15.27:0/4159807956 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed0ea834e0 con 0x55ed0e7021c0 2024-09-21T07:13:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.468+0000 7fefaf400640 10 client.25583 handle_command_reply: tid=0 2024-09-21T07:13:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.468+0000 7fefd73c7740 1 client.25583 shutdown 2024-09-21T07:13:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.468+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 msgr2=0x55ed0eb24220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.468+0000 7fefd73c7740 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb24220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55ed0eb11a20 tx=0x7fefc00095c0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 shutdown_connections 2024-09-21T07:13:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed0e7021c0 0x55ed0eb9cf50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0e76e1b0 0x55ed0eb91f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed0e74d4d0 0x55ed0eb24800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 --2- 172.21.15.27:0/4159807956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed0e7452f0 0x55ed0eb24220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 >> 172.21.15.27:0/4159807956 conn(0x55ed0e762300 msgr2=0x55ed0eb0faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefaf400640 0 client.25583 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 shutdown_connections 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 1 -- 172.21.15.27:0/4159807956 wait complete. 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.469+0000 7fefd73c7740 20 client.25583 trim_cache size 0 max 16384 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:13:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:13:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:13:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:13:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fefb00b6710 msgr2=0x7fefb00b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fefb00b6710 0x7fefb00b8c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0e4f50 tx=0x7fefbc005460 comp rx=0 tx=0).stop 2024-09-21T07:13:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8103910 msgr2=0x7fefc819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8103910 0x7fefc819c530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00e9f0 tx=0x7fefbc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 shutdown_connections 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fefc8109b90 0x7fefc81a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fefb00b6710 0x7fefb00b8c20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc8109240 0x7fefc819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 --2- 172.21.15.27:0/3519337985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc8103910 0x7fefc819c530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 >> 172.21.15.27:0/3519337985 conn(0x7fefc80ff330 msgr2=0x7fefc81068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.470+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 shutdown_connections 2024-09-21T07:13:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:11.471+0000 7fefcec00640 1 -- 172.21.15.27:0/3519337985 wait complete. 2024-09-21T07:13:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:13.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:13 smithi027 ceph-mon[30779]: pgmap v383: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 95 KiB/s wr, 29 op/s 2024-09-21T07:13:13.199 INFO:teuthology.orchestra.run.smithi027.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T07:13:13.200 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T07:13:13.200 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T07:13:13.200 DEBUG:tasks.quiescer:Yielding 2024-09-21T07:13:13.200 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T07:13:13.217 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:13:13.217 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T07:13:13.217 INFO:tasks.workunit:timeout=3h 2024-09-21T07:13:13.217 INFO:tasks.workunit:cleanup=True 2024-09-21T07:13:13.217 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T07:13:13.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 07:11:01.597822492 +0000 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 07:11:01.597822492 +0000 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 07:12:43.130217691 +0000 2024-09-21T07:13:13.278 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 07:11:01.597822492 +0000 2024-09-21T07:13:13.278 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T07:13:13.279 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T07:13:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:13 smithi119 ceph-mon[34519]: pgmap v383: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 95 KiB/s wr, 29 op/s 2024-09-21T07:13:13.388 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:13:13.427 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T07:13:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:13 smithi153 ceph-mon[34747]: pgmap v383: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 95 KiB/s wr, 29 op/s 2024-09-21T07:13:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 -- 172.21.15.27:0/3708411637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410b6c0 msgr2=0x7f07c410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 --2- 172.21.15.27:0/3708411637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410b6c0 0x7f07c410bad0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009900 tx=0x7f07b002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 -- 172.21.15.27:0/3708411637 shutdown_connections 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 --2- 172.21.15.27:0/3708411637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c410c010 0x7f07c4112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 --2- 172.21.15.27:0/3708411637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410b6c0 0x7f07c410bad0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 --2- 172.21.15.27:0/3708411637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c4105d80 0x7f07c4106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 -- 172.21.15.27:0/3708411637 >> 172.21.15.27:0/3708411637 conn(0x7f07c4101810 msgr2=0x7f07c4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 -- 172.21.15.27:0/3708411637 shutdown_connections 2024-09-21T07:13:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.264+0000 7f07cc600640 1 -- 172.21.15.27:0/3708411637 wait complete. 2024-09-21T07:13:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.265+0000 7f07cc600640 1 Processor -- start 2024-09-21T07:13:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07cc600640 1 -- start start 2024-09-21T07:13:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07cc600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c4105d80 0x7f07c419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07cc600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c410b6c0 0x7f07c419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07cc600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410c010 0x7f07c41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07cc600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4115d20 con 0x7f07c410c010 2024-09-21T07:13:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07cc600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4115ba0 con 0x7f07c410b6c0 2024-09-21T07:13:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07cc600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4115ea0 con 0x7f07c4105d80 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07ca600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410c010 0x7f07c41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07c9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c4105d80 0x7f07c419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07c9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c4105d80 0x7f07c419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40704/0 (socket says 172.21.15.27:40704) 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07ca600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410c010 0x7f07c41a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43406/0 (socket says 172.21.15.27:43406) 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.266+0000 7f07c9c00640 1 -- 172.21.15.27:0/3832773661 learned_addr learned my addr 172.21.15.27:0/3832773661 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07c9200640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c410b6c0 0x7f07c419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07c9c00640 1 -- 172.21.15.27:0/3832773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c410b6c0 msgr2=0x7f07c419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07c9c00640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c410b6c0 0x7f07c419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07c9c00640 1 -- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410c010 msgr2=0x7f07c41a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07c9c00640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410c010 0x7f07c41a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07c9c00640 1 -- 172.21.15.27:0/3832773661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c41a2640 con 0x7f07c4105d80 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07ca600640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410c010 0x7f07c41a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.267+0000 7f07c9c00640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c4105d80 0x7f07c419c6d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.268+0000 7f07c2000640 1 -- 172.21.15.27:0/3832773661 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b800cd80 con 0x7f07c4105d80 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.268+0000 7f07c2000640 1 -- 172.21.15.27:0/3832773661 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b80108e0 con 0x7f07c4105d80 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.268+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c41a2930 con 0x7f07c4105d80 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.268+0000 7f07c2000640 1 -- 172.21.15.27:0/3832773661 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8018980 con 0x7f07c4105d80 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.268+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c41a9090 con 0x7f07c4105d80 2024-09-21T07:13:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.269+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c4106190 con 0x7f07c4105d80 2024-09-21T07:13:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.271+0000 7f07c2000640 1 -- 172.21.15.27:0/3832773661 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f07b8010430 con 0x7f07c4105d80 2024-09-21T07:13:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.273+0000 7f07c2000640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07ac0b6710 0x7f07ac0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.273+0000 7f07c2000640 1 -- 172.21.15.27:0/3832773661 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f07b80e8200 con 0x7f07c4105d80 2024-09-21T07:13:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.273+0000 7f07c9200640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07ac0b6710 0x7f07ac0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:14.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.274+0000 7f07c9200640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07ac0b6710 0x7f07ac0b8c20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f07b002a9e0 tx=0x7f07b0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:14.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.275+0000 7f07c2000640 1 -- 172.21.15.27:0/3832773661 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07b80b2610 con 0x7f07c4105d80 2024-09-21T07:13:14.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.517+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07c419d800 con 0x7f07c4105d80 2024-09-21T07:13:14.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.518+0000 7f07c2000640 1 -- 172.21.15.27:0/3832773661 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f07b80b7550 con 0x7f07c4105d80 2024-09-21T07:13:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:14.521 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:13:14.521 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:13:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07ac0b6710 msgr2=0x7f07ac0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07ac0b6710 0x7f07ac0b8c20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f07b002a9e0 tx=0x7f07b0005a20 comp rx=0 tx=0).stop 2024-09-21T07:13:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c4105d80 msgr2=0x7f07c419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c4105d80 0x7f07c419c6d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 shutdown_connections 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07ac0b6710 0x7f07ac0b8c20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07c410c010 0x7f07c41a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c410b6c0 0x7f07c419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 --2- 172.21.15.27:0/3832773661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c4105d80 0x7f07c419c6d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.521+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 >> 172.21.15.27:0/3832773661 conn(0x7f07c4101810 msgr2=0x7f07c410ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.522+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 shutdown_connections 2024-09-21T07:13:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:14.522+0000 7f07cc600640 1 -- 172.21.15.27:0/3832773661 wait complete. 2024-09-21T07:13:15.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:15 smithi027 ceph-mon[30779]: pgmap v384: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:13:15.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:15 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3832773661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:15.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T07:09:56.473755+0000', 'modified': '2024-09-21T07:12:44.098389+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24773, 'mds_1': 24748, 'mds_2': 24740}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24713': {'gid': 24713, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/3894045960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 3894045960}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3894045960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24721': {'gid': 24721, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/1555665457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 1555665457}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 1555665457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24736': {'gid': 24736, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6839/2081315981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 2081315981}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 2081315981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24740': {'gid': 24740, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.119:6837/4207522386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 4207522386}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 4207522386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24748': {'gid': 24748, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.27:6839/609554839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 609554839}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 609554839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24773': {'gid': 24773, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.119:6839/1565967470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1565967470}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1565967470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24773, 'qdb_cluster': [24740, 24748, 24773]} 2024-09-21T07:13:15.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:13:15.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T07:13:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:15 smithi153 ceph-mon[34747]: pgmap v384: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:13:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:15 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3832773661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:15.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:15 smithi119 ceph-mon[34519]: pgmap v384: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:13:15.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:15 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3832773661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:16.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 -- 172.21.15.27:0/359204721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0114570 msgr2=0x7f7cb0116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 --2- 172.21.15.27:0/359204721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0114570 0x7f7cb0116990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009be0 tx=0x7f7ca802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 -- 172.21.15.27:0/359204721 shutdown_connections 2024-09-21T07:13:16.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 --2- 172.21.15.27:0/359204721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 0x7f7cb0114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 --2- 172.21.15.27:0/359204721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0114570 0x7f7cb0116990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 --2- 172.21.15.27:0/359204721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb0077940 0x7f7cb0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 -- 172.21.15.27:0/359204721 >> 172.21.15.27:0/359204721 conn(0x7f7cb0101680 msgr2=0x7f7cb0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:16.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 -- 172.21.15.27:0/359204721 shutdown_connections 2024-09-21T07:13:16.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.382+0000 7f7cb6600640 1 -- 172.21.15.27:0/359204721 wait complete. 2024-09-21T07:13:16.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.383+0000 7f7cb6600640 1 Processor -- start 2024-09-21T07:13:16.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.383+0000 7f7cb6600640 1 -- start start 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0077940 0x7f7cb01a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb6600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 0x7f7cb01a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb6600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb0114570 0x7f7cb01a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb6600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0119ef0 con 0x7f7cb0114570 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb6600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0119d70 con 0x7f7cb0077940 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb6600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb011a070 con 0x7f7cb0078290 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb5c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb0114570 0x7f7cb01a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 0x7f7cb01a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cb5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0077940 0x7f7cb01a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 0x7f7cb01a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40726/0 (socket says 172.21.15.27:40726) 2024-09-21T07:13:16.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cafe00640 1 -- 172.21.15.27:0/262664802 learned_addr learned my addr 172.21.15.27:0/262664802 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:16.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cafe00640 1 -- 172.21.15.27:0/262664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0077940 msgr2=0x7f7cb01a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.384+0000 7f7cafe00640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0077940 0x7f7cb01a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cafe00640 1 -- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb0114570 msgr2=0x7f7cb01a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cafe00640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb0114570 0x7f7cb01a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cafe00640 1 -- 172.21.15.27:0/262664802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb01a6810 con 0x7f7cb0078290 2024-09-21T07:13:16.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cafe00640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 0x7f7cb01a0e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:16.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cad600640 1 -- 172.21.15.27:0/262664802 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00ccc0 con 0x7f7cb0078290 2024-09-21T07:13:16.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb01a6b00 con 0x7f7cb0078290 2024-09-21T07:13:16.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cad600640 1 -- 172.21.15.27:0/262664802 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c9c0054d0 con 0x7f7cb0078290 2024-09-21T07:13:16.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cad600640 1 -- 172.21.15.27:0/262664802 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c010430 con 0x7f7cb0078290 2024-09-21T07:13:16.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.385+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb01ad260 con 0x7f7cb0078290 2024-09-21T07:13:16.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 Processor -- start 2024-09-21T07:13:16.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 -- start start 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cad600640 1 -- 172.21.15.27:0/262664802 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f7c9c004f40 con 0x7f7cb0078290 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168ef30a0 0x5621692c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562169239d50 con 0x562168efb280 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562169229180 con 0x562168ef30a0 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56216922cf90 con 0x562168f1bf60 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.389+0000 7f7cb5c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168ef30a0 0x5621692c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40738/0 (socket says 172.21.15.27:40738) 2024-09-21T07:13:16.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 -- 172.21.15.27:0/1526542520 learned_addr learned my addr 172.21.15.27:0/1526542520 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:16.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cafe00640 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 -- 172.21.15.27:0/1526542520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168ef30a0 msgr2=0x5621692c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168ef30a0 0x5621692c2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 -- 172.21.15.27:0/1526542520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 msgr2=0x5621692c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 -- 172.21.15.27:0/1526542520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621691f3a70 con 0x562168f1bf60 2024-09-21T07:13:16.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cafe00640 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.390+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692c2670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ca802acb0 tx=0x7f7ca8030420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe6f3648f4f0e339 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.391+0000 7f7ca2000640 1 -- 172.21.15.27:0/1526542520 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8041030 con 0x562168f1bf60 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.391+0000 7f7ca2000640 1 -- 172.21.15.27:0/1526542520 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca8003e20 con 0x562168f1bf60 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.391+0000 7f7ca2000640 1 -- 172.21.15.27:0/1526542520 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8036040 con 0x562168f1bf60 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.391+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/1526542520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 msgr2=0x5621692c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.391+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692c2670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ca802acb0 tx=0x7f7ca8030420 comp rx=0 tx=0).stop 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cad600640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c980b6710 0x7f7c980b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/1526542520 shutdown_connections 2024-09-21T07:13:16.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168ef30a0 0x5621692c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/1526542520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692c2670 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/1526542520 >> 172.21.15.27:0/1526542520 conn(0x562168f10350 msgr2=0x5621692c0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:16.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cad600640 1 -- 172.21.15.27:0/262664802 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7c9c0e8110 con 0x7f7cb0078290 2024-09-21T07:13:16.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/1526542520 shutdown_connections 2024-09-21T07:13:16.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c980b6710 0x7f7c980b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.392+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/1526542520 wait complete. 2024-09-21T07:13:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 -- start start 2024-09-21T07:13:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c980b6710 0x7f7c980b8c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8034040 tx=0x7f7ca802a6c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 Processor -- start 2024-09-21T07:13:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168ef30a0 0x5621692ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168efb280 0x5621692ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692da0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56216908a5f0 con 0x562168ef30a0 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5621691c4460 con 0x562168efb280 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.393+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621692d0df0 con 0x562168f1bf60 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cbe8c0740 10 client.0 _ll_register_callbacks cb 0x5621691d4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cb5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168ef30a0 0x5621692ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168efb280 0x5621692ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cb5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168ef30a0 0x5621692ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43436/0 (socket says 172.21.15.27:43436) 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cb5200640 1 -- 172.21.15.27:0/719274148 learned_addr learned my addr 172.21.15.27:0/719274148 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168efb280 0x5621692ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:57134/0 (socket says 172.21.15.27:57134) 2024-09-21T07:13:16.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692da0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cafe00640 1 -- 172.21.15.27:0/719274148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 msgr2=0x5621692da0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cafe00640 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692da0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cafe00640 1 -- 172.21.15.27:0/719274148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168ef30a0 msgr2=0x5621692ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cafe00640 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168ef30a0 0x5621692ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cafe00640 1 -- 172.21.15.27:0/719274148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562168f87540 con 0x562168efb280 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.394+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7cafe00640 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168efb280 0x5621692ce7b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0e53f0 tx=0x7f7c9c005460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd56a4a2117f8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7c96a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7c96a00640 1 -- 172.21.15.27:0/719274148 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0ad390 con 0x562168efb280 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621691c6a70 con 0x562168efb280 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7c96a00640 1 -- 172.21.15.27:0/719274148 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7c9c0ea0d0 con 0x562168efb280 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7c96a00640 1 -- 172.21.15.27:0/719274148 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c9c0ad1b0 con 0x562168efb280 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7cbe8c0740 10 client.25601 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:16.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5621691e16f0 con 0x562168efb280 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.395+0000 7f7c96a00640 1 -- 172.21.15.27:0/719274148 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5621691c6a70 con 0x562168efb280 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7c96a00640 1 -- 172.21.15.27:0/719274148 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7c9c0b21a0 con 0x562168efb280 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7cbe8c0740 10 client.25601 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7cbe8c0740 10 client.25601 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7cbe8c0740 20 client.25601 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eda8e0 0x5621692e0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7cbe8c0740 4 client.25601 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56216923bca0 con 0x562168eda8e0 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.396+0000 7f7cb5200640 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eda8e0 0x5621692e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.397+0000 7f7cb5200640 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eda8e0 0x5621692e0740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cc2a00fc82b8850 server_cookie=3a4edad1b4c575d0 in_seq=0 out_seq=0 2024-09-21T07:13:16.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.397+0000 7f7c96a00640 10 client.25601 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:16.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7c96a00640 1 -- 172.21.15.27:0/719274148 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56216923bca0 con 0x562168eda8e0 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7c96a00640 10 client.25601 handle_command_reply: tid=0 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7cbe8c0740 1 client.25601 shutdown 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168efb280 msgr2=0x5621692ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168efb280 0x5621692ce7b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0e53f0 tx=0x7f7c9c005460 comp rx=0 tx=0).stop 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 shutdown_connections 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eda8e0 0x5621692e0740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.398+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168f1bf60 0x5621692da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.399+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168efb280 0x5621692ce7b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.399+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/719274148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168ef30a0 0x5621692ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.399+0000 7f7c96a00640 0 client.25601 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.399+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 >> 172.21.15.27:0/719274148 conn(0x562168f10350 msgr2=0x5621692c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.399+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 shutdown_connections 2024-09-21T07:13:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.399+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/719274148 wait complete. 2024-09-21T07:13:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.399+0000 7f7cbe8c0740 20 client.25601 trim_cache size 0 max 16384 2024-09-21T07:13:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 Processor -- start 2024-09-21T07:13:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 -- start start 2024-09-21T07:13:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5621692dea80 con 0x562168efb280 2024-09-21T07:13:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5621692df150 con 0x562168f1bf60 2024-09-21T07:13:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.418+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621692df820 con 0x562168ef30a0 2024-09-21T07:13:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43440/0 (socket says 172.21.15.27:43440) 2024-09-21T07:13:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cafe00640 1 -- 172.21.15.27:0/3844091935 learned_addr learned my addr 172.21.15.27:0/3844091935 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cb5200640 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cb5200640 1 -- 172.21.15.27:0/3844091935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 msgr2=0x5621692c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cb5200640 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cb5200640 1 -- 172.21.15.27:0/3844091935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 msgr2=0x5621692ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cb5200640 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cb5200640 1 -- 172.21.15.27:0/3844091935 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562168edab10 con 0x562168ef30a0 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.419+0000 7f7cafe00640 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.420+0000 7f7cb5200640 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692ce230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5621692e10b0 tx=0x7f7ca4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f93a3ff4d9ae10d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.420+0000 7f7ca1600640 1 -- 172.21.15.27:0/3844091935 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4003d20 con 0x562168ef30a0 2024-09-21T07:13:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.420+0000 7f7ca1600640 1 -- 172.21.15.27:0/3844091935 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca4003ec0 con 0x562168ef30a0 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.420+0000 7f7ca1600640 1 -- 172.21.15.27:0/3844091935 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4024470 con 0x562168ef30a0 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/3844091935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 msgr2=0x5621692ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692ce230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5621692e10b0 tx=0x7f7ca4009260 comp rx=0 tx=0).stop 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/3844091935 shutdown_connections 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/3844091935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692ce230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/3844091935 >> 172.21.15.27:0/3844091935 conn(0x562168f10350 msgr2=0x5621692de180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/3844091935 shutdown_connections 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/3844091935 wait complete. 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.421+0000 7f7cbe8c0740 1 -- start start 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 1 Processor -- start 2024-09-21T07:13:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692e7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5621692e05c0 con 0x562168efb280 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5621692dfef0 con 0x562168f1bf60 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621692e0c90 con 0x562168ef30a0 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cbe8c0740 10 client.0 _ll_register_callbacks cb 0x7f7c9c004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cb5200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.422+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40770/0 (socket says 172.21.15.27:40770) 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 -- 172.21.15.27:0/7126586 learned_addr learned my addr 172.21.15.27:0/7126586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5c00640 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 -- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 msgr2=0x5621692e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692e7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 -- 172.21.15.27:0/7126586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 msgr2=0x5621692c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 -- 172.21.15.27:0/7126586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621692e7f20 con 0x562168ef30a0 2024-09-21T07:13:16.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7cb5200640 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692c76c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4003c30 tx=0x7f7ca4003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7c1b0d67ba1713 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:16.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.423+0000 7f7c96a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.424+0000 7f7c96a00640 1 -- 172.21.15.27:0/7126586 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4030020 con 0x562168ef30a0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.424+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca4003ec0 con 0x562168ef30a0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.424+0000 7f7c96a00640 1 -- 172.21.15.27:0/7126586 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7ca4005430 con 0x562168ef30a0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.424+0000 7f7c96a00640 1 -- 172.21.15.27:0/7126586 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca403b050 con 0x562168ef30a0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.424+0000 7f7cbe8c0740 10 client.25606 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.424+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5621692e9180 con 0x562168ef30a0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.424+0000 7f7c96a00640 1 -- 172.21.15.27:0/7126586 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca4003ec0 con 0x562168ef30a0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7c96a00640 1 -- 172.21.15.27:0/7126586 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7ca40247d0 con 0x562168ef30a0 2024-09-21T07:13:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7cbe8c0740 10 client.25606 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7cbe8c0740 10 client.25606 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T07:13:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7cbe8c0740 20 client.25606 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eafc90 0x56216934e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7cbe8c0740 4 client.25606 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:13:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "status"}) -- 0x562169237750 con 0x562168eafc90 2024-09-21T07:13:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.425+0000 7f7cafe00640 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eafc90 0x56216934e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cafe00640 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eafc90 0x56216934e9b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4d2f97fdaabb7bc server_cookie=a0c1106e4339bbe1 in_seq=0 out_seq=0 2024-09-21T07:13:16.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7c96a00640 10 client.25606 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:16.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7c96a00640 1 -- 172.21.15.27:0/7126586 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562169237750 con 0x562168eafc90 2024-09-21T07:13:16.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7c96a00640 10 client.25606 handle_command_reply: tid=0 2024-09-21T07:13:16.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 client.25606 shutdown 2024-09-21T07:13:16.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 msgr2=0x5621692c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692c76c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4003c30 tx=0x7f7ca4003c60 comp rx=0 tx=0).stop 2024-09-21T07:13:16.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 shutdown_connections 2024-09-21T07:13:16.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x562168eafc90 0x56216934e9b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562168f1bf60 0x5621692e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562168efb280 0x5621692c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 --2- 172.21.15.27:0/7126586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562168ef30a0 0x5621692c76c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 >> 172.21.15.27:0/7126586 conn(0x562168f10350 msgr2=0x5621692c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:16.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.426+0000 7f7c96a00640 0 client.25606 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:16.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 shutdown_connections 2024-09-21T07:13:16.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cbe8c0740 1 -- 172.21.15.27:0/7126586 wait complete. 2024-09-21T07:13:16.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cbe8c0740 20 client.25606 trim_cache size 0 max 16384 2024-09-21T07:13:16.435 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:16.435 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:13:16.435 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24740, 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 147.89355586799999, 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:13:16.436 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T07:13:16.437 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 225.07470376699999 2024-09-21T07:13:16.437 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:16.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c980b6710 msgr2=0x7f7c980b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c980b6710 0x7f7c980b8c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8034040 tx=0x7f7ca802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 msgr2=0x7f7cb01a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 0x7f7cb01a0e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c00edc0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 shutdown_connections 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb0078290 0x7f7cb01a0e40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c980b6710 0x7f7c980b8c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb0114570 0x7f7cb01a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 --2- 172.21.15.27:0/262664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb0077940 0x7f7cb01a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.427+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 >> 172.21.15.27:0/262664802 conn(0x7f7cb0101680 msgr2=0x7f7cb0116570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.428+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 shutdown_connections 2024-09-21T07:13:16.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:16.428+0000 7f7cb6600640 1 -- 172.21.15.27:0/262664802 wait complete. 2024-09-21T07:13:17.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:17 smithi027 ceph-mon[30779]: pgmap v385: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:13:17.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.g status 2024-09-21T07:13:17.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:17 smithi153 ceph-mon[34747]: pgmap v385: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:13:17.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:17 smithi119 ceph-mon[34519]: pgmap v385: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:13:17.720 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.293+0000 7fadbfe00640 1 -- 172.21.15.27:0/1314001389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 msgr2=0x7fadc0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.293+0000 7fadbfe00640 1 --2- 172.21.15.27:0/1314001389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc0105fd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009860 tx=0x7fadb002a340 comp rx=0 tx=0).stop 2024-09-21T07:13:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.294+0000 7fadbfe00640 1 -- 172.21.15.27:0/1314001389 shutdown_connections 2024-09-21T07:13:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.294+0000 7fadbfe00640 1 --2- 172.21.15.27:0/1314001389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadc010be50 0x7fadc0112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.294+0000 7fadbfe00640 1 --2- 172.21.15.27:0/1314001389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc010b500 0x7fadc010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.294+0000 7fadbfe00640 1 --2- 172.21.15.27:0/1314001389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc0105fd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.294+0000 7fadbfe00640 1 -- 172.21.15.27:0/1314001389 >> 172.21.15.27:0/1314001389 conn(0x7fadc0101670 msgr2=0x7fadc0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.294+0000 7fadbfe00640 1 -- 172.21.15.27:0/1314001389 shutdown_connections 2024-09-21T07:13:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.294+0000 7fadbfe00640 1 -- 172.21.15.27:0/1314001389 wait complete. 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 Processor -- start 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 -- start start 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadc010b500 0x7fadc019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc010be50 0x7fadc01a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fadc0115b10 con 0x7fadc0105bc0 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadc0115990 con 0x7fadc010be50 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.295+0000 7fadbfe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fadc0115c90 con 0x7fadc010b500 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56676/0 (socket says 172.21.15.27:56676) 2024-09-21T07:13:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 -- 172.21.15.27:0/805266458 learned_addr learned my addr 172.21.15.27:0/805266458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbf400640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc010be50 0x7fadc01a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbe000640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadc010b500 0x7fadc019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 -- 172.21.15.27:0/805266458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadc010b500 msgr2=0x7fadc019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadc010b500 0x7fadc019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 -- 172.21.15.27:0/805266458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc010be50 msgr2=0x7fadc01a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc010be50 0x7fadc01a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbea00640 1 -- 172.21.15.27:0/805266458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc01a2450 con 0x7fadc0105bc0 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.296+0000 7fadbf400640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc010be50 0x7fadc01a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.297+0000 7fadbea00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc019c4e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fadb00303f0 tx=0x7fadb0032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.297+0000 7fadc4a00640 1 -- 172.21.15.27:0/805266458 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0040020 con 0x7fadc0105bc0 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.297+0000 7fadc4a00640 1 -- 172.21.15.27:0/805266458 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb0030dd0 con 0x7fadc0105bc0 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.297+0000 7fadc4a00640 1 -- 172.21.15.27:0/805266458 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0037040 con 0x7fadc0105bc0 2024-09-21T07:13:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.297+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc01a26e0 con 0x7fadc0105bc0 2024-09-21T07:13:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.297+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc01a8ea0 con 0x7fadc0105bc0 2024-09-21T07:13:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.300+0000 7fadcca5c740 1 Processor -- start 2024-09-21T07:13:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.300+0000 7fadcca5c740 1 -- start start 2024-09-21T07:13:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadc4a00640 1 -- 172.21.15.27:0/805266458 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fadb0043020 con 0x7fadc0105bc0 2024-09-21T07:13:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fa96670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6c70a0 0x559f2fa96bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6cf280 0x559f2fa970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadcca5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559f2fa0dd50 con 0x559f2f6c70a0 2024-09-21T07:13:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadcca5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559f2f9fd180 con 0x559f2f6eff60 2024-09-21T07:13:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadcca5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f2fa00f90 con 0x559f2f6cf280 2024-09-21T07:13:18.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadbea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6c70a0 0x559f2fa96bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadbea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6c70a0 0x559f2fa96bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56690/0 (socket says 172.21.15.27:56690) 2024-09-21T07:13:18.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadbea00640 1 -- 172.21.15.27:0/557210204 learned_addr learned my addr 172.21.15.27:0/557210204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:18.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fa96670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.301+0000 7fadbe000640 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6cf280 0x559f2fa970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.302+0000 7fadbe000640 1 -- 172.21.15.27:0/557210204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 msgr2=0x559f2fa96670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.302+0000 7fadbe000640 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fa96670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.302+0000 7fadbe000640 1 -- 172.21.15.27:0/557210204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6c70a0 msgr2=0x559f2fa96bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.302+0000 7fadbe000640 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6c70a0 0x559f2fa96bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.302+0000 7fadbe000640 1 -- 172.21.15.27:0/557210204 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2f9c7a70 con 0x559f2f6cf280 2024-09-21T07:13:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.302+0000 7fadbf400640 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fa96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.302+0000 7fadbe000640 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6cf280 0x559f2fa970f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fadc006c610 tx=0x7fadac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8101eaf5478886 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.303+0000 7fadabe00640 1 -- 172.21.15.27:0/557210204 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac004050 con 0x559f2f6cf280 2024-09-21T07:13:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.303+0000 7fadabe00640 1 -- 172.21.15.27:0/557210204 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac0041f0 con 0x559f2f6cf280 2024-09-21T07:13:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.303+0000 7fadabe00640 1 -- 172.21.15.27:0/557210204 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac019690 con 0x559f2f6cf280 2024-09-21T07:13:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.303+0000 7fadc4a00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fada40b6710 0x7fada40b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.303+0000 7fadc4a00640 1 -- 172.21.15.27:0/805266458 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fadb0107860 con 0x7fadc0105bc0 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.303+0000 7fadbf400640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fada40b6710 0x7fada40b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 -- 172.21.15.27:0/557210204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6cf280 msgr2=0x559f2fa970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6cf280 0x559f2fa970f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fadc006c610 tx=0x7fadac011810 comp rx=0 tx=0).stop 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 -- 172.21.15.27:0/557210204 shutdown_connections 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6cf280 0x559f2fa970f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6c70a0 0x559f2fa96bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 --2- 172.21.15.27:0/557210204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fa96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 -- 172.21.15.27:0/557210204 >> 172.21.15.27:0/557210204 conn(0x559f2f6e4350 msgr2=0x559f2fa94560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadbf400640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fada40b6710 0x7fada40b8c20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009fd0 tx=0x7fadb8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 -- 172.21.15.27:0/557210204 shutdown_connections 2024-09-21T07:13:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.304+0000 7fadcca5c740 1 -- 172.21.15.27:0/557210204 wait complete. 2024-09-21T07:13:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 -- start start 2024-09-21T07:13:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 Processor -- start 2024-09-21T07:13:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2faa2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2faae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559f2f85e5f0 con 0x559f2f6cf280 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559f2f998460 con 0x559f2f6eff60 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.305+0000 7fadcca5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f2faa4e50 con 0x559f2f6c70a0 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadcca5c740 10 client.0 _ll_register_callbacks cb 0x559f2f9a8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbe000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2faa2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42414/0 (socket says 172.21.15.27:42414) 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 -- 172.21.15.27:0/1817518112 learned_addr learned my addr 172.21.15.27:0/1817518112 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbf400640 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2faae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 -- 172.21.15.27:0/1817518112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 msgr2=0x559f2faae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2faae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 -- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 msgr2=0x559f2faa2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2faa2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbea00640 1 -- 172.21.15.27:0/1817518112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2f75b540 con 0x559f2f6c70a0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.306+0000 7fadbe000640 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2faa2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fadbea00640 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fadb00cce90 tx=0x7fadb0032f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a62a8031c80c56b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fada8c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fada8c00640 1 -- 172.21.15.27:0/1817518112 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00c97e0 con 0x559f2f6c70a0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fada8c00640 1 -- 172.21.15.27:0/1817518112 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fadb0107450 con 0x559f2f6c70a0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f2f99aa70 con 0x559f2f6c70a0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fada8c00640 1 -- 172.21.15.27:0/1817518112 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fadb00cc900 con 0x559f2f6c70a0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fadcca5c740 10 client.25618 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.307+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f2f9b56f0 con 0x559f2f6c70a0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fada8c00640 1 -- 172.21.15.27:0/1817518112 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559f2f99aa70 con 0x559f2f6c70a0 2024-09-21T07:13:18.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fada8c00640 1 -- 172.21.15.27:0/1817518112 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fadb004d060 con 0x559f2f6c70a0 2024-09-21T07:13:18.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fadcca5c740 10 client.25618 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:18.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fadcca5c740 10 client.25618 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T07:13:18.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fadcca5c740 20 client.25618 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:18.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fadcca5c740 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f6ae8e0 0x559f2fab4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fadcca5c740 4 client.25618 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:18.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f2fa0fca0 con 0x559f2f6ae8e0 2024-09-21T07:13:18.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.308+0000 7fadbe000640 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f6ae8e0 0x559f2fab4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.309+0000 7fadbe000640 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f6ae8e0 0x559f2fab4510 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e30e3e41ebfdc1d server_cookie=45310a579879115d in_seq=0 out_seq=0 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.309+0000 7fada8c00640 10 client.25618 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fada8c00640 1 -- 172.21.15.27:0/1817518112 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f2fa0fca0 con 0x559f2f6ae8e0 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fada8c00640 10 client.25618 handle_command_reply: tid=0 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 client.25618 shutdown 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 msgr2=0x559f2faa21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fadb00cce90 tx=0x7fadb0032f10 comp rx=0 tx=0).stop 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 shutdown_connections 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f6ae8e0 0x559f2fab4510 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2faae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2faa2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 --2- 172.21.15.27:0/1817518112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fada8c00640 0 client.25618 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:18.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.310+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 >> 172.21.15.27:0/1817518112 conn(0x559f2f6e4350 msgr2=0x559f2fa99850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:18.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.311+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 shutdown_connections 2024-09-21T07:13:18.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.311+0000 7fadcca5c740 1 -- 172.21.15.27:0/1817518112 wait complete. 2024-09-21T07:13:18.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.311+0000 7fadcca5c740 20 client.25618 trim_cache size 0 max 16384 2024-09-21T07:13:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 Processor -- start 2024-09-21T07:13:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 -- start start 2024-09-21T07:13:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6cf280 0x559f2faa2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6eff60 0x559f2fa9d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559f2fab2850 con 0x559f2f6eff60 2024-09-21T07:13:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559f2fab2f20 con 0x559f2f6cf280 2024-09-21T07:13:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.330+0000 7fadcca5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f2fab35f0 con 0x559f2f6c70a0 2024-09-21T07:13:18.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6cf280 0x559f2faa2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6eff60 0x559f2fa9d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6cf280 0x559f2faa2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34130/0 (socket says 172.21.15.27:34130) 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 -- 172.21.15.27:0/647272936 learned_addr learned my addr 172.21.15.27:0/647272936 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbea00640 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 -- 172.21.15.27:0/647272936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 msgr2=0x559f2faa21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 -- 172.21.15.27:0/647272936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6eff60 msgr2=0x559f2fa9d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6eff60 0x559f2fa9d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 -- 172.21.15.27:0/647272936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2f9b56f0 con 0x559f2f6cf280 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbea00640 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.331+0000 7fadbe000640 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6cf280 0x559f2faa2700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadac005fd0 tx=0x7fadac009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48de2e04c204362 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadab400640 1 -- 172.21.15.27:0/647272936 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac005040 con 0x559f2f6cf280 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadab400640 1 -- 172.21.15.27:0/647272936 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac0035f0 con 0x559f2f6cf280 2024-09-21T07:13:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadab400640 1 -- 172.21.15.27:0/647272936 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac004460 con 0x559f2f6cf280 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadcca5c740 1 -- 172.21.15.27:0/647272936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6cf280 msgr2=0x559f2faa2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadcca5c740 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6cf280 0x559f2faa2700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadac005fd0 tx=0x7fadac009600 comp rx=0 tx=0).stop 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadcca5c740 1 -- 172.21.15.27:0/647272936 shutdown_connections 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadcca5c740 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6eff60 0x559f2fa9d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadcca5c740 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6cf280 0x559f2faa2700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadcca5c740 1 --2- 172.21.15.27:0/647272936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2faa21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.332+0000 7fadcca5c740 1 -- 172.21.15.27:0/647272936 >> 172.21.15.27:0/647272936 conn(0x559f2f6e4350 msgr2=0x559f2fab1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.333+0000 7fadcca5c740 1 -- 172.21.15.27:0/647272936 shutdown_connections 2024-09-21T07:13:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.333+0000 7fadcca5c740 1 -- 172.21.15.27:0/647272936 wait complete. 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.333+0000 7fadcca5c740 1 -- start start 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.333+0000 7fadcca5c740 1 Processor -- start 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2fab6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2fab72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadcca5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fabbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadcca5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559f2fab4390 con 0x559f2f6cf280 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadcca5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559f2fab3cc0 con 0x559f2f6eff60 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadcca5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f2fab4a60 con 0x559f2f6c70a0 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadcca5c740 10 client.0 _ll_register_callbacks cb 0x7fadb0030f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadbe000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2fab72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fabbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadbea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2fab6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fabbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34142/0 (socket says 172.21.15.27:34142) 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.334+0000 7fadbf400640 1 -- 172.21.15.27:0/3288948377 learned_addr learned my addr 172.21.15.27:0/3288948377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadbea00640 1 -- 172.21.15.27:0/3288948377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 msgr2=0x559f2fabbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadbea00640 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fabbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadbea00640 1 -- 172.21.15.27:0/3288948377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 msgr2=0x559f2fab72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadbea00640 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2fab72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadbea00640 1 -- 172.21.15.27:0/3288948377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2fabc280 con 0x559f2f6c70a0 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadbf400640 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fabbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadbea00640 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2fab6cf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fadb0104bd0 tx=0x7fadb0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=362114b9932ba4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fada8c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fada8c00640 1 -- 172.21.15.27:0/3288948377 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00ccae0 con 0x559f2f6c70a0 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadac0035f0 con 0x559f2f6c70a0 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.335+0000 7fada8c00640 1 -- 172.21.15.27:0/3288948377 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fadb00f0b10 con 0x559f2f6c70a0 2024-09-21T07:13:18.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fada8c00640 1 -- 172.21.15.27:0/3288948377 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fadb00ccf60 con 0x559f2f6c70a0 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fadcca5c740 10 client.25624 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f2fabc280 con 0x559f2f6c70a0 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fada8c00640 1 -- 172.21.15.27:0/3288948377 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac0035f0 con 0x559f2f6c70a0 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fada8c00640 1 -- 172.21.15.27:0/3288948377 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fadb00cbe60 con 0x559f2f6c70a0 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fadcca5c740 10 client.25624 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fadcca5c740 10 client.25624 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fadcca5c740 20 client.25624 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.336+0000 7fadcca5c740 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f683c90 0x559f2fb22be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.337+0000 7fadcca5c740 4 client.25624 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.337+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "status"}) -- 0x559f2fa0b750 con 0x559f2f683c90 2024-09-21T07:13:18.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.337+0000 7fadbe000640 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f683c90 0x559f2fb22be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.337+0000 7fadbe000640 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f683c90 0x559f2fb22be0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=910141f48f1f8e75 server_cookie=3c8792cadf511e5 in_seq=0 out_seq=0 2024-09-21T07:13:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.337+0000 7fada8c00640 10 client.25624 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fada8c00640 1 -- 172.21.15.27:0/3288948377 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559f2fa0b750 con 0x559f2f683c90 2024-09-21T07:13:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fada8c00640 10 client.25624 handle_command_reply: tid=0 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 client.25624 shutdown 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 msgr2=0x559f2fab6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2fab6cf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fadb0104bd0 tx=0x7fadb0005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 shutdown_connections 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559f2f683c90 0x559f2fb22be0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f2f6eff60 0x559f2fabbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559f2f6cf280 0x559f2fab72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fada8c00640 0 client.25624 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 --2- 172.21.15.27:0/3288948377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f2f6c70a0 0x559f2fab6cf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 >> 172.21.15.27:0/3288948377 conn(0x559f2f6e4350 msgr2=0x559f2faae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 shutdown_connections 2024-09-21T07:13:18.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 1 -- 172.21.15.27:0/3288948377 wait complete. 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.338+0000 7fadcca5c740 20 client.25624 trim_cache size 0 max 16384 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24748, 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 150.81284916199999, 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:13:18.344 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 220.870885163 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fada40b6710 msgr2=0x7fada40b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fada40b6710 0x7fada40b8c20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009fd0 tx=0x7fadb8009500 comp rx=0 tx=0).stop 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 msgr2=0x7fadc019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc019c4e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fadb00303f0 tx=0x7fadb0032da0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 shutdown_connections 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadc010b500 0x7fadc019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fada40b6710 0x7fada40b8c20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc010be50 0x7fadc01a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 --2- 172.21.15.27:0/805266458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadc0105bc0 0x7fadc019c4e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:18.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.339+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 >> 172.21.15.27:0/805266458 conn(0x7fadc0101670 msgr2=0x7fadc010ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:18.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.340+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 shutdown_connections 2024-09-21T07:13:18.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:18.340+0000 7fadbfe00640 1 -- 172.21.15.27:0/805266458 wait complete. 2024-09-21T07:13:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:19 smithi027 ceph-mon[30779]: pgmap v386: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:13:19.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T07:13:19.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:19 smithi153 ceph-mon[34747]: pgmap v386: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:13:19.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:19 smithi119 ceph-mon[34519]: pgmap v386: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:13:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:20.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 -- 172.21.15.27:0/3963857591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf410b500 msgr2=0x7fbbf410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 --2- 172.21.15.27:0/3963857591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf410b500 0x7fbbf410b910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009900 tx=0x7fbbe002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 -- 172.21.15.27:0/3963857591 shutdown_connections 2024-09-21T07:13:20.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 --2- 172.21.15.27:0/3963857591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf410be50 0x7fbbf4112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 --2- 172.21.15.27:0/3963857591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf410b500 0x7fbbf410b910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 --2- 172.21.15.27:0/3963857591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf4105bc0 0x7fbbf4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 -- 172.21.15.27:0/3963857591 >> 172.21.15.27:0/3963857591 conn(0x7fbbf4101630 msgr2=0x7fbbf4103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:20.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 -- 172.21.15.27:0/3963857591 shutdown_connections 2024-09-21T07:13:20.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.206+0000 7fbbfac00640 1 -- 172.21.15.27:0/3963857591 wait complete. 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.207+0000 7fbbfac00640 1 Processor -- start 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.207+0000 7fbbfac00640 1 -- start start 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.207+0000 7fbbfac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf4105bc0 0x7fbbf419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbfac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf410b500 0x7fbbf419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbfac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf410be50 0x7fbbf41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbfac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4115b10 con 0x7fbbf4105bc0 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbfac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4115990 con 0x7fbbf410b500 2024-09-21T07:13:20.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbfac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4115c90 con 0x7fbbf410be50 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf4105bc0 0x7fbbf419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbf8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf410b500 0x7fbbf419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf4105bc0 0x7fbbf419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56736/0 (socket says 172.21.15.27:56736) 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbf9800640 1 -- 172.21.15.27:0/1967024751 learned_addr learned my addr 172.21.15.27:0/1967024751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.208+0000 7fbbfa200640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf410be50 0x7fbbf41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbfa200640 1 -- 172.21.15.27:0/1967024751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf410b500 msgr2=0x7fbbf419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbfa200640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf410b500 0x7fbbf419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbfa200640 1 -- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf4105bc0 msgr2=0x7fbbf419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbfa200640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf4105bc0 0x7fbbf419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbfa200640 1 -- 172.21.15.27:0/1967024751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf41a2410 con 0x7fbbf410be50 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf4105bc0 0x7fbbf419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:20.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbfa200640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf410be50 0x7fbbf41a1ce0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009fd0 tx=0x7fbbec00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:20.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbf2000640 1 -- 172.21.15.27:0/1967024751 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00ce20 con 0x7fbbf410be50 2024-09-21T07:13:20.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf41a2700 con 0x7fbbf410be50 2024-09-21T07:13:20.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.209+0000 7fbbf2000640 1 -- 172.21.15.27:0/1967024751 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbec005020 con 0x7fbbf410be50 2024-09-21T07:13:20.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.210+0000 7fbbf2000640 1 -- 172.21.15.27:0/1967024751 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec010690 con 0x7fbbf410be50 2024-09-21T07:13:20.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.210+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf41a8e60 con 0x7fbbf410be50 2024-09-21T07:13:20.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.214+0000 7fbc02ff7740 1 Processor -- start 2024-09-21T07:13:20.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbbf2000640 1 -- 172.21.15.27:0/1967024751 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fbbec0054d0 con 0x7fbbf410be50 2024-09-21T07:13:20.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbc02ff7740 1 -- start start 2024-09-21T07:13:20.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70894670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af70894bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704cd280 0x55af708950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af7080bd50 con 0x55af704cd280 2024-09-21T07:13:20.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55af707fb180 con 0x55af704c50a0 2024-09-21T07:13:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.215+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af707fef90 con 0x55af704edf60 2024-09-21T07:13:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af70894bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70894670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af70894bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34178/0 (socket says 172.21.15.27:34178) 2024-09-21T07:13:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 -- 172.21.15.27:0/1031764141 learned_addr learned my addr 172.21.15.27:0/1031764141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704cd280 0x55af708950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 -- 172.21.15.27:0/1031764141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 msgr2=0x55af70894670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70894670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 -- 172.21.15.27:0/1031764141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704cd280 msgr2=0x55af708950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704cd280 0x55af708950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 -- 172.21.15.27:0/1031764141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af707c5a70 con 0x55af704c50a0 2024-09-21T07:13:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704cd280 0x55af708950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:13:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.216+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af70894bb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8011630 tx=0x7fbbe8011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62070245ec6321f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.217+0000 7fbbf2000640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbbdc0b6790 0x7fbbdc0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.217+0000 7fbbe6a00640 1 -- 172.21.15.27:0/1031764141 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe801e070 con 0x55af704c50a0 2024-09-21T07:13:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.217+0000 7fbbe6a00640 1 -- 172.21.15.27:0/1031764141 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe800fba0 con 0x55af704c50a0 2024-09-21T07:13:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.217+0000 7fbbf2000640 1 -- 172.21.15.27:0/1967024751 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fbbec0e8290 con 0x7fbbf410be50 2024-09-21T07:13:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.217+0000 7fbbe6a00640 1 -- 172.21.15.27:0/1031764141 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8019c80 con 0x55af704c50a0 2024-09-21T07:13:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.217+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbbdc0b6790 0x7fbbdc0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1031764141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 msgr2=0x55af70894bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af70894bb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8011630 tx=0x7fbbe8011af0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbbdc0b6790 0x7fbbdc0b8ca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009230 tx=0x7fbbe0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1031764141 shutdown_connections 2024-09-21T07:13:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704cd280 0x55af708950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af70894bb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1031764141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70894670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1031764141 >> 172.21.15.27:0/1031764141 conn(0x55af704e2350 msgr2=0x55af70892560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1031764141 shutdown_connections 2024-09-21T07:13:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.218+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1031764141 wait complete. 2024-09-21T07:13:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.219+0000 7fbc02ff7740 1 -- start start 2024-09-21T07:13:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.219+0000 7fbc02ff7740 1 Processor -- start 2024-09-21T07:13:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.219+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af708a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.219+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704cd280 0x55af708a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.219+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704edf60 0x55af708ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.219+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af7065c5f0 con 0x55af704edf60 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55af70796460 con 0x55af704c50a0 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af708a2ea0 con 0x55af704cd280 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbc02ff7740 10 client.0 _ll_register_callbacks cb 0x55af707a6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af708a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af708a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34182/0 (socket says 172.21.15.27:34182) 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 -- 172.21.15.27:0/1575903969 learned_addr learned my addr 172.21.15.27:0/1575903969 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704cd280 0x55af708a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbfa200640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704edf60 0x55af708ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 -- 172.21.15.27:0/1575903969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704cd280 msgr2=0x55af708a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704cd280 0x55af708a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 -- 172.21.15.27:0/1575903969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704edf60 msgr2=0x55af708ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704edf60 0x55af708ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 -- 172.21.15.27:0/1575903969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af70559540 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704cd280 0x55af708a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af708a01a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbbe801c040 tx=0x7fbbe8019600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71323eb7eea0058b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.220+0000 7fbbdb400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.221+0000 7fbbdb400640 1 -- 172.21.15.27:0/1575903969 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8026020 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.221+0000 7fbbdb400640 1 -- 172.21.15.27:0/1575903969 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fbbe80055c0 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.221+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af70798a70 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.221+0000 7fbbdb400640 1 -- 172.21.15.27:0/1575903969 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe8031050 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.221+0000 7fbc02ff7740 10 client.25637 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.221+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af707b36f0 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbbdb400640 1 -- 172.21.15.27:0/1575903969 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55af70798a70 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbbdb400640 1 -- 172.21.15.27:0/1575903969 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbe8021050 con 0x55af704c50a0 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbc02ff7740 10 client.25637 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbc02ff7740 10 client.25637 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:13:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbc02ff7740 20 client.25637 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af704ac8e0 0x55af708b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbc02ff7740 4 client.25637 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af7080dca0 con 0x55af704ac8e0 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.222+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af704ac8e0 0x55af708b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.223+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af704ac8e0 0x55af708b23f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d402314c684eda server_cookie=baa801ba6d75bc10 in_seq=0 out_seq=0 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.223+0000 7fbbdb400640 10 client.25637 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbbdb400640 1 -- 172.21.15.27:0/1575903969 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af7080dca0 con 0x55af704ac8e0 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbbdb400640 10 client.25637 handle_command_reply: tid=0 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 client.25637 shutdown 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 msgr2=0x55af708a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af708a01a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbbe801c040 tx=0x7fbbe8019600 comp rx=0 tx=0).stop 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 shutdown_connections 2024-09-21T07:13:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af704ac8e0 0x55af708b23f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704cd280 0x55af708a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704edf60 0x55af708ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1575903969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704c50a0 0x55af708a01a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbbdb400640 0 client.25637 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 >> 172.21.15.27:0/1575903969 conn(0x55af704e2350 msgr2=0x55af70896060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 shutdown_connections 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.224+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1575903969 wait complete. 2024-09-21T07:13:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.225+0000 7fbc02ff7740 20 client.25637 trim_cache size 0 max 16384 2024-09-21T07:13:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.243+0000 7fbc02ff7740 1 Processor -- start 2024-09-21T07:13:20.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbc02ff7740 1 -- start start 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708ac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70899750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af708b0730 con 0x55af704c50a0 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55af708b0e00 con 0x55af704cd280 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af708b14d0 con 0x55af704edf60 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56768/0 (socket says 172.21.15.27:56768) 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbbf9800640 1 -- 172.21.15.27:0/1430658220 learned_addr learned my addr 172.21.15.27:0/1430658220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.244+0000 7fbbfa200640 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70899750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbf9800640 1 -- 172.21.15.27:0/1430658220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 msgr2=0x55af70899750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70899750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbf9800640 1 -- 172.21.15.27:0/1430658220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 msgr2=0x55af708ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708ac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbf9800640 1 -- 172.21.15.27:0/1430658220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af70896960 con 0x55af704c50a0 2024-09-21T07:13:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbf9800640 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708b0f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8011b30 tx=0x7fbbe800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd3e03b3f00ca31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbe6000640 1 -- 172.21.15.27:0/1430658220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8026070 con 0x55af704c50a0 2024-09-21T07:13:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbe6000640 1 -- 172.21.15.27:0/1430658220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8005410 con 0x55af704c50a0 2024-09-21T07:13:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.245+0000 7fbbe6000640 1 -- 172.21.15.27:0/1430658220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe801e040 con 0x55af704c50a0 2024-09-21T07:13:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1430658220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 msgr2=0x55af708b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708b0f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8011b30 tx=0x7fbbe800f6f0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1430658220 shutdown_connections 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af70899750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/1430658220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708b0f80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1430658220 >> 172.21.15.27:0/1430658220 conn(0x55af704e2350 msgr2=0x55af7089b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1430658220 shutdown_connections 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.246+0000 7fbc02ff7740 1 -- 172.21.15.27:0/1430658220 wait complete. 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.247+0000 7fbc02ff7740 1 -- start start 2024-09-21T07:13:20.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.247+0000 7fbc02ff7740 1 Processor -- start 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.247+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbc02ff7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af708b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af708b2270 con 0x55af704c50a0 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55af708b1ba0 con 0x55af704cd280 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbc02ff7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af708b2940 con 0x55af704edf60 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbc02ff7740 10 client.0 _ll_register_callbacks cb 0x7fbbe801ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708bae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56772/0 (socket says 172.21.15.27:56772) 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf9800640 1 -- 172.21.15.27:0/478328467 learned_addr learned my addr 172.21.15.27:0/478328467 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbfa200640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af708b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf8e00640 1 -- 172.21.15.27:0/478328467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 msgr2=0x55af708b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af708b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf8e00640 1 -- 172.21.15.27:0/478328467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 msgr2=0x55af708bae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708bae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf8e00640 1 -- 172.21.15.27:0/478328467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af708b4f50 con 0x55af704cd280 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.248+0000 7fbbf9800640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.249+0000 7fbbf8e00640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708b42d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbbe002a910 tx=0x7fbbe0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d32dab1469c25a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.249+0000 7fbbdb400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.249+0000 7fbbdb400640 1 -- 172.21.15.27:0/478328467 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0005040 con 0x55af704cd280 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.249+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe8005410 con 0x55af704cd280 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.249+0000 7fbbdb400640 1 -- 172.21.15.27:0/478328467 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fbbe00051e0 con 0x55af704cd280 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.249+0000 7fbbdb400640 1 -- 172.21.15.27:0/478328467 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe0060050 con 0x55af704cd280 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.249+0000 7fbc02ff7740 10 client.25643 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.250+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af7091d8c0 con 0x55af704cd280 2024-09-21T07:13:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.250+0000 7fbbdb400640 1 -- 172.21.15.27:0/478328467 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8005410 con 0x55af704cd280 2024-09-21T07:13:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.250+0000 7fbbdb400640 1 -- 172.21.15.27:0/478328467 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbe0044040 con 0x55af704cd280 2024-09-21T07:13:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.250+0000 7fbc02ff7740 10 client.25643 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.250+0000 7fbc02ff7740 10 client.25643 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:13:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.250+0000 7fbc02ff7740 20 client.25643 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.250+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af70481c90 0x55af70921160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.251+0000 7fbc02ff7740 4 client.25643 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.251+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "status"}) -- 0x55af70809750 con 0x55af70481c90 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.251+0000 7fbbf9800640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af70481c90 0x55af70921160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.251+0000 7fbbf9800640 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af70481c90 0x55af70921160 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e845049e7566a87 server_cookie=f7399ea6f2db2100 in_seq=0 out_seq=0 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.251+0000 7fbbdb400640 10 client.25643 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbbdb400640 1 -- 172.21.15.27:0/478328467 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55af70809750 con 0x55af70481c90 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbbdb400640 10 client.25643 handle_command_reply: tid=0 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 client.25643 shutdown 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 msgr2=0x55af708b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708b42d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbbe002a910 tx=0x7fbbe0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 shutdown_connections 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af70481c90 0x55af70921160 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af704edf60 0x55af708b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af704cd280 0x55af708b42d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 --2- 172.21.15.27:0/478328467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af704c50a0 0x55af708bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbbdb400640 0 client.25643 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 >> 172.21.15.27:0/478328467 conn(0x55af704e2350 msgr2=0x55af708956a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 shutdown_connections 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 1 -- 172.21.15.27:0/478328467 wait complete. 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.252+0000 7fbc02ff7740 20 client.25643 trim_cache size 0 max 16384 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "0ff1e276-77e7-11ef-baf5-efdc52797490", 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24773, 2024-09-21T07:13:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 202.91178193299999, 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 217.06909707200001 2024-09-21T07:13:20.258 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbbdc0b6790 msgr2=0x7fbbdc0b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbbdc0b6790 0x7fbbdc0b8ca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009230 tx=0x7fbbe0031400 comp rx=0 tx=0).stop 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf410be50 msgr2=0x7fbbf41a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf410be50 0x7fbbf41a1ce0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009fd0 tx=0x7fbbec00ef10 comp rx=0 tx=0).stop 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 shutdown_connections 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf410be50 0x7fbbf41a1ce0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbbdc0b6790 0x7fbbdc0b8ca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf410b500 0x7fbbf419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 --2- 172.21.15.27:0/1967024751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf4105bc0 0x7fbbf419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 >> 172.21.15.27:0/1967024751 conn(0x7fbbf4101630 msgr2=0x7fbbf4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 shutdown_connections 2024-09-21T07:13:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:20.253+0000 7fbbfac00640 1 -- 172.21.15.27:0/1967024751 wait complete. 2024-09-21T07:13:21.071 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T07:13:21.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:21 smithi027 ceph-mon[30779]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-21T07:13:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:21 smithi153 ceph-mon[34747]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-21T07:13:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:21 smithi119 ceph-mon[34519]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-21T07:13:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:23 smithi027 ceph-mon[30779]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-21T07:13:23.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:23 smithi153 ceph-mon[34747]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-21T07:13:23.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:23 smithi119 ceph-mon[34519]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.684+0000 7f6271400640 1 -- 172.21.15.27:0/433092842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c105cc0 msgr2=0x7f626c1060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.684+0000 7f6271400640 1 --2- 172.21.15.27:0/433092842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c105cc0 0x7f626c1060d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f625c009900 tx=0x7f625c02a410 comp rx=0 tx=0).stop 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.685+0000 7f6271400640 1 -- 172.21.15.27:0/433092842 shutdown_connections 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.685+0000 7f6271400640 1 --2- 172.21.15.27:0/433092842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c10bf40 0x7f626c112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.685+0000 7f6271400640 1 --2- 172.21.15.27:0/433092842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c10b5f0 0x7f626c10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.685+0000 7f6271400640 1 --2- 172.21.15.27:0/433092842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c105cc0 0x7f626c1060d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.685+0000 7f6271400640 1 -- 172.21.15.27:0/433092842 >> 172.21.15.27:0/433092842 conn(0x7f626c1016e0 msgr2=0x7f626c103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.685+0000 7f6271400640 1 -- 172.21.15.27:0/433092842 shutdown_connections 2024-09-21T07:13:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.685+0000 7f6271400640 1 -- 172.21.15.27:0/433092842 wait complete. 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 Processor -- start 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 -- start start 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c105cc0 0x7f626c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c10b5f0 0x7f626c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c10bf40 0x7f626c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f626c115c00 con 0x7f626c10bf40 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f626c115a80 con 0x7f626c10b5f0 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.686+0000 7f6271400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f626c115d80 con 0x7f626c105cc0 2024-09-21T07:13:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c105cc0 0x7f626c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c105cc0 0x7f626c19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42512/0 (socket says 172.21.15.27:42512) 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 -- 172.21.15.27:0/1854823290 learned_addr learned my addr 172.21.15.27:0/1854823290 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f6263e00640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c10b5f0 0x7f626c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626b200640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c10bf40 0x7f626c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 -- 172.21.15.27:0/1854823290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c10b5f0 msgr2=0x7f626c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c10b5f0 0x7f626c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 -- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c10bf40 msgr2=0x7f626c1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c10bf40 0x7f626c1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 -- 172.21.15.27:0/1854823290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c1a25b0 con 0x7f626c105cc0 2024-09-21T07:13:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.687+0000 7f626a800640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c105cc0 0x7f626c19c680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f625c005940 tx=0x7f625c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.688+0000 7f6263400640 1 -- 172.21.15.27:0/1854823290 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c03f030 con 0x7f626c105cc0 2024-09-21T07:13:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.688+0000 7f6263400640 1 -- 172.21.15.27:0/1854823290 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625c0044c0 con 0x7f626c105cc0 2024-09-21T07:13:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.688+0000 7f6263400640 1 -- 172.21.15.27:0/1854823290 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c035070 con 0x7f626c105cc0 2024-09-21T07:13:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.688+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c1a2840 con 0x7f626c105cc0 2024-09-21T07:13:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.688+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a9030 con 0x7f626c105cc0 2024-09-21T07:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 Processor -- start 2024-09-21T07:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 -- start start 2024-09-21T07:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34262c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 0x55bf34263170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf342636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf341d86b0 con 0x55bf33ebe1b0 2024-09-21T07:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bf341c7ae0 con 0x55bf33e952f0 2024-09-21T07:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.691+0000 7f62785d4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf341cb8f0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f626b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34262c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf342636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f626b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34262c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56802/0 (socket says 172.21.15.27:56802) 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f626b200640 1 -- 172.21.15.27:0/1534669141 learned_addr learned my addr 172.21.15.27:0/1534669141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263400640 1 -- 172.21.15.27:0/1854823290 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f625c042030 con 0x7f626c105cc0 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263e00640 1 -- 172.21.15.27:0/1534669141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 msgr2=0x55bf34263170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263e00640 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 0x55bf34263170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263e00640 1 -- 172.21.15.27:0/1534669141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 msgr2=0x55bf34262c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263e00640 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34262c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263e00640 1 -- 172.21.15.27:0/1534669141 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf34194780 con 0x55bf33e9d4d0 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f626b200640 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34262c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.692+0000 7f6263e00640 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf342636b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f626c06ca40 tx=0x7f6254011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed01379021cc84de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.693+0000 7f625b400640 1 -- 172.21.15.27:0/1534669141 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254004050 con 0x55bf33e9d4d0 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.693+0000 7f625b400640 1 -- 172.21.15.27:0/1534669141 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62540054d0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.693+0000 7f625b400640 1 -- 172.21.15.27:0/1534669141 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254004c80 con 0x55bf33e9d4d0 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 -- 172.21.15.27:0/1534669141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 msgr2=0x55bf342636b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf342636b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f626c06ca40 tx=0x7f6254011810 comp rx=0 tx=0).stop 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 -- 172.21.15.27:0/1534669141 shutdown_connections 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf342636b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 0x55bf34263170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 --2- 172.21.15.27:0/1534669141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34262c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 -- 172.21.15.27:0/1534669141 >> 172.21.15.27:0/1534669141 conn(0x55bf33eb2300 msgr2=0x55bf34260b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f6263400640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62500b6790 0x7f62500b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 -- 172.21.15.27:0/1534669141 shutdown_connections 2024-09-21T07:13:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f62785d4740 1 -- 172.21.15.27:0/1534669141 wait complete. 2024-09-21T07:13:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f6263400640 1 -- 172.21.15.27:0/1854823290 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f625c107f60 con 0x7f626c105cc0 2024-09-21T07:13:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.694+0000 7f626b200640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62500b6790 0x7f62500b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.695+0000 7f62785d4740 1 -- start start 2024-09-21T07:13:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.695+0000 7f626b200640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62500b6790 0x7f62500b8ca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6264005fd0 tx=0x7f6264009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.695+0000 7f62785d4740 1 Processor -- start 2024-09-21T07:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.695+0000 7f62785d4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f62785d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf3426e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f62785d4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f62785d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf340ffab0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f62785d4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bf3416e570 con 0x55bf33ebe1b0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f62785d4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf342713b0 con 0x55bf33e952f0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f62785d4740 10 client.0 _ll_register_callbacks cb 0x55bf3417d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f6263e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf3426e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f6263e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf3426e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56804/0 (socket says 172.21.15.27:56804) 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f6263e00640 1 -- 172.21.15.27:0/3859826271 learned_addr learned my addr 172.21.15.27:0/3859826271 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f626a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f626b200640 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f626a800640 1 -- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 msgr2=0x55bf3427a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f626a800640 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f626a800640 1 -- 172.21.15.27:0/3859826271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 msgr2=0x55bf3426e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f626a800640 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf3426e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.696+0000 7f626a800640 1 -- 172.21.15.27:0/3859826271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf33f29790 con 0x55bf33e952f0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.697+0000 7f626a800640 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f625c105240 tx=0x7f625c0d71e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70609b69eb77b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.697+0000 7f624fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.697+0000 7f624fe00640 1 -- 172.21.15.27:0/3859826271 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c049040 con 0x55bf33e952f0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.697+0000 7f624fe00640 1 -- 172.21.15.27:0/3859826271 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f625c0ccba0 con 0x55bf33e952f0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.697+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62540054d0 con 0x55bf33e952f0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.697+0000 7f624fe00640 1 -- 172.21.15.27:0/3859826271 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f625c0cd0b0 con 0x55bf33e952f0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.697+0000 7f62785d4740 10 client.25654 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf33fa2620 con 0x55bf33e952f0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f624fe00640 1 -- 172.21.15.27:0/3859826271 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62540054d0 con 0x55bf33e952f0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f624fe00640 1 -- 172.21.15.27:0/3859826271 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f625c0cebc0 con 0x55bf33e952f0 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f62785d4740 10 client.25654 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f62785d4740 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f62785d4740 20 client.25654 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f62785d4740 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e7cd80 0x55bf342803b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f62785d4740 4 client.25654 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf341da600 con 0x55bf33e7cd80 2024-09-21T07:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.698+0000 7f6263e00640 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e7cd80 0x55bf342803b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.699+0000 7f6263e00640 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e7cd80 0x55bf342803b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4daeb11959209427 server_cookie=5e90df71f839548d in_seq=0 out_seq=0 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.699+0000 7f624fe00640 10 client.25654 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f624fe00640 1 -- 172.21.15.27:0/3859826271 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf341da600 con 0x55bf33e7cd80 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f624fe00640 10 client.25654 handle_command_reply: tid=0 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 client.25654 shutdown 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 msgr2=0x55bf3426e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f625c105240 tx=0x7f625c0d71e0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 shutdown_connections 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e7cd80 0x55bf342803b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf3426e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 --2- 172.21.15.27:0/3859826271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f624fe00640 0 client.25654 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 >> 172.21.15.27:0/3859826271 conn(0x55bf33eb2300 msgr2=0x55bf34263c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 shutdown_connections 2024-09-21T07:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 1 -- 172.21.15.27:0/3859826271 wait complete. 2024-09-21T07:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.700+0000 7f62785d4740 20 client.25654 trim_cache size 0 max 16384 2024-09-21T07:13:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 Processor -- start 2024-09-21T07:13:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 -- start start 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf34264370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b57c0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b3090 con 0x55bf33ebe1b0 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.716+0000 7f62785d4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b0ee0 con 0x55bf33e952f0 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf34264370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf34264370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56806/0 (socket says 172.21.15.27:56806) 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 -- 172.21.15.27:0/1678749823 learned_addr learned my addr 172.21.15.27:0/1678749823 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f626a800640 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f626b200640 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 -- 172.21.15.27:0/1678749823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 msgr2=0x55bf3426e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 -- 172.21.15.27:0/1678749823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 msgr2=0x55bf3427a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 -- 172.21.15.27:0/1678749823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf342684c0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f626b200640 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f6263e00640 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf34264370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6254009b40 tx=0x7f6254005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf32829ebdadf54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.717+0000 7f625aa00640 1 -- 172.21.15.27:0/1678749823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540045b0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f625aa00640 1 -- 172.21.15.27:0/1678749823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254003aa0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f625aa00640 1 -- 172.21.15.27:0/1678749823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540199d0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 -- 172.21.15.27:0/1678749823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 msgr2=0x55bf34264370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf34264370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6254009b40 tx=0x7f6254005d50 comp rx=0 tx=0).stop 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 -- 172.21.15.27:0/1678749823 shutdown_connections 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33ebe1b0 0x55bf3427a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33e9d4d0 0x55bf34264370 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 --2- 172.21.15.27:0/1678749823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e952f0 0x55bf3426e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 -- 172.21.15.27:0/1678749823 >> 172.21.15.27:0/1678749823 conn(0x55bf33eb2300 msgr2=0x55bf34263c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.718+0000 7f62785d4740 1 -- 172.21.15.27:0/1678749823 shutdown_connections 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.719+0000 7f62785d4740 1 -- 172.21.15.27:0/1678749823 wait complete. 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.719+0000 7f62785d4740 1 -- start start 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.719+0000 7f62785d4740 1 Processor -- start 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f62785d4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 0x55bf34272610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f62785d4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf34272b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f62785d4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34273090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f62785d4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b1550 con 0x55bf33ebe1b0 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f62785d4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b29c0 con 0x55bf33e952f0 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f62785d4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b1c20 con 0x55bf33e9d4d0 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f62785d4740 10 client.0 _ll_register_callbacks cb 0x7f625c1050d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f626b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34273090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf34272b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf34272b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42554/0 (socket says 172.21.15.27:42554) 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 -- 172.21.15.27:0/2966876137 learned_addr learned my addr 172.21.15.27:0/2966876137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f626b200640 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34273090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56812/0 (socket says 172.21.15.27:56812) 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f626a800640 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 0x55bf34272610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 -- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 msgr2=0x55bf34272610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 0x55bf34272610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 -- 172.21.15.27:0/2966876137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 msgr2=0x55bf34273090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34273090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.720+0000 7f6263e00640 1 -- 172.21.15.27:0/2966876137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf342e9dc0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f6263e00640 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf34272b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6254011d10 tx=0x7f6254019430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52eb946e853fca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f624fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f624fe00640 1 -- 172.21.15.27:0/2966876137 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540035f0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6254003aa0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f624fe00640 1 -- 172.21.15.27:0/2966876137 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f6254003790 con 0x55bf33e9d4d0 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f624fe00640 1 -- 172.21.15.27:0/2966876137 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6254042050 con 0x55bf33e9d4d0 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f62785d4740 10 client.25660 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.721+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf342eb050 con 0x55bf33e9d4d0 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f624fe00640 1 -- 172.21.15.27:0/2966876137 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254003aa0 con 0x55bf33e9d4d0 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f624fe00640 1 -- 172.21.15.27:0/2966876137 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f625402b020 con 0x55bf33e9d4d0 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f62785d4740 10 client.25660 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f62785d4740 10 client.25660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f62785d4740 20 client.25660 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f62785d4740 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e521c0 0x55bf342ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f62785d4740 4 client.25660 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:13:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf341d60b0 con 0x55bf33e521c0 2024-09-21T07:13:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.722+0000 7f626a800640 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e521c0 0x55bf342ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.723+0000 7f626a800640 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e521c0 0x55bf342ed560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d78a74e42aa03a server_cookie=c24fb3b7f4043158 in_seq=0 out_seq=0 2024-09-21T07:13:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.723+0000 7f624fe00640 10 client.25660 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f624fe00640 1 -- 172.21.15.27:0/2966876137 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf341d60b0 con 0x55bf33e521c0 2024-09-21T07:13:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f624fe00640 10 client.25660 handle_command_reply: tid=0 2024-09-21T07:13:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 client.25660 shutdown 2024-09-21T07:13:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 msgr2=0x55bf34272b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf34272b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6254011d10 tx=0x7f6254019430 comp rx=0 tx=0).stop 2024-09-21T07:13:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 shutdown_connections 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf33e521c0 0x55bf342ed560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf33e9d4d0 0x55bf34272b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf33ebe1b0 0x55bf34273090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f624fe00640 0 client.25660 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 --2- 172.21.15.27:0/2966876137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf33e952f0 0x55bf34272610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 >> 172.21.15.27:0/2966876137 conn(0x55bf33eb2300 msgr2=0x55bf3426f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 shutdown_connections 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 1 -- 172.21.15.27:0/2966876137 wait complete. 2024-09-21T07:13:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.732+0000 7f62785d4740 20 client.25660 trim_cache size 0 max 16384 2024-09-21T07:13:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:13:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:13:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:13:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:13:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62500b6790 msgr2=0x7f62500b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62500b6790 0x7f62500b8ca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6264005fd0 tx=0x7f6264009500 comp rx=0 tx=0).stop 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c105cc0 msgr2=0x7f626c19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c105cc0 0x7f626c19c680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f625c005940 tx=0x7f625c0040c0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 shutdown_connections 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62500b6790 0x7f62500b8ca0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c10bf40 0x7f626c1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f626c10b5f0 0x7f626c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 --2- 172.21.15.27:0/1854823290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c105cc0 0x7f626c19c680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 >> 172.21.15.27:0/1854823290 conn(0x7f626c1016e0 msgr2=0x7f626c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 shutdown_connections 2024-09-21T07:13:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:24.733+0000 7f6271400640 1 -- 172.21.15.27:0/1854823290 wait complete. 2024-09-21T07:13:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:25 smithi027 ceph-mon[30779]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-21T07:13:25.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:25 smithi153 ceph-mon[34747]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-21T07:13:25.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:25 smithi119 ceph-mon[34519]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-21T07:13:27.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:27 smithi027 ceph-mon[30779]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-21T07:13:27.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:27 smithi153 ceph-mon[34747]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-21T07:13:27.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:27 smithi119 ceph-mon[34519]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-21T07:13:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:29 smithi027 ceph-mon[30779]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T07:13:29.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:29 smithi153 ceph-mon[34747]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T07:13:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:29 smithi119 ceph-mon[34519]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T07:13:30.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:13:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 -- 172.21.15.27:0/3212410745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 msgr2=0x7f465810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 --2- 172.21.15.27:0/3212410745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 0x7f465810bae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4644009860 tx=0x7f464402a340 comp rx=0 tx=0).stop 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 -- 172.21.15.27:0/3212410745 shutdown_connections 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 --2- 172.21.15.27:0/3212410745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 0x7f465810bae0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 --2- 172.21.15.27:0/3212410745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465810c020 0x7f4658112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 --2- 172.21.15.27:0/3212410745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 0x7f46581061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 -- 172.21.15.27:0/3212410745 >> 172.21.15.27:0/3212410745 conn(0x7f4658101860 msgr2=0x7f4658103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 -- 172.21.15.27:0/3212410745 shutdown_connections 2024-09-21T07:13:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.069+0000 7f465dc00640 1 -- 172.21.15.27:0/3212410745 wait complete. 2024-09-21T07:13:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.070+0000 7f465dc00640 1 Processor -- start 2024-09-21T07:13:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.070+0000 7f465dc00640 1 -- start start 2024-09-21T07:13:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f465dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 0x7f465819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f465dc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 0x7f465819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f465dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465810c020 0x7f46581a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f465dc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4658115ce0 con 0x7f465810c020 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f465dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4658115b60 con 0x7f4658105d90 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f465dc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4658115e60 con 0x7f465810b6d0 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f4656a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 0x7f465819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f4656000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 0x7f465819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f4656a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 0x7f465819c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41202/0 (socket says 172.21.15.27:41202) 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f4656a00640 1 -- 172.21.15.27:0/1571304678 learned_addr learned my addr 172.21.15.27:0/1571304678 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.071+0000 7f4657400640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465810c020 0x7f46581a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.072+0000 7f4656a00640 1 -- 172.21.15.27:0/1571304678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 msgr2=0x7f465819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.072+0000 7f4656a00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 0x7f465819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.072+0000 7f4656a00640 1 -- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465810c020 msgr2=0x7f46581a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.072+0000 7f4656a00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465810c020 0x7f46581a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.072+0000 7f4656a00640 1 -- 172.21.15.27:0/1571304678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46581a26e0 con 0x7f4658105d90 2024-09-21T07:13:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.072+0000 7f4656a00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 0x7f465819c770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f464800e9c0 tx=0x7f464800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.073+0000 7f464f400640 1 -- 172.21.15.27:0/1571304678 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464800cd80 con 0x7f4658105d90 2024-09-21T07:13:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.073+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46581a29d0 con 0x7f4658105d90 2024-09-21T07:13:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.073+0000 7f464f400640 1 -- 172.21.15.27:0/1571304678 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46480035f0 con 0x7f4658105d90 2024-09-21T07:13:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.073+0000 7f464f400640 1 -- 172.21.15.27:0/1571304678 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648010690 con 0x7f4658105d90 2024-09-21T07:13:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.073+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581a9130 con 0x7f4658105d90 2024-09-21T07:13:31.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 Processor -- start 2024-09-21T07:13:31.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 -- start start 2024-09-21T07:13:31.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f464f400640 1 -- 172.21.15.27:0/1571304678 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f4648003fb0 con 0x7f4658105d90 2024-09-21T07:13:31.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb7fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b8390 0x55912eb807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55912eaf6f70 con 0x55912e7d9070 2024-09-21T07:13:31.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55912eae63a0 con 0x55912e7b8390 2024-09-21T07:13:31.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.077+0000 7f46647cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55912eaea1b0 con 0x55912e7b01b0 2024-09-21T07:13:31.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4657400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb7fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4657400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb7fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60346/0 (socket says 172.21.15.27:60346) 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4657400640 1 -- 172.21.15.27:0/3274675528 learned_addr learned my addr 172.21.15.27:0/3274675528 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656000640 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b8390 0x55912eb807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656000640 1 -- 172.21.15.27:0/3274675528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 msgr2=0x55912eb802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656000640 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656000640 1 -- 172.21.15.27:0/3274675528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 msgr2=0x55912eb7fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656000640 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb7fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656000640 1 -- 172.21.15.27:0/3274675528 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912eab0c90 con 0x55912e7b8390 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.078+0000 7f4656a00640 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:31.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.079+0000 7f4656000640 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b8390 0x55912eb807e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f464402a860 tx=0x7f46440095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee30afcd440c7eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:31.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.079+0000 7f4643e00640 1 -- 172.21.15.27:0/3274675528 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644041070 con 0x55912e7b8390 2024-09-21T07:13:31.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.079+0000 7f4643e00640 1 -- 172.21.15.27:0/3274675528 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46440044d0 con 0x55912e7b8390 2024-09-21T07:13:31.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.079+0000 7f4643e00640 1 -- 172.21.15.27:0/3274675528 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644037040 con 0x55912e7b8390 2024-09-21T07:13:31.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f464f400640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f463c0b6710 0x7f463c0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f46647cd740 1 -- 172.21.15.27:0/3274675528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b8390 msgr2=0x55912eb807e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f46647cd740 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b8390 0x55912eb807e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f464402a860 tx=0x7f46440095a0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f4657400640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f463c0b6710 0x7f463c0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f46647cd740 1 -- 172.21.15.27:0/3274675528 shutdown_connections 2024-09-21T07:13:31.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f46647cd740 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f46647cd740 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b8390 0x55912eb807e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f46647cd740 1 --2- 172.21.15.27:0/3274675528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb7fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f464f400640 1 -- 172.21.15.27:0/1571304678 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f46480e7ef0 con 0x7f4658105d90 2024-09-21T07:13:31.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.080+0000 7f46647cd740 1 -- 172.21.15.27:0/3274675528 >> 172.21.15.27:0/3274675528 conn(0x55912e7cd460 msgr2=0x55912eb7dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:31.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.081+0000 7f46647cd740 1 -- 172.21.15.27:0/3274675528 shutdown_connections 2024-09-21T07:13:31.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.081+0000 7f46647cd740 1 -- 172.21.15.27:0/3274675528 wait complete. 2024-09-21T07:13:31.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.081+0000 7f46647cd740 1 -- start start 2024-09-21T07:13:31.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f4657400640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f463c0b6710 0x7f463c0b8c20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4650005fd0 tx=0x7f4650009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:31.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 1 Processor -- start 2024-09-21T07:13:31.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb8b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b8390 0x55912eb8b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb977d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55912e947700 con 0x55912e7b8390 2024-09-21T07:13:31.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55912ea81540 con 0x55912e7d9070 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55912eb8e4e0 con 0x55912e7b01b0 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.082+0000 7f46647cd740 10 client.0 _ll_register_callbacks cb 0x55912ea91430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b8390 0x55912eb8b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb8b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb8b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50682/0 (socket says 172.21.15.27:50682) 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b8390 0x55912eb8b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60356/0 (socket says 172.21.15.27:60356) 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 -- 172.21.15.27:0/326019342 learned_addr learned my addr 172.21.15.27:0/326019342 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4657400640 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 -- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 msgr2=0x55912eb977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 -- 172.21.15.27:0/326019342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b8390 msgr2=0x55912eb8b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b8390 0x55912eb8b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656a00640 1 -- 172.21.15.27:0/326019342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912e844650 con 0x55912e7b01b0 2024-09-21T07:13:31.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.083+0000 7f4656000640 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b8390 0x55912eb8b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f4656a00640 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb8b480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f464800c6a0 tx=0x7f4648003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b164e8f6ea647f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f4640c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f4640c00640 1 -- 172.21.15.27:0/326019342 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480ad5f0 con 0x55912e7b01b0 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f4640c00640 1 -- 172.21.15.27:0/326019342 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f46480aeb50 con 0x55912e7b01b0 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55912ea83b50 con 0x55912e7b01b0 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f4640c00640 1 -- 172.21.15.27:0/326019342 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46480acf90 con 0x55912e7b01b0 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f46647cd740 10 client.25672 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.084+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55912ea9e910 con 0x55912e7b01b0 2024-09-21T07:13:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f4640c00640 1 -- 172.21.15.27:0/326019342 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55912ea83b50 con 0x55912e7b01b0 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f4640c00640 1 -- 172.21.15.27:0/326019342 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f46480eb020 con 0x55912e7b01b0 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f46647cd740 10 client.25672 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f46647cd740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f46647cd740 20 client.25672 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f46647cd740 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e7979f0 0x55912eb9d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f46647cd740 4 client.25672 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.085+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55912eaf8ec0 con 0x55912e7979f0 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.086+0000 7f4656000640 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e7979f0 0x55912eb9d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.086+0000 7f4656000640 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e7979f0 0x55912eb9d570 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d9054eb47463e1 server_cookie=80e41bea7c3b1f64 in_seq=0 out_seq=0 2024-09-21T07:13:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.086+0000 7f4640c00640 10 client.25672 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f4640c00640 1 -- 172.21.15.27:0/326019342 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55912eaf8ec0 con 0x55912e7979f0 2024-09-21T07:13:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f4640c00640 10 client.25672 handle_command_reply: tid=0 2024-09-21T07:13:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 client.25672 shutdown 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 msgr2=0x55912eb8b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb8b480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f464800c6a0 tx=0x7f4648003cd0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 shutdown_connections 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e7979f0 0x55912eb9d570 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b8390 0x55912eb8b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 --2- 172.21.15.27:0/326019342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b01b0 0x55912eb8b480 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.087+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 >> 172.21.15.27:0/326019342 conn(0x55912e7cd460 msgr2=0x55912eb82f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.088+0000 7f4640c00640 0 client.25672 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.088+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 shutdown_connections 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.088+0000 7f46647cd740 1 -- 172.21.15.27:0/326019342 wait complete. 2024-09-21T07:13:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.088+0000 7f46647cd740 20 client.25672 trim_cache size 0 max 16384 2024-09-21T07:13:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 Processor -- start 2024-09-21T07:13:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 -- start start 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b01b0 0x55912eb86c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb88670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b58c0 con 0x55912e7d9070 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b3190 con 0x55912e7b01b0 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f46647cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b0fe0 con 0x55912e7b8390 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.107+0000 7f4656a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b01b0 0x55912eb86c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b01b0 0x55912eb86c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41238/0 (socket says 172.21.15.27:41238) 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 -- 172.21.15.27:0/950287475 learned_addr learned my addr 172.21.15.27:0/950287475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4657400640 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb88670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656000640 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 -- 172.21.15.27:0/950287475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 msgr2=0x55912eb8d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 -- 172.21.15.27:0/950287475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 msgr2=0x55912eb88670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb88670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 -- 172.21.15.27:0/950287475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912e797c20 con 0x55912e7b01b0 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656000640 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.108+0000 7f4656a00640 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b01b0 0x55912eb86c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4648003e20 tx=0x7f46480e5850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95f554fa63692546 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.109+0000 7f4643400640 1 -- 172.21.15.27:0/950287475 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480ad170 con 0x55912e7b01b0 2024-09-21T07:13:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.109+0000 7f4643400640 1 -- 172.21.15.27:0/950287475 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46480eb740 con 0x55912e7b01b0 2024-09-21T07:13:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.109+0000 7f4643400640 1 -- 172.21.15.27:0/950287475 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480b2300 con 0x55912e7b01b0 2024-09-21T07:13:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 -- 172.21.15.27:0/950287475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b01b0 msgr2=0x55912eb86c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b01b0 0x55912eb86c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4648003e20 tx=0x7f46480e5850 comp rx=0 tx=0).stop 2024-09-21T07:13:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 -- 172.21.15.27:0/950287475 shutdown_connections 2024-09-21T07:13:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7d9070 0x55912eb88670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 --2- 172.21.15.27:0/950287475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7b01b0 0x55912eb86c30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 -- 172.21.15.27:0/950287475 >> 172.21.15.27:0/950287475 conn(0x55912e7cd460 msgr2=0x55912eb8c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:31.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 -- 172.21.15.27:0/950287475 shutdown_connections 2024-09-21T07:13:31.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.110+0000 7f46647cd740 1 -- 172.21.15.27:0/950287475 wait complete. 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.111+0000 7f46647cd740 1 -- start start 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.111+0000 7f46647cd740 1 Processor -- start 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.111+0000 7f46647cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b01b0 0x55912eb94070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.111+0000 7f46647cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f46647cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb8fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f46647cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b1650 con 0x55912e7b01b0 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f46647cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b2ac0 con 0x55912e7d9070 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f46647cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b1d20 con 0x55912e7b8390 2024-09-21T07:13:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f46647cd740 10 client.0 _ll_register_callbacks cb 0x7f4648003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b01b0 0x55912eb94070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50706/0 (socket says 172.21.15.27:50706) 2024-09-21T07:13:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 -- 172.21.15.27:0/3035728074 learned_addr learned my addr 172.21.15.27:0/3035728074 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4657400640 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb8fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 -- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 msgr2=0x55912eb8fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb8fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 -- 172.21.15.27:0/3035728074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b01b0 msgr2=0x55912eb94070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b01b0 0x55912eb94070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656000640 1 -- 172.21.15.27:0/3035728074 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912eb903a0 con 0x55912e7b8390 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.112+0000 7f4656a00640 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b01b0 0x55912eb94070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.113+0000 7f4656000640 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8f570 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4644005940 tx=0x7f464402aa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0e00fa528deb4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.113+0000 7f4640c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.113+0000 7f4640c00640 1 -- 172.21.15.27:0/3035728074 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464404e070 con 0x55912e7b8390 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.113+0000 7f4640c00640 1 -- 172.21.15.27:0/3035728074 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4644004040 con 0x55912e7b8390 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.113+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46480eb740 con 0x55912e7b8390 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f4640c00640 1 -- 172.21.15.27:0/3035728074 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4644037040 con 0x55912e7b8390 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f46647cd740 10 client.25678 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:31.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55912ec07ff0 con 0x55912e7b8390 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f4640c00640 1 -- 172.21.15.27:0/3035728074 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46480eb740 con 0x55912e7b8390 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f4640c00640 1 -- 172.21.15.27:0/3035728074 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f464403d480 con 0x55912e7b8390 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f46647cd740 10 client.25678 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f46647cd740 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f46647cd740 20 client.25678 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.114+0000 7f46647cd740 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e76cda0 0x55912ec0a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.115+0000 7f46647cd740 4 client.25678 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.115+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55912eaf4970 con 0x55912e76cda0 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.115+0000 7f4656a00640 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e76cda0 0x55912ec0a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.115+0000 7f4640c00640 10 client.25678 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.115+0000 7f4656a00640 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e76cda0 0x55912ec0a330 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d464d41cffa71d server_cookie=2c49c7700e62a71a in_seq=0 out_seq=0 2024-09-21T07:13:31.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f4640c00640 1 -- 172.21.15.27:0/3035728074 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55912eaf4970 con 0x55912e76cda0 2024-09-21T07:13:31.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f4640c00640 10 client.25678 handle_command_reply: tid=0 2024-09-21T07:13:31.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 client.25678 shutdown 2024-09-21T07:13:31.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 msgr2=0x55912eb8f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8f570 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4644005940 tx=0x7f464402aa40 comp rx=0 tx=0).stop 2024-09-21T07:13:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 shutdown_connections 2024-09-21T07:13:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55912e76cda0 0x55912ec0a330 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55912e7b8390 0x55912eb8f570 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55912e7d9070 0x55912eb8fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f4640c00640 0 client.25678 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 --2- 172.21.15.27:0/3035728074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55912e7b01b0 0x55912eb94070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 >> 172.21.15.27:0/3035728074 conn(0x55912e7cd460 msgr2=0x55912eb89d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 shutdown_connections 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.116+0000 7f46647cd740 1 -- 172.21.15.27:0/3035728074 wait complete. 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f46647cd740 20 client.25678 trim_cache size 0 max 16384 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:13:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:13:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:13:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:13:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:13:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:13:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:31.122 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f463c0b6710 msgr2=0x7f463c0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f463c0b6710 0x7f463c0b8c20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4650005fd0 tx=0x7f4650009500 comp rx=0 tx=0).stop 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 msgr2=0x7f465819c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 0x7f465819c770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f464800e9c0 tx=0x7f464800edc0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 shutdown_connections 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f465810b6d0 0x7f465819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f463c0b6710 0x7f463c0b8c20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465810c020 0x7f46581a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 --2- 172.21.15.27:0/1571304678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658105d90 0x7f465819c770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.117+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 >> 172.21.15.27:0/1571304678 conn(0x7f4658101860 msgr2=0x7f4658103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.118+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 shutdown_connections 2024-09-21T07:13:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:31.118+0000 7f465dc00640 1 -- 172.21.15.27:0/1571304678 wait complete. 2024-09-21T07:13:31.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:31 smithi027 ceph-mon[30779]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T07:13:31.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:31 smithi153 ceph-mon[34747]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T07:13:31.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:31 smithi119 ceph-mon[34519]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T07:13:31.879 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:13:31.880 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:13:32.287 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 -- 172.21.15.27:0/318487833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed4109c40 msgr2=0x7f7ed4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/318487833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed4109c40 0x7f7ed4110470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009ca0 tx=0x7f7ecc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 -- 172.21.15.27:0/318487833 shutdown_connections 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/318487833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed4109c40 0x7f7ed4110470 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/318487833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed41092f0 0x7f7ed4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/318487833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed41039b0 0x7f7ed4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 -- 172.21.15.27:0/318487833 >> 172.21.15.27:0/318487833 conn(0x7f7ed40ff440 msgr2=0x7f7ed4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.955+0000 7f7ed3e00640 1 -- 172.21.15.27:0/318487833 shutdown_connections 2024-09-21T07:13:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.956+0000 7f7ed3e00640 1 -- 172.21.15.27:0/318487833 wait complete. 2024-09-21T07:13:32.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 Processor -- start 2024-09-21T07:13:32.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 -- start start 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed41039b0 0x7f7ed4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed41092f0 0x7f7ed4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed4109c40 0x7f7ed4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed4113950 con 0x7f7ed41039b0 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed41137d0 con 0x7f7ed41092f0 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.957+0000 7f7ed3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed4113ad0 con 0x7f7ed4109c40 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed41039b0 0x7f7ed4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:32.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed41092f0 0x7f7ed4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:32.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed41039b0 0x7f7ed4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60370/0 (socket says 172.21.15.27:60370) 2024-09-21T07:13:32.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2a00640 1 -- 172.21.15.27:0/1026619554 learned_addr learned my addr 172.21.15.27:0/1026619554 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:32.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed3400640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed4109c40 0x7f7ed4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2000640 1 -- 172.21.15.27:0/1026619554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed4109c40 msgr2=0x7f7ed4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2000640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed4109c40 0x7f7ed4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2000640 1 -- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed41039b0 msgr2=0x7f7ed4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2000640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed41039b0 0x7f7ed4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.958+0000 7f7ed2000640 1 -- 172.21.15.27:0/1026619554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed41a8600 con 0x7f7ed41092f0 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.959+0000 7f7ed2000640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed41092f0 0x7f7ed4075c50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ed406b990 tx=0x7f7ec000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.959+0000 7f7ecb400640 1 -- 172.21.15.27:0/1026619554 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000cc40 con 0x7f7ed41092f0 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.959+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed41a88f0 con 0x7f7ed41092f0 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.959+0000 7f7ecb400640 1 -- 172.21.15.27:0/1026619554 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec000cde0 con 0x7f7ed41092f0 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.959+0000 7f7ecb400640 1 -- 172.21.15.27:0/1026619554 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0018760 con 0x7f7ed41092f0 2024-09-21T07:13:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.959+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed41a8e30 con 0x7f7ed41092f0 2024-09-21T07:13:32.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.960+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb40052c0 con 0x7f7ed41092f0 2024-09-21T07:13:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.964+0000 7f7ecb400640 1 -- 172.21.15.27:0/1026619554 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f7ec0018900 con 0x7f7ed41092f0 2024-09-21T07:13:32.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.965+0000 7f7ecb400640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7eb80b6710 0x7f7eb80b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:32.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.966+0000 7f7ecb400640 1 -- 172.21.15.27:0/1026619554 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7ec00e7e60 con 0x7f7ed41092f0 2024-09-21T07:13:32.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.966+0000 7f7ed2a00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7eb80b6710 0x7f7eb80b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:32.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.966+0000 7f7ecb400640 1 -- 172.21.15.27:0/1026619554 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ec00b2270 con 0x7f7ed41092f0 2024-09-21T07:13:32.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:32.967+0000 7f7ed2a00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7eb80b6710 0x7f7eb80b8c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4077160 tx=0x7f7ec4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:33.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.226+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eb40055b0 con 0x7f7ed41092f0 2024-09-21T07:13:33.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.227+0000 7f7ecb400640 1 -- 172.21.15.27:0/1026619554 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f7ec00b71b0 con 0x7f7ed41092f0 2024-09-21T07:13:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:33.230 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:13:33.230 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:13:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7eb80b6710 msgr2=0x7f7eb80b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7eb80b6710 0x7f7eb80b8c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4077160 tx=0x7f7ec4005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed41092f0 msgr2=0x7f7ed4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed41092f0 0x7f7ed4075c50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ed406b990 tx=0x7f7ec000ee50 comp rx=0 tx=0).stop 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 shutdown_connections 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed4109c40 0x7f7ed4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7eb80b6710 0x7f7eb80b8c20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed41092f0 0x7f7ed4075c50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 --2- 172.21.15.27:0/1026619554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ed41039b0 0x7f7ed4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.230+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 >> 172.21.15.27:0/1026619554 conn(0x7f7ed40ff440 msgr2=0x7f7ed4106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.231+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 shutdown_connections 2024-09-21T07:13:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:33.231+0000 7f7ed3e00640 1 -- 172.21.15.27:0/1026619554 wait complete. 2024-09-21T07:13:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:33 smithi027 ceph-mon[30779]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-21T07:13:33.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:33 smithi153 ceph-mon[34747]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-21T07:13:33.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:33 smithi119 ceph-mon[34519]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-21T07:13:33.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:13:34.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:34 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1026619554' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:34.405 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:34.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:34 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1026619554' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:34.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:34 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1026619554' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 -- 172.21.15.27:0/2079770335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 msgr2=0x7fa624110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 --2- 172.21.15.27:0/2079770335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 0x7fa624110110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009ca0 tx=0x7fa61c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 -- 172.21.15.27:0/2079770335 shutdown_connections 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 --2- 172.21.15.27:0/2079770335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa624105110 0x7fa62410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 --2- 172.21.15.27:0/2079770335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 0x7fa624110110 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 --2- 172.21.15.27:0/2079770335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 0x7fa624104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 -- 172.21.15.27:0/2079770335 >> 172.21.15.27:0/2079770335 conn(0x7fa6240fe2c0 msgr2=0x7fa624100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 -- 172.21.15.27:0/2079770335 shutdown_connections 2024-09-21T07:13:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.042+0000 7fa623200640 1 -- 172.21.15.27:0/2079770335 wait complete. 2024-09-21T07:13:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 Processor -- start 2024-09-21T07:13:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 -- start start 2024-09-21T07:13:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 0x7fa62419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa624105110 0x7fa62419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 0x7fa62419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241138a0 con 0x7fa62410dc20 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa624113720 con 0x7fa6241047c0 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.044+0000 7fa623200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa624113a20 con 0x7fa624105110 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 0x7fa62419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 0x7fa62419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41292/0 (socket says 172.21.15.27:41292) 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 -- 172.21.15.27:0/3297923413 learned_addr learned my addr 172.21.15.27:0/3297923413 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621400640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa624105110 0x7fa62419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa622800640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 0x7fa62419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 -- 172.21.15.27:0/3297923413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa624105110 msgr2=0x7fa62419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa624105110 0x7fa62419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 -- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 msgr2=0x7fa62419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 0x7fa62419fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.045+0000 7fa621e00640 1 -- 172.21.15.27:0/3297923413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6241a01c0 con 0x7fa6241047c0 2024-09-21T07:13:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.046+0000 7fa621e00640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 0x7fa62419a250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa61400e9f0 tx=0x7fa61400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.046+0000 7fa61aa00640 1 -- 172.21.15.27:0/3297923413 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400cd80 con 0x7fa6241047c0 2024-09-21T07:13:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.046+0000 7fa61aa00640 1 -- 172.21.15.27:0/3297923413 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6140108e0 con 0x7fa6241047c0 2024-09-21T07:13:35.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.046+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6241a04b0 con 0x7fa6241047c0 2024-09-21T07:13:35.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.046+0000 7fa61aa00640 1 -- 172.21.15.27:0/3297923413 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614018980 con 0x7fa6241047c0 2024-09-21T07:13:35.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.046+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6241a6c10 con 0x7fa6241047c0 2024-09-21T07:13:35.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.049+0000 7fa62fe12740 1 Processor -- start 2024-09-21T07:13:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa62fe12740 1 -- start start 2024-09-21T07:13:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be1576af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be1539b1b0 0x55be1576b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1576b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa62fe12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55be156e1fb0 con 0x55be153c4070 2024-09-21T07:13:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa62fe12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55be156d13e0 con 0x55be153a3390 2024-09-21T07:13:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa62fe12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55be156d51f0 con 0x55be1539b1b0 2024-09-21T07:13:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa622800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be1576af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa622800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be1576af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60396/0 (socket says 172.21.15.27:60396) 2024-09-21T07:13:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.050+0000 7fa622800640 1 -- 172.21.15.27:0/3603221508 learned_addr learned my addr 172.21.15.27:0/3603221508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa621400640 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1576b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa622800640 1 -- 172.21.15.27:0/3603221508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be1539b1b0 msgr2=0x55be1576b4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa622800640 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be1539b1b0 0x55be1576b4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa622800640 1 -- 172.21.15.27:0/3603221508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 msgr2=0x55be1576b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa622800640 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1576b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa622800640 1 -- 172.21.15.27:0/3603221508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be1569bcd0 con 0x55be153c4070 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa621400640 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1576b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa61aa00640 1 -- 172.21.15.27:0/3297923413 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fa614010430 con 0x7fa6241047c0 2024-09-21T07:13:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.051+0000 7fa622800640 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be1576af70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005cc0 tx=0x7fa61c02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7c6818653aff74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.052+0000 7fa60f400640 1 -- 172.21.15.27:0/3603221508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c031cf0 con 0x55be153c4070 2024-09-21T07:13:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.052+0000 7fa60f400640 1 -- 172.21.15.27:0/3603221508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c036040 con 0x55be153c4070 2024-09-21T07:13:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.052+0000 7fa60f400640 1 -- 172.21.15.27:0/3603221508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004040 con 0x55be153c4070 2024-09-21T07:13:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 -- 172.21.15.27:0/3603221508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 msgr2=0x55be1576af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be1576af70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005cc0 tx=0x7fa61c02f420 comp rx=0 tx=0).stop 2024-09-21T07:13:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 -- 172.21.15.27:0/3603221508 shutdown_connections 2024-09-21T07:13:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be1539b1b0 0x55be1576b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1576b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3603221508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be1576af70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 -- 172.21.15.27:0/3603221508 >> 172.21.15.27:0/3603221508 conn(0x55be153b8460 msgr2=0x55be15768e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa61aa00640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6080b66c0 0x7fa6080b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.053+0000 7fa62fe12740 1 -- 172.21.15.27:0/3603221508 shutdown_connections 2024-09-21T07:13:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.054+0000 7fa61aa00640 1 -- 172.21.15.27:0/3297923413 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa6140e7ff0 con 0x7fa6241047c0 2024-09-21T07:13:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.054+0000 7fa62fe12740 1 -- 172.21.15.27:0/3603221508 wait complete. 2024-09-21T07:13:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.054+0000 7fa621400640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6080b66c0 0x7fa6080b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.054+0000 7fa62fe12740 1 -- start start 2024-09-21T07:13:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 1 Processor -- start 2024-09-21T07:13:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa621400640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6080b66c0 0x7fa6080b8bd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa62410ffb0 tx=0x7fa610005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15776790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153a3390 0x55be15776d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be157829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55be15532700 con 0x55be153c4070 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55be1566c580 con 0x55be1539b1b0 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55be157796f0 con 0x55be153a3390 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.055+0000 7fa62fe12740 10 client.0 _ll_register_callbacks cb 0x55be1567c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15776790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa622800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be157829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15776790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41308/0 (socket says 172.21.15.27:41308) 2024-09-21T07:13:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 -- 172.21.15.27:0/3012336181 learned_addr learned my addr 172.21.15.27:0/3012336181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621400640 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153a3390 0x55be15776d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 -- 172.21.15.27:0/3012336181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153a3390 msgr2=0x55be15776d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153a3390 0x55be15776d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 -- 172.21.15.27:0/3012336181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 msgr2=0x55be157829e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be157829e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621e00640 1 -- 172.21.15.27:0/3012336181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be1542f650 con 0x55be1539b1b0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.056+0000 7fa621400640 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153a3390 0x55be15776d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa621e00640 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15776790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6140e5410 tx=0x7fa6140ad750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5942e83b0a5738b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa603e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa603e00640 1 -- 172.21.15.27:0/3012336181 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140ad0f0 con 0x55be1539b1b0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa603e00640 1 -- 172.21.15.27:0/3012336181 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa614018b20 con 0x55be1539b1b0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be1566eb90 con 0x55be1539b1b0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa603e00640 1 -- 172.21.15.27:0/3012336181 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa614010720 con 0x55be1539b1b0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa62fe12740 10 client.25694 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.057+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be15689950 con 0x55be1539b1b0 2024-09-21T07:13:35.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa603e00640 1 -- 172.21.15.27:0/3012336181 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55be1566eb90 con 0x55be1539b1b0 2024-09-21T07:13:35.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa603e00640 1 -- 172.21.15.27:0/3012336181 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa6140ac650 con 0x55be1539b1b0 2024-09-21T07:13:35.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa62fe12740 10 client.25694 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:35.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa62fe12740 10 client.25694 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:13:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa62fe12740 20 client.25694 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be153829f0 0x55be157888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa62fe12740 4 client.25694 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.058+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be156e3f00 con 0x55be153829f0 2024-09-21T07:13:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.059+0000 7fa621400640 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be153829f0 0x55be157888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.059+0000 7fa621400640 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be153829f0 0x55be157888d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b81aadc8dec41e server_cookie=35a902107ee9c49c in_seq=0 out_seq=0 2024-09-21T07:13:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.059+0000 7fa603e00640 10 client.25694 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.060+0000 7fa603e00640 1 -- 172.21.15.27:0/3012336181 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be156e3f00 con 0x55be153829f0 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.060+0000 7fa603e00640 10 client.25694 handle_command_reply: tid=0 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.060+0000 7fa62fe12740 1 client.25694 shutdown 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.060+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 msgr2=0x55be15776790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.060+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15776790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6140e5410 tx=0x7fa6140ad750 comp rx=0 tx=0).stop 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 shutdown_connections 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be153829f0 0x55be157888d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153a3390 0x55be15776d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153c4070 0x55be157829e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 --2- 172.21.15.27:0/3012336181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15776790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa603e00640 0 client.25694 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 >> 172.21.15.27:0/3012336181 conn(0x55be153b8460 msgr2=0x55be15767600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 shutdown_connections 2024-09-21T07:13:35.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 1 -- 172.21.15.27:0/3012336181 wait complete. 2024-09-21T07:13:35.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.061+0000 7fa62fe12740 20 client.25694 trim_cache size 0 max 16384 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 Processor -- start 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 -- start start 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15773980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153a3390 0x55be15773ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be1578e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241b11c0 con 0x55be153a3390 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241aea90 con 0x55be1539b1b0 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.076+0000 7fa62fe12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241ac8e0 con 0x55be153c4070 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa621400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153a3390 0x55be15773ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa621400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153a3390 0x55be15773ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60408/0 (socket says 172.21.15.27:60408) 2024-09-21T07:13:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa621400640 1 -- 172.21.15.27:0/103525272 learned_addr learned my addr 172.21.15.27:0/103525272 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa621e00640 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15773980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa622800640 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be1578e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa622800640 1 -- 172.21.15.27:0/103525272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 msgr2=0x55be15773980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa622800640 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15773980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa622800640 1 -- 172.21.15.27:0/103525272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153a3390 msgr2=0x55be15773ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa622800640 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153a3390 0x55be15773ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa622800640 1 -- 172.21.15.27:0/103525272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be15768610 con 0x55be153c4070 2024-09-21T07:13:35.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.077+0000 7fa621e00640 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15773980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:35.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.078+0000 7fa621400640 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153a3390 0x55be15773ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:35.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.078+0000 7fa622800640 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be1578e070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009ca0 tx=0x7fa61c003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdbf4e025e3b37c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:35.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.082+0000 7fa60ea00640 1 -- 172.21.15.27:0/103525272 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c053070 con 0x55be153c4070 2024-09-21T07:13:35.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.082+0000 7fa60ea00640 1 -- 172.21.15.27:0/103525272 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c036040 con 0x55be153c4070 2024-09-21T07:13:35.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.082+0000 7fa60ea00640 1 -- 172.21.15.27:0/103525272 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c031880 con 0x55be153c4070 2024-09-21T07:13:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 -- 172.21.15.27:0/103525272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 msgr2=0x55be1578e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be1578e070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009ca0 tx=0x7fa61c003c30 comp rx=0 tx=0).stop 2024-09-21T07:13:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 -- 172.21.15.27:0/103525272 shutdown_connections 2024-09-21T07:13:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be1578e070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be153a3390 0x55be15773ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 --2- 172.21.15.27:0/103525272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1539b1b0 0x55be15773980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 -- 172.21.15.27:0/103525272 >> 172.21.15.27:0/103525272 conn(0x55be153b8460 msgr2=0x55be15775010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 -- 172.21.15.27:0/103525272 shutdown_connections 2024-09-21T07:13:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.083+0000 7fa62fe12740 1 -- 172.21.15.27:0/103525272 wait complete. 2024-09-21T07:13:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.084+0000 7fa62fe12740 1 -- start start 2024-09-21T07:13:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.084+0000 7fa62fe12740 1 Processor -- start 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.084+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be1539b1b0 0x55be1577ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.084+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1577f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.084+0000 7fa62fe12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be15779900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa62fe12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241acf50 con 0x55be1539b1b0 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa62fe12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241ae3c0 con 0x55be153a3390 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa62fe12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241ad620 con 0x55be153c4070 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa62fe12740 10 client.0 _ll_register_callbacks cb 0x7fa6140a98f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa621400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1577f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.088 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:35 smithi027 ceph-mon[30779]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa621e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be1539b1b0 0x55be1577ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa621e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be1539b1b0 0x55be1577ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60420/0 (socket says 172.21.15.27:60420) 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa621400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1577f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41322/0 (socket says 172.21.15.27:41322) 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa621e00640 1 -- 172.21.15.27:0/2037608142 learned_addr learned my addr 172.21.15.27:0/2037608142 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.085+0000 7fa622800640 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be15779900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa621e00640 1 -- 172.21.15.27:0/2037608142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 msgr2=0x55be15779900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa621e00640 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be15779900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa621e00640 1 -- 172.21.15.27:0/2037608142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 msgr2=0x55be1577f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa621e00640 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1577f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa621e00640 1 -- 172.21.15.27:0/2037608142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be1577a090 con 0x55be1539b1b0 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa621e00640 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be1539b1b0 0x55be1577ebe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6140a9f50 tx=0x7fa614010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d204fe2544082e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa603e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa603e00640 1 -- 172.21.15.27:0/2037608142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140c7060 con 0x55be1539b1b0 2024-09-21T07:13:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa603e00640 1 -- 172.21.15.27:0/2037608142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa6140c3b30 con 0x55be1539b1b0 2024-09-21T07:13:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.086+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61c036040 con 0x55be1539b1b0 2024-09-21T07:13:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa603e00640 1 -- 172.21.15.27:0/2037608142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6140ad0f0 con 0x55be1539b1b0 2024-09-21T07:13:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa62fe12740 10 client.15672 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be157f3c70 con 0x55be1539b1b0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa603e00640 1 -- 172.21.15.27:0/2037608142 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c036040 con 0x55be1539b1b0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa603e00640 1 -- 172.21.15.27:0/2037608142 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa6140b68e0 con 0x55be1539b1b0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa62fe12740 10 client.15672 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa62fe12740 10 client.15672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa62fe12740 20 client.15672 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.087+0000 7fa62fe12740 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be15357da0 0x55be157f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.088+0000 7fa62fe12740 4 client.15672 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.088+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be156df9b0 con 0x55be15357da0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.088+0000 7fa621400640 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be15357da0 0x55be157f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa603e00640 10 client.15672 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa621400640 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be15357da0 0x55be157f6450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57717cbfd9a5afcb server_cookie=fcb4c4728ccb35f2 in_seq=0 out_seq=0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa603e00640 1 -- 172.21.15.27:0/2037608142 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be156df9b0 con 0x55be15357da0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa603e00640 10 client.15672 handle_command_reply: tid=0 2024-09-21T07:13:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 client.15672 shutdown 2024-09-21T07:13:35.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be1539b1b0 msgr2=0x55be1577ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be1539b1b0 0x55be1577ebe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6140a9f50 tx=0x7fa614010ca0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 shutdown_connections 2024-09-21T07:13:35.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55be15357da0 0x55be157f6450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be153c4070 0x55be15779900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be153a3390 0x55be1577f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 --2- 172.21.15.27:0/2037608142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be1539b1b0 0x55be1577ebe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa603e00640 0 client.15672 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.089+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 >> 172.21.15.27:0/2037608142 conn(0x55be153b8460 msgr2=0x55be15767600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.090+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 shutdown_connections 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.090+0000 7fa62fe12740 1 -- 172.21.15.27:0/2037608142 wait complete. 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.090+0000 7fa62fe12740 20 client.15672 trim_cache size 0 max 16384 2024-09-21T07:13:35.093 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:13:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6080b66c0 msgr2=0x7fa6080b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6080b66c0 0x7fa6080b8bd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa62410ffb0 tx=0x7fa610005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 msgr2=0x7fa62419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 0x7fa62419a250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa61400e9f0 tx=0x7fa61400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 shutdown_connections 2024-09-21T07:13:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa624105110 0x7fa62419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6080b66c0 0x7fa6080b8bd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa62410dc20 0x7fa62419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 --2- 172.21.15.27:0/3297923413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6241047c0 0x7fa62419a250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 >> 172.21.15.27:0/3297923413 conn(0x7fa6240fe2c0 msgr2=0x7fa624107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 shutdown_connections 2024-09-21T07:13:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:35.091+0000 7fa623200640 1 -- 172.21.15.27:0/3297923413 wait complete. 2024-09-21T07:13:35.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:35 smithi153 ceph-mon[34747]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T07:13:35.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:35 smithi119 ceph-mon[34519]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T07:13:35.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:13:36.072 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T07:13:36.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:13:36.324 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:37.014 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.015+0000 7f1f89c00640 1 -- 172.21.15.27:0/1084279182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8410fff0 msgr2=0x7f1f84112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.015+0000 7f1f89c00640 1 --2- 172.21.15.27:0/1084279182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8410fff0 0x7f1f84112410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009ca0 tx=0x7f1f7c02a940 comp rx=0 tx=0).stop 2024-09-21T07:13:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.016+0000 7f1f89c00640 1 -- 172.21.15.27:0/1084279182 shutdown_connections 2024-09-21T07:13:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.016+0000 7f1f89c00640 1 --2- 172.21.15.27:0/1084279182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f84102d90 0x7f1f8410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.016+0000 7f1f89c00640 1 --2- 172.21.15.27:0/1084279182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8410fff0 0x7f1f84112410 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.016+0000 7f1f89c00640 1 --2- 172.21.15.27:0/1084279182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102440 0x7f1f84102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.016+0000 7f1f89c00640 1 -- 172.21.15.27:0/1084279182 >> 172.21.15.27:0/1084279182 conn(0x7f1f840fe2c0 msgr2=0x7f1f84100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.016+0000 7f1f89c00640 1 -- 172.21.15.27:0/1084279182 shutdown_connections 2024-09-21T07:13:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.016+0000 7f1f89c00640 1 -- 172.21.15.27:0/1084279182 wait complete. 2024-09-21T07:13:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.017+0000 7f1f89c00640 1 Processor -- start 2024-09-21T07:13:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.017+0000 7f1f89c00640 1 -- start start 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.017+0000 7f1f89c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f84102440 0x7f1f84074dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f89c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102d90 0x7f1f8406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f89c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8410fff0 0x7f1f8406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f89c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84115ae0 con 0x7f1f84102440 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f89c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84115960 con 0x7f1f84102d90 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f89c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84115c60 con 0x7f1f8410fff0 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f89200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8410fff0 0x7f1f8406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f84102440 0x7f1f84074dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102d90 0x7f1f8406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102d90 0x7f1f8406e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41328/0 (socket says 172.21.15.27:41328) 2024-09-21T07:13:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.018+0000 7f1f83400640 1 -- 172.21.15.27:0/2428471454 learned_addr learned my addr 172.21.15.27:0/2428471454 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f83400640 1 -- 172.21.15.27:0/2428471454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8410fff0 msgr2=0x7f1f8406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f83400640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8410fff0 0x7f1f8406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f83400640 1 -- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f84102440 msgr2=0x7f1f84074dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f83400640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f84102440 0x7f1f84074dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f83400640 1 -- 172.21.15.27:0/2428471454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8406edb0 con 0x7f1f84102d90 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f83400640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102d90 0x7f1f8406e0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c600 tx=0x7f1f7000cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f80c00640 1 -- 172.21.15.27:0/2428471454 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70005020 con 0x7f1f84102d90 2024-09-21T07:13:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f84073a40 con 0x7f1f84102d90 2024-09-21T07:13:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f80c00640 1 -- 172.21.15.27:0/2428471454 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f70005670 con 0x7f1f84102d90 2024-09-21T07:13:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.019+0000 7f1f80c00640 1 -- 172.21.15.27:0/2428471454 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000b910 con 0x7f1f84102d90 2024-09-21T07:13:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.020+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f84073f80 con 0x7f1f84102d90 2024-09-21T07:13:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.025+0000 7f1f80c00640 1 -- 172.21.15.27:0/2428471454 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f1f700049c0 con 0x7f1f84102d90 2024-09-21T07:13:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.025+0000 7f1f91e29740 1 Processor -- start 2024-09-21T07:13:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.025+0000 7f1f91e29740 1 -- start start 2024-09-21T07:13:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f91e29740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5581a9b7cfb0 con 0x5581a983e390 2024-09-21T07:13:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f91e29740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5581a9b6c3e0 con 0x5581a98361b0 2024-09-21T07:13:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f91e29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5581a9b701f0 con 0x5581a985f070 2024-09-21T07:13:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f89200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f89200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c05f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50784/0 (socket says 172.21.15.27:50784) 2024-09-21T07:13:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41342/0 (socket says 172.21.15.27:41342) 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.026+0000 7f1f89200640 1 -- 172.21.15.27:0/2173458008 learned_addr learned my addr 172.21.15.27:0/2173458008 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.027+0000 7f1f83400640 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.027+0000 7f1f89200640 1 -- 172.21.15.27:0/2173458008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 msgr2=0x5581a9c064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.027+0000 7f1f89200640 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.027+0000 7f1f89200640 1 -- 172.21.15.27:0/2173458008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 msgr2=0x5581a9c069f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.027+0000 7f1f89200640 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c069f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.027+0000 7f1f89200640 1 -- 172.21.15.27:0/2173458008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a9b36cd0 con 0x5581a985f070 2024-09-21T07:13:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.027+0000 7f1f83400640 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.028+0000 7f1f80c00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f680b6710 0x7f1f680b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.028+0000 7f1f89200640 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c05f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009ca0 tx=0x7f1f7c0099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8adf0b314735a927 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.028+0000 7f1f80c00640 1 -- 172.21.15.27:0/2428471454 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1f700efda0 con 0x7f1f84102d90 2024-09-21T07:13:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.028+0000 7f1f79600640 1 -- 172.21.15.27:0/2173458008 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c034040 con 0x5581a985f070 2024-09-21T07:13:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.028+0000 7f1f83e00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f680b6710 0x7f1f680b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.028+0000 7f1f79600640 1 -- 172.21.15.27:0/2173458008 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c003e20 con 0x5581a985f070 2024-09-21T07:13:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.028+0000 7f1f79600640 1 -- 172.21.15.27:0/2173458008 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c02fe70 con 0x5581a985f070 2024-09-21T07:13:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.029+0000 7f1f91e29740 1 -- 172.21.15.27:0/2173458008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 msgr2=0x5581a9c05f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.029+0000 7f1f91e29740 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c05f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009ca0 tx=0x7f1f7c0099f0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.029+0000 7f1f83e00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f680b6710 0x7f1f680b8c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x5581a9c04bf0 tx=0x7f1f74005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.030+0000 7f1f91e29740 1 -- 172.21.15.27:0/2173458008 shutdown_connections 2024-09-21T07:13:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.030+0000 7f1f91e29740 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.030+0000 7f1f91e29740 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.030+0000 7f1f91e29740 1 --2- 172.21.15.27:0/2173458008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c05f70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.030+0000 7f1f91e29740 1 -- 172.21.15.27:0/2173458008 >> 172.21.15.27:0/2173458008 conn(0x5581a9853460 msgr2=0x5581a9c03e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.030+0000 7f1f91e29740 1 -- 172.21.15.27:0/2173458008 shutdown_connections 2024-09-21T07:13:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.030+0000 7f1f91e29740 1 -- 172.21.15.27:0/2173458008 wait complete. 2024-09-21T07:13:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.031+0000 7f1f91e29740 1 -- start start 2024-09-21T07:13:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.031+0000 7f1f91e29740 1 Processor -- start 2024-09-21T07:13:37.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c11bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f91e29740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5581a99cd700 con 0x5581a983e390 2024-09-21T07:13:37.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f91e29740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5581a9b07580 con 0x5581a98361b0 2024-09-21T07:13:37.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f91e29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5581a9c146f0 con 0x5581a985f070 2024-09-21T07:13:37.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f91e29740 10 client.0 _ll_register_callbacks cb 0x5581a9b17470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:37.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c11bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c11670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41352/0 (socket says 172.21.15.27:41352) 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.032+0000 7f1f83e00640 1 -- 172.21.15.27:0/1934892357 learned_addr learned my addr 172.21.15.27:0/1934892357 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f89200640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f83400640 1 -- 172.21.15.27:0/1934892357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 msgr2=0x5581a9c1da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f83400640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c1da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f83400640 1 -- 172.21.15.27:0/1934892357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 msgr2=0x5581a9c11670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f83400640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c11670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f83400640 1 -- 172.21.15.27:0/1934892357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a98ca650 con 0x5581a983e390 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f83e00640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f89200640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f83400640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c11bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1f700ed080 tx=0x7f1f700ed700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca373b5cb75ade26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.033+0000 7f1f6e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:13:37.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.034+0000 7f1f6e000640 1 -- 172.21.15.27:0/1934892357 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f700b5290 con 0x5581a983e390 2024-09-21T07:13:37.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.034+0000 7f1f6e000640 1 -- 172.21.15.27:0/1934892357 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1f700b10d0 con 0x5581a983e390 2024-09-21T07:13:37.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.034+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581a9b09b90 con 0x5581a983e390 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.034+0000 7f1f6e000640 1 -- 172.21.15.27:0/1934892357 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f700ba1b0 con 0x5581a983e390 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.034+0000 7f1f91e29740 10 client.15684 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.034+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581a9b24950 con 0x5581a983e390 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f6e000640 1 -- 172.21.15.27:0/1934892357 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5581a9b09b90 con 0x5581a983e390 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f6e000640 1 -- 172.21.15.27:0/1934892357 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f7000bc40 con 0x5581a983e390 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f91e29740 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f91e29740 10 client.15684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:13:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f91e29740 20 client.15684 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a981d9f0 0x5581a9c23920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f91e29740 4 client.15684 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.035+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581a9b7ef00 con 0x5581a981d9f0 2024-09-21T07:13:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.036+0000 7f1f83e00640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a981d9f0 0x5581a9c23920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.037+0000 7f1f6e000640 10 client.15684 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.037+0000 7f1f83e00640 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a981d9f0 0x5581a9c23920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70305dad2dec5654 server_cookie=685fbbbca8bf6d4b in_seq=0 out_seq=0 2024-09-21T07:13:37.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.038+0000 7f1f6e000640 1 -- 172.21.15.27:0/1934892357 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581a9b7ef00 con 0x5581a981d9f0 2024-09-21T07:13:37.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.038+0000 7f1f6e000640 10 client.15684 handle_command_reply: tid=0 2024-09-21T07:13:37.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.038+0000 7f1f91e29740 1 client.15684 shutdown 2024-09-21T07:13:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 msgr2=0x5581a9c11bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c11bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1f700ed080 tx=0x7f1f700ed700 comp rx=0 tx=0).stop 2024-09-21T07:13:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 shutdown_connections 2024-09-21T07:13:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a981d9f0 0x5581a9c23920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a985f070 0x5581a9c1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a983e390 0x5581a9c11bb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1934892357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a98361b0 0x5581a9c11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f6e000640 0 client.15684 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 >> 172.21.15.27:0/1934892357 conn(0x5581a9853460 msgr2=0x5581a9c02600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 shutdown_connections 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.039+0000 7f1f91e29740 1 -- 172.21.15.27:0/1934892357 wait complete. 2024-09-21T07:13:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.040+0000 7f1f91e29740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T07:13:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.056+0000 7f1f91e29740 1 Processor -- start 2024-09-21T07:13:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.056+0000 7f1f91e29740 1 -- start start 2024-09-21T07:13:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a98361b0 0x5581a9c132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 0x5581a9c1d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c0e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a98361b0 0x5581a9c132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a98361b0 0x5581a9c132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60472/0 (socket says 172.21.15.27:60472) 2024-09-21T07:13:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f83e00640 1 -- 172.21.15.27:0/3247067683 learned_addr learned my addr 172.21.15.27:0/3247067683 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f841b9980 con 0x5581a98361b0 2024-09-21T07:13:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f841b7250 con 0x5581a985f070 2024-09-21T07:13:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f841b50e0 con 0x5581a983e390 2024-09-21T07:13:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.057+0000 7f1f89200640 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c0e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.058+0000 7f1f83e00640 1 -- 172.21.15.27:0/3247067683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 msgr2=0x5581a9c1d910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.058+0000 7f1f83e00640 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 0x5581a9c1d910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.058+0000 7f1f83e00640 1 -- 172.21.15.27:0/3247067683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 msgr2=0x5581a9c0e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.058+0000 7f1f83e00640 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c0e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.058+0000 7f1f83e00640 1 -- 172.21.15.27:0/3247067683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a9c0c2a0 con 0x5581a98361b0 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.058+0000 7f1f83e00640 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a98361b0 0x5581a9c132a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5581a9c24090 tx=0x7f1f7401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523e3a9e1dfd26a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.059+0000 7f1f78c00640 1 -- 172.21.15.27:0/3247067683 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74003e00 con 0x5581a98361b0 2024-09-21T07:13:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.059+0000 7f1f78c00640 1 -- 172.21.15.27:0/3247067683 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7401fd30 con 0x5581a98361b0 2024-09-21T07:13:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.059+0000 7f1f78c00640 1 -- 172.21.15.27:0/3247067683 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74004a00 con 0x5581a98361b0 2024-09-21T07:13:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.060+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a98361b0 msgr2=0x5581a9c132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.060+0000 7f1f91e29740 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a98361b0 0x5581a9c132a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5581a9c24090 tx=0x7f1f7401b000 comp rx=0 tx=0).stop 2024-09-21T07:13:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.061+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 shutdown_connections 2024-09-21T07:13:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.061+0000 7f1f91e29740 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 0x5581a9c1d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.061+0000 7f1f91e29740 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c0e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.061+0000 7f1f91e29740 1 --2- 172.21.15.27:0/3247067683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a98361b0 0x5581a9c132a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.061+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 >> 172.21.15.27:0/3247067683 conn(0x5581a9853460 msgr2=0x5581a9c12b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.061+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 shutdown_connections 2024-09-21T07:13:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.061+0000 7f1f91e29740 1 -- 172.21.15.27:0/3247067683 wait complete. 2024-09-21T07:13:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.062+0000 7f1f91e29740 1 -- start start 2024-09-21T07:13:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.062+0000 7f1f91e29740 1 Processor -- start 2024-09-21T07:13:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.063+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 0x5581a9c19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.063+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c1a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.063+0000 7f1f91e29740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a97f2da0 0x5581a9c14930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.063+0000 7f1f91e29740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f841b5730 con 0x5581a97f2da0 2024-09-21T07:13:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.063+0000 7f1f91e29740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f841b6b80 con 0x5581a985f070 2024-09-21T07:13:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.063+0000 7f1f91e29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f841b5de0 con 0x5581a983e390 2024-09-21T07:13:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.063+0000 7f1f91e29740 10 client.0 _ll_register_callbacks cb 0x7f1f700048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f89200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a97f2da0 0x5581a9c14930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c1a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c1a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41374/0 (socket says 172.21.15.27:41374) 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f89200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a97f2da0 0x5581a9c14930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60476/0 (socket says 172.21.15.27:60476) 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 -- 172.21.15.27:0/1530318027 learned_addr learned my addr 172.21.15.27:0/1530318027 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83e00640 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 0x5581a9c19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 -- 172.21.15.27:0/1530318027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 msgr2=0x5581a9c19b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 0x5581a9c19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 -- 172.21.15.27:0/1530318027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a97f2da0 msgr2=0x5581a9c14930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a97f2da0 0x5581a9c14930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.064+0000 7f1f83400640 1 -- 172.21.15.27:0/1530318027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a9c15040 con 0x5581a985f070 2024-09-21T07:13:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.065+0000 7f1f83400640 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c1a160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f700b5480 tx=0x7f1f700b4e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eae8864ae4083ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.065+0000 7f1f6e000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.065+0000 7f1f6e000640 1 -- 172.21.15.27:0/1530318027 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f700b1620 con 0x5581a985f070 2024-09-21T07:13:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.065+0000 7f1f6e000640 1 -- 172.21.15.27:0/1530318027 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1f700b12d0 con 0x5581a985f070 2024-09-21T07:13:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.065+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f7401fd30 con 0x5581a985f070 2024-09-21T07:13:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.066+0000 7f1f6e000640 1 -- 172.21.15.27:0/1530318027 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f700b5290 con 0x5581a985f070 2024-09-21T07:13:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.066+0000 7f1f91e29740 10 client.25718 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.066+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581a9c9fcb0 con 0x5581a985f070 2024-09-21T07:13:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.068+0000 7f1f6e000640 1 -- 172.21.15.27:0/1530318027 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7401fd30 con 0x5581a985f070 2024-09-21T07:13:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.068+0000 7f1f6e000640 1 -- 172.21.15.27:0/1530318027 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f700be040 con 0x5581a985f070 2024-09-21T07:13:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.069+0000 7f1f91e29740 10 client.25718 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.069+0000 7f1f91e29740 10 client.25718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:13:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.069+0000 7f1f91e29740 20 client.25718 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.069+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a9ca0410 0x5581a9ca2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.069+0000 7f1f91e29740 4 client.25718 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:13:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.069+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581a9b7a9b0 con 0x5581a9ca0410 2024-09-21T07:13:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.069+0000 7f1f83e00640 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a9ca0410 0x5581a9ca2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.070+0000 7f1f83e00640 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a9ca0410 0x5581a9ca2830 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ca7e19bd0fbc2cb server_cookie=afb10b8162135957 in_seq=0 out_seq=0 2024-09-21T07:13:37.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.070+0000 7f1f6e000640 10 client.25718 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f6e000640 1 -- 172.21.15.27:0/1530318027 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581a9b7a9b0 con 0x5581a9ca0410 2024-09-21T07:13:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f6e000640 10 client.25718 handle_command_reply: tid=0 2024-09-21T07:13:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 client.25718 shutdown 2024-09-21T07:13:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 msgr2=0x5581a9c1a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c1a160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f700b5480 tx=0x7f1f700b4e00 comp rx=0 tx=0).stop 2024-09-21T07:13:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 shutdown_connections 2024-09-21T07:13:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5581a9ca0410 0x5581a9ca2830 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5581a97f2da0 0x5581a9c14930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a985f070 0x5581a9c1a160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 --2- 172.21.15.27:0/1530318027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581a983e390 0x5581a9c19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f6e000640 0 client.25718 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:13:37.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.071+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 >> 172.21.15.27:0/1530318027 conn(0x5581a9853460 msgr2=0x5581a9c02600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.072+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 shutdown_connections 2024-09-21T07:13:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.072+0000 7f1f91e29740 1 -- 172.21.15.27:0/1530318027 wait complete. 2024-09-21T07:13:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.072+0000 7f1f91e29740 20 client.25718 trim_cache size 0 max 16384 2024-09-21T07:13:37.075 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f680b6710 msgr2=0x7f1f680b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f680b6710 0x7f1f680b8c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x5581a9c04bf0 tx=0x7f1f74005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102d90 msgr2=0x7f1f8406e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102d90 0x7f1f8406e0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c600 tx=0x7f1f7000cac0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 shutdown_connections 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8410fff0 0x7f1f8406e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f680b6710 0x7f1f680b8c20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84102d90 0x7f1f8406e0a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 --2- 172.21.15.27:0/2428471454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f84102440 0x7f1f84074dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 >> 172.21.15.27:0/2428471454 conn(0x7f1f840fe2c0 msgr2=0x7f1f841010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 shutdown_connections 2024-09-21T07:13:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:37.073+0000 7f1f89c00640 1 -- 172.21.15.27:0/2428471454 wait complete. 2024-09-21T07:13:37.330 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:37 smithi027 ceph-mon[30779]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-21T07:13:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:37 smithi153 ceph-mon[34747]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-21T07:13:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:37 smithi119 ceph-mon[34519]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.942+0000 7ffba6200640 1 -- 172.21.15.27:0/3038923976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 msgr2=0x7ffba010d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.942+0000 7ffba6200640 1 --2- 172.21.15.27:0/3038923976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 0x7ffba010d5d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009900 tx=0x7ffb8c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.943+0000 7ffba6200640 1 -- 172.21.15.27:0/3038923976 shutdown_connections 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.943+0000 7ffba6200640 1 --2- 172.21.15.27:0/3038923976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffba010dba0 0x7ffba0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.943+0000 7ffba6200640 1 --2- 172.21.15.27:0/3038923976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 0x7ffba010d5d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.943+0000 7ffba6200640 1 --2- 172.21.15.27:0/3038923976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba0104740 0x7ffba0104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.943+0000 7ffba6200640 1 -- 172.21.15.27:0/3038923976 >> 172.21.15.27:0/3038923976 conn(0x7ffba00fe230 msgr2=0x7ffba0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.943+0000 7ffba6200640 1 -- 172.21.15.27:0/3038923976 shutdown_connections 2024-09-21T07:13:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.943+0000 7ffba6200640 1 -- 172.21.15.27:0/3038923976 wait complete. 2024-09-21T07:13:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.944+0000 7ffba6200640 1 Processor -- start 2024-09-21T07:13:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.944+0000 7ffba6200640 1 -- start start 2024-09-21T07:13:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba6200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffba0104740 0x7ffba019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 0x7ffba019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba010dba0 0x7ffba01a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba6200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01137e0 con 0x7ffba0105090 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba6200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0113660 con 0x7ffba010dba0 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba6200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0113960 con 0x7ffba0104740 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 0x7ffba019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba010dba0 0x7ffba01a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 0x7ffba019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60500/0 (socket says 172.21.15.27:60500) 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffb9fe00640 1 -- 172.21.15.27:0/978475598 learned_addr learned my addr 172.21.15.27:0/978475598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.945+0000 7ffba4e00640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffba0104740 0x7ffba019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffba5800640 1 -- 172.21.15.27:0/978475598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffba0104740 msgr2=0x7ffba019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffba5800640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffba0104740 0x7ffba019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffba5800640 1 -- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 msgr2=0x7ffba019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffba5800640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 0x7ffba019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffba5800640 1 -- 172.21.15.27:0/978475598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba01a23b0 con 0x7ffba010dba0 2024-09-21T07:13:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffba5800640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba010dba0 0x7ffba01a1cb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009fd0 tx=0x7ffb9800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffb9d600640 1 -- 172.21.15.27:0/978475598 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800ce20 con 0x7ffba010dba0 2024-09-21T07:13:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffb9d600640 1 -- 172.21.15.27:0/978475598 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb98005020 con 0x7ffba010dba0 2024-09-21T07:13:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba01a26a0 con 0x7ffba010dba0 2024-09-21T07:13:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.946+0000 7ffb9d600640 1 -- 172.21.15.27:0/978475598 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98010690 con 0x7ffba010dba0 2024-09-21T07:13:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.947+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba01a8e30 con 0x7ffba010dba0 2024-09-21T07:13:37.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.950+0000 7ffbae7c6740 1 Processor -- start 2024-09-21T07:13:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.952+0000 7ffb9d600640 1 -- 172.21.15.27:0/978475598 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7ffb980054d0 con 0x7ffba010dba0 2024-09-21T07:13:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffbae7c6740 1 -- start start 2024-09-21T07:13:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6d71b0 0x55959ea7be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea7c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea7c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55959e9f16b0 con 0x55959e6b64d0 2024-09-21T07:13:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55959e9e0ae0 con 0x55959e6d71b0 2024-09-21T07:13:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55959e9e48f0 con 0x55959e6ae2f0 2024-09-21T07:13:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.953+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea7c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6d71b0 0x55959ea7be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffba4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea7c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6d71b0 0x55959ea7be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41400/0 (socket says 172.21.15.27:41400) 2024-09-21T07:13:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffb9fe00640 1 -- 172.21.15.27:0/927593088 learned_addr learned my addr 172.21.15.27:0/927593088 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffba4e00640 1 -- 172.21.15.27:0/927593088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6d71b0 msgr2=0x55959ea7be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffba4e00640 1 --2- 172.21.15.27:0/927593088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6d71b0 0x55959ea7be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffba4e00640 1 -- 172.21.15.27:0/927593088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 msgr2=0x55959ea7c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffba4e00640 1 --2- 172.21.15.27:0/927593088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea7c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffba4e00640 1 -- 172.21.15.27:0/927593088 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959e9ad780 con 0x55959e6ae2f0 2024-09-21T07:13:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffba4e00640 1 --2- 172.21.15.27:0/927593088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea7c350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009c50 tx=0x7ffb94011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b2417429080d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.954+0000 7ffb9d600640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb880b6710 0x7ffb880b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.955+0000 7ffb92000640 1 -- 172.21.15.27:0/927593088 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94004050 con 0x55959e6ae2f0 2024-09-21T07:13:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.955+0000 7ffb9d600640 1 -- 172.21.15.27:0/978475598 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ffb980e82f0 con 0x7ffba010dba0 2024-09-21T07:13:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.955+0000 7ffb92000640 1 -- 172.21.15.27:0/927593088 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb940054d0 con 0x55959e6ae2f0 2024-09-21T07:13:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.955+0000 7ffb92000640 1 -- 172.21.15.27:0/927593088 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94004c40 con 0x55959e6ae2f0 2024-09-21T07:13:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.955+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb880b6710 0x7ffb880b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 -- 172.21.15.27:0/927593088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 msgr2=0x55959ea7c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb880b6710 0x7ffb880b8c20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0096c0 tx=0x7ffb8c0095d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/927593088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea7c350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009c50 tx=0x7ffb94011810 comp rx=0 tx=0).stop 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 -- 172.21.15.27:0/927593088 shutdown_connections 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/927593088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea7c350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/927593088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea7c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/927593088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6d71b0 0x55959ea7be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 -- 172.21.15.27:0/927593088 >> 172.21.15.27:0/927593088 conn(0x55959e6cb300 msgr2=0x55959ea79d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 -- 172.21.15.27:0/927593088 shutdown_connections 2024-09-21T07:13:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.956+0000 7ffbae7c6740 1 -- 172.21.15.27:0/927593088 wait complete. 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.957+0000 7ffbae7c6740 1 -- start start 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.957+0000 7ffbae7c6740 1 Processor -- start 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6ae2f0 0x55959ea873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea87e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55959e918ab0 con 0x55959e6ae2f0 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55959e987570 con 0x55959e6b64d0 2024-09-21T07:13:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55959ea8a590 con 0x55959e6d71b0 2024-09-21T07:13:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffbae7c6740 10 client.0 _ll_register_callbacks cb 0x55959e996100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6ae2f0 0x55959ea873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6ae2f0 0x55959ea873a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58808/0 (socket says 172.21.15.27:58808) 2024-09-21T07:13:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba4e00640 1 -- 172.21.15.27:0/3367410254 learned_addr learned my addr 172.21.15.27:0/3367410254 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba5800640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea87e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba4e00640 1 -- 172.21.15.27:0/3367410254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 msgr2=0x55959ea87e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba4e00640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea87e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba4e00640 1 -- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 msgr2=0x55959ea878e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.958+0000 7ffba4e00640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea878e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.959+0000 7ffba4e00640 1 -- 172.21.15.27:0/3367410254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959e742790 con 0x55959e6ae2f0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.959+0000 7ffba5800640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.959+0000 7ffba4e00640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6ae2f0 0x55959ea873a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000c00 tx=0x7ffb94004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97f7bf8bb4cba62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.959+0000 7ffb86a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.959+0000 7ffb86a00640 1 -- 172.21.15.27:0/3367410254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94027070 con 0x55959e6ae2f0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.959+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb940054d0 con 0x55959e6ae2f0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.959+0000 7ffb86a00640 1 -- 172.21.15.27:0/3367410254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ffb94003a00 con 0x55959e6ae2f0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffb86a00640 1 -- 172.21.15.27:0/3367410254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb94033050 con 0x55959e6ae2f0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 10 client.15702 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55959e7bb620 con 0x55959e6ae2f0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffb86a00640 1 -- 172.21.15.27:0/3367410254 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb940054d0 con 0x55959e6ae2f0 2024-09-21T07:13:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffb86a00640 1 -- 172.21.15.27:0/3367410254 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffb94005020 con 0x55959e6ae2f0 2024-09-21T07:13:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 10 client.15702 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 20 client.15702 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e695d80 0x55959ea99410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 4 client.15702 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.960+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55959e9f3600 con 0x55959e695d80 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.961+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e695d80 0x55959ea99410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.962+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e695d80 0x55959ea99410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5eee96bb8853990 server_cookie=c342740cf29b49e3 in_seq=0 out_seq=0 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.962+0000 7ffb86a00640 10 client.15702 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.962+0000 7ffb86a00640 1 -- 172.21.15.27:0/3367410254 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55959e9f3600 con 0x55959e695d80 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.962+0000 7ffb86a00640 10 client.15702 handle_command_reply: tid=0 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.962+0000 7ffbae7c6740 1 client.15702 shutdown 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6ae2f0 msgr2=0x55959ea873a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6ae2f0 0x55959ea873a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000c00 tx=0x7ffb94004740 comp rx=0 tx=0).stop 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 shutdown_connections 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e695d80 0x55959ea99410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3367410254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6ae2f0 0x55959ea873a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 >> 172.21.15.27:0/3367410254 conn(0x55959e6cb300 msgr2=0x55959ea7ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffb86a00640 0 client.15702 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 shutdown_connections 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3367410254 wait complete. 2024-09-21T07:13:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.963+0000 7ffbae7c6740 20 client.15702 trim_cache size 0 max 16384 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 Processor -- start 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 -- start start 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6ae2f0 0x55959ea9eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea9f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea84570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b56c0 con 0x55959e6b64d0 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b2f90 con 0x55959e6ae2f0 2024-09-21T07:13:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.980+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b0de0 con 0x55959e6d71b0 2024-09-21T07:13:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea9f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6ae2f0 0x55959ea9eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea9f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58820/0 (socket says 172.21.15.27:58820) 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffb9fe00640 1 -- 172.21.15.27:0/2717602978 learned_addr learned my addr 172.21.15.27:0/2717602978 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba5800640 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea84570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba4e00640 1 -- 172.21.15.27:0/2717602978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 msgr2=0x55959ea84570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba4e00640 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea84570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba4e00640 1 -- 172.21.15.27:0/2717602978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 msgr2=0x55959ea9f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba4e00640 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea9f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba4e00640 1 -- 172.21.15.27:0/2717602978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959ea7ecc0 con 0x55959e6ae2f0 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba5800640 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea84570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea9f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.981+0000 7ffba4e00640 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6ae2f0 0x55959ea9eaf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb940053f0 tx=0x7ffb94004650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67bb98267b176147 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.982+0000 7ffb91600640 1 -- 172.21.15.27:0/2717602978 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94027070 con 0x55959e6ae2f0 2024-09-21T07:13:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.982+0000 7ffb91600640 1 -- 172.21.15.27:0/2717602978 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb94004360 con 0x55959e6ae2f0 2024-09-21T07:13:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.982+0000 7ffb91600640 1 -- 172.21.15.27:0/2717602978 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94003800 con 0x55959e6ae2f0 2024-09-21T07:13:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 -- 172.21.15.27:0/2717602978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6ae2f0 msgr2=0x55959ea9eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6ae2f0 0x55959ea9eaf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb940053f0 tx=0x7ffb94004650 comp rx=0 tx=0).stop 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 -- 172.21.15.27:0/2717602978 shutdown_connections 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6d71b0 0x55959ea84570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6b64d0 0x55959ea9f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/2717602978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6ae2f0 0x55959ea9eaf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 -- 172.21.15.27:0/2717602978 >> 172.21.15.27:0/2717602978 conn(0x55959e6cb300 msgr2=0x55959ea74a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 -- 172.21.15.27:0/2717602978 shutdown_connections 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 -- 172.21.15.27:0/2717602978 wait complete. 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.983+0000 7ffbae7c6740 1 -- start start 2024-09-21T07:13:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 1 Processor -- start 2024-09-21T07:13:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6d71b0 0x55959eafbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b1450 con 0x55959e6d71b0 2024-09-21T07:13:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b28c0 con 0x55959e6b64d0 2024-09-21T07:13:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b1b20 con 0x55959e6ae2f0 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffbae7c6740 10 client.0 _ll_register_callbacks cb 0x7ffb8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.984+0000 7ffba5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6d71b0 0x55959eafbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffba4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea8d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52986/0 (socket says 172.21.15.27:52986) 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffb9fe00640 1 -- 172.21.15.27:0/3174438798 learned_addr learned my addr 172.21.15.27:0/3174438798 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffba4e00640 1 -- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 msgr2=0x55959ea8d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffba4e00640 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea8d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffba4e00640 1 -- 172.21.15.27:0/3174438798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6d71b0 msgr2=0x55959eafbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffba4e00640 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6d71b0 0x55959eafbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffba4e00640 1 -- 172.21.15.27:0/3174438798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959eafc360 con 0x55959e6ae2f0 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.985+0000 7ffba5800640 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6d71b0 0x55959eafbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffba4e00640 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea8d120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb940053f0 tx=0x7ffb94004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d93119fee88aa4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffb86a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffb86a00640 1 -- 172.21.15.27:0/3174438798 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94027040 con 0x55959e6ae2f0 2024-09-21T07:13:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb94004360 con 0x55959e6ae2f0 2024-09-21T07:13:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffb86a00640 1 -- 172.21.15.27:0/3174438798 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ffb94023a40 con 0x55959e6ae2f0 2024-09-21T07:13:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffb86a00640 1 -- 172.21.15.27:0/3174438798 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb940237f0 con 0x55959e6ae2f0 2024-09-21T07:13:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffbae7c6740 10 client.25729 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.986+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55959eb036a0 con 0x55959e6ae2f0 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffb86a00640 1 -- 172.21.15.27:0/3174438798 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb94004360 con 0x55959e6ae2f0 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffb86a00640 1 -- 172.21.15.27:0/3174438798 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffb9402c640 con 0x55959e6ae2f0 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffbae7c6740 10 client.25729 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffbae7c6740 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffbae7c6740 20 client.25729 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e66b1c0 0x55959eb05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffbae7c6740 4 client.25729 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55959e9ef0b0 con 0x55959e66b1c0 2024-09-21T07:13:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.987+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e66b1c0 0x55959eb05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.988+0000 7ffb9fe00640 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e66b1c0 0x55959eb05d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6959bf9f6101b6ea server_cookie=455cac8f79189dba in_seq=0 out_seq=0 2024-09-21T07:13:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.988+0000 7ffb86a00640 10 client.25729 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.994+0000 7ffb86a00640 1 -- 172.21.15.27:0/3174438798 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55959e9ef0b0 con 0x55959e66b1c0 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.994+0000 7ffb86a00640 10 client.25729 handle_command_reply: tid=0 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.994+0000 7ffbae7c6740 1 client.25729 shutdown 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.994+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 msgr2=0x55959ea8d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.994+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea8d120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb940053f0 tx=0x7ffb94004590 comp rx=0 tx=0).stop 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.994+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 shutdown_connections 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55959e66b1c0 0x55959eb05d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55959e6d71b0 0x55959eafbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55959e6b64d0 0x55959ea8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 1 --2- 172.21.15.27:0/3174438798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55959e6ae2f0 0x55959ea8d120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffb86a00640 0 client.25729 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 >> 172.21.15.27:0/3174438798 conn(0x55959e6cb300 msgr2=0x55959ea7ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 shutdown_connections 2024-09-21T07:13:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 1 -- 172.21.15.27:0/3174438798 wait complete. 2024-09-21T07:13:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.995+0000 7ffbae7c6740 20 client.25729 trim_cache size 0 max 16384 2024-09-21T07:13:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:13:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:13:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:13:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb880b6710 msgr2=0x7ffb880b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb880b6710 0x7ffb880b8c20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0096c0 tx=0x7ffb8c0095d0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba010dba0 msgr2=0x7ffba01a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba010dba0 0x7ffba01a1cb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009fd0 tx=0x7ffb9800ef10 comp rx=0 tx=0).stop 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 shutdown_connections 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffb880b6710 0x7ffb880b8c20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba010dba0 0x7ffba01a1cb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffba0105090 0x7ffba019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 --2- 172.21.15.27:0/978475598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffba0104740 0x7ffba019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 >> 172.21.15.27:0/978475598 conn(0x7ffba00fe230 msgr2=0x7ffba010cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 shutdown_connections 2024-09-21T07:13:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:37.996+0000 7ffba6200640 1 -- 172.21.15.27:0/978475598 wait complete. 2024-09-21T07:13:38.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 -- 172.21.15.27:0/3276473833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441023e0 msgr2=0x7f20441027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 --2- 172.21.15.27:0/3276473833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441023e0 0x7f20441027f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2034009900 tx=0x7f203402a480 comp rx=0 tx=0).stop 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 -- 172.21.15.27:0/3276473833 shutdown_connections 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 --2- 172.21.15.27:0/3276473833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2044102d30 0x7f204410fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 --2- 172.21.15.27:0/3276473833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f204410ff90 0x7f20441123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 --2- 172.21.15.27:0/3276473833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441023e0 0x7f20441027f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 -- 172.21.15.27:0/3276473833 >> 172.21.15.27:0/3276473833 conn(0x7f20440fe2c0 msgr2=0x7f2044100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 -- 172.21.15.27:0/3276473833 shutdown_connections 2024-09-21T07:13:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.384+0000 7f2049400640 1 -- 172.21.15.27:0/3276473833 wait complete. 2024-09-21T07:13:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.386+0000 7f2049400640 1 Processor -- start 2024-09-21T07:13:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.386+0000 7f2049400640 1 -- start start 2024-09-21T07:13:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.387+0000 7f2049400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2044102d30 0x7f204419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.387+0000 7f2049400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f204410ff90 0x7f204419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.387+0000 7f2049400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441a1da0 0x7f20441a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.387+0000 7f2049400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2044115ad0 con 0x7f20441a1da0 2024-09-21T07:13:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.387+0000 7f2049400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2044115950 con 0x7f2044102d30 2024-09-21T07:13:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.387+0000 7f2049400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2044115c50 con 0x7f204410ff90 2024-09-21T07:13:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.388+0000 7f2043e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2044102d30 0x7f204419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.388+0000 7f2043400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f204410ff90 0x7f204419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.388+0000 7f2043e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2044102d30 0x7f204419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53008/0 (socket says 172.21.15.27:53008) 2024-09-21T07:13:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.388+0000 7f2043e00640 1 -- 172.21.15.27:0/2106132844 learned_addr learned my addr 172.21.15.27:0/2106132844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.388+0000 7f2048a00640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441a1da0 0x7f20441a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:38.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.388+0000 7f2043e00640 1 -- 172.21.15.27:0/2106132844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f204410ff90 msgr2=0x7f204419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.388+0000 7f2043e00640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f204410ff90 0x7f204419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.389+0000 7f2043e00640 1 -- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441a1da0 msgr2=0x7f20441a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.389+0000 7f2043e00640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441a1da0 0x7f20441a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.389+0000 7f2043e00640 1 -- 172.21.15.27:0/2106132844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20441a2980 con 0x7f2044102d30 2024-09-21T07:13:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.389+0000 7f2043e00640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2044102d30 0x7f204419c620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f203402aa70 tx=0x7f2034003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.390+0000 7f2040c00640 1 -- 172.21.15.27:0/2106132844 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203403f020 con 0x7f2044102d30 2024-09-21T07:13:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.390+0000 7f2040c00640 1 -- 172.21.15.27:0/2106132844 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2034031b60 con 0x7f2044102d30 2024-09-21T07:13:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.390+0000 7f2040c00640 1 -- 172.21.15.27:0/2106132844 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034035070 con 0x7f2044102d30 2024-09-21T07:13:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.390+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20441a8f20 con 0x7f2044102d30 2024-09-21T07:13:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.390+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20441a92c0 con 0x7f2044102d30 2024-09-21T07:13:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.392+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20441027f0 con 0x7f2044102d30 2024-09-21T07:13:38.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.396+0000 7f2040c00640 1 -- 172.21.15.27:0/2106132844 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f2034042020 con 0x7f2044102d30 2024-09-21T07:13:38.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.398+0000 7f2040c00640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20280b6790 0x7f20280b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:38.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.398+0000 7f2040c00640 1 -- 172.21.15.27:0/2106132844 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f2034108270 con 0x7f2044102d30 2024-09-21T07:13:38.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.399+0000 7f2040c00640 1 -- 172.21.15.27:0/2106132844 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2034049070 con 0x7f2044102d30 2024-09-21T07:13:38.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.399+0000 7f2043400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20280b6790 0x7f20280b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:38.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.400+0000 7f2043400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20280b6790 0x7f20280b8ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2044112450 tx=0x7f2030005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:38.523 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:38.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.697+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f204402e670 con 0x7f2044102d30 2024-09-21T07:13:38.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.698+0000 7f2040c00640 1 -- 172.21.15.27:0/2106132844 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f20340ce2a0 con 0x7f2044102d30 2024-09-21T07:13:38.701 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:38.701 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:13:38.701 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:13:38.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20280b6790 msgr2=0x7f20280b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20280b6790 0x7f20280b8ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2044112450 tx=0x7f2030005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2044102d30 msgr2=0x7f204419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2044102d30 0x7f204419c620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f203402aa70 tx=0x7f2034003c10 comp rx=0 tx=0).stop 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 shutdown_connections 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f204410ff90 0x7f204419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f20280b6790 0x7f20280b8ca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20441a1da0 0x7f20441a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 --2- 172.21.15.27:0/2106132844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2044102d30 0x7f204419c620 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:38.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.702+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 >> 172.21.15.27:0/2106132844 conn(0x7f20440fe2c0 msgr2=0x7f2044101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:38.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.703+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 shutdown_connections 2024-09-21T07:13:38.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:38.704+0000 7f2049400640 1 -- 172.21.15.27:0/2106132844 wait complete. 2024-09-21T07:13:39.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:39 smithi119 ceph-mon[34519]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:13:39.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:39 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2106132844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:39 smithi027 ceph-mon[30779]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:13:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:39 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2106132844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 -- 172.21.15.27:0/721220315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 msgr2=0x7f095c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 --2- 172.21.15.27:0/721220315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 0x7f095c110470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0954009ca0 tx=0x7f095402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 -- 172.21.15.27:0/721220315 shutdown_connections 2024-09-21T07:13:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 --2- 172.21.15.27:0/721220315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 0x7f095c110470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 --2- 172.21.15.27:0/721220315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095c1092f0 0x7f095c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 --2- 172.21.15.27:0/721220315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 0x7f095c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 -- 172.21.15.27:0/721220315 >> 172.21.15.27:0/721220315 conn(0x7f095c0ff440 msgr2=0x7f095c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 -- 172.21.15.27:0/721220315 shutdown_connections 2024-09-21T07:13:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.399+0000 7f0962000640 1 -- 172.21.15.27:0/721220315 wait complete. 2024-09-21T07:13:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 Processor -- start 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 -- start start 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 0x7f095c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095c1092f0 0x7f095c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 0x7f095c1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f095c113900 con 0x7f095c109c40 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f095c113780 con 0x7f095c1092f0 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.401+0000 7f0962000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f095c113a80 con 0x7f095c1039b0 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 0x7f095c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 0x7f095c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57028/0 (socket says 172.21.15.27:57028) 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 -- 172.21.15.27:0/429370249 learned_addr learned my addr 172.21.15.27:0/429370249 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0961600640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 0x7f095c1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f095be00640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095c1092f0 0x7f095c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 -- 172.21.15.27:0/429370249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095c1092f0 msgr2=0x7f095c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095c1092f0 0x7f095c19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 -- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 msgr2=0x7f095c1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 0x7f095c1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0960c00640 1 -- 172.21.15.27:0/429370249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1a2460 con 0x7f095c1039b0 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.402+0000 7f0961600640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 0x7f095c1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.403+0000 7f0960c00640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 0x7f095c19c4f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f095000e9f0 tx=0x7f095000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.403+0000 7f0959600640 1 -- 172.21.15.27:0/429370249 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095000cd80 con 0x7f095c1039b0 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.403+0000 7f0959600640 1 -- 172.21.15.27:0/429370249 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09500054d0 con 0x7f095c1039b0 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.403+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1a2750 con 0x7f095c1039b0 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.403+0000 7f0959600640 1 -- 172.21.15.27:0/429370249 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950010690 con 0x7f095c1039b0 2024-09-21T07:13:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.403+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a8eb0 con 0x7f095c1039b0 2024-09-21T07:13:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.407+0000 7f096a23a740 1 Processor -- start 2024-09-21T07:13:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.408+0000 7f096a23a740 1 -- start start 2024-09-21T07:13:39.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.408+0000 7f096a23a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d2bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.408+0000 7f096a23a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.408+0000 7f096a23a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.408+0000 7f096a23a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c918ca2fb0 con 0x55c918964390 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.408+0000 7f096a23a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c918c923e0 con 0x55c91895c1b0 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.408+0000 7f096a23a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c918c961f0 con 0x55c918985070 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d2bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f095be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d2c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d2bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57032/0 (socket says 172.21.15.27:57032) 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 -- 172.21.15.27:0/964682776 learned_addr learned my addr 172.21.15.27:0/964682776 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0959600640 1 -- 172.21.15.27:0/429370249 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f0950004e70 con 0x7f095c1039b0 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 -- 172.21.15.27:0/964682776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 msgr2=0x55c918d2c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 -- 172.21.15.27:0/964682776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 msgr2=0x55c918d2c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d2c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f0961600640 1 -- 172.21.15.27:0/964682776 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c918c5ccd0 con 0x55c918985070 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.409+0000 7f095be00640 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d2c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.410+0000 7f0961600640 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d2bf70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0954009470 tx=0x7f09540094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b01a28a84a835aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.410+0000 7f094dc00640 1 -- 172.21.15.27:0/964682776 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954031cf0 con 0x55c918985070 2024-09-21T07:13:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.410+0000 7f094dc00640 1 -- 172.21.15.27:0/964682776 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0954036040 con 0x55c918985070 2024-09-21T07:13:39.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.410+0000 7f094dc00640 1 -- 172.21.15.27:0/964682776 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004240 con 0x55c918985070 2024-09-21T07:13:39.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.411+0000 7f096a23a740 1 -- 172.21.15.27:0/964682776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 msgr2=0x55c918d2bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.411+0000 7f096a23a740 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d2bf70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0954009470 tx=0x7f09540094a0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.411+0000 7f096a23a740 1 -- 172.21.15.27:0/964682776 shutdown_connections 2024-09-21T07:13:39.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.411+0000 7f096a23a740 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d2c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.411+0000 7f096a23a740 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.411+0000 7f096a23a740 1 --2- 172.21.15.27:0/964682776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d2bf70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.411+0000 7f096a23a740 1 -- 172.21.15.27:0/964682776 >> 172.21.15.27:0/964682776 conn(0x55c918979460 msgr2=0x55c918d29e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:39.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.412+0000 7f096a23a740 1 -- 172.21.15.27:0/964682776 shutdown_connections 2024-09-21T07:13:39.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.412+0000 7f096a23a740 1 -- 172.21.15.27:0/964682776 wait complete. 2024-09-21T07:13:39.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.412+0000 7f0959600640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09440b6620 0x7f09440b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.412+0000 7f0959600640 1 -- 172.21.15.27:0/429370249 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f09500e8160 con 0x7f095c1039b0 2024-09-21T07:13:39.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.412+0000 7f095be00640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09440b6620 0x7f09440b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.412+0000 7f096a23a740 1 -- start start 2024-09-21T07:13:39.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.413+0000 7f096a23a740 1 Processor -- start 2024-09-21T07:13:39.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.413+0000 7f095be00640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09440b6620 0x7f09440b8b30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f095c108a20 tx=0x7f0948005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:39.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.413+0000 7f096a23a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d37710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.413+0000 7f096a23a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d37c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f096a23a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f096a23a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c918af3700 con 0x55c918964390 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f096a23a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c918c2d580 con 0x55c91895c1b0 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f096a23a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c918d3a6f0 con 0x55c918985070 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d37710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d37710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53048/0 (socket says 172.21.15.27:53048) 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f096a23a740 10 client.0 _ll_register_callbacks cb 0x55c918c3d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 -- 172.21.15.27:0/1037423551 learned_addr learned my addr 172.21.15.27:0/1037423551 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f095be00640 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d37c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 -- 172.21.15.27:0/1037423551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 msgr2=0x55c918d439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 -- 172.21.15.27:0/1037423551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 msgr2=0x55c918d37c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:39.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d37c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.414+0000 7f0960c00640 1 -- 172.21.15.27:0/1037423551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9189f0650 con 0x55c91895c1b0 2024-09-21T07:13:39.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f0960c00640 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d37710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f095000c6a0 tx=0x7f09500ad920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57fab334b859ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:39.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f0942000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:39.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f0942000640 1 -- 172.21.15.27:0/1037423551 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500b2020 con 0x55c91895c1b0 2024-09-21T07:13:39.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c918c2fb90 con 0x55c91895c1b0 2024-09-21T07:13:39.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f0942000640 1 -- 172.21.15.27:0/1037423551 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f09500e56f0 con 0x55c91895c1b0 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f0942000640 1 -- 172.21.15.27:0/1037423551 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09500e5970 con 0x55c91895c1b0 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f096a23a740 10 client.25751 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c918c4a950 con 0x55c91895c1b0 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.415+0000 7f0942000640 1 -- 172.21.15.27:0/1037423551 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c918c2fb90 con 0x55c91895c1b0 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f0942000640 1 -- 172.21.15.27:0/1037423551 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f09500a93e0 con 0x55c91895c1b0 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f096a23a740 10 client.25751 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f096a23a740 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f096a23a740 20 client.25751 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:39.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f096a23a740 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c9189439f0 0x55c918d49840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f096a23a740 4 client.25751 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:39.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c918ca4f00 con 0x55c9189439f0 2024-09-21T07:13:39.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.416+0000 7f095be00640 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c9189439f0 0x55c918d49840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.417+0000 7f095be00640 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c9189439f0 0x55c918d49840 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2de45cd31fb7d0b server_cookie=efeb92e466472bc9 in_seq=0 out_seq=0 2024-09-21T07:13:39.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.417+0000 7f0942000640 10 client.25751 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.418+0000 7f0942000640 1 -- 172.21.15.27:0/1037423551 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c918ca4f00 con 0x55c9189439f0 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.418+0000 7f0942000640 10 client.25751 handle_command_reply: tid=0 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.418+0000 7f096a23a740 1 client.25751 shutdown 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.418+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 msgr2=0x55c918d37710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.418+0000 7f096a23a740 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d37710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f095000c6a0 tx=0x7f09500ad920 comp rx=0 tx=0).stop 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.418+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 shutdown_connections 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.418+0000 7f096a23a740 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c9189439f0 0x55c918d49840 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f096a23a740 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918985070 0x55c918d439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f096a23a740 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d37c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f096a23a740 1 --2- 172.21.15.27:0/1037423551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d37710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 >> 172.21.15.27:0/1037423551 conn(0x55c918979460 msgr2=0x55c918d28600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:39.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f0942000640 0 client.25751 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:39.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 shutdown_connections 2024-09-21T07:13:39.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f096a23a740 1 -- 172.21.15.27:0/1037423551 wait complete. 2024-09-21T07:13:39.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.419+0000 7f096a23a740 20 client.25751 trim_cache size 0 max 16384 2024-09-21T07:13:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:39 smithi153 ceph-mon[34747]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T07:13:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:39 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2106132844' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:39.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.433+0000 7f096a23a740 1 Processor -- start 2024-09-21T07:13:39.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.433+0000 7f096a23a740 1 -- start start 2024-09-21T07:13:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f096a23a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f096a23a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918964390 0x55c918d39340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f096a23a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918985070 0x55c918d34940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f096a23a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1b5660 con 0x55c918985070 2024-09-21T07:13:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f096a23a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1b2f30 con 0x55c91895c1b0 2024-09-21T07:13:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f096a23a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1b0d80 con 0x55c918964390 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f0960c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f095be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918964390 0x55c918d39340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f0960c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53052/0 (socket says 172.21.15.27:53052) 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f095be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918964390 0x55c918d39340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57036/0 (socket says 172.21.15.27:57036) 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f0960c00640 1 -- 172.21.15.27:0/2661278817 learned_addr learned my addr 172.21.15.27:0/2661278817 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.434+0000 7f0961600640 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918985070 0x55c918d34940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f0960c00640 1 -- 172.21.15.27:0/2661278817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918964390 msgr2=0x55c918d39340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f0960c00640 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918964390 0x55c918d39340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f0960c00640 1 -- 172.21.15.27:0/2661278817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918985070 msgr2=0x55c918d34940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f0960c00640 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918985070 0x55c918d34940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f0960c00640 1 -- 172.21.15.27:0/2661278817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c918d29610 con 0x55c91895c1b0 2024-09-21T07:13:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f0960c00640 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2d960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f09500d3040 tx=0x7f0950009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc6a0896b928b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f094d200640 1 -- 172.21.15.27:0/2661278817 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500b2040 con 0x55c91895c1b0 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.435+0000 7f094d200640 1 -- 172.21.15.27:0/2661278817 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09500ae190 con 0x55c91895c1b0 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.436+0000 7f094d200640 1 -- 172.21.15.27:0/2661278817 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500b74b0 con 0x55c91895c1b0 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 -- 172.21.15.27:0/2661278817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 msgr2=0x55c918d2d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2d960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f09500d3040 tx=0x7f0950009790 comp rx=0 tx=0).stop 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 -- 172.21.15.27:0/2661278817 shutdown_connections 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c918964390 0x55c918d39340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918985070 0x55c918d34940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 --2- 172.21.15.27:0/2661278817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91895c1b0 0x55c918d2d960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 -- 172.21.15.27:0/2661278817 >> 172.21.15.27:0/2661278817 conn(0x55c918979460 msgr2=0x55c918d38c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 -- 172.21.15.27:0/2661278817 shutdown_connections 2024-09-21T07:13:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 -- 172.21.15.27:0/2661278817 wait complete. 2024-09-21T07:13:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.437+0000 7f096a23a740 1 -- start start 2024-09-21T07:13:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 1 Processor -- start 2024-09-21T07:13:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91895c1b0 0x55c918d3fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d40150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c918985070 0x55c918d3a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1b13f0 con 0x55c918964390 2024-09-21T07:13:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1b2860 con 0x55c918985070 2024-09-21T07:13:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1b1ac0 con 0x55c91895c1b0 2024-09-21T07:13:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.438+0000 7f096a23a740 10 client.0 _ll_register_callbacks cb 0x7f09500a9930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0960c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91895c1b0 0x55c918d3fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0960c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91895c1b0 0x55c918d3fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57042/0 (socket says 172.21.15.27:57042) 2024-09-21T07:13:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0960c00640 1 -- 172.21.15.27:0/2035128553 learned_addr learned my addr 172.21.15.27:0/2035128553 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0961600640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c918985070 0x55c918d3a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f095be00640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d40150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0961600640 1 -- 172.21.15.27:0/2035128553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91895c1b0 msgr2=0x55c918d3fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0961600640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91895c1b0 0x55c918d3fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0961600640 1 -- 172.21.15.27:0/2035128553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 msgr2=0x55c918d40150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0961600640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d40150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0961600640 1 -- 172.21.15.27:0/2035128553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c918d3b040 con 0x55c918985070 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.439+0000 7f0960c00640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91895c1b0 0x55c918d3fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f0961600640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c918985070 0x55c918d3a900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0954005cc0 tx=0x7f095402f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef6b4f939ec142db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f0942000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f0942000640 1 -- 172.21.15.27:0/2035128553 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954051020 con 0x55c918985070 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f0942000640 1 -- 172.21.15.27:0/2035128553 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0954031d80 con 0x55c918985070 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09500ae190 con 0x55c918985070 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f0942000640 1 -- 172.21.15.27:0/2035128553 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f095405c050 con 0x55c918985070 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f096a23a740 10 client.25757 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.440+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c918db4bc0 con 0x55c918985070 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f0942000640 1 -- 172.21.15.27:0/2035128553 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09500ae190 con 0x55c918985070 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f0942000640 1 -- 172.21.15.27:0/2035128553 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0954004040 con 0x55c918985070 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f096a23a740 10 client.25757 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f096a23a740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f096a23a740 20 client.25757 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f096a23a740 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c918918da0 0x55c918db6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f096a23a740 4 client.25757 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c918ca09b0 con 0x55c918918da0 2024-09-21T07:13:39.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.441+0000 7f0960c00640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c918918da0 0x55c918db6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f0960c00640 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c918918da0 0x55c918db6f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646b95d814cb59e server_cookie=1ff3b551723b807c in_seq=0 out_seq=0 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f0942000640 10 client.25757 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f0942000640 1 -- 172.21.15.27:0/2035128553 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c918ca09b0 con 0x55c918918da0 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f0942000640 10 client.25757 handle_command_reply: tid=0 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f096a23a740 1 client.25757 shutdown 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c918985070 msgr2=0x55c918d3a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f096a23a740 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c918985070 0x55c918d3a900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0954005cc0 tx=0x7f095402f310 comp rx=0 tx=0).stop 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.442+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 shutdown_connections 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c918918da0 0x55c918db6f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c918985070 0x55c918d3a900 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c918964390 0x55c918d40150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 1 --2- 172.21.15.27:0/2035128553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91895c1b0 0x55c918d3fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f0942000640 0 client.25757 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 >> 172.21.15.27:0/2035128553 conn(0x55c918979460 msgr2=0x55c918d28d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 shutdown_connections 2024-09-21T07:13:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 1 -- 172.21.15.27:0/2035128553 wait complete. 2024-09-21T07:13:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.443+0000 7f096a23a740 20 client.25757 trim_cache size 0 max 16384 2024-09-21T07:13:39.447 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09440b6620 msgr2=0x7f09440b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09440b6620 0x7f09440b8b30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f095c108a20 tx=0x7f0948005f90 comp rx=0 tx=0).stop 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 msgr2=0x7f095c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 0x7f095c19c4f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f095000e9f0 tx=0x7f095000edf0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 shutdown_connections 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09440b6620 0x7f09440b8b30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c109c40 0x7f095c1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095c1092f0 0x7f095c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 --2- 172.21.15.27:0/429370249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f095c1039b0 0x7f095c19c4f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 >> 172.21.15.27:0/429370249 conn(0x7f095c0ff440 msgr2=0x7f095c1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 shutdown_connections 2024-09-21T07:13:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:39.444+0000 7f0962000640 1 -- 172.21.15.27:0/429370249 wait complete. 2024-09-21T07:13:39.702 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T07:13:39.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T07:13:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:41.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.172+0000 7f954de00640 1 -- 172.21.15.27:0/2337456694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109330 msgr2=0x7f9548109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.172+0000 7f954de00640 1 --2- 172.21.15.27:0/2337456694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109330 0x7f9548109740 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9534009900 tx=0x7f953402a410 comp rx=0 tx=0).stop 2024-09-21T07:13:41.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.173+0000 7f954de00640 1 -- 172.21.15.27:0/2337456694 shutdown_connections 2024-09-21T07:13:41.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.173+0000 7f954de00640 1 --2- 172.21.15.27:0/2337456694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9548109c80 0x7f95481104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.173+0000 7f954de00640 1 --2- 172.21.15.27:0/2337456694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109330 0x7f9548109740 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.173+0000 7f954de00640 1 --2- 172.21.15.27:0/2337456694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 0x7f9548103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.173+0000 7f954de00640 1 -- 172.21.15.27:0/2337456694 >> 172.21.15.27:0/2337456694 conn(0x7f95480ff480 msgr2=0x7f95481018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:41.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.173+0000 7f954de00640 1 -- 172.21.15.27:0/2337456694 shutdown_connections 2024-09-21T07:13:41.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.173+0000 7f954de00640 1 -- 172.21.15.27:0/2337456694 wait complete. 2024-09-21T07:13:41.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.174+0000 7f954de00640 1 Processor -- start 2024-09-21T07:13:41.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954de00640 1 -- start start 2024-09-21T07:13:41.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 0x7f954819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9548109330 0x7f954819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109c80 0x7f95481a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954de00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9548113990 con 0x7f9548109c80 2024-09-21T07:13:41.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954de00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9548113810 con 0x7f9548109330 2024-09-21T07:13:41.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954de00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9548113b10 con 0x7f95481039f0 2024-09-21T07:13:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 0x7f954819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 0x7f954819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57056/0 (socket says 172.21.15.27:57056) 2024-09-21T07:13:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.175+0000 7f954ca00640 1 -- 172.21.15.27:0/2260714800 learned_addr learned my addr 172.21.15.27:0/2260714800 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f954d400640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109c80 0x7f95481a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f9547e00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9548109330 0x7f954819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f954ca00640 1 -- 172.21.15.27:0/2260714800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9548109330 msgr2=0x7f954819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f954ca00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9548109330 0x7f954819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f954ca00640 1 -- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109c80 msgr2=0x7f95481a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f954ca00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109c80 0x7f95481a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f954ca00640 1 -- 172.21.15.27:0/2260714800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95481a24d0 con 0x7f95481039f0 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.176+0000 7f954ca00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 0x7f954819c560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e9f0 tx=0x7f953c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.177+0000 7f9545600640 1 -- 172.21.15.27:0/2260714800 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c00cd80 con 0x7f95481039f0 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.177+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95481a27c0 con 0x7f95481039f0 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.177+0000 7f9545600640 1 -- 172.21.15.27:0/2260714800 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c0108e0 con 0x7f95481039f0 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.177+0000 7f9545600640 1 -- 172.21.15.27:0/2260714800 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c018980 con 0x7f95481039f0 2024-09-21T07:13:41.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.177+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95481a8f20 con 0x7f95481039f0 2024-09-21T07:13:41.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.180+0000 7f955620e740 1 Processor -- start 2024-09-21T07:13:41.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.180+0000 7f955620e740 1 -- start start 2024-09-21T07:13:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f955620e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f890f840 0x55a5f8cb7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f955620e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88e6980 0x55a5f8cb7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f955620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88eeb60 0x55a5f8cb7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f955620e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5f8c2cca0 con 0x55a5f890f840 2024-09-21T07:13:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f955620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5f8c1c0d0 con 0x55a5f88eeb60 2024-09-21T07:13:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f955620e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a5f8c1fee0 con 0x55a5f88e6980 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f954d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f890f840 0x55a5f8cb7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f954d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f890f840 0x55a5f8cb7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58918/0 (socket says 172.21.15.27:58918) 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f954d400640 1 -- 172.21.15.27:0/2999409811 learned_addr learned my addr 172.21.15.27:0/2999409811 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.181+0000 7f9547e00640 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88eeb60 0x55a5f8cb7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f954d400640 1 -- 172.21.15.27:0/2999409811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88e6980 msgr2=0x55a5f8cb7a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f954d400640 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88e6980 0x55a5f8cb7a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f954d400640 1 -- 172.21.15.27:0/2999409811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88eeb60 msgr2=0x55a5f8cb7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f954d400640 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88eeb60 0x55a5f8cb7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f954d400640 1 -- 172.21.15.27:0/2999409811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5f8bce400 con 0x55a5f890f840 2024-09-21T07:13:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f9547e00640 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88eeb60 0x55a5f8cb7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f9545600640 1 -- 172.21.15.27:0/2260714800 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f953c010430 con 0x7f95481039f0 2024-09-21T07:13:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.182+0000 7f954d400640 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f890f840 0x55a5f8cb7510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9540009fd0 tx=0x7f95400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4e7d4438cd63b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.183+0000 7f9539c00640 1 -- 172.21.15.27:0/2999409811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540004d20 con 0x55a5f890f840 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.183+0000 7f9539c00640 1 -- 172.21.15.27:0/2999409811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9540004050 con 0x55a5f890f840 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.183+0000 7f9539c00640 1 -- 172.21.15.27:0/2999409811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540005280 con 0x55a5f890f840 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 -- 172.21.15.27:0/2999409811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f890f840 msgr2=0x55a5f8cb7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f890f840 0x55a5f8cb7510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9540009fd0 tx=0x7f95400118d0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 -- 172.21.15.27:0/2999409811 shutdown_connections 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88e6980 0x55a5f8cb7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88eeb60 0x55a5f8cb7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 --2- 172.21.15.27:0/2999409811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f890f840 0x55a5f8cb7510 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 -- 172.21.15.27:0/2999409811 >> 172.21.15.27:0/2999409811 conn(0x55a5f8903990 msgr2=0x55a5f8cb5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 -- 172.21.15.27:0/2999409811 shutdown_connections 2024-09-21T07:13:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.184+0000 7f955620e740 1 -- 172.21.15.27:0/2999409811 wait complete. 2024-09-21T07:13:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.185+0000 7f955620e740 1 -- start start 2024-09-21T07:13:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.185+0000 7f9545600640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95300b6690 0x7f95300b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.185+0000 7f9545600640 1 -- 172.21.15.27:0/2260714800 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f953c0e7fa0 con 0x7f95481039f0 2024-09-21T07:13:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.185+0000 7f955620e740 1 Processor -- start 2024-09-21T07:13:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.185+0000 7f954d400640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95300b6690 0x7f95300b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f955620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88e6980 0x55a5f8cc2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f955620e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88eeb60 0x55a5f8cc31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f955620e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8ccefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f955620e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5f8a7d7a0 con 0x55a5f88eeb60 2024-09-21T07:13:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f955620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5f8bc1a10 con 0x55a5f88e6980 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f955620e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a5f8cc5cf0 con 0x55a5f890f840 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f955620e740 10 client.0 _ll_register_callbacks cb 0x55a5f8bd1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f9547e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88eeb60 0x55a5f8cc31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f9547e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88eeb60 0x55a5f8cc31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58924/0 (socket says 172.21.15.27:58924) 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f9547e00640 1 -- 172.21.15.27:0/170822660 learned_addr learned my addr 172.21.15.27:0/170822660 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f954d400640 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8ccefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f954ca00640 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88e6980 0x55a5f8cc2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.186+0000 7f954d400640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95300b6690 0x7f95300b8ba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9540011dd0 tx=0x7f9540005fd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.187+0000 7f9547e00640 1 -- 172.21.15.27:0/170822660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 msgr2=0x55a5f8ccefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.187+0000 7f9547e00640 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8ccefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.187+0000 7f9547e00640 1 -- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88e6980 msgr2=0x55a5f8cc2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.187+0000 7f9547e00640 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88e6980 0x55a5f8cc2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.187+0000 7f9547e00640 1 -- 172.21.15.27:0/170822660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5f8989b90 con 0x55a5f88eeb60 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.187+0000 7f9547e00640 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88eeb60 0x55a5f8cc31c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9534000c00 tx=0x7f95340043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d1dccef0ab1f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.187+0000 7f952e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.188+0000 7f952e000640 1 -- 172.21.15.27:0/170822660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534035040 con 0x55a5f88eeb60 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.188+0000 7f952e000640 1 -- 172.21.15.27:0/170822660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9534005450 con 0x55a5f88eeb60 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.188+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9540004050 con 0x55a5f88eeb60 2024-09-21T07:13:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.188+0000 7f952e000640 1 -- 172.21.15.27:0/170822660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9534005650 con 0x55a5f88eeb60 2024-09-21T07:13:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.188+0000 7f955620e740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.188+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5f8cd1a80 con 0x55a5f88eeb60 2024-09-21T07:13:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.188+0000 7f952e000640 1 -- 172.21.15.27:0/170822660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9540004050 con 0x55a5f88eeb60 2024-09-21T07:13:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f952e000640 1 -- 172.21.15.27:0/170822660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9534003a30 con 0x55a5f88eeb60 2024-09-21T07:13:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f955620e740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f955620e740 10 client.15744 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:13:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f955620e740 20 client.15744 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:41.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f955620e740 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f88ce1c0 0x55a5f8cd5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f955620e740 4 client.15744 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:41.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5f8c2ebf0 con 0x55a5f88ce1c0 2024-09-21T07:13:41.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.189+0000 7f954ca00640 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f88ce1c0 0x55a5f8cd5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.190+0000 7f954ca00640 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f88ce1c0 0x55a5f8cd5260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7422dbf2e8a2ac9d server_cookie=92a6c11db90be42 in_seq=0 out_seq=0 2024-09-21T07:13:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.190+0000 7f952e000640 10 client.15744 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.190+0000 7f952e000640 1 -- 172.21.15.27:0/170822660 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5f8c2ebf0 con 0x55a5f88ce1c0 2024-09-21T07:13:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.190+0000 7f952e000640 10 client.15744 handle_command_reply: tid=0 2024-09-21T07:13:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 client.15744 shutdown 2024-09-21T07:13:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88eeb60 msgr2=0x55a5f8cc31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88eeb60 0x55a5f8cc31c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9534000c00 tx=0x7f95340043b0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 shutdown_connections 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f88ce1c0 0x55a5f8cd5260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8ccefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88eeb60 0x55a5f8cc31c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 --2- 172.21.15.27:0/170822660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88e6980 0x55a5f8cc2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 >> 172.21.15.27:0/170822660 conn(0x55a5f8903990 msgr2=0x55a5f8cba6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f952e000640 0 client.15744 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 shutdown_connections 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 1 -- 172.21.15.27:0/170822660 wait complete. 2024-09-21T07:13:41.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.191+0000 7f955620e740 20 client.15744 trim_cache size 0 max 16384 2024-09-21T07:13:41.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 Processor -- start 2024-09-21T07:13:41.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 -- start start 2024-09-21T07:13:41.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:41 smithi027 ceph-mon[30779]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-21T07:13:41.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cbad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8cbfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88a3570 0x55a5f8cc03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b5710 con 0x55a5f88e6980 2024-09-21T07:13:41.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b2fe0 con 0x55a5f88a3570 2024-09-21T07:13:41.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.210+0000 7f955620e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b0e30 con 0x55a5f890f840 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cbad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cbad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58940/0 (socket says 172.21.15.27:58940) 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 -- 172.21.15.27:0/1055401019 learned_addr learned my addr 172.21.15.27:0/1055401019 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f9547e00640 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8cbfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954d400640 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88a3570 0x55a5f8cc03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 -- 172.21.15.27:0/1055401019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 msgr2=0x55a5f8cbfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8cbfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 -- 172.21.15.27:0/1055401019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88a3570 msgr2=0x55a5f8cc03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88a3570 0x55a5f8cc03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.211+0000 7f954ca00640 1 -- 172.21.15.27:0/1055401019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5f8cc53a0 con 0x55a5f88e6980 2024-09-21T07:13:41.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.212+0000 7f954ca00640 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cbad20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e9c0 tx=0x7f953c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b9bf06b474e6af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.212+0000 7f9539200640 1 -- 172.21.15.27:0/1055401019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c0ad490 con 0x55a5f88e6980 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.212+0000 7f9539200640 1 -- 172.21.15.27:0/1055401019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c0b23b0 con 0x55a5f88e6980 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.212+0000 7f9539200640 1 -- 172.21.15.27:0/1055401019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c018b20 con 0x55a5f88e6980 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.213+0000 7f955620e740 1 -- 172.21.15.27:0/1055401019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 msgr2=0x55a5f8cbad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.213+0000 7f955620e740 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cbad20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e9c0 tx=0x7f953c018420 comp rx=0 tx=0).stop 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.213+0000 7f955620e740 1 -- 172.21.15.27:0/1055401019 shutdown_connections 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.213+0000 7f955620e740 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f890f840 0x55a5f8cbfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.213+0000 7f955620e740 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f88a3570 0x55a5f8cc03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.213+0000 7f955620e740 1 --2- 172.21.15.27:0/1055401019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cbad20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.213+0000 7f955620e740 1 -- 172.21.15.27:0/1055401019 >> 172.21.15.27:0/1055401019 conn(0x55a5f8903990 msgr2=0x55a5f8cba610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:41.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.214+0000 7f955620e740 1 -- 172.21.15.27:0/1055401019 shutdown_connections 2024-09-21T07:13:41.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.214+0000 7f955620e740 1 -- 172.21.15.27:0/1055401019 wait complete. 2024-09-21T07:13:41.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.214+0000 7f955620e740 1 -- start start 2024-09-21T07:13:41.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.214+0000 7f955620e740 1 Processor -- start 2024-09-21T07:13:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f955620e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88a3570 0x55a5f8cc7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f955620e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f955620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f890f840 0x55a5f8d38ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f955620e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b14a0 con 0x55a5f88e6980 2024-09-21T07:13:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f955620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b2910 con 0x55a5f890f840 2024-09-21T07:13:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f955620e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b1b70 con 0x55a5f88a3570 2024-09-21T07:13:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f955620e740 10 client.0 _ll_register_callbacks cb 0x7f9534009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f9547e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cc7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f954ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88a3570 0x55a5f8cc7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f9547e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cc7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58954/0 (socket says 172.21.15.27:58954) 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.215+0000 7f9547e00640 1 -- 172.21.15.27:0/3114727817 learned_addr learned my addr 172.21.15.27:0/3114727817 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f954d400640 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f890f840 0x55a5f8d38ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f9547e00640 1 -- 172.21.15.27:0/3114727817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88a3570 msgr2=0x55a5f8cc7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f9547e00640 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88a3570 0x55a5f8cc7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f9547e00640 1 -- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f890f840 msgr2=0x55a5f8d38ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f9547e00640 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f890f840 0x55a5f8d38ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f9547e00640 1 -- 172.21.15.27:0/3114727817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5f8d39310 con 0x55a5f88e6980 2024-09-21T07:13:41.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f954d400640 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f890f840 0x55a5f8d38ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:41.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f9547e00640 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cc7e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f953402a450 tx=0x7f95340314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f4c7dbfd0b24f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:41.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.216+0000 7f952e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:13:41.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.217+0000 7f952e000640 1 -- 172.21.15.27:0/3114727817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534059060 con 0x55a5f88e6980 2024-09-21T07:13:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.217+0000 7f952e000640 1 -- 172.21.15.27:0/3114727817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9534031b40 con 0x55a5f88e6980 2024-09-21T07:13:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.217+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953c0b23b0 con 0x55a5f88e6980 2024-09-21T07:13:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.217+0000 7f952e000640 1 -- 172.21.15.27:0/3114727817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9534035070 con 0x55a5f88e6980 2024-09-21T07:13:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 10 client.15750 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5f8d405f0 con 0x55a5f88e6980 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f952e000640 1 -- 172.21.15.27:0/3114727817 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c0b23b0 con 0x55a5f88e6980 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f952e000640 1 -- 172.21.15.27:0/3114727817 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9534004750 con 0x55a5f88e6980 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 10 client.15750 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 20 client.15750 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f8d40de0 0x55a5f8d43200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 4 client.15750 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T07:13:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.218+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x55a5f8c2a6a0 con 0x55a5f8d40de0 2024-09-21T07:13:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.219+0000 7f954ca00640 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f8d40de0 0x55a5f8d43200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.219+0000 7f954ca00640 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f8d40de0 0x55a5f8d43200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be870eff2fb9e4fb server_cookie=9938b3ab4e571ac9 in_seq=0 out_seq=0 2024-09-21T07:13:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.219+0000 7f952e000640 10 client.15750 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.232+0000 7f952e000640 1 -- 172.21.15.27:0/3114727817 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55a5f8c2a6a0 con 0x55a5f8d40de0 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.232+0000 7f952e000640 10 client.15750 handle_command_reply: tid=0 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.232+0000 7f955620e740 1 client.15750 shutdown 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.232+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 msgr2=0x55a5f8cc7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.232+0000 7f955620e740 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cc7e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f953402a450 tx=0x7f95340314a0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 shutdown_connections 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a5f8d40de0 0x55a5f8d43200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f890f840 0x55a5f8d38ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5f88e6980 0x55a5f8cc7e10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 --2- 172.21.15.27:0/3114727817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5f88a3570 0x55a5f8cc7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f952e000640 0 client.15750 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 >> 172.21.15.27:0/3114727817 conn(0x55a5f8903990 msgr2=0x55a5f8cc40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:41.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 shutdown_connections 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 1 -- 172.21.15.27:0/3114727817 wait complete. 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.233+0000 7f955620e740 20 client.15750 trim_cache size 0 max 16384 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 2, 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "2b5ff92": { 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 2, 2024-09-21T07:13:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 134.5, 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95300b6690 msgr2=0x7f95300b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95300b6690 0x7f95300b8ba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9540011dd0 tx=0x7f9540005fd0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 msgr2=0x7f954819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 0x7f954819c560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e9f0 tx=0x7f953c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 shutdown_connections 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95300b6690 0x7f95300b8ba0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9548109c80 0x7f95481a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9548109330 0x7f954819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 --2- 172.21.15.27:0/2260714800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95481039f0 0x7f954819c560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 >> 172.21.15.27:0/2260714800 conn(0x7f95480ff480 msgr2=0x7f954810e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 shutdown_connections 2024-09-21T07:13:41.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:41.234+0000 7f954de00640 1 -- 172.21.15.27:0/2260714800 wait complete. 2024-09-21T07:13:41.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:13:41.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:13:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:41 smithi153 ceph-mon[34747]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-21T07:13:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:41 smithi119 ceph-mon[34519]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-21T07:13:41.998 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2b5ff92', quiesce took 5.9 seconds. Will release after: 8.6 2024-09-21T07:13:42.165 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.818+0000 7fc672000640 1 -- 172.21.15.27:0/2175348493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c105d80 msgr2=0x7fc66c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.818+0000 7fc672000640 1 --2- 172.21.15.27:0/2175348493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c105d80 0x7fc66c106190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009900 tx=0x7fc65c02a410 comp rx=0 tx=0).stop 2024-09-21T07:13:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.819+0000 7fc672000640 1 -- 172.21.15.27:0/2175348493 shutdown_connections 2024-09-21T07:13:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.819+0000 7fc672000640 1 --2- 172.21.15.27:0/2175348493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc66c10c010 0x7fc66c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.819+0000 7fc672000640 1 --2- 172.21.15.27:0/2175348493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c10b6c0 0x7fc66c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.819+0000 7fc672000640 1 --2- 172.21.15.27:0/2175348493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c105d80 0x7fc66c106190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.819+0000 7fc672000640 1 -- 172.21.15.27:0/2175348493 >> 172.21.15.27:0/2175348493 conn(0x7fc66c101810 msgr2=0x7fc66c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.819+0000 7fc672000640 1 -- 172.21.15.27:0/2175348493 shutdown_connections 2024-09-21T07:13:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.819+0000 7fc672000640 1 -- 172.21.15.27:0/2175348493 wait complete. 2024-09-21T07:13:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.820+0000 7fc672000640 1 Processor -- start 2024-09-21T07:13:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.820+0000 7fc672000640 1 -- start start 2024-09-21T07:13:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.820+0000 7fc672000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc66c105d80 0x7fc66c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc672000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c10b6c0 0x7fc66c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc672000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c10c010 0x7fc66c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc672000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c115cd0 con 0x7fc66c10c010 2024-09-21T07:13:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc672000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c115b50 con 0x7fc66c10b6c0 2024-09-21T07:13:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc672000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c115e50 con 0x7fc66c105d80 2024-09-21T07:13:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c10b6c0 0x7fc66c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc66be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c10c010 0x7fc66c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c10b6c0 0x7fc66c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53094/0 (socket says 172.21.15.27:53094) 2024-09-21T07:13:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc66aa00640 1 -- 172.21.15.27:0/1445104059 learned_addr learned my addr 172.21.15.27:0/1445104059 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.821+0000 7fc66b400640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc66c105d80 0x7fc66c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc66b400640 1 -- 172.21.15.27:0/1445104059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c10b6c0 msgr2=0x7fc66c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc66b400640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c10b6c0 0x7fc66c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc66b400640 1 -- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c10c010 msgr2=0x7fc66c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc66b400640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c10c010 0x7fc66c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc66b400640 1 -- 172.21.15.27:0/1445104059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c1a2690 con 0x7fc66c105d80 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc66b400640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc66c105d80 0x7fc66c19c720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02a930 tx=0x7fc65c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc663e00640 1 -- 172.21.15.27:0/1445104059 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c03f030 con 0x7fc66c105d80 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.822+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c1a2920 con 0x7fc66c105d80 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.823+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c1a90e0 con 0x7fc66c105d80 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.823+0000 7fc663e00640 1 -- 172.21.15.27:0/1445104059 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65c004490 con 0x7fc66c105d80 2024-09-21T07:13:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.823+0000 7fc663e00640 1 -- 172.21.15.27:0/1445104059 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c049060 con 0x7fc66c105d80 2024-09-21T07:13:42.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.827+0000 7fc663e00640 1 -- 172.21.15.27:0/1445104059 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fc65c042030 con 0x7fc66c105d80 2024-09-21T07:13:42.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.827+0000 7fc678cc7740 1 Processor -- start 2024-09-21T07:13:42.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.828+0000 7fc678cc7740 1 -- start start 2024-09-21T07:13:42.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.828+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ea7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.828+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708ea8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.828+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708ea8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.828+0000 7fc678cc7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557708e1de90 con 0x557708b00350 2024-09-21T07:13:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.828+0000 7fc678cc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557708e0d2c0 con 0x557708ad7490 2024-09-21T07:13:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.828+0000 7fc678cc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557708e110d0 con 0x557708adf670 2024-09-21T07:13:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708ea8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc663e00640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc6500b6710 0x7fc6500b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708ea8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53098/0 (socket says 172.21.15.27:53098) 2024-09-21T07:13:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66b400640 1 -- 172.21.15.27:0/1559032404 learned_addr learned my addr 172.21.15.27:0/1559032404 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66aa00640 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708ea8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66be00640 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ea7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc663e00640 1 -- 172.21.15.27:0/1445104059 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc65c107950 con 0x7fc66c105d80 2024-09-21T07:13:42.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66be00640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc6500b6710 0x7fc6500b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66b400640 1 -- 172.21.15.27:0/1559032404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 msgr2=0x557708ea8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.829+0000 7fc66b400640 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708ea8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc66b400640 1 -- 172.21.15.27:0/1559032404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 msgr2=0x557708ea7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc66b400640 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ea7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc66b400640 1 -- 172.21.15.27:0/1559032404 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557708b6b930 con 0x557708ad7490 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc66b400640 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708ea8250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc65c006010 tx=0x7fc65c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7043ba811fc70c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc657e00640 1 -- 172.21.15.27:0/1559032404 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c035030 con 0x557708ad7490 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc657e00640 1 -- 172.21.15.27:0/1559032404 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65c0d1d60 con 0x557708ad7490 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc657e00640 1 -- 172.21.15.27:0/1559032404 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c105140 con 0x557708ad7490 2024-09-21T07:13:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.830+0000 7fc66be00640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc6500b6710 0x7fc6500b8c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc664005fd0 tx=0x7fc664009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.831+0000 7fc678cc7740 1 -- 172.21.15.27:0/1559032404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 msgr2=0x557708ea8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.831+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708ea8250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc65c006010 tx=0x7fc65c02a410 comp rx=0 tx=0).stop 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 -- 172.21.15.27:0/1559032404 shutdown_connections 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708ea8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708ea8250 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1559032404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ea7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 -- 172.21.15.27:0/1559032404 >> 172.21.15.27:0/1559032404 conn(0x557708af4740 msgr2=0x557708ea5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 -- 172.21.15.27:0/1559032404 shutdown_connections 2024-09-21T07:13:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 -- 172.21.15.27:0/1559032404 wait complete. 2024-09-21T07:13:42.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.832+0000 7fc678cc7740 1 -- start start 2024-09-21T07:13:42.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.833+0000 7fc678cc7740 1 Processor -- start 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.833+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708eb3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.833+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708eb3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.833+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ebf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc678cc7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557708c6e890 con 0x557708b00350 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc678cc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557708da8160 con 0x557708ad7490 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc678cc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557708e0d2c0 con 0x557708adf670 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc678cc7740 10 client.0 _ll_register_callbacks cb 0x557708dc4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708eb3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708eb3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53112/0 (socket says 172.21.15.27:53112) 2024-09-21T07:13:42.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 -- 172.21.15.27:0/1390882124 learned_addr learned my addr 172.21.15.27:0/1390882124 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:42.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66aa00640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708eb3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66be00640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ebf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 -- 172.21.15.27:0/1390882124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 msgr2=0x557708eb3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708eb3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 -- 172.21.15.27:0/1390882124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 msgr2=0x557708ebf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ebf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.834+0000 7fc66b400640 1 -- 172.21.15.27:0/1390882124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557708ebfa90 con 0x557708ad7490 2024-09-21T07:13:42.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.835+0000 7fc66aa00640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708eb3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:42.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.835+0000 7fc66b400640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708eb3960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc65c03d040 tx=0x7fc65c0c9830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1373987fc673ea9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.835+0000 7fc654c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.835+0000 7fc654c00640 1 -- 172.21.15.27:0/1390882124 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c035070 con 0x557708ad7490 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.835+0000 7fc654c00640 1 -- 172.21.15.27:0/1390882124 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc65c1050e0 con 0x557708ad7490 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.835+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc65c0d1d60 con 0x557708ad7490 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.835+0000 7fc654c00640 1 -- 172.21.15.27:0/1390882124 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc65c0d6a20 con 0x557708ad7490 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 10 client.25787 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557708ec23c0 con 0x557708ad7490 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc654c00640 1 -- 172.21.15.27:0/1390882124 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65c0d1d60 con 0x557708ad7490 2024-09-21T07:13:42.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc654c00640 1 -- 172.21.15.27:0/1390882124 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc65c04c060 con 0x557708ad7490 2024-09-21T07:13:42.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 10 client.25787 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:42.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:42.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 20 client.25787 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:42.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708abecd0 0x557708ec5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 4 client.25787 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:42.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.836+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557708e1fde0 con 0x557708abecd0 2024-09-21T07:13:42.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.837+0000 7fc66aa00640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708abecd0 0x557708ec5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.837+0000 7fc66aa00640 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708abecd0 0x557708ec5740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1a49ef73704264 server_cookie=7fa9481006d11177 in_seq=0 out_seq=0 2024-09-21T07:13:42.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.837+0000 7fc654c00640 10 client.25787 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:42.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.838+0000 7fc654c00640 1 -- 172.21.15.27:0/1390882124 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557708e1fde0 con 0x557708abecd0 2024-09-21T07:13:42.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.838+0000 7fc654c00640 10 client.25787 handle_command_reply: tid=0 2024-09-21T07:13:42.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.838+0000 7fc678cc7740 1 client.25787 shutdown 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 msgr2=0x557708eb3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708eb3960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc65c03d040 tx=0x7fc65c0c9830 comp rx=0 tx=0).stop 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 shutdown_connections 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708abecd0 0x557708ec5740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708adf670 0x557708eb3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708b00350 0x557708ebf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1390882124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708ad7490 0x557708eb3960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc654c00640 0 client.25787 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:42.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 >> 172.21.15.27:0/1390882124 conn(0x557708af4740 msgr2=0x557708ea4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:42.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 shutdown_connections 2024-09-21T07:13:42.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 1 -- 172.21.15.27:0/1390882124 wait complete. 2024-09-21T07:13:42.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.839+0000 7fc678cc7740 20 client.25787 trim_cache size 0 max 16384 2024-09-21T07:13:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.858+0000 7fc678cc7740 1 Processor -- start 2024-09-21T07:13:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.858+0000 7fc678cc7740 1 -- start start 2024-09-21T07:13:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ea4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ea47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc678cc7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c012710 con 0x557708adf670 2024-09-21T07:13:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc678cc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00ffe0 con 0x557708b00350 2024-09-21T07:13:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc678cc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00de30 con 0x557708ad7490 2024-09-21T07:13:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ea47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ea47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58998/0 (socket says 172.21.15.27:58998) 2024-09-21T07:13:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc66aa00640 1 -- 172.21.15.27:0/1670616938 learned_addr learned my addr 172.21.15.27:0/1670616938 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc66b400640 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ea4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.859+0000 7fc66be00640 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc66b400640 1 -- 172.21.15.27:0/1670616938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 msgr2=0x557708eace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc66b400640 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc66b400640 1 -- 172.21.15.27:0/1670616938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 msgr2=0x557708ea47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc66b400640 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ea47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc66b400640 1 -- 172.21.15.27:0/1670616938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557708eb6140 con 0x557708ad7490 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc66b400640 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ea4280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005f70 tx=0x7fc65c0cd100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8771e565ef51d7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc657400640 1 -- 172.21.15.27:0/1670616938 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0c9980 con 0x557708ad7490 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc657400640 1 -- 172.21.15.27:0/1670616938 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65c0d6ca0 con 0x557708ad7490 2024-09-21T07:13:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.860+0000 7fc657400640 1 -- 172.21.15.27:0/1670616938 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c104ee0 con 0x557708ad7490 2024-09-21T07:13:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.861+0000 7fc678cc7740 1 -- 172.21.15.27:0/1670616938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 msgr2=0x557708ea4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.861+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ea4280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005f70 tx=0x7fc65c0cd100 comp rx=0 tx=0).stop 2024-09-21T07:13:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 -- 172.21.15.27:0/1670616938 shutdown_connections 2024-09-21T07:13:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ea47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 --2- 172.21.15.27:0/1670616938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ea4280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 -- 172.21.15.27:0/1670616938 >> 172.21.15.27:0/1670616938 conn(0x557708af4740 msgr2=0x557708eb1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:42.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 -- 172.21.15.27:0/1670616938 shutdown_connections 2024-09-21T07:13:42.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 -- 172.21.15.27:0/1670616938 wait complete. 2024-09-21T07:13:42.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.862+0000 7fc678cc7740 1 -- start start 2024-09-21T07:13:42.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.863+0000 7fc678cc7740 1 Processor -- start 2024-09-21T07:13:42.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ebb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ebb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc678cc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eb63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc678cc7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00e4a0 con 0x557708adf670 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc678cc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00f910 con 0x557708b00350 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc678cc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00eb70 con 0x557708ad7490 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc678cc7740 10 client.0 _ll_register_callbacks cb 0x7fc664009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc66b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ebb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc66b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ebb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57124/0 (socket says 172.21.15.27:57124) 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc66b400640 1 -- 172.21.15.27:0/265204220 learned_addr learned my addr 172.21.15.27:0/265204220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc66aa00640 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ebb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc66be00640 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eb63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc66b400640 1 -- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 msgr2=0x557708eb63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.864+0000 7fc66b400640 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eb63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.865+0000 7fc66b400640 1 -- 172.21.15.27:0/265204220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 msgr2=0x557708ebb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.865+0000 7fc66b400640 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ebb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.865+0000 7fc66b400640 1 -- 172.21.15.27:0/265204220 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557708ea6990 con 0x557708ad7490 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.865+0000 7fc66b400640 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ebb3f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0ccc10 tx=0x7fc65c0cc230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6783c6d05b0aafd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.865+0000 7fc654c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.865+0000 7fc654c00640 1 -- 172.21.15.27:0/265204220 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0e8060 con 0x557708ad7490 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.865+0000 7fc654c00640 1 -- 172.21.15.27:0/265204220 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc65c1050f0 con 0x557708ad7490 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc65c0d6ca0 con 0x557708ad7490 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc654c00640 1 -- 172.21.15.27:0/265204220 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc65c0ed050 con 0x557708ad7490 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 10 client.25783 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557708ea6990 con 0x557708ad7490 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc654c00640 1 -- 172.21.15.27:0/265204220 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65c0d6ca0 con 0x557708ad7490 2024-09-21T07:13:42.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc654c00640 1 -- 172.21.15.27:0/265204220 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc65c078b70 con 0x557708ad7490 2024-09-21T07:13:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 10 client.25783 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 20 client.25783 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708a94080 0x557708f329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 4 client.25783 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:13:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.866+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557708e1b890 con 0x557708a94080 2024-09-21T07:13:42.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.867+0000 7fc66aa00640 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708a94080 0x557708f329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:42.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.868+0000 7fc66aa00640 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708a94080 0x557708f329c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c363705edd422fb9 server_cookie=38f602915a6df049 in_seq=0 out_seq=0 2024-09-21T07:13:42.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.868+0000 7fc654c00640 10 client.25783 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:42.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.869+0000 7fc654c00640 1 -- 172.21.15.27:0/265204220 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557708e1b890 con 0x557708a94080 2024-09-21T07:13:42.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.869+0000 7fc654c00640 10 client.25783 handle_command_reply: tid=0 2024-09-21T07:13:42.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.869+0000 7fc678cc7740 1 client.25783 shutdown 2024-09-21T07:13:42.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.869+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 msgr2=0x557708ebb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.869+0000 7fc678cc7740 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ebb3f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0ccc10 tx=0x7fc65c0cc230 comp rx=0 tx=0).stop 2024-09-21T07:13:42.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 shutdown_connections 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557708a94080 0x557708f329c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557708b00350 0x557708eb63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557708adf670 0x557708ebb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 --2- 172.21.15.27:0/265204220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708ad7490 0x557708ebb3f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc654c00640 0 client.25783 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 >> 172.21.15.27:0/265204220 conn(0x557708af4740 msgr2=0x557708eb1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 shutdown_connections 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 1 -- 172.21.15.27:0/265204220 wait complete. 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.870+0000 7fc678cc7740 20 client.25783 trim_cache size 0 max 16384 2024-09-21T07:13:42.873 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:13:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "d0ace04d-6f23-4748-8446-09ba2828b765", 2024-09-21T07:13:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:13:42.874 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.871+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc6500b6710 msgr2=0x7fc6500b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.871+0000 7fc672000640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc6500b6710 0x7fc6500b8c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc664005fd0 tx=0x7fc664009500 comp rx=0 tx=0).stop 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.871+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc66c105d80 msgr2=0x7fc66c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.871+0000 7fc672000640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc66c105d80 0x7fc66c19c720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02a930 tx=0x7fc65c031a20 comp rx=0 tx=0).stop 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 shutdown_connections 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc6500b6710 0x7fc6500b8c20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c10c010 0x7fc66c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c10b6c0 0x7fc66c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 --2- 172.21.15.27:0/1445104059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc66c105d80 0x7fc66c19c720 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:42.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 >> 172.21.15.27:0/1445104059 conn(0x7fc66c101810 msgr2=0x7fc66c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:42.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 shutdown_connections 2024-09-21T07:13:42.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:42.872+0000 7fc672000640 1 -- 172.21.15.27:0/1445104059 wait complete. 2024-09-21T07:13:43.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:43 smithi027 ceph-mon[30779]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:13:43.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:43 smithi153 ceph-mon[34747]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:13:43.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:43 smithi119 ceph-mon[34519]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:13:43.746 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:13:44.170 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:44.362 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:44 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:13:44.362 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:44 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:13:44.362 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:44 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:13:44.362 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:44 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:13:44.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:44 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:13:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:44 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:13:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:44 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:13:44.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:44 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:13:44.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:44 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:13:44.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:44 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:13:44.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:44 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:13:44.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:44 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:13:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.943+0000 7fe13fa00640 1 -- 172.21.15.27:0/36620428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe138102f40 msgr2=0x7fe13810fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.943+0000 7fe13fa00640 1 --2- 172.21.15.27:0/36620428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe138102f40 0x7fe13810fc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe124009900 tx=0x7fe12402a480 comp rx=0 tx=0).stop 2024-09-21T07:13:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.943+0000 7fe13fa00640 1 -- 172.21.15.27:0/36620428 shutdown_connections 2024-09-21T07:13:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.943+0000 7fe13fa00640 1 --2- 172.21.15.27:0/36620428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe138102f40 0x7fe13810fc60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.943+0000 7fe13fa00640 1 --2- 172.21.15.27:0/36620428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381101a0 0x7fe1381125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.943+0000 7fe13fa00640 1 --2- 172.21.15.27:0/36620428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1381025f0 0x7fe138102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.943+0000 7fe13fa00640 1 -- 172.21.15.27:0/36620428 >> 172.21.15.27:0/36620428 conn(0x7fe138079740 msgr2=0x7fe138079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.944+0000 7fe13fa00640 1 -- 172.21.15.27:0/36620428 shutdown_connections 2024-09-21T07:13:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.944+0000 7fe13fa00640 1 -- 172.21.15.27:0/36620428 wait complete. 2024-09-21T07:13:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 Processor -- start 2024-09-21T07:13:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 -- start start 2024-09-21T07:13:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381025f0 0x7fe1381a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe138102f40 0x7fe1381a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1381101a0 0x7fe1381a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe138115c60 con 0x7fe138102f40 2024-09-21T07:13:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe138115ae0 con 0x7fe1381025f0 2024-09-21T07:13:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.946+0000 7fe13fa00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe138115de0 con 0x7fe1381101a0 2024-09-21T07:13:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.947+0000 7fe13d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381025f0 0x7fe1381a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.947+0000 7fe13d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381025f0 0x7fe1381a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53158/0 (socket says 172.21.15.27:53158) 2024-09-21T07:13:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.947+0000 7fe13d000640 1 -- 172.21.15.27:0/1689316665 learned_addr learned my addr 172.21.15.27:0/1689316665 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.947+0000 7fe137e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe138102f40 0x7fe1381a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.947+0000 7fe13da00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1381101a0 0x7fe1381a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.947+0000 7fe137e00640 1 -- 172.21.15.27:0/1689316665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1381101a0 msgr2=0x7fe1381a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.947+0000 7fe137e00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1381101a0 0x7fe1381a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.948+0000 7fe137e00640 1 -- 172.21.15.27:0/1689316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381025f0 msgr2=0x7fe1381a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.948+0000 7fe137e00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381025f0 0x7fe1381a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.948+0000 7fe137e00640 1 -- 172.21.15.27:0/1689316665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13810eb50 con 0x7fe138102f40 2024-09-21T07:13:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.948+0000 7fe13d000640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381025f0 0x7fe1381a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.948+0000 7fe137e00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe138102f40 0x7fe1381a1cb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe12402a9a0 tx=0x7fe124031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.948+0000 7fe135600640 1 -- 172.21.15.27:0/1689316665 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12403f030 con 0x7fe138102f40 2024-09-21T07:13:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.949+0000 7fe135600640 1 -- 172.21.15.27:0/1689316665 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe124004490 con 0x7fe138102f40 2024-09-21T07:13:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.949+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13810ede0 con 0x7fe138102f40 2024-09-21T07:13:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.949+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13810f310 con 0x7fe138102f40 2024-09-21T07:13:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.949+0000 7fe135600640 1 -- 172.21.15.27:0/1689316665 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124035070 con 0x7fe138102f40 2024-09-21T07:13:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.953+0000 7fe135600640 1 -- 172.21.15.27:0/1689316665 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7fe124042030 con 0x7fe138102f40 2024-09-21T07:13:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 Processor -- start 2024-09-21T07:13:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 -- start start 2024-09-21T07:13:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5654cc51df70 con 0x5654cc1d71b0 2024-09-21T07:13:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5654cc50d3a0 con 0x5654cc1df390 2024-09-21T07:13:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.954+0000 7fe14669d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5654cc5111b0 con 0x5654cc200070 2024-09-21T07:13:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.955+0000 7fe13da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.955+0000 7fe13da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5a6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57136/0 (socket says 172.21.15.27:57136) 2024-09-21T07:13:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.955+0000 7fe13da00640 1 -- 172.21.15.27:0/2883163892 learned_addr learned my addr 172.21.15.27:0/2883163892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.955+0000 7fe13d000640 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.955+0000 7fe137e00640 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.956+0000 7fe135600640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1200b6810 0x7fe1200b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.956+0000 7fe13d000640 1 -- 172.21.15.27:0/2883163892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 msgr2=0x5654cc5a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.956+0000 7fe13d000640 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.956+0000 7fe13d000640 1 -- 172.21.15.27:0/2883163892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 msgr2=0x5654cc5a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.956+0000 7fe13d000640 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.956+0000 7fe135600640 1 -- 172.21.15.27:0/1689316665 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe124107c30 con 0x7fe138102f40 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.956+0000 7fe13d000640 1 -- 172.21.15.27:0/2883163892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cc4d7c90 con 0x5654cc1d71b0 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.957+0000 7fe13d000640 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5a72a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe1381a31c0 tx=0x7fe12c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a5b98663db448a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.957+0000 7fe13da00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1200b6810 0x7fe1200b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.957+0000 7fe12a000640 1 -- 172.21.15.27:0/2883163892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c004030 con 0x5654cc1d71b0 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.958+0000 7fe12a000640 1 -- 172.21.15.27:0/2883163892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe12c005020 con 0x5654cc1d71b0 2024-09-21T07:13:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.958+0000 7fe12a000640 1 -- 172.21.15.27:0/2883163892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c004030 con 0x5654cc1d71b0 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 -- 172.21.15.27:0/2883163892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 msgr2=0x5654cc5a72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe13da00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1200b6810 0x7fe1200b8d20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe130003b10 tx=0x7fe130004ab0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5a72a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe1381a31c0 tx=0x7fe12c0119a0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 -- 172.21.15.27:0/2883163892 shutdown_connections 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5a72a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 --2- 172.21.15.27:0/2883163892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 -- 172.21.15.27:0/2883163892 >> 172.21.15.27:0/2883163892 conn(0x5654cc1f4460 msgr2=0x5654cc5a4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:44.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 -- 172.21.15.27:0/2883163892 shutdown_connections 2024-09-21T07:13:44.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.959+0000 7fe14669d740 1 -- 172.21.15.27:0/2883163892 wait complete. 2024-09-21T07:13:44.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.960+0000 7fe14669d740 1 -- start start 2024-09-21T07:13:44.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.960+0000 7fe14669d740 1 Processor -- start 2024-09-21T07:13:44.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe14669d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe14669d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe14669d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc200070 0x5654cc5be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe14669d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5654cc36e700 con 0x5654cc200070 2024-09-21T07:13:44.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe14669d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5654cc4a8540 con 0x5654cc1df390 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe14669d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5654cc5b5540 con 0x5654cc1d71b0 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe14669d740 10 client.0 _ll_register_callbacks cb 0x5654cc4b8430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe13d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe13da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc200070 0x5654cc5be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe13d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57150/0 (socket says 172.21.15.27:57150) 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe13d000640 1 -- 172.21.15.27:0/2098674511 learned_addr learned my addr 172.21.15.27:0/2098674511 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe13da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc200070 0x5654cc5be8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59044/0 (socket says 172.21.15.27:59044) 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.961+0000 7fe137e00640 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe13d000640 1 -- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 msgr2=0x5654cc5b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe13d000640 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe13d000640 1 -- 172.21.15.27:0/2098674511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc200070 msgr2=0x5654cc5be8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe13d000640 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc200070 0x5654cc5be8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe13d000640 1 -- 172.21.15.27:0/2098674511 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cc26b650 con 0x5654cc1d71b0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe13d000640 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5b24e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe12c01b040 tx=0x7fe12c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fbb0fd4456b34f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe11ea00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.962+0000 7fe11ea00640 1 -- 172.21.15.27:0/2098674511 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c028020 con 0x5654cc1d71b0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.963+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654cc4aab50 con 0x5654cc1d71b0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.963+0000 7fe11ea00640 1 -- 172.21.15.27:0/2098674511 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe12c016040 con 0x5654cc1d71b0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.963+0000 7fe11ea00640 1 -- 172.21.15.27:0/2098674511 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe12c033050 con 0x5654cc1d71b0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.963+0000 7fe14669d740 10 client.25795 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.963+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654cc4c5910 con 0x5654cc1d71b0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.963+0000 7fe11ea00640 1 -- 172.21.15.27:0/2098674511 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5654cc4aab50 con 0x5654cc1d71b0 2024-09-21T07:13:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe11ea00640 1 -- 172.21.15.27:0/2098674511 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe12c00fa90 con 0x5654cc1d71b0 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe14669d740 10 client.25795 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe14669d740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe14669d740 20 client.25795 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe14669d740 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc1be9f0 0x5654cc5c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe14669d740 4 client.25795 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654cc51fec0 con 0x5654cc1be9f0 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.964+0000 7fe137e00640 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc1be9f0 0x5654cc5c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.965+0000 7fe11ea00640 10 client.25795 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.965+0000 7fe137e00640 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc1be9f0 0x5654cc5c4590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0aacd2f1874297 server_cookie=c4065ab96a463e09 in_seq=0 out_seq=0 2024-09-21T07:13:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe11ea00640 1 -- 172.21.15.27:0/2098674511 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654cc51fec0 con 0x5654cc1be9f0 2024-09-21T07:13:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe11ea00640 10 client.25795 handle_command_reply: tid=0 2024-09-21T07:13:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 client.25795 shutdown 2024-09-21T07:13:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 msgr2=0x5654cc5b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5b24e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe12c01b040 tx=0x7fe12c003820 comp rx=0 tx=0).stop 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 shutdown_connections 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc1be9f0 0x5654cc5c4590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc200070 0x5654cc5be8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 --2- 172.21.15.27:0/2098674511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5b24e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 >> 172.21.15.27:0/2098674511 conn(0x5654cc1f4460 msgr2=0x5654cc5a9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe11ea00640 0 client.25795 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:44.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.966+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 shutdown_connections 2024-09-21T07:13:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.967+0000 7fe14669d740 1 -- 172.21.15.27:0/2098674511 wait complete. 2024-09-21T07:13:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.967+0000 7fe14669d740 20 client.25795 trim_cache size 0 max 16384 2024-09-21T07:13:44.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.985+0000 7fe14669d740 1 Processor -- start 2024-09-21T07:13:44.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.986+0000 7fe14669d740 1 -- start start 2024-09-21T07:13:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.986+0000 7fe14669d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.986+0000 7fe14669d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1df390 0x5654cc5b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.986+0000 7fe14669d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc200070 0x5654cc5aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.986+0000 7fe14669d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe1180126d0 con 0x5654cc1df390 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.986+0000 7fe14669d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800ffa0 con 0x5654cc200070 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.986+0000 7fe14669d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800ddf0 con 0x5654cc1d71b0 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe137e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1df390 0x5654cc5b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe137e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1df390 0x5654cc5b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59052/0 (socket says 172.21.15.27:59052) 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe137e00640 1 -- 172.21.15.27:0/3701047383 learned_addr learned my addr 172.21.15.27:0/3701047383 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe13d000640 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe13da00640 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc200070 0x5654cc5aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe13d000640 1 -- 172.21.15.27:0/3701047383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc200070 msgr2=0x5654cc5aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe13d000640 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc200070 0x5654cc5aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe13d000640 1 -- 172.21.15.27:0/3701047383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1df390 msgr2=0x5654cc5b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe13d000640 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1df390 0x5654cc5b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe13d000640 1 -- 172.21.15.27:0/3701047383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cc5b37a0 con 0x5654cc1d71b0 2024-09-21T07:13:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.987+0000 7fe137e00640 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1df390 0x5654cc5b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.988+0000 7fe13d000640 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5adc90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe12c023450 tx=0x7fe12c009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a85ca28a129866 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.988+0000 7fe129600640 1 -- 172.21.15.27:0/3701047383 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c028070 con 0x5654cc1d71b0 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.988+0000 7fe129600640 1 -- 172.21.15.27:0/3701047383 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe12c023e00 con 0x5654cc1d71b0 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.988+0000 7fe129600640 1 -- 172.21.15.27:0/3701047383 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0054c0 con 0x5654cc1d71b0 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 -- 172.21.15.27:0/3701047383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 msgr2=0x5654cc5adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5adc90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe12c023450 tx=0x7fe12c009590 comp rx=0 tx=0).stop 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 -- 172.21.15.27:0/3701047383 shutdown_connections 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc200070 0x5654cc5aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1df390 0x5654cc5b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 --2- 172.21.15.27:0/3701047383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc1d71b0 0x5654cc5adc90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 -- 172.21.15.27:0/3701047383 >> 172.21.15.27:0/3701047383 conn(0x5654cc1f4460 msgr2=0x5654cc5a9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 -- 172.21.15.27:0/3701047383 shutdown_connections 2024-09-21T07:13:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.989+0000 7fe14669d740 1 -- 172.21.15.27:0/3701047383 wait complete. 2024-09-21T07:13:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.990+0000 7fe14669d740 1 -- start start 2024-09-21T07:13:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.990+0000 7fe14669d740 1 Processor -- start 2024-09-21T07:13:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe14669d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe14669d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe14669d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe14669d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800e460 con 0x5654cc1d71b0 2024-09-21T07:13:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe14669d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800f8d0 con 0x5654cc1df390 2024-09-21T07:13:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe14669d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800eb30 con 0x5654cc200070 2024-09-21T07:13:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe14669d740 10 client.0 _ll_register_callbacks cb 0x7fe12c009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.991+0000 7fe13d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe13d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5b6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59058/0 (socket says 172.21.15.27:59058) 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe13d000640 1 -- 172.21.15.27:0/2054539939 learned_addr learned my addr 172.21.15.27:0/2054539939 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe137e00640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe13da00640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe137e00640 1 -- 172.21.15.27:0/2054539939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 msgr2=0x5654cc5b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe137e00640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe137e00640 1 -- 172.21.15.27:0/2054539939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 msgr2=0x5654cc5b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe137e00640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe137e00640 1 -- 172.21.15.27:0/2054539939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cc62df60 con 0x5654cc1df390 2024-09-21T07:13:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.992+0000 7fe13d000640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:13:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.993+0000 7fe137e00640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b6db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1240cd310 tx=0x7fe12402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec5cfb2ee76d6d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.993+0000 7fe11ea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.993+0000 7fe11ea00640 1 -- 172.21.15.27:0/2054539939 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124049040 con 0x5654cc1df390 2024-09-21T07:13:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.993+0000 7fe11ea00640 1 -- 172.21.15.27:0/2054539939 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe1240c98e0 con 0x5654cc1df390 2024-09-21T07:13:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.993+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe12c023e00 con 0x5654cc1df390 2024-09-21T07:13:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.994+0000 7fe11ea00640 1 -- 172.21.15.27:0/2054539939 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1240cd120 con 0x5654cc1df390 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.994+0000 7fe14669d740 10 client.25811 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.994+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654cc62f1f0 con 0x5654cc1df390 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.994+0000 7fe11ea00640 1 -- 172.21.15.27:0/2054539939 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe12c023e00 con 0x5654cc1df390 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe11ea00640 1 -- 172.21.15.27:0/2054539939 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe1240c8fe0 con 0x5654cc1df390 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe14669d740 10 client.25811 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe14669d740 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe14669d740 20 client.25811 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe14669d740 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc193da0 0x5654cc631a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe14669d740 4 client.25811 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:13:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654cc51b970 con 0x5654cc193da0 2024-09-21T07:13:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.995+0000 7fe13d000640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc193da0 0x5654cc631a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.996+0000 7fe13d000640 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc193da0 0x5654cc631a30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1faa1a3f3431b1 server_cookie=604aa0e505e95c36 in_seq=0 out_seq=0 2024-09-21T07:13:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.996+0000 7fe11ea00640 10 client.25811 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.996+0000 7fe11ea00640 1 -- 172.21.15.27:0/2054539939 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x5654cc51b970 con 0x5654cc193da0 2024-09-21T07:13:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.996+0000 7fe11ea00640 10 client.25811 handle_command_reply: tid=0 2024-09-21T07:13:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.996+0000 7fe14669d740 1 client.25811 shutdown 2024-09-21T07:13:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.996+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 msgr2=0x5654cc5b6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.996+0000 7fe14669d740 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b6db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1240cd310 tx=0x7fe12402a480 comp rx=0 tx=0).stop 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 shutdown_connections 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5654cc193da0 0x5654cc631a30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654cc200070 0x5654cc5b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe11ea00640 0 client.25811 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654cc1df390 0x5654cc5b6db0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 --2- 172.21.15.27:0/2054539939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5654cc1d71b0 0x5654cc5b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 >> 172.21.15.27:0/2054539939 conn(0x5654cc1f4460 msgr2=0x5654cc5b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 shutdown_connections 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.997+0000 7fe14669d740 1 -- 172.21.15.27:0/2054539939 wait complete. 2024-09-21T07:13:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.998+0000 7fe14669d740 20 client.25811 trim_cache size 0 max 16384 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "d0ace04d-6f23-4748-8446-09ba2828b765", 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:45.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:13:45.002 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1200b6810 msgr2=0x7fe1200b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:45.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1200b6810 0x7fe1200b8d20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe130003b10 tx=0x7fe130004ab0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe138102f40 msgr2=0x7fe1381a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe138102f40 0x7fe1381a1cb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe12402a9a0 tx=0x7fe124031a20 comp rx=0 tx=0).stop 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 shutdown_connections 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1381101a0 0x7fe1381a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1200b6810 0x7fe1200b8d20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe138102f40 0x7fe1381a1cb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 --2- 172.21.15.27:0/1689316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1381025f0 0x7fe1381a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 >> 172.21.15.27:0/1689316665 conn(0x7fe138079740 msgr2=0x7fe138105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 shutdown_connections 2024-09-21T07:13:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:44.999+0000 7fe13fa00640 1 -- 172.21.15.27:0/1689316665 wait complete. 2024-09-21T07:13:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:45 smithi027 ceph-mon[30779]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T07:13:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:45 smithi153 ceph-mon[34747]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T07:13:45.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:45 smithi119 ceph-mon[34519]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T07:13:45.865 INFO:tasks.cephfs.filesystem:scrub status for tag:d0ace04d-6f23-4748-8446-09ba2828b765 - {'path': '/', 'tag': 'd0ace04d-6f23-4748-8446-09ba2828b765', 'options': 'recursive,force'} 2024-09-21T07:13:46.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:46 smithi153 ceph-mon[34747]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T07:13:46.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:46 smithi119 ceph-mon[34519]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T07:13:46.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:46 smithi027 ceph-mon[30779]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T07:13:48.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:48 smithi119 ceph-mon[34519]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T07:13:48.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:48 smithi027 ceph-mon[30779]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T07:13:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:48 smithi153 ceph-mon[34747]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T07:13:50.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:13:50.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:50 smithi119 ceph-mon[34519]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T07:13:50.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:50 smithi027 ceph-mon[30779]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T07:13:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:50 smithi153 ceph-mon[34747]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T07:13:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 -- 172.21.15.27:0/2949403703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957010bd90 msgr2=0x7f95701125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 --2- 172.21.15.27:0/2949403703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957010bd90 0x7f95701125c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9568009ca0 tx=0x7f956802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 -- 172.21.15.27:0/2949403703 shutdown_connections 2024-09-21T07:13:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 --2- 172.21.15.27:0/2949403703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957010bd90 0x7f95701125c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 --2- 172.21.15.27:0/2949403703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f957010b440 0x7f957010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 --2- 172.21.15.27:0/2949403703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9570105b10 0x7f9570105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 -- 172.21.15.27:0/2949403703 >> 172.21.15.27:0/2949403703 conn(0x7f95701014f0 msgr2=0x7f9570103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 -- 172.21.15.27:0/2949403703 shutdown_connections 2024-09-21T07:13:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.279+0000 7f9575400640 1 -- 172.21.15.27:0/2949403703 wait complete. 2024-09-21T07:13:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.281+0000 7f9575400640 1 Processor -- start 2024-09-21T07:13:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.281+0000 7f9575400640 1 -- start start 2024-09-21T07:13:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f9575400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9570105b10 0x7f957019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f9575400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f957010b440 0x7f957019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f9575400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f957010bd90 0x7f95701a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f9575400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9570115aa0 con 0x7f957010b440 2024-09-21T07:13:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f9575400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9570115920 con 0x7f9570105b10 2024-09-21T07:13:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f9575400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9570115c20 con 0x7f957010bd90 2024-09-21T07:13:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9570105b10 0x7f957019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f956f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f957010b440 0x7f957019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9570105b10 0x7f957019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37926/0 (socket says 172.21.15.27:37926) 2024-09-21T07:13:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.282+0000 7f956fe00640 1 -- 172.21.15.27:0/1669300137 learned_addr learned my addr 172.21.15.27:0/1669300137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f9574a00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f957010bd90 0x7f95701a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f956fe00640 1 -- 172.21.15.27:0/1669300137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f957010bd90 msgr2=0x7f95701a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f956fe00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f957010bd90 0x7f95701a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f956fe00640 1 -- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f957010b440 msgr2=0x7f957019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f956fe00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f957010b440 0x7f957019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f956fe00640 1 -- 172.21.15.27:0/1669300137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95701a2470 con 0x7f9570105b10 2024-09-21T07:13:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f9574a00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f957010bd90 0x7f95701a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.283+0000 7f956fe00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9570105b10 0x7f957019c540 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f956000e9f0 tx=0x7f956000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.284+0000 7f956cc00640 1 -- 172.21.15.27:0/1669300137 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956000cd80 con 0x7f9570105b10 2024-09-21T07:13:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.284+0000 7f956cc00640 1 -- 172.21.15.27:0/1669300137 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600054d0 con 0x7f9570105b10 2024-09-21T07:13:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.284+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95701a2760 con 0x7f9570105b10 2024-09-21T07:13:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.284+0000 7f956cc00640 1 -- 172.21.15.27:0/1669300137 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560010430 con 0x7f9570105b10 2024-09-21T07:13:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.284+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95701a8ef0 con 0x7f9570105b10 2024-09-21T07:13:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.288+0000 7f956cc00640 1 -- 172.21.15.27:0/1669300137 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f9560005020 con 0x7f9570105b10 2024-09-21T07:13:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.288+0000 7f957da9c740 1 Processor -- start 2024-09-21T07:13:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.288+0000 7f957da9c740 1 -- start start 2024-09-21T07:13:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f957da9c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f957da9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f957da9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f957da9c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56545d4266b0 con 0x56545d10c1b0 2024-09-21T07:13:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f957da9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56545d415ae0 con 0x56545d0e32f0 2024-09-21T07:13:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f957da9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56545d4198f0 con 0x56545d0eb4d0 2024-09-21T07:13:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f9574a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.289+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37928/0 (socket says 172.21.15.27:37928) 2024-09-21T07:13:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 -- 172.21.15.27:0/3109134097 learned_addr learned my addr 172.21.15.27:0/3109134097 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956f400640 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 -- 172.21.15.27:0/3109134097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 msgr2=0x56545d4b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 -- 172.21.15.27:0/3109134097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 msgr2=0x56545d4b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 -- 172.21.15.27:0/3109134097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56545d3e2780 con 0x56545d0e32f0 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.290+0000 7f956fe00640 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4b1350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f95600e52e0 tx=0x7f95600f01a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f229fd1b7cee31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.291+0000 7f956cc00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95540b6790 0x7f95540b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.291+0000 7f9565200640 1 -- 172.21.15.27:0/3109134097 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600fb070 con 0x56545d0e32f0 2024-09-21T07:13:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.291+0000 7f9565200640 1 -- 172.21.15.27:0/3109134097 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9560005310 con 0x56545d0e32f0 2024-09-21T07:13:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.291+0000 7f9565200640 1 -- 172.21.15.27:0/3109134097 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600ad7c0 con 0x56545d0e32f0 2024-09-21T07:13:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.291+0000 7f956cc00640 1 -- 172.21.15.27:0/1669300137 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f95600e80c0 con 0x7f9570105b10 2024-09-21T07:13:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.291+0000 7f9574a00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95540b6790 0x7f95540b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 -- 172.21.15.27:0/3109134097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 msgr2=0x56545d4b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4b1350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f95600e52e0 tx=0x7f95600f01a0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f9574a00640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95540b6790 0x7f95540b8ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9568006010 tx=0x7f956802f3f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 -- 172.21.15.27:0/3109134097 shutdown_connections 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4b1350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 --2- 172.21.15.27:0/3109134097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 -- 172.21.15.27:0/3109134097 >> 172.21.15.27:0/3109134097 conn(0x56545d100300 msgr2=0x56545d4aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.292+0000 7f957da9c740 1 -- 172.21.15.27:0/3109134097 shutdown_connections 2024-09-21T07:13:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.293+0000 7f957da9c740 1 -- 172.21.15.27:0/3109134097 wait complete. 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.293+0000 7f957da9c740 1 -- start start 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.293+0000 7f957da9c740 1 Processor -- start 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.294+0000 7f957da9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0e32f0 0x56545d4bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.294+0000 7f957da9c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d0eb4d0 0x56545d4bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.294+0000 7f957da9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d10c1b0 0x56545d4bce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.294+0000 7f957da9c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56545d34dab0 con 0x56545d0eb4d0 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.294+0000 7f957da9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56545d3bc570 con 0x56545d10c1b0 2024-09-21T07:13:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.294+0000 7f957da9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56545d4bf590 con 0x56545d0e32f0 2024-09-21T07:13:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.294+0000 7f957da9c740 10 client.0 _ll_register_callbacks cb 0x56545d3cb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f9574a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d10c1b0 0x56545d4bce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0e32f0 0x56545d4bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0e32f0 0x56545d4bc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55712/0 (socket says 172.21.15.27:55712) 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 -- 172.21.15.27:0/46307354 learned_addr learned my addr 172.21.15.27:0/46307354 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956f400640 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d0eb4d0 0x56545d4bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 -- 172.21.15.27:0/46307354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d10c1b0 msgr2=0x56545d4bce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d10c1b0 0x56545d4bce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 -- 172.21.15.27:0/46307354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d0eb4d0 msgr2=0x56545d4bc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d0eb4d0 0x56545d4bc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 -- 172.21.15.27:0/46307354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56545d177790 con 0x56545d0e32f0 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f956fe00640 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0e32f0 0x56545d4bc400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9560004a60 tx=0x7f9560009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725ba3dcf359326e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.295+0000 7f9559600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.296+0000 7f9559600640 1 -- 172.21.15.27:0/46307354 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600ebbb0 con 0x56545d0e32f0 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.296+0000 7f9559600640 1 -- 172.21.15.27:0/46307354 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f95600e5650 con 0x56545d0e32f0 2024-09-21T07:13:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.296+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56545d428600 con 0x56545d0e32f0 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.296+0000 7f9559600640 1 -- 172.21.15.27:0/46307354 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95600ad160 con 0x56545d0e32f0 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.296+0000 7f957da9c740 10 client.25813 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.296+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56545d1f0620 con 0x56545d0e32f0 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.296+0000 7f9559600640 1 -- 172.21.15.27:0/46307354 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56545d428600 con 0x56545d0e32f0 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f9559600640 1 -- 172.21.15.27:0/46307354 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f95600b2020 con 0x56545d0e32f0 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f957da9c740 10 client.25813 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f957da9c740 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f957da9c740 20 client.25813 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f957da9c740 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0cad80 0x56545d4ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f957da9c740 4 client.25813 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56545d4240b0 con 0x56545d0cad80 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.297+0000 7f956f400640 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0cad80 0x56545d4ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.298+0000 7f956f400640 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0cad80 0x56545d4ce3a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6589cf715efe7a0f server_cookie=619c93a056cf389e in_seq=0 out_seq=0 2024-09-21T07:13:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.298+0000 7f9559600640 10 client.25813 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.299+0000 7f9559600640 1 -- 172.21.15.27:0/46307354 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56545d4240b0 con 0x56545d0cad80 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.299+0000 7f9559600640 10 client.25813 handle_command_reply: tid=0 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.299+0000 7f957da9c740 1 client.25813 shutdown 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.299+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0e32f0 msgr2=0x56545d4bc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.299+0000 7f957da9c740 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0e32f0 0x56545d4bc400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9560004a60 tx=0x7f9560009720 comp rx=0 tx=0).stop 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 shutdown_connections 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0cad80 0x56545d4ce3a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d10c1b0 0x56545d4bce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d0eb4d0 0x56545d4bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 --2- 172.21.15.27:0/46307354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0e32f0 0x56545d4bc400 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f9559600640 0 client.25813 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 >> 172.21.15.27:0/46307354 conn(0x56545d100300 msgr2=0x56545d4b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 shutdown_connections 2024-09-21T07:13:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 1 -- 172.21.15.27:0/46307354 wait complete. 2024-09-21T07:13:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.300+0000 7f957da9c740 20 client.25813 trim_cache size 0 max 16384 2024-09-21T07:13:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.329+0000 7f957da9c740 1 Processor -- start 2024-09-21T07:13:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.329+0000 7f957da9c740 1 -- start start 2024-09-21T07:13:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f957da9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f957da9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f957da9c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f957da9c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b56b0 con 0x56545d10c1b0 2024-09-21T07:13:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f957da9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b2f80 con 0x56545d0e32f0 2024-09-21T07:13:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f957da9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b0dd0 con 0x56545d0eb4d0 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f9574a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f9574a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59166/0 (socket says 172.21.15.27:59166) 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37954/0 (socket says 172.21.15.27:37954) 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f9574a00640 1 -- 172.21.15.27:0/1296448859 learned_addr learned my addr 172.21.15.27:0/1296448859 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.330+0000 7f956f400640 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.331+0000 7f956fe00640 1 -- 172.21.15.27:0/1296448859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 msgr2=0x56545d4b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.331+0000 7f956fe00640 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.331+0000 7f956fe00640 1 -- 172.21.15.27:0/1296448859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 msgr2=0x56545d4b95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.331+0000 7f956fe00640 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.331+0000 7f956fe00640 1 -- 172.21.15.27:0/1296448859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56545d4afa90 con 0x56545d0e32f0 2024-09-21T07:13:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.331+0000 7f956fe00640 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4add20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f95600ad980 tx=0x7f95600a9900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=436b8535023fec13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.332+0000 7f955be00640 1 -- 172.21.15.27:0/1296448859 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600ad340 con 0x56545d0e32f0 2024-09-21T07:13:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.332+0000 7f955be00640 1 -- 172.21.15.27:0/1296448859 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600f06a0 con 0x56545d0e32f0 2024-09-21T07:13:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.332+0000 7f955be00640 1 -- 172.21.15.27:0/1296448859 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600eb6a0 con 0x56545d0e32f0 2024-09-21T07:13:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.333+0000 7f957da9c740 1 -- 172.21.15.27:0/1296448859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 msgr2=0x56545d4add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.333+0000 7f957da9c740 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4add20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f95600ad980 tx=0x7f95600a9900 comp rx=0 tx=0).stop 2024-09-21T07:13:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.333+0000 7f957da9c740 1 -- 172.21.15.27:0/1296448859 shutdown_connections 2024-09-21T07:13:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.333+0000 7f957da9c740 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.333+0000 7f957da9c740 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d4b95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.333+0000 7f957da9c740 1 --2- 172.21.15.27:0/1296448859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4add20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.333+0000 7f957da9c740 1 -- 172.21.15.27:0/1296448859 >> 172.21.15.27:0/1296448859 conn(0x56545d100300 msgr2=0x56545d4b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.334+0000 7f957da9c740 1 -- 172.21.15.27:0/1296448859 shutdown_connections 2024-09-21T07:13:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.334+0000 7f957da9c740 1 -- 172.21.15.27:0/1296448859 wait complete. 2024-09-21T07:13:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.334+0000 7f957da9c740 1 -- start start 2024-09-21T07:13:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.334+0000 7f957da9c740 1 Processor -- start 2024-09-21T07:13:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f957da9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f957da9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f957da9c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d52dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f957da9c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b1440 con 0x56545d10c1b0 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f957da9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b28b0 con 0x56545d0e32f0 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f957da9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b1b10 con 0x56545d0eb4d0 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f957da9c740 10 client.0 _ll_register_callbacks cb 0x7f95600f9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.335+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37956/0 (socket says 172.21.15.27:37956) 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 -- 172.21.15.27:0/554104428 learned_addr learned my addr 172.21.15.27:0/554104428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956f400640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f9574a00640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d52dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 -- 172.21.15.27:0/554104428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 msgr2=0x56545d4d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 -- 172.21.15.27:0/554104428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 msgr2=0x56545d52dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d52dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 -- 172.21.15.27:0/554104428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56545d52e3f0 con 0x56545d0e32f0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f9574a00640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d52dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f956fe00640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4d1220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9560009640 tx=0x7f95600b2290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7e0a5fa5bf96a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.336+0000 7f9559600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.337+0000 7f9559600640 1 -- 172.21.15.27:0/554104428 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600ad340 con 0x56545d0e32f0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.337+0000 7f9559600640 1 -- 172.21.15.27:0/554104428 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f95600ebb40 con 0x56545d0e32f0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.337+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95600f06a0 con 0x56545d0e32f0 2024-09-21T07:13:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.337+0000 7f9559600640 1 -- 172.21.15.27:0/554104428 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95600aa040 con 0x56545d0e32f0 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.337+0000 7f957da9c740 10 client.25829 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.337+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56545d538a00 con 0x56545d0e32f0 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.337+0000 7f9559600640 1 -- 172.21.15.27:0/554104428 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600f06a0 con 0x56545d0e32f0 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.338+0000 7f9559600640 1 -- 172.21.15.27:0/554104428 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f95600e5650 con 0x56545d0e32f0 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.338+0000 7f957da9c740 10 client.25829 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.338+0000 7f957da9c740 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.338+0000 7f957da9c740 20 client.25829 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.338+0000 7f957da9c740 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0a01c0 0x56545d53afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.338+0000 7f957da9c740 4 client.25829 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:13:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.338+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56545d4214e0 con 0x56545d0a01c0 2024-09-21T07:13:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.339+0000 7f956f400640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0a01c0 0x56545d53afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.339+0000 7f956f400640 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0a01c0 0x56545d53afd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35d6eb328ab6482 server_cookie=ebc4c80c5943fd17 in_seq=0 out_seq=0 2024-09-21T07:13:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.339+0000 7f9559600640 10 client.25829 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.347+0000 7f9559600640 1 -- 172.21.15.27:0/554104428 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56545d4214e0 con 0x56545d0a01c0 2024-09-21T07:13:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.347+0000 7f9559600640 10 client.25829 handle_command_reply: tid=0 2024-09-21T07:13:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.347+0000 7f957da9c740 1 client.25829 shutdown 2024-09-21T07:13:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 msgr2=0x56545d4d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4d1220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9560009640 tx=0x7f95600b2290 comp rx=0 tx=0).stop 2024-09-21T07:13:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 shutdown_connections 2024-09-21T07:13:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56545d0a01c0 0x56545d53afd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56545d0eb4d0 0x56545d4d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56545d10c1b0 0x56545d52dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 --2- 172.21.15.27:0/554104428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56545d0e32f0 0x56545d4d1220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 >> 172.21.15.27:0/554104428 conn(0x56545d100300 msgr2=0x56545d4adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f9559600640 0 client.25829 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 shutdown_connections 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 1 -- 172.21.15.27:0/554104428 wait complete. 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.348+0000 7f957da9c740 20 client.25829 trim_cache size 0 max 16384 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:13:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:13:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95540b6790 msgr2=0x7f95540b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95540b6790 0x7f95540b8ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9568006010 tx=0x7f956802f3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9570105b10 msgr2=0x7f957019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9570105b10 0x7f957019c540 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f956000e9f0 tx=0x7f956000edf0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 shutdown_connections 2024-09-21T07:13:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f957010bd90 0x7f95701a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f95540b6790 0x7f95540b8ca0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f957010b440 0x7f957019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 --2- 172.21.15.27:0/1669300137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9570105b10 0x7f957019c540 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.349+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 >> 172.21.15.27:0/1669300137 conn(0x7f95701014f0 msgr2=0x7f957010a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.350+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 shutdown_connections 2024-09-21T07:13:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:13:51.350+0000 7f9575400640 1 -- 172.21.15.27:0/1669300137 wait complete. 2024-09-21T07:13:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 -- 172.21.15.27:0/1946389034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a340757b0 msgr2=0x7f9a34075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 --2- 172.21.15.27:0/1946389034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a340757b0 0x7f9a34075bc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009900 tx=0x7f9a2802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 -- 172.21.15.27:0/1946389034 shutdown_connections 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 --2- 172.21.15.27:0/1946389034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a3410ff10 0x7f9a34112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 --2- 172.21.15.27:0/1946389034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a34076100 0x7f9a3410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 --2- 172.21.15.27:0/1946389034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a340757b0 0x7f9a34075bc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 -- 172.21.15.27:0/1946389034 >> 172.21.15.27:0/1946389034 conn(0x7f9a340ff440 msgr2=0x7f9a34101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 -- 172.21.15.27:0/1946389034 shutdown_connections 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.922+0000 7f9a3b000640 1 -- 172.21.15.27:0/1946389034 wait complete. 2024-09-21T07:13:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.923+0000 7f9a3b000640 1 Processor -- start 2024-09-21T07:13:51.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.924+0000 7f9a3b000640 1 -- start start 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.924+0000 7f9a3b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a340757b0 0x7f9a3419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.924+0000 7f9a3b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a34076100 0x7f9a3419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.924+0000 7f9a3b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a3410ff10 0x7f9a341a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.924+0000 7f9a3b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34115ac0 con 0x7f9a340757b0 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.924+0000 7f9a3b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34115940 con 0x7f9a3410ff10 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.924+0000 7f9a3b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34115c40 con 0x7f9a34076100 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a3410ff10 0x7f9a341a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a3410ff10 0x7f9a341a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37978/0 (socket says 172.21.15.27:37978) 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 -- 172.21.15.27:0/3428234874 learned_addr learned my addr 172.21.15.27:0/3428234874 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a39200640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a34076100 0x7f9a3419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 -- 172.21.15.27:0/3428234874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a34076100 msgr2=0x7f9a3419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a34076100 0x7f9a3419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 -- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a340757b0 msgr2=0x7f9a3419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a340757b0 0x7f9a3419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.925+0000 7f9a3a600640 1 -- 172.21.15.27:0/3428234874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a341a2420 con 0x7f9a3410ff10 2024-09-21T07:13:51.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.926+0000 7f9a3a600640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a3410ff10 0x7f9a341a1cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00eb10 tx=0x7f9a2c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.926+0000 7f9a32000640 1 -- 172.21.15.27:0/3428234874 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c004b20 con 0x7f9a3410ff10 2024-09-21T07:13:51.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.926+0000 7f9a32000640 1 -- 172.21.15.27:0/3428234874 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a2c004cc0 con 0x7f9a3410ff10 2024-09-21T07:13:51.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.926+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a341a2710 con 0x7f9a3410ff10 2024-09-21T07:13:51.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.926+0000 7f9a32000640 1 -- 172.21.15.27:0/3428234874 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c010670 con 0x7f9a3410ff10 2024-09-21T07:13:51.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.927+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a341a8e70 con 0x7f9a3410ff10 2024-09-21T07:13:51.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.928+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a34075bc0 con 0x7f9a3410ff10 2024-09-21T07:13:51.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.930+0000 7f9a32000640 1 -- 172.21.15.27:0/3428234874 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f9a2c005020 con 0x7f9a3410ff10 2024-09-21T07:13:51.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.932+0000 7f9a32000640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a1c0b6710 0x7f9a1c0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:51.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.932+0000 7f9a32000640 1 -- 172.21.15.27:0/3428234874 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9a2c0e83a0 con 0x7f9a3410ff10 2024-09-21T07:13:51.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.932+0000 7f9a39c00640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a1c0b6710 0x7f9a1c0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:51.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.933+0000 7f9a39c00640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a1c0b6710 0x7f9a1c0b8c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9a2802a910 tx=0x7f9a28031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:51.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:51.936+0000 7f9a32000640 1 -- 172.21.15.27:0/3428234874 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a2c0ae3d0 con 0x7f9a3410ff10 2024-09-21T07:13:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.178+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a3419d5e0 con 0x7f9a3410ff10 2024-09-21T07:13:52.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.179+0000 7f9a32000640 1 -- 172.21.15.27:0/3428234874 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f9a2c0b27b0 con 0x7f9a3410ff10 2024-09-21T07:13:52.181 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:13:52.181 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:13:52.181 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:13:52.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a1c0b6710 msgr2=0x7f9a1c0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a1c0b6710 0x7f9a1c0b8c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9a2802a910 tx=0x7f9a28031400 comp rx=0 tx=0).stop 2024-09-21T07:13:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a3410ff10 msgr2=0x7f9a341a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a3410ff10 0x7f9a341a1cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00eb10 tx=0x7f9a2c00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:13:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 shutdown_connections 2024-09-21T07:13:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a34076100 0x7f9a3419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a1c0b6710 0x7f9a1c0b8c20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:52.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a3410ff10 0x7f9a341a1cf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:52.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 --2- 172.21.15.27:0/3428234874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a340757b0 0x7f9a3419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:52.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 >> 172.21.15.27:0/3428234874 conn(0x7f9a340ff440 msgr2=0x7f9a34109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:52.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.182+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 shutdown_connections 2024-09-21T07:13:52.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:52.183+0000 7f9a3b000640 1 -- 172.21.15.27:0/3428234874 wait complete. 2024-09-21T07:13:52.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:52 smithi027 ceph-mon[30779]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T07:13:52.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:52 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3428234874' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:52.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:52 smithi119 ceph-mon[34519]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T07:13:52.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:52 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3428234874' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:52.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:52 smithi153 ceph-mon[34747]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T07:13:52.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:52 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3428234874' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:13:52.973 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 2b5ff92 --release --await' 2024-09-21T07:13:52.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 2b5ff92 --release --await 2024-09-21T07:13:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.250+0000 7f7271400640 1 -- 172.21.15.27:0/417632663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1092f0 msgr2=0x7f726c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.250+0000 7f7271400640 1 --2- 172.21.15.27:0/417632663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1092f0 0x7f726c109700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7258009900 tx=0x7f725802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.251+0000 7f7271400640 1 -- 172.21.15.27:0/417632663 shutdown_connections 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.251+0000 7f7271400640 1 --2- 172.21.15.27:0/417632663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 0x7f726c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.251+0000 7f7271400640 1 --2- 172.21.15.27:0/417632663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1092f0 0x7f726c109700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.251+0000 7f7271400640 1 --2- 172.21.15.27:0/417632663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f726c1039b0 0x7f726c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.251+0000 7f7271400640 1 -- 172.21.15.27:0/417632663 >> 172.21.15.27:0/417632663 conn(0x7f726c0ff400 msgr2=0x7f726c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:54.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.251+0000 7f7271400640 1 -- 172.21.15.27:0/417632663 shutdown_connections 2024-09-21T07:13:54.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.251+0000 7f7271400640 1 -- 172.21.15.27:0/417632663 wait complete. 2024-09-21T07:13:54.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 Processor -- start 2024-09-21T07:13:54.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 -- start start 2024-09-21T07:13:54.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1039b0 0x7f726c1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f726c1092f0 0x7f726c1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 0x7f726c1ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f726c113950 con 0x7f726c1039b0 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1137d0 con 0x7f726c109c40 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.253+0000 7f7271400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f726c113ad0 con 0x7f726c1092f0 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.254+0000 7f7270a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 0x7f726c1ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.254+0000 7f7270a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 0x7f726c1ac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38000/0 (socket says 172.21.15.27:38000) 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.254+0000 7f7270a00640 1 -- 172.21.15.27:0/2281520017 learned_addr learned my addr 172.21.15.27:0/2281520017 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.254+0000 7f726b400640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f726c1092f0 0x7f726c1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.254+0000 7f7270a00640 1 -- 172.21.15.27:0/2281520017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f726c1092f0 msgr2=0x7f726c1a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.254+0000 7f7270a00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f726c1092f0 0x7f726c1a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.254+0000 7f7270a00640 1 -- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1039b0 msgr2=0x7f726c1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.255+0000 7f726be00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1039b0 0x7f726c1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.255+0000 7f7270a00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1039b0 0x7f726c1a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.255+0000 7f7270a00640 1 -- 172.21.15.27:0/2281520017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c1a23d0 con 0x7f726c109c40 2024-09-21T07:13:54.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.256+0000 7f726be00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1039b0 0x7f726c1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:13:54.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.256+0000 7f7270a00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 0x7f726c1ac670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f726400eb10 tx=0x7f726400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:54.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.256+0000 7f7268c00640 1 -- 172.21.15.27:0/2281520017 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264004b20 con 0x7f726c109c40 2024-09-21T07:13:54.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.256+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726c1acc70 con 0x7f726c109c40 2024-09-21T07:13:54.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.256+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726c1ad180 con 0x7f726c109c40 2024-09-21T07:13:54.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.257+0000 7f7268c00640 1 -- 172.21.15.27:0/2281520017 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264004cc0 con 0x7f726c109c40 2024-09-21T07:13:54.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.257+0000 7f7268c00640 1 -- 172.21.15.27:0/2281520017 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264010670 con 0x7f726c109c40 2024-09-21T07:13:54.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.261+0000 7f7279658740 1 Processor -- start 2024-09-21T07:13:54.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.263+0000 7f7268c00640 1 -- 172.21.15.27:0/2281520017 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154694+0+0 (secure 0 0 0) 0x7f7264010830 con 0x7f726c109c40 2024-09-21T07:13:54.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.263+0000 7f7279658740 1 -- start start 2024-09-21T07:13:54.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.263+0000 7f7279658740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6eab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.263+0000 7f7279658740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.263+0000 7f7279658740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eabdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.263+0000 7f7279658740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a1e6e22290 con 0x55a1e6b04520 2024-09-21T07:13:54.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.264+0000 7f7279658740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a1e6e116c0 con 0x55a1e6adb660 2024-09-21T07:13:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.264+0000 7f7279658740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a1e6e154d0 con 0x55a1e6ae3840 2024-09-21T07:13:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.264+0000 7f726b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eabdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.265+0000 7f726b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eabdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55774/0 (socket says 172.21.15.27:55774) 2024-09-21T07:13:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.265+0000 7f726b400640 1 -- 172.21.15.27:0/3925280970 learned_addr learned my addr 172.21.15.27:0/3925280970 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.265+0000 7f726b400640 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6eab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.265+0000 7f726be00640 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.265+0000 7f7268c00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f72500b6710 0x7f72500b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.266+0000 7f7268c00640 1 -- 172.21.15.27:0/2281520017 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f72640e8110 con 0x7f726c109c40 2024-09-21T07:13:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.266+0000 7f726be00640 1 -- 172.21.15.27:0/3925280970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 msgr2=0x55a1e6eabdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.266+0000 7f726be00640 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eabdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.266+0000 7f726be00640 1 -- 172.21.15.27:0/3925280970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 msgr2=0x55a1e6eab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.266+0000 7f726be00640 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6eab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.266+0000 7f726be00640 1 -- 172.21.15.27:0/3925280970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e6d9e460 con 0x55a1e6adb660 2024-09-21T07:13:54.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.267+0000 7f7270a00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f72500b6710 0x7f72500b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.267+0000 7f726be00640 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eab880 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f726c110510 tx=0x7f725c010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04b40ba57e39bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:54.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.268+0000 7f7261200640 1 -- 172.21.15.27:0/3925280970 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c010da0 con 0x55a1e6adb660 2024-09-21T07:13:54.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.268+0000 7f7261200640 1 -- 172.21.15.27:0/3925280970 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725c01b930 con 0x55a1e6adb660 2024-09-21T07:13:54.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.268+0000 7f7261200640 1 -- 172.21.15.27:0/3925280970 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c0129f0 con 0x55a1e6adb660 2024-09-21T07:13:54.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.268+0000 7f7270a00640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f72500b6710 0x7f72500b8c20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72640e5450 tx=0x7f7264009a70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:54.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 -- 172.21.15.27:0/3925280970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 msgr2=0x55a1e6eab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eab880 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f726c110510 tx=0x7f725c010370 comp rx=0 tx=0).stop 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 -- 172.21.15.27:0/3925280970 shutdown_connections 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eabdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eab880 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 --2- 172.21.15.27:0/3925280970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6eab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 -- 172.21.15.27:0/3925280970 >> 172.21.15.27:0/3925280970 conn(0x55a1e6af8910 msgr2=0x55a1e6ea9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.269+0000 7f7279658740 1 -- 172.21.15.27:0/3925280970 shutdown_connections 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.270+0000 7f7279658740 1 -- 172.21.15.27:0/3925280970 wait complete. 2024-09-21T07:13:54.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.270+0000 7f7279658740 1 -- start start 2024-09-21T07:13:54.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.271+0000 7f7279658740 1 Processor -- start 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.271+0000 7f7279658740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eb6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.271+0000 7f7279658740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f7279658740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6ed3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f7279658740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a1e6c72bb0 con 0x55a1e6b04520 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f7279658740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a1e6dac6c0 con 0x55a1e6adb660 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f7279658740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a1e6ecab30 con 0x55a1e6ae3840 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f7279658740 10 client.0 _ll_register_callbacks cb 0x55a1e6dc7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:54.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f726b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eb6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f726b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eb6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38004/0 (socket says 172.21.15.27:38004) 2024-09-21T07:13:54.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f726b400640 1 -- 172.21.15.27:0/2975294477 learned_addr learned my addr 172.21.15.27:0/2975294477 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f726b400640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6ed3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.272+0000 7f726be00640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f726b400640 1 -- 172.21.15.27:0/2975294477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 msgr2=0x55a1e6eb73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f726b400640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f726b400640 1 -- 172.21.15.27:0/2975294477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 msgr2=0x55a1e6ed3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f726b400640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6ed3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f726b400640 1 -- 172.21.15.27:0/2975294477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e6b6fb00 con 0x55a1e6adb660 2024-09-21T07:13:54.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f726be00640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f726b400640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eb6ea0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f725802a910 tx=0x7f725802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b122b571fa4857 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.273+0000 7f7255600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.274+0000 7f7255600640 1 -- 172.21.15.27:0/2975294477 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258035040 con 0x55a1e6adb660 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.274+0000 7f7255600640 1 -- 172.21.15.27:0/2975294477 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7258005020 con 0x55a1e6adb660 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.274+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f725c01b930 con 0x55a1e6adb660 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.274+0000 7f7255600640 1 -- 172.21.15.27:0/2975294477 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7258005220 con 0x55a1e6adb660 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.274+0000 7f7279658740 10 client.25847 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.274+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1e6be6960 con 0x55a1e6adb660 2024-09-21T07:13:54.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7255600640 1 -- 172.21.15.27:0/2975294477 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72580053c0 con 0x55a1e6adb660 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7255600640 1 -- 172.21.15.27:0/2975294477 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f725803a040 con 0x55a1e6adb660 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7279658740 10 client.25847 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7279658740 10 client.25847 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7279658740 20 client.25847 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7279658740 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6ac2ea0 0x55a1e6ed9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7279658740 4 client.25847 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.275+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f725c01b930 con 0x55a1e6ac2ea0 2024-09-21T07:13:54.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.276+0000 7f726be00640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6ac2ea0 0x55a1e6ed9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.277+0000 7f726be00640 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6ac2ea0 0x55a1e6ed9da0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a9575502a78797 server_cookie=e29eab76b39eee58 in_seq=0 out_seq=0 2024-09-21T07:13:54.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.277+0000 7f7255600640 10 client.25847 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:54.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.278+0000 7f7255600640 1 -- 172.21.15.27:0/2975294477 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f725c01b930 con 0x55a1e6ac2ea0 2024-09-21T07:13:54.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.278+0000 7f7255600640 10 client.25847 handle_command_reply: tid=0 2024-09-21T07:13:54.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.278+0000 7f7279658740 1 client.25847 shutdown 2024-09-21T07:13:54.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.278+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 msgr2=0x55a1e6eb6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.278+0000 7f7279658740 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eb6ea0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f725802a910 tx=0x7f725802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 shutdown_connections 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6ac2ea0 0x55a1e6ed9da0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6b04520 0x55a1e6ed3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 --2- 172.21.15.27:0/2975294477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6eb6ea0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7255600640 0 client.25847 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 >> 172.21.15.27:0/2975294477 conn(0x55a1e6af8910 msgr2=0x55a1e6eae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:54.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 shutdown_connections 2024-09-21T07:13:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 1 -- 172.21.15.27:0/2975294477 wait complete. 2024-09-21T07:13:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.279+0000 7f7279658740 20 client.25847 trim_cache size 0 max 16384 2024-09-21T07:13:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.316+0000 7f7279658740 1 Processor -- start 2024-09-21T07:13:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.316+0000 7f7279658740 1 -- start start 2024-09-21T07:13:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f7279658740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6af8910 0x55a1e6eb6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f7279658740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6adb660 0x55a1e6eb03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f7279658740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f7279658740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f724c012710 con 0x55a1e6af8910 2024-09-21T07:13:54.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f7279658740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f724c00ffe0 con 0x55a1e6ae3840 2024-09-21T07:13:54.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f7279658740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f724c00de30 con 0x55a1e6adb660 2024-09-21T07:13:54.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f726be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6adb660 0x55a1e6eb03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f726be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6adb660 0x55a1e6eb03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55788/0 (socket says 172.21.15.27:55788) 2024-09-21T07:13:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.317+0000 7f726be00640 1 -- 172.21.15.27:0/3499218581 learned_addr learned my addr 172.21.15.27:0/3499218581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.318+0000 7f726b400640 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6af8910 0x55a1e6eb6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.318+0000 7f726b400640 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.319+0000 7f726b400640 1 -- 172.21.15.27:0/3499218581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6adb660 msgr2=0x55a1e6eb03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.319+0000 7f726b400640 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6adb660 0x55a1e6eb03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.319+0000 7f726b400640 1 -- 172.21.15.27:0/3499218581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6ae3840 msgr2=0x55a1e6eb0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.319+0000 7f726b400640 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.319+0000 7f726b400640 1 -- 172.21.15.27:0/3499218581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e6eb2530 con 0x55a1e6af8910 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.319+0000 7f726b400640 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6af8910 0x55a1e6eb6e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7258031640 tx=0x7f7258000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7e7f50d7baa2aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.320+0000 7f7257e00640 1 -- 172.21.15.27:0/3499218581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258035070 con 0x55a1e6af8910 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.320+0000 7f7257e00640 1 -- 172.21.15.27:0/3499218581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7258031830 con 0x55a1e6af8910 2024-09-21T07:13:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.320+0000 7f7257e00640 1 -- 172.21.15.27:0/3499218581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258004dd0 con 0x55a1e6af8910 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.321+0000 7f7279658740 1 -- 172.21.15.27:0/3499218581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6af8910 msgr2=0x55a1e6eb6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.321+0000 7f7279658740 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6af8910 0x55a1e6eb6e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7258031640 tx=0x7f7258000bc0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.322+0000 7f7279658740 1 -- 172.21.15.27:0/3499218581 shutdown_connections 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.322+0000 7f7279658740 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6adb660 0x55a1e6eb03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.322+0000 7f7279658740 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6ae3840 0x55a1e6eb0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.322+0000 7f7279658740 1 --2- 172.21.15.27:0/3499218581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6af8910 0x55a1e6eb6e80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.322+0000 7f7279658740 1 -- 172.21.15.27:0/3499218581 >> 172.21.15.27:0/3499218581 conn(0x55a1e6eb7560 msgr2=0x55a1e6eb5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.322+0000 7f7279658740 1 -- 172.21.15.27:0/3499218581 shutdown_connections 2024-09-21T07:13:54.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.322+0000 7f7279658740 1 -- 172.21.15.27:0/3499218581 wait complete. 2024-09-21T07:13:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.323+0000 7f7279658740 1 -- start start 2024-09-21T07:13:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.323+0000 7f7279658740 1 Processor -- start 2024-09-21T07:13:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f7279658740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6f3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f7279658740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6ae3840 0x55a1e6f3e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f7279658740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6af8910 0x55a1e6ecfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f7279658740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f724c00e4a0 con 0x55a1e6ae3840 2024-09-21T07:13:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f7279658740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f724c00f910 con 0x55a1e6adb660 2024-09-21T07:13:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f7279658740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f724c00eb70 con 0x55a1e6af8910 2024-09-21T07:13:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f7279658740 10 client.0 _ll_register_callbacks cb 0x7f7258009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:13:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.324+0000 7f726be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6ae3840 0x55a1e6f3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6f3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6f3e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38018/0 (socket says 172.21.15.27:38018) 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 -- 172.21.15.27:0/2657869258 learned_addr learned my addr 172.21.15.27:0/2657869258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6af8910 0x55a1e6ecfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 -- 172.21.15.27:0/2657869258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6af8910 msgr2=0x55a1e6ecfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6af8910 0x55a1e6ecfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 -- 172.21.15.27:0/2657869258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6ae3840 msgr2=0x55a1e6f3e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6ae3840 0x55a1e6f3e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726b400640 1 -- 172.21.15.27:0/2657869258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e6ed0480 con 0x55a1e6adb660 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.325+0000 7f726be00640 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6ae3840 0x55a1e6f3e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:13:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.327+0000 7f726b400640 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6f3e0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f72580096c0 tx=0x7f72580035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc079eb1606c1595 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:13:54.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.327+0000 7f7255600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:13:54.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.327+0000 7f7255600640 1 -- 172.21.15.27:0/2657869258 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258035040 con 0x55a1e6adb660 2024-09-21T07:13:54.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.328+0000 7f7255600640 1 -- 172.21.15.27:0/2657869258 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7258005020 con 0x55a1e6adb660 2024-09-21T07:13:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.328+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7258031830 con 0x55a1e6adb660 2024-09-21T07:13:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.328+0000 7f7255600640 1 -- 172.21.15.27:0/2657869258 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72580053c0 con 0x55a1e6adb660 2024-09-21T07:13:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.328+0000 7f7279658740 10 client.25850 fetch_fsmap learned FSMap version 27 2024-09-21T07:13:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.328+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1e6f56150 con 0x55a1e6adb660 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7255600640 1 -- 172.21.15.27:0/2657869258 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7258031830 con 0x55a1e6adb660 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7255600640 1 -- 172.21.15.27:0/2657869258 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f725803d610 con 0x55a1e6adb660 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7279658740 10 client.25850 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7279658740 10 client.25850 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7279658740 20 client.25850 populate_metadata read hostname 'smithi027' 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7279658740 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6b04520 0x55a1e6f58640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7279658740 4 client.25850 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2b5ff92", "release": true, "await": true}] 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.329+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2b5ff92", "release": true, "await": true}) -- 0x55a1e6e241e0 con 0x55a1e6b04520 2024-09-21T07:13:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.330+0000 7f726be00640 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6b04520 0x55a1e6f58640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:13:54.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.330+0000 7f7255600640 10 client.25850 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:54.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.330+0000 7f726be00640 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6b04520 0x55a1e6f58640 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b84bcd4a107c4c server_cookie=3496dd9767014d89 in_seq=0 out_seq=0 2024-09-21T07:13:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.333+0000 7f7255600640 1 -- 172.21.15.27:0/2657869258 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55a1e6e241e0 con 0x55a1e6b04520 2024-09-21T07:13:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7255600640 10 client.25850 handle_command_reply: tid=0 2024-09-21T07:13:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 client.25850 shutdown 2024-09-21T07:13:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 msgr2=0x55a1e6f3e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6f3e0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f72580096c0 tx=0x7f72580035f0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 shutdown_connections 2024-09-21T07:13:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a1e6b04520 0x55a1e6f58640 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e6af8910 0x55a1e6ecfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a1e6ae3840 0x55a1e6f3e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7255600640 0 client.25850 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:13:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 --2- 172.21.15.27:0/2657869258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1e6adb660 0x55a1e6f3e0a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.334+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 >> 172.21.15.27:0/2657869258 conn(0x55a1e6eb7560 msgr2=0x55a1e6eac270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.335+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 shutdown_connections 2024-09-21T07:13:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.336+0000 7f7279658740 1 -- 172.21.15.27:0/2657869258 wait complete. 2024-09-21T07:13:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.336+0000 7f7279658740 20 client.25850 trim_cache size 0 max 16384 2024-09-21T07:13:54.338 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:13:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 4, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "2b5ff92": { 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 4, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 134.5, 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:13:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:13:54.340 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f72500b6710 msgr2=0x7f72500b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f72500b6710 0x7f72500b8c20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72640e5450 tx=0x7f7264009a70 comp rx=0 tx=0).stop 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 msgr2=0x7f726c1ac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 0x7f726c1ac670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f726400eb10 tx=0x7f726400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 shutdown_connections 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f726c1092f0 0x7f726c1a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f72500b6710 0x7f72500b8c20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f726c109c40 0x7f726c1ac670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 --2- 172.21.15.27:0/2281520017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f726c1039b0 0x7f726c1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.337+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 >> 172.21.15.27:0/2281520017 conn(0x7f726c0ff400 msgr2=0x7f726c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.338+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 shutdown_connections 2024-09-21T07:13:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:13:54.338+0000 7f7271400640 1 -- 172.21.15.27:0/2281520017 wait complete. 2024-09-21T07:13:54.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:54 smithi027 ceph-mon[30779]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T07:13:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:54 smithi119 ceph-mon[34519]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T07:13:54.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:54 smithi153 ceph-mon[34747]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T07:13:55.137 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2b5ff92', total seconds elapsed: 19.1 2024-09-21T07:13:56.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:56 smithi119 ceph-mon[34519]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-21T07:13:56.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:56 smithi027 ceph-mon[30779]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-21T07:13:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:56 smithi153 ceph-mon[34747]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-21T07:13:57.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:57 smithi027 ceph-mon[30779]: mgrmap e27: x(active, since 11m), standbys: z, y 2024-09-21T07:13:57.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:57 smithi153 ceph-mon[34747]: mgrmap e27: x(active, since 11m), standbys: z, y 2024-09-21T07:13:58.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:57 smithi119 ceph-mon[34519]: mgrmap e27: x(active, since 11m), standbys: z, y 2024-09-21T07:13:58.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:58 smithi027 ceph-mon[30779]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T07:13:58.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:58 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:13:58.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:58 smithi153 ceph-mon[34747]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T07:13:58.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:58 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:13:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:58 smithi119 ceph-mon[34519]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T07:13:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:58 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:13:59.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:13:59 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:13:59.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:13:59 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:14:00.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:13:59 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:14:00.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:00 smithi027 ceph-mon[30779]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T07:14:00.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:14:00.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:14:00.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:14:00.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:00 smithi153 ceph-mon[34747]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T07:14:00.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:14:00.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:14:00.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:14:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:00 smithi119 ceph-mon[34519]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T07:14:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:14:01.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:14:01.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:14:02.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:02 smithi027 ceph-mon[30779]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:14:02.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:02 smithi153 ceph-mon[34747]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:14:03.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:02 smithi119 ceph-mon[34519]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:14:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.662+0000 7f8022000640 1 -- 172.21.15.27:0/2604120091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c0760a0 msgr2=0x7f801c10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.662+0000 7f8022000640 1 --2- 172.21.15.27:0/2604120091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c0760a0 0x7f801c10f980 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8008009900 tx=0x7f800802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.662+0000 7f8022000640 1 -- 172.21.15.27:0/2604120091 shutdown_connections 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.662+0000 7f8022000640 1 --2- 172.21.15.27:0/2604120091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801c10fec0 0x7f801c1122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.662+0000 7f8022000640 1 --2- 172.21.15.27:0/2604120091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c0760a0 0x7f801c10f980 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.662+0000 7f8022000640 1 --2- 172.21.15.27:0/2604120091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f801c075750 0x7f801c075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.662+0000 7f8022000640 1 -- 172.21.15.27:0/2604120091 >> 172.21.15.27:0/2604120091 conn(0x7f801c0ff330 msgr2=0x7f801c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.663+0000 7f8022000640 1 -- 172.21.15.27:0/2604120091 shutdown_connections 2024-09-21T07:14:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.663+0000 7f8022000640 1 -- 172.21.15.27:0/2604120091 wait complete. 2024-09-21T07:14:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.664+0000 7f8022000640 1 Processor -- start 2024-09-21T07:14:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.665+0000 7f8022000640 1 -- start start 2024-09-21T07:14:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.665+0000 7f8022000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c075750 0x7f801c107360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.665+0000 7f8022000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f801c0760a0 0x7f801c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.665+0000 7f8022000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801c10fec0 0x7f801c103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.665+0000 7f8022000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f801c115a80 con 0x7f801c10fec0 2024-09-21T07:14:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.665+0000 7f8022000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f801c115900 con 0x7f801c075750 2024-09-21T07:14:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.665+0000 7f8022000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f801c115c00 con 0x7f801c0760a0 2024-09-21T07:14:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f801be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f801c0760a0 0x7f801c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f801be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f801c0760a0 0x7f801c103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:60326/0 (socket says 172.21.15.27:60326) 2024-09-21T07:14:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f801be00640 1 -- 172.21.15.27:0/226295370 learned_addr learned my addr 172.21.15.27:0/226295370 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8021600640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801c10fec0 0x7f801c103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8020c00640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c075750 0x7f801c107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8021600640 1 -- 172.21.15.27:0/226295370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f801c0760a0 msgr2=0x7f801c103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8021600640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f801c0760a0 0x7f801c103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8021600640 1 -- 172.21.15.27:0/226295370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c075750 msgr2=0x7f801c107360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8021600640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c075750 0x7f801c107360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8021600640 1 -- 172.21.15.27:0/226295370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c074060 con 0x7f801c10fec0 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.666+0000 7f8020c00640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c075750 0x7f801c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.667+0000 7f8021600640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801c10fec0 0x7f801c103d60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8014009fd0 tx=0x7f801400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.667+0000 7f8019600640 1 -- 172.21.15.27:0/226295370 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400ce20 con 0x7f801c10fec0 2024-09-21T07:14:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.667+0000 7f8019600640 1 -- 172.21.15.27:0/226295370 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8014005020 con 0x7f801c10fec0 2024-09-21T07:14:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.667+0000 7f8019600640 1 -- 172.21.15.27:0/226295370 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014010690 con 0x7f801c10fec0 2024-09-21T07:14:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.668+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c074350 con 0x7f801c10fec0 2024-09-21T07:14:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.668+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c074820 con 0x7f801c10fec0 2024-09-21T07:14:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.672+0000 7f802a5f6740 1 Processor -- start 2024-09-21T07:14:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.674+0000 7f8019600640 1 -- 172.21.15.27:0/226295370 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f80140054d0 con 0x7f801c10fec0 2024-09-21T07:14:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.674+0000 7f802a5f6740 1 -- start start 2024-09-21T07:14:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb800f2f0 0x55ccb83dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f802a5f6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ccb83526b0 con 0x55ccb80174d0 2024-09-21T07:14:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f802a5f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ccb8341ae0 con 0x55ccb800f2f0 2024-09-21T07:14:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f802a5f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ccb83458f0 con 0x55ccb80381b0 2024-09-21T07:14:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f8020c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb800f2f0 0x55ccb83dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.675+0000 7f801be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f8020c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb800f2f0 0x55ccb83dd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38864/0 (socket says 172.21.15.27:38864) 2024-09-21T07:14:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f8020c00640 1 -- 172.21.15.27:0/2684075828 learned_addr learned my addr 172.21.15.27:0/2684075828 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f801be00640 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83dce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:60334/0 (socket says 172.21.15.27:60334) 2024-09-21T07:14:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f801be00640 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f8020c00640 1 -- 172.21.15.27:0/2684075828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 msgr2=0x55ccb83dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f8020c00640 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83dce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f8020c00640 1 -- 172.21.15.27:0/2684075828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 msgr2=0x55ccb83dd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f8020c00640 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83dd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.676+0000 7f8020c00640 1 -- 172.21.15.27:0/2684075828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb830e780 con 0x55ccb800f2f0 2024-09-21T07:14:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.678+0000 7f8020c00640 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb800f2f0 0x55ccb83dd350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8010011520 tx=0x7f80100119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa6f7a6e50326d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.678+0000 7f8019600640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80040b6700 0x7f80040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.678+0000 7f800dc00640 1 -- 172.21.15.27:0/2684075828 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010011250 con 0x55ccb800f2f0 2024-09-21T07:14:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.679+0000 7f800dc00640 1 -- 172.21.15.27:0/2684075828 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80100054d0 con 0x55ccb800f2f0 2024-09-21T07:14:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.679+0000 7f800dc00640 1 -- 172.21.15.27:0/2684075828 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801000f840 con 0x55ccb800f2f0 2024-09-21T07:14:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.679+0000 7f801be00640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80040b6700 0x7f80040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.679+0000 7f8019600640 1 -- 172.21.15.27:0/226295370 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f80140e8150 con 0x7f801c10fec0 2024-09-21T07:14:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 -- 172.21.15.27:0/2684075828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb800f2f0 msgr2=0x55ccb83dd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb800f2f0 0x55ccb83dd350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8010011520 tx=0x7f80100119e0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f801be00640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80040b6700 0x7f80040b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c00 tx=0x7f8008005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 -- 172.21.15.27:0/2684075828 shutdown_connections 2024-09-21T07:14:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83dd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb800f2f0 0x55ccb83dd350 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2684075828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 -- 172.21.15.27:0/2684075828 >> 172.21.15.27:0/2684075828 conn(0x55ccb802c300 msgr2=0x55ccb83dad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 -- 172.21.15.27:0/2684075828 shutdown_connections 2024-09-21T07:14:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.680+0000 7f802a5f6740 1 -- 172.21.15.27:0/2684075828 wait complete. 2024-09-21T07:14:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.681+0000 7f802a5f6740 1 -- start start 2024-09-21T07:14:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.681+0000 7f802a5f6740 1 Processor -- start 2024-09-21T07:14:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.682+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.682+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.682+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f802a5f6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ccb8279ab0 con 0x55ccb800f2f0 2024-09-21T07:14:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f802a5f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ccb82e8570 con 0x55ccb80174d0 2024-09-21T07:14:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f802a5f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ccb83eb590 con 0x55ccb80381b0 2024-09-21T07:14:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f8020c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f8020c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83e83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40782/0 (socket says 172.21.15.27:40782) 2024-09-21T07:14:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f8020c00640 1 -- 172.21.15.27:0/1122282121 learned_addr learned my addr 172.21.15.27:0/1122282121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f802a5f6740 10 client.0 _ll_register_callbacks cb 0x55ccb82f7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f8021600640 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.683+0000 7f801be00640 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8021600640 1 -- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 msgr2=0x55ccb83e8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8021600640 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8021600640 1 -- 172.21.15.27:0/1122282121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 msgr2=0x55ccb83e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8021600640 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8021600640 1 -- 172.21.15.27:0/1122282121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb80a3790 con 0x55ccb80381b0 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8021600640 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e8e70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80140e54f0 tx=0x7f80140ad420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf6260c0aa29595 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8002000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8002000640 1 -- 172.21.15.27:0/1122282121 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140ad700 con 0x55ccb80381b0 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80100054d0 con 0x55ccb80381b0 2024-09-21T07:14:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.684+0000 7f8002000640 1 -- 172.21.15.27:0/1122282121 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f80140d10d0 con 0x55ccb80381b0 2024-09-21T07:14:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.685+0000 7f8002000640 1 -- 172.21.15.27:0/1122282121 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80140d51a0 con 0x55ccb80381b0 2024-09-21T07:14:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.685+0000 7f802a5f6740 10 client.25855 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.685+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ccb811c620 con 0x55ccb80381b0 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.685+0000 7f8002000640 1 -- 172.21.15.27:0/1122282121 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80100054d0 con 0x55ccb80381b0 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f8002000640 1 -- 172.21.15.27:0/1122282121 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f80140a9040 con 0x55ccb80381b0 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f802a5f6740 10 client.25855 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f802a5f6740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f802a5f6740 20 client.25855 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7ff6d80 0x55ccb83fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f802a5f6740 4 client.25855 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccb8354600 con 0x55ccb7ff6d80 2024-09-21T07:14:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.686+0000 7f8020c00640 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7ff6d80 0x55ccb83fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.687+0000 7f8020c00640 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7ff6d80 0x55ccb83fa5a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69412972450d0529 server_cookie=15399dcf9e6923e7 in_seq=0 out_seq=0 2024-09-21T07:14:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.687+0000 7f8002000640 10 client.25855 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.688+0000 7f8002000640 1 -- 172.21.15.27:0/1122282121 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccb8354600 con 0x55ccb7ff6d80 2024-09-21T07:14:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.688+0000 7f8002000640 10 client.25855 handle_command_reply: tid=0 2024-09-21T07:14:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.688+0000 7f802a5f6740 1 client.25855 shutdown 2024-09-21T07:14:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 msgr2=0x55ccb83e8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e8e70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80140e54f0 tx=0x7f80140ad420 comp rx=0 tx=0).stop 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 shutdown_connections 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7ff6d80 0x55ccb83fa5a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e8e70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1122282121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f8002000640 0 client.25855 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 >> 172.21.15.27:0/1122282121 conn(0x55ccb802c300 msgr2=0x55ccb83dde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 shutdown_connections 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 1 -- 172.21.15.27:0/1122282121 wait complete. 2024-09-21T07:14:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.689+0000 7f802a5f6740 20 client.25855 trim_cache size 0 max 16384 2024-09-21T07:14:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 Processor -- start 2024-09-21T07:14:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 -- start start 2024-09-21T07:14:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1c21a0 con 0x55ccb800f2f0 2024-09-21T07:14:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1bfa70 con 0x55ccb80174d0 2024-09-21T07:14:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.720+0000 7f802a5f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1bd8c0 con 0x55ccb80381b0 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.721+0000 7f8020c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.721+0000 7f801be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.721+0000 7f801be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38892/0 (socket says 172.21.15.27:38892) 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.721+0000 7f8020c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83ea020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40798/0 (socket says 172.21.15.27:40798) 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.721+0000 7f801be00640 1 -- 172.21.15.27:0/2740759812 learned_addr learned my addr 172.21.15.27:0/2740759812 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.721+0000 7f8021600640 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.722+0000 7f801be00640 1 -- 172.21.15.27:0/2740759812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 msgr2=0x55ccb83e5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.722+0000 7f801be00640 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.722+0000 7f801be00640 1 -- 172.21.15.27:0/2740759812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 msgr2=0x55ccb83ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.722+0000 7f801be00640 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.722+0000 7f801be00640 1 -- 172.21.15.27:0/2740759812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb83dfc20 con 0x55ccb80174d0 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.722+0000 7f8021600640 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.722+0000 7f801be00640 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e55c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f800802a9e0 tx=0x7f8008009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc90287246611735 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.723+0000 7f800d200640 1 -- 172.21.15.27:0/2740759812 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80080314a0 con 0x55ccb80174d0 2024-09-21T07:14:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.723+0000 7f800d200640 1 -- 172.21.15.27:0/2740759812 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8008051070 con 0x55ccb80174d0 2024-09-21T07:14:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.723+0000 7f800d200640 1 -- 172.21.15.27:0/2740759812 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800803da10 con 0x55ccb80174d0 2024-09-21T07:14:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 -- 172.21.15.27:0/2740759812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 msgr2=0x55ccb83e55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e55c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f800802a9e0 tx=0x7f8008009260 comp rx=0 tx=0).stop 2024-09-21T07:14:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 -- 172.21.15.27:0/2740759812 shutdown_connections 2024-09-21T07:14:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb80381b0 0x55ccb83e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80174d0 0x55ccb83e55c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 --2- 172.21.15.27:0/2740759812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb800f2f0 0x55ccb83ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 -- 172.21.15.27:0/2740759812 >> 172.21.15.27:0/2740759812 conn(0x55ccb802c300 msgr2=0x55ccb83e9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 -- 172.21.15.27:0/2740759812 shutdown_connections 2024-09-21T07:14:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.724+0000 7f802a5f6740 1 -- 172.21.15.27:0/2740759812 wait complete. 2024-09-21T07:14:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.725+0000 7f802a5f6740 1 -- start start 2024-09-21T07:14:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.725+0000 7f802a5f6740 1 Processor -- start 2024-09-21T07:14:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb800f2f0 0x55ccb83f1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f802a5f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80381b0 0x55ccb83eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f802a5f6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1bdf30 con 0x55ccb80174d0 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f802a5f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1bf3a0 con 0x55ccb80381b0 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f802a5f6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1be600 con 0x55ccb800f2f0 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f802a5f6740 10 client.0 _ll_register_callbacks cb 0x7f8014005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f8021600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80381b0 0x55ccb83eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f8021600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80381b0 0x55ccb83eecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38896/0 (socket says 172.21.15.27:38896) 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f8021600640 1 -- 172.21.15.27:0/1060427763 learned_addr learned my addr 172.21.15.27:0/1060427763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f8020c00640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb800f2f0 0x55ccb83f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.726+0000 7f801be00640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f8021600640 1 -- 172.21.15.27:0/1060427763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb800f2f0 msgr2=0x55ccb83f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f8021600640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb800f2f0 0x55ccb83f1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f8021600640 1 -- 172.21.15.27:0/1060427763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 msgr2=0x55ccb83ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f8021600640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f8021600640 1 -- 172.21.15.27:0/1060427763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb83ef440 con 0x55ccb80381b0 2024-09-21T07:14:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f801be00640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f8021600640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80381b0 0x55ccb83eecb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80140aa1c0 tx=0x7f8014009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b91c1308dff0e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.727+0000 7f8002000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.728+0000 7f8002000640 1 -- 172.21.15.27:0/1060427763 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140c7020 con 0x55ccb80381b0 2024-09-21T07:14:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.728+0000 7f8002000640 1 -- 172.21.15.27:0/1060427763 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f80140b3ae0 con 0x55ccb80381b0 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.728+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8008051070 con 0x55ccb80381b0 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.728+0000 7f8002000640 1 -- 172.21.15.27:0/1060427763 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80140ad700 con 0x55ccb80381b0 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.728+0000 7f802a5f6740 10 client.25868 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.728+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ccb8465020 con 0x55ccb80381b0 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f8002000640 1 -- 172.21.15.27:0/1060427763 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8008051070 con 0x55ccb80381b0 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f8002000640 1 -- 172.21.15.27:0/1060427763 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f80140a9040 con 0x55ccb80381b0 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f802a5f6740 10 client.25868 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f802a5f6740 10 client.25868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f802a5f6740 20 client.25868 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7fcc1c0 0x55ccb8467800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f802a5f6740 4 client.25868 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:14:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccb83ead60 con 0x55ccb7fcc1c0 2024-09-21T07:14:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.729+0000 7f8020c00640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7fcc1c0 0x55ccb8467800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.730+0000 7f8020c00640 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7fcc1c0 0x55ccb8467800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342e4fb6c82e68b9 server_cookie=f730adeb5d159844 in_seq=0 out_seq=0 2024-09-21T07:14:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.730+0000 7f8002000640 10 client.25868 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.736+0000 7f8002000640 1 -- 172.21.15.27:0/1060427763 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccb83ead60 con 0x55ccb7fcc1c0 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f8002000640 10 client.25868 handle_command_reply: tid=0 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 client.25868 shutdown 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80381b0 msgr2=0x55ccb83eecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80381b0 0x55ccb83eecb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80140aa1c0 tx=0x7f8014009590 comp rx=0 tx=0).stop 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 shutdown_connections 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ccb7fcc1c0 0x55ccb8467800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ccb80381b0 0x55ccb83eecb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ccb80174d0 0x55ccb83ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f8002000640 0 client.25868 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 --2- 172.21.15.27:0/1060427763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb800f2f0 0x55ccb83f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 >> 172.21.15.27:0/1060427763 conn(0x55ccb802c300 msgr2=0x55ccb83dde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 shutdown_connections 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 1 -- 172.21.15.27:0/1060427763 wait complete. 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.737+0000 7f802a5f6740 20 client.25868 trim_cache size 0 max 16384 2024-09-21T07:14:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:14:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:14:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:14:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:14:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.738+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80040b6700 msgr2=0x7f80040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.738+0000 7f8022000640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80040b6700 0x7f80040b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c00 tx=0x7f8008005a20 comp rx=0 tx=0).stop 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.738+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801c10fec0 msgr2=0x7f801c103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.738+0000 7f8022000640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801c10fec0 0x7f801c103d60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8014009fd0 tx=0x7f801400ef10 comp rx=0 tx=0).stop 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 shutdown_connections 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f801c0760a0 0x7f801c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f80040b6700 0x7f80040b8c10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f801c10fec0 0x7f801c103d60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 --2- 172.21.15.27:0/226295370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c075750 0x7f801c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 >> 172.21.15.27:0/226295370 conn(0x7f801c0ff330 msgr2=0x7f801c109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 shutdown_connections 2024-09-21T07:14:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:04.739+0000 7f8022000640 1 -- 172.21.15.27:0/226295370 wait complete. 2024-09-21T07:14:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:04 smithi027 ceph-mon[30779]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-21T07:14:04.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:04 smithi153 ceph-mon[34747]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-21T07:14:05.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:04 smithi119 ceph-mon[34519]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-21T07:14:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:06 smithi027 ceph-mon[30779]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T07:14:06.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:06 smithi153 ceph-mon[34747]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T07:14:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:06 smithi119 ceph-mon[34519]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T07:14:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:08 smithi027 ceph-mon[30779]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:14:08.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:08 smithi153 ceph-mon[34747]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:14:09.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:08 smithi119 ceph-mon[34519]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:14:10.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:10 smithi119 ceph-mon[34519]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-21T07:14:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:10 smithi027 ceph-mon[30779]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-21T07:14:10.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:10 smithi153 ceph-mon[34747]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-21T07:14:12.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:12 smithi027 ceph-mon[30779]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:14:12.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:12 smithi153 ceph-mon[34747]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:14:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:12 smithi119 ceph-mon[34519]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T07:14:14.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:14 smithi027 ceph-mon[30779]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-21T07:14:14.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:14 smithi153 ceph-mon[34747]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-21T07:14:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:14 smithi119 ceph-mon[34519]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-21T07:14:15.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:14:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:16.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:16 smithi027 ceph-mon[30779]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:16.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:16 smithi153 ceph-mon[34747]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:16 smithi119 ceph-mon[34519]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 -- 172.21.15.27:0/1107586411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 msgr2=0x7f6ad0110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 --2- 172.21.15.27:0/1107586411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 0x7f6ad0110110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009be0 tx=0x7f6ac802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 -- 172.21.15.27:0/1107586411 shutdown_connections 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 --2- 172.21.15.27:0/1107586411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad0105110 0x7f6ad010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 --2- 172.21.15.27:0/1107586411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 0x7f6ad0110110 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 --2- 172.21.15.27:0/1107586411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 0x7f6ad0104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 -- 172.21.15.27:0/1107586411 >> 172.21.15.27:0/1107586411 conn(0x7f6ad00fe2c0 msgr2=0x7f6ad0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 -- 172.21.15.27:0/1107586411 shutdown_connections 2024-09-21T07:14:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.176+0000 7f6acfe00640 1 -- 172.21.15.27:0/1107586411 wait complete. 2024-09-21T07:14:17.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.178+0000 7f6acfe00640 1 Processor -- start 2024-09-21T07:14:17.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.178+0000 7f6acfe00640 1 -- start start 2024-09-21T07:14:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.178+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 0x7f6ad019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.178+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad0105110 0x7f6ad019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 0x7f6ad01a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acfe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0113880 con 0x7f6ad01047c0 2024-09-21T07:14:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0113700 con 0x7f6ad010dc20 2024-09-21T07:14:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acfe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0113a00 con 0x7f6ad0105110 2024-09-21T07:14:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 0x7f6ad019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 0x7f6ad019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36674/0 (socket says 172.21.15.27:36674) 2024-09-21T07:14:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acea00640 1 -- 172.21.15.27:0/2977362118 learned_addr learned my addr 172.21.15.27:0/2977362118 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6acf400640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 0x7f6ad01a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.179+0000 7f6ace000640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad0105110 0x7f6ad019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6acea00640 1 -- 172.21.15.27:0/2977362118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad0105110 msgr2=0x7f6ad019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6acea00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad0105110 0x7f6ad019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6acea00640 1 -- 172.21.15.27:0/2977362118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 msgr2=0x7f6ad01a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6acea00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 0x7f6ad01a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6acea00640 1 -- 172.21.15.27:0/2977362118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad01a2420 con 0x7f6ad01047c0 2024-09-21T07:14:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6ace000640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad0105110 0x7f6ad019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6acf400640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 0x7f6ad01a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.180+0000 7f6acea00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 0x7f6ad019c4b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e9c0 tx=0x7f6ac000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.181+0000 7f6ac7400640 1 -- 172.21.15.27:0/2977362118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000ccc0 con 0x7f6ad01047c0 2024-09-21T07:14:17.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.181+0000 7f6ac7400640 1 -- 172.21.15.27:0/2977362118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac00054d0 con 0x7f6ad01047c0 2024-09-21T07:14:17.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.181+0000 7f6ac7400640 1 -- 172.21.15.27:0/2977362118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0010430 con 0x7f6ad01047c0 2024-09-21T07:14:17.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.181+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad01a2710 con 0x7f6ad01047c0 2024-09-21T07:14:17.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.181+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad01a8e70 con 0x7f6ad01047c0 2024-09-21T07:14:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.186+0000 7f6adc460740 1 Processor -- start 2024-09-21T07:14:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.186+0000 7f6adc460740 1 -- start start 2024-09-21T07:14:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6adc460740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2fedf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6adc460740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c1e1b0 0x5578b2fee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6adc460740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b2fee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6adc460740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578b2f64f70 con 0x5578b2c26390 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6adc460740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5578b2f543a0 con 0x5578b2c47070 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6adc460740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578b2f581b0 con 0x5578b2c1e1b0 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6acf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2fedf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6acf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2fedf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53600/0 (socket says 172.21.15.27:53600) 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6acf400640 1 -- 172.21.15.27:0/2042940082 learned_addr learned my addr 172.21.15.27:0/2042940082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6ac7400640 1 -- 172.21.15.27:0/2977362118 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6ac0004f80 con 0x7f6ad01047c0 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.187+0000 7f6ace000640 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b2fee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.188+0000 7f6acf400640 1 -- 172.21.15.27:0/2042940082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c1e1b0 msgr2=0x5578b2fee470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:14:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.188+0000 7f6acf400640 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c1e1b0 0x5578b2fee470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.188+0000 7f6acf400640 1 -- 172.21.15.27:0/2042940082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 msgr2=0x5578b2fee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.189+0000 7f6acf400640 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b2fee9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.189+0000 7f6acf400640 1 -- 172.21.15.27:0/2042940082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b2f1ec90 con 0x5578b2c47070 2024-09-21T07:14:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.189+0000 7f6ace000640 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b2fee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.189+0000 7f6acf400640 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2fedf30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009be0 tx=0x7f6ac80307e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7fde6bb5c95832 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.190+0000 7f6ac7400640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ab40b6780 0x7f6ab40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.190+0000 7f6abbe00640 1 -- 172.21.15.27:0/2042940082 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8041030 con 0x5578b2c47070 2024-09-21T07:14:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.190+0000 7f6abbe00640 1 -- 172.21.15.27:0/2042940082 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac8030970 con 0x5578b2c47070 2024-09-21T07:14:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.190+0000 7f6abbe00640 1 -- 172.21.15.27:0/2042940082 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8036040 con 0x5578b2c47070 2024-09-21T07:14:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.190+0000 7f6ac7400640 1 -- 172.21.15.27:0/2977362118 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f6ac00e7ef0 con 0x7f6ad01047c0 2024-09-21T07:14:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.190+0000 7f6acea00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ab40b6780 0x7f6ab40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6adc460740 1 -- 172.21.15.27:0/2042940082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 msgr2=0x5578b2fedf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6adc460740 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2fedf30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009be0 tx=0x7f6ac80307e0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6acea00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ab40b6780 0x7f6ab40b8c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5578b2fecbb0 tx=0x7f6ac0005710 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6adc460740 1 -- 172.21.15.27:0/2042940082 shutdown_connections 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6adc460740 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c1e1b0 0x5578b2fee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6adc460740 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b2fee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6adc460740 1 --2- 172.21.15.27:0/2042940082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2fedf30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.192+0000 7f6adc460740 1 -- 172.21.15.27:0/2042940082 >> 172.21.15.27:0/2042940082 conn(0x5578b2c3b460 msgr2=0x5578b2febe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.193+0000 7f6adc460740 1 -- 172.21.15.27:0/2042940082 shutdown_connections 2024-09-21T07:14:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.193+0000 7f6adc460740 1 -- 172.21.15.27:0/2042940082 wait complete. 2024-09-21T07:14:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.194+0000 7f6adc460740 1 -- start start 2024-09-21T07:14:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.194+0000 7f6adc460740 1 Processor -- start 2024-09-21T07:14:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6adc460740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c1e1b0 0x5578b3001e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6adc460740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b30023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6adc460740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c47070 0x5578b300e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6adc460740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578b2db5700 con 0x5578b2c26390 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6adc460740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5578b2eef540 con 0x5578b2c1e1b0 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6adc460740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578b3004de0 con 0x5578b2c47070 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6adc460740 10 client.0 _ll_register_callbacks cb 0x5578b2eff430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6ace000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b30023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6ace000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b30023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36688/0 (socket says 172.21.15.27:36688) 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.195+0000 7f6ace000640 1 -- 172.21.15.27:0/705887281 learned_addr learned my addr 172.21.15.27:0/705887281 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.196+0000 7f6acf400640 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c47070 0x5578b300e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.196+0000 7f6acf400640 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c1e1b0 0x5578b3001e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.197+0000 7f6acf400640 1 -- 172.21.15.27:0/705887281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c47070 msgr2=0x5578b300e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.197+0000 7f6acf400640 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c47070 0x5578b300e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.197+0000 7f6acf400640 1 -- 172.21.15.27:0/705887281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 msgr2=0x5578b30023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.197+0000 7f6acf400640 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b30023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.197+0000 7f6acf400640 1 -- 172.21.15.27:0/705887281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b2cb2650 con 0x5578b2c1e1b0 2024-09-21T07:14:17.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.198+0000 7f6acf400640 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c1e1b0 0x5578b3001e90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8004af0 tx=0x7f6ac802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97158edfb494063 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.198+0000 7f6ab8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.198+0000 7f6ab8c00640 1 -- 172.21.15.27:0/705887281 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8036070 con 0x5578b2c1e1b0 2024-09-21T07:14:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.198+0000 7f6ab8c00640 1 -- 172.21.15.27:0/705887281 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f6ac8003b30 con 0x5578b2c1e1b0 2024-09-21T07:14:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.198+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5578b2ef1b50 con 0x5578b2c1e1b0 2024-09-21T07:14:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.199+0000 7f6ab8c00640 1 -- 172.21.15.27:0/705887281 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6ac8041040 con 0x5578b2c1e1b0 2024-09-21T07:14:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.199+0000 7f6adc460740 10 client.25880 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.199+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578b2f0c910 con 0x5578b2c1e1b0 2024-09-21T07:14:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.200+0000 7f6ab8c00640 1 -- 172.21.15.27:0/705887281 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5578b2ef1b50 con 0x5578b2c1e1b0 2024-09-21T07:14:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6ab8c00640 1 -- 172.21.15.27:0/705887281 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6ac803d7d0 con 0x5578b2c1e1b0 2024-09-21T07:14:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6adc460740 10 client.25880 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6adc460740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6adc460740 20 client.25880 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6adc460740 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2c059f0 0x5578b3013d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6adc460740 4 client.25880 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578b2f66ec0 con 0x5578b2c059f0 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.201+0000 7f6ace000640 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2c059f0 0x5578b3013d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.203+0000 7f6ace000640 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2c059f0 0x5578b3013d90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0a55efa4f97573 server_cookie=6f2f9520fd12b8d5 in_seq=0 out_seq=0 2024-09-21T07:14:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.203+0000 7f6ab8c00640 10 client.25880 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.203+0000 7f6ab8c00640 1 -- 172.21.15.27:0/705887281 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578b2f66ec0 con 0x5578b2c059f0 2024-09-21T07:14:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.203+0000 7f6ab8c00640 10 client.25880 handle_command_reply: tid=0 2024-09-21T07:14:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.204+0000 7f6adc460740 1 client.25880 shutdown 2024-09-21T07:14:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.204+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c1e1b0 msgr2=0x5578b3001e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.204+0000 7f6adc460740 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c1e1b0 0x5578b3001e90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8004af0 tx=0x7f6ac802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 shutdown_connections 2024-09-21T07:14:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2c059f0 0x5578b3013d90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c47070 0x5578b300e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c26390 0x5578b30023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 --2- 172.21.15.27:0/705887281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c1e1b0 0x5578b3001e90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6ab8c00640 0 client.25880 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 >> 172.21.15.27:0/705887281 conn(0x5578b2c3b460 msgr2=0x5578b2fea5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 shutdown_connections 2024-09-21T07:14:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 1 -- 172.21.15.27:0/705887281 wait complete. 2024-09-21T07:14:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.205+0000 7f6adc460740 20 client.25880 trim_cache size 0 max 16384 2024-09-21T07:14:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.226+0000 7f6adc460740 1 Processor -- start 2024-09-21T07:14:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.226+0000 7f6adc460740 1 -- start start 2024-09-21T07:14:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.226+0000 7f6adc460740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6adc460740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b2fef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6adc460740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2feb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6adc460740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01b13a0 con 0x5578b2c1e1b0 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6adc460740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01aec70 con 0x5578b2c47070 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6adc460740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01acb00 con 0x5578b2c26390 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6acf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6acf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36694/0 (socket says 172.21.15.27:36694) 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6acf400640 1 -- 172.21.15.27:0/4041929970 learned_addr learned my addr 172.21.15.27:0/4041929970 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6acf400640 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2feb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.227+0000 7f6ace000640 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b2fef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.228+0000 7f6acf400640 1 -- 172.21.15.27:0/4041929970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 msgr2=0x5578b2fef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.228+0000 7f6acf400640 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b2fef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.228+0000 7f6acf400640 1 -- 172.21.15.27:0/4041929970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 msgr2=0x5578b2feb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.228+0000 7f6acf400640 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2feb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.228+0000 7f6acf400640 1 -- 172.21.15.27:0/4041929970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b2fe7460 con 0x5578b2c1e1b0 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.228+0000 7f6acf400640 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3003d70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009300 tx=0x7f6ac8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b5275d7b83848d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.229+0000 7f6abb400640 1 -- 172.21.15.27:0/4041929970 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8036030 con 0x5578b2c1e1b0 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.229+0000 7f6abb400640 1 -- 172.21.15.27:0/4041929970 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac80047c0 con 0x5578b2c1e1b0 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.229+0000 7f6abb400640 1 -- 172.21.15.27:0/4041929970 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8030cd0 con 0x5578b2c1e1b0 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 -- 172.21.15.27:0/4041929970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 msgr2=0x5578b3003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3003d70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009300 tx=0x7f6ac8004a40 comp rx=0 tx=0).stop 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 -- 172.21.15.27:0/4041929970 shutdown_connections 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b2fef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b2feb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 --2- 172.21.15.27:0/4041929970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3003d70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 -- 172.21.15.27:0/4041929970 >> 172.21.15.27:0/4041929970 conn(0x5578b2c3b460 msgr2=0x5578b2feef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 -- 172.21.15.27:0/4041929970 shutdown_connections 2024-09-21T07:14:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.230+0000 7f6adc460740 1 -- 172.21.15.27:0/4041929970 wait complete. 2024-09-21T07:14:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.231+0000 7f6adc460740 1 -- start start 2024-09-21T07:14:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.231+0000 7f6adc460740 1 Processor -- start 2024-09-21T07:14:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.232+0000 7f6adc460740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3014d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.232+0000 7f6adc460740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b3015280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.232+0000 7f6adc460740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b30157c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.232+0000 7f6adc460740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01ad150 con 0x5578b2c1e1b0 2024-09-21T07:14:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.232+0000 7f6adc460740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01ae5a0 con 0x5578b2c47070 2024-09-21T07:14:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.232+0000 7f6adc460740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01ad800 con 0x5578b2c26390 2024-09-21T07:14:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.232+0000 7f6adc460740 10 client.0 _ll_register_callbacks cb 0x7f6ac8004710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6acf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b30157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6acf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3014d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b3015280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b3015280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58068/0 (socket says 172.21.15.27:58068) 2024-09-21T07:14:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6acf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3014d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36696/0 (socket says 172.21.15.27:36696) 2024-09-21T07:14:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 -- 172.21.15.27:0/1716447315 learned_addr learned my addr 172.21.15.27:0/1716447315 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 -- 172.21.15.27:0/1716447315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 msgr2=0x5578b30157c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b30157c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 -- 172.21.15.27:0/1716447315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 msgr2=0x5578b3014d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3014d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.233+0000 7f6ace000640 1 -- 172.21.15.27:0/1716447315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b307d510 con 0x5578b2c26390 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.234+0000 7f6ace000640 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b3015280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5578b2f544f0 tx=0x7f6abc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122943505d5ffb0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.234+0000 7f6ab8c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.234+0000 7f6ab8c00640 1 -- 172.21.15.27:0/1716447315 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc01e440 con 0x5578b2c26390 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.235+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ac80047c0 con 0x5578b2c26390 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.235+0000 7f6ab8c00640 1 -- 172.21.15.27:0/1716447315 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f6abc01e5e0 con 0x5578b2c26390 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.235+0000 7f6ab8c00640 1 -- 172.21.15.27:0/1716447315 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6abc01e7e0 con 0x5578b2c26390 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.235+0000 7f6adc460740 10 client.25876 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.235+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578b307e7e0 con 0x5578b2c26390 2024-09-21T07:14:17.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6ab8c00640 1 -- 172.21.15.27:0/1716447315 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac80047c0 con 0x5578b2c26390 2024-09-21T07:14:17.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6ab8c00640 1 -- 172.21.15.27:0/1716447315 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6abc012b00 con 0x5578b2c26390 2024-09-21T07:14:17.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6adc460740 10 client.25876 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:17.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6adc460740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:17.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6adc460740 20 client.25876 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:17.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6adc460740 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2bdada0 0x5578b3081000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6adc460740 4 client.25876 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:14:17.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.236+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578b2f62970 con 0x5578b2bdada0 2024-09-21T07:14:17.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.237+0000 7f6acf400640 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2bdada0 0x5578b3081000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.237+0000 7f6acf400640 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2bdada0 0x5578b3081000 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde648b4dbc90703 server_cookie=b7ed0118f214e614 in_seq=0 out_seq=0 2024-09-21T07:14:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.237+0000 7f6ab8c00640 10 client.25876 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6ab8c00640 1 -- 172.21.15.27:0/1716447315 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x5578b2f62970 con 0x5578b2bdada0 2024-09-21T07:14:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6ab8c00640 10 client.25876 handle_command_reply: tid=0 2024-09-21T07:14:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 client.25876 shutdown 2024-09-21T07:14:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 msgr2=0x5578b3015280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b3015280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5578b2f544f0 tx=0x7f6abc009b60 comp rx=0 tx=0).stop 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 shutdown_connections 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578b2bdada0 0x5578b3081000 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578b2c26390 0x5578b3015280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578b2c47070 0x5578b30157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 --2- 172.21.15.27:0/1716447315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578b2c1e1b0 0x5578b3014d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 >> 172.21.15.27:0/1716447315 conn(0x5578b2c3b460 msgr2=0x5578b3000ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.238+0000 7f6ab8c00640 0 client.25876 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.239+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 shutdown_connections 2024-09-21T07:14:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.239+0000 7f6adc460740 1 -- 172.21.15.27:0/1716447315 wait complete. 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.239+0000 7f6adc460740 20 client.25876 trim_cache size 0 max 16384 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:14:17.242 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:17.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ab40b6780 msgr2=0x7f6ab40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ab40b6780 0x7f6ab40b8c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5578b2fecbb0 tx=0x7f6ac0005710 comp rx=0 tx=0).stop 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 msgr2=0x7f6ad019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 0x7f6ad019c4b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e9c0 tx=0x7f6ac000ee80 comp rx=0 tx=0).stop 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 shutdown_connections 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ad0105110 0x7f6ad019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6ab40b6780 0x7f6ab40b8c90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad010dc20 0x7f6ad01a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 --2- 172.21.15.27:0/2977362118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad01047c0 0x7f6ad019c4b0 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e9c0 tx=0x7f6ac000ee80 comp rx=0 tx=0).stop 2024-09-21T07:14:17.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.240+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 >> 172.21.15.27:0/2977362118 conn(0x7f6ad00fe2c0 msgr2=0x7f6ad01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.241+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 shutdown_connections 2024-09-21T07:14:17.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:17.241+0000 7f6acfe00640 1 -- 172.21.15.27:0/2977362118 wait complete. 2024-09-21T07:14:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 -- 172.21.15.27:0/1301201321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934103ae0 msgr2=0x7fb934103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 --2- 172.21.15.27:0/1301201321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934103ae0 0x7fb934103ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb924009900 tx=0x7fb92402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 -- 172.21.15.27:0/1301201321 shutdown_connections 2024-09-21T07:14:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 --2- 172.21.15.27:0/1301201321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934109d60 0x7fb934110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 --2- 172.21.15.27:0/1301201321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934109410 0x7fb934109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 --2- 172.21.15.27:0/1301201321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934103ae0 0x7fb934103ef0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 -- 172.21.15.27:0/1301201321 >> 172.21.15.27:0/1301201321 conn(0x7fb9340ff520 msgr2=0x7fb934101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 -- 172.21.15.27:0/1301201321 shutdown_connections 2024-09-21T07:14:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.953+0000 7fb93a400640 1 -- 172.21.15.27:0/1301201321 wait complete. 2024-09-21T07:14:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.955+0000 7fb93a400640 1 Processor -- start 2024-09-21T07:14:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.955+0000 7fb93a400640 1 -- start start 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.955+0000 7fb93a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934103ae0 0x7fb934075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.955+0000 7fb93a400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934109410 0x7fb934075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb93a400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934109d60 0x7fb934076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb93a400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb934113a20 con 0x7fb934103ae0 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb93a400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341138a0 con 0x7fb934109410 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb93a400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb934113ba0 con 0x7fb934109d60 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934103ae0 0x7fb934075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934103ae0 0x7fb934075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36716/0 (socket says 172.21.15.27:36716) 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 -- 172.21.15.27:0/3900546168 learned_addr learned my addr 172.21.15.27:0/3900546168 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb92aa00640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934109410 0x7fb934075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933e00640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934109d60 0x7fb934076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 -- 172.21.15.27:0/3900546168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934109d60 msgr2=0x7fb934076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934109d60 0x7fb934076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 -- 172.21.15.27:0/3900546168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934109410 msgr2=0x7fb934075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934109410 0x7fb934075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.956+0000 7fb933400640 1 -- 172.21.15.27:0/3900546168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9341a8730 con 0x7fb934103ae0 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.957+0000 7fb933400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934103ae0 0x7fb934075710 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb924005940 tx=0x7fb9240040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.957+0000 7fb930c00640 1 -- 172.21.15.27:0/3900546168 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92403f030 con 0x7fb934103ae0 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.957+0000 7fb930c00640 1 -- 172.21.15.27:0/3900546168 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9240044c0 con 0x7fb934103ae0 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.957+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9341a89c0 con 0x7fb934103ae0 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.957+0000 7fb930c00640 1 -- 172.21.15.27:0/3900546168 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924035070 con 0x7fb934103ae0 2024-09-21T07:14:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.957+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9341a8eb0 con 0x7fb934103ae0 2024-09-21T07:14:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 Processor -- start 2024-09-21T07:14:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 -- start start 2024-09-21T07:14:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622503fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225040170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x5562250406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556224fb56b0 con 0x556224c722f0 2024-09-21T07:14:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556224fa4ae0 con 0x556224c7a4d0 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb94161c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556224fa88f0 con 0x556224c9b1b0 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb930c00640 1 -- 172.21.15.27:0/3900546168 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb924042030 con 0x7fb934103ae0 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.961+0000 7fb933e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622503fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622503fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45492/0 (socket says 172.21.15.27:45492) 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 -- 172.21.15.27:0/3153057870 learned_addr learned my addr 172.21.15.27:0/3153057870 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb92aa00640 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x5562250406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933400640 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225040170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 -- 172.21.15.27:0/3153057870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 msgr2=0x5562250406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x5562250406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 -- 172.21.15.27:0/3153057870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 msgr2=0x556225040170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225040170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 -- 172.21.15.27:0/3153057870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556224f71780 con 0x556224c9b1b0 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.962+0000 7fb933e00640 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622503fc30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb92c011470 tx=0x7fb92c011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11fa79555e7e274f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.963+0000 7fb928c00640 1 -- 172.21.15.27:0/3153057870 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c01e070 con 0x556224c9b1b0 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.963+0000 7fb928c00640 1 -- 172.21.15.27:0/3153057870 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92c003c30 con 0x556224c9b1b0 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.963+0000 7fb928c00640 1 -- 172.21.15.27:0/3153057870 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00f900 con 0x556224c9b1b0 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb930c00640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9180b66b0 0x7fb9180b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb933400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9180b66b0 0x7fb9180b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb930c00640 1 -- 172.21.15.27:0/3900546168 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb924107dc0 con 0x7fb934103ae0 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 -- 172.21.15.27:0/3153057870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 msgr2=0x55622503fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622503fc30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb92c011470 tx=0x7fb92c011930 comp rx=0 tx=0).stop 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 -- 172.21.15.27:0/3153057870 shutdown_connections 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x5562250406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225040170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 --2- 172.21.15.27:0/3153057870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622503fc30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 -- 172.21.15.27:0/3153057870 >> 172.21.15.27:0/3153057870 conn(0x556224c8f300 msgr2=0x55622503db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 -- 172.21.15.27:0/3153057870 shutdown_connections 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.964+0000 7fb94161c740 1 -- 172.21.15.27:0/3153057870 wait complete. 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.965+0000 7fb933400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9180b66b0 0x7fb9180b8bc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55622503e8b0 tx=0x7fb92402a3f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.965+0000 7fb94161c740 1 -- start start 2024-09-21T07:14:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.965+0000 7fb94161c740 1 Processor -- start 2024-09-21T07:14:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb94161c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c722f0 0x55622504b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb94161c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x55622504b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb94161c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c9b1b0 0x55622504bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb94161c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556224edcab0 con 0x556224c9b1b0 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb94161c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556224f4b570 con 0x556224c7a4d0 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb94161c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55622504e3b0 con 0x556224c722f0 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb94161c740 10 client.0 _ll_register_callbacks cb 0x556224f5a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb933e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c9b1b0 0x55622504bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb933e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c9b1b0 0x55622504bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47678/0 (socket says 172.21.15.27:47678) 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb933e00640 1 -- 172.21.15.27:0/1437803973 learned_addr learned my addr 172.21.15.27:0/1437803973 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb92aa00640 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x55622504b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.966+0000 7fb933e00640 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c722f0 0x55622504b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb92aa00640 1 -- 172.21.15.27:0/1437803973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c722f0 msgr2=0x55622504b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb92aa00640 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c722f0 0x55622504b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb92aa00640 1 -- 172.21.15.27:0/1437803973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c9b1b0 msgr2=0x55622504bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb92aa00640 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c9b1b0 0x55622504bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb92aa00640 1 -- 172.21.15.27:0/1437803973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556224d06790 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb92aa00640 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x55622504b7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb93406c640 tx=0x7fb91c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7b97eace7ac08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb921600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb921600640 1 -- 172.21.15.27:0/1437803973 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00f8f0 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb92c003c30 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.967+0000 7fb921600640 1 -- 172.21.15.27:0/1437803973 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb91c00fa90 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb921600640 1 -- 172.21.15.27:0/1437803973 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb91c00fc90 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 10 client.25898 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556224d7f620 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb921600640 1 -- 172.21.15.27:0/1437803973 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92c003c30 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb921600640 1 -- 172.21.15.27:0/1437803973 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb91c01c420 con 0x556224c7a4d0 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 10 client.25898 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 10 client.25898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 20 client.25898 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c59d80 0x55622505d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 4 client.25898 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.968+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556224fb7600 con 0x556224c59d80 2024-09-21T07:14:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.969+0000 7fb933e00640 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c59d80 0x55622505d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.969+0000 7fb921600640 10 client.25898 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.969+0000 7fb933e00640 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c59d80 0x55622505d4a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b0e61318e20384 server_cookie=db5a1b9b042fbd4b in_seq=0 out_seq=0 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.970+0000 7fb921600640 1 -- 172.21.15.27:0/1437803973 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556224fb7600 con 0x556224c59d80 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.970+0000 7fb921600640 10 client.25898 handle_command_reply: tid=0 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.970+0000 7fb94161c740 1 client.25898 shutdown 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.970+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 msgr2=0x55622504b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.970+0000 7fb94161c740 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x55622504b7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb93406c640 tx=0x7fb91c012b60 comp rx=0 tx=0).stop 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 shutdown_connections 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c59d80 0x55622505d4a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c9b1b0 0x55622504bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x55622504b7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 --2- 172.21.15.27:0/1437803973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c722f0 0x55622504b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb921600640 0 client.25898 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 >> 172.21.15.27:0/1437803973 conn(0x556224c8f300 msgr2=0x556225040c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 shutdown_connections 2024-09-21T07:14:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 1 -- 172.21.15.27:0/1437803973 wait complete. 2024-09-21T07:14:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.971+0000 7fb94161c740 20 client.25898 trim_cache size 0 max 16384 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.986+0000 7fb94161c740 1 Processor -- start 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.986+0000 7fb94161c740 1 -- start start 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb94161c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x55622503cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb94161c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225041620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb94161c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622504ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb94161c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b5890 con 0x556224c722f0 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb94161c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b3160 con 0x556224c7a4d0 2024-09-21T07:14:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb94161c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b0fb0 con 0x556224c9b1b0 2024-09-21T07:14:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225041620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225041620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54076/0 (socket says 172.21.15.27:54076) 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 -- 172.21.15.27:0/1012572817 learned_addr learned my addr 172.21.15.27:0/1012572817 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb933e00640 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x55622503cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 -- 172.21.15.27:0/1012572817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 msgr2=0x55622504ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622504ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 -- 172.21.15.27:0/1012572817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 msgr2=0x55622503cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x55622503cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.987+0000 7fb92aa00640 1 -- 172.21.15.27:0/1012572817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556225041f30 con 0x556224c7a4d0 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.988+0000 7fb92aa00640 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225041620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb91c01ca80 tx=0x7fb91c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d6a970b6c2e589 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.988+0000 7fb923e00640 1 -- 172.21.15.27:0/1012572817 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c02f070 con 0x556224c7a4d0 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.988+0000 7fb923e00640 1 -- 172.21.15.27:0/1012572817 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91c00fe30 con 0x556224c7a4d0 2024-09-21T07:14:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.988+0000 7fb923e00640 1 -- 172.21.15.27:0/1012572817 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c01cd00 con 0x556224c7a4d0 2024-09-21T07:14:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 -- 172.21.15.27:0/1012572817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 msgr2=0x556225041620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225041620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb91c01ca80 tx=0x7fb91c005850 comp rx=0 tx=0).stop 2024-09-21T07:14:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 -- 172.21.15.27:0/1012572817 shutdown_connections 2024-09-21T07:14:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x55622504ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225041620 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 --2- 172.21.15.27:0/1012572817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x55622503cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 -- 172.21.15.27:0/1012572817 >> 172.21.15.27:0/1012572817 conn(0x556224c8f300 msgr2=0x55622504c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 -- 172.21.15.27:0/1012572817 shutdown_connections 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.989+0000 7fb94161c740 1 -- 172.21.15.27:0/1012572817 wait complete. 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.990+0000 7fb94161c740 1 -- start start 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.990+0000 7fb94161c740 1 Processor -- start 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.990+0000 7fb94161c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225051710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.990+0000 7fb94161c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225051c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.990+0000 7fb94161c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x556225052190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.990+0000 7fb94161c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b1620 con 0x556224c722f0 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb94161c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b2a90 con 0x556224c7a4d0 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb94161c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b1cf0 con 0x556224c9b1b0 2024-09-21T07:14:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb94161c740 10 client.0 _ll_register_callbacks cb 0x7fb91c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225051710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb92aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225051c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225051710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47690/0 (socket says 172.21.15.27:47690) 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 -- 172.21.15.27:0/121854472 learned_addr learned my addr 172.21.15.27:0/121854472 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x556225052190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 -- 172.21.15.27:0/121854472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 msgr2=0x556225052190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x556225052190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 -- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 msgr2=0x556225051c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225051c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb933e00640 1 -- 172.21.15.27:0/121854472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562250c6f50 con 0x556224c722f0 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.991+0000 7fb92aa00640 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225051c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb933e00640 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225051710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009a70 tx=0x7fb92c0048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44905ff732d86f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb921600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb921600640 1 -- 172.21.15.27:0/121854472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c01e040 con 0x556224c722f0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb91c00fe30 con 0x556224c722f0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb921600640 1 -- 172.21.15.27:0/121854472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb92c004920 con 0x556224c722f0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb921600640 1 -- 172.21.15.27:0/121854472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb92c03d050 con 0x556224c722f0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb94161c740 10 client.15873 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.992+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562250c81e0 con 0x556224c722f0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb921600640 1 -- 172.21.15.27:0/121854472 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91c00fe30 con 0x556224c722f0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb921600640 1 -- 172.21.15.27:0/121854472 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb92c024040 con 0x556224c722f0 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb94161c740 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb94161c740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb94161c740 20 client.15873 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb94161c740 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c2f1c0 0x5562250ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb94161c740 4 client.15873 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:14:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556224fb30b0 con 0x556224c2f1c0 2024-09-21T07:14:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.993+0000 7fb92aa00640 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c2f1c0 0x5562250ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.994+0000 7fb92aa00640 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c2f1c0 0x5562250ca840 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4622bc080c46d4b server_cookie=9f0f425cc9613495 in_seq=0 out_seq=0 2024-09-21T07:14:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:17.994+0000 7fb921600640 10 client.15873 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb921600640 1 -- 172.21.15.27:0/121854472 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556224fb30b0 con 0x556224c2f1c0 2024-09-21T07:14:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb921600640 10 client.15873 handle_command_reply: tid=0 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 client.15873 shutdown 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 msgr2=0x556225051710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225051710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009a70 tx=0x7fb92c0048b0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 shutdown_connections 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556224c2f1c0 0x5562250ca840 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556224c9b1b0 0x556225052190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556224c7a4d0 0x556225051c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 --2- 172.21.15.27:0/121854472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556224c722f0 0x556225051710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 >> 172.21.15.27:0/121854472 conn(0x556224c8f300 msgr2=0x55622504d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb921600640 0 client.15873 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 shutdown_connections 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.000+0000 7fb94161c740 1 -- 172.21.15.27:0/121854472 wait complete. 2024-09-21T07:14:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb94161c740 20 client.15873 trim_cache size 0 max 16384 2024-09-21T07:14:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:14:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:14:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:14:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:14:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9180b66b0 msgr2=0x7fb9180b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9180b66b0 0x7fb9180b8bc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55622503e8b0 tx=0x7fb92402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934103ae0 msgr2=0x7fb934075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934103ae0 0x7fb934075710 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb924005940 tx=0x7fb9240040c0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 shutdown_connections 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb934109d60 0x7fb934076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9180b66b0 0x7fb9180b8bc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb934109410 0x7fb934075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 --2- 172.21.15.27:0/3900546168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb934103ae0 0x7fb934075710 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.001+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 >> 172.21.15.27:0/3900546168 conn(0x7fb9340ff520 msgr2=0x7fb934101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.002+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 shutdown_connections 2024-09-21T07:14:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:18.002+0000 7fb93a400640 1 -- 172.21.15.27:0/3900546168 wait complete. 2024-09-21T07:14:18.068 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:14:18.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:14:18.468 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:18.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:18 smithi027 ceph-mon[30779]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:14:18.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:18 smithi153 ceph-mon[34747]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:14:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:18 smithi119 ceph-mon[34519]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:14:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 -- 172.21.15.27:0/2233157669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1868105d80 msgr2=0x7f1868106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 --2- 172.21.15.27:0/2233157669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1868105d80 0x7f1868106190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1858009860 tx=0x7f185802a340 comp rx=0 tx=0).stop 2024-09-21T07:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 -- 172.21.15.27:0/2233157669 shutdown_connections 2024-09-21T07:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 --2- 172.21.15.27:0/2233157669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186810c010 0x7f1868112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 --2- 172.21.15.27:0/2233157669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f186810b6c0 0x7f186810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 --2- 172.21.15.27:0/2233157669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1868105d80 0x7f1868106190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 -- 172.21.15.27:0/2233157669 >> 172.21.15.27:0/2233157669 conn(0x7f1868101810 msgr2=0x7f1868103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 -- 172.21.15.27:0/2233157669 shutdown_connections 2024-09-21T07:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.341+0000 7f186f000640 1 -- 172.21.15.27:0/2233157669 wait complete. 2024-09-21T07:14:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.342+0000 7f186f000640 1 Processor -- start 2024-09-21T07:14:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.342+0000 7f186f000640 1 -- start start 2024-09-21T07:14:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.342+0000 7f186f000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1868105d80 0x7f186819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f186f000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f186810b6c0 0x7f186819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f186f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186810c010 0x7f18681a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f186f000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1868115cd0 con 0x7f186810b6c0 2024-09-21T07:14:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f186f000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1868115b50 con 0x7f186810c010 2024-09-21T07:14:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f186f000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1868115e50 con 0x7f1868105d80 2024-09-21T07:14:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f1867e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1868105d80 0x7f186819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f1867e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1868105d80 0x7f186819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45530/0 (socket says 172.21.15.27:45530) 2024-09-21T07:14:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f1867e00640 1 -- 172.21.15.27:0/340354263 learned_addr learned my addr 172.21.15.27:0/340354263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f186d000640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186810c010 0x7f18681a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.343+0000 7f1867400640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f186810b6c0 0x7f186819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1867e00640 1 -- 172.21.15.27:0/340354263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186810c010 msgr2=0x7f18681a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1867e00640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186810c010 0x7f18681a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1867e00640 1 -- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f186810b6c0 msgr2=0x7f186819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1867e00640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f186810b6c0 0x7f186819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1867e00640 1 -- 172.21.15.27:0/340354263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18681a2610 con 0x7f1868105d80 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1867400640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f186810b6c0 0x7f186819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1867e00640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1868105d80 0x7f186819c6a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1858035040 tx=0x7f1858032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1864c00640 1 -- 172.21.15.27:0/340354263 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858037040 con 0x7f1868105d80 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18681a28a0 con 0x7f1868105d80 2024-09-21T07:14:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.344+0000 7f1864c00640 1 -- 172.21.15.27:0/340354263 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1858041070 con 0x7f1868105d80 2024-09-21T07:14:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.345+0000 7f1864c00640 1 -- 172.21.15.27:0/340354263 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858003850 con 0x7f1868105d80 2024-09-21T07:14:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.345+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a9060 con 0x7f1868105d80 2024-09-21T07:14:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.346+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1868107000 con 0x7f1868105d80 2024-09-21T07:14:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.351+0000 7f1864c00640 1 -- 172.21.15.27:0/340354263 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f18580039f0 con 0x7f1868105d80 2024-09-21T07:14:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.354+0000 7f1864c00640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f184c0b6680 0x7f184c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.355+0000 7f1864c00640 1 -- 172.21.15.27:0/340354263 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1858107ed0 con 0x7f1868105d80 2024-09-21T07:14:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.355+0000 7f1864c00640 1 -- 172.21.15.27:0/340354263 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1858108330 con 0x7f1868105d80 2024-09-21T07:14:19.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.356+0000 7f1867400640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f184c0b6680 0x7f184c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.358+0000 7f1867400640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f184c0b6680 0x7f184c0b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f186810b1a0 tx=0x7f1854009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:19.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.599+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f186802e640 con 0x7f1868105d80 2024-09-21T07:14:19.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.600+0000 7f1864c00640 1 -- 172.21.15.27:0/340354263 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f18580ce070 con 0x7f1868105d80 2024-09-21T07:14:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:14:19.602 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:14:19.603 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:14:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.603+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f184c0b6680 msgr2=0x7f184c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.603+0000 7f186f000640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f184c0b6680 0x7f184c0b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f186810b1a0 tx=0x7f1854009260 comp rx=0 tx=0).stop 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.603+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1868105d80 msgr2=0x7f186819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.603+0000 7f186f000640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1868105d80 0x7f186819c6a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1858035040 tx=0x7f1858032d50 comp rx=0 tx=0).stop 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.603+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 shutdown_connections 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.603+0000 7f186f000640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f184c0b6680 0x7f184c0b8b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.603+0000 7f186f000640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186810c010 0x7f18681a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.604+0000 7f186f000640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f186810b6c0 0x7f186819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.604+0000 7f186f000640 1 --2- 172.21.15.27:0/340354263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1868105d80 0x7f186819c6a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.604+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 >> 172.21.15.27:0/340354263 conn(0x7f1868101810 msgr2=0x7f1868103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.604+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 shutdown_connections 2024-09-21T07:14:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:19.604+0000 7f186f000640 1 -- 172.21.15.27:0/340354263 wait complete. 2024-09-21T07:14:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:20 smithi027 ceph-mon[30779]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:14:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:20 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/340354263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:20.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:20 smithi153 ceph-mon[34747]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:14:20.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:20 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/340354263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:20 smithi119 ceph-mon[34519]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:14:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:20 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/340354263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:22.066 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T07:14:22.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:14:22.902 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:22 smithi027 ceph-mon[30779]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:22.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:22 smithi153 ceph-mon[34747]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:22 smithi119 ceph-mon[34519]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:23.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:14:23.512 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 -- 172.21.15.27:0/2823934800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f41094a0 msgr2=0x7ff6f41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2823934800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f41094a0 0x7ff6f41098b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009900 tx=0x7ff6e002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 -- 172.21.15.27:0/2823934800 shutdown_connections 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2823934800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6f4109df0 0x7ff6f4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2823934800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f41094a0 0x7ff6f41098b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2823934800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6f4103b60 0x7ff6f4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 -- 172.21.15.27:0/2823934800 >> 172.21.15.27:0/2823934800 conn(0x7ff6f407b850 msgr2=0x7ff6f407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 -- 172.21.15.27:0/2823934800 shutdown_connections 2024-09-21T07:14:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.232+0000 7ff6fc200640 1 -- 172.21.15.27:0/2823934800 wait complete. 2024-09-21T07:14:24.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 Processor -- start 2024-09-21T07:14:24.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 -- start start 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f4103b60 0x7ff6f419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6f41094a0 0x7ff6f419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6f4109df0 0x7ff6f419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4113ab0 con 0x7ff6f4109df0 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4113930 con 0x7ff6f4103b60 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.234+0000 7ff6fc200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4113c30 con 0x7ff6f41094a0 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.235+0000 7ff6f9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f4103b60 0x7ff6f419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.235+0000 7ff6f9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f4103b60 0x7ff6f419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54120/0 (socket says 172.21.15.27:54120) 2024-09-21T07:14:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.235+0000 7ff6f9800640 1 -- 172.21.15.27:0/2362464319 learned_addr learned my addr 172.21.15.27:0/2362464319 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:24.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.235+0000 7ff6fa200640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6f4109df0 0x7ff6f419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:24.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.235+0000 7ff6f8e00640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6f41094a0 0x7ff6f419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f9800640 1 -- 172.21.15.27:0/2362464319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6f41094a0 msgr2=0x7ff6f419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f9800640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6f41094a0 0x7ff6f419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f9800640 1 -- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6f4109df0 msgr2=0x7ff6f419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f9800640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6f4109df0 0x7ff6f419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f9800640 1 -- 172.21.15.27:0/2362464319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f41a0450 con 0x7ff6f4103b60 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f8e00640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6f41094a0 0x7ff6f419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f9800640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f4103b60 0x7ff6f419a4e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e9f0 tx=0x7ff6e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f2000640 1 -- 172.21.15.27:0/2362464319 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800cd80 con 0x7ff6f4103b60 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f2000640 1 -- 172.21.15.27:0/2362464319 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e80108e0 con 0x7ff6f4103b60 2024-09-21T07:14:24.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.236+0000 7ff6f2000640 1 -- 172.21.15.27:0/2362464319 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8018980 con 0x7ff6f4103b60 2024-09-21T07:14:24.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.237+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f41a0740 con 0x7ff6f4103b60 2024-09-21T07:14:24.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.237+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f41a6ea0 con 0x7ff6f4103b60 2024-09-21T07:14:24.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.238+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f4104de0 con 0x7ff6f4103b60 2024-09-21T07:14:24.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.241+0000 7ff6f2000640 1 -- 172.21.15.27:0/2362464319 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff6e8010430 con 0x7ff6f4103b60 2024-09-21T07:14:24.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.242+0000 7ff6f2000640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6dc0b6700 0x7ff6dc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.243+0000 7ff6f2000640 1 -- 172.21.15.27:0/2362464319 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff6e80e8330 con 0x7ff6f4103b60 2024-09-21T07:14:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.243+0000 7ff6f8e00640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6dc0b6700 0x7ff6dc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.244+0000 7ff6f8e00640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6dc0b6700 0x7ff6dc0b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009230 tx=0x7ff6e0000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.244+0000 7ff6f2000640 1 -- 172.21.15.27:0/2362464319 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff6e80ae4d0 con 0x7ff6f4103b60 2024-09-21T07:14:24.304 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:24.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.523+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6f4103f70 con 0x7ff6f4103b60 2024-09-21T07:14:24.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.524+0000 7ff6f2000640 1 -- 172.21.15.27:0/2362464319 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7ff6e80b28b0 con 0x7ff6f4103b60 2024-09-21T07:14:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:14:24.527 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:14:24.527 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6dc0b6700 msgr2=0x7ff6dc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6dc0b6700 0x7ff6dc0b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009230 tx=0x7ff6e0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f4103b60 msgr2=0x7ff6f419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f4103b60 0x7ff6f419a4e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e9f0 tx=0x7ff6e800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 shutdown_connections 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6f41094a0 0x7ff6f419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6dc0b6700 0x7ff6dc0b8c10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6f4109df0 0x7ff6f419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 --2- 172.21.15.27:0/2362464319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6f4103b60 0x7ff6f419a4e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:24.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.526+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 >> 172.21.15.27:0/2362464319 conn(0x7ff6f407b850 msgr2=0x7ff6f41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:24.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.527+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 shutdown_connections 2024-09-21T07:14:24.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:24.527+0000 7ff6fc200640 1 -- 172.21.15.27:0/2362464319 wait complete. 2024-09-21T07:14:24.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:24 smithi027 ceph-mon[30779]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T07:14:24.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:24 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2362464319' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:24.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:24 smithi153 ceph-mon[34747]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T07:14:24.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:24 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2362464319' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:24 smithi119 ceph-mon[34519]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T07:14:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:24 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2362464319' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:25.272 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 55% (6985/12697) Updating files: 56% (7111/12697) Updating files: 57% (7238/12697) Updating files: 58% (7365/12697) Updating files: 59% (7492/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 90% (11455/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T07:14:25.341 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T07:14:25.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T07:14:25.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 -- 172.21.15.27:0/3129677358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa41079f0 msgr2=0x7effa4107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 --2- 172.21.15.27:0/3129677358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa41079f0 0x7effa4107ed0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009c10 tx=0x7eff9c02a6e0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 -- 172.21.15.27:0/3129677358 shutdown_connections 2024-09-21T07:14:25.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 --2- 172.21.15.27:0/3129677358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa41079f0 0x7effa4107ed0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 --2- 172.21.15.27:0/3129677358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa4102f00 0x7effa4107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 --2- 172.21.15.27:0/3129677358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa41025b0 0x7effa41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 -- 172.21.15.27:0/3129677358 >> 172.21.15.27:0/3129677358 conn(0x7effa40fe2c0 msgr2=0x7effa4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.699+0000 7effa9e00640 1 -- 172.21.15.27:0/3129677358 shutdown_connections 2024-09-21T07:14:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.700+0000 7effa9e00640 1 -- 172.21.15.27:0/3129677358 wait complete. 2024-09-21T07:14:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.701+0000 7effa9e00640 1 Processor -- start 2024-09-21T07:14:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.701+0000 7effa9e00640 1 -- start start 2024-09-21T07:14:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.702+0000 7effa9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa41025b0 0x7effa41a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.702+0000 7effa9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa4102f00 0x7effa41a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.702+0000 7effa9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa41079f0 0x7effa41a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.702+0000 7effa9e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7effa4119e90 con 0x7effa41025b0 2024-09-21T07:14:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.702+0000 7effa9e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7effa4119d10 con 0x7effa4102f00 2024-09-21T07:14:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.702+0000 7effa9e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effa411a010 con 0x7effa41079f0 2024-09-21T07:14:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa41025b0 0x7effa41a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa41025b0 0x7effa41a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47730/0 (socket says 172.21.15.27:47730) 2024-09-21T07:14:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa8a00640 1 -- 172.21.15.27:0/314459912 learned_addr learned my addr 172.21.15.27:0/314459912 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa9400640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa41079f0 0x7effa41a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa3e00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa4102f00 0x7effa41a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa9400640 1 -- 172.21.15.27:0/314459912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa4102f00 msgr2=0x7effa41a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa9400640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa4102f00 0x7effa41a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa9400640 1 -- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa41025b0 msgr2=0x7effa41a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa9400640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa41025b0 0x7effa41a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa9400640 1 -- 172.21.15.27:0/314459912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa41a67d0 con 0x7effa41079f0 2024-09-21T07:14:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.703+0000 7effa8a00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa41025b0 0x7effa41a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.704+0000 7effa9400640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa41079f0 0x7effa41a60a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7eff9c034040 tx=0x7eff9c003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.704+0000 7effa1600640 1 -- 172.21.15.27:0/314459912 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c041020 con 0x7effa41079f0 2024-09-21T07:14:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.704+0000 7effa1600640 1 -- 172.21.15.27:0/314459912 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff9c0318f0 con 0x7effa41079f0 2024-09-21T07:14:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.704+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41a6a60 con 0x7effa41079f0 2024-09-21T07:14:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.704+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41ad220 con 0x7effa41079f0 2024-09-21T07:14:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.704+0000 7effa1600640 1 -- 172.21.15.27:0/314459912 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c036040 con 0x7effa41079f0 2024-09-21T07:14:25.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.708+0000 7effa1600640 1 -- 172.21.15.27:0/314459912 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7eff9c044020 con 0x7effa41079f0 2024-09-21T07:14:25.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.708+0000 7effb207a740 1 Processor -- start 2024-09-21T07:14:25.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.708+0000 7effb207a740 1 -- start start 2024-09-21T07:14:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effb207a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f737f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effb207a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effb207a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f7389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effb207a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c73f6aefb0 con 0x55c73f370390 2024-09-21T07:14:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effb207a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c73f69e3e0 con 0x55c73f3681b0 2024-09-21T07:14:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effb207a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c73f6a21f0 con 0x55c73f391070 2024-09-21T07:14:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effa3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f737f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effa8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.709+0000 7effa3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f737f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45564/0 (socket says 172.21.15.27:45564) 2024-09-21T07:14:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.710+0000 7effa3e00640 1 -- 172.21.15.27:0/7342440 learned_addr learned my addr 172.21.15.27:0/7342440 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.710+0000 7effa3e00640 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f7389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.710+0000 7effa3e00640 1 -- 172.21.15.27:0/7342440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 msgr2=0x55c73f7384b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.710+0000 7effa3e00640 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7384b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.710+0000 7effa3e00640 1 -- 172.21.15.27:0/7342440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 msgr2=0x55c73f7389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.710+0000 7effa3e00640 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f7389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.710+0000 7effa3e00640 1 -- 172.21.15.27:0/7342440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73f668cd0 con 0x55c73f391070 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.711+0000 7effa1600640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff8c0b6700 0x7eff8c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.711+0000 7effa3e00640 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f737f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff90009690 tx=0x7eff90009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e84151f28ff03a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.711+0000 7effa8a00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff8c0b6700 0x7eff8c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.711+0000 7effa1600640 1 -- 172.21.15.27:0/314459912 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7eff9c108590 con 0x7effa41079f0 2024-09-21T07:14:25.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.711+0000 7eff95c00640 1 -- 172.21.15.27:0/7342440 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90004050 con 0x55c73f391070 2024-09-21T07:14:25.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.711+0000 7eff95c00640 1 -- 172.21.15.27:0/7342440 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff900054d0 con 0x55c73f391070 2024-09-21T07:14:25.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.711+0000 7eff95c00640 1 -- 172.21.15.27:0/7342440 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90004b90 con 0x55c73f391070 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effb207a740 1 -- 172.21.15.27:0/7342440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 msgr2=0x55c73f737f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effa8a00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff8c0b6700 0x7eff8c0b8c10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7eff980095f0 tx=0x7eff98005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effb207a740 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f737f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff90009690 tx=0x7eff90009620 comp rx=0 tx=0).stop 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effb207a740 1 -- 172.21.15.27:0/7342440 shutdown_connections 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effb207a740 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f7389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effb207a740 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effb207a740 1 --2- 172.21.15.27:0/7342440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f737f70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.712+0000 7effb207a740 1 -- 172.21.15.27:0/7342440 >> 172.21.15.27:0/7342440 conn(0x55c73f385460 msgr2=0x55c73f735e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.713+0000 7effb207a740 1 -- 172.21.15.27:0/7342440 shutdown_connections 2024-09-21T07:14:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.713+0000 7effb207a740 1 -- 172.21.15.27:0/7342440 wait complete. 2024-09-21T07:14:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.713+0000 7effb207a740 1 -- start start 2024-09-21T07:14:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 1 Processor -- start 2024-09-21T07:14:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f743f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f74fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c73f4ff700 con 0x55c73f370390 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c73f639580 con 0x55c73f3681b0 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c73f746750 con 0x55c73f391070 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effb207a740 10 client.0 _ll_register_callbacks cb 0x55c73f649470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.714+0000 7effa3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f743f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f743f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47746/0 (socket says 172.21.15.27:47746) 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa3e00640 1 -- 172.21.15.27:0/399097978 learned_addr learned my addr 172.21.15.27:0/399097978 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54160/0 (socket says 172.21.15.27:54160) 2024-09-21T07:14:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa9400640 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f74fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa8a00640 1 -- 172.21.15.27:0/399097978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 msgr2=0x55c73f74fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa8a00640 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f74fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa8a00640 1 -- 172.21.15.27:0/399097978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 msgr2=0x55c73f743f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa8a00640 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f743f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa8a00640 1 -- 172.21.15.27:0/399097978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73f3fc650 con 0x55c73f3681b0 2024-09-21T07:14:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.715+0000 7effa9400640 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f74fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.716+0000 7effa8a00640 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7439f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff9800fb70 tx=0x7eff98004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58bda3f531b96717 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:25.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.716+0000 7eff8a000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:25.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.716+0000 7eff8a000640 1 -- 172.21.15.27:0/399097978 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9802e060 con 0x55c73f3681b0 2024-09-21T07:14:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.716+0000 7eff8a000640 1 -- 172.21.15.27:0/399097978 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7eff98018030 con 0x55c73f3681b0 2024-09-21T07:14:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.716+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c73f63bb90 con 0x55c73f3681b0 2024-09-21T07:14:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.717+0000 7eff8a000640 1 -- 172.21.15.27:0/399097978 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff98035050 con 0x55c73f3681b0 2024-09-21T07:14:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.717+0000 7effb207a740 10 client.25928 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.717+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c73f656950 con 0x55c73f3681b0 2024-09-21T07:14:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.717+0000 7eff8a000640 1 -- 172.21.15.27:0/399097978 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c73f63bb90 con 0x55c73f3681b0 2024-09-21T07:14:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7eff8a000640 1 -- 172.21.15.27:0/399097978 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff980297c0 con 0x55c73f3681b0 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7effb207a740 10 client.25928 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7effb207a740 10 client.25928 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7effb207a740 20 client.25928 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7effb207a740 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f34f9f0 0x55c73f7559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7effb207a740 4 client.25928 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73f6b0f00 con 0x55c73f34f9f0 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.718+0000 7effa3e00640 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f34f9f0 0x55c73f7559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.719+0000 7effa3e00640 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f34f9f0 0x55c73f7559c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23af219e6ed43992 server_cookie=c83cced57f6e6819 in_seq=0 out_seq=0 2024-09-21T07:14:25.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.719+0000 7eff8a000640 10 client.25928 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:14:25.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7eff8a000640 1 -- 172.21.15.27:0/399097978 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c73f6b0f00 con 0x55c73f34f9f0 2024-09-21T07:14:25.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7eff8a000640 10 client.25928 handle_command_reply: tid=0 2024-09-21T07:14:25.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 client.25928 shutdown 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 msgr2=0x55c73f7439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7439f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff9800fb70 tx=0x7eff98004730 comp rx=0 tx=0).stop 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 shutdown_connections 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f34f9f0 0x55c73f7559c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f391070 0x55c73f74fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f370390 0x55c73f743f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 --2- 172.21.15.27:0/399097978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f7439f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 >> 172.21.15.27:0/399097978 conn(0x55c73f385460 msgr2=0x55c73f73b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.720+0000 7eff8a000640 0 client.25928 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.721+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 shutdown_connections 2024-09-21T07:14:25.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.721+0000 7effb207a740 1 -- 172.21.15.27:0/399097978 wait complete. 2024-09-21T07:14:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.721+0000 7effb207a740 20 client.25928 trim_cache size 0 max 16384 2024-09-21T07:14:25.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 Processor -- start 2024-09-21T07:14:25.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 -- start start 2024-09-21T07:14:25.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f73e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f370390 0x55c73f745420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f74f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b9a30 con 0x55c73f391070 2024-09-21T07:14:25.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b7300 con 0x55c73f3681b0 2024-09-21T07:14:25.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.750+0000 7effb207a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b5150 con 0x55c73f370390 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f74f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f74f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47762/0 (socket says 172.21.15.27:47762) 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f370390 0x55c73f745420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f73e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 -- 172.21.15.27:0/1070458242 learned_addr learned my addr 172.21.15.27:0/1070458242 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 -- 172.21.15.27:0/1070458242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f370390 msgr2=0x55c73f745420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f370390 0x55c73f745420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 -- 172.21.15.27:0/1070458242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 msgr2=0x55c73f73e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f73e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.751+0000 7effa9400640 1 -- 172.21.15.27:0/1070458242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73f7363a0 con 0x55c73f391070 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.752+0000 7effa3e00640 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f370390 0x55c73f745420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.752+0000 7effa8a00640 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f73e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:25.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.752+0000 7effa9400640 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f74f940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0cdd30 tx=0x7eff9c0cd7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc89969368078f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:25.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.752+0000 7eff95200640 1 -- 172.21.15.27:0/1070458242 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0ce730 con 0x55c73f391070 2024-09-21T07:14:25.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.752+0000 7eff95200640 1 -- 172.21.15.27:0/1070458242 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff9c0d2b10 con 0x55c73f391070 2024-09-21T07:14:25.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.752+0000 7eff95200640 1 -- 172.21.15.27:0/1070458242 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c030bc0 con 0x55c73f391070 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.753+0000 7effb207a740 1 -- 172.21.15.27:0/1070458242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 msgr2=0x55c73f74f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.753+0000 7effb207a740 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f74f940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0cdd30 tx=0x7eff9c0cd7a0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.754+0000 7effb207a740 1 -- 172.21.15.27:0/1070458242 shutdown_connections 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.754+0000 7effb207a740 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f370390 0x55c73f745420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.754+0000 7effb207a740 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f74f940 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.754+0000 7effb207a740 1 --2- 172.21.15.27:0/1070458242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f3681b0 0x55c73f73e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.754+0000 7effb207a740 1 -- 172.21.15.27:0/1070458242 >> 172.21.15.27:0/1070458242 conn(0x55c73f385460 msgr2=0x55c73f744d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:25.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.754+0000 7effb207a740 1 -- 172.21.15.27:0/1070458242 shutdown_connections 2024-09-21T07:14:25.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.754+0000 7effb207a740 1 -- 172.21.15.27:0/1070458242 wait complete. 2024-09-21T07:14:25.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.755+0000 7effb207a740 1 -- start start 2024-09-21T07:14:25.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.755+0000 7effb207a740 1 Processor -- start 2024-09-21T07:14:25.757 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:25.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effb207a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f3681b0 0x55c73f74b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effb207a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f370390 0x55c73f74bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effb207a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f746960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effb207a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b57c0 con 0x55c73f391070 2024-09-21T07:14:25.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effb207a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b6c30 con 0x55c73f370390 2024-09-21T07:14:25.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effb207a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b5e90 con 0x55c73f3681b0 2024-09-21T07:14:25.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effb207a740 10 client.0 _ll_register_callbacks cb 0x7eff9800fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:25.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effa8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f3681b0 0x55c73f74b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effa9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f746960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effa8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f3681b0 0x55c73f74b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45590/0 (socket says 172.21.15.27:45590) 2024-09-21T07:14:25.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effa8a00640 1 -- 172.21.15.27:0/663802053 learned_addr learned my addr 172.21.15.27:0/663802053 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:25.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.756+0000 7effa3e00640 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f370390 0x55c73f74bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7effa8a00640 1 -- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f370390 msgr2=0x55c73f74bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7effa8a00640 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f370390 0x55c73f74bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7effa8a00640 1 -- 172.21.15.27:0/663802053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 msgr2=0x55c73f746960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:25.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7effa8a00640 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f746960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7effa8a00640 1 -- 172.21.15.27:0/663802053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73f7442a0 con 0x55c73f3681b0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7effa8a00640 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f3681b0 0x55c73f74b9b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff98000c00 tx=0x7eff98005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2dd39e368f2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7eff8a000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.757+0000 7eff8a000640 1 -- 172.21.15.27:0/663802053 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98004250 con 0x55c73f3681b0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.758+0000 7eff8a000640 1 -- 172.21.15.27:0/663802053 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7eff980043f0 con 0x55c73f3681b0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.758+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff9c0d2b10 con 0x55c73f3681b0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.758+0000 7eff8a000640 1 -- 172.21.15.27:0/663802053 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff98029420 con 0x55c73f3681b0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.758+0000 7effb207a740 10 client.25921 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.758+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c73f7c0830 con 0x55c73f3681b0 2024-09-21T07:14:25.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.758+0000 7eff8a000640 1 -- 172.21.15.27:0/663802053 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff9c0d2b10 con 0x55c73f3681b0 2024-09-21T07:14:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7eff8a000640 1 -- 172.21.15.27:0/663802053 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff980035f0 con 0x55c73f3681b0 2024-09-21T07:14:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7effb207a740 10 client.25921 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7effb207a740 10 client.25921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7effb207a740 20 client.25921 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7effb207a740 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f324da0 0x55c73f7c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7effb207a740 4 client.25921 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c73f6ac9b0 con 0x55c73f324da0 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.759+0000 7effa3e00640 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f324da0 0x55c73f7c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.760+0000 7effa3e00640 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f324da0 0x55c73f7c3010 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78fde2c2e46f19b8 server_cookie=733fd0aff83e2156 in_seq=0 out_seq=0 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.760+0000 7eff8a000640 10 client.25921 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.760+0000 7eff8a000640 1 -- 172.21.15.27:0/663802053 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c73f6ac9b0 con 0x55c73f324da0 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.760+0000 7eff8a000640 10 client.25921 handle_command_reply: tid=0 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.760+0000 7effb207a740 1 client.25921 shutdown 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.760+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f3681b0 msgr2=0x55c73f74b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.760+0000 7effb207a740 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f3681b0 0x55c73f74b9b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff98000c00 tx=0x7eff98005d80 comp rx=0 tx=0).stop 2024-09-21T07:14:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 shutdown_connections 2024-09-21T07:14:25.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c73f324da0 0x55c73f7c3010 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73f391070 0x55c73f746960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73f370390 0x55c73f74bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 --2- 172.21.15.27:0/663802053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73f3681b0 0x55c73f74b9b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7eff8a000640 0 client.25921 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:14:25.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 >> 172.21.15.27:0/663802053 conn(0x55c73f385460 msgr2=0x55c73f734600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:25.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 shutdown_connections 2024-09-21T07:14:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 1 -- 172.21.15.27:0/663802053 wait complete. 2024-09-21T07:14:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.761+0000 7effb207a740 20 client.25921 trim_cache size 0 max 16384 2024-09-21T07:14:25.765 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:14:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff8c0b6700 msgr2=0x7eff8c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff8c0b6700 0x7eff8c0b8c10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7eff980095f0 tx=0x7eff98005f90 comp rx=0 tx=0).stop 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa41079f0 msgr2=0x7effa41a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa41079f0 0x7effa41a60a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7eff9c034040 tx=0x7eff9c003b80 comp rx=0 tx=0).stop 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 shutdown_connections 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effa41079f0 0x7effa41a60a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7eff8c0b6700 0x7eff8c0b8c10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa4102f00 0x7effa41a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 --2- 172.21.15.27:0/314459912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effa41025b0 0x7effa41a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 >> 172.21.15.27:0/314459912 conn(0x7effa40fe2c0 msgr2=0x7effa4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 shutdown_connections 2024-09-21T07:14:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:25.762+0000 7effa9e00640 1 -- 172.21.15.27:0/314459912 wait complete. 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T07:14:26.087 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T07:14:26.088 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T07:14:26.103 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T07:14:26.189 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T07:14:26.194 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T07:14:26.194 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.711+0000 7f18af000640 1 -- 172.21.15.27:0/1577037543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a810c040 msgr2=0x7f18a8112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.711+0000 7f18af000640 1 --2- 172.21.15.27:0/1577037543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a810c040 0x7f18a8112870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009ca0 tx=0x7f18a002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.711+0000 7f18af000640 1 -- 172.21.15.27:0/1577037543 shutdown_connections 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.711+0000 7f18af000640 1 --2- 172.21.15.27:0/1577037543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a810c040 0x7f18a8112870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.711+0000 7f18af000640 1 --2- 172.21.15.27:0/1577037543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a810b6f0 0x7f18a810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.711+0000 7f18af000640 1 --2- 172.21.15.27:0/1577037543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18a8105db0 0x7f18a81061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.711+0000 7f18af000640 1 -- 172.21.15.27:0/1577037543 >> 172.21.15.27:0/1577037543 conn(0x7f18a81017e0 msgr2=0x7f18a8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.712+0000 7f18af000640 1 -- 172.21.15.27:0/1577037543 shutdown_connections 2024-09-21T07:14:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.712+0000 7f18af000640 1 -- 172.21.15.27:0/1577037543 wait complete. 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 Processor -- start 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 -- start start 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8105db0 0x7f18a819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18a810b6f0 0x7f18a819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a810c040 0x7f18a81a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18a8115d50 con 0x7f18a8105db0 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18a8115bd0 con 0x7f18a810c040 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18af000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18a8115ed0 con 0x7f18a810b6f0 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.713+0000 7f18a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8105db0 0x7f18a819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8105db0 0x7f18a819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47788/0 (socket says 172.21.15.27:47788) 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7e00640 1 -- 172.21.15.27:0/2606409617 learned_addr learned my addr 172.21.15.27:0/2606409617 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7400640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18a810b6f0 0x7f18a819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18ad000640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a810c040 0x7f18a81a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7e00640 1 -- 172.21.15.27:0/2606409617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18a810b6f0 msgr2=0x7f18a819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7e00640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18a810b6f0 0x7f18a819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7e00640 1 -- 172.21.15.27:0/2606409617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a810c040 msgr2=0x7f18a81a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7e00640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a810c040 0x7f18a81a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7e00640 1 -- 172.21.15.27:0/2606409617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a81a25c0 con 0x7f18a8105db0 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.714+0000 7f18a7400640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18a810b6f0 0x7f18a819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.715+0000 7f18a7e00640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8105db0 0x7f18a819c650 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f189800e9f0 tx=0x7f189800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.715+0000 7f18a4c00640 1 -- 172.21.15.27:0/2606409617 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189800cd80 con 0x7f18a8105db0 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.715+0000 7f18a4c00640 1 -- 172.21.15.27:0/2606409617 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18980108e0 con 0x7f18a8105db0 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.715+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a81a28b0 con 0x7f18a8105db0 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.715+0000 7f18a4c00640 1 -- 172.21.15.27:0/2606409617 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898018980 con 0x7f18a8105db0 2024-09-21T07:14:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.715+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a81a9010 con 0x7f18a8105db0 2024-09-21T07:14:26.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.719+0000 7f18b5c39740 1 Processor -- start 2024-09-21T07:14:26.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.719+0000 7f18b5c39740 1 -- start start 2024-09-21T07:14:26.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c54350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c54890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb088bb60 0x558fb0c54dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18b5c39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558fb0bc9ca0 con 0x558fb0883980 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18b5c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558fb0bb90d0 con 0x558fb08ac840 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18b5c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558fb0bbcee0 con 0x558fb088bb60 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18a4c00640 1 -- 172.21.15.27:0/2606409617 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1898010430 con 0x7f18a8105db0 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18ad000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c54350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18a7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb088bb60 0x558fb0c54dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18ad000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c54350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54202/0 (socket says 172.21.15.27:54202) 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18ad000640 1 -- 172.21.15.27:0/1332591652 learned_addr learned my addr 172.21.15.27:0/1332591652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.720+0000 7f18a7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb088bb60 0x558fb0c54dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45604/0 (socket says 172.21.15.27:45604) 2024-09-21T07:14:26.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.721+0000 7f18a7e00640 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c54890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.721+0000 7f18ad000640 1 -- 172.21.15.27:0/1332591652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb088bb60 msgr2=0x558fb0c54dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.721+0000 7f18ad000640 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb088bb60 0x558fb0c54dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.721+0000 7f18ad000640 1 -- 172.21.15.27:0/1332591652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 msgr2=0x558fb0c54890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.721+0000 7f18ad000640 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c54890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.721+0000 7f18ad000640 1 -- 172.21.15.27:0/1332591652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb0b6b400 con 0x558fb08ac840 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.721+0000 7f18ad000640 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c54350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009ca0 tx=0x7f18a0009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91505db9a13364b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.722+0000 7f189d600640 1 -- 172.21.15.27:0/1332591652 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0031cf0 con 0x558fb08ac840 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.722+0000 7f189d600640 1 -- 172.21.15.27:0/1332591652 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a0036030 con 0x558fb08ac840 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.722+0000 7f189d600640 1 -- 172.21.15.27:0/1332591652 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00042a0 con 0x558fb08ac840 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.722+0000 7f18a4c00640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f188c0b6700 0x7f188c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18a4c00640 1 -- 172.21.15.27:0/2606409617 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f18980e7d80 con 0x7f18a8105db0 2024-09-21T07:14:26.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 -- 172.21.15.27:0/1332591652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 msgr2=0x558fb0c54350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c54350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009ca0 tx=0x7f18a0009430 comp rx=0 tx=0).stop 2024-09-21T07:14:26.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18a7e00640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f188c0b6700 0x7f188c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 -- 172.21.15.27:0/1332591652 shutdown_connections 2024-09-21T07:14:26.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb088bb60 0x558fb0c54dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c54890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1332591652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c54350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 -- 172.21.15.27:0/1332591652 >> 172.21.15.27:0/1332591652 conn(0x558fb08a0990 msgr2=0x558fb0c52240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 -- 172.21.15.27:0/1332591652 shutdown_connections 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18b5c39740 1 -- 172.21.15.27:0/1332591652 wait complete. 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.723+0000 7f18a7e00640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f188c0b6700 0x7f188c0b8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x558fb0c52fd0 tx=0x7f1898000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 -- start start 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 Processor -- start 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c5ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558fb0a1a7a0 con 0x558fb088bb60 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558fb0b5ea10 con 0x558fb08ac840 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558fb0c62950 con 0x558fb0883980 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.724+0000 7f18b5c39740 10 client.0 _ll_register_callbacks cb 0x558fb0b6ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c5ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c5ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47808/0 (socket says 172.21.15.27:47808) 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 -- 172.21.15.27:0/1296532469 learned_addr learned my addr 172.21.15.27:0/1296532469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18ad000640 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18ad000640 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 -- 172.21.15.27:0/1296532469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 msgr2=0x558fb0c5f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 -- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 msgr2=0x558fb0c604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.725+0000 7f18a7400640 1 -- 172.21.15.27:0/1296532469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb0926b90 con 0x558fb088bb60 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f18a7400640 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c5ff30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1894012d20 tx=0x7f189400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73387492b1fa6847 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:26.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f1892000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:26.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f1892000640 1 -- 172.21.15.27:0/1296532469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894012110 con 0x558fb088bb60 2024-09-21T07:14:26.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f1892000640 1 -- 172.21.15.27:0/1296532469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1894004050 con 0x558fb088bb60 2024-09-21T07:14:26.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a0036030 con 0x558fb088bb60 2024-09-21T07:14:26.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f1892000640 1 -- 172.21.15.27:0/1296532469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1894025050 con 0x558fb088bb60 2024-09-21T07:14:26.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f18b5c39740 10 client.15915 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:26.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.726+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558fb0c6e700 con 0x558fb088bb60 2024-09-21T07:14:26.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f1892000640 1 -- 172.21.15.27:0/1296532469 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a0036030 con 0x558fb088bb60 2024-09-21T07:14:26.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f1892000640 1 -- 172.21.15.27:0/1296532469 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1894010430 con 0x558fb088bb60 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f18b5c39740 10 client.15915 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f18b5c39740 10 client.15915 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f18b5c39740 20 client.15915 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb086b1c0 0x558fb0c71ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f18b5c39740 4 client.15915 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fb0bcbbf0 con 0x558fb086b1c0 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.727+0000 7f18ad000640 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb086b1c0 0x558fb0c71ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.728+0000 7f18ad000640 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb086b1c0 0x558fb0c71ca0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2584b1f3bfaa054 server_cookie=2fb34a2f88ca7475 in_seq=0 out_seq=0 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.728+0000 7f1892000640 10 client.15915 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f1892000640 1 -- 172.21.15.27:0/1296532469 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558fb0bcbbf0 con 0x558fb086b1c0 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f1892000640 10 client.15915 handle_command_reply: tid=0 2024-09-21T07:14:26.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 client.15915 shutdown 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 msgr2=0x558fb0c5ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c5ff30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1894012d20 tx=0x7f189400f790 comp rx=0 tx=0).stop 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 shutdown_connections 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb086b1c0 0x558fb0c71ca0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c5ff30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1296532469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f1892000640 0 client.15915 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.729+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 >> 172.21.15.27:0/1296532469 conn(0x558fb08a0990 msgr2=0x558fb0c574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.730+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 shutdown_connections 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.730+0000 7f18b5c39740 1 -- 172.21.15.27:0/1296532469 wait complete. 2024-09-21T07:14:26.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.730+0000 7f18b5c39740 20 client.15915 trim_cache size 0 max 16384 2024-09-21T07:14:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 Processor -- start 2024-09-21T07:14:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 -- start start 2024-09-21T07:14:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c55a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c5cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18a81b57a0 con 0x558fb088bb60 2024-09-21T07:14:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18a81b3070 con 0x558fb08ac840 2024-09-21T07:14:26.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.748+0000 7f18b5c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18a81b0ec0 con 0x558fb0883980 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c55a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c55a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47818/0 (socket says 172.21.15.27:47818) 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 -- 172.21.15.27:0/1376509315 learned_addr learned my addr 172.21.15.27:0/1376509315 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18ad000640 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18ad000640 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c5cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 -- 172.21.15.27:0/1376509315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 msgr2=0x558fb0c5fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 -- 172.21.15.27:0/1376509315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 msgr2=0x558fb0c5cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c5cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 -- 172.21.15.27:0/1376509315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb0c4dc50 con 0x558fb088bb60 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18ad000640 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c5cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.749+0000 7f18a7400640 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c55a90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739e0326474b5b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:26.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.750+0000 7f189cc00640 1 -- 172.21.15.27:0/1376509315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894000e60 con 0x558fb088bb60 2024-09-21T07:14:26.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.750+0000 7f189cc00640 1 -- 172.21.15.27:0/1376509315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1894005560 con 0x558fb088bb60 2024-09-21T07:14:26.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.750+0000 7f189cc00640 1 -- 172.21.15.27:0/1376509315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894003850 con 0x558fb088bb60 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 -- 172.21.15.27:0/1376509315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 msgr2=0x558fb0c55a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c55a90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894012320 comp rx=0 tx=0).stop 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 -- 172.21.15.27:0/1376509315 shutdown_connections 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb08ac840 0x558fb0c5cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb088bb60 0x558fb0c55a90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 --2- 172.21.15.27:0/1376509315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb0883980 0x558fb0c5fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 -- 172.21.15.27:0/1376509315 >> 172.21.15.27:0/1376509315 conn(0x558fb08a0990 msgr2=0x558fb0c55380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 -- 172.21.15.27:0/1376509315 shutdown_connections 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 -- 172.21.15.27:0/1376509315 wait complete. 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.751+0000 7f18b5c39740 1 -- start start 2024-09-21T07:14:26.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 1 Processor -- start 2024-09-21T07:14:26.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c68370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb088bb60 0x558fb0c64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb08ac840 0x558fb0c64dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18a81b1530 con 0x558fb0883980 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18a81b29a0 con 0x558fb088bb60 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18a81b1c00 con 0x558fb08ac840 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.752+0000 7f18b5c39740 10 client.0 _ll_register_callbacks cb 0x7f1894009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18ad000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c68370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18a7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb088bb60 0x558fb0c64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18ad000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c68370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47824/0 (socket says 172.21.15.27:47824) 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18ad000640 1 -- 172.21.15.27:0/2525016074 learned_addr learned my addr 172.21.15.27:0/2525016074 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18ad000640 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb08ac840 0x558fb0c64dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18a7400640 1 -- 172.21.15.27:0/2525016074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb08ac840 msgr2=0x558fb0c64dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18a7400640 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb08ac840 0x558fb0c64dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18a7400640 1 -- 172.21.15.27:0/2525016074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 msgr2=0x558fb0c68370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18a7400640 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c68370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.753+0000 7f18a7400640 1 -- 172.21.15.27:0/2525016074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb0c654d0 con 0x558fb088bb60 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f18a7400640 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb088bb60 0x558fb0c64880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f189402b040 tx=0x7f1894010a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca6a23b8d2f91cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f1892000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f1892000640 1 -- 172.21.15.27:0/2525016074 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894037060 con 0x558fb088bb60 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1894005560 con 0x558fb088bb60 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f1892000640 1 -- 172.21.15.27:0/2525016074 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f18940052e0 con 0x558fb088bb60 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f1892000640 1 -- 172.21.15.27:0/2525016074 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f189403e050 con 0x558fb088bb60 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f18b5c39740 10 client.25952 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.754+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558fb0cdd510 con 0x558fb088bb60 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f1892000640 1 -- 172.21.15.27:0/2525016074 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1894005560 con 0x558fb088bb60 2024-09-21T07:14:26.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f1892000640 1 -- 172.21.15.27:0/2525016074 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f189401c420 con 0x558fb088bb60 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f18b5c39740 10 client.25952 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f18b5c39740 10 client.25952 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f18b5c39740 20 client.25952 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f18b5c39740 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb0840570 0x558fb0cdfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f18b5c39740 4 client.25952 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x558fb0bc76a0 con 0x558fb0840570 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.755+0000 7f18ad000640 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb0840570 0x558fb0cdfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:26.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.756+0000 7f18ad000640 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb0840570 0x558fb0cdfbd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a37dacd091e7f94 server_cookie=4188e37c7dfbc14f in_seq=0 out_seq=0 2024-09-21T07:14:26.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.756+0000 7f1892000640 10 client.25952 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f1892000640 1 -- 172.21.15.27:0/2525016074 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558fb0bc76a0 con 0x558fb0840570 2024-09-21T07:14:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f1892000640 10 client.25952 handle_command_reply: tid=0 2024-09-21T07:14:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 client.25952 shutdown 2024-09-21T07:14:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb088bb60 msgr2=0x558fb0c64880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb088bb60 0x558fb0c64880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f189402b040 tx=0x7f1894010a70 comp rx=0 tx=0).stop 2024-09-21T07:14:26.766 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:26 smithi027 ceph-mon[30779]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 shutdown_connections 2024-09-21T07:14:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558fb0840570 0x558fb0cdfbd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb08ac840 0x558fb0c64dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fb088bb60 0x558fb0c64880 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 --2- 172.21.15.27:0/2525016074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fb0883980 0x558fb0c68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f1892000640 0 client.25952 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 >> 172.21.15.27:0/2525016074 conn(0x558fb08a0990 msgr2=0x558fb0c55380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.763+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 shutdown_connections 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.764+0000 7f18b5c39740 1 -- 172.21.15.27:0/2525016074 wait complete. 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.764+0000 7f18b5c39740 20 client.25952 trim_cache size 0 max 16384 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 6, 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:14:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "7b608863": { 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 6, 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.0, 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:14:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:14:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:26.769 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:14:26.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f188c0b6700 msgr2=0x7f188c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f188c0b6700 0x7f188c0b8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x558fb0c52fd0 tx=0x7f1898000bc0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8105db0 msgr2=0x7f18a819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8105db0 0x7f18a819c650 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f189800e9f0 tx=0x7f189800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 shutdown_connections 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18a810b6f0 0x7f18a819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f188c0b6700 0x7f188c0b8c10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18a810c040 0x7f18a81a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 --2- 172.21.15.27:0/2606409617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8105db0 0x7f18a819c650 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 >> 172.21.15.27:0/2606409617 conn(0x7f18a81017e0 msgr2=0x7f18a8108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 shutdown_connections 2024-09-21T07:14:26.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:26.765+0000 7f18af000640 1 -- 172.21.15.27:0/2606409617 wait complete. 2024-09-21T07:14:26.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:26 smithi153 ceph-mon[34747]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:26 smithi119 ceph-mon[34519]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:27.421 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:14:28.232 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7b608863', quiesce took 6.2 seconds. Will release after: 10.8 2024-09-21T07:14:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:28.270 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T07:14:28.383 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T07:14:28.451 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T07:14:28.455 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T07:14:28.455 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T07:14:28.525 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T07:14:28.532 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:14:28.532 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T07:14:28.610 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-21T07:14:28.611 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-21T07:14:28.611 DEBUG:teuthology.orchestra.run.smithi027:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-21T07:14:28.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:28 smithi027 ceph-mon[30779]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:28 smithi153 ceph-mon[34747]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 -- 172.21.15.27:0/2437581779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc103990 msgr2=0x7fc8dc103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 --2- 172.21.15.27:0/2437581779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc103990 0x7fc8dc103da0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009900 tx=0x7fc8cc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 -- 172.21.15.27:0/2437581779 shutdown_connections 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 --2- 172.21.15.27:0/2437581779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8dc1092d0 0x7fc8dc1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 --2- 172.21.15.27:0/2437581779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8dc109c20 0x7fc8dc110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 --2- 172.21.15.27:0/2437581779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc103990 0x7fc8dc103da0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 -- 172.21.15.27:0/2437581779 >> 172.21.15.27:0/2437581779 conn(0x7fc8dc0ff3e0 msgr2=0x7fc8dc101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 -- 172.21.15.27:0/2437581779 shutdown_connections 2024-09-21T07:14:28.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.974+0000 7fc8e1400640 1 -- 172.21.15.27:0/2437581779 wait complete. 2024-09-21T07:14:28.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.975+0000 7fc8e1400640 1 Processor -- start 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8e1400640 1 -- start start 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8e1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8dc103990 0x7fc8dc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8e1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc1092d0 0x7fc8dc19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8e1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8dc109c20 0x7fc8dc19fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8e1400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc113930 con 0x7fc8dc1092d0 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8e1400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1137b0 con 0x7fc8dc103990 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8e1400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc113ab0 con 0x7fc8dc109c20 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8db400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc1092d0 0x7fc8dc19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.976+0000 7fc8db400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc1092d0 0x7fc8dc19a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39328/0 (socket says 172.21.15.27:39328) 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8db400640 1 -- 172.21.15.27:0/4091663911 learned_addr learned my addr 172.21.15.27:0/4091663911 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8dc103990 0x7fc8dc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8e0a00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8dc109c20 0x7fc8dc19fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8dbe00640 1 -- 172.21.15.27:0/4091663911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8dc109c20 msgr2=0x7fc8dc19fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8dc109c20 0x7fc8dc19fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8dbe00640 1 -- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc1092d0 msgr2=0x7fc8dc19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc1092d0 0x7fc8dc19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8dbe00640 1 -- 172.21.15.27:0/4091663911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc1a0250 con 0x7fc8dc103990 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8db400640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc1092d0 0x7fc8dc19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.977+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8dc103990 0x7fc8dc19a2e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc02a9e0 tx=0x7fc8cc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.978+0000 7fc8d8c00640 1 -- 172.21.15.27:0/4091663911 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc03f070 con 0x7fc8dc103990 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.978+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc1a04e0 con 0x7fc8dc103990 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.978+0000 7fc8d8c00640 1 -- 172.21.15.27:0/4091663911 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc031b50 con 0x7fc8dc103990 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.978+0000 7fc8d8c00640 1 -- 172.21.15.27:0/4091663911 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc035030 con 0x7fc8dc103990 2024-09-21T07:14:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.978+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc1a6ca0 con 0x7fc8dc103990 2024-09-21T07:14:28.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.981+0000 7fc8e9690740 1 Processor -- start 2024-09-21T07:14:28.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.981+0000 7fc8e9690740 1 -- start start 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.981+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604064b8070 0x56040685ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.981+0000 7fc8d8c00640 1 -- 172.21.15.27:0/4091663911 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc8cc042070 con 0x7fc8dc103990 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56040648f1b0 0x56040685f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040685f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e9690740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5604067d5fb0 con 0x560406497390 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e9690740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5604067c53e0 con 0x5604064b8070 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e9690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5604067c91f0 con 0x56040648f1b0 2024-09-21T07:14:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604064b8070 0x56040685ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8db400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040685f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604064b8070 0x56040685ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56922/0 (socket says 172.21.15.27:56922) 2024-09-21T07:14:28.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8e0a00640 1 -- 172.21.15.27:0/861363498 learned_addr learned my addr 172.21.15.27:0/861363498 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:28.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56040648f1b0 0x56040685f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8db400640 1 -- 172.21.15.27:0/861363498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56040648f1b0 msgr2=0x56040685f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8db400640 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56040648f1b0 0x56040685f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8db400640 1 -- 172.21.15.27:0/861363498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604064b8070 msgr2=0x56040685ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8db400640 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604064b8070 0x56040685ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.982+0000 7fc8db400640 1 -- 172.21.15.27:0/861363498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040678fcd0 con 0x560406497390 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.983+0000 7fc8db400640 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040685f9f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80114c0 tx=0x7fc8c8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4e5d2ff4584593 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.983+0000 7fc8d1200640 1 -- 172.21.15.27:0/861363498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c801e070 con 0x560406497390 2024-09-21T07:14:28.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.983+0000 7fc8d1200640 1 -- 172.21.15.27:0/861363498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c80035f0 con 0x560406497390 2024-09-21T07:14:28.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.983+0000 7fc8d1200640 1 -- 172.21.15.27:0/861363498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c800f7a0 con 0x560406497390 2024-09-21T07:14:28.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.984+0000 7fc8d8c00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8c00b6680 0x7fc8c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.984+0000 7fc8d8c00640 1 -- 172.21.15.27:0/4091663911 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc8cc107fe0 con 0x7fc8dc103990 2024-09-21T07:14:28.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.984+0000 7fc8e0a00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8c00b6680 0x7fc8c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.984+0000 7fc8e9690740 1 -- 172.21.15.27:0/861363498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 msgr2=0x56040685f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.984+0000 7fc8e9690740 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040685f9f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80114c0 tx=0x7fc8c8009f90 comp rx=0 tx=0).stop 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e9690740 1 -- 172.21.15.27:0/861363498 shutdown_connections 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e9690740 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56040648f1b0 0x56040685f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e9690740 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040685f9f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e9690740 1 --2- 172.21.15.27:0/861363498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5604064b8070 0x56040685ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e9690740 1 -- 172.21.15.27:0/861363498 >> 172.21.15.27:0/861363498 conn(0x5604064ac460 msgr2=0x56040685ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:28.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e9690740 1 -- 172.21.15.27:0/861363498 shutdown_connections 2024-09-21T07:14:28.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e0a00640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8c00b6680 0x7fc8c00b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40116b0 tx=0x7fc8d4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:28.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.985+0000 7fc8e9690740 1 -- 172.21.15.27:0/861363498 wait complete. 2024-09-21T07:14:28.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 -- start start 2024-09-21T07:14:28.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 Processor -- start 2024-09-21T07:14:28.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56040648f1b0 0x56040686a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560406497390 0x56040686ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560406626700 con 0x56040648f1b0 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560406760580 con 0x560406497390 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56040686d750 con 0x5604064b8070 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.986+0000 7fc8e9690740 10 client.0 _ll_register_callbacks cb 0x560406770470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560406497390 0x56040686ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56040648f1b0 0x56040686a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560406497390 0x56040686ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56924/0 (socket says 172.21.15.27:56924) 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 -- 172.21.15.27:0/1359105806 learned_addr learned my addr 172.21.15.27:0/1359105806 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8e0a00640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 -- 172.21.15.27:0/1359105806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 msgr2=0x56040687ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 -- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56040648f1b0 msgr2=0x56040686a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:28.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56040648f1b0 0x56040686a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8db400640 1 -- 172.21.15.27:0/1359105806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560406523650 con 0x560406497390 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8e0a00640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.987+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56040648f1b0 0x56040686a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8db400640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560406497390 0x56040686ac10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8012fd0 tx=0x7fc8c80038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=988c818b7eea3cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8c5600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8c5600640 1 -- 172.21.15.27:0/1359105806 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8026020 con 0x560406497390 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560406762b90 con 0x560406497390 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8c5600640 1 -- 172.21.15.27:0/1359105806 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc8c8003a90 con 0x560406497390 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8c5600640 1 -- 172.21.15.27:0/1359105806 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8c8031050 con 0x560406497390 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8e9690740 10 client.25964 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.988+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56040677d950 con 0x560406497390 2024-09-21T07:14:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8c5600640 1 -- 172.21.15.27:0/1359105806 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560406762b90 con 0x560406497390 2024-09-21T07:14:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8c5600640 1 -- 172.21.15.27:0/1359105806 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8c8021030 con 0x560406497390 2024-09-21T07:14:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8e9690740 10 client.25964 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8e9690740 10 client.25964 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:14:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8e9690740 20 client.25964 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8e9690740 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5604064769f0 0x560406880b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8e9690740 4 client.25964 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.989+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604067d7f00 con 0x5604064769f0 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.990+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5604064769f0 0x560406880b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.990+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5604064769f0 0x560406880b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b04ce1964dc55e4 server_cookie=e36a0fff02dbd81e in_seq=0 out_seq=0 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.990+0000 7fc8c5600640 10 client.25964 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8c5600640 1 -- 172.21.15.27:0/1359105806 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604067d7f00 con 0x5604064769f0 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8c5600640 10 client.25964 handle_command_reply: tid=0 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 client.25964 shutdown 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560406497390 msgr2=0x56040686ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560406497390 0x56040686ac10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8012fd0 tx=0x7fc8c80038a0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 shutdown_connections 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5604064769f0 0x560406880b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560406497390 0x56040686ac10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 --2- 172.21.15.27:0/1359105806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56040648f1b0 0x56040686a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8c5600640 0 client.25964 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:14:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 >> 172.21.15.27:0/1359105806 conn(0x5604064ac460 msgr2=0x56040685b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 shutdown_connections 2024-09-21T07:14:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.991+0000 7fc8e9690740 1 -- 172.21.15.27:0/1359105806 wait complete. 2024-09-21T07:14:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:28.992+0000 7fc8e9690740 20 client.25964 trim_cache size 0 max 16384 2024-09-21T07:14:29.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.005+0000 7fc8e9690740 1 Processor -- start 2024-09-21T07:14:29.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e9690740 1 -- start start 2024-09-21T07:14:29.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406860a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040686c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e9690740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1b12b0 con 0x560406497390 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e9690740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1aeb80 con 0x56040648f1b0 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e9690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ac9d0 con 0x5604064b8070 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406860a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8dbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406860a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56928/0 (socket says 172.21.15.27:56928) 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8db400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040686c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8dbe00640 1 -- 172.21.15.27:0/2128382257 learned_addr learned my addr 172.21.15.27:0/2128382257 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.006+0000 7fc8e0a00640 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.007+0000 7fc8dbe00640 1 -- 172.21.15.27:0/2128382257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 msgr2=0x56040687add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.007+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.007+0000 7fc8dbe00640 1 -- 172.21.15.27:0/2128382257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 msgr2=0x56040686c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:29.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.007+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040686c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.007+0000 7fc8dbe00640 1 -- 172.21.15.27:0/2128382257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560406861260 con 0x56040648f1b0 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.007+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406860a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc02a9b0 tx=0x7fc8cc10a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5ba616167e056c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.007+0000 7fc8c7e00640 1 -- 172.21.15.27:0/2128382257 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc049070 con 0x56040648f1b0 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8c7e00640 1 -- 172.21.15.27:0/2128382257 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc031d00 con 0x56040648f1b0 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8c7e00640 1 -- 172.21.15.27:0/2128382257 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0c92c0 con 0x56040648f1b0 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8e9690740 1 -- 172.21.15.27:0/2128382257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 msgr2=0x560406860a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8e9690740 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406860a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc02a9b0 tx=0x7fc8cc10a4e0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8e9690740 1 -- 172.21.15.27:0/2128382257 shutdown_connections 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8e9690740 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604064b8070 0x56040687add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8e9690740 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560406497390 0x56040686c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8e9690740 1 --2- 172.21.15.27:0/2128382257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406860a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.008+0000 7fc8e9690740 1 -- 172.21.15.27:0/2128382257 >> 172.21.15.27:0/2128382257 conn(0x5604064ac460 msgr2=0x560406864080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:29.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.009+0000 7fc8e9690740 1 -- 172.21.15.27:0/2128382257 shutdown_connections 2024-09-21T07:14:29.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.009+0000 7fc8e9690740 1 -- 172.21.15.27:0/2128382257 wait complete. 2024-09-21T07:14:29.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.009+0000 7fc8e9690740 1 -- start start 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.009+0000 7fc8e9690740 1 Processor -- start 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406872d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560406497390 0x5604068732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e9690740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5604064b8070 0x560406873810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e9690740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ad040 con 0x5604064b8070 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e9690740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ae4b0 con 0x56040648f1b0 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e9690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ad710 con 0x560406497390 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e9690740 10 client.0 _ll_register_callbacks cb 0x7fc8d4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5604064b8070 0x560406873810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8db400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560406497390 0x5604068732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5604064b8070 0x560406873810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39384/0 (socket says 172.21.15.27:39384) 2024-09-21T07:14:29.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8e0a00640 1 -- 172.21.15.27:0/3791253018 learned_addr learned my addr 172.21.15.27:0/3791253018 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8db400640 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560406497390 0x5604068732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33004/0 (socket says 172.21.15.27:33004) 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.010+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406872d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8db400640 1 -- 172.21.15.27:0/3791253018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 msgr2=0x560406872d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8db400640 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406872d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8db400640 1 -- 172.21.15.27:0/3791253018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5604064b8070 msgr2=0x560406873810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8db400640 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5604064b8070 0x560406873810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8db400640 1 -- 172.21.15.27:0/3791253018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604068ea360 con 0x560406497390 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8db400640 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560406497390 0x5604068732d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004aa0 tx=0x7fc8c80042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ec638623450d18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8c5600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8c5600640 1 -- 172.21.15.27:0/3791253018 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c803a060 con 0x560406497390 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.011+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5604067d39b0 con 0x560406497390 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8c5600640 1 -- 172.21.15.27:0/3791253018 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc8c8003910 con 0x560406497390 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8c5600640 1 -- 172.21.15.27:0/3791253018 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8c8041050 con 0x560406497390 2024-09-21T07:14:29.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8e9690740 10 client.25957 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:29.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604068eb570 con 0x560406497390 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8c5600640 1 -- 172.21.15.27:0/3791253018 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5604067d39b0 con 0x560406497390 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8c5600640 1 -- 172.21.15.27:0/3791253018 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8c8021040 con 0x560406497390 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8e9690740 10 client.25957 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8e9690740 10 client.25957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8e9690740 20 client.25957 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.012+0000 7fc8e9690740 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x56040644bda0 0x5604068edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.013+0000 7fc8e9690740 4 client.25957 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.013+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604067d0de0 con 0x56040644bda0 2024-09-21T07:14:29.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.013+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x56040644bda0 0x5604068edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.013+0000 7fc8dbe00640 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x56040644bda0 0x5604068edc10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae0d58a6a43b935a server_cookie=d6b7fa89366634fc in_seq=0 out_seq=0 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.013+0000 7fc8c5600640 10 client.25957 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8c5600640 1 -- 172.21.15.27:0/3791253018 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604067d0de0 con 0x56040644bda0 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8c5600640 10 client.25957 handle_command_reply: tid=0 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 client.25957 shutdown 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560406497390 msgr2=0x5604068732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560406497390 0x5604068732d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004aa0 tx=0x7fc8c80042c0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 shutdown_connections 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x56040644bda0 0x5604068edc10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560406497390 0x5604068732d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5604064b8070 0x560406873810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 --2- 172.21.15.27:0/3791253018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56040648f1b0 0x560406872d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 >> 172.21.15.27:0/3791253018 conn(0x5604064ac460 msgr2=0x56040686c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:29.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8c5600640 0 client.25957 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:14:29.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 shutdown_connections 2024-09-21T07:14:29.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 1 -- 172.21.15.27:0/3791253018 wait complete. 2024-09-21T07:14:29.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.014+0000 7fc8e9690740 20 client.25957 trim_cache size 0 max 16384 2024-09-21T07:14:29.019 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:14:29.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8c00b6680 msgr2=0x7fc8c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8c00b6680 0x7fc8c00b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40116b0 tx=0x7fc8d4009500 comp rx=0 tx=0).stop 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8dc103990 msgr2=0x7fc8dc19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8dc103990 0x7fc8dc19a2e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc02a9e0 tx=0x7fc8cc003b90 comp rx=0 tx=0).stop 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 shutdown_connections 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8dc109c20 0x7fc8dc19fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8c00b6680 0x7fc8c00b8b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc1092d0 0x7fc8dc19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 --2- 172.21.15.27:0/4091663911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8dc103990 0x7fc8dc19a2e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 >> 172.21.15.27:0/4091663911 conn(0x7fc8dc0ff3e0 msgr2=0x7fc8dc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 shutdown_connections 2024-09-21T07:14:29.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:29.015+0000 7fc8e1400640 1 -- 172.21.15.27:0/4091663911 wait complete. 2024-09-21T07:14:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:28 smithi119 ceph-mon[34519]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:29.769 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:14:30.159 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.810+0000 7f67a2e00640 1 -- 172.21.15.27:0/775938165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1092f0 msgr2=0x7f679c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.810+0000 7f67a2e00640 1 --2- 172.21.15.27:0/775938165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1092f0 0x7f679c109700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6788009900 tx=0x7f678802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.810+0000 7f67a2e00640 1 -- 172.21.15.27:0/775938165 shutdown_connections 2024-09-21T07:14:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.810+0000 7f67a2e00640 1 --2- 172.21.15.27:0/775938165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f679c109c40 0x7f679c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.810+0000 7f67a2e00640 1 --2- 172.21.15.27:0/775938165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1092f0 0x7f679c109700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.810+0000 7f67a2e00640 1 --2- 172.21.15.27:0/775938165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1039b0 0x7f679c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.811+0000 7f67a2e00640 1 -- 172.21.15.27:0/775938165 >> 172.21.15.27:0/775938165 conn(0x7f679c0ff400 msgr2=0x7f679c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.811+0000 7f67a2e00640 1 -- 172.21.15.27:0/775938165 shutdown_connections 2024-09-21T07:14:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.811+0000 7f67a2e00640 1 -- 172.21.15.27:0/775938165 wait complete. 2024-09-21T07:14:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 Processor -- start 2024-09-21T07:14:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 -- start start 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1039b0 0x7f679c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1092f0 0x7f679c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f679c109c40 0x7f679c1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f679c113900 con 0x7f679c1039b0 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f679c113780 con 0x7f679c1092f0 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.812+0000 7f67a2e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f679c113a80 con 0x7f679c109c40 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1092f0 0x7f679c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1039b0 0x7f679c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1092f0 0x7f679c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56974/0 (socket says 172.21.15.27:56974) 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1000640 1 -- 172.21.15.27:0/1887660894 learned_addr learned my addr 172.21.15.27:0/1887660894 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a2400640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f679c109c40 0x7f679c1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1a00640 1 -- 172.21.15.27:0/1887660894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f679c109c40 msgr2=0x7f679c1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1a00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f679c109c40 0x7f679c1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1a00640 1 -- 172.21.15.27:0/1887660894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1092f0 msgr2=0x7f679c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1a00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1092f0 0x7f679c19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1a00640 1 -- 172.21.15.27:0/1887660894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c1a2440 con 0x7f679c1039b0 2024-09-21T07:14:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.813+0000 7f67a1000640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1092f0 0x7f679c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.814+0000 7f67a1a00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1039b0 0x7f679c19c4d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f679000e9f0 tx=0x7f679000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.814+0000 7f679a000640 1 -- 172.21.15.27:0/1887660894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679000ccc0 con 0x7f679c1039b0 2024-09-21T07:14:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.814+0000 7f679a000640 1 -- 172.21.15.27:0/1887660894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67900108e0 con 0x7f679c1039b0 2024-09-21T07:14:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.814+0000 7f679a000640 1 -- 172.21.15.27:0/1887660894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790018880 con 0x7f679c1039b0 2024-09-21T07:14:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.814+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679c1a2730 con 0x7f679c1039b0 2024-09-21T07:14:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.814+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679c1a8e90 con 0x7f679c1039b0 2024-09-21T07:14:30.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.817+0000 7f67ab218740 1 Processor -- start 2024-09-21T07:14:30.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.817+0000 7f67ab218740 1 -- start start 2024-09-21T07:14:30.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67ab218740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdafcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67ab218740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdafd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67ab218740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd735390 0x55d1bdafd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67ab218740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d1bda73fb0 con 0x55d1bd72d1b0 2024-09-21T07:14:30.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67ab218740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d1bda633e0 con 0x55d1bd756070 2024-09-21T07:14:30.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67ab218740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d1bda671f0 con 0x55d1bd735390 2024-09-21T07:14:30.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f679a000640 1 -- 172.21.15.27:0/1887660894 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6790010430 con 0x7f679c1039b0 2024-09-21T07:14:30.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67a2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdafcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67a1000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd735390 0x55d1bdafd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67a2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdafcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56986/0 (socket says 172.21.15.27:56986) 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67a1000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd735390 0x55d1bdafd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33044/0 (socket says 172.21.15.27:33044) 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67a2400640 1 -- 172.21.15.27:0/1296742841 learned_addr learned my addr 172.21.15.27:0/1296742841 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.818+0000 7f67a1a00640 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdafd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.819+0000 7f67a2400640 1 -- 172.21.15.27:0/1296742841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd735390 msgr2=0x55d1bdafd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.819+0000 7f67a2400640 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd735390 0x55d1bdafd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.819+0000 7f67a2400640 1 -- 172.21.15.27:0/1296742841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 msgr2=0x55d1bdafd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.819+0000 7f67a2400640 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdafd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.819+0000 7f67a2400640 1 -- 172.21.15.27:0/1296742841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bda2dcd0 con 0x55d1bd756070 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.819+0000 7f67a2400640 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdafcf70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6794009fd0 tx=0x7f67940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f4dd1869eb76a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.819+0000 7f678ea00640 1 -- 172.21.15.27:0/1296742841 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794004d20 con 0x55d1bd756070 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.820+0000 7f678ea00640 1 -- 172.21.15.27:0/1296742841 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6794004050 con 0x55d1bd756070 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.820+0000 7f678ea00640 1 -- 172.21.15.27:0/1296742841 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794005280 con 0x55d1bd756070 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.820+0000 7f679a000640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67840b6680 0x7f67840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.820+0000 7f679a000640 1 -- 172.21.15.27:0/1887660894 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f67900e7e00 con 0x7f679c1039b0 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.820+0000 7f67a1a00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67840b6680 0x7f67840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 -- 172.21.15.27:0/1296742841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 msgr2=0x55d1bdafcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdafcf70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6794009fd0 tx=0x7f67940118d0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 -- 172.21.15.27:0/1296742841 shutdown_connections 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd735390 0x55d1bdafd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdafd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 --2- 172.21.15.27:0/1296742841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdafcf70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 -- 172.21.15.27:0/1296742841 >> 172.21.15.27:0/1296742841 conn(0x55d1bd74a460 msgr2=0x55d1bdafae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:30.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 -- 172.21.15.27:0/1296742841 shutdown_connections 2024-09-21T07:14:30.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67ab218740 1 -- 172.21.15.27:0/1296742841 wait complete. 2024-09-21T07:14:30.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.821+0000 7f67a1a00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67840b6680 0x7f67840b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55d1bdafbbf0 tx=0x7f67900eb8b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:30.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 -- start start 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 Processor -- start 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb10e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdb1d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d1bd8c4700 con 0x55d1bd735390 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d1bd9fe580 con 0x55d1bd756070 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d1bdb13e80 con 0x55d1bd72d1b0 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.822+0000 7f67ab218740 10 client.0 _ll_register_callbacks cb 0x55d1bda0e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a2400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb10e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39416/0 (socket says 172.21.15.27:39416) 2024-09-21T07:14:30.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 -- 172.21.15.27:0/3579204776 learned_addr learned my addr 172.21.15.27:0/3579204776 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a2400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb10e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33060/0 (socket says 172.21.15.27:33060) 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a2400640 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdb1d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 -- 172.21.15.27:0/3579204776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd72d1b0 msgr2=0x55d1bdb10e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb10e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 -- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 msgr2=0x55d1bdb1d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdb1d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 -- 172.21.15.27:0/3579204776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bd7c1650 con 0x55d1bd735390 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a2400640 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdb1d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f67a1000640 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb113b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6788006010 tx=0x7f6788031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09edb7c383e97ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.823+0000 7f6783400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:30.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.824+0000 7f6783400640 1 -- 172.21.15.27:0/3579204776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788035040 con 0x55d1bd735390 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.824+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1bda00b90 con 0x55d1bd735390 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.824+0000 7f6783400640 1 -- 172.21.15.27:0/3579204776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f6788004450 con 0x55d1bd735390 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.824+0000 7f6783400640 1 -- 172.21.15.27:0/3579204776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6788050050 con 0x55d1bd735390 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.824+0000 7f67ab218740 10 client.15951 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.824+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1bda1b950 con 0x55d1bd735390 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.824+0000 7f6783400640 1 -- 172.21.15.27:0/3579204776 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d1bda00b90 con 0x55d1bd735390 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f6783400640 1 -- 172.21.15.27:0/3579204776 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6788004c40 con 0x55d1bd735390 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f67ab218740 10 client.15951 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f67ab218740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f67ab218740 20 client.15951 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f67ab218740 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd7149f0 0x55d1bdb22f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f67ab218740 4 client.15951 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:30.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1bda75f00 con 0x55d1bd7149f0 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.825+0000 7f67a2400640 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd7149f0 0x55d1bdb22f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.826+0000 7f67a2400640 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd7149f0 0x55d1bdb22f90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9045ebea7bf190d1 server_cookie=46d4090b79eccf13 in_seq=0 out_seq=0 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.826+0000 7f6783400640 10 client.15951 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f6783400640 1 -- 172.21.15.27:0/3579204776 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1bda75f00 con 0x55d1bd7149f0 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f6783400640 10 client.15951 handle_command_reply: tid=0 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 client.15951 shutdown 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 msgr2=0x55d1bdb113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb113b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6788006010 tx=0x7f6788031740 comp rx=0 tx=0).stop 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 shutdown_connections 2024-09-21T07:14:30.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd7149f0 0x55d1bdb22f90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd756070 0x55d1bdb1d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb113b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 --2- 172.21.15.27:0/3579204776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb10e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 >> 172.21.15.27:0/3579204776 conn(0x55d1bd74a460 msgr2=0x55d1bdaf9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f6783400640 0 client.15951 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 shutdown_connections 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 1 -- 172.21.15.27:0/3579204776 wait complete. 2024-09-21T07:14:30.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.827+0000 7f67ab218740 20 client.15951 trim_cache size 0 max 16384 2024-09-21T07:14:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.841+0000 7f67ab218740 1 Processor -- start 2024-09-21T07:14:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.841+0000 7f67ab218740 1 -- start start 2024-09-21T07:14:30.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67ab218740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67ab218740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd735390 0x55d1bdafe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67ab218740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb1d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67ab218740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f679c1b1340 con 0x55d1bd72d1b0 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67ab218740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f679c1aec10 con 0x55d1bd735390 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67ab218740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f679c1acaa0 con 0x55d1bd756070 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67a2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67a1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd735390 0x55d1bdafe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67a2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb12d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39430/0 (socket says 172.21.15.27:39430) 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67a2400640 1 -- 172.21.15.27:0/4089276735 learned_addr learned my addr 172.21.15.27:0/4089276735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67a2400640 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb1d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.842+0000 7f67a1000640 1 -- 172.21.15.27:0/4089276735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 msgr2=0x55d1bdb1d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f67a1000640 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb1d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f67a1000640 1 -- 172.21.15.27:0/4089276735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 msgr2=0x55d1bdb12d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f67a1000640 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb12d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f67a1000640 1 -- 172.21.15.27:0/4089276735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bdafb7e0 con 0x55d1bd735390 2024-09-21T07:14:30.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f67a2400640 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb12d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:30.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f67a1000640 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd735390 0x55d1bdafe6b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6788003810 tx=0x7f6788004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359eb046b41403d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:30.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f678e000640 1 -- 172.21.15.27:0/4089276735 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788035030 con 0x55d1bd735390 2024-09-21T07:14:30.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f678e000640 1 -- 172.21.15.27:0/4089276735 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67880054f0 con 0x55d1bd735390 2024-09-21T07:14:30.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.843+0000 7f678e000640 1 -- 172.21.15.27:0/4089276735 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788003ea0 con 0x55d1bd735390 2024-09-21T07:14:30.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 -- 172.21.15.27:0/4089276735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd735390 msgr2=0x55d1bdafe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd735390 0x55d1bdafe6b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6788003810 tx=0x7f6788004420 comp rx=0 tx=0).stop 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 -- 172.21.15.27:0/4089276735 shutdown_connections 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb1d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd735390 0x55d1bdafe6b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 --2- 172.21.15.27:0/4089276735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb12d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 -- 172.21.15.27:0/4089276735 >> 172.21.15.27:0/4089276735 conn(0x55d1bd74a460 msgr2=0x55d1bdafdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 -- 172.21.15.27:0/4089276735 shutdown_connections 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.844+0000 7f67ab218740 1 -- 172.21.15.27:0/4089276735 wait complete. 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.845+0000 7f67ab218740 1 -- start start 2024-09-21T07:14:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.845+0000 7f67ab218740 1 Processor -- start 2024-09-21T07:14:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67ab218740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb19320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67ab218740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67ab218740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67ab218740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f679c1ad0f0 con 0x55d1bd735390 2024-09-21T07:14:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67ab218740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f679c1ae540 con 0x55d1bd72d1b0 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67ab218740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f679c1ad7a0 con 0x55d1bd756070 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67ab218740 10 client.0 _ll_register_callbacks cb 0x7f6788005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb19860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39434/0 (socket says 172.21.15.27:39434) 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb19320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 -- 172.21.15.27:0/1776107796 learned_addr learned my addr 172.21.15.27:0/1776107796 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a2400640 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 -- 172.21.15.27:0/1776107796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 msgr2=0x55d1bdb140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 -- 172.21.15.27:0/1776107796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd72d1b0 msgr2=0x55d1bdb19320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb19320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a1000640 1 -- 172.21.15.27:0/1776107796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bdb14640 con 0x55d1bd735390 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.846+0000 7f67a2400640 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f67a1000640 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb19860 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6788000c00 tx=0x7f6788031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93849de181226a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f6783400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f6783400640 1 -- 172.21.15.27:0/1776107796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788063060 con 0x55d1bd735390 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f6783400640 1 -- 172.21.15.27:0/1776107796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f6788031970 con 0x55d1bd735390 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67880054f0 con 0x55d1bd735390 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f6783400640 1 -- 172.21.15.27:0/1776107796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f678806a050 con 0x55d1bd735390 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f67ab218740 10 client.15957 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.847+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1bdb8e130 con 0x55d1bd735390 2024-09-21T07:14:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f6783400640 1 -- 172.21.15.27:0/1776107796 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67880054f0 con 0x55d1bd735390 2024-09-21T07:14:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f6783400640 1 -- 172.21.15.27:0/1776107796 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6788032690 con 0x55d1bd735390 2024-09-21T07:14:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f67ab218740 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f67ab218740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f67ab218740 20 client.15957 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f67ab218740 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd6e9da0 0x55d1bdb90850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f67ab218740 4 client.15957 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:14:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.848+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1bda719b0 con 0x55d1bd6e9da0 2024-09-21T07:14:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.849+0000 7f67a2400640 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd6e9da0 0x55d1bdb90850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.849+0000 7f67a2400640 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd6e9da0 0x55d1bdb90850 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3961a1a30befe046 server_cookie=fdfbe7fe528c21ba in_seq=0 out_seq=0 2024-09-21T07:14:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.849+0000 7f6783400640 10 client.15957 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.849+0000 7f6783400640 1 -- 172.21.15.27:0/1776107796 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1bda719b0 con 0x55d1bd6e9da0 2024-09-21T07:14:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.849+0000 7f6783400640 10 client.15957 handle_command_reply: tid=0 2024-09-21T07:14:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.849+0000 7f67ab218740 1 client.15957 shutdown 2024-09-21T07:14:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 msgr2=0x55d1bdb19860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb19860 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6788000c00 tx=0x7f6788031400 comp rx=0 tx=0).stop 2024-09-21T07:14:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 shutdown_connections 2024-09-21T07:14:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1bd6e9da0 0x55d1bdb90850 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1bd756070 0x55d1bdb140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1bd735390 0x55d1bdb19860 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 --2- 172.21.15.27:0/1776107796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1bd72d1b0 0x55d1bdb19320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 >> 172.21.15.27:0/1776107796 conn(0x55d1bd74a460 msgr2=0x55d1bdafe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f6783400640 0 client.15957 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 shutdown_connections 2024-09-21T07:14:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 1 -- 172.21.15.27:0/1776107796 wait complete. 2024-09-21T07:14:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.850+0000 7f67ab218740 20 client.15957 trim_cache size 0 max 16384 2024-09-21T07:14:30.860 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:14:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67840b6680 msgr2=0x7f67840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67840b6680 0x7f67840b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55d1bdafbbf0 tx=0x7f67900eb8b0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1039b0 msgr2=0x7f679c19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1039b0 0x7f679c19c4d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f679000e9f0 tx=0x7f679000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 shutdown_connections 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f679c109c40 0x7f679c1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67840b6680 0x7f67840b8b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c1092f0 0x7f679c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 --2- 172.21.15.27:0/1887660894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f679c1039b0 0x7f679c19c4d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 >> 172.21.15.27:0/1887660894 conn(0x7f679c0ff400 msgr2=0x7f679c10e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 shutdown_connections 2024-09-21T07:14:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:30.851+0000 7f67a2e00640 1 -- 172.21.15.27:0/1887660894 wait complete. 2024-09-21T07:14:30.876 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:30 smithi027 ceph-mon[30779]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:30.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:30 smithi153 ceph-mon[34747]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:30 smithi119 ceph-mon[34519]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.197+0000 7fc1a5600640 1 -- 172.21.15.27:0/2290674580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 msgr2=0x7fc1a01103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.197+0000 7fc1a5600640 1 --2- 172.21.15.27:0/2290674580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a01103b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc198009ca0 tx=0x7fc19802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.197+0000 7fc1a5600640 1 -- 172.21.15.27:0/2290674580 shutdown_connections 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.197+0000 7fc1a5600640 1 --2- 172.21.15.27:0/2290674580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a01103b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.197+0000 7fc1a5600640 1 --2- 172.21.15.27:0/2290674580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0109230 0x7fc1a0109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.197+0000 7fc1a5600640 1 --2- 172.21.15.27:0/2290674580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0103900 0x7fc1a0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.198+0000 7fc1a5600640 1 -- 172.21.15.27:0/2290674580 >> 172.21.15.27:0/2290674580 conn(0x7fc1a00ff360 msgr2=0x7fc1a01017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.198+0000 7fc1a5600640 1 -- 172.21.15.27:0/2290674580 shutdown_connections 2024-09-21T07:14:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.198+0000 7fc1a5600640 1 -- 172.21.15.27:0/2290674580 wait complete. 2024-09-21T07:14:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 Processor -- start 2024-09-21T07:14:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 -- start start 2024-09-21T07:14:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0103900 0x7fc1a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0109230 0x7fc1a0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0113890 con 0x7fc1a0109b80 2024-09-21T07:14:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0113710 con 0x7fc1a0103900 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a5600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0113a10 con 0x7fc1a0109230 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.199+0000 7fc1a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0109230 0x7fc1a0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0103900 0x7fc1a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc1a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39444/0 (socket says 172.21.15.27:39444) 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0103900 0x7fc1a0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:57030/0 (socket says 172.21.15.27:57030) 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc1a4c00640 1 -- 172.21.15.27:0/1569365136 learned_addr learned my addr 172.21.15.27:0/1569365136 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0109230 0x7fc1a0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33102/0 (socket says 172.21.15.27:33102) 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 -- 172.21.15.27:0/1569365136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0103900 msgr2=0x7fc1a0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0103900 0x7fc1a0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 -- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 msgr2=0x7fc1a0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 -- 172.21.15.27:0/1569365136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a01a8520 con 0x7fc1a0109230 2024-09-21T07:14:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc19fe00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0103900 0x7fc1a0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.200+0000 7fc197e00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0109230 0x7fc1a0075c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1a006c840 tx=0x7fc18800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.201+0000 7fc19d600640 1 -- 172.21.15.27:0/1569365136 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18800cc40 con 0x7fc1a0109230 2024-09-21T07:14:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.201+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a01a8810 con 0x7fc1a0109230 2024-09-21T07:14:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.201+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a01a8db0 con 0x7fc1a0109230 2024-09-21T07:14:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.201+0000 7fc19d600640 1 -- 172.21.15.27:0/1569365136 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18800cde0 con 0x7fc1a0109230 2024-09-21T07:14:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.201+0000 7fc19d600640 1 -- 172.21.15.27:0/1569365136 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188018640 con 0x7fc1a0109230 2024-09-21T07:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 Processor -- start 2024-09-21T07:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 -- start start 2024-09-21T07:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5596a36326b0 con 0x5596a33181b0 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5596a3621ae0 con 0x5596a32ef2f0 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.204+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5596a36258f0 con 0x5596a32f74d0 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:57034/0 (socket says 172.21.15.27:57034) 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 -- 172.21.15.27:0/2066938300 learned_addr learned my addr 172.21.15.27:0/2066938300 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 -- 172.21.15.27:0/2066938300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 msgr2=0x5596a36bd890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36bd890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 -- 172.21.15.27:0/2066938300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 msgr2=0x5596a36bce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36bce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 -- 172.21.15.27:0/2066938300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a35ee780 con 0x5596a32ef2f0 2024-09-21T07:14:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19d600640 1 -- 172.21.15.27:0/1569365136 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc188010430 con 0x7fc1a0109230 2024-09-21T07:14:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.205+0000 7fc19fe00640 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36bd350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc190011520 tx=0x7fc1900119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930cafe6ffb07027 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.206+0000 7fc195200640 1 -- 172.21.15.27:0/2066938300 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190004020 con 0x5596a32ef2f0 2024-09-21T07:14:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.206+0000 7fc195200640 1 -- 172.21.15.27:0/2066938300 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1900041c0 con 0x5596a32ef2f0 2024-09-21T07:14:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.206+0000 7fc195200640 1 -- 172.21.15.27:0/2066938300 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19000f910 con 0x5596a32ef2f0 2024-09-21T07:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2066938300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 msgr2=0x5596a36bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36bd350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc190011520 tx=0x7fc1900119e0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2066938300 shutdown_connections 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36bd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36bd350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2066938300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36bce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2066938300 >> 172.21.15.27:0/2066938300 conn(0x5596a330c300 msgr2=0x5596a36bad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2066938300 shutdown_connections 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2066938300 wait complete. 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.207+0000 7fc19d600640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1840b65c0 0x7fc1840b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.208+0000 7fc19fe00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1840b65c0 0x7fc1840b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.208+0000 7fc19d600640 1 -- 172.21.15.27:0/1569365136 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc1880e7ba0 con 0x7fc1a0109230 2024-09-21T07:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.208+0000 7fc1adc3e740 1 -- start start 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.208+0000 7fc1adc3e740 1 Processor -- start 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.208+0000 7fc19fe00640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1840b65c0 0x7fc1840b8ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc190004190 tx=0x7fc190005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a32f74d0 0x5596a36c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a33181b0 0x5596a36c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5596a3559ab0 con 0x5596a32f74d0 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5596a35c8570 con 0x5596a32ef2f0 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5596a36cb590 con 0x5596a33181b0 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1adc3e740 10 client.0 _ll_register_callbacks cb 0x5596a35d7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc197e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a32f74d0 0x5596a36c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc197e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a32f74d0 0x5596a36c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39462/0 (socket says 172.21.15.27:39462) 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc197e00640 1 -- 172.21.15.27:0/3854033959 learned_addr learned my addr 172.21.15.27:0/3854033959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc19fe00640 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.209+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a33181b0 0x5596a36c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc197e00640 1 -- 172.21.15.27:0/3854033959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a33181b0 msgr2=0x5596a36c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc197e00640 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a33181b0 0x5596a36c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc197e00640 1 -- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 msgr2=0x5596a36c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc197e00640 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc197e00640 1 -- 172.21.15.27:0/3854033959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a3383790 con 0x5596a32f74d0 2024-09-21T07:14:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc197e00640 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a32f74d0 0x5596a36c8990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc1880aa960 tx=0x7fc1880ace60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8fe41f4f85aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc18d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc18d600640 1 -- 172.21.15.27:0/3854033959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1880b7060 con 0x5596a32f74d0 2024-09-21T07:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc18d600640 1 -- 172.21.15.27:0/3854033959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc1880a9040 con 0x5596a32f74d0 2024-09-21T07:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1900041c0 con 0x5596a32f74d0 2024-09-21T07:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc18d600640 1 -- 172.21.15.27:0/3854033959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1880a9240 con 0x5596a32f74d0 2024-09-21T07:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc1adc3e740 10 client.15969 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.210+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596a33fc620 con 0x5596a32f74d0 2024-09-21T07:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc18d600640 1 -- 172.21.15.27:0/3854033959 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1900041c0 con 0x5596a32f74d0 2024-09-21T07:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc18d600640 1 -- 172.21.15.27:0/3854033959 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc1880b65a0 con 0x5596a32f74d0 2024-09-21T07:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc1adc3e740 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc1adc3e740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc1adc3e740 20 client.15969 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32d6d80 0x5596a36da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc1adc3e740 4 client.15969 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596a3634600 con 0x5596a32d6d80 2024-09-21T07:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.211+0000 7fc19fe00640 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32d6d80 0x5596a36da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.212+0000 7fc19fe00640 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32d6d80 0x5596a36da5d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1568c69e4decdcee server_cookie=252fbe862e4c4e2d in_seq=0 out_seq=0 2024-09-21T07:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.212+0000 7fc18d600640 10 client.15969 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc18d600640 1 -- 172.21.15.27:0/3854033959 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596a3634600 con 0x5596a32d6d80 2024-09-21T07:14:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc18d600640 10 client.15969 handle_command_reply: tid=0 2024-09-21T07:14:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 client.15969 shutdown 2024-09-21T07:14:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a32f74d0 msgr2=0x5596a36c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a32f74d0 0x5596a36c8990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc1880aa960 tx=0x7fc1880ace60 comp rx=0 tx=0).stop 2024-09-21T07:14:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 shutdown_connections 2024-09-21T07:14:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32d6d80 0x5596a36da5d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a33181b0 0x5596a36c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a32f74d0 0x5596a36c8990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/3854033959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 >> 172.21.15.27:0/3854033959 conn(0x5596a330c300 msgr2=0x5596a36bde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc18d600640 0 client.15969 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 shutdown_connections 2024-09-21T07:14:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.213+0000 7fc1adc3e740 1 -- 172.21.15.27:0/3854033959 wait complete. 2024-09-21T07:14:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.214+0000 7fc1adc3e740 20 client.15969 trim_cache size 0 max 16384 2024-09-21T07:14:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 Processor -- start 2024-09-21T07:14:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 -- start start 2024-09-21T07:14:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32ef2f0 0x5596a36b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32f74d0 0x5596a36c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c0126b0 con 0x5596a33181b0 2024-09-21T07:14:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c00ff80 con 0x5596a32f74d0 2024-09-21T07:14:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.229+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c00ddd0 con 0x5596a32ef2f0 2024-09-21T07:14:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32ef2f0 0x5596a36b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39464/0 (socket says 172.21.15.27:39464) 2024-09-21T07:14:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 -- 172.21.15.27:0/158867658 learned_addr learned my addr 172.21.15.27:0/158867658 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc197e00640 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32f74d0 0x5596a36c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 -- 172.21.15.27:0/158867658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32ef2f0 msgr2=0x5596a36b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32ef2f0 0x5596a36b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 -- 172.21.15.27:0/158867658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32f74d0 msgr2=0x5596a36c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32f74d0 0x5596a36c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 -- 172.21.15.27:0/158867658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a36d4810 con 0x5596a33181b0 2024-09-21T07:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc197e00640 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32f74d0 0x5596a36c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.230+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36c5b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc1980308e0 tx=0x7fc198005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3c49e0cd3ce446 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.231+0000 7fc18fe00640 1 -- 172.21.15.27:0/158867658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198030c80 con 0x5596a33181b0 2024-09-21T07:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.231+0000 7fc18fe00640 1 -- 172.21.15.27:0/158867658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19804b070 con 0x5596a33181b0 2024-09-21T07:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.231+0000 7fc18fe00640 1 -- 172.21.15.27:0/158867658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198031440 con 0x5596a33181b0 2024-09-21T07:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.231+0000 7fc1adc3e740 1 -- 172.21.15.27:0/158867658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 msgr2=0x5596a36c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.231+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36c5b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc1980308e0 tx=0x7fc198005d80 comp rx=0 tx=0).stop 2024-09-21T07:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 -- 172.21.15.27:0/158867658 shutdown_connections 2024-09-21T07:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36c5b80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32f74d0 0x5596a36c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/158867658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32ef2f0 0x5596a36b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 -- 172.21.15.27:0/158867658 >> 172.21.15.27:0/158867658 conn(0x5596a330c300 msgr2=0x5596a36b9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 -- 172.21.15.27:0/158867658 shutdown_connections 2024-09-21T07:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 -- 172.21.15.27:0/158867658 wait complete. 2024-09-21T07:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 -- start start 2024-09-21T07:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.232+0000 7fc1adc3e740 1 Processor -- start 2024-09-21T07:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1adc3e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c00e440 con 0x5596a33181b0 2024-09-21T07:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c00f8b0 con 0x5596a32ef2f0 2024-09-21T07:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1adc3e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c00eb10 con 0x5596a32f74d0 2024-09-21T07:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1adc3e740 10 client.0 _ll_register_callbacks cb 0x7fc1880056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39468/0 (socket says 172.21.15.27:39468) 2024-09-21T07:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc1a4c00640 1 -- 172.21.15.27:0/2333436748 learned_addr learned my addr 172.21.15.27:0/2333436748 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc19fe00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.233+0000 7fc197e00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc1a4c00640 1 -- 172.21.15.27:0/2333436748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 msgr2=0x5596a36ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc1a4c00640 1 -- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 msgr2=0x5596a36d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc1a4c00640 1 -- 172.21.15.27:0/2333436748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a36cf130 con 0x5596a33181b0 2024-09-21T07:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc19fe00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc197e00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc1a4c00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36ce9f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc198009bf0 tx=0x7fc198009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c135af00536c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc18d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc18d600640 1 -- 172.21.15.27:0/2333436748 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198003d80 con 0x5596a33181b0 2024-09-21T07:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc18d600640 1 -- 172.21.15.27:0/2333436748 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc198003f20 con 0x5596a33181b0 2024-09-21T07:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.234+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19804b070 con 0x5596a33181b0 2024-09-21T07:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc18d600640 1 -- 172.21.15.27:0/2333436748 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc198004160 con 0x5596a33181b0 2024-09-21T07:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596a3744ed0 con 0x5596a33181b0 2024-09-21T07:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc18d600640 1 -- 172.21.15.27:0/2333436748 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19804b070 con 0x5596a33181b0 2024-09-21T07:14:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc18d600640 1 -- 172.21.15.27:0/2333436748 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc19803c5a0 con 0x5596a33181b0 2024-09-21T07:14:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 20 client.15975 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32ac1c0 0x5596a3747630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 4 client.15975 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:14:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.235+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596a36300b0 con 0x5596a32ac1c0 2024-09-21T07:14:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.236+0000 7fc19fe00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32ac1c0 0x5596a3747630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.236+0000 7fc19fe00640 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32ac1c0 0x5596a3747630 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585b3eddbb8d2767 server_cookie=48155fcbc63423dc in_seq=0 out_seq=0 2024-09-21T07:14:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.236+0000 7fc18d600640 10 client.15975 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc18d600640 1 -- 172.21.15.27:0/2333436748 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596a36300b0 con 0x5596a32ac1c0 2024-09-21T07:14:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc18d600640 10 client.15975 handle_command_reply: tid=0 2024-09-21T07:14:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 client.15975 shutdown 2024-09-21T07:14:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 msgr2=0x5596a36ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36ce9f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc198009bf0 tx=0x7fc198009650 comp rx=0 tx=0).stop 2024-09-21T07:14:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 shutdown_connections 2024-09-21T07:14:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5596a32ac1c0 0x5596a3747630 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5596a32f74d0 0x5596a36ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596a33181b0 0x5596a36ce9f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc18d600640 0 client.15975 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 --2- 172.21.15.27:0/2333436748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596a32ef2f0 0x5596a36d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 >> 172.21.15.27:0/2333436748 conn(0x5596a330c300 msgr2=0x5596a36c9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 shutdown_connections 2024-09-21T07:14:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 1 -- 172.21.15.27:0/2333436748 wait complete. 2024-09-21T07:14:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1adc3e740 20 client.15975 trim_cache size 0 max 16384 2024-09-21T07:14:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:14:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:14:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:14:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:14:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1840b65c0 msgr2=0x7fc1840b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1840b65c0 0x7fc1840b8ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc190004190 tx=0x7fc190005850 comp rx=0 tx=0).stop 2024-09-21T07:14:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0109230 msgr2=0x7fc1a0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0109230 0x7fc1a0075c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1a006c840 tx=0x7fc18800ee50 comp rx=0 tx=0).stop 2024-09-21T07:14:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 shutdown_connections 2024-09-21T07:14:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0109230 0x7fc1a0075c50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1840b65c0 0x7fc1840b8ad0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a0109b80 0x7fc1a0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 --2- 172.21.15.27:0/1569365136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1a0103900 0x7fc1a0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 >> 172.21.15.27:0/1569365136 conn(0x7fc1a00ff360 msgr2=0x7fc1a01067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 shutdown_connections 2024-09-21T07:14:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:31.239+0000 7fc1a5600640 1 -- 172.21.15.27:0/1569365136 wait complete. 2024-09-21T07:14:32.647 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:14:32.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:14:32.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:32 smithi153 ceph-mon[34747]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T07:14:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:32 smithi119 ceph-mon[34519]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T07:14:33.043 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:32 smithi027 ceph-mon[30779]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T07:14:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 -- 172.21.15.27:0/2877870268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 msgr2=0x7f3748116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 --2- 172.21.15.27:0/2877870268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 0x7f3748116900 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3740009ca0 tx=0x7f374002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 -- 172.21.15.27:0/2877870268 shutdown_connections 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 --2- 172.21.15.27:0/2877870268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3748075dd0 0x7f37480761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 --2- 172.21.15.27:0/2877870268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 0x7f3748116900 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 --2- 172.21.15.27:0/2877870268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 0x7f37480757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 -- 172.21.15.27:0/2877870268 >> 172.21.15.27:0/2877870268 conn(0x7f374807b850 msgr2=0x7f374807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 -- 172.21.15.27:0/2877870268 shutdown_connections 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.708+0000 7f374f400640 1 -- 172.21.15.27:0/2877870268 wait complete. 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.709+0000 7f374f400640 1 Processor -- start 2024-09-21T07:14:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.710+0000 7f374f400640 1 -- start start 2024-09-21T07:14:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.710+0000 7f374f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3748075dd0 0x7f37481a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.710+0000 7f374f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 0x7f37481a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.710+0000 7f374f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 0x7f37481a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.710+0000 7f374f400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f374811a0d0 con 0x7f3748114430 2024-09-21T07:14:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.710+0000 7f374f400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3748119f50 con 0x7f3748077380 2024-09-21T07:14:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.710+0000 7f374f400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f374811a250 con 0x7f3748075dd0 2024-09-21T07:14:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 0x7f37481a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f374ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3748075dd0 0x7f37481a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f374d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 0x7f37481a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 0x7f37481a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:57094/0 (socket says 172.21.15.27:57094) 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 -- 172.21.15.27:0/43771644 learned_addr learned my addr 172.21.15.27:0/43771644 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 -- 172.21.15.27:0/43771644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3748075dd0 msgr2=0x7f37481a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3748075dd0 0x7f37481a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 -- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 msgr2=0x7f37481a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 0x7f37481a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 -- 172.21.15.27:0/43771644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37481a6970 con 0x7f3748077380 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.711+0000 7f373fe00640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 0x7f37481a0fa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f373000e9f0 tx=0x7f373000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.712+0000 7f3746000640 1 -- 172.21.15.27:0/43771644 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373000ccc0 con 0x7f3748077380 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.712+0000 7f3746000640 1 -- 172.21.15.27:0/43771644 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37300054d0 con 0x7f3748077380 2024-09-21T07:14:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.712+0000 7f3746000640 1 -- 172.21.15.27:0/43771644 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730010430 con 0x7f3748077380 2024-09-21T07:14:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.712+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37481a6c60 con 0x7f3748077380 2024-09-21T07:14:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.712+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37481ad3c0 con 0x7f3748077380 2024-09-21T07:14:33.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 Processor -- start 2024-09-21T07:14:33.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 -- start start 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666808f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666808f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556668004e90 con 0x556667cbe490 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556667ff42c0 con 0x556667cc6670 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.715+0000 7f3755ff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556667ff80d0 con 0x556667ce7350 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f3746000640 1 -- 172.21.15.27:0/43771644 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f37300046c0 con 0x7f3748077380 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666808f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33130/0 (socket says 172.21.15.27:33130) 2024-09-21T07:14:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 -- 172.21.15.27:0/1522325090 learned_addr learned my addr 172.21.15.27:0/1522325090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f373fe00640 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666808f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 -- 172.21.15.27:0/1522325090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 msgr2=0x55666808f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666808f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 -- 172.21.15.27:0/1522325090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 msgr2=0x55666808f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666808f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.716+0000 7f374d400640 1 -- 172.21.15.27:0/1522325090 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556667d52930 con 0x556667ce7350 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.717+0000 7f374d400640 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ed10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3740009ca0 tx=0x7f3740009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95251bcc99f61324 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.717+0000 7f373dc00640 1 -- 172.21.15.27:0/1522325090 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740031cf0 con 0x556667ce7350 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.717+0000 7f373dc00640 1 -- 172.21.15.27:0/1522325090 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3740036030 con 0x556667ce7350 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.717+0000 7f373dc00640 1 -- 172.21.15.27:0/1522325090 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740004240 con 0x556667ce7350 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3746000640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f372c0b6780 0x7f372c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 -- 172.21.15.27:0/1522325090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 msgr2=0x55666808ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ed10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3740009ca0 tx=0x7f3740009430 comp rx=0 tx=0).stop 2024-09-21T07:14:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3746000640 1 -- 172.21.15.27:0/43771644 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f37300e7db0 con 0x7f3748077380 2024-09-21T07:14:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 -- 172.21.15.27:0/1522325090 shutdown_connections 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666808f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666808f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 --2- 172.21.15.27:0/1522325090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ed10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 -- 172.21.15.27:0/1522325090 >> 172.21.15.27:0/1522325090 conn(0x556667cdb740 msgr2=0x55666808cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f374ca00640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f372c0b6780 0x7f372c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.718+0000 7f3755ff8740 1 -- 172.21.15.27:0/1522325090 shutdown_connections 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.719+0000 7f3755ff8740 1 -- 172.21.15.27:0/1522325090 wait complete. 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.719+0000 7f374ca00640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f372c0b6780 0x7f372c0b8c90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55666808d990 tx=0x7f3738005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.719+0000 7f3755ff8740 1 -- start start 2024-09-21T07:14:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.719+0000 7f3755ff8740 1 Processor -- start 2024-09-21T07:14:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cbe490 0x55666809a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666809ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667ce7350 0x5566680a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f3755ff8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556667e55890 con 0x556667ce7350 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f3755ff8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556667f8f160 con 0x556667cc6670 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f3755ff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55666809d310 con 0x556667cbe490 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f3755ff8740 10 client.0 _ll_register_callbacks cb 0x556667faba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f374ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cbe490 0x55666809a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f374d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667ce7350 0x5566680a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f374d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667ce7350 0x5566680a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39512/0 (socket says 172.21.15.27:39512) 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f374d400640 1 -- 172.21.15.27:0/204531225 learned_addr learned my addr 172.21.15.27:0/204531225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.720+0000 7f373fe00640 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666809ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.721+0000 7f374d400640 1 -- 172.21.15.27:0/204531225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cbe490 msgr2=0x55666809a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.721+0000 7f374d400640 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cbe490 0x55666809a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.721+0000 7f374d400640 1 -- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 msgr2=0x55666809ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.721+0000 7f374d400640 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666809ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.721+0000 7f374d400640 1 -- 172.21.15.27:0/204531225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566680a6ba0 con 0x556667ce7350 2024-09-21T07:14:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.721+0000 7f374d400640 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667ce7350 0x5566680a6630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f374002ade0 tx=0x7f37400094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a634f53a24ce4871 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.721+0000 7f3736000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3736000640 1 -- 172.21.15.27:0/204531225 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374004a040 con 0x556667ce7350 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3740036030 con 0x556667ce7350 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3736000640 1 -- 172.21.15.27:0/204531225 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3740031650 con 0x556667ce7350 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3736000640 1 -- 172.21.15.27:0/204531225 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3740003970 con 0x556667ce7350 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3755ff8740 10 client.15987 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566680a94d0 con 0x556667ce7350 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3736000640 1 -- 172.21.15.27:0/204531225 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3740036030 con 0x556667ce7350 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.722+0000 7f3736000640 1 -- 172.21.15.27:0/204531225 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f374004d030 con 0x556667ce7350 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.723+0000 7f3755ff8740 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.723+0000 7f3755ff8740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.723+0000 7f3755ff8740 20 client.15987 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.723+0000 7f3755ff8740 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667ca5cd0 0x5566680ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.723+0000 7f3755ff8740 4 client.15987 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.723+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556668006de0 con 0x556667ca5cd0 2024-09-21T07:14:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.723+0000 7f374ca00640 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667ca5cd0 0x5566680ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.724+0000 7f374ca00640 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667ca5cd0 0x5566680ac830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e496fcac56f4ca54 server_cookie=f8da3eea1d4c00af in_seq=0 out_seq=0 2024-09-21T07:14:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.724+0000 7f3736000640 10 client.15987 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3736000640 1 -- 172.21.15.27:0/204531225 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556668006de0 con 0x556667ca5cd0 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3736000640 10 client.15987 handle_command_reply: tid=0 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 client.15987 shutdown 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667ce7350 msgr2=0x5566680a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667ce7350 0x5566680a6630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f374002ade0 tx=0x7f37400094a0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 shutdown_connections 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667ca5cd0 0x5566680ac830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667ce7350 0x5566680a6630 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x55666809ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 --2- 172.21.15.27:0/204531225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cbe490 0x55666809a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 >> 172.21.15.27:0/204531225 conn(0x556667cdb740 msgr2=0x55666808b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3736000640 0 client.15987 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 shutdown_connections 2024-09-21T07:14:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.725+0000 7f3755ff8740 1 -- 172.21.15.27:0/204531225 wait complete. 2024-09-21T07:14:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.726+0000 7f3755ff8740 20 client.15987 trim_cache size 0 max 16384 2024-09-21T07:14:33.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 Processor -- start 2024-09-21T07:14:33.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 -- start start 2024-09-21T07:14:33.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x556668094e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x556668099f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37481b9b40 con 0x556667cbe490 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37481b7410 con 0x556667cc6670 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.744+0000 7f3755ff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37481b5260 con 0x556667ce7350 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x556668094e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x556668094e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39518/0 (socket says 172.21.15.27:39518) 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 -- 172.21.15.27:0/3526688550 learned_addr learned my addr 172.21.15.27:0/3526688550 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f373fe00640 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x556668099f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374d400640 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 -- 172.21.15.27:0/3526688550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 msgr2=0x55666808ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 -- 172.21.15.27:0/3526688550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 msgr2=0x556668099f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x556668099f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f374ca00640 1 -- 172.21.15.27:0/3526688550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556668095f50 con 0x556667cbe490 2024-09-21T07:14:33.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.745+0000 7f373fe00640 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x556668099f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:33.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.746+0000 7f374ca00640 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x556668094e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5566680ace30 tx=0x7f373801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315ad6bbc024fc45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:33.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.746+0000 7f373d200640 1 -- 172.21.15.27:0/3526688550 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373802b470 con 0x556667cbe490 2024-09-21T07:14:33.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.746+0000 7f373d200640 1 -- 172.21.15.27:0/3526688550 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373802bac0 con 0x556667cbe490 2024-09-21T07:14:33.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.746+0000 7f373d200640 1 -- 172.21.15.27:0/3526688550 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373801fba0 con 0x556667cbe490 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 -- 172.21.15.27:0/3526688550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 msgr2=0x556668094e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x556668094e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5566680ace30 tx=0x7f373801b000 comp rx=0 tx=0).stop 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 -- 172.21.15.27:0/3526688550 shutdown_connections 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667ce7350 0x55666808ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667cc6670 0x556668099f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3526688550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x556668094e30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 -- 172.21.15.27:0/3526688550 >> 172.21.15.27:0/3526688550 conn(0x556667cdb740 msgr2=0x55666808c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 -- 172.21.15.27:0/3526688550 shutdown_connections 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 -- 172.21.15.27:0/3526688550 wait complete. 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.747+0000 7f3755ff8740 1 -- start start 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 1 Processor -- start 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666809e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cc6670 0x55666809eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667ce7350 0x556668116090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37481b58d0 con 0x556667cbe490 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37481b6d40 con 0x556667ce7350 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37481b5fa0 con 0x556667cc6670 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f3755ff8740 10 client.0 _ll_register_callbacks cb 0x7f37380097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f374ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666809e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f374ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666809e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39528/0 (socket says 172.21.15.27:39528) 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f374ca00640 1 -- 172.21.15.27:0/3849635035 learned_addr learned my addr 172.21.15.27:0/3849635035 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.748+0000 7f374d400640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667ce7350 0x556668116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f373fe00640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cc6670 0x55666809eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f374d400640 1 -- 172.21.15.27:0/3849635035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cc6670 msgr2=0x55666809eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f374d400640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cc6670 0x55666809eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f374d400640 1 -- 172.21.15.27:0/3849635035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 msgr2=0x55666809e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f374d400640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666809e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f374d400640 1 -- 172.21.15.27:0/3849635035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566681165d0 con 0x556667ce7350 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f373fe00640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cc6670 0x55666809eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f374ca00640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666809e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f374d400640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667ce7350 0x556668116090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37400309a0 tx=0x7f3740030cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0ee0a64d42841d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.749+0000 7f3736000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3736000640 1 -- 172.21.15.27:0/3849635035 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37400305c0 con 0x556667ce7350 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373802bac0 con 0x556667ce7350 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3736000640 1 -- 172.21.15.27:0/3849635035 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3740031450 con 0x556667ce7350 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3736000640 1 -- 172.21.15.27:0/3849635035 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3740030ec0 con 0x556667ce7350 2024-09-21T07:14:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 10 client.26024 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556668117760 con 0x556667ce7350 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3736000640 1 -- 172.21.15.27:0/3849635035 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373802bac0 con 0x556667ce7350 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3736000640 1 -- 172.21.15.27:0/3849635035 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f374004d030 con 0x556667ce7350 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 10 client.26024 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 20 client.26024 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667c7b080 0x556668119c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 4 client.26024 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.750+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556668002890 con 0x556667c7b080 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.751+0000 7f374ca00640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667c7b080 0x556668119c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.751+0000 7f374ca00640 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667c7b080 0x556668119c50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91139382f45f364 server_cookie=b2561f9c1f14bb36 in_seq=0 out_seq=0 2024-09-21T07:14:33.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.751+0000 7f3736000640 10 client.26024 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3736000640 1 -- 172.21.15.27:0/3849635035 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556668002890 con 0x556667c7b080 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3736000640 10 client.26024 handle_command_reply: tid=0 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 client.26024 shutdown 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667ce7350 msgr2=0x556668116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667ce7350 0x556668116090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37400309a0 tx=0x7f3740030cd0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 shutdown_connections 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556667c7b080 0x556668119c50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556667cc6670 0x55666809eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556667ce7350 0x556668116090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 --2- 172.21.15.27:0/3849635035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556667cbe490 0x55666809e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 >> 172.21.15.27:0/3849635035 conn(0x556667cdb740 msgr2=0x55666809bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3736000640 0 client.26024 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 shutdown_connections 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.753+0000 7f3755ff8740 1 -- 172.21.15.27:0/3849635035 wait complete. 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f3755ff8740 20 client.26024 trim_cache size 0 max 16384 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "a21b9482-1919-4fdf-88af-92fc0d74616b", 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:14:33.759 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f372c0b6780 msgr2=0x7f372c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f372c0b6780 0x7f372c0b8c90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55666808d990 tx=0x7f3738005f90 comp rx=0 tx=0).stop 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 msgr2=0x7f37481a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 0x7f37481a0fa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f373000e9f0 tx=0x7f373000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 shutdown_connections 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f372c0b6780 0x7f372c0b8c90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748114430 0x7f37481a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3748077380 0x7f37481a0fa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 --2- 172.21.15.27:0/43771644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3748075dd0 0x7f37481a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:33.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.754+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 >> 172.21.15.27:0/43771644 conn(0x7f374807b850 msgr2=0x7f3748112d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:33.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.755+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 shutdown_connections 2024-09-21T07:14:33.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:33.755+0000 7f374f400640 1 -- 172.21.15.27:0/43771644 wait complete. 2024-09-21T07:14:34.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:14:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:34 smithi153 ceph-mon[34747]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T07:14:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:34 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:14:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:34 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:14:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:34 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:14:34.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:34 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:14:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:34 smithi119 ceph-mon[34519]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T07:14:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:34 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:14:35.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:34 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:14:35.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:34 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:14:35.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:34 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:14:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:34 smithi027 ceph-mon[30779]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T07:14:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:34 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:14:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:34 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:14:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:34 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:14:35.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:34 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:14:35.134 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 -- 172.21.15.27:0/2689613232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8077940 msgr2=0x7f3ef8077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 --2- 172.21.15.27:0/2689613232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8077940 0x7f3ef8077d50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009900 tx=0x7f3ee802a400 comp rx=0 tx=0).stop 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 -- 172.21.15.27:0/2689613232 shutdown_connections 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 --2- 172.21.15.27:0/2689613232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef8114740 0x7f3ef8116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 --2- 172.21.15.27:0/2689613232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef8078290 0x7f3ef8114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 --2- 172.21.15.27:0/2689613232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8077940 0x7f3ef8077d50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 -- 172.21.15.27:0/2689613232 >> 172.21.15.27:0/2689613232 conn(0x7f3ef8101810 msgr2=0x7f3ef8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.732+0000 7f3eff200640 1 -- 172.21.15.27:0/2689613232 shutdown_connections 2024-09-21T07:14:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.733+0000 7f3eff200640 1 -- 172.21.15.27:0/2689613232 wait complete. 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 Processor -- start 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 -- start start 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef8077940 0x7f3ef81a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef8078290 0x7f3ef81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8114740 0x7f3ef81a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef811a0b0 con 0x7f3ef8077940 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef8119f30 con 0x7f3ef8078290 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3eff200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef811a230 con 0x7f3ef8114740 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef8077940 0x7f3ef81a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3efd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8114740 0x7f3ef81a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef8077940 0x7f3ef81a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39544/0 (socket says 172.21.15.27:39544) 2024-09-21T07:14:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.734+0000 7f3ef7e00640 1 -- 172.21.15.27:0/3078903379 learned_addr learned my addr 172.21.15.27:0/3078903379 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.735+0000 7f3eef400640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef8078290 0x7f3ef81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.735+0000 7f3efd200640 1 -- 172.21.15.27:0/3078903379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef8078290 msgr2=0x7f3ef81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.735+0000 7f3efd200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef8078290 0x7f3ef81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.735+0000 7f3efd200640 1 -- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef8077940 msgr2=0x7f3ef81a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.735+0000 7f3efd200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef8077940 0x7f3ef81a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.735+0000 7f3efd200640 1 -- 172.21.15.27:0/3078903379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef81a6a30 con 0x7f3ef8114740 2024-09-21T07:14:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.735+0000 7f3efd200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8114740 0x7f3ef81a6300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000ec50 tx=0x7f3ef000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.736+0000 7f3ef5600640 1 -- 172.21.15.27:0/3078903379 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0003b70 con 0x7f3ef8114740 2024-09-21T07:14:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.736+0000 7f3ef5600640 1 -- 172.21.15.27:0/3078903379 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef0003d10 con 0x7f3ef8114740 2024-09-21T07:14:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.736+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef81a6d20 con 0x7f3ef8114740 2024-09-21T07:14:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.736+0000 7f3ef5600640 1 -- 172.21.15.27:0/3078903379 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0010830 con 0x7f3ef8114740 2024-09-21T07:14:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.736+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef81ad480 con 0x7f3ef8114740 2024-09-21T07:14:35.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.739+0000 7f3f05e30740 1 Processor -- start 2024-09-21T07:14:35.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.739+0000 7f3ef5600640 1 -- 172.21.15.27:0/3078903379 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3ef0003ec0 con 0x7f3ef8114740 2024-09-21T07:14:35.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.739+0000 7f3f05e30740 1 -- start start 2024-09-21T07:14:35.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3f05e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c767049f70 con 0x55c766d2c070 2024-09-21T07:14:35.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3f05e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c7670393a0 con 0x55c766d031b0 2024-09-21T07:14:35.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3f05e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c76703d1b0 con 0x55c766d0b390 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:57172/0 (socket says 172.21.15.27:57172) 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3ef7e00640 1 -- 172.21.15.27:0/2502875523 learned_addr learned my addr 172.21.15.27:0/2502875523 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39560/0 (socket says 172.21.15.27:39560) 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.740+0000 7f3eef400640 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef7e00640 1 -- 172.21.15.27:0/2502875523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 msgr2=0x55c7670d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef7e00640 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670d35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef7e00640 1 -- 172.21.15.27:0/2502875523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 msgr2=0x55c7670d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef7e00640 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670d2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef7e00640 1 -- 172.21.15.27:0/2502875523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c767003c90 con 0x55c766d031b0 2024-09-21T07:14:35.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3eef400640 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:35.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef7e00640 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670d30a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258a68ab2d926c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:35.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef5600640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3edc0b6700 0x7f3edc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.741+0000 7f3ef5600640 1 -- 172.21.15.27:0/3078903379 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3ef00e7f50 con 0x7f3ef8114740 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.742+0000 7f3eef400640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3edc0b6700 0x7f3edc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.742+0000 7f3eed600640 1 -- 172.21.15.27:0/2502875523 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8040070 con 0x55c766d031b0 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.742+0000 7f3eed600640 1 -- 172.21.15.27:0/2502875523 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee8004030 con 0x55c766d031b0 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.742+0000 7f3eed600640 1 -- 172.21.15.27:0/2502875523 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8035040 con 0x55c766d031b0 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 -- 172.21.15.27:0/2502875523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 msgr2=0x55c7670d30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670d30a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8031b10 comp rx=0 tx=0).stop 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3eef400640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3edc0b6700 0x7f3edc0b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009730 tx=0x7f3ee0015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 -- 172.21.15.27:0/2502875523 shutdown_connections 2024-09-21T07:14:35.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670d30a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 --2- 172.21.15.27:0/2502875523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 -- 172.21.15.27:0/2502875523 >> 172.21.15.27:0/2502875523 conn(0x55c766d20460 msgr2=0x55c7670d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:35.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 -- 172.21.15.27:0/2502875523 shutdown_connections 2024-09-21T07:14:35.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.743+0000 7f3f05e30740 1 -- 172.21.15.27:0/2502875523 wait complete. 2024-09-21T07:14:35.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 -- start start 2024-09-21T07:14:35.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 Processor -- start 2024-09-21T07:14:35.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d031b0 0x55c7670de780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670decc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670ea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c766e9a700 con 0x55c766d2c070 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c766fd4540 con 0x55c766d0b390 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.744+0000 7f3f05e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c7670e12e0 con 0x55c766d031b0 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3f05e30740 10 client.0 _ll_register_callbacks cb 0x55c766fe4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d031b0 0x55c7670de780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3eef400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670decc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3efd200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d031b0 0x55c7670de780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33170/0 (socket says 172.21.15.27:33170) 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3ef7e00640 1 -- 172.21.15.27:0/1414802511 learned_addr learned my addr 172.21.15.27:0/1414802511 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3eef400640 1 -- 172.21.15.27:0/1414802511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d031b0 msgr2=0x55c7670de780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3eef400640 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d031b0 0x55c7670de780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3eef400640 1 -- 172.21.15.27:0/1414802511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 msgr2=0x55c7670ea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3eef400640 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670ea600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3eef400640 1 -- 172.21.15.27:0/1414802511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c766d97650 con 0x55c766d0b390 2024-09-21T07:14:35.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.745+0000 7f3efd200640 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3eef400640 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670decc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ef807a1f0 tx=0x7f3ee0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89278a36e15e7fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3ee6000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3ee6000640 1 -- 172.21.15.27:0/1414802511 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00107c0 con 0x55c766d0b390 2024-09-21T07:14:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c766fd6b50 con 0x55c766d0b390 2024-09-21T07:14:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3ee6000640 1 -- 172.21.15.27:0/1414802511 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3ee0023420 con 0x55c766d0b390 2024-09-21T07:14:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3ee6000640 1 -- 172.21.15.27:0/1414802511 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ee0010960 con 0x55c766d0b390 2024-09-21T07:14:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3f05e30740 10 client.26036 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.746+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c766ff1910 con 0x55c766d0b390 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3ee6000640 1 -- 172.21.15.27:0/1414802511 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c766fd6b50 con 0x55c766d0b390 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3ee6000640 1 -- 172.21.15.27:0/1414802511 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3ee0022380 con 0x55c766d0b390 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3f05e30740 10 client.26036 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3f05e30740 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3f05e30740 20 client.26036 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cea9f0 0x55c7670f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3f05e30740 4 client.26036 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c76704bec0 con 0x55c766cea9f0 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.747+0000 7f3ef7e00640 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cea9f0 0x55c7670f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.748+0000 7f3ef7e00640 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cea9f0 0x55c7670f06e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=205a118871024dcb server_cookie=ea60ce45498cb17e in_seq=0 out_seq=0 2024-09-21T07:14:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.748+0000 7f3ee6000640 10 client.26036 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:35.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3ee6000640 1 -- 172.21.15.27:0/1414802511 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c76704bec0 con 0x55c766cea9f0 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3ee6000640 10 client.26036 handle_command_reply: tid=0 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 client.26036 shutdown 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 msgr2=0x55c7670decc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670decc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ef807a1f0 tx=0x7f3ee0005f90 comp rx=0 tx=0).stop 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 shutdown_connections 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cea9f0 0x55c7670f06e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670decc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3ee6000640 0 client.26036 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1414802511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d031b0 0x55c7670de780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 >> 172.21.15.27:0/1414802511 conn(0x55c766d20460 msgr2=0x55c7670d5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 shutdown_connections 2024-09-21T07:14:35.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.749+0000 7f3f05e30740 1 -- 172.21.15.27:0/1414802511 wait complete. 2024-09-21T07:14:35.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.750+0000 7f3f05e30740 20 client.26036 trim_cache size 0 max 16384 2024-09-21T07:14:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.768+0000 7f3f05e30740 1 Processor -- start 2024-09-21T07:14:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.768+0000 7f3f05e30740 1 -- start start 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d031b0 0x55c7670ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d2c070 0x55c7670d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3f05e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b9c50 con 0x55c766d031b0 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3f05e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b7520 con 0x55c766d0b390 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3f05e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b5370 con 0x55c766d2c070 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d031b0 0x55c7670ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d031b0 0x55c7670ea500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39578/0 (socket says 172.21.15.27:39578) 2024-09-21T07:14:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3eef400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3ef7e00640 1 -- 172.21.15.27:0/1262768771 learned_addr learned my addr 172.21.15.27:0/1262768771 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3efd200640 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d2c070 0x55c7670d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3eef400640 1 -- 172.21.15.27:0/1262768771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d2c070 msgr2=0x55c7670d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.769+0000 7f3eef400640 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d2c070 0x55c7670d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.770+0000 7f3eef400640 1 -- 172.21.15.27:0/1262768771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d031b0 msgr2=0x55c7670ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.770+0000 7f3eef400640 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d031b0 0x55c7670ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.770+0000 7f3eef400640 1 -- 172.21.15.27:0/1262768771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7670cac70 con 0x55c766d0b390 2024-09-21T07:14:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.770+0000 7f3eef400640 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670eaa40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00227c0 tx=0x7f3ee0010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0b88b6b84c8de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.770+0000 7f3eecc00640 1 -- 172.21.15.27:0/1262768771 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00107c0 con 0x55c766d0b390 2024-09-21T07:14:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.770+0000 7f3eecc00640 1 -- 172.21.15.27:0/1262768771 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee0022380 con 0x55c766d0b390 2024-09-21T07:14:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.770+0000 7f3eecc00640 1 -- 172.21.15.27:0/1262768771 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0035420 con 0x55c766d0b390 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 -- 172.21.15.27:0/1262768771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 msgr2=0x55c7670eaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670eaa40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00227c0 tx=0x7f3ee0010430 comp rx=0 tx=0).stop 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 -- 172.21.15.27:0/1262768771 shutdown_connections 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d2c070 0x55c7670d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d0b390 0x55c7670eaa40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 --2- 172.21.15.27:0/1262768771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d031b0 0x55c7670ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 -- 172.21.15.27:0/1262768771 >> 172.21.15.27:0/1262768771 conn(0x55c766d20460 msgr2=0x55c7670dce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 -- 172.21.15.27:0/1262768771 shutdown_connections 2024-09-21T07:14:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.771+0000 7f3f05e30740 1 -- 172.21.15.27:0/1262768771 wait complete. 2024-09-21T07:14:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.772+0000 7f3f05e30740 1 -- start start 2024-09-21T07:14:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.772+0000 7f3f05e30740 1 Processor -- start 2024-09-21T07:14:35.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670e2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3f05e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3f05e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b59e0 con 0x55c766d2c070 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3f05e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b6e50 con 0x55c766d031b0 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3f05e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b60b0 con 0x55c766d0b390 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3f05e30740 10 client.0 _ll_register_callbacks cb 0x7f3ee0004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670e2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39588/0 (socket says 172.21.15.27:39588) 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 -- 172.21.15.27:0/3329664105 learned_addr learned my addr 172.21.15.27:0/3329664105 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3eef400640 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 -- 172.21.15.27:0/3329664105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 msgr2=0x55c7670e29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670e29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 -- 172.21.15.27:0/3329664105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 msgr2=0x55c7670e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.773+0000 7f3efd200640 1 -- 172.21.15.27:0/3329664105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c76715a0a0 con 0x55c766d2c070 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3efd200640 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670e2f00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00aa020 tx=0x7f3ef0004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ba043d17f42e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:35.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3ee6000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:35.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3ee6000640 1 -- 172.21.15.27:0/3329664105 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00a91c0 con 0x55c766d2c070 2024-09-21T07:14:35.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3ee6000640 1 -- 172.21.15.27:0/3329664105 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3ef00a93a0 con 0x55c766d2c070 2024-09-21T07:14:35.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee0022380 con 0x55c766d2c070 2024-09-21T07:14:35.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3ee6000640 1 -- 172.21.15.27:0/3329664105 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ef0009820 con 0x55c766d2c070 2024-09-21T07:14:35.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3f05e30740 10 client.16011 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:35.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.774+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c76715b2d0 con 0x55c766d2c070 2024-09-21T07:14:35.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3ee6000640 1 -- 172.21.15.27:0/3329664105 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee0022380 con 0x55c766d2c070 2024-09-21T07:14:35.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3ee6000640 1 -- 172.21.15.27:0/3329664105 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3ef00a95a0 con 0x55c766d2c070 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3f05e30740 10 client.16011 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3f05e30740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3f05e30740 20 client.16011 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3f05e30740 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cbfda0 0x55c76715da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3f05e30740 4 client.16011 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c767047970 con 0x55c766cbfda0 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.775+0000 7f3ef7e00640 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cbfda0 0x55c76715da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.776+0000 7f3ef7e00640 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cbfda0 0x55c76715da90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cfb890517e53ba server_cookie=4f0358c38d650aa8 in_seq=0 out_seq=0 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.776+0000 7f3ee6000640 10 client.16011 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.776+0000 7f3ee6000640 1 -- 172.21.15.27:0/3329664105 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55c767047970 con 0x55c766cbfda0 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.776+0000 7f3ee6000640 10 client.16011 handle_command_reply: tid=0 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.776+0000 7f3f05e30740 1 client.16011 shutdown 2024-09-21T07:14:35.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 msgr2=0x55c7670e2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670e2f00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00aa020 tx=0x7f3ef0004f10 comp rx=0 tx=0).stop 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 shutdown_connections 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c766cbfda0 0x55c76715da90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c766d0b390 0x55c7670e29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c766d2c070 0x55c7670e2f00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 --2- 172.21.15.27:0/3329664105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c766d031b0 0x55c7670e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 >> 172.21.15.27:0/3329664105 conn(0x55c766d20460 msgr2=0x55c7670d3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3ee6000640 0 client.16011 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 shutdown_connections 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 1 -- 172.21.15.27:0/3329664105 wait complete. 2024-09-21T07:14:35.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.777+0000 7f3f05e30740 20 client.16011 trim_cache size 0 max 16384 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "a21b9482-1919-4fdf-88af-92fc0d74616b", 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:14:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:35.784 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:14:35.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3edc0b6700 msgr2=0x7f3edc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3edc0b6700 0x7f3edc0b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009730 tx=0x7f3ee0015040 comp rx=0 tx=0).stop 2024-09-21T07:14:35.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8114740 msgr2=0x7f3ef81a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:35.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8114740 0x7f3ef81a6300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000ec50 tx=0x7f3ef000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 shutdown_connections 2024-09-21T07:14:35.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef8114740 0x7f3ef81a6300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3edc0b6700 0x7f3edc0b8c10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ef8078290 0x7f3ef81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 --2- 172.21.15.27:0/3078903379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef8077940 0x7f3ef81a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:35.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 >> 172.21.15.27:0/3078903379 conn(0x7f3ef8101810 msgr2=0x7f3ef8116800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:35.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 shutdown_connections 2024-09-21T07:14:35.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:35.778+0000 7f3eff200640 1 -- 172.21.15.27:0/3078903379 wait complete. 2024-09-21T07:14:36.629 INFO:tasks.cephfs.filesystem:scrub status for tag:a21b9482-1919-4fdf-88af-92fc0d74616b - {'path': '/', 'tag': 'a21b9482-1919-4fdf-88af-92fc0d74616b', 'options': 'recursive,force'} 2024-09-21T07:14:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:36 smithi027 ceph-mon[30779]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:36.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:36 smithi153 ceph-mon[34747]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:36 smithi119 ceph-mon[34519]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:38.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:38 smithi153 ceph-mon[34747]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:39.032 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:14:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:38 smithi119 ceph-mon[34519]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:39.085 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:38 smithi027 ceph-mon[30779]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.026+0000 7f6026800640 1 -- 172.21.15.27:0/2748977658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 msgr2=0x7f6020106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.026+0000 7f6026800640 1 --2- 172.21.15.27:0/2748977658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 0x7f6020106190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6010009900 tx=0x7f601002a410 comp rx=0 tx=0).stop 2024-09-21T07:14:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.026+0000 7f6026800640 1 -- 172.21.15.27:0/2748977658 shutdown_connections 2024-09-21T07:14:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.026+0000 7f6026800640 1 --2- 172.21.15.27:0/2748977658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f602010c010 0x7f6020112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.026+0000 7f6026800640 1 --2- 172.21.15.27:0/2748977658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 0x7f602010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.026+0000 7f6026800640 1 --2- 172.21.15.27:0/2748977658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 0x7f6020106190 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.027+0000 7f6026800640 1 -- 172.21.15.27:0/2748977658 >> 172.21.15.27:0/2748977658 conn(0x7f6020101810 msgr2=0x7f6020103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.027+0000 7f6026800640 1 -- 172.21.15.27:0/2748977658 shutdown_connections 2024-09-21T07:14:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.027+0000 7f6026800640 1 -- 172.21.15.27:0/2748977658 wait complete. 2024-09-21T07:14:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.028+0000 7f6026800640 1 Processor -- start 2024-09-21T07:14:40.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.028+0000 7f6026800640 1 -- start start 2024-09-21T07:14:40.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f6026800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 0x7f602019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:40.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f6026800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 0x7f602019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:40.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f6026800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f602010c010 0x7f60201a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:40.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f6026800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6020115cd0 con 0x7f6020105d80 2024-09-21T07:14:40.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f6026800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6020115b50 con 0x7f602010b6c0 2024-09-21T07:14:40.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f6026800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6020115e50 con 0x7f602010c010 2024-09-21T07:14:40.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 0x7f602019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:40.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 0x7f602019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:40.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 0x7f602019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46002/0 (socket says 172.21.15.27:46002) 2024-09-21T07:14:40.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 -- 172.21.15.27:0/359119571 learned_addr learned my addr 172.21.15.27:0/359119571 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601fe00640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f602010c010 0x7f60201a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 -- 172.21.15.27:0/359119571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f602010c010 msgr2=0x7f60201a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f602010c010 0x7f60201a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 -- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 msgr2=0x7f602019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 0x7f602019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.029+0000 7f601ea00640 1 -- 172.21.15.27:0/359119571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60201a2650 con 0x7f602010b6c0 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.030+0000 7f601ea00640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 0x7f602019cc80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f600c00e9f0 tx=0x7f600c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.030+0000 7f6017e00640 1 -- 172.21.15.27:0/359119571 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c00ccc0 con 0x7f602010b6c0 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.030+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60201a2940 con 0x7f602010b6c0 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.030+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60201a90a0 con 0x7f602010b6c0 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.030+0000 7f6017e00640 1 -- 172.21.15.27:0/359119571 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600c0108e0 con 0x7f602010b6c0 2024-09-21T07:14:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.031+0000 7f6017e00640 1 -- 172.21.15.27:0/359119571 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c026060 con 0x7f602010b6c0 2024-09-21T07:14:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.031+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60000052c0 con 0x7f602010b6c0 2024-09-21T07:14:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.035+0000 7f6017e00640 1 -- 172.21.15.27:0/359119571 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f600c010430 con 0x7f602010b6c0 2024-09-21T07:14:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.036+0000 7f6017e00640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60040b6700 0x7f60040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.036+0000 7f6017e00640 1 -- 172.21.15.27:0/359119571 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f600c0e7c90 con 0x7f602010b6c0 2024-09-21T07:14:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.037+0000 7f601f400640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60040b6700 0x7f60040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.037+0000 7f6017e00640 1 -- 172.21.15.27:0/359119571 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f600c0ade30 con 0x7f602010b6c0 2024-09-21T07:14:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.037+0000 7f601f400640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60040b6700 0x7f60040b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6010031440 tx=0x7f6010009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:40.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.277+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60000055b0 con 0x7f602010b6c0 2024-09-21T07:14:40.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.278+0000 7f6017e00640 1 -- 172.21.15.27:0/359119571 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f600c0b2210 con 0x7f602010b6c0 2024-09-21T07:14:40.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:14:40.281 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:14:40.281 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60040b6700 msgr2=0x7f60040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60040b6700 0x7f60040b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6010031440 tx=0x7f6010009560 comp rx=0 tx=0).stop 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 msgr2=0x7f602019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 0x7f602019cc80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f600c00e9f0 tx=0x7f600c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 shutdown_connections 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f602010c010 0x7f60201a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60040b6700 0x7f60040b8c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f602010b6c0 0x7f602019cc80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 --2- 172.21.15.27:0/359119571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6020105d80 0x7f602019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 >> 172.21.15.27:0/359119571 conn(0x7f6020101810 msgr2=0x7f6020103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 shutdown_connections 2024-09-21T07:14:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:40.281+0000 7f6026800640 1 -- 172.21.15.27:0/359119571 wait complete. 2024-09-21T07:14:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:40 smithi153 ceph-mon[34747]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:40 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/359119571' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:40 smithi119 ceph-mon[34519]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:40 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/359119571' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:41.041 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:40 smithi027 ceph-mon[30779]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:41.041 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:40 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/359119571' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:14:41.082 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 7b608863 --release --await' 2024-09-21T07:14:41.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 7b608863 --release --await 2024-09-21T07:14:41.470 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:14:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.094+0000 7fa45ea00640 1 -- 172.21.15.27:0/2531880875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581092f0 msgr2=0x7fa458109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.094+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2531880875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581092f0 0x7fa458109700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa444009900 tx=0x7fa44402a410 comp rx=0 tx=0).stop 2024-09-21T07:14:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.095+0000 7fa45ea00640 1 -- 172.21.15.27:0/2531880875 shutdown_connections 2024-09-21T07:14:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.095+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2531880875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa458109c40 0x7fa458110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.095+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2531880875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581092f0 0x7fa458109700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.095+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2531880875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4581039b0 0x7fa458103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.095+0000 7fa45ea00640 1 -- 172.21.15.27:0/2531880875 >> 172.21.15.27:0/2531880875 conn(0x7fa4580ff400 msgr2=0x7fa458101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:42.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.095+0000 7fa45ea00640 1 -- 172.21.15.27:0/2531880875 shutdown_connections 2024-09-21T07:14:42.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.095+0000 7fa45ea00640 1 -- 172.21.15.27:0/2531880875 wait complete. 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.096+0000 7fa45ea00640 1 Processor -- start 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.096+0000 7fa45ea00640 1 -- start start 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.096+0000 7fa45ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581039b0 0x7fa45819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.096+0000 7fa45ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4581092f0 0x7fa45819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa458109c40 0x7fa45819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45ea00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa458113900 con 0x7fa4581039b0 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45ea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa458113780 con 0x7fa458109c40 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45ea00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa458113a80 con 0x7fa4581092f0 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581039b0 0x7fa45819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4581092f0 0x7fa45819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581039b0 0x7fa45819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58888/0 (socket says 172.21.15.27:58888) 2024-09-21T07:14:42.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4581092f0 0x7fa45819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56736/0 (socket says 172.21.15.27:56736) 2024-09-21T07:14:42.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45d600640 1 -- 172.21.15.27:0/2103970951 learned_addr learned my addr 172.21.15.27:0/2103970951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:42.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45e000640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa458109c40 0x7fa45819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45cc00640 1 -- 172.21.15.27:0/2103970951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa458109c40 msgr2=0x7fa45819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45cc00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa458109c40 0x7fa45819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45cc00640 1 -- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581039b0 msgr2=0x7fa45819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45cc00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581039b0 0x7fa45819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45cc00640 1 -- 172.21.15.27:0/2103970951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4581a0240 con 0x7fa4581092f0 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45d600640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581039b0 0x7fa45819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.097+0000 7fa45e000640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa458109c40 0x7fa45819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.098+0000 7fa45cc00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4581092f0 0x7fa45819a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa44402a930 tx=0x7fa444031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.098+0000 7fa456000640 1 -- 172.21.15.27:0/2103970951 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44403f070 con 0x7fa4581092f0 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.098+0000 7fa456000640 1 -- 172.21.15.27:0/2103970951 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa444004490 con 0x7fa4581092f0 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.098+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4581a04d0 con 0x7fa4581092f0 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.098+0000 7fa456000640 1 -- 172.21.15.27:0/2103970951 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444035070 con 0x7fa4581092f0 2024-09-21T07:14:42.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.098+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4581a6c90 con 0x7fa4581092f0 2024-09-21T07:14:42.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.102+0000 7fa466e00740 1 Processor -- start 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.102+0000 7fa466e00740 1 -- start start 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa466e00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa466e00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa466e00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 0x55cedb4aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa466e00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cedb425290 con 0x55cedb0e6840 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa466e00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cedb4146c0 con 0x55cedb0de660 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa466e00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cedb4184d0 con 0x55cedb107520 2024-09-21T07:14:42.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa45e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa45e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56744/0 (socket says 172.21.15.27:56744) 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa45e000640 1 -- 172.21.15.27:0/3430122353 learned_addr learned my addr 172.21.15.27:0/3430122353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.103+0000 7fa45d600640 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa45e000640 1 -- 172.21.15.27:0/3430122353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 msgr2=0x55cedb4ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa45e000640 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa45e000640 1 -- 172.21.15.27:0/3430122353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 msgr2=0x55cedb4aedc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa45e000640 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 0x55cedb4aedc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa45e000640 1 -- 172.21.15.27:0/3430122353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cedb3a1460 con 0x55cedb107520 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa456000640 1 -- 172.21.15.27:0/2103970951 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa44403b460 con 0x7fa4581092f0 2024-09-21T07:14:42.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa45e000640 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4ae340 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa4500117e0 tx=0x7fa450011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70920c7746d4f4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:42.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.104+0000 7fa44a600640 1 -- 172.21.15.27:0/3430122353 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450011310 con 0x55cedb107520 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.105+0000 7fa44a600640 1 -- 172.21.15.27:0/3430122353 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450003e20 con 0x55cedb107520 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.105+0000 7fa44a600640 1 -- 172.21.15.27:0/3430122353 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4500044f0 con 0x55cedb107520 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.105+0000 7fa466e00740 1 -- 172.21.15.27:0/3430122353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 msgr2=0x55cedb4ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.105+0000 7fa466e00740 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4ae340 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa4500117e0 tx=0x7fa450011ca0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.105+0000 7fa466e00740 1 -- 172.21.15.27:0/3430122353 shutdown_connections 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.106+0000 7fa466e00740 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 0x55cedb4aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.106+0000 7fa466e00740 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.106+0000 7fa466e00740 1 --2- 172.21.15.27:0/3430122353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4ae340 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.106+0000 7fa466e00740 1 -- 172.21.15.27:0/3430122353 >> 172.21.15.27:0/3430122353 conn(0x55cedb0fb910 msgr2=0x55cedb4ac230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.106+0000 7fa466e00740 1 -- 172.21.15.27:0/3430122353 shutdown_connections 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.106+0000 7fa466e00740 1 -- 172.21.15.27:0/3430122353 wait complete. 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.106+0000 7fa456000640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa4400b6590 0x7fa4400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.107+0000 7fa466e00740 1 -- start start 2024-09-21T07:14:42.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.107+0000 7fa456000640 1 -- 172.21.15.27:0/2103970951 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa444107c20 con 0x7fa4581092f0 2024-09-21T07:14:42.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.107+0000 7fa45e000640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa4400b6590 0x7fa4400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.107+0000 7fa466e00740 1 Processor -- start 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.107+0000 7fa45e000640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa4400b6590 0x7fa4400b8aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4500041d0 tx=0x7fa450005fd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa466e00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0de660 0x55cedb4b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa466e00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0e6840 0x55cedb4ba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa466e00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa466e00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cedb275bb0 con 0x55cedb107520 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa466e00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cedb3af6c0 con 0x55cedb0e6840 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa466e00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cedb4bcb20 con 0x55cedb0de660 2024-09-21T07:14:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa466e00740 10 client.0 _ll_register_callbacks cb 0x55cedb3ca490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa45e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa45cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0e6840 0x55cedb4ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa45e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4bfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58900/0 (socket says 172.21.15.27:58900) 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa45e000640 1 -- 172.21.15.27:0/2996366656 learned_addr learned my addr 172.21.15.27:0/2996366656 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.108+0000 7fa45d600640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0de660 0x55cedb4b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45cc00640 1 -- 172.21.15.27:0/2996366656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0de660 msgr2=0x55cedb4b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45cc00640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0de660 0x55cedb4b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45cc00640 1 -- 172.21.15.27:0/2996366656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 msgr2=0x55cedb4bfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45cc00640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4bfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45cc00640 1 -- 172.21.15.27:0/2996366656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cedb172b00 con 0x55cedb0e6840 2024-09-21T07:14:42.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45d600640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0de660 0x55cedb4b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:42.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45e000640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4bfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:42.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa45cc00640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0e6840 0x55cedb4ba420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa444104fc0 tx=0x7fa444105500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e19579f6b2ab59a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:42.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa43ea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:42.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa43ea00640 1 -- 172.21.15.27:0/2996366656 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444048040 con 0x55cedb0e6840 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa450003e20 con 0x55cedb0e6840 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.109+0000 7fa43ea00640 1 -- 172.21.15.27:0/2996366656 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa4440cc9d0 con 0x55cedb0e6840 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa43ea00640 1 -- 172.21.15.27:0/2996366656 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4440cd280 con 0x55cedb0e6840 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 10 client.26060 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cedb1e9960 con 0x55cedb0e6840 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa43ea00640 1 -- 172.21.15.27:0/2996366656 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450003e20 con 0x55cedb0e6840 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa43ea00640 1 -- 172.21.15.27:0/2996366656 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4440cc560 con 0x55cedb0e6840 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 10 client.26060 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 10 client.26060 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 20 client.26060 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb0c5ea0 0x55cedb4d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 4 client.26060 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:42.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.110+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cedb4271e0 con 0x55cedb0c5ea0 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.111+0000 7fa45d600640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb0c5ea0 0x55cedb4d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.111+0000 7fa45d600640 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb0c5ea0 0x55cedb4d45b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919cf035664aada server_cookie=220e3a82200e15cf in_seq=0 out_seq=0 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.111+0000 7fa43ea00640 10 client.26060 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa43ea00640 1 -- 172.21.15.27:0/2996366656 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cedb4271e0 con 0x55cedb0c5ea0 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa43ea00640 10 client.26060 handle_command_reply: tid=0 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 client.26060 shutdown 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0e6840 msgr2=0x55cedb4ba420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0e6840 0x55cedb4ba420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa444104fc0 tx=0x7fa444105500 comp rx=0 tx=0).stop 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 shutdown_connections 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb0c5ea0 0x55cedb4d45b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4bfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0e6840 0x55cedb4ba420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 --2- 172.21.15.27:0/2996366656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0de660 0x55cedb4b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 >> 172.21.15.27:0/2996366656 conn(0x55cedb0fb910 msgr2=0x55cedb4b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:42.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.112+0000 7fa43ea00640 0 client.26060 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:42.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.113+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 shutdown_connections 2024-09-21T07:14:42.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.113+0000 7fa466e00740 1 -- 172.21.15.27:0/2996366656 wait complete. 2024-09-21T07:14:42.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.113+0000 7fa466e00740 20 client.26060 trim_cache size 0 max 16384 2024-09-21T07:14:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 Processor -- start 2024-09-21T07:14:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 -- start start 2024-09-21T07:14:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0e6840 0x55cedb4b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581b11b0 con 0x55cedb107520 2024-09-21T07:14:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581aea80 con 0x55cedb0de660 2024-09-21T07:14:42.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.131+0000 7fa466e00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581ac8d0 con 0x55cedb0e6840 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0e6840 0x55cedb4b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58914/0 (socket says 172.21.15.27:58914) 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45e000640 1 -- 172.21.15.27:0/3276040222 learned_addr learned my addr 172.21.15.27:0/3276040222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45d600640 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45cc00640 1 -- 172.21.15.27:0/3276040222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 msgr2=0x55cedb4ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45cc00640 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45cc00640 1 -- 172.21.15.27:0/3276040222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 msgr2=0x55cedb4b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45cc00640 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45cc00640 1 -- 172.21.15.27:0/3276040222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cedb4b5590 con 0x55cedb0e6840 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.132+0000 7fa45cc00640 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0e6840 0x55cedb4b1b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4440ca9e0 tx=0x7fa4440c9010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1c700dd58faae23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.133+0000 7fa449c00640 1 -- 172.21.15.27:0/3276040222 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444048070 con 0x55cedb0e6840 2024-09-21T07:14:42.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.133+0000 7fa449c00640 1 -- 172.21.15.27:0/3276040222 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4440d21a0 con 0x55cedb0e6840 2024-09-21T07:14:42.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.133+0000 7fa449c00640 1 -- 172.21.15.27:0/3276040222 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4440c90c0 con 0x55cedb0e6840 2024-09-21T07:14:42.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 -- 172.21.15.27:0/3276040222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0e6840 msgr2=0x55cedb4b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0e6840 0x55cedb4b1b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4440ca9e0 tx=0x7fa4440c9010 comp rx=0 tx=0).stop 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 -- 172.21.15.27:0/3276040222 shutdown_connections 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb0e6840 0x55cedb4b1b50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb107520 0x55cedb4b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 --2- 172.21.15.27:0/3276040222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 -- 172.21.15.27:0/3276040222 >> 172.21.15.27:0/3276040222 conn(0x55cedb0fb910 msgr2=0x55cedb4b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 -- 172.21.15.27:0/3276040222 shutdown_connections 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 -- 172.21.15.27:0/3276040222 wait complete. 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.134+0000 7fa466e00740 1 -- start start 2024-09-21T07:14:42.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 1 Processor -- start 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 0x55cedb4cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581acf40 con 0x55cedb0e6840 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581ae3b0 con 0x55cedb0de660 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581ad610 con 0x55cedb107520 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa466e00740 10 client.0 _ll_register_callbacks cb 0x7fa444005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa45d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa45d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46058/0 (socket says 172.21.15.27:46058) 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa45d600640 1 -- 172.21.15.27:0/432465026 learned_addr learned my addr 172.21.15.27:0/432465026 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa45e000640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.135+0000 7fa45cc00640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 0x55cedb4cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa45d600640 1 -- 172.21.15.27:0/432465026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 msgr2=0x55cedb4c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa45d600640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa45d600640 1 -- 172.21.15.27:0/432465026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 msgr2=0x55cedb4cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa45d600640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 0x55cedb4cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa45d600640 1 -- 172.21.15.27:0/432465026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cedb4c9f10 con 0x55cedb0de660 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa45e000640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa45d600640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4cea20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa44c004700 tx=0x7fa44c004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5243f3d98d6e245a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa43ea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa43ea00640 1 -- 172.21.15.27:0/432465026 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c003d20 con 0x55cedb0de660 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.136+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4440d21a0 con 0x55cedb0de660 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa43ea00640 1 -- 172.21.15.27:0/432465026 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa44c005020 con 0x55cedb0de660 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa43ea00640 1 -- 172.21.15.27:0/432465026 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa44c005220 con 0x55cedb0de660 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 10 client.26066 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cedb53f6a0 con 0x55cedb0de660 2024-09-21T07:14:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa43ea00640 1 -- 172.21.15.27:0/432465026 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4440d21a0 con 0x55cedb0de660 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa43ea00640 1 -- 172.21.15.27:0/432465026 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa44c010430 con 0x55cedb0de660 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 10 client.26066 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 10 client.26066 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 20 client.26066 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb09b250 0x55cedb541e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 4 client.26066 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7b608863", "release": true, "await": true}] 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.137+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7b608863", "release": true, "await": true}) -- 0x55cedb422c90 con 0x55cedb09b250 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.138+0000 7fa45cc00640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb09b250 0x55cedb541e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.138+0000 7fa45cc00640 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb09b250 0x55cedb541e90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c61c47c9bdc259 server_cookie=fac0273a4fa219c1 in_seq=0 out_seq=0 2024-09-21T07:14:42.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.138+0000 7fa43ea00640 10 client.26066 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:42.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa43ea00640 1 -- 172.21.15.27:0/432465026 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55cedb422c90 con 0x55cedb09b250 2024-09-21T07:14:42.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa43ea00640 10 client.26066 handle_command_reply: tid=0 2024-09-21T07:14:42.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 client.26066 shutdown 2024-09-21T07:14:42.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 msgr2=0x55cedb4cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4cea20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa44c004700 tx=0x7fa44c004410 comp rx=0 tx=0).stop 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 shutdown_connections 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cedb09b250 0x55cedb541e90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cedb107520 0x55cedb4c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cedb0e6840 0x55cedb4cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 --2- 172.21.15.27:0/432465026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cedb0de660 0x55cedb4cea20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 >> 172.21.15.27:0/432465026 conn(0x55cedb0fb910 msgr2=0x55cedb4bb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa43ea00640 0 client.26066 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 shutdown_connections 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.143+0000 7fa466e00740 1 -- 172.21.15.27:0/432465026 wait complete. 2024-09-21T07:14:42.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa466e00740 20 client.26066 trim_cache size 0 max 16384 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 8, 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "7b608863": { 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 8, 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:14:42.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.0, 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:42.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:14:42.149 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:14:42.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa4400b6590 msgr2=0x7fa4400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa4400b6590 0x7fa4400b8aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4500041d0 tx=0x7fa450005fd0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4581092f0 msgr2=0x7fa45819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:42.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4581092f0 0x7fa45819a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa44402a930 tx=0x7fa444031a20 comp rx=0 tx=0).stop 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 shutdown_connections 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4581092f0 0x7fa45819a870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa4400b6590 0x7fa4400b8aa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa458109c40 0x7fa45819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 --2- 172.21.15.27:0/2103970951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4581039b0 0x7fa45819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.144+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 >> 172.21.15.27:0/2103970951 conn(0x7fa4580ff400 msgr2=0x7fa458101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.145+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 shutdown_connections 2024-09-21T07:14:42.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:14:42.145+0000 7fa45ea00640 1 -- 172.21.15.27:0/2103970951 wait complete. 2024-09-21T07:14:42.156 INFO:tasks.workunit.client.0.smithi027.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-21T07:14:42.158 INFO:tasks.workunit.client.0.smithi027.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-21T07:14:42.158 INFO:tasks.workunit.client.0.smithi027.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-21T07:14:42.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:42 smithi153 ceph-mon[34747]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:42.943 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:42 smithi027 ceph-mon[30779]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:42 smithi119 ceph-mon[34519]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T07:14:43.419 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7b608863', total seconds elapsed: 21.4 2024-09-21T07:14:43.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:43 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:14:44.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:43 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:14:44.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:43 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 -- 172.21.15.27:0/2876282376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 msgr2=0x7f9d6c075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 --2- 172.21.15.27:0/2876282376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c075b60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009900 tx=0x7f9d5c02a410 comp rx=0 tx=0).stop 2024-09-21T07:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 -- 172.21.15.27:0/2876282376 shutdown_connections 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 --2- 172.21.15.27:0/2876282376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d6c1100b0 0x7f9d6c1124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 --2- 172.21.15.27:0/2876282376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d6c0760a0 0x7f9d6c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 --2- 172.21.15.27:0/2876282376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c075b60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 -- 172.21.15.27:0/2876282376 >> 172.21.15.27:0/2876282376 conn(0x7f9d6c0ff500 msgr2=0x7f9d6c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 -- 172.21.15.27:0/2876282376 shutdown_connections 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.448+0000 7f9d72200640 1 -- 172.21.15.27:0/2876282376 wait complete. 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.449+0000 7f9d72200640 1 Processor -- start 2024-09-21T07:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.449+0000 7f9d72200640 1 -- start start 2024-09-21T07:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d72200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d72200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d6c0760a0 0x7f9d6c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d72200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d6c1100b0 0x7f9d6c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d72200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c115c20 con 0x7f9d6c075750 2024-09-21T07:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d72200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c115aa0 con 0x7f9d6c1100b0 2024-09-21T07:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d72200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c115da0 con 0x7f9d6c0760a0 2024-09-21T07:14:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58940/0 (socket says 172.21.15.27:58940) 2024-09-21T07:14:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 -- 172.21.15.27:0/465343136 learned_addr learned my addr 172.21.15.27:0/465343136 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d62a00640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d6c0760a0 0x7f9d6c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6be00640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d6c1100b0 0x7f9d6c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 -- 172.21.15.27:0/465343136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d6c0760a0 msgr2=0x7f9d6c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d6c0760a0 0x7f9d6c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 -- 172.21.15.27:0/465343136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d6c1100b0 msgr2=0x7f9d6c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d6c1100b0 0x7f9d6c1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.450+0000 7f9d6b400640 1 -- 172.21.15.27:0/465343136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6c1a2550 con 0x7f9d6c075750 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.451+0000 7f9d6b400640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c19c620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005940 tx=0x7f9d5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.451+0000 7f9d68c00640 1 -- 172.21.15.27:0/465343136 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c03f030 con 0x7f9d6c075750 2024-09-21T07:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.451+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6c1a27e0 con 0x7f9d6c075750 2024-09-21T07:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.451+0000 7f9d68c00640 1 -- 172.21.15.27:0/465343136 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5c0044c0 con 0x7f9d6c075750 2024-09-21T07:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.451+0000 7f9d68c00640 1 -- 172.21.15.27:0/465343136 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c049060 con 0x7f9d6c075750 2024-09-21T07:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.451+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6c1a8fd0 con 0x7f9d6c075750 2024-09-21T07:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.454+0000 7f9d793f5740 1 Processor -- start 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.454+0000 7f9d793f5740 1 -- start start 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3053e1b0 0x55ed308e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed305152f0 0x55ed308e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d68c00640 1 -- 172.21.15.27:0/465343136 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9d5c042030 con 0x7f9d6c075750 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308e36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d793f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ed308586b0 con 0x55ed3053e1b0 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d793f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ed30847ae0 con 0x55ed3051d4d0 2024-09-21T07:14:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d793f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed3084b8f0 con 0x55ed305152f0 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3053e1b0 0x55ed308e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3053e1b0 0x55ed308e2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58954/0 (socket says 172.21.15.27:58954) 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d6be00640 1 -- 172.21.15.27:0/733261502 learned_addr learned my addr 172.21.15.27:0/733261502 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d6b400640 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed305152f0 0x55ed308e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.455+0000 7f9d62a00640 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308e36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d6be00640 1 -- 172.21.15.27:0/733261502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed305152f0 msgr2=0x55ed308e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d6be00640 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed305152f0 0x55ed308e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d6be00640 1 -- 172.21.15.27:0/733261502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 msgr2=0x55ed308e36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d6be00640 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308e36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d6be00640 1 -- 172.21.15.27:0/733261502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed30814780 con 0x55ed3053e1b0 2024-09-21T07:14:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d6be00640 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3053e1b0 0x55ed308e2c30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009fd0 tx=0x7f9d640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1de80f8e6a514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d5be00640 1 -- 172.21.15.27:0/733261502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64004d20 con 0x55ed3053e1b0 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d5be00640 1 -- 172.21.15.27:0/733261502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d64004050 con 0x55ed3053e1b0 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.456+0000 7f9d5be00640 1 -- 172.21.15.27:0/733261502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64005220 con 0x55ed3053e1b0 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d68c00640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d500b6680 0x7f9d500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d793f5740 1 -- 172.21.15.27:0/733261502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3053e1b0 msgr2=0x55ed308e2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d793f5740 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3053e1b0 0x55ed308e2c30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009fd0 tx=0x7f9d640118d0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d68c00640 1 -- 172.21.15.27:0/465343136 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9d5c108190 con 0x7f9d6c075750 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d6b400640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d500b6680 0x7f9d500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d793f5740 1 -- 172.21.15.27:0/733261502 shutdown_connections 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d793f5740 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed305152f0 0x55ed308e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d793f5740 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308e36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d793f5740 1 --2- 172.21.15.27:0/733261502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3053e1b0 0x55ed308e2c30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.457+0000 7f9d793f5740 1 -- 172.21.15.27:0/733261502 >> 172.21.15.27:0/733261502 conn(0x55ed30532300 msgr2=0x55ed308e0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.458+0000 7f9d793f5740 1 -- 172.21.15.27:0/733261502 shutdown_connections 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.458+0000 7f9d793f5740 1 -- 172.21.15.27:0/733261502 wait complete. 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.458+0000 7f9d6b400640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d500b6680 0x7f9d500b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55ed308e18b0 tx=0x7f9d5c10bc40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.458+0000 7f9d793f5740 1 -- start start 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.458+0000 7f9d793f5740 1 Processor -- start 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308ee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d793f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ed3077fab0 con 0x55ed3051d4d0 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d793f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ed307ee570 con 0x55ed305152f0 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d793f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed308f1400 con 0x55ed3053e1b0 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d793f5740 10 client.0 _ll_register_callbacks cb 0x55ed307fd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308ee6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58956/0 (socket says 172.21.15.27:58956) 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d62a00640 1 -- 172.21.15.27:0/1918668175 learned_addr learned my addr 172.21.15.27:0/1918668175 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d6be00640 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308ee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.459+0000 7f9d6be00640 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d62a00640 1 -- 172.21.15.27:0/1918668175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 msgr2=0x55ed308eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d62a00640 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d62a00640 1 -- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 msgr2=0x55ed308ee1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d62a00640 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308ee1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d62a00640 1 -- 172.21.15.27:0/1918668175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed305a9790 con 0x55ed3051d4d0 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d62a00640 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308ee6f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d54012700 tx=0x7f9d54012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e30b278a568e931 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d58c00640 1 -- 172.21.15.27:0/1918668175 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400f8c0 con 0x55ed3051d4d0 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d58c00640 1 -- 172.21.15.27:0/1918668175 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9d5400fa60 con 0x55ed3051d4d0 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.460+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d64004050 con 0x55ed3051d4d0 2024-09-21T07:14:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d58c00640 1 -- 172.21.15.27:0/1918668175 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d54025050 con 0x55ed3051d4d0 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 10 client.16044 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed30622620 con 0x55ed3051d4d0 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d58c00640 1 -- 172.21.15.27:0/1918668175 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d64004050 con 0x55ed3051d4d0 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d58c00640 1 -- 172.21.15.27:0/1918668175 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9d54010430 con 0x55ed3051d4d0 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 10 client.16044 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 20 client.16044 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304fcd80 0x55ed309003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 4 client.16044 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.461+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed3085a600 con 0x55ed304fcd80 2024-09-21T07:14:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.462+0000 7f9d6be00640 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304fcd80 0x55ed309003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.462+0000 7f9d6be00640 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304fcd80 0x55ed309003a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8db1b92cead048 server_cookie=33783403635d8b9b in_seq=0 out_seq=0 2024-09-21T07:14:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.462+0000 7f9d58c00640 10 client.16044 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d58c00640 1 -- 172.21.15.27:0/1918668175 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed3085a600 con 0x55ed304fcd80 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d58c00640 10 client.16044 handle_command_reply: tid=0 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 client.16044 shutdown 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 msgr2=0x55ed308ee6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308ee6f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d54012700 tx=0x7f9d54012bc0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 shutdown_connections 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304fcd80 0x55ed309003a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308ee6f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 --2- 172.21.15.27:0/1918668175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d58c00640 0 client.16044 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.463+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 >> 172.21.15.27:0/1918668175 conn(0x55ed30532300 msgr2=0x55ed308e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.464+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 shutdown_connections 2024-09-21T07:14:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.464+0000 7f9d793f5740 1 -- 172.21.15.27:0/1918668175 wait complete. 2024-09-21T07:14:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.464+0000 7f9d793f5740 20 client.16044 trim_cache size 0 max 16384 2024-09-21T07:14:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 Processor -- start 2024-09-21T07:14:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 -- start start 2024-09-21T07:14:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed305152f0 0x55ed308df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308eb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1b5790 con 0x55ed305152f0 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1b3060 con 0x55ed3051d4d0 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.479+0000 7f9d793f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1b0eb0 con 0x55ed3053e1b0 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed305152f0 0x55ed308df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed305152f0 0x55ed308df7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58972/0 (socket says 172.21.15.27:58972) 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 -- 172.21.15.27:0/3818665700 learned_addr learned my addr 172.21.15.27:0/3818665700 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d62a00640 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 -- 172.21.15.27:0/3818665700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 msgr2=0x55ed308eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 -- 172.21.15.27:0/3818665700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 msgr2=0x55ed308eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308eb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.480+0000 7f9d6be00640 1 -- 172.21.15.27:0/3818665700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed308ef470 con 0x55ed305152f0 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.481+0000 7f9d6be00640 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed305152f0 0x55ed308df7f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d64011dd0 tx=0x7f9d64009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3489cd6133eefbe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.481+0000 7f9d5b400640 1 -- 172.21.15.27:0/3818665700 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64004c80 con 0x55ed305152f0 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.481+0000 7f9d5b400640 1 -- 172.21.15.27:0/3818665700 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d64004e20 con 0x55ed305152f0 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.481+0000 7f9d5b400640 1 -- 172.21.15.27:0/3818665700 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64005630 con 0x55ed305152f0 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 -- 172.21.15.27:0/3818665700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed305152f0 msgr2=0x55ed308df7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed305152f0 0x55ed308df7f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d64011dd0 tx=0x7f9d64009500 comp rx=0 tx=0).stop 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 -- 172.21.15.27:0/3818665700 shutdown_connections 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed308eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed3051d4d0 0x55ed308eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 --2- 172.21.15.27:0/3818665700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed305152f0 0x55ed308df7f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 -- 172.21.15.27:0/3818665700 >> 172.21.15.27:0/3818665700 conn(0x55ed30532300 msgr2=0x55ed308df0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 -- 172.21.15.27:0/3818665700 shutdown_connections 2024-09-21T07:14:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 -- 172.21.15.27:0/3818665700 wait complete. 2024-09-21T07:14:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 -- start start 2024-09-21T07:14:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.482+0000 7f9d793f5740 1 Processor -- start 2024-09-21T07:14:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d793f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed30962df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d793f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1b1520 con 0x55ed3051d4d0 2024-09-21T07:14:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d793f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1b2990 con 0x55ed305152f0 2024-09-21T07:14:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d793f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1b1bf0 con 0x55ed3053e1b0 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d793f5740 10 client.0 _ll_register_callbacks cb 0x7f9d54009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58986/0 (socket says 172.21.15.27:58986) 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d62a00640 1 -- 172.21.15.27:0/4276296341 learned_addr learned my addr 172.21.15.27:0/4276296341 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d6be00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed30962df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.483+0000 7f9d6be00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d62a00640 1 -- 172.21.15.27:0/4276296341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 msgr2=0x55ed30962df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d62a00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed30962df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d62a00640 1 -- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 msgr2=0x55ed308f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d62a00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d62a00640 1 -- 172.21.15.27:0/4276296341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed30963460 con 0x55ed3051d4d0 2024-09-21T07:14:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d6be00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d62a00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308f4590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9d54010880 tx=0x7f9d540106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943ff651958b4bab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d58c00640 1 -- 172.21.15.27:0/4276296341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54011d90 con 0x55ed3051d4d0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d58c00640 1 -- 172.21.15.27:0/4276296341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9d54022040 con 0x55ed3051d4d0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.484+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d64004e20 con 0x55ed3051d4d0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d58c00640 1 -- 172.21.15.27:0/4276296341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d54010a20 con 0x55ed3051d4d0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 10 client.16050 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed3096a790 con 0x55ed3051d4d0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d58c00640 1 -- 172.21.15.27:0/4276296341 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d64004e20 con 0x55ed3051d4d0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d58c00640 1 -- 172.21.15.27:0/4276296341 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9d5400f790 con 0x55ed3051d4d0 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 20 client.16050 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304d21c0 0x55ed3096cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 4 client.16050 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:14:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.485+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed308560b0 con 0x55ed304d21c0 2024-09-21T07:14:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.486+0000 7f9d6be00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304d21c0 0x55ed3096cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.486+0000 7f9d6be00640 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304d21c0 0x55ed3096cee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e4b2397b52af2 server_cookie=7bc8c15b02fb3c85 in_seq=0 out_seq=0 2024-09-21T07:14:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.486+0000 7f9d58c00640 10 client.16050 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.495+0000 7f9d58c00640 1 -- 172.21.15.27:0/4276296341 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed308560b0 con 0x55ed304d21c0 2024-09-21T07:14:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.495+0000 7f9d58c00640 10 client.16050 handle_command_reply: tid=0 2024-09-21T07:14:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.495+0000 7f9d793f5740 1 client.16050 shutdown 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.495+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 msgr2=0x55ed308f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.495+0000 7f9d793f5740 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308f4590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9d54010880 tx=0x7f9d540106c0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 shutdown_connections 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ed304d21c0 0x55ed3096cee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed3053e1b0 0x55ed30962df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ed3051d4d0 0x55ed308f4590 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 --2- 172.21.15.27:0/4276296341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed305152f0 0x55ed308f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 >> 172.21.15.27:0/4276296341 conn(0x55ed30532300 msgr2=0x55ed308f95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d58c00640 0 client.16050 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 shutdown_connections 2024-09-21T07:14:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 1 -- 172.21.15.27:0/4276296341 wait complete. 2024-09-21T07:14:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d793f5740 20 client.16050 trim_cache size 0 max 16384 2024-09-21T07:14:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:14:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:14:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:14:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:14:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d500b6680 msgr2=0x7f9d500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d72200640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d500b6680 0x7f9d500b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55ed308e18b0 tx=0x7f9d5c10bc40 comp rx=0 tx=0).stop 2024-09-21T07:14:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 msgr2=0x7f9d6c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.496+0000 7f9d72200640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c19c620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005940 tx=0x7f9d5c0040c0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 shutdown_connections 2024-09-21T07:14:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d6c0760a0 0x7f9d6c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d500b6680 0x7f9d500b8b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d6c1100b0 0x7f9d6c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 --2- 172.21.15.27:0/465343136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d6c075750 0x7f9d6c19c620 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 >> 172.21.15.27:0/465343136 conn(0x7f9d6c0ff500 msgr2=0x7f9d6c109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 shutdown_connections 2024-09-21T07:14:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:44.497+0000 7f9d72200640 1 -- 172.21.15.27:0/465343136 wait complete. 2024-09-21T07:14:44.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:44 smithi153 ceph-mon[34747]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T07:14:45.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:44 smithi119 ceph-mon[34519]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T07:14:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:44 smithi027 ceph-mon[30779]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T07:14:46.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:46 smithi153 ceph-mon[34747]: pgmap v430: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-21T07:14:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:46 smithi119 ceph-mon[34519]: pgmap v430: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-21T07:14:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:46 smithi027 ceph-mon[30779]: pgmap v430: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-21T07:14:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:48 smithi153 ceph-mon[34747]: pgmap v431: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-21T07:14:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:48 smithi119 ceph-mon[34519]: pgmap v431: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-21T07:14:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:48 smithi027 ceph-mon[30779]: pgmap v431: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-21T07:14:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:50 smithi153 ceph-mon[34747]: pgmap v432: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-21T07:14:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:50 smithi119 ceph-mon[34519]: pgmap v432: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-21T07:14:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:50 smithi027 ceph-mon[30779]: pgmap v432: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-21T07:14:52.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:52 smithi153 ceph-mon[34747]: pgmap v433: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:14:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:52 smithi119 ceph-mon[34519]: pgmap v433: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:14:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:52 smithi027 ceph-mon[30779]: pgmap v433: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:14:54.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:54 smithi153 ceph-mon[34747]: pgmap v434: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:14:55.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:54 smithi119 ceph-mon[34519]: pgmap v434: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:14:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:54 smithi027 ceph-mon[30779]: pgmap v434: 129 pgs: 129 active+clean; 1.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:14:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:56 smithi153 ceph-mon[34747]: pgmap v435: 129 pgs: 129 active+clean; 1.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T07:14:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:56 smithi119 ceph-mon[34519]: pgmap v435: 129 pgs: 129 active+clean; 1.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T07:14:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:56 smithi027 ceph-mon[30779]: pgmap v435: 129 pgs: 129 active+clean; 1.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.700+0000 7f2a7d800640 1 -- 172.21.15.27:0/2945718358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 msgr2=0x7f2a781125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.700+0000 7f2a7d800640 1 --2- 172.21.15.27:0/2945718358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 0x7f2a781125d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009900 tx=0x7f2a7002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.700+0000 7f2a7d800640 1 -- 172.21.15.27:0/2945718358 shutdown_connections 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.700+0000 7f2a7d800640 1 --2- 172.21.15.27:0/2945718358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a7810b450 0x7f2a7810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.700+0000 7f2a7d800640 1 --2- 172.21.15.27:0/2945718358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 0x7f2a781125d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.700+0000 7f2a7d800640 1 --2- 172.21.15.27:0/2945718358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a78105b20 0x7f2a78105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.701+0000 7f2a7d800640 1 -- 172.21.15.27:0/2945718358 >> 172.21.15.27:0/2945718358 conn(0x7f2a78101560 msgr2=0x7f2a781039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.701+0000 7f2a7d800640 1 -- 172.21.15.27:0/2945718358 shutdown_connections 2024-09-21T07:14:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.701+0000 7f2a7d800640 1 -- 172.21.15.27:0/2945718358 wait complete. 2024-09-21T07:14:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 Processor -- start 2024-09-21T07:14:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 -- start start 2024-09-21T07:14:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a78105b20 0x7f2a7819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7810b450 0x7f2a7819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 0x7f2a781a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78115a60 con 0x7f2a7810bda0 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781158e0 con 0x7f2a7810b450 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.702+0000 7f2a7d800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78115be0 con 0x7f2a78105b20 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a78105b20 0x7f2a7819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a6f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7810b450 0x7f2a7819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a78105b20 0x7f2a7819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58414/0 (socket says 172.21.15.27:58414) 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a6f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7810b450 0x7f2a7819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41260/0 (socket says 172.21.15.27:41260) 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 -- 172.21.15.27:0/3132711146 learned_addr learned my addr 172.21.15.27:0/3132711146 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 0x7f2a781a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 -- 172.21.15.27:0/3132711146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7810b450 msgr2=0x7f2a7819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7810b450 0x7f2a7819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 -- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 msgr2=0x7f2a781a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 0x7f2a781a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 -- 172.21.15.27:0/3132711146 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a781a2390 con 0x7f2a78105b20 2024-09-21T07:14:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a6f400640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7810b450 0x7f2a7819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.703+0000 7f2a77e00640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a78105b20 0x7f2a7819c460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ebc0 tx=0x7f2a6800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.704+0000 7f2a75600640 1 -- 172.21.15.27:0/3132711146 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68010630 con 0x7f2a78105b20 2024-09-21T07:14:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.704+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a781a2680 con 0x7f2a78105b20 2024-09-21T07:14:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.704+0000 7f2a75600640 1 -- 172.21.15.27:0/3132711146 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a68010c80 con 0x7f2a78105b20 2024-09-21T07:14:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.704+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a781a8e10 con 0x7f2a78105b20 2024-09-21T07:14:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.704+0000 7f2a75600640 1 -- 172.21.15.27:0/3132711146 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68018aa0 con 0x7f2a78105b20 2024-09-21T07:14:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 Processor -- start 2024-09-21T07:14:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 -- start start 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f5de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f5e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d133ed36b0 con 0x55d133b902f0 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d133ec2ae0 con 0x55d133bb91b0 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a85e87740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d133ec68f0 con 0x55d133b984d0 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.707+0000 7f2a7ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f5de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f5de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41276/0 (socket says 172.21.15.27:41276) 2024-09-21T07:14:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 -- 172.21.15.27:0/1993458678 learned_addr learned my addr 172.21.15.27:0/1993458678 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a6f400640 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f5e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a75600640 1 -- 172.21.15.27:0/3132711146 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2a680107d0 con 0x7f2a78105b20 2024-09-21T07:14:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 -- 172.21.15.27:0/1993458678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 msgr2=0x55d133f5e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f5e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 -- 172.21.15.27:0/1993458678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 msgr2=0x55d133f5e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 -- 172.21.15.27:0/1993458678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d133e8f780 con 0x55d133bb91b0 2024-09-21T07:14:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.708+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f5de10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a70033000 tx=0x7f2a70032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab3deff231e531f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.709+0000 7f2a6d600640 1 -- 172.21.15.27:0/1993458678 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70035040 con 0x55d133bb91b0 2024-09-21T07:14:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.709+0000 7f2a6d600640 1 -- 172.21.15.27:0/1993458678 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a70004030 con 0x55d133bb91b0 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.709+0000 7f2a6d600640 1 -- 172.21.15.27:0/1993458678 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70005550 con 0x55d133bb91b0 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.709+0000 7f2a85e87740 1 -- 172.21.15.27:0/1993458678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 msgr2=0x55d133f5de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.709+0000 7f2a85e87740 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f5de10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a70033000 tx=0x7f2a70032b20 comp rx=0 tx=0).stop 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a85e87740 1 -- 172.21.15.27:0/1993458678 shutdown_connections 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a85e87740 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f5e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a85e87740 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a85e87740 1 --2- 172.21.15.27:0/1993458678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f5de10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a85e87740 1 -- 172.21.15.27:0/1993458678 >> 172.21.15.27:0/1993458678 conn(0x55d133bad300 msgr2=0x55d133f5bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a75600640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a5c0b6680 0x7f2a5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a85e87740 1 -- 172.21.15.27:0/1993458678 shutdown_connections 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a85e87740 1 -- 172.21.15.27:0/1993458678 wait complete. 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a75600640 1 -- 172.21.15.27:0/3132711146 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f2a680e8120 con 0x7f2a78105b20 2024-09-21T07:14:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.710+0000 7f2a6f400640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a5c0b6680 0x7f2a5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 -- start start 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a6f400640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a5c0b6680 0x7f2a5c0b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2a78109e60 tx=0x7f2a60005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 Processor -- start 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f69530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b984d0 0x55d133f69a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133bb91b0 0x55d133f75850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d133dfaab0 con 0x55d133b984d0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d133e69570 con 0x55d133b902f0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.711+0000 7f2a85e87740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d133f6c530 con 0x55d133bb91b0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a85e87740 10 client.0 _ll_register_callbacks cb 0x55d133e78100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b984d0 0x55d133f69a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f69530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b984d0 0x55d133f69a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60162/0 (socket says 172.21.15.27:60162) 2024-09-21T07:14:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f69530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41284/0 (socket says 172.21.15.27:41284) 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 -- 172.21.15.27:0/823426286 learned_addr learned my addr 172.21.15.27:0/823426286 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133bb91b0 0x55d133f75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 -- 172.21.15.27:0/823426286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133bb91b0 msgr2=0x55d133f75850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133bb91b0 0x55d133f75850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 -- 172.21.15.27:0/823426286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 msgr2=0x55d133f69530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f69530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a6f400640 1 -- 172.21.15.27:0/823426286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d133c24790 con 0x55d133b984d0 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.712+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133bb91b0 0x55d133f75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a6f400640 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b984d0 0x55d133f69a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2a60003640 tx=0x7f2a60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cb46c725fe5c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a66000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a66000640 1 -- 172.21.15.27:0/823426286 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000f890 con 0x55d133b984d0 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a66000640 1 -- 172.21.15.27:0/823426286 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f2a6000fa30 con 0x55d133b984d0 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a70004030 con 0x55d133b984d0 2024-09-21T07:14:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a66000640 1 -- 172.21.15.27:0/823426286 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a6000fc30 con 0x55d133b984d0 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a85e87740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d133c9d620 con 0x55d133b984d0 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.713+0000 7f2a66000640 1 -- 172.21.15.27:0/823426286 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a70004030 con 0x55d133b984d0 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a66000640 1 -- 172.21.15.27:0/823426286 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2a60027070 con 0x55d133b984d0 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a85e87740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a85e87740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a85e87740 20 client.16059 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a85e87740 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b77d80 0x55d133f7b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a85e87740 4 client.16059 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d133ed5600 con 0x55d133b77d80 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.714+0000 7f2a77e00640 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b77d80 0x55d133f7b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.715+0000 7f2a77e00640 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b77d80 0x55d133f7b690 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe287dccb0181b19 server_cookie=76756b70d3eda36b in_seq=0 out_seq=0 2024-09-21T07:14:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.715+0000 7f2a66000640 10 client.16059 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a66000640 1 -- 172.21.15.27:0/823426286 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d133ed5600 con 0x55d133b77d80 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a66000640 10 client.16059 handle_command_reply: tid=0 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 client.16059 shutdown 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b984d0 msgr2=0x55d133f69a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b984d0 0x55d133f69a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2a60003640 tx=0x7f2a60004730 comp rx=0 tx=0).stop 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 shutdown_connections 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b77d80 0x55d133f7b690 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133bb91b0 0x55d133f75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b984d0 0x55d133f69a70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 --2- 172.21.15.27:0/823426286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f69530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 >> 172.21.15.27:0/823426286 conn(0x55d133bad300 msgr2=0x55d133f5ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a66000640 0 client.16059 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 shutdown_connections 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 1 -- 172.21.15.27:0/823426286 wait complete. 2024-09-21T07:14:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.716+0000 7f2a85e87740 20 client.16059 trim_cache size 0 max 16384 2024-09-21T07:14:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 Processor -- start 2024-09-21T07:14:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 -- start start 2024-09-21T07:14:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b5650 con 0x55d133b902f0 2024-09-21T07:14:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b2f20 con 0x55d133bb91b0 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a85e87740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b0d70 con 0x55d133b984d0 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60166/0 (socket says 172.21.15.27:60166) 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a77e00640 1 -- 172.21.15.27:0/3394820230 learned_addr learned my addr 172.21.15.27:0/3394820230 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.732+0000 7f2a6f400640 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f75750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a77e00640 1 -- 172.21.15.27:0/3394820230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 msgr2=0x55d133f75750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a77e00640 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f75750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a77e00640 1 -- 172.21.15.27:0/3394820230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 msgr2=0x55d133f66700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a77e00640 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f66700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a77e00640 1 -- 172.21.15.27:0/3394820230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d133f6a7f0 con 0x55d133b902f0 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f66700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a77e00640 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5ad20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2a68000f80 tx=0x7f2a68018520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5d0548c298a24b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a6cc00640 1 -- 172.21.15.27:0/3394820230 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68010ac0 con 0x55d133b902f0 2024-09-21T07:14:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a6cc00640 1 -- 172.21.15.27:0/3394820230 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a680ad7e0 con 0x55d133b902f0 2024-09-21T07:14:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.733+0000 7f2a6cc00640 1 -- 172.21.15.27:0/3394820230 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680b71b0 con 0x55d133b902f0 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 -- 172.21.15.27:0/3394820230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 msgr2=0x55d133f5ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5ad20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2a68000f80 tx=0x7f2a68018520 comp rx=0 tx=0).stop 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 -- 172.21.15.27:0/3394820230 shutdown_connections 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f75750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133bb91b0 0x55d133f66700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 --2- 172.21.15.27:0/3394820230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133b902f0 0x55d133f5ad20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 -- 172.21.15.27:0/3394820230 >> 172.21.15.27:0/3394820230 conn(0x55d133bad300 msgr2=0x55d133f5bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 -- 172.21.15.27:0/3394820230 shutdown_connections 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.734+0000 7f2a85e87740 1 -- 172.21.15.27:0/3394820230 wait complete. 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 -- start start 2024-09-21T07:14:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 Processor -- start 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f71f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133bb91b0 0x55d133f6f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b13e0 con 0x55d133bb91b0 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b2850 con 0x55d133b902f0 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b1ab0 con 0x55d133b984d0 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.735+0000 7f2a85e87740 10 client.0 _ll_register_callbacks cb 0x7f2a60009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f71f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f71f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41296/0 (socket says 172.21.15.27:41296) 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 -- 172.21.15.27:0/846386772 learned_addr learned my addr 172.21.15.27:0/846386772 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a6f400640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133bb91b0 0x55d133f6f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 -- 172.21.15.27:0/846386772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 msgr2=0x55d133f6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 -- 172.21.15.27:0/846386772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133bb91b0 msgr2=0x55d133f6f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133bb91b0 0x55d133f6f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 -- 172.21.15.27:0/846386772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d133f700e0 con 0x55d133b902f0 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a7ce00640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133bb91b0 0x55d133f6f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a77e00640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f71f70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a680e55d0 tx=0x7f2a680aa1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbe8acc4a1ae3202 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.736+0000 7f2a66000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a66000640 1 -- 172.21.15.27:0/846386772 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680b73b0 con 0x55d133b902f0 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a66000640 1 -- 172.21.15.27:0/846386772 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f2a680acf30 con 0x55d133b902f0 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a680ad7e0 con 0x55d133b902f0 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a66000640 1 -- 172.21.15.27:0/846386772 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a68010ac0 con 0x55d133b902f0 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 10 client.26102 fetch_fsmap learned FSMap version 27 2024-09-21T07:14:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d133fe5f70 con 0x55d133b902f0 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a66000640 1 -- 172.21.15.27:0/846386772 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a680ad7e0 con 0x55d133b902f0 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a66000640 1 -- 172.21.15.27:0/846386772 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2a680cb020 con 0x55d133b902f0 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 10 client.26102 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 20 client.26102 populate_metadata read hostname 'smithi027' 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b4d1c0 0x55d133fe85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 4 client.26102 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.737+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d133ed10b0 con 0x55d133b4d1c0 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.738+0000 7f2a6f400640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b4d1c0 0x55d133fe85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.738+0000 7f2a6f400640 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b4d1c0 0x55d133fe85c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461512a90aa7bee server_cookie=15a805033d229561 in_seq=0 out_seq=0 2024-09-21T07:14:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.738+0000 7f2a66000640 10 client.26102 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a66000640 1 -- 172.21.15.27:0/846386772 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d133ed10b0 con 0x55d133b4d1c0 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a66000640 10 client.26102 handle_command_reply: tid=0 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a85e87740 1 client.26102 shutdown 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 msgr2=0x55d133f71f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a85e87740 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f71f70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a680e55d0 tx=0x7f2a680aa1e0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 shutdown_connections 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a85e87740 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d133b4d1c0 0x55d133fe85c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.751+0000 7f2a85e87740 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d133b984d0 0x55d133f6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a66000640 0 client.26102 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a85e87740 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d133bb91b0 0x55d133f6f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a85e87740 1 --2- 172.21.15.27:0/846386772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d133b902f0 0x55d133f71f70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 >> 172.21.15.27:0/846386772 conn(0x55d133bad300 msgr2=0x55d133f6a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 shutdown_connections 2024-09-21T07:14:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a85e87740 1 -- 172.21.15.27:0/846386772 wait complete. 2024-09-21T07:14:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a85e87740 20 client.26102 trim_cache size 0 max 16384 2024-09-21T07:14:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:14:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:14:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:14:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a5c0b6680 msgr2=0x7f2a5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a7d800640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a5c0b6680 0x7f2a5c0b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2a78109e60 tx=0x7f2a60005f90 comp rx=0 tx=0).stop 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a78105b20 msgr2=0x7f2a7819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.752+0000 7f2a7d800640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a78105b20 0x7f2a7819c460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ebc0 tx=0x7f2a6800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 shutdown_connections 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a5c0b6680 0x7f2a5c0b8b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7810bda0 0x7f2a781a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7810b450 0x7f2a7819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 --2- 172.21.15.27:0/3132711146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a78105b20 0x7f2a7819c460 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 >> 172.21.15.27:0/3132711146 conn(0x7f2a78101560 msgr2=0x7f2a78103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:14:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 shutdown_connections 2024-09-21T07:14:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:14:57.753+0000 7f2a7d800640 1 -- 172.21.15.27:0/3132711146 wait complete. 2024-09-21T07:14:58.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:14:58 smithi153 ceph-mon[34747]: pgmap v436: 129 pgs: 129 active+clean; 1.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-21T07:14:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:14:58 smithi119 ceph-mon[34519]: pgmap v436: 129 pgs: 129 active+clean; 1.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-21T07:14:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:14:58 smithi027 ceph-mon[30779]: pgmap v436: 129 pgs: 129 active+clean; 1.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-21T07:15:00.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:00 smithi153 ceph-mon[34747]: pgmap v437: 129 pgs: 129 active+clean; 1.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-09-21T07:15:00.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:15:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:00 smithi119 ceph-mon[34519]: pgmap v437: 129 pgs: 129 active+clean; 1.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-09-21T07:15:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:15:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:00 smithi027 ceph-mon[30779]: pgmap v437: 129 pgs: 129 active+clean; 1.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-09-21T07:15:01.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:15:01.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:15:01.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:15:01.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:15:02.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:15:02.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:15:02.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:15:02.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:15:02.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:15:02.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:15:03.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:02 smithi119 ceph-mon[34519]: pgmap v438: 129 pgs: 129 active+clean; 1.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-21T07:15:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:02 smithi027 ceph-mon[30779]: pgmap v438: 129 pgs: 129 active+clean; 1.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-21T07:15:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:02 smithi153 ceph-mon[34747]: pgmap v438: 129 pgs: 129 active+clean; 1.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-21T07:15:05.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:04 smithi119 ceph-mon[34519]: pgmap v439: 129 pgs: 129 active+clean; 1.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-21T07:15:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:04 smithi027 ceph-mon[30779]: pgmap v439: 129 pgs: 129 active+clean; 1.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-21T07:15:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:04 smithi153 ceph-mon[34747]: pgmap v439: 129 pgs: 129 active+clean; 1.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-21T07:15:06.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:15:06.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:06 smithi153 ceph-mon[34747]: pgmap v440: 129 pgs: 129 active+clean; 1.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-21T07:15:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:06 smithi119 ceph-mon[34519]: pgmap v440: 129 pgs: 129 active+clean; 1.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-21T07:15:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:06 smithi027 ceph-mon[30779]: pgmap v440: 129 pgs: 129 active+clean; 1.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 -- 172.21.15.27:0/483530489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4078290 msgr2=0x7f5bb4114210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 --2- 172.21.15.27:0/483530489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4078290 0x7f5bb4114210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009900 tx=0x7f5ba002a410 comp rx=0 tx=0).stop 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 -- 172.21.15.27:0/483530489 shutdown_connections 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 --2- 172.21.15.27:0/483530489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4078290 0x7f5bb4114210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 --2- 172.21.15.27:0/483530489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bb4114750 0x7f5bb4116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 --2- 172.21.15.27:0/483530489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb4077940 0x7f5bb4077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 -- 172.21.15.27:0/483530489 >> 172.21.15.27:0/483530489 conn(0x7f5bb4101860 msgr2=0x7f5bb4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 -- 172.21.15.27:0/483530489 shutdown_connections 2024-09-21T07:15:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.687+0000 7f5bb9400640 1 -- 172.21.15.27:0/483530489 wait complete. 2024-09-21T07:15:07.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.688+0000 7f5bb9400640 1 Processor -- start 2024-09-21T07:15:07.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.688+0000 7f5bb9400640 1 -- start start 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bb4077940 0x7f5bb41a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb4078290 0x7f5bb41a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4114750 0x7f5bb41a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb9400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb411a0a0 con 0x7f5bb4078290 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb9400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4119f20 con 0x7f5bb4077940 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb9400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb411a220 con 0x7f5bb4114750 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bb4077940 0x7f5bb41a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb3200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4114750 0x7f5bb41a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bb4077940 0x7f5bb41a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:43400/0 (socket says 172.21.15.27:43400) 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb2800640 1 -- 172.21.15.27:0/795307594 learned_addr learned my addr 172.21.15.27:0/795307594 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.689+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb4078290 0x7f5bb41a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.690+0000 7f5bb3200640 1 -- 172.21.15.27:0/795307594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bb4077940 msgr2=0x7f5bb41a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.690+0000 7f5bb3200640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bb4077940 0x7f5bb41a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.690+0000 7f5bb3200640 1 -- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb4078290 msgr2=0x7f5bb41a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.690+0000 7f5bb3200640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb4078290 0x7f5bb41a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.690+0000 7f5bb3200640 1 -- 172.21.15.27:0/795307594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb41a6a00 con 0x7f5bb4114750 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.690+0000 7f5bb3200640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4114750 0x7f5bb41a62d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009fd0 tx=0x7f5bac00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.690+0000 7f5bab400640 1 -- 172.21.15.27:0/795307594 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00ce20 con 0x7f5bb4114750 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.691+0000 7f5bab400640 1 -- 172.21.15.27:0/795307594 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac005020 con 0x7f5bb4114750 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.691+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb41a6cf0 con 0x7f5bb4114750 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.691+0000 7f5bab400640 1 -- 172.21.15.27:0/795307594 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac010690 con 0x7f5bb4114750 2024-09-21T07:15:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.691+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb41ad450 con 0x7f5bb4114750 2024-09-21T07:15:07.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.694+0000 7f5bc0077740 1 Processor -- start 2024-09-21T07:15:07.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bab400640 1 -- 172.21.15.27:0/795307594 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5bac0054d0 con 0x7f5bb4114750 2024-09-21T07:15:07.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bc0077740 1 -- start start 2024-09-21T07:15:07.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1daa5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1daa62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6de390 0x55af1daa67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bc0077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af1da1cf70 con 0x55af1d6ff070 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bc0077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55af1da0c3a0 con 0x55af1d6de390 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.695+0000 7f5bc0077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af1da101b0 con 0x55af1d6d61b0 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1daa62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1daa62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40596/0 (socket says 172.21.15.27:40596) 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 -- 172.21.15.27:0/367624441 learned_addr learned my addr 172.21.15.27:0/367624441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1daa5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6de390 0x55af1daa67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 -- 172.21.15.27:0/367624441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6de390 msgr2=0x55af1daa67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6de390 0x55af1daa67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 -- 172.21.15.27:0/367624441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 msgr2=0x55af1daa5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1daa5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.696+0000 7f5bb2800640 1 -- 172.21.15.27:0/367624441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1d9d6c90 con 0x55af1d6d61b0 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.697+0000 7f5bb2800640 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1daa62a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009dc0 tx=0x7f5ba4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=498e9a29bc1d1e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.697+0000 7f5bab400640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b980b6700 0x7f5b980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.697+0000 7f5b9f400640 1 -- 172.21.15.27:0/367624441 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4005220 con 0x55af1d6d61b0 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.697+0000 7f5b9f400640 1 -- 172.21.15.27:0/367624441 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba4004c50 con 0x55af1d6d61b0 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.697+0000 7f5b9f400640 1 -- 172.21.15.27:0/367624441 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40197d0 con 0x55af1d6d61b0 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.697+0000 7f5bab400640 1 -- 172.21.15.27:0/795307594 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5bac0e8130 con 0x7f5bb4114750 2024-09-21T07:15:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.697+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b980b6700 0x7f5b980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 -- 172.21.15.27:0/367624441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 msgr2=0x55af1daa62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1daa62a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009dc0 tx=0x7f5ba4011810 comp rx=0 tx=0).stop 2024-09-21T07:15:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b980b6700 0x7f5b980b8c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009230 tx=0x7f5ba0000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 -- 172.21.15.27:0/367624441 shutdown_connections 2024-09-21T07:15:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1daa62a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6de390 0x55af1daa67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 --2- 172.21.15.27:0/367624441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1daa5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 -- 172.21.15.27:0/367624441 >> 172.21.15.27:0/367624441 conn(0x55af1d6f3460 msgr2=0x55af1daa3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.698+0000 7f5bc0077740 1 -- 172.21.15.27:0/367624441 shutdown_connections 2024-09-21T07:15:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.699+0000 7f5bc0077740 1 -- 172.21.15.27:0/367624441 wait complete. 2024-09-21T07:15:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.699+0000 7f5bc0077740 1 -- start start 2024-09-21T07:15:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.699+0000 7f5bc0077740 1 Processor -- start 2024-09-21T07:15:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6d61b0 0x55af1dab1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dab1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1dabd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bc0077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55af1d86d700 con 0x55af1d6ff070 2024-09-21T07:15:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bc0077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55af1d9a7540 con 0x55af1d6d61b0 2024-09-21T07:15:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bc0077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af1dab4540 con 0x55af1d6de390 2024-09-21T07:15:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bc0077740 10 client.0 _ll_register_callbacks cb 0x55af1d9b7430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bb1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dab1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bb1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dab1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40606/0 (socket says 172.21.15.27:40606) 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bb1e00640 1 -- 172.21.15.27:0/3505811641 learned_addr learned my addr 172.21.15.27:0/3505811641 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.700+0000 7f5bb3200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1dabd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5bb2800640 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6d61b0 0x55af1dab1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5bb3200640 1 -- 172.21.15.27:0/3505811641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 msgr2=0x55af1dab1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5bb3200640 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dab1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5bb3200640 1 -- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6d61b0 msgr2=0x55af1dab1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5bb3200640 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6d61b0 0x55af1dab1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5bb3200640 1 -- 172.21.15.27:0/3505811641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1d76a650 con 0x55af1d6ff070 2024-09-21T07:15:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5bb3200640 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1dabd890 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0e54d0 tx=0x7f5bac0ad400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5872de1c58de2b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.701+0000 7f5b93e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.702+0000 7f5b93e00640 1 -- 172.21.15.27:0/3505811641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0ae2d0 con 0x55af1d6ff070 2024-09-21T07:15:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.702+0000 7f5b93e00640 1 -- 172.21.15.27:0/3505811641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5bac0d10d0 con 0x55af1d6ff070 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.702+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af1d9a9b50 con 0x55af1d6ff070 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.702+0000 7f5b93e00640 1 -- 172.21.15.27:0/3505811641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5bac0d51a0 con 0x55af1d6ff070 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.702+0000 7f5bc0077740 10 client.16077 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.702+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af1d9c4910 con 0x55af1d6ff070 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.702+0000 7f5b93e00640 1 -- 172.21.15.27:0/3505811641 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55af1d9a9b50 con 0x55af1d6ff070 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5b93e00640 1 -- 172.21.15.27:0/3505811641 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5bac0ac8a0 con 0x55af1d6ff070 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5bc0077740 10 client.16077 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5bc0077740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5bc0077740 20 client.16077 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d6bd9f0 0x55af1dac37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5bc0077740 4 client.16077 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af1da1eec0 con 0x55af1d6bd9f0 2024-09-21T07:15:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.703+0000 7f5bb2800640 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d6bd9f0 0x55af1dac37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.704+0000 7f5bb2800640 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d6bd9f0 0x55af1dac37b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e622e9ae63af547e server_cookie=e5040795c30f6ec3 in_seq=0 out_seq=0 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.704+0000 7f5b93e00640 10 client.16077 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5b93e00640 1 -- 172.21.15.27:0/3505811641 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af1da1eec0 con 0x55af1d6bd9f0 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5b93e00640 10 client.16077 handle_command_reply: tid=0 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 client.16077 shutdown 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 msgr2=0x55af1dabd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1dabd890 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0e54d0 tx=0x7f5bac0ad400 comp rx=0 tx=0).stop 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 shutdown_connections 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d6bd9f0 0x55af1dac37b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dab1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5b93e00640 0 client.16077 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6ff070 0x55af1dabd890 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3505811641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6d61b0 0x55af1dab1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.705+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 >> 172.21.15.27:0/3505811641 conn(0x55af1d6f3460 msgr2=0x55af1daa8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.706+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 shutdown_connections 2024-09-21T07:15:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.706+0000 7f5bc0077740 1 -- 172.21.15.27:0/3505811641 wait complete. 2024-09-21T07:15:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.706+0000 7f5bc0077740 20 client.16077 trim_cache size 0 max 16384 2024-09-21T07:15:07.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.724+0000 7f5bc0077740 1 Processor -- start 2024-09-21T07:15:07.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bc0077740 1 -- start start 2024-09-21T07:15:07.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6d61b0 0x55af1daa9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dac9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dac9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bc0077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41b9b80 con 0x55af1d6d61b0 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bc0077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41b7450 con 0x55af1d6ff070 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bc0077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41b52a0 con 0x55af1d6de390 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6d61b0 0x55af1daa9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dac9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.725+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6d61b0 0x55af1daa9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50318/0 (socket says 172.21.15.27:50318) 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb2800640 1 -- 172.21.15.27:0/614367572 learned_addr learned my addr 172.21.15.27:0/614367572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:07.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dac9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb3200640 1 -- 172.21.15.27:0/614367572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 msgr2=0x55af1dac9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb3200640 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dac9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb3200640 1 -- 172.21.15.27:0/614367572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6d61b0 msgr2=0x55af1daa9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb3200640 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6d61b0 0x55af1daa9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb3200640 1 -- 172.21.15.27:0/614367572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1daa8490 con 0x55af1d6ff070 2024-09-21T07:15:07.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.726+0000 7f5bb3200640 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dac9540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0aa1a0 tx=0x7f5bac0ad470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f85df8afe2a4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:07.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.727+0000 7f5b9ea00640 1 -- 172.21.15.27:0/614367572 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0b26b0 con 0x55af1d6ff070 2024-09-21T07:15:07.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.727+0000 7f5b9ea00640 1 -- 172.21.15.27:0/614367572 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0b75f0 con 0x55af1d6ff070 2024-09-21T07:15:07.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.727+0000 7f5b9ea00640 1 -- 172.21.15.27:0/614367572 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0b3840 con 0x55af1d6ff070 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 -- 172.21.15.27:0/614367572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 msgr2=0x55af1dac9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dac9540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0aa1a0 tx=0x7f5bac0ad470 comp rx=0 tx=0).stop 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 -- 172.21.15.27:0/614367572 shutdown_connections 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6de390 0x55af1dac9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dac9540 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 --2- 172.21.15.27:0/614367572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6d61b0 0x55af1daa9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 -- 172.21.15.27:0/614367572 >> 172.21.15.27:0/614367572 conn(0x55af1d6f3460 msgr2=0x55af1daa8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 -- 172.21.15.27:0/614367572 shutdown_connections 2024-09-21T07:15:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.728+0000 7f5bc0077740 1 -- 172.21.15.27:0/614367572 wait complete. 2024-09-21T07:15:07.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.729+0000 7f5bc0077740 1 -- start start 2024-09-21T07:15:07.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.729+0000 7f5bc0077740 1 Processor -- start 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1dab7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6de390 0x55af1dab7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bc0077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dab8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bc0077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41b5910 con 0x55af1d6de390 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bc0077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41b6d80 con 0x55af1d6ff070 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bc0077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41b5fe0 con 0x55af1d6d61b0 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bc0077740 10 client.0 _ll_register_callbacks cb 0x7f5bac0ad520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1dab7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bb2800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1dab7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40626/0 (socket says 172.21.15.27:40626) 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bb2800640 1 -- 172.21.15.27:0/3599246225 learned_addr learned my addr 172.21.15.27:0/3599246225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.730+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6de390 0x55af1dab7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb2800640 1 -- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 msgr2=0x55af1dab8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb3200640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dab8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb2800640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dab8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb2800640 1 -- 172.21.15.27:0/3599246225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6de390 msgr2=0x55af1dab7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:07.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb2800640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6de390 0x55af1dab7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb2800640 1 -- 172.21.15.27:0/3599246225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1db2d210 con 0x55af1d6d61b0 2024-09-21T07:15:07.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb3200640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dab8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:15:07.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5bb2800640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1dab7790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009250 tx=0x7f5ba4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ac5464f100c137 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:07.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5b93e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.731+0000 7f5b93e00640 1 -- 172.21.15.27:0/3599246225 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4032060 con 0x55af1d6d61b0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bac0b75f0 con 0x55af1d6d61b0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5b93e00640 1 -- 172.21.15.27:0/3599246225 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5ba40035f0 con 0x55af1d6d61b0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5b93e00640 1 -- 172.21.15.27:0/3599246225 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ba4039050 con 0x55af1d6d61b0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5bc0077740 10 client.26104 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af1db2e400 con 0x55af1d6d61b0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5b93e00640 1 -- 172.21.15.27:0/3599246225 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0b75f0 con 0x55af1d6d61b0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5b93e00640 1 -- 172.21.15.27:0/3599246225 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5ba402c070 con 0x55af1d6d61b0 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.732+0000 7f5bc0077740 10 client.26104 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.733+0000 7f5bc0077740 10 client.26104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.733+0000 7f5bc0077740 20 client.26104 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.733+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d692da0 0x55af1db309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.733+0000 7f5bc0077740 4 client.26104 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:15:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.733+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af1da1a970 con 0x55af1d692da0 2024-09-21T07:15:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.733+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d692da0 0x55af1db309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bb1e00640 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d692da0 0x55af1db309d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9428a5d7a0c9256 server_cookie=d25f3d168bcd9448 in_seq=0 out_seq=0 2024-09-21T07:15:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5b93e00640 10 client.26104 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5b93e00640 1 -- 172.21.15.27:0/3599246225 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55af1da1a970 con 0x55af1d692da0 2024-09-21T07:15:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5b93e00640 10 client.26104 handle_command_reply: tid=0 2024-09-21T07:15:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 client.26104 shutdown 2024-09-21T07:15:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 msgr2=0x55af1dab7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1dab7790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009250 tx=0x7f5ba4009f90 comp rx=0 tx=0).stop 2024-09-21T07:15:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 shutdown_connections 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55af1d692da0 0x55af1db309d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55af1d6ff070 0x55af1dab8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55af1d6de390 0x55af1dab7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5b93e00640 0 client.26104 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 --2- 172.21.15.27:0/3599246225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af1d6d61b0 0x55af1dab7790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.734+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 >> 172.21.15.27:0/3599246225 conn(0x55af1d6f3460 msgr2=0x55af1daa6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.735+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 shutdown_connections 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.735+0000 7f5bc0077740 1 -- 172.21.15.27:0/3599246225 wait complete. 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.735+0000 7f5bc0077740 20 client.26104 trim_cache size 0 max 16384 2024-09-21T07:15:07.739 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:07.742 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b980b6700 msgr2=0x7f5b980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b980b6700 0x7f5b980b8c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009230 tx=0x7f5ba0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4114750 msgr2=0x7f5bb41a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4114750 0x7f5bb41a62d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009fd0 tx=0x7f5bac00ef10 comp rx=0 tx=0).stop 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 shutdown_connections 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bb4114750 0x7f5bb41a62d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b980b6700 0x7f5b980b8c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb4078290 0x7f5bb41a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 --2- 172.21.15.27:0/795307594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bb4077940 0x7f5bb41a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 >> 172.21.15.27:0/795307594 conn(0x7f5bb4101860 msgr2=0x7f5bb4116810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 shutdown_connections 2024-09-21T07:15:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:07.736+0000 7f5bb9400640 1 -- 172.21.15.27:0/795307594 wait complete. 2024-09-21T07:15:08.648 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:15:08.648 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:15:08.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:08 smithi027 ceph-mon[30779]: pgmap v441: 129 pgs: 129 active+clean; 1.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-21T07:15:09.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:08 smithi119 ceph-mon[34519]: pgmap v441: 129 pgs: 129 active+clean; 1.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-21T07:15:09.042 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:08 smithi153 ceph-mon[34747]: pgmap v441: 129 pgs: 129 active+clean; 1.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 -- 172.21.15.27:0/3158768626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 msgr2=0x7faddc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 --2- 172.21.15.27:0/3158768626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 0x7faddc110620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009ca0 tx=0x7fadd402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 -- 172.21.15.27:0/3158768626 shutdown_connections 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 --2- 172.21.15.27:0/3158768626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faddc1094a0 0x7faddc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 --2- 172.21.15.27:0/3158768626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 0x7faddc110620 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 --2- 172.21.15.27:0/3158768626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faddc103b60 0x7faddc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 -- 172.21.15.27:0/3158768626 >> 172.21.15.27:0/3158768626 conn(0x7faddc07b850 msgr2=0x7faddc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 -- 172.21.15.27:0/3158768626 shutdown_connections 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.639+0000 7fade0e00640 1 -- 172.21.15.27:0/3158768626 wait complete. 2024-09-21T07:15:09.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 Processor -- start 2024-09-21T07:15:09.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 -- start start 2024-09-21T07:15:09.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faddc103b60 0x7faddc19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:09.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faddc1094a0 0x7faddc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 0x7faddc1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113ab0 con 0x7faddc109df0 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113930 con 0x7faddc1094a0 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.641+0000 7fade0e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113c30 con 0x7faddc103b60 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faddc103b60 0x7faddc19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadda800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 0x7faddc1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faddc103b60 0x7faddc19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41218/0 (socket says 172.21.15.27:41218) 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd9e00640 1 -- 172.21.15.27:0/3517178696 learned_addr learned my addr 172.21.15.27:0/3517178696 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd3e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faddc1094a0 0x7faddc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd3e00640 1 -- 172.21.15.27:0/3517178696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faddc103b60 msgr2=0x7faddc19c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd3e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faddc103b60 0x7faddc19c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd3e00640 1 -- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 msgr2=0x7faddc1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd3e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 0x7faddc1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd3e00640 1 -- 172.21.15.27:0/3517178696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1a2570 con 0x7faddc1094a0 2024-09-21T07:15:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.642+0000 7fadd3e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faddc1094a0 0x7faddc19cba0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faddc06c9f0 tx=0x7fadc400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:09.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.643+0000 7fadd2a00640 1 -- 172.21.15.27:0/3517178696 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc400cc40 con 0x7faddc1094a0 2024-09-21T07:15:09.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.643+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1a2860 con 0x7faddc1094a0 2024-09-21T07:15:09.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.643+0000 7fadd2a00640 1 -- 172.21.15.27:0/3517178696 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc400cde0 con 0x7faddc1094a0 2024-09-21T07:15:09.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.643+0000 7fadd2a00640 1 -- 172.21.15.27:0/3517178696 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4018760 con 0x7faddc1094a0 2024-09-21T07:15:09.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.643+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a8fc0 con 0x7faddc1094a0 2024-09-21T07:15:09.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.644+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc103f70 con 0x7faddc1094a0 2024-09-21T07:15:09.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.647+0000 7fadd2a00640 1 -- 172.21.15.27:0/3517178696 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fadc4010a90 con 0x7faddc1094a0 2024-09-21T07:15:09.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.648+0000 7fadd2a00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fadc00b6680 0x7fadc00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:09.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.648+0000 7fadd2a00640 1 -- 172.21.15.27:0/3517178696 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fadc40e8220 con 0x7faddc1094a0 2024-09-21T07:15:09.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.649+0000 7fadd9e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fadc00b6680 0x7fadc00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:09.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.649+0000 7fadd9e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fadc00b6680 0x7fadc00b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009c90 tx=0x7fadcc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.650+0000 7fadd2a00640 1 -- 172.21.15.27:0/3517178696 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fadc40b27a0 con 0x7faddc1094a0 2024-09-21T07:15:09.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.887+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faddc19d730 con 0x7faddc1094a0 2024-09-21T07:15:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.889+0000 7fadd2a00640 1 -- 172.21.15.27:0/3517178696 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fadc40b76e0 con 0x7faddc1094a0 2024-09-21T07:15:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:15:09.891 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:15:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:15:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fadc00b6680 msgr2=0x7fadc00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fadc00b6680 0x7fadc00b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009c90 tx=0x7fadcc005f90 comp rx=0 tx=0).stop 2024-09-21T07:15:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faddc1094a0 msgr2=0x7faddc19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faddc1094a0 0x7faddc19cba0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faddc06c9f0 tx=0x7fadc400ee50 comp rx=0 tx=0).stop 2024-09-21T07:15:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 shutdown_connections 2024-09-21T07:15:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fadc00b6680 0x7fadc00b8b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faddc109df0 0x7faddc1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faddc1094a0 0x7faddc19cba0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 --2- 172.21.15.27:0/3517178696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faddc103b60 0x7faddc19c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 >> 172.21.15.27:0/3517178696 conn(0x7faddc07b850 msgr2=0x7faddc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 shutdown_connections 2024-09-21T07:15:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:09.891+0000 7fade0e00640 1 -- 172.21.15.27:0/3517178696 wait complete. 2024-09-21T07:15:10.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:15:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 -- 172.21.15.27:0/2973976215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250103b00 msgr2=0x7fb250103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 --2- 172.21.15.27:0/2973976215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250103b00 0x7fb250103f10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb240009900 tx=0x7fb24002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 -- 172.21.15.27:0/2973976215 shutdown_connections 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 --2- 172.21.15.27:0/2973976215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb250109d80 0x7fb2501105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 --2- 172.21.15.27:0/2973976215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250109430 0x7fb250109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 --2- 172.21.15.27:0/2973976215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250103b00 0x7fb250103f10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 -- 172.21.15.27:0/2973976215 >> 172.21.15.27:0/2973976215 conn(0x7fb2500ff500 msgr2=0x7fb250101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 -- 172.21.15.27:0/2973976215 shutdown_connections 2024-09-21T07:15:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.947+0000 7fb254c00640 1 -- 172.21.15.27:0/2973976215 wait complete. 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.948+0000 7fb254c00640 1 Processor -- start 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb254c00640 1 -- start start 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb254c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250103b00 0x7fb25019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb254c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250109430 0x7fb25019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb254c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb250109d80 0x7fb25019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb254c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb250113a40 con 0x7fb250103b00 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb254c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501138c0 con 0x7fb250109430 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb254c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb250113bc0 con 0x7fb250109d80 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb24de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250103b00 0x7fb25019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb24de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250103b00 0x7fb25019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37248/0 (socket says 172.21.15.27:37248) 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb24de00640 1 -- 172.21.15.27:0/1769058089 learned_addr learned my addr 172.21.15.27:0/1769058089 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.949+0000 7fb24d400640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250109430 0x7fb25019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.950+0000 7fb24e800640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb250109d80 0x7fb25019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.950+0000 7fb24de00640 1 -- 172.21.15.27:0/1769058089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb250109d80 msgr2=0x7fb25019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.950+0000 7fb24de00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb250109d80 0x7fb25019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.950+0000 7fb24de00640 1 -- 172.21.15.27:0/1769058089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250109430 msgr2=0x7fb25019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.950+0000 7fb24de00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250109430 0x7fb25019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.950+0000 7fb24de00640 1 -- 172.21.15.27:0/1769058089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2501a03d0 con 0x7fb250103b00 2024-09-21T07:15:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.950+0000 7fb24de00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250103b00 0x7fb25019a4a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb240005940 tx=0x7fb2400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.951+0000 7fb246a00640 1 -- 172.21.15.27:0/1769058089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24003f030 con 0x7fb250103b00 2024-09-21T07:15:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.951+0000 7fb246a00640 1 -- 172.21.15.27:0/1769058089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2400044c0 con 0x7fb250103b00 2024-09-21T07:15:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.951+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2501a0660 con 0x7fb250103b00 2024-09-21T07:15:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.951+0000 7fb246a00640 1 -- 172.21.15.27:0/1769058089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240035070 con 0x7fb250103b00 2024-09-21T07:15:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.951+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501a6e50 con 0x7fb250103b00 2024-09-21T07:15:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.955+0000 7fb246a00640 1 -- 172.21.15.27:0/1769058089 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb240042030 con 0x7fb250103b00 2024-09-21T07:15:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.955+0000 7fb25bcbb740 1 Processor -- start 2024-09-21T07:15:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb25bcbb740 1 -- start start 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb246a00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2340b6680 0x7fb2340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fc6ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc6f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a94d0 0x55a90fc6f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a90fbe46b0 con 0x55a90f8a94d0 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a90fbd3ae0 con 0x55a90f8ca1b0 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a90fbd78f0 con 0x55a90f8a12f0 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.956+0000 7fb246a00640 1 -- 172.21.15.27:0/1769058089 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb240107e30 con 0x7fb250103b00 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fc6ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc6f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fc6ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40006/0 (socket says 172.21.15.27:40006) 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24de00640 1 -- 172.21.15.27:0/1107333554 learned_addr learned my addr 172.21.15.27:0/1107333554 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc6f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41248/0 (socket says 172.21.15.27:41248) 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24e800640 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a94d0 0x55a90fc6f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24e800640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2340b6680 0x7fb2340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 -- 172.21.15.27:0/1107333554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 msgr2=0x55a90fc6ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fc6ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 -- 172.21.15.27:0/1107333554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a94d0 msgr2=0x55a90fc6f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a94d0 0x55a90fc6f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 -- 172.21.15.27:0/1107333554 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90fba0780 con 0x55a90f8a12f0 2024-09-21T07:15:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.957+0000 7fb24d400640 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc6f170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb25006c9b0 tx=0x7fb23c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea5bf914d2a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.958+0000 7fb23b000640 1 -- 172.21.15.27:0/1107333554 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c004050 con 0x55a90f8a12f0 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.958+0000 7fb24e800640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2340b6680 0x7fb2340b8b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb248009b80 tx=0x7fb248009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.958+0000 7fb23b000640 1 -- 172.21.15.27:0/1107333554 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c0054d0 con 0x55a90f8a12f0 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.958+0000 7fb23b000640 1 -- 172.21.15.27:0/1107333554 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c004c60 con 0x55a90f8a12f0 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 -- 172.21.15.27:0/1107333554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 msgr2=0x55a90fc6f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc6f170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb25006c9b0 tx=0x7fb23c011810 comp rx=0 tx=0).stop 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 -- 172.21.15.27:0/1107333554 shutdown_connections 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc6f170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a94d0 0x55a90fc6f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/1107333554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fc6ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 -- 172.21.15.27:0/1107333554 >> 172.21.15.27:0/1107333554 conn(0x55a90f8be300 msgr2=0x55a90fc6cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 -- 172.21.15.27:0/1107333554 shutdown_connections 2024-09-21T07:15:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 -- 172.21.15.27:0/1107333554 wait complete. 2024-09-21T07:15:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.959+0000 7fb25bcbb740 1 -- start start 2024-09-21T07:15:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 1 Processor -- start 2024-09-21T07:15:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc7a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a94d0 0x55a90fc7a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc7ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a90fb0bab0 con 0x55a90f8ca1b0 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a90fb7a570 con 0x55a90f8a94d0 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a90fc7d3b0 con 0x55a90f8a12f0 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb25bcbb740 10 client.0 _ll_register_callbacks cb 0x55a90fb89100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.960+0000 7fb24e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc7ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc7a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc7ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37278/0 (socket says 172.21.15.27:37278) 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a94d0 0x55a90fc7a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24e800640 1 -- 172.21.15.27:0/2146705792 learned_addr learned my addr 172.21.15.27:0/2146705792 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24de00640 1 -- 172.21.15.27:0/2146705792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a94d0 msgr2=0x55a90fc7a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24de00640 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a94d0 0x55a90fc7a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24de00640 1 -- 172.21.15.27:0/2146705792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 msgr2=0x55a90fc7ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24de00640 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc7ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24de00640 1 -- 172.21.15.27:0/2146705792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90f935790 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb24de00640 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc7a1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2400c7fd0 tx=0x7fb240105740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24698e600a5ef066 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb22f400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb22f400640 1 -- 172.21.15.27:0/2146705792 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240049040 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb22f400640 1 -- 172.21.15.27:0/2146705792 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb2400d7060 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.961+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23c0054d0 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb22f400640 1 -- 172.21.15.27:0/2146705792 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb2400cd040 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 10 client.26122 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a90f9ae620 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb22f400640 1 -- 172.21.15.27:0/2146705792 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c0054d0 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb22f400640 1 -- 172.21.15.27:0/2146705792 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb2400c9220 con 0x55a90f8a12f0 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 10 client.26122 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 10 client.26122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 20 client.26122 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f888d80 0x55a90fc907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 4 client.26122 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.962+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a90fbe6600 con 0x55a90f888d80 2024-09-21T07:15:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.963+0000 7fb24d400640 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f888d80 0x55a90fc907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.963+0000 7fb24d400640 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f888d80 0x55a90fc907b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43d3b954f343452 server_cookie=2d671461df3abc33 in_seq=0 out_seq=0 2024-09-21T07:15:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.963+0000 7fb22f400640 10 client.26122 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb22f400640 1 -- 172.21.15.27:0/2146705792 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a90fbe6600 con 0x55a90f888d80 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb22f400640 10 client.26122 handle_command_reply: tid=0 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 client.26122 shutdown 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 msgr2=0x55a90fc7a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc7a1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2400c7fd0 tx=0x7fb240105740 comp rx=0 tx=0).stop 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 shutdown_connections 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f888d80 0x55a90fc907b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc7ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a94d0 0x55a90fc7a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2146705792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a12f0 0x55a90fc7a1d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 >> 172.21.15.27:0/2146705792 conn(0x55a90f8be300 msgr2=0x55a90fc6fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.964+0000 7fb22f400640 0 client.26122 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.965+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 shutdown_connections 2024-09-21T07:15:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.965+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2146705792 wait complete. 2024-09-21T07:15:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.965+0000 7fb25bcbb740 20 client.26122 trim_cache size 0 max 16384 2024-09-21T07:15:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.980+0000 7fb25bcbb740 1 Processor -- start 2024-09-21T07:15:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.980+0000 7fb25bcbb740 1 -- start start 2024-09-21T07:15:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a12f0 0x55a90fc6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc8a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501b1460 con 0x55a90f8ca1b0 2024-09-21T07:15:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501aed30 con 0x55a90f8a12f0 2024-09-21T07:15:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501acb80 con 0x55a90f8a94d0 2024-09-21T07:15:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb24e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb24e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37290/0 (socket says 172.21.15.27:37290) 2024-09-21T07:15:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb24e800640 1 -- 172.21.15.27:0/2738059442 learned_addr learned my addr 172.21.15.27:0/2738059442 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb24d400640 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc8a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.981+0000 7fb24de00640 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a12f0 0x55a90fc6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb24d400640 1 -- 172.21.15.27:0/2738059442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a12f0 msgr2=0x55a90fc6baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb24d400640 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a12f0 0x55a90fc6baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb24d400640 1 -- 172.21.15.27:0/2738059442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 msgr2=0x55a90fc773a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb24d400640 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc773a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb24d400640 1 -- 172.21.15.27:0/2738059442 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90fc75d00 con 0x55a90f8a94d0 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb24d400640 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc8a160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb23c004880 tx=0x7fb23c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7230bfb674338ba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb23a600640 1 -- 172.21.15.27:0/2738059442 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c004220 con 0x55a90f8a94d0 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb23a600640 1 -- 172.21.15.27:0/2738059442 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c0043c0 con 0x55a90f8a94d0 2024-09-21T07:15:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.982+0000 7fb23a600640 1 -- 172.21.15.27:0/2738059442 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c019690 con 0x55a90f8a94d0 2024-09-21T07:15:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2738059442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 msgr2=0x55a90fc8a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc8a160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb23c004880 tx=0x7fb23c011810 comp rx=0 tx=0).stop 2024-09-21T07:15:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2738059442 shutdown_connections 2024-09-21T07:15:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc8a160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8ca1b0 0x55a90fc773a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/2738059442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8a12f0 0x55a90fc6baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2738059442 >> 172.21.15.27:0/2738059442 conn(0x55a90f8be300 msgr2=0x55a90fc89a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2738059442 shutdown_connections 2024-09-21T07:15:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.983+0000 7fb25bcbb740 1 -- 172.21.15.27:0/2738059442 wait complete. 2024-09-21T07:15:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.984+0000 7fb25bcbb740 1 -- start start 2024-09-21T07:15:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.984+0000 7fb25bcbb740 1 Processor -- start 2024-09-21T07:15:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a12f0 0x55a90fc84ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb25bcbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fcfa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501ad1f0 con 0x55a90f8a12f0 2024-09-21T07:15:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501ae660 con 0x55a90f8ca1b0 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb25bcbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501ad8c0 con 0x55a90f8a94d0 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb25bcbb740 10 client.0 _ll_register_callbacks cb 0x7fb240105170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a12f0 0x55a90fc84ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a12f0 0x55a90fc84ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37304/0 (socket says 172.21.15.27:37304) 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24de00640 1 -- 172.21.15.27:0/704226384 learned_addr learned my addr 172.21.15.27:0/704226384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24d400640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24e800640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fcfa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24d400640 1 -- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 msgr2=0x55a90fcfa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24d400640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fcfa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24d400640 1 -- 172.21.15.27:0/704226384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a12f0 msgr2=0x55a90fc84ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24d400640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a12f0 0x55a90fc84ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.985+0000 7fb24d400640 1 -- 172.21.15.27:0/704226384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90fcfa6c0 con 0x55a90f8a94d0 2024-09-21T07:15:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb24de00640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a12f0 0x55a90fc84ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb24d400640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc85440 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00a010 tx=0x7fb23c0097e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb019e8c82cd927 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb22f400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb22f400640 1 -- 172.21.15.27:0/704226384 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0037f0 con 0x55a90f8a94d0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23c0043c0 con 0x55a90f8a94d0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb22f400640 1 -- 172.21.15.27:0/704226384 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb23c003990 con 0x55a90f8a94d0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb22f400640 1 -- 172.21.15.27:0/704226384 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb23c003bd0 con 0x55a90f8a94d0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb25bcbb740 10 client.26128 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.986+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a90fcfb800 con 0x55a90f8a94d0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb22f400640 1 -- 172.21.15.27:0/704226384 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c0043c0 con 0x55a90f8a94d0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb22f400640 1 -- 172.21.15.27:0/704226384 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb23c019430 con 0x55a90f8a94d0 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb25bcbb740 10 client.26128 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb25bcbb740 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb25bcbb740 20 client.26128 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f85e1c0 0x55a90fcfdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb25bcbb740 4 client.26128 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:15:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a90fbe20b0 con 0x55a90f85e1c0 2024-09-21T07:15:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.987+0000 7fb24de00640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f85e1c0 0x55a90fcfdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.988+0000 7fb22f400640 10 client.26128 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:10.988+0000 7fb24de00640 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f85e1c0 0x55a90fcfdc90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd932bdc9bff9488 server_cookie=532641e19cc6cb92 in_seq=0 out_seq=0 2024-09-21T07:15:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb22f400640 1 -- 172.21.15.27:0/704226384 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a90fbe20b0 con 0x55a90f85e1c0 2024-09-21T07:15:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb22f400640 10 client.26128 handle_command_reply: tid=0 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 client.26128 shutdown 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 msgr2=0x55a90fc85440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc85440 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00a010 tx=0x7fb23c0097e0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 shutdown_connections 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a90f85e1c0 0x55a90fcfdc90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a90f8a94d0 0x55a90fc85440 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a90f8ca1b0 0x55a90fcfa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 --2- 172.21.15.27:0/704226384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a90f8a12f0 0x55a90fc84ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb22f400640 0 client.26128 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.000+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 >> 172.21.15.27:0/704226384 conn(0x55a90f8be300 msgr2=0x55a90fc6b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.001+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 shutdown_connections 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.001+0000 7fb25bcbb740 1 -- 172.21.15.27:0/704226384 wait complete. 2024-09-21T07:15:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.001+0000 7fb25bcbb740 20 client.26128 trim_cache size 0 max 16384 2024-09-21T07:15:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:15:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:15:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:15:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.001+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2340b6680 msgr2=0x7fb2340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.001+0000 7fb254c00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2340b6680 0x7fb2340b8b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb248009b80 tx=0x7fb248009500 comp rx=0 tx=0).stop 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.001+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250103b00 msgr2=0x7fb25019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.001+0000 7fb254c00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250103b00 0x7fb25019a4a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb240005940 tx=0x7fb2400040c0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 shutdown_connections 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb250109d80 0x7fb25019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2340b6680 0x7fb2340b8b90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb250109430 0x7fb25019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 --2- 172.21.15.27:0/1769058089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb250103b00 0x7fb25019a4a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 >> 172.21.15.27:0/1769058089 conn(0x7fb2500ff500 msgr2=0x7fb250108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 shutdown_connections 2024-09-21T07:15:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:11.002+0000 7fb254c00640 1 -- 172.21.15.27:0/1769058089 wait complete. 2024-09-21T07:15:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:10 smithi119 ceph-mon[34519]: pgmap v442: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-21T07:15:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:10 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3517178696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:10 smithi027 ceph-mon[30779]: pgmap v442: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-21T07:15:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:10 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3517178696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:11.135 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:10 smithi153 ceph-mon[34747]: pgmap v442: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-21T07:15:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:10 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3517178696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:11.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 -- 172.21.15.27:0/389788173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 msgr2=0x7ff600103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 --2- 172.21.15.27:0/389788173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600103f70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009900 tx=0x7ff5f002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 -- 172.21.15.27:0/389788173 shutdown_connections 2024-09-21T07:15:11.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 --2- 172.21.15.27:0/389788173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff600109df0 0x7ff600110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 --2- 172.21.15.27:0/389788173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6001094a0 0x7ff6001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 --2- 172.21.15.27:0/389788173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600103f70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 -- 172.21.15.27:0/389788173 >> 172.21.15.27:0/389788173 conn(0x7ff60007b850 msgr2=0x7ff60007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 -- 172.21.15.27:0/389788173 shutdown_connections 2024-09-21T07:15:11.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.733+0000 7ff606200640 1 -- 172.21.15.27:0/389788173 wait complete. 2024-09-21T07:15:11.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.734+0000 7ff606200640 1 Processor -- start 2024-09-21T07:15:11.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.735+0000 7ff606200640 1 -- start start 2024-09-21T07:15:11.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.735+0000 7ff606200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.735+0000 7ff606200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6001094a0 0x7ff600075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.735+0000 7ff606200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600109df0 0x7ff600076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.735+0000 7ff606200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff600113ab0 con 0x7ff6001094a0 2024-09-21T07:15:11.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.735+0000 7ff606200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff600113930 con 0x7ff600109df0 2024-09-21T07:15:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.735+0000 7ff606200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff600113c30 con 0x7ff600103b60 2024-09-21T07:15:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6001094a0 0x7ff600075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41288/0 (socket says 172.21.15.27:41288) 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 -- 172.21.15.27:0/1055327439 learned_addr learned my addr 172.21.15.27:0/1055327439 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600109df0 0x7ff600076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 -- 172.21.15.27:0/1055327439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600109df0 msgr2=0x7ff600076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600109df0 0x7ff600076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 -- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6001094a0 msgr2=0x7ff600075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6001094a0 0x7ff600075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5ff400640 1 -- 172.21.15.27:0/1055327439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6001a87a0 con 0x7ff600103b60 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.736+0000 7ff5fea00640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6001094a0 0x7ff600075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:11.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.737+0000 7ff5ff400640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0005940 tx=0x7ff5f00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.737+0000 7ff5f7e00640 1 -- 172.21.15.27:0/1055327439 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f003f030 con 0x7ff600103b60 2024-09-21T07:15:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.738+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6001a8a30 con 0x7ff600103b60 2024-09-21T07:15:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.738+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6001a8f60 con 0x7ff600103b60 2024-09-21T07:15:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.738+0000 7ff5f7e00640 1 -- 172.21.15.27:0/1055327439 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f00044c0 con 0x7ff600103b60 2024-09-21T07:15:11.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.738+0000 7ff5f7e00640 1 -- 172.21.15.27:0/1055327439 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0035070 con 0x7ff600103b60 2024-09-21T07:15:11.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.741+0000 7ff60ce06740 1 Processor -- start 2024-09-21T07:15:11.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.741+0000 7ff60ce06740 1 -- start start 2024-09-21T07:15:11.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.741+0000 7ff5f7e00640 1 -- 172.21.15.27:0/1055327439 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff5f0042030 con 0x7ff600103b60 2024-09-21T07:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.741+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39381a070 0x55a393bc0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bc10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bc1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff60ce06740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a393b37fb0 con 0x55a3937f11b0 2024-09-21T07:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff60ce06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a393b273e0 con 0x55a39381a070 2024-09-21T07:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff60ce06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a393b2b1f0 con 0x55a3937f9390 2024-09-21T07:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bc10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bc10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37330/0 (socket says 172.21.15.27:37330) 2024-09-21T07:15:11.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 -- 172.21.15.27:0/410320561 learned_addr learned my addr 172.21.15.27:0/410320561 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:11.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39381a070 0x55a393bc0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5fea00640 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bc1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 -- 172.21.15.27:0/410320561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 msgr2=0x55a393bc1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bc1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 -- 172.21.15.27:0/410320561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39381a070 msgr2=0x55a393bc0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39381a070 0x55a393bc0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.742+0000 7ff5ff400640 1 -- 172.21.15.27:0/410320561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a393af1cd0 con 0x55a3937f11b0 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.743+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39381a070 0x55a393bc0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.743+0000 7ff5ff400640 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bc10e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0105130 tx=0x7ff5f01100e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9027f5931e3994e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:11.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.743+0000 7ff5ebe00640 1 -- 172.21.15.27:0/410320561 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0110d20 con 0x55a3937f11b0 2024-09-21T07:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.743+0000 7ff5ebe00640 1 -- 172.21.15.27:0/410320561 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f0115070 con 0x55a3937f11b0 2024-09-21T07:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.743+0000 7ff5ebe00640 1 -- 172.21.15.27:0/410320561 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0118680 con 0x55a3937f11b0 2024-09-21T07:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff5f7e00640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5e40b6700 0x7ff5e40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff60ce06740 1 -- 172.21.15.27:0/410320561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 msgr2=0x55a393bc10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff60ce06740 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bc10e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0105130 tx=0x7ff5f01100e0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff5f7e00640 1 -- 172.21.15.27:0/1055327439 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff5f0107e50 con 0x7ff600103b60 2024-09-21T07:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5e40b6700 0x7ff5e40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff60ce06740 1 -- 172.21.15.27:0/410320561 shutdown_connections 2024-09-21T07:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff60ce06740 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bc1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff60ce06740 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bc10e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff60ce06740 1 --2- 172.21.15.27:0/410320561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39381a070 0x55a393bc0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.744+0000 7ff60ce06740 1 -- 172.21.15.27:0/410320561 >> 172.21.15.27:0/410320561 conn(0x55a39380e460 msgr2=0x55a393bbea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.745+0000 7ff60ce06740 1 -- 172.21.15.27:0/410320561 shutdown_connections 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.745+0000 7ff60ce06740 1 -- 172.21.15.27:0/410320561 wait complete. 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.745+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5e40b6700 0x7ff5e40b8c10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009d50 tx=0x7ff5f8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.745+0000 7ff60ce06740 1 -- start start 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 1 Processor -- start 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bcc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39381a070 0x55a393bd83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a393988700 con 0x55a3937f9390 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a393ac2580 con 0x55a3937f11b0 2024-09-21T07:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a393bcf140 con 0x55a39381a070 2024-09-21T07:15:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.746+0000 7ff60ce06740 10 client.0 _ll_register_callbacks cb 0x55a393ad2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bcc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bcc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40074/0 (socket says 172.21.15.27:40074) 2024-09-21T07:15:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 -- 172.21.15.27:0/2403414897 learned_addr learned my addr 172.21.15.27:0/2403414897 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5fea00640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39381a070 0x55a393bd83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 -- 172.21.15.27:0/2403414897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39381a070 msgr2=0x55a393bd83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39381a070 0x55a393bd83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 -- 172.21.15.27:0/2403414897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 msgr2=0x55a393bccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 -- 172.21.15.27:0/2403414897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a393885650 con 0x55a3937f11b0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39381a070 0x55a393bd83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5ff400640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bcc7d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0031440 tx=0x7ff5f00c9d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d717c8e558cb59c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.747+0000 7ff5e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff5e8c00640 1 -- 172.21.15.27:0/2403414897 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0049070 con 0x55a3937f11b0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a393ac4b90 con 0x55a3937f11b0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff5e8c00640 1 -- 172.21.15.27:0/2403414897 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff5f00d7080 con 0x55a3937f11b0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff5e8c00640 1 -- 172.21.15.27:0/2403414897 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5f00c9f40 con 0x55a3937f11b0 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff60ce06740 10 client.26153 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:11.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a393adf950 con 0x55a3937f11b0 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff5e8c00640 1 -- 172.21.15.27:0/2403414897 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a393ac4b90 con 0x55a3937f11b0 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.748+0000 7ff5e8c00640 1 -- 172.21.15.27:0/2403414897 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff5f0110b20 con 0x55a3937f11b0 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.749+0000 7ff60ce06740 10 client.26153 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.749+0000 7ff60ce06740 10 client.26153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.749+0000 7ff60ce06740 20 client.26153 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.749+0000 7ff60ce06740 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a3937d89f0 0x55a393bde260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.749+0000 7ff60ce06740 4 client.26153 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.749+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a393b39f00 con 0x55a3937d89f0 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.749+0000 7ff5fea00640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a3937d89f0 0x55a393bde260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.750+0000 7ff5fea00640 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a3937d89f0 0x55a393bde260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d82b302c96462eb server_cookie=87fe92ae14a7c756 in_seq=0 out_seq=0 2024-09-21T07:15:11.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.750+0000 7ff5e8c00640 10 client.26153 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.750+0000 7ff5e8c00640 1 -- 172.21.15.27:0/2403414897 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a393b39f00 con 0x55a3937d89f0 2024-09-21T07:15:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.750+0000 7ff5e8c00640 10 client.26153 handle_command_reply: tid=0 2024-09-21T07:15:11.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.750+0000 7ff60ce06740 1 client.26153 shutdown 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 msgr2=0x55a393bcc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bcc7d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0031440 tx=0x7ff5f00c9d90 comp rx=0 tx=0).stop 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 shutdown_connections 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a3937d89f0 0x55a393bde260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39381a070 0x55a393bd83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff5e8c00640 0 client.26153 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 --2- 172.21.15.27:0/2403414897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bcc7d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 >> 172.21.15.27:0/2403414897 conn(0x55a39380e460 msgr2=0x55a393bc3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 shutdown_connections 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 1 -- 172.21.15.27:0/2403414897 wait complete. 2024-09-21T07:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.751+0000 7ff60ce06740 20 client.26153 trim_cache size 0 max 16384 2024-09-21T07:15:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.765+0000 7ff60ce06740 1 Processor -- start 2024-09-21T07:15:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.765+0000 7ff60ce06740 1 -- start start 2024-09-21T07:15:11.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.765+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39380e460 0x55a393bc7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.765+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bbe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff60ce06740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b5890 con 0x55a3937f11b0 2024-09-21T07:15:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff60ce06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b3160 con 0x55a39380e460 2024-09-21T07:15:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff60ce06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b0fb0 con 0x55a3937f9390 2024-09-21T07:15:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39380e460 0x55a393bc7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37348/0 (socket says 172.21.15.27:37348) 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5fea00640 1 -- 172.21.15.27:0/225092702 learned_addr learned my addr 172.21.15.27:0/225092702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bbe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5ff400640 1 -- 172.21.15.27:0/225092702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 msgr2=0x55a393bbe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5ff400640 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bbe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5ff400640 1 -- 172.21.15.27:0/225092702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 msgr2=0x55a393bce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5ff400640 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5ff400640 1 -- 172.21.15.27:0/225092702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a393bc2e90 con 0x55a39380e460 2024-09-21T07:15:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.766+0000 7ff5fea00640 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.767+0000 7ff5ff400640 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39380e460 0x55a393bc7d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff5f011ae70 tx=0x7ff5f002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79a563cacc14038 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.767+0000 7ff5eb400640 1 -- 172.21.15.27:0/225092702 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0049070 con 0x55a39380e460 2024-09-21T07:15:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.767+0000 7ff5eb400640 1 -- 172.21.15.27:0/225092702 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f0115030 con 0x55a39380e460 2024-09-21T07:15:11.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.767+0000 7ff5eb400640 1 -- 172.21.15.27:0/225092702 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0105680 con 0x55a39380e460 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 -- 172.21.15.27:0/225092702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39380e460 msgr2=0x55a393bc7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39380e460 0x55a393bc7d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff5f011ae70 tx=0x7ff5f002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 -- 172.21.15.27:0/225092702 shutdown_connections 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3937f9390 0x55a393bbe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f11b0 0x55a393bce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 --2- 172.21.15.27:0/225092702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a39380e460 0x55a393bc7d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 -- 172.21.15.27:0/225092702 >> 172.21.15.27:0/225092702 conn(0x55a393bd8b50 msgr2=0x55a393bcdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 -- 172.21.15.27:0/225092702 shutdown_connections 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.768+0000 7ff60ce06740 1 -- 172.21.15.27:0/225092702 wait complete. 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 -- start start 2024-09-21T07:15:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 Processor -- start 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bd08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bd0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39380e460 0x55a393c50270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b1620 con 0x55a3937f9390 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b2a90 con 0x55a3937f11b0 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b1cf0 con 0x55a39380e460 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.769+0000 7ff60ce06740 10 client.0 _ll_register_callbacks cb 0x7ff5f010bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bd08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5fea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bd0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bd08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40082/0 (socket says 172.21.15.27:40082) 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 -- 172.21.15.27:0/1756668906 learned_addr learned my addr 172.21.15.27:0/1756668906 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ffe00640 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39380e460 0x55a393c50270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 -- 172.21.15.27:0/1756668906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39380e460 msgr2=0x55a393c50270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39380e460 0x55a393c50270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 -- 172.21.15.27:0/1756668906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 msgr2=0x55a393bd0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bd0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 -- 172.21.15.27:0/1756668906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a393c507b0 con 0x55a3937f11b0 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5ff400640 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bd08b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5f010b3a0 tx=0x7ff5f00d24a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39449d6faa66906b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.770+0000 7ff5e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff5e8c00640 1 -- 172.21.15.27:0/1756668906 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0049070 con 0x55a3937f11b0 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff5e8c00640 1 -- 172.21.15.27:0/1756668906 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff5f00d6e80 con 0x55a3937f11b0 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f0115030 con 0x55a3937f11b0 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff5e8c00640 1 -- 172.21.15.27:0/1756668906 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5f00c9060 con 0x55a3937f11b0 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 10 client.26159 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a393c507b0 con 0x55a3937f11b0 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff5e8c00640 1 -- 172.21.15.27:0/1756668906 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f0115030 con 0x55a3937f11b0 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff5e8c00640 1 -- 172.21.15.27:0/1756668906 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff5f0031ac0 con 0x55a3937f11b0 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 10 client.26159 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 10 client.26159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 20 client.26159 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a39381a070 0x55a393c53c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 4 client.26159 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:15:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.771+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a393b359b0 con 0x55a39381a070 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.772+0000 7ff5fea00640 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a39381a070 0x55a393c53c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.772+0000 7ff5fea00640 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a39381a070 0x55a393c53c60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ae59f39c6db1202 server_cookie=cc90f0f7cd37b05d in_seq=0 out_seq=0 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.772+0000 7ff5e8c00640 10 client.26159 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff5e8c00640 1 -- 172.21.15.27:0/1756668906 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a393b359b0 con 0x55a39381a070 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff5e8c00640 10 client.26159 handle_command_reply: tid=0 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 client.26159 shutdown 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 msgr2=0x55a393bd08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bd08b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5f010b3a0 tx=0x7ff5f00d24a0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 shutdown_connections 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55a39381a070 0x55a393c53c60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a39380e460 0x55a393c50270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a3937f9390 0x55a393bd0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 --2- 172.21.15.27:0/1756668906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3937f11b0 0x55a393bd08b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 >> 172.21.15.27:0/1756668906 conn(0x55a393bd8b50 msgr2=0x55a393bcb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff5e8c00640 0 client.26159 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 shutdown_connections 2024-09-21T07:15:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 1 -- 172.21.15.27:0/1756668906 wait complete. 2024-09-21T07:15:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.773+0000 7ff60ce06740 20 client.26159 trim_cache size 0 max 16384 2024-09-21T07:15:11.778 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:15:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5e40b6700 msgr2=0x7ff5e40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5e40b6700 0x7ff5e40b8c10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009d50 tx=0x7ff5f8009500 comp rx=0 tx=0).stop 2024-09-21T07:15:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 msgr2=0x7ff600075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0005940 tx=0x7ff5f00040c0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 shutdown_connections 2024-09-21T07:15:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5e40b6700 0x7ff5e40b8c10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600109df0 0x7ff600076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6001094a0 0x7ff600075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 --2- 172.21.15.27:0/1055327439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff600103b60 0x7ff600075710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 >> 172.21.15.27:0/1055327439 conn(0x7ff60007b850 msgr2=0x7ff600107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 shutdown_connections 2024-09-21T07:15:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:11.774+0000 7ff606200640 1 -- 172.21.15.27:0/1055327439 wait complete. 2024-09-21T07:15:12.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:15:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:12 smithi119 ceph-mon[34519]: pgmap v443: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-21T07:15:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:12 smithi027 ceph-mon[30779]: pgmap v443: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-21T07:15:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:12 smithi153 ceph-mon[34747]: pgmap v443: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-21T07:15:13.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 -- 172.21.15.27:0/3836123281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0109df0 msgr2=0x7f27c0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 --2- 172.21.15.27:0/3836123281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0109df0 0x7f27c0110620 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009ca0 tx=0x7f27b802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 -- 172.21.15.27:0/3836123281 shutdown_connections 2024-09-21T07:15:13.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 --2- 172.21.15.27:0/3836123281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 0x7f27c01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 --2- 172.21.15.27:0/3836123281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0109df0 0x7f27c0110620 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 --2- 172.21.15.27:0/3836123281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27c0103b60 0x7f27c0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 -- 172.21.15.27:0/3836123281 >> 172.21.15.27:0/3836123281 conn(0x7f27c007b850 msgr2=0x7f27c007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:13.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 -- 172.21.15.27:0/3836123281 shutdown_connections 2024-09-21T07:15:13.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.735+0000 7f27c7200640 1 -- 172.21.15.27:0/3836123281 wait complete. 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.736+0000 7f27c7200640 1 Processor -- start 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.736+0000 7f27c7200640 1 -- start start 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27c7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0103b60 0x7f27c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27c7200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 0x7f27c0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27c7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27c0109df0 0x7f27c0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27c7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0113b00 con 0x7f27c0103b60 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27c7200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0113980 con 0x7f27c0109df0 2024-09-21T07:15:13.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27c7200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0113c80 con 0x7f27c01094a0 2024-09-21T07:15:13.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27b7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 0x7f27c0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0103b60 0x7f27c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27b7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 0x7f27c0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41348/0 (socket says 172.21.15.27:41348) 2024-09-21T07:15:13.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27b7e00640 1 -- 172.21.15.27:0/2499283924 learned_addr learned my addr 172.21.15.27:0/2499283924 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:13.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.737+0000 7f27c5200640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27c0109df0 0x7f27c0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.738+0000 7f27b7e00640 1 -- 172.21.15.27:0/2499283924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27c0109df0 msgr2=0x7f27c0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.738+0000 7f27b7e00640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27c0109df0 0x7f27c0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.738+0000 7f27b7e00640 1 -- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0103b60 msgr2=0x7f27c0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.738+0000 7f27b7e00640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0103b60 0x7f27c0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.738+0000 7f27b7e00640 1 -- 172.21.15.27:0/2499283924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c01a8710 con 0x7f27c01094a0 2024-09-21T07:15:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.738+0000 7f27bfe00640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0103b60 0x7f27c0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.738+0000 7f27b7e00640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 0x7f27c0075c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27a800d750 tx=0x7f27a800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:13.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.741+0000 7f27bd600640 1 -- 172.21.15.27:0/2499283924 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8017070 con 0x7f27c01094a0 2024-09-21T07:15:13.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.741+0000 7f27bd600640 1 -- 172.21.15.27:0/2499283924 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a80054d0 con 0x7f27c01094a0 2024-09-21T07:15:13.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.741+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c01a8a00 con 0x7f27c01094a0 2024-09-21T07:15:13.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.741+0000 7f27bd600640 1 -- 172.21.15.27:0/2499283924 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8004c30 con 0x7f27c01094a0 2024-09-21T07:15:13.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.741+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c01a8f40 con 0x7f27c01094a0 2024-09-21T07:15:13.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.744+0000 7f27cde7e740 1 Processor -- start 2024-09-21T07:15:13.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27cde7e740 1 -- start start 2024-09-21T07:15:13.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27bd600640 1 -- 172.21.15.27:0/2499283924 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f27a8004dd0 con 0x7f27c01094a0 2024-09-21T07:15:13.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6fe2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fe32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6fe3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27cde7e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5612b6f59fb0 con 0x5612b6c131b0 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27cde7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5612b6f493e0 con 0x5612b6c3c070 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27cde7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5612b6f4d1f0 con 0x5612b6c1b390 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27b7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6fe3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fe32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fe32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37386/0 (socket says 172.21.15.27:37386) 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27bfe00640 1 -- 172.21.15.27:0/1202037891 learned_addr learned my addr 172.21.15.27:0/1202037891 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27b7e00640 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6fe3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41362/0 (socket says 172.21.15.27:41362) 2024-09-21T07:15:13.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.745+0000 7f27c5200640 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6fe2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.746+0000 7f27c5200640 1 -- 172.21.15.27:0/1202037891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 msgr2=0x5612b6fe3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.746+0000 7f27c5200640 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6fe3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.746+0000 7f27c5200640 1 -- 172.21.15.27:0/1202037891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 msgr2=0x5612b6fe32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.746+0000 7f27c5200640 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fe32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.746+0000 7f27c5200640 1 -- 172.21.15.27:0/1202037891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b6f13cd0 con 0x5612b6c3c070 2024-09-21T07:15:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.746+0000 7f27bfe00640 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fe32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.746+0000 7f27c5200640 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6fe2da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005e30 tx=0x7f27b8009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b38ea411e6d8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:13.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.747+0000 7f27bd600640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f27a40b6700 0x7f27a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.747+0000 7f27b5600640 1 -- 172.21.15.27:0/1202037891 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8030800 con 0x5612b6c3c070 2024-09-21T07:15:13.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.747+0000 7f27bd600640 1 -- 172.21.15.27:0/2499283924 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f27a80e73a0 con 0x7f27c01094a0 2024-09-21T07:15:13.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.747+0000 7f27b5600640 1 -- 172.21.15.27:0/1202037891 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b8036030 con 0x5612b6c3c070 2024-09-21T07:15:13.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.747+0000 7f27b5600640 1 -- 172.21.15.27:0/1202037891 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004440 con 0x5612b6c3c070 2024-09-21T07:15:13.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.747+0000 7f27bfe00640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f27a40b6700 0x7f27a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 -- 172.21.15.27:0/1202037891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 msgr2=0x5612b6fe2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6fe2da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005e30 tx=0x7f27b8009950 comp rx=0 tx=0).stop 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27bfe00640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f27a40b6700 0x7f27a40b8c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5612b6fe1a20 tx=0x7f27b0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 -- 172.21.15.27:0/1202037891 shutdown_connections 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6fe3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fe32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 --2- 172.21.15.27:0/1202037891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6fe2da0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 -- 172.21.15.27:0/1202037891 >> 172.21.15.27:0/1202037891 conn(0x5612b6c30460 msgr2=0x5612b6fe0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 -- 172.21.15.27:0/1202037891 shutdown_connections 2024-09-21T07:15:13.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.748+0000 7f27cde7e740 1 -- 172.21.15.27:0/1202037891 wait complete. 2024-09-21T07:15:13.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.749+0000 7f27cde7e740 1 -- start start 2024-09-21T07:15:13.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.749+0000 7f27cde7e740 1 Processor -- start 2024-09-21T07:15:13.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.749+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.749+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6ffa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27cde7e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5612b6daa700 con 0x5612b6c131b0 2024-09-21T07:15:13.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27cde7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5612b6ee4580 con 0x5612b6c3c070 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27cde7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5612b6ff1520 con 0x5612b6c1b390 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27cde7e740 10 client.0 _ll_register_callbacks cb 0x5612b6ef4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fee5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37398/0 (socket says 172.21.15.27:37398) 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27bfe00640 1 -- 172.21.15.27:0/829060334 learned_addr learned my addr 172.21.15.27:0/829060334 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27b7e00640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27c5200640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6ffa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27b7e00640 1 -- 172.21.15.27:0/829060334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 msgr2=0x5612b6ffa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27b7e00640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6ffa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27b7e00640 1 -- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 msgr2=0x5612b6fee5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27b7e00640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fee5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27b7e00640 1 -- 172.21.15.27:0/829060334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b6ca7650 con 0x5612b6c1b390 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.750+0000 7f27bfe00640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fee5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:13.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27b7e00640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feeb20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f27a80e4740 tx=0x7f27a800dee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff7e4dab7c5b1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27ae000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27ae000640 1 -- 172.21.15.27:0/829060334 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8020070 con 0x5612b6c1b390 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27ae000640 1 -- 172.21.15.27:0/829060334 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f27a80940f0 con 0x5612b6c1b390 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612b6ee6b90 con 0x5612b6c1b390 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27ae000640 1 -- 172.21.15.27:0/829060334 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27a80af050 con 0x5612b6c1b390 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27cde7e740 10 client.26158 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612b6f01950 con 0x5612b6c1b390 2024-09-21T07:15:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.751+0000 7f27ae000640 1 -- 172.21.15.27:0/829060334 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5612b6ee6b90 con 0x5612b6c1b390 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27ae000640 1 -- 172.21.15.27:0/829060334 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27a80a8840 con 0x5612b6c1b390 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27cde7e740 10 client.26158 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27cde7e740 10 client.26158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27cde7e740 20 client.26158 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27cde7e740 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bfa9f0 0x5612b7000590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27cde7e740 4 client.26158 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612b6f5bf00 con 0x5612b6bfa9f0 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.752+0000 7f27bfe00640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bfa9f0 0x5612b7000590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.753+0000 7f27bfe00640 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bfa9f0 0x5612b7000590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef58d507a4b43250 server_cookie=c66b9b1a6b7d02e in_seq=0 out_seq=0 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.753+0000 7f27ae000640 10 client.26158 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27ae000640 1 -- 172.21.15.27:0/829060334 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612b6f5bf00 con 0x5612b6bfa9f0 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27ae000640 10 client.26158 handle_command_reply: tid=0 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 client.26158 shutdown 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 msgr2=0x5612b6feeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feeb20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f27a80e4740 tx=0x7f27a800dee0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 shutdown_connections 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bfa9f0 0x5612b7000590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feeb20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b6ffa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 --2- 172.21.15.27:0/829060334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6fee5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27ae000640 0 client.26158 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 >> 172.21.15.27:0/829060334 conn(0x5612b6c30460 msgr2=0x5612b6fe5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 shutdown_connections 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 1 -- 172.21.15.27:0/829060334 wait complete. 2024-09-21T07:15:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.754+0000 7f27cde7e740 20 client.26158 trim_cache size 0 max 16384 2024-09-21T07:15:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.768+0000 7f27cde7e740 1 Processor -- start 2024-09-21T07:15:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27cde7e740 1 -- start start 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c131b0 0x5612b6fe44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c3c070 0x5612b6febd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27cde7e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01b5750 con 0x5612b6c3c070 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27cde7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01b3020 con 0x5612b6c131b0 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27cde7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01b0e70 con 0x5612b6c1b390 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27c5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c3c070 0x5612b6febd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27c5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c3c070 0x5612b6febd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37412/0 (socket says 172.21.15.27:37412) 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27c5200640 1 -- 172.21.15.27:0/504956395 learned_addr learned my addr 172.21.15.27:0/504956395 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.769+0000 7f27bfe00640 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c131b0 0x5612b6fe44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27b7e00640 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27bfe00640 1 -- 172.21.15.27:0/504956395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 msgr2=0x5612b6feb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27bfe00640 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27bfe00640 1 -- 172.21.15.27:0/504956395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c3c070 msgr2=0x5612b6febd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27bfe00640 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c3c070 0x5612b6febd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27bfe00640 1 -- 172.21.15.27:0/504956395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b6fe5360 con 0x5612b6c131b0 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27b7e00640 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27c5200640 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c3c070 0x5612b6febd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.770+0000 7f27bfe00640 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c131b0 0x5612b6fe44e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5612b6fef360 tx=0x7f27b001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81790d1e5eeb070 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27b4c00640 1 -- 172.21.15.27:0/504956395 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b001da20 con 0x5612b6c131b0 2024-09-21T07:15:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27b4c00640 1 -- 172.21.15.27:0/504956395 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b00054d0 con 0x5612b6c131b0 2024-09-21T07:15:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27b4c00640 1 -- 172.21.15.27:0/504956395 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004c40 con 0x5612b6c131b0 2024-09-21T07:15:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27cde7e740 1 -- 172.21.15.27:0/504956395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c131b0 msgr2=0x5612b6fe44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27cde7e740 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c131b0 0x5612b6fe44e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5612b6fef360 tx=0x7f27b001b000 comp rx=0 tx=0).stop 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27cde7e740 1 -- 172.21.15.27:0/504956395 shutdown_connections 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27cde7e740 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6feb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27cde7e740 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c3c070 0x5612b6febd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27cde7e740 1 --2- 172.21.15.27:0/504956395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c131b0 0x5612b6fe44e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.771+0000 7f27cde7e740 1 -- 172.21.15.27:0/504956395 >> 172.21.15.27:0/504956395 conn(0x5612b6c30460 msgr2=0x5612b6fe3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.772+0000 7f27cde7e740 1 -- 172.21.15.27:0/504956395 shutdown_connections 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.772+0000 7f27cde7e740 1 -- 172.21.15.27:0/504956395 wait complete. 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.772+0000 7f27cde7e740 1 -- start start 2024-09-21T07:15:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.772+0000 7f27cde7e740 1 Processor -- start 2024-09-21T07:15:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6ff2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6ff3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27cde7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b7069f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27cde7e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01b14e0 con 0x5612b6c131b0 2024-09-21T07:15:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27cde7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01b2950 con 0x5612b6c3c070 2024-09-21T07:15:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27cde7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01b1bb0 con 0x5612b6c1b390 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27cde7e740 10 client.0 _ll_register_callbacks cb 0x7f27a8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6ff2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6ff2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37414/0 (socket says 172.21.15.27:37414) 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27bfe00640 1 -- 172.21.15.27:0/4208945451 learned_addr learned my addr 172.21.15.27:0/4208945451 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27c5200640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b7069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.773+0000 7f27b7e00640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6ff3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27bfe00640 1 -- 172.21.15.27:0/4208945451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 msgr2=0x5612b6ff3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27bfe00640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6ff3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27bfe00640 1 -- 172.21.15.27:0/4208945451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 msgr2=0x5612b7069f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27bfe00640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b7069f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27bfe00640 1 -- 172.21.15.27:0/4208945451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b706a460 con 0x5612b6c131b0 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27c5200640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b7069f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27bfe00640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6ff2e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f27b0019040 tx=0x7f27b0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da2236828fe2cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27ae000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27ae000640 1 -- 172.21.15.27:0/4208945451 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b003e060 con 0x5612b6c131b0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27ae000640 1 -- 172.21.15.27:0/4208945451 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f27b001db60 con 0x5612b6c131b0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.774+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b00054d0 con 0x5612b6c131b0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27ae000640 1 -- 172.21.15.27:0/4208945451 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27b0045050 con 0x5612b6c131b0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 10 client.16143 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612b706b670 con 0x5612b6c131b0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27ae000640 1 -- 172.21.15.27:0/4208945451 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b00054d0 con 0x5612b6c131b0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27ae000640 1 -- 172.21.15.27:0/4208945451 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27b002f070 con 0x5612b6c131b0 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 10 client.16143 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 10 client.16143 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 20 client.16143 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bcfda0 0x5612b706dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 4 client.16143 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:15:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.775+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612b6f579b0 con 0x5612b6bcfda0 2024-09-21T07:15:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.776+0000 7f27b7e00640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bcfda0 0x5612b706dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.776+0000 7f27b7e00640 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bcfda0 0x5612b706dd80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=744137b4fa5d78d0 server_cookie=4f73d6c1be627087 in_seq=0 out_seq=0 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.776+0000 7f27ae000640 10 client.16143 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27ae000640 1 -- 172.21.15.27:0/4208945451 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612b6f579b0 con 0x5612b6bcfda0 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27ae000640 10 client.16143 handle_command_reply: tid=0 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 client.16143 shutdown 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 msgr2=0x5612b6ff2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6ff2e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f27b0019040 tx=0x7f27b0005710 comp rx=0 tx=0).stop 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 shutdown_connections 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5612b6bcfda0 0x5612b706dd80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612b6c1b390 0x5612b6ff3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612b6c3c070 0x5612b7069f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 --2- 172.21.15.27:0/4208945451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5612b6c131b0 0x5612b6ff2e10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 >> 172.21.15.27:0/4208945451 conn(0x5612b6c30460 msgr2=0x5612b6fe6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27ae000640 0 client.16143 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 shutdown_connections 2024-09-21T07:15:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 1 -- 172.21.15.27:0/4208945451 wait complete. 2024-09-21T07:15:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.777+0000 7f27cde7e740 20 client.16143 trim_cache size 0 max 16384 2024-09-21T07:15:13.782 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:15:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f27a40b6700 msgr2=0x7f27a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f27a40b6700 0x7f27a40b8c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5612b6fe1a20 tx=0x7f27b0005f90 comp rx=0 tx=0).stop 2024-09-21T07:15:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 msgr2=0x7f27c0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 0x7f27c0075c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27a800d750 tx=0x7f27a800dc10 comp rx=0 tx=0).stop 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 shutdown_connections 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c01094a0 0x7f27c0075c50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f27a40b6700 0x7f27a40b8c10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27c0109df0 0x7f27c0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 --2- 172.21.15.27:0/2499283924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27c0103b60 0x7f27c0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 >> 172.21.15.27:0/2499283924 conn(0x7f27c007b850 msgr2=0x7f27c0107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 shutdown_connections 2024-09-21T07:15:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:13.778+0000 7f27c7200640 1 -- 172.21.15.27:0/2499283924 wait complete. 2024-09-21T07:15:14.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:15:14.991 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:15:14.992 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T07:15:14.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:15:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:14 smithi119 ceph-mon[34519]: pgmap v444: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 24 op/s 2024-09-21T07:15:15.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:14 smithi027 ceph-mon[30779]: pgmap v444: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 24 op/s 2024-09-21T07:15:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:14 smithi153 ceph-mon[34747]: pgmap v444: 129 pgs: 129 active+clean; 1.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 24 op/s 2024-09-21T07:15:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:15.856 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 -- 172.21.15.27:0/683431409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c1094a0 msgr2=0x7f415c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 --2- 172.21.15.27:0/683431409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c1094a0 0x7f415c1098b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4148009900 tx=0x7f414802a410 comp rx=0 tx=0).stop 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 -- 172.21.15.27:0/683431409 shutdown_connections 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 --2- 172.21.15.27:0/683431409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f415c109df0 0x7f415c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 --2- 172.21.15.27:0/683431409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c1094a0 0x7f415c1098b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 --2- 172.21.15.27:0/683431409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f415c103b60 0x7f415c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 -- 172.21.15.27:0/683431409 >> 172.21.15.27:0/683431409 conn(0x7f415c07b850 msgr2=0x7f415c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 -- 172.21.15.27:0/683431409 shutdown_connections 2024-09-21T07:15:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.880+0000 7f4161400640 1 -- 172.21.15.27:0/683431409 wait complete. 2024-09-21T07:15:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.882+0000 7f4161400640 1 Processor -- start 2024-09-21T07:15:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.882+0000 7f4161400640 1 -- start start 2024-09-21T07:15:15.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f4161400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c103b60 0x7f415c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f4161400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f415c1094a0 0x7f415c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f4161400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f415c109df0 0x7f415c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f4161400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f415c113ab0 con 0x7f415c103b60 2024-09-21T07:15:15.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f4161400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f415c113930 con 0x7f415c109df0 2024-09-21T07:15:15.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f4161400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f415c113c30 con 0x7f415c1094a0 2024-09-21T07:15:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f415a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c103b60 0x7f415c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f415a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c103b60 0x7f415c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37434/0 (socket says 172.21.15.27:37434) 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.883+0000 7f415a800640 1 -- 172.21.15.27:0/3616345258 learned_addr learned my addr 172.21.15.27:0/3616345258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f4159e00640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f415c1094a0 0x7f415c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f415b200640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f415c109df0 0x7f415c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f4159e00640 1 -- 172.21.15.27:0/3616345258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f415c109df0 msgr2=0x7f415c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f4159e00640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f415c109df0 0x7f415c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f4159e00640 1 -- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c103b60 msgr2=0x7f415c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f4159e00640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c103b60 0x7f415c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f4159e00640 1 -- 172.21.15.27:0/3616345258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c1a8820 con 0x7f415c1094a0 2024-09-21T07:15:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.884+0000 7f415a800640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c103b60 0x7f415c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.885+0000 7f4159e00640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f415c1094a0 0x7f415c075c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f414802a930 tx=0x7f4148031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.885+0000 7f4153400640 1 -- 172.21.15.27:0/3616345258 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414803f070 con 0x7f415c1094a0 2024-09-21T07:15:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.885+0000 7f4153400640 1 -- 172.21.15.27:0/3616345258 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4148004490 con 0x7f415c1094a0 2024-09-21T07:15:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.885+0000 7f4153400640 1 -- 172.21.15.27:0/3616345258 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148035070 con 0x7f415c1094a0 2024-09-21T07:15:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.885+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c1a8ab0 con 0x7f415c1094a0 2024-09-21T07:15:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.885+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c1a8f40 con 0x7f415c1094a0 2024-09-21T07:15:15.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.888+0000 7f4168035740 1 Processor -- start 2024-09-21T07:15:15.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f4168035740 1 -- start start 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f4168035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb0845cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f4168035740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb0808d1b0 0x55bb0845d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f4168035740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb0845d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f4168035740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bb083d3fb0 con 0x55bb08095390 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f4168035740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bb083c33e0 con 0x55bb0808d1b0 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f4168035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb083c71f0 con 0x55bb080b6070 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.889+0000 7f415b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb0845cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb0845cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41418/0 (socket says 172.21.15.27:41418) 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415b200640 1 -- 172.21.15.27:0/1561524432 learned_addr learned my addr 172.21.15.27:0/1561524432 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415a800640 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb0808d1b0 0x55bb0845d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415b200640 1 -- 172.21.15.27:0/1561524432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb0808d1b0 msgr2=0x55bb0845d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415b200640 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb0808d1b0 0x55bb0845d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415b200640 1 -- 172.21.15.27:0/1561524432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 msgr2=0x55bb0845d820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415b200640 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb0845d820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415b200640 1 -- 172.21.15.27:0/1561524432 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0838dcd0 con 0x55bb080b6070 2024-09-21T07:15:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f415a800640 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb0808d1b0 0x55bb0845d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.890+0000 7f4153400640 1 -- 172.21.15.27:0/3616345258 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f414803b460 con 0x7f415c1094a0 2024-09-21T07:15:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.891+0000 7f415b200640 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb0845cda0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4154009f50 tx=0x7f41540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6841197bf2dd8507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.891+0000 7f4147400640 1 -- 172.21.15.27:0/1561524432 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154004d20 con 0x55bb080b6070 2024-09-21T07:15:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.891+0000 7f4147400640 1 -- 172.21.15.27:0/1561524432 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4154004050 con 0x55bb080b6070 2024-09-21T07:15:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.891+0000 7f4147400640 1 -- 172.21.15.27:0/1561524432 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154005280 con 0x55bb080b6070 2024-09-21T07:15:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 -- 172.21.15.27:0/1561524432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 msgr2=0x55bb0845cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb0845cda0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4154009f50 tx=0x7f41540118d0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 -- 172.21.15.27:0/1561524432 shutdown_connections 2024-09-21T07:15:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb0845d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb0808d1b0 0x55bb0845d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 --2- 172.21.15.27:0/1561524432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb0845cda0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 -- 172.21.15.27:0/1561524432 >> 172.21.15.27:0/1561524432 conn(0x55bb080aa460 msgr2=0x55bb0845ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.892+0000 7f4168035740 1 -- 172.21.15.27:0/1561524432 shutdown_connections 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.893+0000 7f4168035740 1 -- 172.21.15.27:0/1561524432 wait complete. 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.893+0000 7f4153400640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41400b6590 0x7f41400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.893+0000 7f4153400640 1 -- 172.21.15.27:0/3616345258 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4148107f50 con 0x7f415c1094a0 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.893+0000 7f415a800640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41400b6590 0x7f41400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.893+0000 7f4168035740 1 -- start start 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f4168035740 1 Processor -- start 2024-09-21T07:15:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f415a800640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41400b6590 0x7f41400b8aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55bb0845ba20 tx=0x7f414c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f4168035740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb0808d1b0 0x55bb084685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f4168035740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb08095390 0x55bb08468ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f4168035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb08474870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f4168035740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bb08224700 con 0x55bb0808d1b0 2024-09-21T07:15:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f4168035740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bb0835e580 con 0x55bb08095390 2024-09-21T07:15:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.894+0000 7f4168035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb0846b580 con 0x55bb080b6070 2024-09-21T07:15:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f4168035740 10 client.0 _ll_register_callbacks cb 0x55bb0836e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb0808d1b0 0x55bb084685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb0808d1b0 0x55bb084685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37446/0 (socket says 172.21.15.27:37446) 2024-09-21T07:15:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415a800640 1 -- 172.21.15.27:0/574535629 learned_addr learned my addr 172.21.15.27:0/574535629 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415b200640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb08474870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f4159e00640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb08095390 0x55bb08468ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415a800640 1 -- 172.21.15.27:0/574535629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 msgr2=0x55bb08474870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415a800640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb08474870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415a800640 1 -- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb08095390 msgr2=0x55bb08468ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.895+0000 7f415a800640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb08095390 0x55bb08468ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.896+0000 7f415a800640 1 -- 172.21.15.27:0/574535629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb08121650 con 0x55bb0808d1b0 2024-09-21T07:15:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.896+0000 7f4159e00640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb08095390 0x55bb08468ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.896+0000 7f415a800640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb0808d1b0 0x55bb084685e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f414c003670 tx=0x7f414c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01065bf28d1de2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.896+0000 7f413be00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.896+0000 7f413be00640 1 -- 172.21.15.27:0/574535629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c00f8b0 con 0x55bb0808d1b0 2024-09-21T07:15:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.896+0000 7f413be00640 1 -- 172.21.15.27:0/574535629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f414c00fa50 con 0x55bb0808d1b0 2024-09-21T07:15:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.897+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb08360b90 con 0x55bb0808d1b0 2024-09-21T07:15:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.897+0000 7f413be00640 1 -- 172.21.15.27:0/574535629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f414c00fc90 con 0x55bb0808d1b0 2024-09-21T07:15:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.897+0000 7f4168035740 10 client.16152 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.897+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb0837b950 con 0x55bb0808d1b0 2024-09-21T07:15:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.898+0000 7f413be00640 1 -- 172.21.15.27:0/574535629 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bb08360b90 con 0x55bb0808d1b0 2024-09-21T07:15:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f413be00640 1 -- 172.21.15.27:0/574535629 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f414c027070 con 0x55bb0808d1b0 2024-09-21T07:15:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f4168035740 10 client.16152 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f4168035740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f4168035740 20 client.16152 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f4168035740 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb080749f0 0x55bb0847a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f4168035740 4 client.16152 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb083d5f00 con 0x55bb080749f0 2024-09-21T07:15:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.899+0000 7f4159e00640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb080749f0 0x55bb0847a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.900+0000 7f413be00640 10 client.16152 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.900+0000 7f4159e00640 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb080749f0 0x55bb0847a6f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b749ad032ed796b6 server_cookie=1827363c8c16fd6a in_seq=0 out_seq=0 2024-09-21T07:15:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f413be00640 1 -- 172.21.15.27:0/574535629 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb083d5f00 con 0x55bb080749f0 2024-09-21T07:15:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f413be00640 10 client.16152 handle_command_reply: tid=0 2024-09-21T07:15:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 client.16152 shutdown 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb0808d1b0 msgr2=0x55bb084685e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb0808d1b0 0x55bb084685e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f414c003670 tx=0x7f414c004730 comp rx=0 tx=0).stop 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 shutdown_connections 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb080749f0 0x55bb0847a6f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb080b6070 0x55bb08474870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb08095390 0x55bb08468ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 --2- 172.21.15.27:0/574535629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb0808d1b0 0x55bb084685e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f413be00640 0 client.16152 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.901+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 >> 172.21.15.27:0/574535629 conn(0x55bb080aa460 msgr2=0x55bb0845ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.902+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 shutdown_connections 2024-09-21T07:15:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.902+0000 7f4168035740 1 -- 172.21.15.27:0/574535629 wait complete. 2024-09-21T07:15:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.902+0000 7f4168035740 20 client.16152 trim_cache size 0 max 16384 2024-09-21T07:15:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 Processor -- start 2024-09-21T07:15:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 -- start start 2024-09-21T07:15:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb084605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb08474770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08049da0 0x55bb08474cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1b59a0 con 0x55bb08095390 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1b3270 con 0x55bb080b6070 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.927+0000 7f4168035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1b10c0 con 0x55bb08049da0 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08049da0 0x55bb08474cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb084605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08049da0 0x55bb08474cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41428/0 (socket says 172.21.15.27:41428) 2024-09-21T07:15:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 -- 172.21.15.27:0/1136961907 learned_addr learned my addr 172.21.15.27:0/1136961907 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f4159e00640 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb08474770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 -- 172.21.15.27:0/1136961907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 msgr2=0x55bb08474770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb08474770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 -- 172.21.15.27:0/1136961907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 msgr2=0x55bb084605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb084605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415b200640 1 -- 172.21.15.27:0/1136961907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0845f090 con 0x55bb08049da0 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f415a800640 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb084605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.928+0000 7f4159e00640 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb08474770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.929+0000 7f415b200640 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08049da0 0x55bb08474cb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4154009270 tx=0x7f4154003a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759d34ce7178b014 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.929+0000 7f4146a00640 1 -- 172.21.15.27:0/1136961907 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154004050 con 0x55bb08049da0 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.929+0000 7f4146a00640 1 -- 172.21.15.27:0/1136961907 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4154004780 con 0x55bb08049da0 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.929+0000 7f4146a00640 1 -- 172.21.15.27:0/1136961907 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154005290 con 0x55bb08049da0 2024-09-21T07:15:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.930+0000 7f4168035740 1 -- 172.21.15.27:0/1136961907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08049da0 msgr2=0x55bb08474cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.930+0000 7f4168035740 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08049da0 0x55bb08474cb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4154009270 tx=0x7f4154003a30 comp rx=0 tx=0).stop 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.930+0000 7f4168035740 1 -- 172.21.15.27:0/1136961907 shutdown_connections 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.930+0000 7f4168035740 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08049da0 0x55bb08474cb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.930+0000 7f4168035740 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb08474770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.930+0000 7f4168035740 1 --2- 172.21.15.27:0/1136961907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08095390 0x55bb084605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.930+0000 7f4168035740 1 -- 172.21.15.27:0/1136961907 >> 172.21.15.27:0/1136961907 conn(0x55bb080aa460 msgr2=0x55bb0845fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.931+0000 7f4168035740 1 -- 172.21.15.27:0/1136961907 shutdown_connections 2024-09-21T07:15:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.931+0000 7f4168035740 1 -- 172.21.15.27:0/1136961907 wait complete. 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.931+0000 7f4168035740 1 -- start start 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.931+0000 7f4168035740 1 Processor -- start 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4168035740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08049da0 0x55bb08470af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4168035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08095390 0x55bb08471090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4168035740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb0846d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4168035740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1b1730 con 0x55bb08049da0 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4168035740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1b2ba0 con 0x55bb080b6070 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4168035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1b1e00 con 0x55bb08095390 2024-09-21T07:15:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4168035740 10 client.0 _ll_register_callbacks cb 0x7f414c004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f415a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08049da0 0x55bb08470af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f415a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08049da0 0x55bb08470af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37464/0 (socket says 172.21.15.27:37464) 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f415a800640 1 -- 172.21.15.27:0/1135122217 learned_addr learned my addr 172.21.15.27:0/1135122217 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.932+0000 7f4159e00640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08095390 0x55bb08471090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f415b200640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb0846d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f415a800640 1 -- 172.21.15.27:0/1135122217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08095390 msgr2=0x55bb08471090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f415a800640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08095390 0x55bb08471090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f415a800640 1 -- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 msgr2=0x55bb0846d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f415a800640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb0846d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f415a800640 1 -- 172.21.15.27:0/1135122217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0846df90 con 0x55bb08049da0 2024-09-21T07:15:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f4159e00640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08095390 0x55bb08471090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f415a800640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08049da0 0x55bb08470af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f414c019040 tx=0x7f414c00fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb3ec2a13dc61db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.933+0000 7f413be00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f413be00640 1 -- 172.21.15.27:0/1135122217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c00fe30 con 0x55bb08049da0 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f413be00640 1 -- 172.21.15.27:0/1135122217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f414c029de0 con 0x55bb08049da0 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4154004780 con 0x55bb08049da0 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f413be00640 1 -- 172.21.15.27:0/1135122217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f414c040050 con 0x55bb08049da0 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f4168035740 10 client.16158 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb084e5b20 con 0x55bb08049da0 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f413be00640 1 -- 172.21.15.27:0/1135122217 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4154004780 con 0x55bb08049da0 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.934+0000 7f413be00640 1 -- 172.21.15.27:0/1135122217 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f414c027070 con 0x55bb08049da0 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.935+0000 7f4168035740 10 client.16158 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.935+0000 7f4168035740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.935+0000 7f4168035740 20 client.16158 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.935+0000 7f4168035740 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb084e6290 0x55bb084e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.935+0000 7f4168035740 4 client.16158 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.935+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb083d19b0 con 0x55bb084e6290 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.935+0000 7f4159e00640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb084e6290 0x55bb084e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4159e00640 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb084e6290 0x55bb084e86b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26eabfdb60093185 server_cookie=989070f6039446a in_seq=0 out_seq=0 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f413be00640 10 client.16158 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f413be00640 1 -- 172.21.15.27:0/1135122217 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb083d19b0 con 0x55bb084e6290 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f413be00640 10 client.16158 handle_command_reply: tid=0 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 client.16158 shutdown 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08049da0 msgr2=0x55bb08470af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08049da0 0x55bb08470af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f414c019040 tx=0x7f414c00fb60 comp rx=0 tx=0).stop 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 shutdown_connections 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bb084e6290 0x55bb084e86b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb08095390 0x55bb08471090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f413be00640 0 client.16158 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb080b6070 0x55bb0846d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 --2- 172.21.15.27:0/1135122217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb08049da0 0x55bb08470af0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.936+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 >> 172.21.15.27:0/1135122217 conn(0x55bb080aa460 msgr2=0x55bb08460630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.937+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 shutdown_connections 2024-09-21T07:15:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.937+0000 7f4168035740 1 -- 172.21.15.27:0/1135122217 wait complete. 2024-09-21T07:15:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.937+0000 7f4168035740 20 client.16158 trim_cache size 0 max 16384 2024-09-21T07:15:15.941 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:15:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41400b6590 msgr2=0x7f41400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41400b6590 0x7f41400b8aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55bb0845ba20 tx=0x7f414c005f90 comp rx=0 tx=0).stop 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f415c1094a0 msgr2=0x7f415c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f415c1094a0 0x7f415c075c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f414802a930 tx=0x7f4148031a20 comp rx=0 tx=0).stop 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 shutdown_connections 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f415c1094a0 0x7f415c075c50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41400b6590 0x7f41400b8aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f415c109df0 0x7f415c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 --2- 172.21.15.27:0/3616345258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f415c103b60 0x7f415c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 >> 172.21.15.27:0/3616345258 conn(0x7f415c07b850 msgr2=0x7f415c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 shutdown_connections 2024-09-21T07:15:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:15.938+0000 7f4161400640 1 -- 172.21.15.27:0/3616345258 wait complete. 2024-09-21T07:15:16.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 -- 172.21.15.27:0/3650203395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 msgr2=0x7f13c8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:16.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 --2- 172.21.15.27:0/3650203395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 0x7f13c8110470 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009ca0 tx=0x7f13c002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:15:16.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 -- 172.21.15.27:0/3650203395 shutdown_connections 2024-09-21T07:15:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 --2- 172.21.15.27:0/3650203395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13c81092f0 0x7f13c8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 --2- 172.21.15.27:0/3650203395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 0x7f13c8110470 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 --2- 172.21.15.27:0/3650203395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 0x7f13c8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 -- 172.21.15.27:0/3650203395 >> 172.21.15.27:0/3650203395 conn(0x7f13c80ff440 msgr2=0x7f13c8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 -- 172.21.15.27:0/3650203395 shutdown_connections 2024-09-21T07:15:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.801+0000 7f13c7200640 1 -- 172.21.15.27:0/3650203395 wait complete. 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 Processor -- start 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 -- start start 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 0x7f13c819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13c81092f0 0x7f13c819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 0x7f13c81a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8113900 con 0x7f13c8109c40 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8113780 con 0x7f13c81039b0 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.802+0000 7f13c7200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8113a80 con 0x7f13c81092f0 2024-09-21T07:15:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 0x7f13c819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13c81092f0 0x7f13c819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 0x7f13c819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40202/0 (socket says 172.21.15.27:40202) 2024-09-21T07:15:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 0x7f13c81a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13c81092f0 0x7f13c819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41468/0 (socket says 172.21.15.27:41468) 2024-09-21T07:15:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 -- 172.21.15.27:0/2540140994 learned_addr learned my addr 172.21.15.27:0/2540140994 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 -- 172.21.15.27:0/2540140994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13c81092f0 msgr2=0x7f13c819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13c81092f0 0x7f13c819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 -- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 msgr2=0x7f13c81a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 0x7f13c81a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.803+0000 7f13c5e00640 1 -- 172.21.15.27:0/2540140994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c81a2420 con 0x7f13c81039b0 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.804+0000 7f13c5e00640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 0x7f13c819c4b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f13b800ea60 tx=0x7f13b800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.804+0000 7f13bea00640 1 -- 172.21.15.27:0/2540140994 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b800cde0 con 0x7f13c81039b0 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.804+0000 7f13bea00640 1 -- 172.21.15.27:0/2540140994 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b80035f0 con 0x7f13c81039b0 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.804+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c81a2710 con 0x7f13c81039b0 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.804+0000 7f13bea00640 1 -- 172.21.15.27:0/2540140994 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8010690 con 0x7f13c81039b0 2024-09-21T07:15:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.804+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a8e70 con 0x7f13c81039b0 2024-09-21T07:15:16.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.805+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c8103dc0 con 0x7f13c81039b0 2024-09-21T07:15:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.808+0000 7f13bea00640 1 -- 172.21.15.27:0/2540140994 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f13b80037a0 con 0x7f13c81039b0 2024-09-21T07:15:16.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.809+0000 7f13bea00640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13ac0b6680 0x7f13ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:16.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.809+0000 7f13bea00640 1 -- 172.21.15.27:0/2540140994 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f13b80e8060 con 0x7f13c81039b0 2024-09-21T07:15:16.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.809+0000 7f13c5400640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13ac0b6680 0x7f13ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:16.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.810+0000 7f13c5400640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13ac0b6680 0x7f13ac0b8b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f13c8105cf0 tx=0x7f13b4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:16.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:16.811+0000 7f13bea00640 1 -- 172.21.15.27:0/2540140994 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13b80ae200 con 0x7f13c81039b0 2024-09-21T07:15:16.894 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:16 smithi027 ceph-mon[30779]: pgmap v445: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-09-21T07:15:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:16 smithi119 ceph-mon[34519]: pgmap v445: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-09-21T07:15:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.050+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13c819d5e0 con 0x7f13c81039b0 2024-09-21T07:15:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.051+0000 7f13bea00640 1 -- 172.21.15.27:0/2540140994 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f13b80b25e0 con 0x7f13c81039b0 2024-09-21T07:15:17.053 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:15:17.053 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:15:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:15:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13ac0b6680 msgr2=0x7f13ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13ac0b6680 0x7f13ac0b8b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f13c8105cf0 tx=0x7f13b4005f90 comp rx=0 tx=0).stop 2024-09-21T07:15:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 msgr2=0x7f13c819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 0x7f13c819c4b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f13b800ea60 tx=0x7f13b800ef20 comp rx=0 tx=0).stop 2024-09-21T07:15:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 shutdown_connections 2024-09-21T07:15:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13c81092f0 0x7f13c819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13ac0b6680 0x7f13ac0b8b90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13c8109c40 0x7f13c81a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 --2- 172.21.15.27:0/2540140994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81039b0 0x7f13c819c4b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 >> 172.21.15.27:0/2540140994 conn(0x7f13c80ff440 msgr2=0x7f13c810e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 shutdown_connections 2024-09-21T07:15:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:17.054+0000 7f13c7200640 1 -- 172.21.15.27:0/2540140994 wait complete. 2024-09-21T07:15:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:16 smithi153 ceph-mon[34747]: pgmap v445: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-09-21T07:15:17.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:17 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2540140994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:18.018 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T07:15:18.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T07:15:18.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:17 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2540140994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:18.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:17 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2540140994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:18.329 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:15:18.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:15:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:18.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:18 smithi027 ceph-mon[30779]: pgmap v446: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-21T07:15:19.033 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:18 smithi119 ceph-mon[34519]: pgmap v446: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 -- 172.21.15.27:0/1576637106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86181094c0 msgr2=0x7f86181098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 --2- 172.21.15.27:0/1576637106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86181094c0 0x7f86181098d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8600009900 tx=0x7f860002a410 comp rx=0 tx=0).stop 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 -- 172.21.15.27:0/1576637106 shutdown_connections 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 --2- 172.21.15.27:0/1576637106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618109e10 0x7f8618110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 --2- 172.21.15.27:0/1576637106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86181094c0 0x7f86181098d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 --2- 172.21.15.27:0/1576637106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8618103b80 0x7f8618103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 -- 172.21.15.27:0/1576637106 >> 172.21.15.27:0/1576637106 conn(0x7f86180ff5d0 msgr2=0x7f8618101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 -- 172.21.15.27:0/1576637106 shutdown_connections 2024-09-21T07:15:19.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.055+0000 7f861ce00640 1 -- 172.21.15.27:0/1576637106 wait complete. 2024-09-21T07:15:19.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.056+0000 7f861ce00640 1 Processor -- start 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f861ce00640 1 -- start start 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f861ce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618103b80 0x7f861819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f861ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86181094c0 0x7f861819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f861ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8618109e10 0x7f86181a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f861ce00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8618113ad0 con 0x7f8618109e10 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f861ce00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8618113950 con 0x7f86181094c0 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f861ce00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8618113c50 con 0x7f8618103b80 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f8615e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618103b80 0x7f861819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f860fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86181094c0 0x7f861819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f8615e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618103b80 0x7f861819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41916/0 (socket says 172.21.15.27:41916) 2024-09-21T07:15:19.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.057+0000 7f8615e00640 1 -- 172.21.15.27:0/2570449439 learned_addr learned my addr 172.21.15.27:0/2570449439 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f8616800640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8618109e10 0x7f86181a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f8615e00640 1 -- 172.21.15.27:0/2570449439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86181094c0 msgr2=0x7f861819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f8615e00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86181094c0 0x7f861819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f8615e00640 1 -- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8618109e10 msgr2=0x7f86181a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f8615e00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8618109e10 0x7f86181a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f8615e00640 1 -- 172.21.15.27:0/2570449439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86181a2670 con 0x7f8618103b80 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f860fe00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86181094c0 0x7f861819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.058+0000 7f8615e00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618103b80 0x7f861819c700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f860800e9f0 tx=0x7f860800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.059+0000 7f860ea00640 1 -- 172.21.15.27:0/2570449439 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860800cd80 con 0x7f8618103b80 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.059+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86181a2960 con 0x7f8618103b80 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.059+0000 7f860ea00640 1 -- 172.21.15.27:0/2570449439 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86080108e0 con 0x7f8618103b80 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.059+0000 7f860ea00640 1 -- 172.21.15.27:0/2570449439 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608018980 con 0x7f8618103b80 2024-09-21T07:15:19.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.059+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86181a90c0 con 0x7f8618103b80 2024-09-21T07:15:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 Processor -- start 2024-09-21T07:15:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 -- start start 2024-09-21T07:15:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268cb7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cb7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cb7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563268c2cca0 con 0x56326890f840 2024-09-21T07:15:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563268c1c0d0 con 0x5632688eeb60 2024-09-21T07:15:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f8623a2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563268c1fee0 con 0x5632688e6980 2024-09-21T07:15:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.063+0000 7f860ea00640 1 -- 172.21.15.27:0/2570449439 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8608010430 con 0x7f8618103b80 2024-09-21T07:15:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8616800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268cb7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8616800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268cb7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40040/0 (socket says 172.21.15.27:40040) 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8616800640 1 -- 172.21.15.27:0/3795139150 learned_addr learned my addr 172.21.15.27:0/3795139150 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f860fe00640 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cb7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8615e00640 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cb7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8615e00640 1 -- 172.21.15.27:0/3795139150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 msgr2=0x563268cb7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8615e00640 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cb7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8615e00640 1 -- 172.21.15.27:0/3795139150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 msgr2=0x563268cb7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8615e00640 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268cb7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.064+0000 7f8615e00640 1 -- 172.21.15.27:0/3795139150 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563268bce400 con 0x5632688e6980 2024-09-21T07:15:19.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.065+0000 7f8615e00640 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cb7890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86080e5120 tx=0x7f86080ee7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c141b10c7ab9f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.065+0000 7f8607400640 1 -- 172.21.15.27:0/3795139150 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080fd070 con 0x5632688e6980 2024-09-21T07:15:19.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.065+0000 7f8607400640 1 -- 172.21.15.27:0/3795139150 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86080eca90 con 0x5632688e6980 2024-09-21T07:15:19.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.065+0000 7f8607400640 1 -- 172.21.15.27:0/3795139150 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080e9070 con 0x5632688e6980 2024-09-21T07:15:19.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.066+0000 7f860ea00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85fc0b6700 0x7f85fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.066+0000 7f860ea00640 1 -- 172.21.15.27:0/2570449439 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f86080e7d80 con 0x7f8618103b80 2024-09-21T07:15:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.066+0000 7f8616800640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85fc0b6700 0x7f85fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.066+0000 7f8623a2d740 1 -- 172.21.15.27:0/3795139150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 msgr2=0x563268cb7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.066+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cb7890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86080e5120 tx=0x7f86080ee7c0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 -- 172.21.15.27:0/3795139150 shutdown_connections 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cb7890 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cb7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3795139150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268cb7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 -- 172.21.15.27:0/3795139150 >> 172.21.15.27:0/3795139150 conn(0x563268903990 msgr2=0x563268cb5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 -- 172.21.15.27:0/3795139150 shutdown_connections 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 -- 172.21.15.27:0/3795139150 wait complete. 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8616800640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85fc0b6700 0x7f85fc0b8c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8610011630 tx=0x7f8610009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.067+0000 7f8623a2d740 1 -- start start 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 1 Processor -- start 2024-09-21T07:15:19.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cc2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cc2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268ccee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563268a7d7a0 con 0x56326890f840 2024-09-21T07:15:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563268bc1a10 con 0x5632688eeb60 2024-09-21T07:15:19.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563268cc5b30 con 0x5632688e6980 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.068+0000 7f8623a2d740 10 client.0 _ll_register_callbacks cb 0x563268bd1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cc2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cc2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55766/0 (socket says 172.21.15.27:55766) 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 -- 172.21.15.27:0/1882601650 learned_addr learned my addr 172.21.15.27:0/1882601650 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f8616800640 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268ccee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f8615e00640 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cc2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 -- 172.21.15.27:0/1882601650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 msgr2=0x563268cc2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cc2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 -- 172.21.15.27:0/1882601650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 msgr2=0x563268ccee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268ccee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.069+0000 7f860fe00640 1 -- 172.21.15.27:0/1882601650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563268989b90 con 0x5632688eeb60 2024-09-21T07:15:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.070+0000 7f860fe00640 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cc2f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8600009900 tx=0x7f86000316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da04f6618f868cce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.070+0000 7f85fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.070+0000 7f85fbe00640 1 -- 172.21.15.27:0/1882601650 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600035040 con 0x5632688eeb60 2024-09-21T07:15:19.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.070+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86080eca90 con 0x5632688eeb60 2024-09-21T07:15:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.071+0000 7f85fbe00640 1 -- 172.21.15.27:0/1882601650 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f8600004240 con 0x5632688eeb60 2024-09-21T07:15:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.071+0000 7f85fbe00640 1 -- 172.21.15.27:0/1882601650 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f860004f050 con 0x5632688eeb60 2024-09-21T07:15:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.071+0000 7f8623a2d740 10 client.26213 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.071+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563268cd18c0 con 0x5632688eeb60 2024-09-21T07:15:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.071+0000 7f85fbe00640 1 -- 172.21.15.27:0/1882601650 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86080eca90 con 0x5632688eeb60 2024-09-21T07:15:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.071+0000 7f85fbe00640 1 -- 172.21.15.27:0/1882601650 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8600004a10 con 0x5632688eeb60 2024-09-21T07:15:19.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.072+0000 7f8623a2d740 10 client.26213 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.072+0000 7f8623a2d740 10 client.26213 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.072+0000 7f8623a2d740 20 client.26213 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.072+0000 7f8623a2d740 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5632688ce1c0 0x563268cd4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.072+0000 7f8623a2d740 4 client.26213 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.072+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563268c2ebf0 con 0x5632688ce1c0 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.072+0000 7f8615e00640 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5632688ce1c0 0x563268cd4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.073+0000 7f8615e00640 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5632688ce1c0 0x563268cd4d30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe815e93c4d96d9 server_cookie=f98a309cd8d6367c in_seq=0 out_seq=0 2024-09-21T07:15:19.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.073+0000 7f85fbe00640 10 client.26213 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.073+0000 7f85fbe00640 1 -- 172.21.15.27:0/1882601650 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563268c2ebf0 con 0x5632688ce1c0 2024-09-21T07:15:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.073+0000 7f85fbe00640 10 client.26213 handle_command_reply: tid=0 2024-09-21T07:15:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.073+0000 7f8623a2d740 1 client.26213 shutdown 2024-09-21T07:15:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 msgr2=0x563268cc2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cc2f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8600009900 tx=0x7f86000316d0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 shutdown_connections 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5632688ce1c0 0x563268cd4d30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56326890f840 0x563268ccee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688eeb60 0x563268cc2f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 --2- 172.21.15.27:0/1882601650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688e6980 0x563268cc2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f85fbe00640 0 client.26213 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 >> 172.21.15.27:0/1882601650 conn(0x563268903990 msgr2=0x563268cba530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 shutdown_connections 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 1 -- 172.21.15.27:0/1882601650 wait complete. 2024-09-21T07:15:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.074+0000 7f8623a2d740 20 client.26213 trim_cache size 0 max 16384 2024-09-21T07:15:19.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.093+0000 7f8623a2d740 1 Processor -- start 2024-09-21T07:15:19.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8623a2d740 1 -- start start 2024-09-21T07:15:19.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cb8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268cb40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8623a2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b58a0 con 0x5632688ce1c0 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8623a2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b3170 con 0x5632688e6980 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8623a2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b0fc0 con 0x5632688eeb60 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8615e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cb8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8615e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cb8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40062/0 (socket says 172.21.15.27:40062) 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.094+0000 7f8615e00640 1 -- 172.21.15.27:0/3205403 learned_addr learned my addr 172.21.15.27:0/3205403 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8616800640 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268cb40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f860fe00640 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8615e00640 1 -- 172.21.15.27:0/3205403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 msgr2=0x563268cb40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8615e00640 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268cb40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8615e00640 1 -- 172.21.15.27:0/3205403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 msgr2=0x563268cced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8615e00640 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8615e00640 1 -- 172.21.15.27:0/3205403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563268cc4e00 con 0x5632688ce1c0 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8615e00640 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cb8d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f86080a9ea0 tx=0x7f860800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e31c61632c22d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.095+0000 7f8606a00640 1 -- 172.21.15.27:0/3205403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080a9770 con 0x5632688ce1c0 2024-09-21T07:15:19.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.096+0000 7f8606a00640 1 -- 172.21.15.27:0/3205403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8608010720 con 0x5632688ce1c0 2024-09-21T07:15:19.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.096+0000 7f8606a00640 1 -- 172.21.15.27:0/3205403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080ad040 con 0x5632688ce1c0 2024-09-21T07:15:19.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 -- 172.21.15.27:0/3205403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 msgr2=0x563268cb8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cb8d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f86080a9ea0 tx=0x7f860800cf20 comp rx=0 tx=0).stop 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 -- 172.21.15.27:0/3205403 shutdown_connections 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268cb40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 --2- 172.21.15.27:0/3205403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cb8d40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 -- 172.21.15.27:0/3205403 >> 172.21.15.27:0/3205403 conn(0x563268903990 msgr2=0x563268cb39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 -- 172.21.15.27:0/3205403 shutdown_connections 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 -- 172.21.15.27:0/3205403 wait complete. 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.097+0000 7f8623a2d740 1 -- start start 2024-09-21T07:15:19.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 1 Processor -- start 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cc74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cc7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268d3e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b1630 con 0x5632688ce1c0 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b2aa0 con 0x5632688e6980 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b1d00 con 0x5632688eeb60 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.098+0000 7f8623a2d740 10 client.0 _ll_register_callbacks cb 0x7f8600009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f860fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cc7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cc74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f860fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cc7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55782/0 (socket says 172.21.15.27:55782) 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f860fe00640 1 -- 172.21.15.27:0/118904632 learned_addr learned my addr 172.21.15.27:0/118904632 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cc74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40076/0 (socket says 172.21.15.27:40076) 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8616800640 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268d3e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 -- 172.21.15.27:0/118904632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 msgr2=0x563268d3e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268d3e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 -- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 msgr2=0x563268cc7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cc7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 -- 172.21.15.27:0/118904632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563268d3ec60 con 0x5632688ce1c0 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f860fe00640 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cc7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f8615e00640 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cc74c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f86080a7fd0 tx=0x7f8608000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c68ecd947914d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.099+0000 7f85fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.100+0000 7f85fbe00640 1 -- 172.21.15.27:0/118904632 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080e9030 con 0x5632688ce1c0 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.100+0000 7f85fbe00640 1 -- 172.21.15.27:0/118904632 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f86080f4070 con 0x5632688ce1c0 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.100+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563268c2a6a0 con 0x5632688ce1c0 2024-09-21T07:15:19.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.100+0000 7f85fbe00640 1 -- 172.21.15.27:0/118904632 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86080ad220 con 0x5632688ce1c0 2024-09-21T07:15:19.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.100+0000 7f8623a2d740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:19.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.100+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563268d3fec0 con 0x5632688ce1c0 2024-09-21T07:15:19.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.100+0000 7f85fbe00640 1 -- 172.21.15.27:0/118904632 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563268c2a6a0 con 0x5632688ce1c0 2024-09-21T07:15:19.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f85fbe00640 1 -- 172.21.15.27:0/118904632 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f86080a9b40 con 0x5632688ce1c0 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f8623a2d740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f8623a2d740 10 client.16182 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f8623a2d740 20 client.16182 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f8623a2d740 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56326890f840 0x563268d41fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f8623a2d740 4 client.16182 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x563268c27ad0 con 0x56326890f840 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.101+0000 7f860fe00640 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56326890f840 0x563268d41fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.102+0000 7f860fe00640 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56326890f840 0x563268d41fe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deee59c668d9fc0a server_cookie=cf87a0bd6478d4a4 in_seq=0 out_seq=0 2024-09-21T07:15:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.102+0000 7f85fbe00640 10 client.16182 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.114+0000 7f85fbe00640 1 -- 172.21.15.27:0/118904632 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563268c27ad0 con 0x56326890f840 2024-09-21T07:15:19.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.114+0000 7f85fbe00640 10 client.16182 handle_command_reply: tid=0 2024-09-21T07:15:19.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.114+0000 7f8623a2d740 1 client.16182 shutdown 2024-09-21T07:15:19.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 msgr2=0x563268cc74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cc74c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f86080a7fd0 tx=0x7f8608000bc0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 shutdown_connections 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56326890f840 0x563268d41fe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5632688eeb60 0x563268d3e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632688e6980 0x563268cc7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f85fbe00640 0 client.16182 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 --2- 172.21.15.27:0/118904632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5632688ce1c0 0x563268cc74c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 >> 172.21.15.27:0/118904632 conn(0x563268903990 msgr2=0x563268cb8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 shutdown_connections 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 1 -- 172.21.15.27:0/118904632 wait complete. 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.115+0000 7f8623a2d740 20 client.16182 trim_cache size 0 max 16384 2024-09-21T07:15:19.118 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 10, 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "7c734edf": { 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 10, 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:19.119 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:19.120 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85fc0b6700 msgr2=0x7f85fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85fc0b6700 0x7f85fc0b8c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8610011630 tx=0x7f8610009500 comp rx=0 tx=0).stop 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618103b80 msgr2=0x7f861819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618103b80 0x7f861819c700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f860800e9f0 tx=0x7f860800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 shutdown_connections 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85fc0b6700 0x7f85fc0b8c10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8618109e10 0x7f86181a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86181094c0 0x7f861819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 --2- 172.21.15.27:0/2570449439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8618103b80 0x7f861819c700 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 >> 172.21.15.27:0/2570449439 conn(0x7f86180ff5d0 msgr2=0x7f8618108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 shutdown_connections 2024-09-21T07:15:19.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.116+0000 7f861ce00640 1 -- 172.21.15.27:0/2570449439 wait complete. 2024-09-21T07:15:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:18 smithi153 ceph-mon[34747]: pgmap v446: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-21T07:15:19.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 -- 172.21.15.27:0/3416765361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab74105bb0 msgr2=0x7fab74105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 --2- 172.21.15.27:0/3416765361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab74105bb0 0x7fab74105fc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fab64009900 tx=0x7fab6402a410 comp rx=0 tx=0).stop 2024-09-21T07:15:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 -- 172.21.15.27:0/3416765361 shutdown_connections 2024-09-21T07:15:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 --2- 172.21.15.27:0/3416765361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7410b4f0 0x7fab7410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 --2- 172.21.15.27:0/3416765361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab7410be40 0x7fab74112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 --2- 172.21.15.27:0/3416765361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab74105bb0 0x7fab74105fc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 -- 172.21.15.27:0/3416765361 >> 172.21.15.27:0/3416765361 conn(0x7fab74101680 msgr2=0x7fab74103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 -- 172.21.15.27:0/3416765361 shutdown_connections 2024-09-21T07:15:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.796+0000 7fab78a00640 1 -- 172.21.15.27:0/3416765361 wait complete. 2024-09-21T07:15:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.797+0000 7fab78a00640 1 Processor -- start 2024-09-21T07:15:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.797+0000 7fab78a00640 1 -- start start 2024-09-21T07:15:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab78a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab74105bb0 0x7fab7419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab78a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7410b4f0 0x7fab7419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab78a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab7410be40 0x7fab741a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab78a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab74115b00 con 0x7fab7410b4f0 2024-09-21T07:15:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab78a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab74115980 con 0x7fab7410be40 2024-09-21T07:15:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab78a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab74115c80 con 0x7fab74105bb0 2024-09-21T07:15:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab69e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7410b4f0 0x7fab7419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab72800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab74105bb0 0x7fab7419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab72800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab74105bb0 0x7fab7419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41978/0 (socket says 172.21.15.27:41978) 2024-09-21T07:15:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab72800640 1 -- 172.21.15.27:0/1900006578 learned_addr learned my addr 172.21.15.27:0/1900006578 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.798+0000 7fab73200640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab7410be40 0x7fab741a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab72800640 1 -- 172.21.15.27:0/1900006578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab7410be40 msgr2=0x7fab741a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab72800640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab7410be40 0x7fab741a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab72800640 1 -- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7410b4f0 msgr2=0x7fab7419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab72800640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7410b4f0 0x7fab7419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab72800640 1 -- 172.21.15.27:0/1900006578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab741a2480 con 0x7fab74105bb0 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab72800640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab74105bb0 0x7fab7419c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab64005940 tx=0x7fab640040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab6be00640 1 -- 172.21.15.27:0/1900006578 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6403f030 con 0x7fab74105bb0 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab6be00640 1 -- 172.21.15.27:0/1900006578 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab640044c0 con 0x7fab74105bb0 2024-09-21T07:15:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab741a2710 con 0x7fab74105bb0 2024-09-21T07:15:19.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab6be00640 1 -- 172.21.15.27:0/1900006578 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64035070 con 0x7fab74105bb0 2024-09-21T07:15:19.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.799+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab741a8ed0 con 0x7fab74105bb0 2024-09-21T07:15:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.803+0000 7fab80075740 1 Processor -- start 2024-09-21T07:15:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.804+0000 7fab80075740 1 -- start start 2024-09-21T07:15:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.804+0000 7fab80075740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee13f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.804+0000 7fab80075740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee14440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.804+0000 7fab80075740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee14980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.804+0000 7fab80075740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55abfed89e90 con 0x55abfea6c350 2024-09-21T07:15:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.804+0000 7fab80075740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55abfed792c0 con 0x55abfea43490 2024-09-21T07:15:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.804+0000 7fab80075740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abfed7d0d0 con 0x55abfea4b670 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab73200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee13f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab69e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee14980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab73200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee13f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40096/0 (socket says 172.21.15.27:40096) 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab73200640 1 -- 172.21.15.27:0/1878537431 learned_addr learned my addr 172.21.15.27:0/1878537431 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab6be00640 1 -- 172.21.15.27:0/1900006578 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fab64042030 con 0x7fab74105bb0 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab69e00640 1 -- 172.21.15.27:0/1878537431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 msgr2=0x55abfee14440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab69e00640 1 --2- 172.21.15.27:0/1878537431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee14440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab69e00640 1 -- 172.21.15.27:0/1878537431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 msgr2=0x55abfee13f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab69e00640 1 --2- 172.21.15.27:0/1878537431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee13f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab69e00640 1 -- 172.21.15.27:0/1878537431 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfead7930 con 0x55abfea4b670 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.805+0000 7fab69e00640 1 --2- 172.21.15.27:0/1878537431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee14980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab7406c630 tx=0x7fab5c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5628353cfadef99b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.806+0000 7fab63400640 1 -- 172.21.15.27:0/1878537431 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c009d70 con 0x55abfea4b670 2024-09-21T07:15:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.806+0000 7fab63400640 1 -- 172.21.15.27:0/1878537431 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab5c0198c0 con 0x55abfea4b670 2024-09-21T07:15:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.806+0000 7fab63400640 1 -- 172.21.15.27:0/1878537431 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0188b0 con 0x55abfea4b670 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.806+0000 7fab80075740 1 -- 172.21.15.27:0/1878537431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 msgr2=0x55abfee14980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 --2- 172.21.15.27:0/1878537431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee14980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab7406c630 tx=0x7fab5c011810 comp rx=0 tx=0).stop 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 -- 172.21.15.27:0/1878537431 shutdown_connections 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 --2- 172.21.15.27:0/1878537431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee14980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 --2- 172.21.15.27:0/1878537431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee14440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 --2- 172.21.15.27:0/1878537431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee13f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 -- 172.21.15.27:0/1878537431 >> 172.21.15.27:0/1878537431 conn(0x55abfea60740 msgr2=0x55abfee11df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab6be00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab580b6700 0x7fab580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 -- 172.21.15.27:0/1878537431 shutdown_connections 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab80075740 1 -- 172.21.15.27:0/1878537431 wait complete. 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab6be00640 1 -- 172.21.15.27:0/1900006578 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab64107e20 con 0x7fab74105bb0 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.807+0000 7fab69e00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab580b6700 0x7fab580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.808+0000 7fab80075740 1 -- start start 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.808+0000 7fab69e00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab580b6700 0x7fab580b8c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fab5c01b040 tx=0x7fab5c009770 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.808+0000 7fab80075740 1 Processor -- start 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab80075740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea43490 0x55abfee1fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab80075740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee20070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab80075740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee2b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab80075740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55abfebda890 con 0x55abfea6c350 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab80075740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55abfed14160 con 0x55abfea4b670 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab80075740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abfee226e0 con 0x55abfea43490 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab80075740 10 client.0 _ll_register_callbacks cb 0x55abfed30a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab69e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee20070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab73200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee2b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab69e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee20070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55818/0 (socket says 172.21.15.27:55818) 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab69e00640 1 -- 172.21.15.27:0/2780936258 learned_addr learned my addr 172.21.15.27:0/2780936258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.809+0000 7fab72800640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea43490 0x55abfee1fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab72800640 1 -- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 msgr2=0x55abfee20070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab72800640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee20070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab72800640 1 -- 172.21.15.27:0/2780936258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 msgr2=0x55abfee2b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab72800640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee2b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab72800640 1 -- 172.21.15.27:0/2780936258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfee2bfc0 con 0x55abfea43490 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab69e00640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee20070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab73200640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee2b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab72800640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea43490 0x55abfee1fb30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fab641051c0 tx=0x7fab64105740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d236e858df4a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab57e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab57e00640 1 -- 172.21.15.27:0/2780936258 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64049050 con 0x55abfea43490 2024-09-21T07:15:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5c0198c0 con 0x55abfea43490 2024-09-21T07:15:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab57e00640 1 -- 172.21.15.27:0/2780936258 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab640ccb20 con 0x55abfea43490 2024-09-21T07:15:19.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab57e00640 1 -- 172.21.15.27:0/2780936258 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab640cd690 con 0x55abfea43490 2024-09-21T07:15:19.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab80075740 10 client.26218 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:19.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.810+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55abfee2e8d0 con 0x55abfea43490 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab57e00640 1 -- 172.21.15.27:0/2780936258 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab6407b020 con 0x55abfea43490 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab57e00640 1 -- 172.21.15.27:0/2780936258 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab640cc590 con 0x55abfea43490 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab80075740 10 client.26218 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab80075740 10 client.26218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab80075740 20 client.26218 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab80075740 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea2acd0 0x55abfee31de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab80075740 4 client.26218 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abfed8bde0 con 0x55abfea2acd0 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.811+0000 7fab69e00640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea2acd0 0x55abfee31de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.812+0000 7fab69e00640 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea2acd0 0x55abfee31de0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b9c39f08ccf810d server_cookie=3495d51ddf5eade9 in_seq=0 out_seq=0 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.812+0000 7fab57e00640 10 client.26218 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab57e00640 1 -- 172.21.15.27:0/2780936258 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55abfed8bde0 con 0x55abfea2acd0 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab57e00640 10 client.26218 handle_command_reply: tid=0 2024-09-21T07:15:19.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 client.26218 shutdown 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea43490 msgr2=0x55abfee1fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea43490 0x55abfee1fb30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fab641051c0 tx=0x7fab64105740 comp rx=0 tx=0).stop 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 shutdown_connections 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea2acd0 0x55abfee31de0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee2b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee20070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 --2- 172.21.15.27:0/2780936258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea43490 0x55abfee1fb30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 >> 172.21.15.27:0/2780936258 conn(0x55abfea60740 msgr2=0x55abfee170e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab57e00640 0 client.26218 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 shutdown_connections 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 1 -- 172.21.15.27:0/2780936258 wait complete. 2024-09-21T07:15:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.813+0000 7fab80075740 20 client.26218 trim_cache size 0 max 16384 2024-09-21T07:15:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.832+0000 7fab80075740 1 Processor -- start 2024-09-21T07:15:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.832+0000 7fab80075740 1 -- start start 2024-09-21T07:15:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab80075740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea43490 0x55abfee19010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab80075740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee19550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab80075740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea6c350 0x55abfee113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab80075740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab741b1310 con 0x55abfea43490 2024-09-21T07:15:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab80075740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aebe0 con 0x55abfea4b670 2024-09-21T07:15:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab80075740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aca30 con 0x55abfea6c350 2024-09-21T07:15:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab72800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea43490 0x55abfee19010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab72800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea43490 0x55abfee19010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40112/0 (socket says 172.21.15.27:40112) 2024-09-21T07:15:19.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab72800640 1 -- 172.21.15.27:0/4167627524 learned_addr learned my addr 172.21.15.27:0/4167627524 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab69e00640 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee19550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab73200640 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea6c350 0x55abfee113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab72800640 1 -- 172.21.15.27:0/4167627524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea6c350 msgr2=0x55abfee113a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab72800640 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea6c350 0x55abfee113a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab72800640 1 -- 172.21.15.27:0/4167627524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 msgr2=0x55abfee19550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.833+0000 7fab72800640 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee19550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.834+0000 7fab72800640 1 -- 172.21.15.27:0/4167627524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfee1de30 con 0x55abfea43490 2024-09-21T07:15:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.834+0000 7fab72800640 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea43490 0x55abfee19010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab6403d040 tx=0x7fab640c9050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86fed1516b911c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.834+0000 7fab62a00640 1 -- 172.21.15.27:0/4167627524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab640cd030 con 0x55abfea43490 2024-09-21T07:15:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.834+0000 7fab62a00640 1 -- 172.21.15.27:0/4167627524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab6407b070 con 0x55abfea43490 2024-09-21T07:15:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.834+0000 7fab62a00640 1 -- 172.21.15.27:0/4167627524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab640cd210 con 0x55abfea43490 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 -- 172.21.15.27:0/4167627524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea43490 msgr2=0x55abfee19010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea43490 0x55abfee19010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab6403d040 tx=0x7fab640c9050 comp rx=0 tx=0).stop 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 -- 172.21.15.27:0/4167627524 shutdown_connections 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea6c350 0x55abfee113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea4b670 0x55abfee19550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 --2- 172.21.15.27:0/4167627524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea43490 0x55abfee19010 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 -- 172.21.15.27:0/4167627524 >> 172.21.15.27:0/4167627524 conn(0x55abfea60740 msgr2=0x55abfee1e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 -- 172.21.15.27:0/4167627524 shutdown_connections 2024-09-21T07:15:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.835+0000 7fab80075740 1 -- 172.21.15.27:0/4167627524 wait complete. 2024-09-21T07:15:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.836+0000 7fab80075740 1 -- start start 2024-09-21T07:15:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.836+0000 7fab80075740 1 Processor -- start 2024-09-21T07:15:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.836+0000 7fab80075740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee23b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.836+0000 7fab80075740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee24070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.836+0000 7fab80075740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab80075740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ad0a0 con 0x55abfea6c350 2024-09-21T07:15:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab80075740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ae510 con 0x55abfea43490 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab80075740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ad770 con 0x55abfea4b670 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab80075740 10 client.0 _ll_register_callbacks cb 0x7fab64005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab73200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab69e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee24070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab72800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee23b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab73200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee245b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40126/0 (socket says 172.21.15.27:40126) 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab69e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee24070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42008/0 (socket says 172.21.15.27:42008) 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab73200640 1 -- 172.21.15.27:0/1609002192 learned_addr learned my addr 172.21.15.27:0/1609002192 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab69e00640 1 -- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 msgr2=0x55abfee23b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab69e00640 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee23b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab69e00640 1 -- 172.21.15.27:0/1609002192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 msgr2=0x55abfee245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab69e00640 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee245b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab69e00640 1 -- 172.21.15.27:0/1609002192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfee9b9a0 con 0x55abfea4b670 2024-09-21T07:15:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.837+0000 7fab72800640 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee23b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab69e00640 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee24070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55abfee32340 tx=0x7fab5c0184f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5762d25054315d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab57e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab57e00640 1 -- 172.21.15.27:0/1609002192 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c009430 con 0x55abfea4b670 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6407b070 con 0x55abfea4b670 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab57e00640 1 -- 172.21.15.27:0/1609002192 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab5c016040 con 0x55abfea4b670 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab57e00640 1 -- 172.21.15.27:0/1609002192 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab5c005040 con 0x55abfea4b670 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab80075740 10 client.26224 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:19.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.838+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55abfee9cc40 con 0x55abfea4b670 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab57e00640 1 -- 172.21.15.27:0/1609002192 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab6407b070 con 0x55abfea4b670 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab57e00640 1 -- 172.21.15.27:0/1609002192 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab5c004cc0 con 0x55abfea4b670 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab80075740 10 client.26224 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab80075740 10 client.26224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab80075740 20 client.26224 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab80075740 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea00080 0x55abfee9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab80075740 4 client.26224 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55abfed87890 con 0x55abfea00080 2024-09-21T07:15:19.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.839+0000 7fab72800640 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea00080 0x55abfee9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:19.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.840+0000 7fab72800640 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea00080 0x55abfee9f230 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfcd68bb4dfbaec1 server_cookie=5f175054eada1fb5 in_seq=0 out_seq=0 2024-09-21T07:15:19.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.840+0000 7fab57e00640 10 client.26224 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab57e00640 1 -- 172.21.15.27:0/1609002192 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55abfed87890 con 0x55abfea00080 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab57e00640 10 client.26224 handle_command_reply: tid=0 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 client.26224 shutdown 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 msgr2=0x55abfee24070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee24070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55abfee32340 tx=0x7fab5c0184f0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 shutdown_connections 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55abfea00080 0x55abfee9f230 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfea4b670 0x55abfee24070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab57e00640 0 client.26224 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55abfea6c350 0x55abfee245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 --2- 172.21.15.27:0/1609002192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abfea43490 0x55abfee23b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 >> 172.21.15.27:0/1609002192 conn(0x55abfea60740 msgr2=0x55abfee19010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 shutdown_connections 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.842+0000 7fab80075740 1 -- 172.21.15.27:0/1609002192 wait complete. 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.843+0000 7fab80075740 20 client.26224 trim_cache size 0 max 16384 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5", 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:15:19.846 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.843+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab580b6700 msgr2=0x7fab580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.843+0000 7fab78a00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab580b6700 0x7fab580b8c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fab5c01b040 tx=0x7fab5c009770 comp rx=0 tx=0).stop 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.843+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab74105bb0 msgr2=0x7fab7419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.843+0000 7fab78a00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab74105bb0 0x7fab7419c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab64005940 tx=0x7fab640040c0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 shutdown_connections 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab580b6700 0x7fab580b8c10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab7410be40 0x7fab741a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7410b4f0 0x7fab7419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 --2- 172.21.15.27:0/1900006578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab74105bb0 0x7fab7419c510 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 >> 172.21.15.27:0/1900006578 conn(0x7fab74101680 msgr2=0x7fab74103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:19.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 shutdown_connections 2024-09-21T07:15:19.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:19.844+0000 7fab78a00640 1 -- 172.21.15.27:0/1900006578 wait complete. 2024-09-21T07:15:20.223 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7c734edf', quiesce took 5.2 seconds. Will release after: 22.6 2024-09-21T07:15:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:20 smithi027 ceph-mon[30779]: pgmap v447: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-21T07:15:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:20 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:15:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:20 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:15:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:20 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:15:20.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:20 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:15:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:20 smithi119 ceph-mon[34519]: pgmap v447: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-21T07:15:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:20 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:15:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:20 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:15:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:20 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:15:21.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:20 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:15:21.039 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:15:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:20 smithi153 ceph-mon[34747]: pgmap v447: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-21T07:15:21.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:20 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:15:21.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:20 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:15:21.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:20 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:15:21.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:20 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 -- 172.21.15.27:0/83067392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 msgr2=0x7f1f34105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 --2- 172.21.15.27:0/83067392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 0x7f1f34105fd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009900 tx=0x7f1f2402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 -- 172.21.15.27:0/83067392 shutdown_connections 2024-09-21T07:15:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 --2- 172.21.15.27:0/83067392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f3410be50 0x7f1f34112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 --2- 172.21.15.27:0/83067392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3410b500 0x7f1f3410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 --2- 172.21.15.27:0/83067392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 0x7f1f34105fd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 -- 172.21.15.27:0/83067392 >> 172.21.15.27:0/83067392 conn(0x7f1f34101670 msgr2=0x7f1f34103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:22.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 -- 172.21.15.27:0/83067392 shutdown_connections 2024-09-21T07:15:22.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.027+0000 7f1f33200640 1 -- 172.21.15.27:0/83067392 wait complete. 2024-09-21T07:15:22.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.028+0000 7f1f33200640 1 Processor -- start 2024-09-21T07:15:22.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.028+0000 7f1f33200640 1 -- start start 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f33200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 0x7f1f3419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f33200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f3410b500 0x7f1f3419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f33200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3410be50 0x7f1f341a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f33200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34115b60 con 0x7f1f34105bc0 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f33200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341159e0 con 0x7f1f3410be50 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f33200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34115ce0 con 0x7f1f3410b500 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 0x7f1f3419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f29400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f3410b500 0x7f1f3419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 0x7f1f3419c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40148/0 (socket says 172.21.15.27:40148) 2024-09-21T07:15:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f31e00640 1 -- 172.21.15.27:0/3593152334 learned_addr learned my addr 172.21.15.27:0/3593152334 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:22.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.029+0000 7f1f32800640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3410be50 0x7f1f341a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f29400640 1 -- 172.21.15.27:0/3593152334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3410be50 msgr2=0x7f1f341a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f29400640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3410be50 0x7f1f341a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f29400640 1 -- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 msgr2=0x7f1f3419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f29400640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 0x7f1f3419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f29400640 1 -- 172.21.15.27:0/3593152334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f341a2580 con 0x7f1f3410b500 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f29400640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f3410b500 0x7f1f3419cbb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f3406c8a0 tx=0x7f1f1c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f2b400640 1 -- 172.21.15.27:0/3593152334 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c00cd00 con 0x7f1f3410b500 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f341a2870 con 0x7f1f3410b500 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f2b400640 1 -- 172.21.15.27:0/3593152334 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c0035f0 con 0x7f1f3410b500 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f2b400640 1 -- 172.21.15.27:0/3593152334 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c010690 con 0x7f1f3410b500 2024-09-21T07:15:22.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.030+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f341a8fd0 con 0x7f1f3410b500 2024-09-21T07:15:22.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.033+0000 7f1f3f67c740 1 Processor -- start 2024-09-21T07:15:22.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.033+0000 7f1f3f67c740 1 -- start start 2024-09-21T07:15:22.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c528604070 0x55c5289aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285db1b0 0x55c5289ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 0x55c5289ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c528921f70 con 0x55c528604070 2024-09-21T07:15:22.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c5289113a0 con 0x55c5285db1b0 2024-09-21T07:15:22.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c5289151b0 con 0x55c5285e3390 2024-09-21T07:15:22.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c528604070 0x55c5289aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c528604070 0x55c5289aaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40152/0 (socket says 172.21.15.27:40152) 2024-09-21T07:15:22.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f32800640 1 -- 172.21.15.27:0/1238610506 learned_addr learned my addr 172.21.15.27:0/1238610506 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:22.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f31e00640 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285db1b0 0x55c5289ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.034+0000 7f1f2b400640 1 -- 172.21.15.27:0/3593152334 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1f1c0037a0 con 0x7f1f3410b500 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f32800640 1 -- 172.21.15.27:0/1238610506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 msgr2=0x55c5289ab9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f32800640 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 0x55c5289ab9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f32800640 1 -- 172.21.15.27:0/1238610506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285db1b0 msgr2=0x55c5289ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f32800640 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285db1b0 0x55c5289ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f32800640 1 -- 172.21.15.27:0/1238610506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5288dbc90 con 0x55c528604070 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f32800640 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c528604070 0x55c5289aaf30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009f50 tx=0x7f1f2c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47e28609b23c350 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f23000640 1 -- 172.21.15.27:0/1238610506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c004d20 con 0x55c528604070 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f23000640 1 -- 172.21.15.27:0/1238610506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f2c004050 con 0x55c528604070 2024-09-21T07:15:22.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.035+0000 7f1f23000640 1 -- 172.21.15.27:0/1238610506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c005280 con 0x55c528604070 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.036+0000 7f1f2b400640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f180b6680 0x7f1f180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.036+0000 7f1f3f67c740 1 -- 172.21.15.27:0/1238610506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c528604070 msgr2=0x55c5289aaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.036+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c528604070 0x55c5289aaf30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009f50 tx=0x7f1f2c0118d0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.036+0000 7f1f2b400640 1 -- 172.21.15.27:0/3593152334 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1f1c0e7d20 con 0x7f1f3410b500 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f180b6680 0x7f1f180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f3f67c740 1 -- 172.21.15.27:0/1238610506 shutdown_connections 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 0x55c5289ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285db1b0 0x55c5289ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/1238610506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c528604070 0x55c5289aaf30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f3f67c740 1 -- 172.21.15.27:0/1238610506 >> 172.21.15.27:0/1238610506 conn(0x55c5285f8460 msgr2=0x55c5289a8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:22.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f3f67c740 1 -- 172.21.15.27:0/1238610506 shutdown_connections 2024-09-21T07:15:22.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.037+0000 7f1f3f67c740 1 -- 172.21.15.27:0/1238610506 wait complete. 2024-09-21T07:15:22.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 -- start start 2024-09-21T07:15:22.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f180b6680 0x7f1f180b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55c5289a9bb0 tx=0x7f1f24005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:22.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 Processor -- start 2024-09-21T07:15:22.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285db1b0 0x55c5289bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285e3390 0x55c5289bfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c528772700 con 0x55c5285e3390 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c5288ac540 con 0x55c528604070 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c5289b9710 con 0x55c5285db1b0 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.038+0000 7f1f3f67c740 10 client.0 _ll_register_callbacks cb 0x55c5288bc430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f29400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285e3390 0x55c5289bfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285db1b0 0x55c5289bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285db1b0 0x55c5289bf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42046/0 (socket says 172.21.15.27:42046) 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f31e00640 1 -- 172.21.15.27:0/4041153258 learned_addr learned my addr 172.21.15.27:0/4041153258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f29400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285e3390 0x55c5289bfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40154/0 (socket says 172.21.15.27:40154) 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f32800640 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f32800640 1 -- 172.21.15.27:0/4041153258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285db1b0 msgr2=0x55c5289bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f32800640 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285db1b0 0x55c5289bf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f32800640 1 -- 172.21.15.27:0/4041153258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285e3390 msgr2=0x55c5289bfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f32800640 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285e3390 0x55c5289bfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f32800640 1 -- 172.21.15.27:0/4041153258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c52866f650 con 0x55c528604070 2024-09-21T07:15:22.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.039+0000 7f1f29400640 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285e3390 0x55c5289bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:22.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f32800640 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c4df0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c004750 tx=0x7f1f2c004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92935df1c7ec2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:22.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f17400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:22.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f17400640 1 -- 172.21.15.27:0/4041153258 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c028020 con 0x55c528604070 2024-09-21T07:15:22.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5288aeb50 con 0x55c528604070 2024-09-21T07:15:22.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f17400640 1 -- 172.21.15.27:0/4041153258 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1f2c004460 con 0x55c528604070 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f17400640 1 -- 172.21.15.27:0/4041153258 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f2c009be0 con 0x55c528604070 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f3f67c740 10 client.26246 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.040+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5288c9910 con 0x55c528604070 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f17400640 1 -- 172.21.15.27:0/4041153258 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5288aeb50 con 0x55c528604070 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f17400640 1 -- 172.21.15.27:0/4041153258 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f2c022470 con 0x55c528604070 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f3f67c740 10 client.26246 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f3f67c740 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f3f67c740 20 client.26246 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5285c29f0 0x55c5289d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f3f67c740 4 client.26246 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c528923ec0 con 0x55c5285c29f0 2024-09-21T07:15:22.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.041+0000 7f1f31e00640 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5285c29f0 0x55c5289d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.042+0000 7f1f31e00640 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5285c29f0 0x55c5289d0ed0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e21d974e8111190 server_cookie=dac87db7564cae73 in_seq=0 out_seq=0 2024-09-21T07:15:22.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.042+0000 7f1f17400640 10 client.26246 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f17400640 1 -- 172.21.15.27:0/4041153258 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c528923ec0 con 0x55c5285c29f0 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f17400640 10 client.26246 handle_command_reply: tid=0 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 client.26246 shutdown 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 msgr2=0x55c5289c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c4df0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c004750 tx=0x7f1f2c004cb0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 shutdown_connections 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5285c29f0 0x55c5289d0ed0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c4df0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285e3390 0x55c5289bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f17400640 0 client.26246 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:22.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/4041153258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285db1b0 0x55c5289bf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.043+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 >> 172.21.15.27:0/4041153258 conn(0x55c5285f8460 msgr2=0x55c5289a75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:22.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.044+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 shutdown_connections 2024-09-21T07:15:22.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.044+0000 7f1f3f67c740 1 -- 172.21.15.27:0/4041153258 wait complete. 2024-09-21T07:15:22.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.044+0000 7f1f3f67c740 20 client.26246 trim_cache size 0 max 16384 2024-09-21T07:15:22.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 Processor -- start 2024-09-21T07:15:22.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 -- start start 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285e3390 0x55c5289a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c528604070 0x55c5289c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341b12d0 con 0x55c5285db1b0 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341aeba0 con 0x55c5285e3390 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.062+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341aca10 con 0x55c528604070 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40170/0 (socket says 172.21.15.27:40170) 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 -- 172.21.15.27:0/3233585658 learned_addr learned my addr 172.21.15.27:0/3233585658 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f29400640 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285e3390 0x55c5289a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f32800640 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c528604070 0x55c5289c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 -- 172.21.15.27:0/3233585658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c528604070 msgr2=0x55c5289c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:22.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c528604070 0x55c5289c3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 -- 172.21.15.27:0/3233585658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285e3390 msgr2=0x55c5289a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:22.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285e3390 0x55c5289a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.063+0000 7f1f31e00640 1 -- 172.21.15.27:0/3233585658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5289bae80 con 0x55c5285db1b0 2024-09-21T07:15:22.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.064+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c35c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f24040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb87fd556bfe563 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:22.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.064+0000 7f1f22600640 1 -- 172.21.15.27:0/3233585658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24035040 con 0x55c5285db1b0 2024-09-21T07:15:22.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.064+0000 7f1f22600640 1 -- 172.21.15.27:0/3233585658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f24031950 con 0x55c5285db1b0 2024-09-21T07:15:22.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.064+0000 7f1f22600640 1 -- 172.21.15.27:0/3233585658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2403d7d0 con 0x55c5285db1b0 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3233585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 msgr2=0x55c5289c35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c35c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f24040000 comp rx=0 tx=0).stop 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3233585658 shutdown_connections 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c528604070 0x55c5289c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5285e3390 0x55c5289a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3233585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c35c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3233585658 >> 172.21.15.27:0/3233585658 conn(0x55c5285f8460 msgr2=0x55c5289b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3233585658 shutdown_connections 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.065+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3233585658 wait complete. 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 -- start start 2024-09-21T07:15:22.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 Processor -- start 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 0x55c5289c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341ad060 con 0x55c5285db1b0 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341ae4d0 con 0x55c528604070 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.066+0000 7f1f3f67c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341ad730 con 0x55c5285e3390 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f3f67c740 10 client.0 _ll_register_callbacks cb 0x7f1f2c003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40176/0 (socket says 172.21.15.27:40176) 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 -- 172.21.15.27:0/3107661988 learned_addr learned my addr 172.21.15.27:0/3107661988 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f29400640 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 0x55c5289c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f32800640 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 -- 172.21.15.27:0/3107661988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 msgr2=0x55c5289c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 0x55c5289c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 -- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 msgr2=0x55c5289c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 -- 172.21.15.27:0/3107661988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5289c8350 con 0x55c5285db1b0 2024-09-21T07:15:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f31e00640 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c1710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1f2402a430 tx=0x7f1f24005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd2499c8676b7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.067+0000 7f1f17400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.068+0000 7f1f17400640 1 -- 172.21.15.27:0/3107661988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2405e020 con 0x55c5285db1b0 2024-09-21T07:15:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.068+0000 7f1f17400640 1 -- 172.21.15.27:0/3107661988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f1f240316b0 con 0x55c5285db1b0 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.068+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f24031950 con 0x55c5285db1b0 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.068+0000 7f1f17400640 1 -- 172.21.15.27:0/3107661988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f24031b00 con 0x55c5285db1b0 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.068+0000 7f1f3f67c740 10 client.16218 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.068+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c528a3be80 con 0x55c5285db1b0 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f17400640 1 -- 172.21.15.27:0/3107661988 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f24031950 con 0x55c5285db1b0 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f17400640 1 -- 172.21.15.27:0/3107661988 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f24044070 con 0x55c5285db1b0 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f3f67c740 10 client.16218 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f3f67c740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f3f67c740 20 client.16218 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c528597da0 0x55c528a3e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f3f67c740 4 client.16218 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c52891f970 con 0x55c528597da0 2024-09-21T07:15:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.069+0000 7f1f29400640 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c528597da0 0x55c528a3e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f29400640 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c528597da0 0x55c528a3e650 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc17ee3f1de1e9a server_cookie=9f7d0b2616dc5bc5 in_seq=0 out_seq=0 2024-09-21T07:15:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f17400640 10 client.16218 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f17400640 1 -- 172.21.15.27:0/3107661988 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x55c52891f970 con 0x55c528597da0 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f17400640 10 client.16218 handle_command_reply: tid=0 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 client.16218 shutdown 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 msgr2=0x55c5289c1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c1710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1f2402a430 tx=0x7f1f24005870 comp rx=0 tx=0).stop 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 shutdown_connections 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c528597da0 0x55c528a3e650 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5285e3390 0x55c5289c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c528604070 0x55c5289c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 --2- 172.21.15.27:0/3107661988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5285db1b0 0x55c5289c1710 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f17400640 0 client.16218 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.070+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 >> 172.21.15.27:0/3107661988 conn(0x55c5285f8460 msgr2=0x55c5289b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.071+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 shutdown_connections 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.071+0000 7f1f3f67c740 1 -- 172.21.15.27:0/3107661988 wait complete. 2024-09-21T07:15:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.071+0000 7f1f3f67c740 20 client.16218 trim_cache size 0 max 16384 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5", 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:15:22.075 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:22.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:22.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:22.078 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:22.078 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:22.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:22.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:22.078 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:15:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.071+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f180b6680 msgr2=0x7f1f180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.071+0000 7f1f33200640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f180b6680 0x7f1f180b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55c5289a9bb0 tx=0x7f1f24005a20 comp rx=0 tx=0).stop 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.071+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f3410b500 msgr2=0x7f1f3419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.071+0000 7f1f33200640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f3410b500 0x7f1f3419cbb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f3406c8a0 tx=0x7f1f1c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 shutdown_connections 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f3410b500 0x7f1f3419cbb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1f180b6680 0x7f1f180b8b90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3410be50 0x7f1f341a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 --2- 172.21.15.27:0/3593152334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f34105bc0 0x7f1f3419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 >> 172.21.15.27:0/3593152334 conn(0x7f1f34101670 msgr2=0x7f1f3410ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 shutdown_connections 2024-09-21T07:15:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:22.072+0000 7f1f33200640 1 -- 172.21.15.27:0/3593152334 wait complete. 2024-09-21T07:15:22.993 INFO:tasks.cephfs.filesystem:scrub status for tag:2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5 - {'path': '/', 'tag': '2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5', 'options': 'recursive,force'} 2024-09-21T07:15:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:22 smithi119 ceph-mon[34519]: pgmap v448: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-21T07:15:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:22 smithi027 ceph-mon[30779]: pgmap v448: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-21T07:15:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:22 smithi153 ceph-mon[34747]: pgmap v448: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 -- 172.21.15.27:0/2911505866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe010feb0 msgr2=0x7febe01122d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 --2- 172.21.15.27:0/2911505866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe010feb0 0x7febe01122d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7febd8009ca0 tx=0x7febd802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 -- 172.21.15.27:0/2911505866 shutdown_connections 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 --2- 172.21.15.27:0/2911505866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe010feb0 0x7febe01122d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 --2- 172.21.15.27:0/2911505866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe00760a0 0x7febe010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 --2- 172.21.15.27:0/2911505866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe0075750 0x7febe0075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 -- 172.21.15.27:0/2911505866 >> 172.21.15.27:0/2911505866 conn(0x7febe00ff360 msgr2=0x7febe01017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 -- 172.21.15.27:0/2911505866 shutdown_connections 2024-09-21T07:15:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.213+0000 7febdf200640 1 -- 172.21.15.27:0/2911505866 wait complete. 2024-09-21T07:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.214+0000 7febdf200640 1 Processor -- start 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdf200640 1 -- start start 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdf200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe0075750 0x7febe019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe00760a0 0x7febe019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdf200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe010feb0 0x7febe01a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdf200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7febe0115a20 con 0x7febe0075750 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7febe01158a0 con 0x7febe00760a0 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdf200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febe0115ba0 con 0x7febe010feb0 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe0075750 0x7febe019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe0075750 0x7febe019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40194/0 (socket says 172.21.15.27:40194) 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.215+0000 7febdde00640 1 -- 172.21.15.27:0/1898411218 learned_addr learned my addr 172.21.15.27:0/1898411218 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdd400640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe00760a0 0x7febe019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febde800640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe010feb0 0x7febe01a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdde00640 1 -- 172.21.15.27:0/1898411218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe010feb0 msgr2=0x7febe01a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdde00640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe010feb0 0x7febe01a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdde00640 1 -- 172.21.15.27:0/1898411218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe00760a0 msgr2=0x7febe019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdde00640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe00760a0 0x7febe019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdde00640 1 -- 172.21.15.27:0/1898411218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01a2310 con 0x7febe0075750 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdd400640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe00760a0 0x7febe019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.216+0000 7febdde00640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe0075750 0x7febe019c3e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febd000ea60 tx=0x7febd000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.217+0000 7febd6a00640 1 -- 172.21.15.27:0/1898411218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000cde0 con 0x7febe0075750 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.217+0000 7febd6a00640 1 -- 172.21.15.27:0/1898411218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd00035f0 con 0x7febe0075750 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.217+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01a2600 con 0x7febe0075750 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.217+0000 7febd6a00640 1 -- 172.21.15.27:0/1898411218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0010690 con 0x7febe0075750 2024-09-21T07:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.217+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01a8d90 con 0x7febe0075750 2024-09-21T07:15:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.220+0000 7febebbee740 1 Processor -- start 2024-09-21T07:15:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.220+0000 7febebbee740 1 -- start start 2024-09-21T07:15:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.220+0000 7febebbee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febebbee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febebbee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febd6a00640 1 -- 172.21.15.27:0/1898411218 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7febd00037a0 con 0x7febe0075750 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febebbee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55992a3696b0 con 0x55992a04f1b0 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febebbee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55992a358ae0 con 0x55992a02e4d0 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febebbee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55992a35c8f0 con 0x55992a0262f0 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40206/0 (socket says 172.21.15.27:40206) 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 -- 172.21.15.27:0/3654145845 learned_addr learned my addr 172.21.15.27:0/3654145845 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febdd400640 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febdde00640 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 -- 172.21.15.27:0/3654145845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 msgr2=0x55992a3f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 -- 172.21.15.27:0/3654145845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 msgr2=0x55992a3f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febde800640 1 -- 172.21.15.27:0/3654145845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992a325780 con 0x55992a04f1b0 2024-09-21T07:15:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febdde00640 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:15:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.221+0000 7febdd400640 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.222+0000 7febde800640 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3f3e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7febd8009800 tx=0x7febd8009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45871306b7c9214b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.222+0000 7febcb000640 1 -- 172.21.15.27:0/3654145845 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80305a0 con 0x55992a04f1b0 2024-09-21T07:15:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.222+0000 7febcb000640 1 -- 172.21.15.27:0/3654145845 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036030 con 0x55992a04f1b0 2024-09-21T07:15:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.222+0000 7febcb000640 1 -- 172.21.15.27:0/3654145845 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8004240 con 0x55992a04f1b0 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febd6a00640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7febc40b6680 0x7febc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 -- 172.21.15.27:0/3654145845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 msgr2=0x55992a3f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3f3e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7febd8009800 tx=0x7febd8009830 comp rx=0 tx=0).stop 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febdde00640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7febc40b6680 0x7febc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febd6a00640 1 -- 172.21.15.27:0/1898411218 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7febd00e7d60 con 0x7febe0075750 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 -- 172.21.15.27:0/3654145845 shutdown_connections 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 --2- 172.21.15.27:0/3654145845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3f3e10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 -- 172.21.15.27:0/3654145845 >> 172.21.15.27:0/3654145845 conn(0x55992a043300 msgr2=0x55992a3f1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 -- 172.21.15.27:0/3654145845 shutdown_connections 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.223+0000 7febebbee740 1 -- 172.21.15.27:0/3654145845 wait complete. 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.224+0000 7febdde00640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7febc40b6680 0x7febc40b8b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7febd00e5210 tx=0x7febd000ef90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.224+0000 7febebbee740 1 -- start start 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.224+0000 7febebbee740 1 Processor -- start 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febebbee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febebbee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a02e4d0 0x55992a3ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febebbee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a3ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febebbee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55992a290ab0 con 0x55992a02e4d0 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febebbee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55992a2ff570 con 0x55992a04f1b0 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febebbee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55992a402530 con 0x55992a0262f0 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febebbee740 10 client.0 _ll_register_callbacks cb 0x55992a30e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a02e4d0 0x55992a3ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a02e4d0 0x55992a3ff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40216/0 (socket says 172.21.15.27:40216) 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 -- 172.21.15.27:0/1178109232 learned_addr learned my addr 172.21.15.27:0/1178109232 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febde800640 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febde800640 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a3ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 -- 172.21.15.27:0/1178109232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 msgr2=0x55992a3ff3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3ff3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 -- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 msgr2=0x55992a3ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a3ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.225+0000 7febdd400640 1 -- 172.21.15.27:0/1178109232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55992a0ba790 con 0x55992a02e4d0 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febdd400640 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a02e4d0 0x55992a3ff920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7febcc0129f0 tx=0x7febcc012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c60892aba582c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febbf400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febbf400640 1 -- 172.21.15.27:0/1178109232 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc00fac0 con 0x55992a02e4d0 2024-09-21T07:15:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febbf400640 1 -- 172.21.15.27:0/1178109232 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7febcc00fc60 con 0x55992a02e4d0 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd8036030 con 0x55992a02e4d0 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febbf400640 1 -- 172.21.15.27:0/1178109232 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febcc00fe60 con 0x55992a02e4d0 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febebbee740 10 client.16230 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55992a133620 con 0x55992a02e4d0 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.226+0000 7febbf400640 1 -- 172.21.15.27:0/1178109232 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036030 con 0x55992a02e4d0 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febbf400640 1 -- 172.21.15.27:0/1178109232 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7febcc010550 con 0x55992a02e4d0 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febebbee740 10 client.16230 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febebbee740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febebbee740 20 client.16230 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febebbee740 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55992a00dd80 0x55992a411480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febebbee740 4 client.16230 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55992a36b600 con 0x55992a00dd80 2024-09-21T07:15:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.227+0000 7febde800640 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55992a00dd80 0x55992a411480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.228+0000 7febde800640 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55992a00dd80 0x55992a411480 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72860f33b4bbc04 server_cookie=2807531dd1a6cb62 in_seq=0 out_seq=0 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.228+0000 7febbf400640 10 client.16230 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.229+0000 7febbf400640 1 -- 172.21.15.27:0/1178109232 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55992a36b600 con 0x55992a00dd80 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.229+0000 7febbf400640 10 client.16230 handle_command_reply: tid=0 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.229+0000 7febebbee740 1 client.16230 shutdown 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a02e4d0 msgr2=0x55992a3ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a02e4d0 0x55992a3ff920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7febcc0129f0 tx=0x7febcc012eb0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 shutdown_connections 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55992a00dd80 0x55992a411480 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a3ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a02e4d0 0x55992a3ff920 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 --2- 172.21.15.27:0/1178109232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a3ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febbf400640 0 client.16230 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 >> 172.21.15.27:0/1178109232 conn(0x55992a043300 msgr2=0x55992a3f4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 shutdown_connections 2024-09-21T07:15:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 1 -- 172.21.15.27:0/1178109232 wait complete. 2024-09-21T07:15:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.230+0000 7febebbee740 20 client.16230 trim_cache size 0 max 16384 2024-09-21T07:15:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.245+0000 7febebbee740 1 Processor -- start 2024-09-21T07:15:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.245+0000 7febebbee740 1 -- start start 2024-09-21T07:15:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febebbee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a401010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febebbee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febebbee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febebbee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7febe01b55a0 con 0x55992a04f1b0 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febebbee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7febe01b2e70 con 0x55992a02e4d0 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febebbee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febe01b0cc0 con 0x55992a0262f0 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3fcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40226/0 (socket says 172.21.15.27:40226) 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 -- 172.21.15.27:0/676656004 learned_addr learned my addr 172.21.15.27:0/676656004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a401010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febdd400640 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 -- 172.21.15.27:0/676656004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 msgr2=0x55992a401010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a401010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 -- 172.21.15.27:0/676656004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 msgr2=0x55992a3fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.246+0000 7febde800640 1 -- 172.21.15.27:0/676656004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992a4117f0 con 0x55992a04f1b0 2024-09-21T07:15:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.247+0000 7febde800640 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3fcaf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febd8006010 tx=0x7febd8009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac106a3ba31de562 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.247+0000 7febca600640 1 -- 172.21.15.27:0/676656004 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8030610 con 0x55992a04f1b0 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.247+0000 7febca600640 1 -- 172.21.15.27:0/676656004 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036030 con 0x55992a04f1b0 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.247+0000 7febca600640 1 -- 172.21.15.27:0/676656004 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8004860 con 0x55992a04f1b0 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 -- 172.21.15.27:0/676656004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 msgr2=0x55992a3fcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3fcaf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febd8006010 tx=0x7febd8009830 comp rx=0 tx=0).stop 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 -- 172.21.15.27:0/676656004 shutdown_connections 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a04f1b0 0x55992a3fcaf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a02e4d0 0x55992a3fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 --2- 172.21.15.27:0/676656004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a0262f0 0x55992a401010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 -- 172.21.15.27:0/676656004 >> 172.21.15.27:0/676656004 conn(0x55992a043300 msgr2=0x55992a400900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 -- 172.21.15.27:0/676656004 shutdown_connections 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 -- 172.21.15.27:0/676656004 wait complete. 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.248+0000 7febebbee740 1 -- start start 2024-09-21T07:15:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 1 Processor -- start 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a0262f0 0x55992a407cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a02e4d0 0x55992a405180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a4056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7febe01b1330 con 0x55992a0262f0 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7febe01b27a0 con 0x55992a04f1b0 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febe01b1a00 con 0x55992a02e4d0 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febebbee740 10 client.0 _ll_register_callbacks cb 0x7febcc009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febde800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a0262f0 0x55992a407cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febde800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a0262f0 0x55992a407cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40236/0 (socket says 172.21.15.27:40236) 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febde800640 1 -- 172.21.15.27:0/3020249023 learned_addr learned my addr 172.21.15.27:0/3020249023 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.249+0000 7febdd400640 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a02e4d0 0x55992a405180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febde800640 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a4056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febde800640 1 -- 172.21.15.27:0/3020249023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a02e4d0 msgr2=0x55992a405180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febde800640 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a02e4d0 0x55992a405180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febde800640 1 -- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 msgr2=0x55992a4056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febde800640 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a4056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febde800640 1 -- 172.21.15.27:0/3020249023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55992a405c00 con 0x55992a0262f0 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febde800640 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a0262f0 0x55992a407cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7febd8009620 tx=0x7febd8004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc76e2b05061ad9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febbf400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febbf400640 1 -- 172.21.15.27:0/3020249023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8062060 con 0x55992a0262f0 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febbf400640 1 -- 172.21.15.27:0/3020249023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7febd8004360 con 0x55992a0262f0 2024-09-21T07:15:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd8036030 con 0x55992a0262f0 2024-09-21T07:15:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.250+0000 7febbf400640 1 -- 172.21.15.27:0/3020249023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febd804a040 con 0x55992a0262f0 2024-09-21T07:15:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 10 client.16236 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55992a47bae0 con 0x55992a0262f0 2024-09-21T07:15:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febbf400640 1 -- 172.21.15.27:0/3020249023 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8036030 con 0x55992a0262f0 2024-09-21T07:15:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febbf400640 1 -- 172.21.15.27:0/3020249023 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7febcc009ce0 con 0x55992a0262f0 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 10 client.16236 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 20 client.16236 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559929fe31c0 0x55992a47de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 4 client.16236 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55992a3670b0 con 0x559929fe31c0 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.251+0000 7febdd400640 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559929fe31c0 0x55992a47de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.252+0000 7febdd400640 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559929fe31c0 0x55992a47de20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bccba43dc7ec02c5 server_cookie=131a61b785a17aca in_seq=0 out_seq=0 2024-09-21T07:15:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.252+0000 7febbf400640 10 client.16236 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febbf400640 1 -- 172.21.15.27:0/3020249023 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55992a3670b0 con 0x559929fe31c0 2024-09-21T07:15:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febbf400640 10 client.16236 handle_command_reply: tid=0 2024-09-21T07:15:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 client.16236 shutdown 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a0262f0 msgr2=0x55992a407cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a0262f0 0x55992a407cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7febd8009620 tx=0x7febd8004a80 comp rx=0 tx=0).stop 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 shutdown_connections 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559929fe31c0 0x55992a47de20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992a02e4d0 0x55992a405180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febbf400640 0 client.16236 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55992a04f1b0 0x55992a4056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 --2- 172.21.15.27:0/3020249023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55992a0262f0 0x55992a407cc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 >> 172.21.15.27:0/3020249023 conn(0x55992a043300 msgr2=0x55992a3f4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 shutdown_connections 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 1 -- 172.21.15.27:0/3020249023 wait complete. 2024-09-21T07:15:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febebbee740 20 client.16236 trim_cache size 0 max 16384 2024-09-21T07:15:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:15:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:15:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:15:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:15:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7febc40b6680 msgr2=0x7febc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7febc40b6680 0x7febc40b8b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7febd00e5210 tx=0x7febd000ef90 comp rx=0 tx=0).stop 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe0075750 msgr2=0x7febe019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe0075750 0x7febe019c3e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febd000ea60 tx=0x7febd000ef20 comp rx=0 tx=0).stop 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 shutdown_connections 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe010feb0 0x7febe01a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7febc40b6680 0x7febc40b8b90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febe00760a0 0x7febe019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 --2- 172.21.15.27:0/1898411218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe0075750 0x7febe019c3e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.267+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 >> 172.21.15.27:0/1898411218 conn(0x7febe00ff360 msgr2=0x7febe01099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.268+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 shutdown_connections 2024-09-21T07:15:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:24.268+0000 7febdf200640 1 -- 172.21.15.27:0/1898411218 wait complete. 2024-09-21T07:15:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:24 smithi119 ceph-mon[34519]: pgmap v449: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-21T07:15:25.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:24 smithi027 ceph-mon[30779]: pgmap v449: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-21T07:15:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:24 smithi153 ceph-mon[34747]: pgmap v449: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-21T07:15:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:26 smithi119 ceph-mon[34519]: pgmap v450: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-21T07:15:27.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:26 smithi027 ceph-mon[30779]: pgmap v450: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-21T07:15:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:26 smithi153 ceph-mon[34747]: pgmap v450: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-21T07:15:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:28 smithi119 ceph-mon[34519]: pgmap v451: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T07:15:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:28 smithi027 ceph-mon[30779]: pgmap v451: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T07:15:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:28 smithi153 ceph-mon[34747]: pgmap v451: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T07:15:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:30 smithi119 ceph-mon[34519]: pgmap v452: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-21T07:15:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:30 smithi027 ceph-mon[30779]: pgmap v452: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-21T07:15:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:30 smithi153 ceph-mon[34747]: pgmap v452: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-21T07:15:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:32 smithi119 ceph-mon[34519]: pgmap v453: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-21T07:15:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:32 smithi027 ceph-mon[30779]: pgmap v453: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-21T07:15:33.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:32 smithi153 ceph-mon[34747]: pgmap v453: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-21T07:15:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:34 smithi119 ceph-mon[34519]: pgmap v454: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-09-21T07:15:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:34 smithi027 ceph-mon[30779]: pgmap v454: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-09-21T07:15:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:34 smithi153 ceph-mon[34747]: pgmap v454: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-09-21T07:15:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:36 smithi027 ceph-mon[30779]: pgmap v455: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-21T07:15:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:36 smithi153 ceph-mon[34747]: pgmap v455: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-21T07:15:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:36 smithi119 ceph-mon[34519]: pgmap v455: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-21T07:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 -- 172.21.15.27:0/1347288806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0103ae0 msgr2=0x7f7ef0103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 --2- 172.21.15.27:0/1347288806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0103ae0 0x7f7ef0103ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009900 tx=0x7f7ee002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 -- 172.21.15.27:0/1347288806 shutdown_connections 2024-09-21T07:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 --2- 172.21.15.27:0/1347288806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0109d60 0x7f7ef0110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 --2- 172.21.15.27:0/1347288806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef0109410 0x7f7ef0109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 --2- 172.21.15.27:0/1347288806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0103ae0 0x7f7ef0103ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 -- 172.21.15.27:0/1347288806 >> 172.21.15.27:0/1347288806 conn(0x7f7ef00ff520 msgr2=0x7f7ef0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 -- 172.21.15.27:0/1347288806 shutdown_connections 2024-09-21T07:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.463+0000 7f7ef6600640 1 -- 172.21.15.27:0/1347288806 wait complete. 2024-09-21T07:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.465+0000 7f7ef6600640 1 Processor -- start 2024-09-21T07:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.465+0000 7f7ef6600640 1 -- start start 2024-09-21T07:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.465+0000 7f7ef6600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0103ae0 0x7f7ef019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7ef6600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0109410 0x7f7ef019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7ef6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef0109d60 0x7f7ef019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7ef6600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0113a20 con 0x7f7ef0103ae0 2024-09-21T07:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7ef6600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01138a0 con 0x7f7ef0109d60 2024-09-21T07:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7ef6600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0113ba0 con 0x7f7ef0109410 2024-09-21T07:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0103ae0 0x7f7ef019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0103ae0 0x7f7ef019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38310/0 (socket says 172.21.15.27:38310) 2024-09-21T07:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 -- 172.21.15.27:0/3322036336 learned_addr learned my addr 172.21.15.27:0/3322036336 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0109410 0x7f7ef019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eefe00640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef0109d60 0x7f7ef019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 -- 172.21.15.27:0/3322036336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0109410 msgr2=0x7f7ef019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0109410 0x7f7ef019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 -- 172.21.15.27:0/3322036336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef0109d60 msgr2=0x7f7ef019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef0109d60 0x7f7ef019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.466+0000 7f7eef400640 1 -- 172.21.15.27:0/3322036336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef01a03d0 con 0x7f7ef0103ae0 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.467+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0109410 0x7f7ef019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.467+0000 7f7eef400640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0103ae0 0x7f7ef019a4a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0005940 tx=0x7f7ee00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.467+0000 7f7eecc00640 1 -- 172.21.15.27:0/3322036336 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee003f030 con 0x7f7ef0103ae0 2024-09-21T07:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.467+0000 7f7eecc00640 1 -- 172.21.15.27:0/3322036336 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee00044c0 con 0x7f7ef0103ae0 2024-09-21T07:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.467+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef01a0660 con 0x7f7ef0103ae0 2024-09-21T07:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.467+0000 7f7eecc00640 1 -- 172.21.15.27:0/3322036336 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0035070 con 0x7f7ef0103ae0 2024-09-21T07:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.467+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01a6e50 con 0x7f7ef0103ae0 2024-09-21T07:15:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.472+0000 7f7eecc00640 1 -- 172.21.15.27:0/3322036336 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7ee0042030 con 0x7f7ef0103ae0 2024-09-21T07:15:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.472+0000 7f7efd6b9740 1 Processor -- start 2024-09-21T07:15:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7efd6b9740 1 -- start start 2024-09-21T07:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e40e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e40eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563e1e37f6b0 con 0x563e1e03c2f0 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563e1e36eae0 con 0x563e1e0651b0 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e1e3728f0 con 0x563e1e0444d0 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.473+0000 7f7eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e40e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e40e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38320/0 (socket says 172.21.15.27:38320) 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e40eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eef400640 1 -- 172.21.15.27:0/60139489 learned_addr learned my addr 172.21.15.27:0/60139489 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eefe00640 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eecc00640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ed40bef10 0x7f7ed40c1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eef400640 1 -- 172.21.15.27:0/60139489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 msgr2=0x563e1e40eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eef400640 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e40eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eef400640 1 -- 172.21.15.27:0/60139489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 msgr2=0x563e1e40e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eef400640 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eefe00640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ed40bef10 0x7f7ed40c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eef400640 1 -- 172.21.15.27:0/60139489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1e33b780 con 0x563e1e03c2f0 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.474+0000 7f7eecc00640 1 -- 172.21.15.27:0/3322036336 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7ee0107dc0 con 0x7f7ef0103ae0 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.475+0000 7f7eef400640 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e40e580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ee01050d0 tx=0x7f7ee00862f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802ca4c5e53b009b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.475+0000 7f7ee4c00640 1 -- 172.21.15.27:0/60139489 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00cd080 con 0x563e1e03c2f0 2024-09-21T07:15:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.475+0000 7f7ee4c00640 1 -- 172.21.15.27:0/60139489 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee00cd420 con 0x563e1e03c2f0 2024-09-21T07:15:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.475+0000 7f7ee4c00640 1 -- 172.21.15.27:0/60139489 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee002e260 con 0x563e1e03c2f0 2024-09-21T07:15:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7eefe00640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ed40bef10 0x7f7ed40c1420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8005c40 tx=0x7f7ee8004ab0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 -- 172.21.15.27:0/60139489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 msgr2=0x563e1e40e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e40e580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ee01050d0 tx=0x7f7ee00862f0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 -- 172.21.15.27:0/60139489 shutdown_connections 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e40eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e40e580 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/60139489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 -- 172.21.15.27:0/60139489 >> 172.21.15.27:0/60139489 conn(0x563e1e059300 msgr2=0x563e1e40bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.476+0000 7f7efd6b9740 1 -- 172.21.15.27:0/60139489 shutdown_connections 2024-09-21T07:15:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.477+0000 7f7efd6b9740 1 -- 172.21.15.27:0/60139489 wait complete. 2024-09-21T07:15:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.477+0000 7f7efd6b9740 1 -- start start 2024-09-21T07:15:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.477+0000 7f7efd6b9740 1 Processor -- start 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e41be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e415380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e415a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563e1e2a6ab0 con 0x563e1e03c2f0 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563e1e315570 con 0x563e1e0651b0 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e1e37f6b0 con 0x563e1e0444d0 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7efd6b9740 10 client.0 _ll_register_callbacks cb 0x563e1e324100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e41be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7eef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e41be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38322/0 (socket says 172.21.15.27:38322) 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7eef400640 1 -- 172.21.15.27:0/347982032 learned_addr learned my addr 172.21.15.27:0/347982032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e415380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.478+0000 7f7eefe00640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e415a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7eef400640 1 -- 172.21.15.27:0/347982032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 msgr2=0x563e1e415380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7eef400640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e415380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7eef400640 1 -- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 msgr2=0x563e1e415a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7eef400640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e415a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7eef400640 1 -- 172.21.15.27:0/347982032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1e0d0790 con 0x563e1e03c2f0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7eefe00640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e415a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7eef400640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e41be60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00d2500 tx=0x7f7ee00c9db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e6e9017ffd70c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7edd600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7edd600640 1 -- 172.21.15.27:0/347982032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0049070 con 0x563e1e03c2f0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee00cd420 con 0x563e1e03c2f0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.479+0000 7f7edd600640 1 -- 172.21.15.27:0/347982032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7ee00d6ff0 con 0x563e1e03c2f0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7edd600640 1 -- 172.21.15.27:0/347982032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ee00cd080 con 0x563e1e03c2f0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 10 client.16248 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e1e149620 con 0x563e1e03c2f0 2024-09-21T07:15:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7edd600640 1 -- 172.21.15.27:0/347982032 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee00cd420 con 0x563e1e03c2f0 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7edd600640 1 -- 172.21.15.27:0/347982032 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7ee00c9260 con 0x563e1e03c2f0 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 10 client.16248 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 20 client.16248 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1e023d80 0x563e1e426df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 4 client.16248 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.480+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e1e381600 con 0x563e1e023d80 2024-09-21T07:15:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.481+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1e023d80 0x563e1e426df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.481+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1e023d80 0x563e1e426df0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1173ce594e4d446 server_cookie=d3fba7bf0d98a424 in_seq=0 out_seq=0 2024-09-21T07:15:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.481+0000 7f7edd600640 10 client.16248 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.482+0000 7f7edd600640 1 -- 172.21.15.27:0/347982032 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e1e381600 con 0x563e1e023d80 2024-09-21T07:15:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.482+0000 7f7edd600640 10 client.16248 handle_command_reply: tid=0 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.482+0000 7f7efd6b9740 1 client.16248 shutdown 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 msgr2=0x563e1e41be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e41be60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00d2500 tx=0x7f7ee00c9db0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 shutdown_connections 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1e023d80 0x563e1e426df0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e415380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e415a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/347982032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e03c2f0 0x563e1e41be60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 >> 172.21.15.27:0/347982032 conn(0x563e1e059300 msgr2=0x563e1e40f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7edd600640 0 client.16248 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 shutdown_connections 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 1 -- 172.21.15.27:0/347982032 wait complete. 2024-09-21T07:15:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.483+0000 7f7efd6b9740 20 client.16248 trim_cache size 0 max 16384 2024-09-21T07:15:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 Processor -- start 2024-09-21T07:15:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 -- start start 2024-09-21T07:15:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e03c2f0 0x563e1e40b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0444d0 0x563e1e40bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01b14a0 con 0x563e1e0444d0 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01aed70 con 0x563e1e0651b0 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.512+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01acbc0 con 0x563e1e03c2f0 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0444d0 0x563e1e40bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0444d0 0x563e1e40bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38330/0 (socket says 172.21.15.27:38330) 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 -- 172.21.15.27:0/3450994337 learned_addr learned my addr 172.21.15.27:0/3450994337 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7eefe00640 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7eef400640 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e03c2f0 0x563e1e40b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 -- 172.21.15.27:0/3450994337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e03c2f0 msgr2=0x563e1e40b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e03c2f0 0x563e1e40b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 -- 172.21.15.27:0/3450994337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 msgr2=0x563e1e40a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.513+0000 7f7ee7e00640 1 -- 172.21.15.27:0/3450994337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1e410c30 con 0x563e1e0444d0 2024-09-21T07:15:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.514+0000 7f7ee7e00640 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0444d0 0x563e1e40bd70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef006c8d0 tx=0x7f7ed8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a14b77b23e29f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.514+0000 7f7edfe00640 1 -- 172.21.15.27:0/3450994337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8005220 con 0x563e1e0444d0 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.514+0000 7f7edfe00640 1 -- 172.21.15.27:0/3450994337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed8004c50 con 0x563e1e0444d0 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.514+0000 7f7edfe00640 1 -- 172.21.15.27:0/3450994337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed801f8f0 con 0x563e1e0444d0 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 -- 172.21.15.27:0/3450994337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0444d0 msgr2=0x563e1e40bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0444d0 0x563e1e40bd70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef006c8d0 tx=0x7f7ed8005f90 comp rx=0 tx=0).stop 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 -- 172.21.15.27:0/3450994337 shutdown_connections 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e0651b0 0x563e1e40a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0444d0 0x563e1e40bd70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/3450994337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e03c2f0 0x563e1e40b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 -- 172.21.15.27:0/3450994337 >> 172.21.15.27:0/3450994337 conn(0x563e1e059300 msgr2=0x563e1e413b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 -- 172.21.15.27:0/3450994337 shutdown_connections 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.515+0000 7f7efd6b9740 1 -- 172.21.15.27:0/3450994337 wait complete. 2024-09-21T07:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.516+0000 7f7efd6b9740 1 -- start start 2024-09-21T07:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.516+0000 7f7efd6b9740 1 Processor -- start 2024-09-21T07:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e03c2f0 0x563e1e421b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e41d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7efd6b9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0651b0 0x563e1e41d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01ad230 con 0x563e1e0651b0 2024-09-21T07:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01ae6a0 con 0x563e1e03c2f0 2024-09-21T07:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7efd6b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01ad900 con 0x563e1e0444d0 2024-09-21T07:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7efd6b9740 10 client.0 _ll_register_callbacks cb 0x7f7ee8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7eefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0651b0 0x563e1e41d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7ee7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e41d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7eefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0651b0 0x563e1e41d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38334/0 (socket says 172.21.15.27:38334) 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7eefe00640 1 -- 172.21.15.27:0/1114887300 learned_addr learned my addr 172.21.15.27:0/1114887300 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.517+0000 7f7eef400640 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e03c2f0 0x563e1e421b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7eefe00640 1 -- 172.21.15.27:0/1114887300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 msgr2=0x563e1e41d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7eefe00640 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e41d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7eefe00640 1 -- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e03c2f0 msgr2=0x563e1e421b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7eefe00640 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e03c2f0 0x563e1e421b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7eefe00640 1 -- 172.21.15.27:0/1114887300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1e41da80 con 0x563e1e0651b0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7eefe00640 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0651b0 0x563e1e41d540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8004280 tx=0x7f7ee8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ced0f73cb6f4344 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7edd600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7edd600640 1 -- 172.21.15.27:0/1114887300 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800f710 con 0x563e1e0651b0 2024-09-21T07:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7edd600640 1 -- 172.21.15.27:0/1114887300 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7ee800f8b0 con 0x563e1e0651b0 2024-09-21T07:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.518+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed8004c50 con 0x563e1e0651b0 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7edd600640 1 -- 172.21.15.27:0/1114887300 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ee8036050 con 0x563e1e0651b0 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 10 client.16254 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e1e4913d0 con 0x563e1e0651b0 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7edd600640 1 -- 172.21.15.27:0/1114887300 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed8004c50 con 0x563e1e0651b0 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7edd600640 1 -- 172.21.15.27:0/1114887300 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7ee802a020 con 0x563e1e0651b0 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 10 client.16254 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 20 client.16254 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1dff91c0 0x563e1e493bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 4 client.16254 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.519+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e1e37d0b0 con 0x563e1dff91c0 2024-09-21T07:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.520+0000 7f7eef400640 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1dff91c0 0x563e1e493bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.520+0000 7f7eef400640 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1dff91c0 0x563e1e493bf0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa6936043935a4a server_cookie=d251d2fd2aa1dc5b in_seq=0 out_seq=0 2024-09-21T07:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.520+0000 7f7edd600640 10 client.16254 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.531+0000 7f7edd600640 1 -- 172.21.15.27:0/1114887300 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e1e37d0b0 con 0x563e1dff91c0 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.531+0000 7f7edd600640 10 client.16254 handle_command_reply: tid=0 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.531+0000 7f7efd6b9740 1 client.16254 shutdown 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0651b0 msgr2=0x563e1e41d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0651b0 0x563e1e41d540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8004280 tx=0x7f7ee8009500 comp rx=0 tx=0).stop 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 shutdown_connections 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563e1dff91c0 0x563e1e493bf0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e1e0444d0 0x563e1e41d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563e1e0651b0 0x563e1e41d540 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 --2- 172.21.15.27:0/1114887300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563e1e03c2f0 0x563e1e421b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7edd600640 0 client.16254 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 >> 172.21.15.27:0/1114887300 conn(0x563e1e059300 msgr2=0x563e1e4060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 shutdown_connections 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 1 -- 172.21.15.27:0/1114887300 wait complete. 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.532+0000 7f7efd6b9740 20 client.16254 trim_cache size 0 max 16384 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ed40bef10 msgr2=0x7f7ed40c1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ed40bef10 0x7f7ed40c1420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8005c40 tx=0x7f7ee8004ab0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0103ae0 msgr2=0x7f7ef019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0103ae0 0x7f7ef019a4a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0005940 tx=0x7f7ee00040c0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 shutdown_connections 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0109410 0x7f7ef019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7ed40bef10 0x7f7ed40c1420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ef0109d60 0x7f7ef019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 --2- 172.21.15.27:0/3322036336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef0103ae0 0x7f7ef019a4a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 >> 172.21.15.27:0/3322036336 conn(0x7f7ef00ff520 msgr2=0x7f7ef0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 shutdown_connections 2024-09-21T07:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:37.533+0000 7f7ef6600640 1 -- 172.21.15.27:0/3322036336 wait complete. 2024-09-21T07:15:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:38 smithi027 ceph-mon[30779]: pgmap v456: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-21T07:15:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:38 smithi153 ceph-mon[34747]: pgmap v456: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-21T07:15:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:38 smithi119 ceph-mon[34519]: pgmap v456: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-21T07:15:41.087 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:40 smithi119 ceph-mon[34519]: pgmap v457: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-21T07:15:41.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:40 smithi027 ceph-mon[30779]: pgmap v457: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-21T07:15:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:40 smithi153 ceph-mon[34747]: pgmap v457: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-21T07:15:42.824 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:15:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:42 smithi027 ceph-mon[30779]: pgmap v458: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-21T07:15:43.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:42 smithi153 ceph-mon[34747]: pgmap v458: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-21T07:15:43.224 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:42 smithi119 ceph-mon[34519]: pgmap v458: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 -- 172.21.15.27:0/3472835594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 msgr2=0x7fe1bc109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 --2- 172.21.15.27:0/3472835594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 0x7fe1bc109700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009900 tx=0x7fe1a802a410 comp rx=0 tx=0).stop 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 -- 172.21.15.27:0/3472835594 shutdown_connections 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 --2- 172.21.15.27:0/3472835594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc109c40 0x7fe1bc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 --2- 172.21.15.27:0/3472835594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 0x7fe1bc109700 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 --2- 172.21.15.27:0/3472835594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc1039b0 0x7fe1bc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 -- 172.21.15.27:0/3472835594 >> 172.21.15.27:0/3472835594 conn(0x7fe1bc0ff400 msgr2=0x7fe1bc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 -- 172.21.15.27:0/3472835594 shutdown_connections 2024-09-21T07:15:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.906+0000 7fe1bb200640 1 -- 172.21.15.27:0/3472835594 wait complete. 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.907+0000 7fe1bb200640 1 Processor -- start 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.907+0000 7fe1bb200640 1 -- start start 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1bb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc1039b0 0x7fe1bc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1bb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 0x7fe1bc19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1bb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc109c40 0x7fe1bc1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1bb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc113950 con 0x7fe1bc1092f0 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1bb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1137d0 con 0x7fe1bc1039b0 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1bb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc113ad0 con 0x7fe1bc109c40 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc1039b0 0x7fe1bc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1b9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 0x7fe1bc19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc1039b0 0x7fe1bc19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55452/0 (socket says 172.21.15.27:55452) 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1b9e00640 1 -- 172.21.15.27:0/1289928511 learned_addr learned my addr 172.21.15.27:0/1289928511 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.908+0000 7fe1ba800640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc109c40 0x7fe1bc1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b9e00640 1 -- 172.21.15.27:0/1289928511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc109c40 msgr2=0x7fe1bc1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b9e00640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc109c40 0x7fe1bc1a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b9e00640 1 -- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 msgr2=0x7fe1bc19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b9e00640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 0x7fe1bc19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b9e00640 1 -- 172.21.15.27:0/1289928511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc1a24b0 con 0x7fe1bc1039b0 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b9400640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 0x7fe1bc19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1ba800640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc109c40 0x7fe1bc1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b9e00640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc1039b0 0x7fe1bc19c540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00e9f0 tx=0x7fe1ac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b2a00640 1 -- 172.21.15.27:0/1289928511 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac00cd80 con 0x7fe1bc1039b0 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc1a27a0 con 0x7fe1bc1039b0 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b2a00640 1 -- 172.21.15.27:0/1289928511 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1ac0054d0 con 0x7fe1bc1039b0 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.909+0000 7fe1b2a00640 1 -- 172.21.15.27:0/1289928511 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac010690 con 0x7fe1bc1039b0 2024-09-21T07:15:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.910+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a8f00 con 0x7fe1bc1039b0 2024-09-21T07:15:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.911+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc104c30 con 0x7fe1bc1039b0 2024-09-21T07:15:43.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.913+0000 7fe1b2a00640 1 -- 172.21.15.27:0/1289928511 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe1ac004e90 con 0x7fe1bc1039b0 2024-09-21T07:15:43.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.915+0000 7fe1b2a00640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1a00b6680 0x7fe1a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.915+0000 7fe1b2a00640 1 -- 172.21.15.27:0/1289928511 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe1ac0e8210 con 0x7fe1bc1039b0 2024-09-21T07:15:43.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.915+0000 7fe1b9400640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1a00b6680 0x7fe1a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:43.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.916+0000 7fe1b9400640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1a00b6680 0x7fe1a00b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe1a802aa00 tx=0x7fe1a8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:43.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:43.916+0000 7fe1b2a00640 1 -- 172.21.15.27:0/1289928511 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1ac0ae3b0 con 0x7fe1bc1039b0 2024-09-21T07:15:44.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.155+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1bc103dc0 con 0x7fe1bc1039b0 2024-09-21T07:15:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.156+0000 7fe1b2a00640 1 -- 172.21.15.27:0/1289928511 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fe1bc103dc0 con 0x7fe1bc1039b0 2024-09-21T07:15:44.159 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:15:44.159 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:15:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:15:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1a00b6680 msgr2=0x7fe1a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1a00b6680 0x7fe1a00b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe1a802aa00 tx=0x7fe1a8005a20 comp rx=0 tx=0).stop 2024-09-21T07:15:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc1039b0 msgr2=0x7fe1bc19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc1039b0 0x7fe1bc19c540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00e9f0 tx=0x7fe1ac00edf0 comp rx=0 tx=0).stop 2024-09-21T07:15:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 shutdown_connections 2024-09-21T07:15:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc109c40 0x7fe1bc1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe1a00b6680 0x7fe1a00b8b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc1092f0 0x7fe1bc19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 --2- 172.21.15.27:0/1289928511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1bc1039b0 0x7fe1bc19c540 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 >> 172.21.15.27:0/1289928511 conn(0x7fe1bc0ff400 msgr2=0x7fe1bc106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 shutdown_connections 2024-09-21T07:15:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:44.159+0000 7fe1bb200640 1 -- 172.21.15.27:0/1289928511 wait complete. 2024-09-21T07:15:44.961 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 7c734edf --release --await' 2024-09-21T07:15:44.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 7c734edf --release --await 2024-09-21T07:15:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:44 smithi027 ceph-mon[30779]: pgmap v459: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-21T07:15:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:44 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1289928511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:44 smithi153 ceph-mon[34747]: pgmap v459: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-21T07:15:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:44 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1289928511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:44 smithi119 ceph-mon[34519]: pgmap v459: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-21T07:15:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:44 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1289928511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:45.352 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 -- 172.21.15.27:0/1690535681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 msgr2=0x7f732410fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 --2- 172.21.15.27:0/1690535681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732410fa30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7310009900 tx=0x7f731002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 -- 172.21.15.27:0/1690535681 shutdown_connections 2024-09-21T07:15:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 --2- 172.21.15.27:0/1690535681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f732410ff70 0x7f7324112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 --2- 172.21.15.27:0/1690535681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732410fa30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 --2- 172.21.15.27:0/1690535681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7324075750 0x7f7324075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 -- 172.21.15.27:0/1690535681 >> 172.21.15.27:0/1690535681 conn(0x7f73240ff400 msgr2=0x7f7324101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 -- 172.21.15.27:0/1690535681 shutdown_connections 2024-09-21T07:15:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.976+0000 7f7328c00640 1 -- 172.21.15.27:0/1690535681 wait complete. 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.977+0000 7f7328c00640 1 Processor -- start 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.977+0000 7f7328c00640 1 -- start start 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.977+0000 7f7328c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7324075750 0x7f732419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7328c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7328c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f732410ff70 0x7f73241a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7328c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7324115ae0 con 0x7f73240760a0 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7328c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7324115960 con 0x7f7324075750 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7328c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7324115c60 con 0x7f732410ff70 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34122/0 (socket says 172.21.15.27:34122) 2024-09-21T07:15:45.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7323400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7324075750 0x7f732419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 -- 172.21.15.27:0/2827090825 learned_addr learned my addr 172.21.15.27:0/2827090825 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:45.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7323e00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f732410ff70 0x7f73241a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 -- 172.21.15.27:0/2827090825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f732410ff70 msgr2=0x7f73241a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:45.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f732410ff70 0x7f73241a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 -- 172.21.15.27:0/2827090825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7324075750 msgr2=0x7f732419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7324075750 0x7f732419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.978+0000 7f7322a00640 1 -- 172.21.15.27:0/2827090825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73241a2440 con 0x7f73240760a0 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.979+0000 7f7322a00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732419ca70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f731002a910 tx=0x7f7310031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.979+0000 7f731be00640 1 -- 172.21.15.27:0/2827090825 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731003f030 con 0x7f73240760a0 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.979+0000 7f731be00640 1 -- 172.21.15.27:0/2827090825 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7310004490 con 0x7f73240760a0 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.979+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73241a26d0 con 0x7f73240760a0 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.979+0000 7f731be00640 1 -- 172.21.15.27:0/2827090825 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310035070 con 0x7f73240760a0 2024-09-21T07:15:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.979+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a8e90 con 0x7f73240760a0 2024-09-21T07:15:45.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 Processor -- start 2024-09-21T07:15:45.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 -- start start 2024-09-21T07:15:45.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f731be00640 1 -- 172.21.15.27:0/2827090825 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7310042030 con 0x7f73240760a0 2024-09-21T07:15:45.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd793c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd793c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd793cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557fd78b3290 con 0x557fd756c660 2024-09-21T07:15:45.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557fd78a26c0 con 0x557fd7595520 2024-09-21T07:15:45.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.983+0000 7f7330ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557fd78a64d0 con 0x557fd7574840 2024-09-21T07:15:45.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd793cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7323e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd793c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7323400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd793c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd793cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50330/0 (socket says 172.21.15.27:50330) 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 -- 172.21.15.27:0/1337213646 learned_addr learned my addr 172.21.15.27:0/1337213646 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 -- 172.21.15.27:0/1337213646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 msgr2=0x557fd793c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd793c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 -- 172.21.15.27:0/1337213646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 msgr2=0x557fd793c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd793c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7322a00640 1 -- 172.21.15.27:0/1337213646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd782f460 con 0x557fd7574840 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.984+0000 7f7323e00640 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd793c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:45.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.985+0000 7f7322a00640 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd793cdc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7310105030 tx=0x7f731010bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa5ee63bcf79616 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:45.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.985+0000 7f730fe00640 1 -- 172.21.15.27:0/1337213646 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310117070 con 0x557fd7574840 2024-09-21T07:15:45.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.985+0000 7f730fe00640 1 -- 172.21.15.27:0/1337213646 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731010d8c0 con 0x557fd7574840 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.985+0000 7f730fe00640 1 -- 172.21.15.27:0/1337213646 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310109030 con 0x557fd7574840 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.985+0000 7f731be00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73080b6700 0x7f73080b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f731be00640 1 -- 172.21.15.27:0/2827090825 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7310107cc0 con 0x7f73240760a0 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7323e00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73080b6700 0x7f73080b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 -- 172.21.15.27:0/1337213646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 msgr2=0x557fd793cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd793cdc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7310105030 tx=0x7f731010bf90 comp rx=0 tx=0).stop 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 -- 172.21.15.27:0/1337213646 shutdown_connections 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd793cdc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd793c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 --2- 172.21.15.27:0/1337213646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd793c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 -- 172.21.15.27:0/1337213646 >> 172.21.15.27:0/1337213646 conn(0x557fd7589910 msgr2=0x557fd793a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 -- 172.21.15.27:0/1337213646 shutdown_connections 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.986+0000 7f7330ffc740 1 -- 172.21.15.27:0/1337213646 wait complete. 2024-09-21T07:15:45.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.987+0000 7f7323e00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73080b6700 0x7f73080b8c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f731c009fd0 tx=0x7f731c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:45.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.987+0000 7f7330ffc740 1 -- start start 2024-09-21T07:15:45.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.987+0000 7f7330ffc740 1 Processor -- start 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd7947f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7574840 0x557fd7948440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7595520 0x557fd7953b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7330ffc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557fd7703bb0 con 0x557fd756c660 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7330ffc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557fd783d6c0 con 0x557fd7574840 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7330ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557fd794a8e0 con 0x557fd7595520 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7330ffc740 10 client.0 _ll_register_callbacks cb 0x557fd7858490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7323400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd7947f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7323400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd7947f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34130/0 (socket says 172.21.15.27:34130) 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7323400640 1 -- 172.21.15.27:0/62228075 learned_addr learned my addr 172.21.15.27:0/62228075 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7322a00640 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7574840 0x557fd7948440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.988+0000 7f7323e00640 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7595520 0x557fd7953b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f7322a00640 1 -- 172.21.15.27:0/62228075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7595520 msgr2=0x557fd7953b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:45.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f7322a00640 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7595520 0x557fd7953b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f7322a00640 1 -- 172.21.15.27:0/62228075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 msgr2=0x557fd7947f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f7322a00640 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd7947f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f7322a00640 1 -- 172.21.15.27:0/62228075 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd7600b00 con 0x557fd7574840 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f7322a00640 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7574840 0x557fd7948440 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7310031440 tx=0x7f7310009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3c9884e9938a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f730cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f730cc00640 1 -- 172.21.15.27:0/62228075 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310049070 con 0x557fd7574840 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731010d8c0 con 0x557fd7574840 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.989+0000 7f730cc00640 1 -- 172.21.15.27:0/62228075 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f73100c9ae0 con 0x557fd7574840 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f730cc00640 1 -- 172.21.15.27:0/62228075 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7310109070 con 0x557fd7574840 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 10 client.26306 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:45.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557fd7677960 con 0x557fd7574840 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f730cc00640 1 -- 172.21.15.27:0/62228075 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731010d8c0 con 0x557fd7574840 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f730cc00640 1 -- 172.21.15.27:0/62228075 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f731010da70 con 0x557fd7574840 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 10 client.26306 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 10 client.26306 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 20 client.26306 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7553ea0 0x557fd79599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 4 client.26306 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.990+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fd78b51e0 con 0x557fd7553ea0 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.991+0000 7f7323400640 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7553ea0 0x557fd79599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:45.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.991+0000 7f7323400640 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7553ea0 0x557fd79599a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cffd5139bfcd4cf4 server_cookie=f67a5feb5e790b89 in_seq=0 out_seq=0 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.991+0000 7f730cc00640 10 client.26306 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f730cc00640 1 -- 172.21.15.27:0/62228075 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557fd78b51e0 con 0x557fd7553ea0 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f730cc00640 10 client.26306 handle_command_reply: tid=0 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 client.26306 shutdown 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7574840 msgr2=0x557fd7948440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7574840 0x557fd7948440 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7310031440 tx=0x7f7310009260 comp rx=0 tx=0).stop 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 shutdown_connections 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7553ea0 0x557fd79599a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7595520 0x557fd7953b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7574840 0x557fd7948440 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f730cc00640 0 client.26306 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 --2- 172.21.15.27:0/62228075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd7947f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.992+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 >> 172.21.15.27:0/62228075 conn(0x557fd7589910 msgr2=0x557fd793f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:45.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.993+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 shutdown_connections 2024-09-21T07:15:45.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.993+0000 7f7330ffc740 1 -- 172.21.15.27:0/62228075 wait complete. 2024-09-21T07:15:45.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:45.993+0000 7f7330ffc740 20 client.26306 trim_cache size 0 max 16384 2024-09-21T07:15:46.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 Processor -- start 2024-09-21T07:15:46.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 -- start start 2024-09-21T07:15:46.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd756c660 0x557fd79430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:46.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd7949be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd7595520 0x557fd79539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b5680 con 0x557fd7595520 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b2f50 con 0x557fd756c660 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.011+0000 7f7330ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b0da0 con 0x557fd7574840 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd7595520 0x557fd79539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7322a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd7949be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd756c660 0x557fd79430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7322a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd7949be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50358/0 (socket says 172.21.15.27:50358) 2024-09-21T07:15:46.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7322a00640 1 -- 172.21.15.27:0/2307385362 learned_addr learned my addr 172.21.15.27:0/2307385362 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323400640 1 -- 172.21.15.27:0/2307385362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 msgr2=0x557fd7949be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323400640 1 --2- 172.21.15.27:0/2307385362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd7949be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323400640 1 -- 172.21.15.27:0/2307385362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd7595520 msgr2=0x557fd79539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323400640 1 --2- 172.21.15.27:0/2307385362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd7595520 0x557fd79539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323400640 1 -- 172.21.15.27:0/2307385362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd793a750 con 0x557fd756c660 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.012+0000 7f7323400640 1 --2- 172.21.15.27:0/2307385362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd756c660 0x557fd79430e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x557fd783d7c0 tx=0x7f7314005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20183ef92f3c4332 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.013+0000 7f730f400640 1 -- 172.21.15.27:0/2307385362 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314004030 con 0x557fd756c660 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.013+0000 7f730f400640 1 -- 172.21.15.27:0/2307385362 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731400f810 con 0x557fd756c660 2024-09-21T07:15:46.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.013+0000 7f730f400640 1 -- 172.21.15.27:0/2307385362 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140038e0 con 0x557fd756c660 2024-09-21T07:15:46.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 -- 172.21.15.27:0/2307385362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd756c660 msgr2=0x557fd79430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:46.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 --2- 172.21.15.27:0/2307385362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd756c660 0x557fd79430e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x557fd783d7c0 tx=0x7f7314005f90 comp rx=0 tx=0).stop 2024-09-21T07:15:46.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 -- 172.21.15.27:0/2307385362 shutdown_connections 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 --2- 172.21.15.27:0/2307385362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd7949be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 --2- 172.21.15.27:0/2307385362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd7595520 0x557fd79539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 --2- 172.21.15.27:0/2307385362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd756c660 0x557fd79430e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 -- 172.21.15.27:0/2307385362 >> 172.21.15.27:0/2307385362 conn(0x557fd7589910 msgr2=0x557fd793d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 -- 172.21.15.27:0/2307385362 shutdown_connections 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 -- 172.21.15.27:0/2307385362 wait complete. 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 -- start start 2024-09-21T07:15:46.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.014+0000 7f7330ffc740 1 Processor -- start 2024-09-21T07:15:46.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd794c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:46.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd794c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:46.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7330ffc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd79bd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:46.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7330ffc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b1410 con 0x557fd756c660 2024-09-21T07:15:46.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7330ffc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b2880 con 0x557fd7595520 2024-09-21T07:15:46.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7330ffc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b1ae0 con 0x557fd7574840 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7330ffc740 10 client.0 _ll_register_callbacks cb 0x7f7314009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7323400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd794c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7323400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd794c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34148/0 (socket says 172.21.15.27:34148) 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7323400640 1 -- 172.21.15.27:0/3945509010 learned_addr learned my addr 172.21.15.27:0/3945509010 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7322a00640 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd794c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.015+0000 7f7323e00640 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd79bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f7323400640 1 -- 172.21.15.27:0/3945509010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 msgr2=0x557fd794c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f7323400640 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd794c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f7323400640 1 -- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 msgr2=0x557fd79bd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f7323400640 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd79bd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f7323400640 1 -- 172.21.15.27:0/3945509010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd79bd7f0 con 0x557fd756c660 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f7323400640 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd794c320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7314005bc0 tx=0x7f7314012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b0fe397f435250 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:46.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f730cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:46.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.016+0000 7f730cc00640 1 -- 172.21.15.27:0/3945509010 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314036060 con 0x557fd756c660 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731400f810 con 0x557fd756c660 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f730cc00640 1 -- 172.21.15.27:0/3945509010 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7314004440 con 0x557fd756c660 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f730cc00640 1 -- 172.21.15.27:0/3945509010 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f731403d050 con 0x557fd756c660 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f7330ffc740 10 client.16278 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557fd79c4b30 con 0x557fd756c660 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f730cc00640 1 -- 172.21.15.27:0/3945509010 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731400f810 con 0x557fd756c660 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f730cc00640 1 -- 172.21.15.27:0/3945509010 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7314003c30 con 0x557fd756c660 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f7330ffc740 10 client.16278 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.017+0000 7f7330ffc740 10 client.16278 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.018+0000 7f7330ffc740 20 client.16278 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.018+0000 7f7330ffc740 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7529250 0x557fd79c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.018+0000 7f7330ffc740 4 client.16278 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7c734edf", "release": true, "await": true}] 2024-09-21T07:15:46.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.018+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7c734edf", "release": true, "await": true}) -- 0x557fd78b0c90 con 0x557fd7529250 2024-09-21T07:15:46.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.018+0000 7f7322a00640 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7529250 0x557fd79c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:46.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.019+0000 7f7322a00640 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7529250 0x557fd79c7260 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a1b2258ce90f3d server_cookie=d5b5b703c3315edc in_seq=0 out_seq=0 2024-09-21T07:15:46.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.019+0000 7f730cc00640 10 client.16278 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f730cc00640 1 -- 172.21.15.27:0/3945509010 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557fd78b0c90 con 0x557fd7529250 2024-09-21T07:15:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f730cc00640 10 client.16278 handle_command_reply: tid=0 2024-09-21T07:15:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 client.16278 shutdown 2024-09-21T07:15:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 msgr2=0x557fd794c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd794c320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7314005bc0 tx=0x7f7314012520 comp rx=0 tx=0).stop 2024-09-21T07:15:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 shutdown_connections 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557fd7529250 0x557fd79c7260 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fd7574840 0x557fd794c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fd7595520 0x557fd79bd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 --2- 172.21.15.27:0/3945509010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fd756c660 0x557fd794c320 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 >> 172.21.15.27:0/3945509010 conn(0x557fd7589910 msgr2=0x557fd793dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f730cc00640 0 client.16278 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 shutdown_connections 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.023+0000 7f7330ffc740 1 -- 172.21.15.27:0/3945509010 wait complete. 2024-09-21T07:15:46.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7330ffc740 20 client.16278 trim_cache size 0 max 16384 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 12, 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "7c734edf": { 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 12, 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:15:46.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:46.028 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73080b6700 msgr2=0x7f73080b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73080b6700 0x7f73080b8c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f731c009fd0 tx=0x7f731c009500 comp rx=0 tx=0).stop 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 msgr2=0x7f732419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732419ca70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f731002a910 tx=0x7f7310031a20 comp rx=0 tx=0).stop 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 shutdown_connections 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f732410ff70 0x7f73241a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73080b6700 0x7f73080b8c10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73240760a0 0x7f732419ca70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.024+0000 7f7328c00640 1 --2- 172.21.15.27:0/2827090825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7324075750 0x7f732419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:46.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.025+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 >> 172.21.15.27:0/2827090825 conn(0x7f73240ff400 msgr2=0x7f7324101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:46.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.025+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 shutdown_connections 2024-09-21T07:15:46.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:46.025+0000 7f7328c00640 1 -- 172.21.15.27:0/2827090825 wait complete. 2024-09-21T07:15:46.907 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7c734edf', total seconds elapsed: 31.9 2024-09-21T07:15:46.907 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T07:15:46.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:15:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:46 smithi027 ceph-mon[30779]: pgmap v460: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-21T07:15:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:46 smithi153 ceph-mon[34747]: pgmap v460: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-21T07:15:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:46 smithi119 ceph-mon[34519]: pgmap v460: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-21T07:15:47.295 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:47.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 -- 172.21.15.27:0/3114782698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa98109e00 msgr2=0x7faa98110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:47.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 --2- 172.21.15.27:0/3114782698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa98109e00 0x7faa98110630 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faa90009ca0 tx=0x7faa9002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:15:47.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 -- 172.21.15.27:0/3114782698 shutdown_connections 2024-09-21T07:15:47.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 --2- 172.21.15.27:0/3114782698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa981094b0 0x7faa981098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:47.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 --2- 172.21.15.27:0/3114782698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa98109e00 0x7faa98110630 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:47.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 --2- 172.21.15.27:0/3114782698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa98103b70 0x7faa98103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:47.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 -- 172.21.15.27:0/3114782698 >> 172.21.15.27:0/3114782698 conn(0x7faa980ff5e0 msgr2=0x7faa98101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:47.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 -- 172.21.15.27:0/3114782698 shutdown_connections 2024-09-21T07:15:47.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.918+0000 7faa9f200640 1 -- 172.21.15.27:0/3114782698 wait complete. 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.919+0000 7faa9f200640 1 Processor -- start 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.919+0000 7faa9f200640 1 -- start start 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa9f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa98103b70 0x7faa9819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa9f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa981094b0 0x7faa9819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa9f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa98109e00 0x7faa9819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa9f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faa98113ac0 con 0x7faa981094b0 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa9f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa98113940 con 0x7faa98103b70 2024-09-21T07:15:47.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa9f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa98113c40 con 0x7faa98109e00 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa981094b0 0x7faa9819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa981094b0 0x7faa9819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34158/0 (socket says 172.21.15.27:34158) 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa97e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa98103b70 0x7faa9819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa97400640 1 -- 172.21.15.27:0/48406454 learned_addr learned my addr 172.21.15.27:0/48406454 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.920+0000 7faa9d200640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa98109e00 0x7faa9819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa97e00640 1 -- 172.21.15.27:0/48406454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa98109e00 msgr2=0x7faa9819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa97e00640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa98109e00 0x7faa9819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa97e00640 1 -- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa981094b0 msgr2=0x7faa9819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa97e00640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa981094b0 0x7faa9819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa97e00640 1 -- 172.21.15.27:0/48406454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981a03c0 con 0x7faa98103b70 2024-09-21T07:15:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa97400640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa981094b0 0x7faa9819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa97e00640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa98103b70 0x7faa9819a450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa8800e9f0 tx=0x7faa8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa94c00640 1 -- 172.21.15.27:0/48406454 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8800ccc0 con 0x7faa98103b70 2024-09-21T07:15:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa94c00640 1 -- 172.21.15.27:0/48406454 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa880108e0 con 0x7faa98103b70 2024-09-21T07:15:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.921+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa981a06b0 con 0x7faa98103b70 2024-09-21T07:15:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.922+0000 7faa94c00640 1 -- 172.21.15.27:0/48406454 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88018880 con 0x7faa98103b70 2024-09-21T07:15:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.922+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981a6e10 con 0x7faa98103b70 2024-09-21T07:15:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.923+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa98103f80 con 0x7faa98103b70 2024-09-21T07:15:47.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.925+0000 7faa94c00640 1 -- 172.21.15.27:0/48406454 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7faa88010430 con 0x7faa98103b70 2024-09-21T07:15:47.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.927+0000 7faa94c00640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faa7c0b6680 0x7faa7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.927+0000 7faa94c00640 1 -- 172.21.15.27:0/48406454 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7faa880e7fb0 con 0x7faa98103b70 2024-09-21T07:15:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.927+0000 7faa97400640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faa7c0b6680 0x7faa7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:47.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.928+0000 7faa97400640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faa7c0b6680 0x7faa7c0b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faa98102ce0 tx=0x7faa8400b440 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:47.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:47.928+0000 7faa94c00640 1 -- 172.21.15.27:0/48406454 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa880b2530 con 0x7faa98103b70 2024-09-21T07:15:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.166+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa9802e640 con 0x7faa98103b70 2024-09-21T07:15:48.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.167+0000 7faa94c00640 1 -- 172.21.15.27:0/48406454 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7faa880b7470 con 0x7faa98103b70 2024-09-21T07:15:48.170 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:15:48.170 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:15:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:15:48.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faa7c0b6680 msgr2=0x7faa7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:48.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faa7c0b6680 0x7faa7c0b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faa98102ce0 tx=0x7faa8400b440 comp rx=0 tx=0).stop 2024-09-21T07:15:48.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa98103b70 msgr2=0x7faa9819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:48.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa98103b70 0x7faa9819a450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa8800e9f0 tx=0x7faa8800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:15:48.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 shutdown_connections 2024-09-21T07:15:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa98109e00 0x7faa9819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faa7c0b6680 0x7faa7c0b8b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa981094b0 0x7faa9819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 --2- 172.21.15.27:0/48406454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa98103b70 0x7faa9819a450 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 >> 172.21.15.27:0/48406454 conn(0x7faa980ff5e0 msgr2=0x7faa98106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.170+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 shutdown_connections 2024-09-21T07:15:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:48.171+0000 7faa9f200640 1 -- 172.21.15.27:0/48406454 wait complete. 2024-09-21T07:15:49.034 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T07:15:49.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T07:15:49.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:48 smithi027 ceph-mon[30779]: pgmap v461: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T07:15:49.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:48 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/48406454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:49.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:48 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:15:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:48 smithi153 ceph-mon[34747]: pgmap v461: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T07:15:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:48 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/48406454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:48 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:15:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:48 smithi119 ceph-mon[34519]: pgmap v461: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T07:15:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:48 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/48406454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:48 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:15:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.091+0000 7f5ea8c00640 1 -- 172.21.15.27:0/2788837128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 msgr2=0x7f5ea40757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.091+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/2788837128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 0x7f5ea40757b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009900 tx=0x7f5e9402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.091+0000 7f5ea8c00640 1 -- 172.21.15.27:0/2788837128 shutdown_connections 2024-09-21T07:15:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.091+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/2788837128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4114290 0x7f5ea4116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.091+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/2788837128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea4075dd0 0x7f5ea40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.091+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/2788837128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 0x7f5ea40757b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.091+0000 7f5ea8c00640 1 -- 172.21.15.27:0/2788837128 >> 172.21.15.27:0/2788837128 conn(0x7f5ea40ff430 msgr2=0x7f5ea4101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.092+0000 7f5ea8c00640 1 -- 172.21.15.27:0/2788837128 shutdown_connections 2024-09-21T07:15:50.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.092+0000 7f5ea8c00640 1 -- 172.21.15.27:0/2788837128 wait complete. 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 Processor -- start 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 -- start start 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4075dd0 0x7f5ea41a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 0x7f5ea41a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea4114290 0x7f5ea41a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea4119ed0 con 0x7f5ea4075dd0 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea4119d50 con 0x7f5ea4114290 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.093+0000 7f5ea8c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea411a050 con 0x7f5ea4077380 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4075dd0 0x7f5ea41a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 0x7f5ea41a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4075dd0 0x7f5ea41a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34668/0 (socket says 172.21.15.27:34668) 2024-09-21T07:15:50.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3400640 1 -- 172.21.15.27:0/3451153428 learned_addr learned my addr 172.21.15.27:0/3451153428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea4114290 0x7f5ea41a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3e00640 1 -- 172.21.15.27:0/3451153428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 msgr2=0x7f5ea41a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 0x7f5ea41a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3e00640 1 -- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4075dd0 msgr2=0x7f5ea41a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4075dd0 0x7f5ea41a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3e00640 1 -- 172.21.15.27:0/3451153428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea41a6810 con 0x7f5ea4114290 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.094+0000 7f5ea3400640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4075dd0 0x7f5ea41a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.095+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea4114290 0x7f5ea41a60e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009fd0 tx=0x7f5e9c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.095+0000 7f5e9be00640 1 -- 172.21.15.27:0/3451153428 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00ce20 con 0x7f5ea4114290 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.095+0000 7f5e9be00640 1 -- 172.21.15.27:0/3451153428 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9c005020 con 0x7f5ea4114290 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.095+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea41a6b00 con 0x7f5ea4114290 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.095+0000 7f5e9be00640 1 -- 172.21.15.27:0/3451153428 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c010690 con 0x7f5ea4114290 2024-09-21T07:15:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.095+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea41ad260 con 0x7f5ea4114290 2024-09-21T07:15:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.098+0000 7f5eb0e24740 1 Processor -- start 2024-09-21T07:15:50.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.099+0000 7f5e9be00640 1 -- 172.21.15.27:0/3451153428 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5e9c0054d0 con 0x7f5ea4114290 2024-09-21T07:15:50.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.099+0000 7f5eb0e24740 1 -- start start 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b563b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b563ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b563bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e1b55b0ca0 con 0x55e1b526a980 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1b55a00d0 con 0x55e1b5293840 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1b55a3ee0 con 0x55e1b5272b60 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b563b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b563b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33288/0 (socket says 172.21.15.27:33288) 2024-09-21T07:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3249623664 learned_addr learned my addr 172.21.15.27:0/3249623664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea3400640 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b563ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b563bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3249623664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 msgr2=0x55e1b563bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b563bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3249623664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 msgr2=0x55e1b563ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b563ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.100+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3249623664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b5552400 con 0x55e1b5293840 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.101+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b563b510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5e90013560 tx=0x7f5e90005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3d02ce4e12d7c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.101+0000 7f5e8fe00640 1 -- 172.21.15.27:0/3249623664 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90004050 con 0x55e1b5293840 2024-09-21T07:15:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.101+0000 7f5e8fe00640 1 -- 172.21.15.27:0/3249623664 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e900041f0 con 0x55e1b5293840 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.101+0000 7f5e8fe00640 1 -- 172.21.15.27:0/3249623664 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000f760 con 0x55e1b5293840 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.101+0000 7f5e9be00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e880b6670 0x7f5e880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5e9be00640 1 -- 172.21.15.27:0/3451153428 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5e9c0e8150 con 0x7f5ea4114290 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5ea3400640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e880b6670 0x7f5e880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3249623664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 msgr2=0x55e1b563b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b563b510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5e90013560 tx=0x7f5e90005850 comp rx=0 tx=0).stop 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3249623664 shutdown_connections 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b563bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b563ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3249623664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b563b510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3249623664 >> 172.21.15.27:0/3249623664 conn(0x55e1b5287990 msgr2=0x55e1b5639400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3249623664 shutdown_connections 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.102+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3249623664 wait complete. 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.103+0000 7f5ea3400640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e880b6670 0x7f5e880b8b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5e94031440 tx=0x7f5e940058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.103+0000 7f5eb0e24740 1 -- start start 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.103+0000 7f5eb0e24740 1 Processor -- start 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5646ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b5647620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e1b54017a0 con 0x55e1b526a980 2024-09-21T07:15:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1b5545a10 con 0x55e1b5293840 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1b5649cf0 con 0x55e1b5272b60 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5eb0e24740 10 client.0 _ll_register_callbacks cb 0x55e1b5555be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5646ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b5647620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5646ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34692/0 (socket says 172.21.15.27:34692) 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.104+0000 7f5ea3400640 1 -- 172.21.15.27:0/821430615 learned_addr learned my addr 172.21.15.27:0/821430615 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5ea3e00640 1 -- 172.21.15.27:0/821430615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 msgr2=0x55e1b56470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5ea3e00640 1 -- 172.21.15.27:0/821430615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 msgr2=0x55e1b5646ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5646ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5ea3e00640 1 -- 172.21.15.27:0/821430615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b530db90 con 0x55e1b5293840 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b5647620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0e54f0 tx=0x7f5e9c0ad420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ef045888534fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5e8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5e8cc00640 1 -- 172.21.15.27:0/821430615 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0b26d0 con 0x55e1b5293840 2024-09-21T07:15:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5e8cc00640 1 -- 172.21.15.27:0/821430615 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5e9c0b21d0 con 0x55e1b5293840 2024-09-21T07:15:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.105+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e900041f0 con 0x55e1b5293840 2024-09-21T07:15:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5e8cc00640 1 -- 172.21.15.27:0/821430615 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e9c0b7610 con 0x55e1b5293840 2024-09-21T07:15:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 10 client.26330 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1b56559a0 con 0x55e1b5293840 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5e8cc00640 1 -- 172.21.15.27:0/821430615 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e900041f0 con 0x55e1b5293840 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5e8cc00640 1 -- 172.21.15.27:0/821430615 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5e9c0a9050 con 0x55e1b5293840 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 10 client.26330 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 10 client.26330 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 20 client.26330 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b52521c0 0x55e1b5658e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 4 client.26330 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.106+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1b55b2bf0 con 0x55e1b52521c0 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.107+0000 7f5ea3400640 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b52521c0 0x55e1b5658e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.107+0000 7f5ea3400640 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b52521c0 0x55e1b5658e20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7c16a8a7786b44 server_cookie=16ec264a77db8a33 in_seq=0 out_seq=0 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.107+0000 7f5e8cc00640 10 client.26330 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.108+0000 7f5e8cc00640 1 -- 172.21.15.27:0/821430615 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e1b55b2bf0 con 0x55e1b52521c0 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.108+0000 7f5e8cc00640 10 client.26330 handle_command_reply: tid=0 2024-09-21T07:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.108+0000 7f5eb0e24740 1 client.26330 shutdown 2024-09-21T07:15:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.108+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 msgr2=0x55e1b5647620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.108+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b5647620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0e54f0 tx=0x7f5e9c0ad420 comp rx=0 tx=0).stop 2024-09-21T07:15:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 shutdown_connections 2024-09-21T07:15:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b52521c0 0x55e1b5658e20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5293840 0x55e1b5647620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/821430615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5646ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5e8cc00640 0 client.26330 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 >> 172.21.15.27:0/821430615 conn(0x55e1b5287990 msgr2=0x55e1b563e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 shutdown_connections 2024-09-21T07:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 1 -- 172.21.15.27:0/821430615 wait complete. 2024-09-21T07:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.109+0000 7f5eb0e24740 20 client.26330 trim_cache size 0 max 16384 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 Processor -- start 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 -- start start 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5287990 0x55e1b563cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5643d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea41b9890 con 0x55e1b526a980 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea41b7160 con 0x55e1b5287990 2024-09-21T07:15:50.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.127+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea41b4fb0 con 0x55e1b5272b60 2024-09-21T07:15:50.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5643d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5643d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34700/0 (socket says 172.21.15.27:34700) 2024-09-21T07:15:50.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3618660506 learned_addr learned my addr 172.21.15.27:0/3618660506 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea3400640 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5287990 0x55e1b563cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3618660506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 msgr2=0x55e1b56442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3618660506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5287990 msgr2=0x55e1b563cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5287990 0x55e1b563cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.128+0000 7f5ea2a00640 1 -- 172.21.15.27:0/3618660506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b563d800 con 0x55e1b526a980 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.129+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5643d90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e90013530 tx=0x7f5e90003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cb94930de0858d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.129+0000 7f5e8f400640 1 -- 172.21.15.27:0/3618660506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e900097f0 con 0x55e1b526a980 2024-09-21T07:15:50.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.129+0000 7f5e8f400640 1 -- 172.21.15.27:0/3618660506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9002f070 con 0x55e1b526a980 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.129+0000 7f5e8f400640 1 -- 172.21.15.27:0/3618660506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000f500 con 0x55e1b526a980 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3618660506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 msgr2=0x55e1b5643d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5643d90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e90013530 tx=0x7f5e90003e00 comp rx=0 tx=0).stop 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3618660506 shutdown_connections 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b56442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b526a980 0x55e1b5643d90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/3618660506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b5287990 0x55e1b563cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3618660506 >> 172.21.15.27:0/3618660506 conn(0x55e1b5634570 msgr2=0x55e1b56379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3618660506 shutdown_connections 2024-09-21T07:15:50.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 -- 172.21.15.27:0/3618660506 wait complete. 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.130+0000 7f5eb0e24740 1 -- start start 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 1 Processor -- start 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b526a980 0x55e1b564b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b564be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b5287990 0x55e1b56bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea41b5620 con 0x55e1b5287990 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea41b6a90 con 0x55e1b526a980 2024-09-21T07:15:50.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea41b5cf0 con 0x55e1b5272b60 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.131+0000 7f5eb0e24740 10 client.0 _ll_register_callbacks cb 0x7f5e9c005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b526a980 0x55e1b564b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b526a980 0x55e1b564b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33318/0 (socket says 172.21.15.27:33318) 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b564be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 -- 172.21.15.27:0/2660546333 learned_addr learned my addr 172.21.15.27:0/2660546333 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3e00640 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b5287990 0x55e1b56bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 -- 172.21.15.27:0/2660546333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 msgr2=0x55e1b564be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b564be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 -- 172.21.15.27:0/2660546333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b5287990 msgr2=0x55e1b56bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b5287990 0x55e1b56bc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 -- 172.21.15.27:0/2660546333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b56bce70 con 0x55e1b526a980 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5ea3400640 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b526a980 0x55e1b564b820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5e94005940 tx=0x7f5e9403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac3f98a655a5b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.132+0000 7f5e8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5e8cc00640 1 -- 172.21.15.27:0/2660546333 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94043070 con 0x55e1b526a980 2024-09-21T07:15:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e9002f070 con 0x55e1b526a980 2024-09-21T07:15:50.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5e8cc00640 1 -- 172.21.15.27:0/2660546333 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5e94004970 con 0x55e1b526a980 2024-09-21T07:15:50.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5e8cc00640 1 -- 172.21.15.27:0/2660546333 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e94063050 con 0x55e1b526a980 2024-09-21T07:15:50.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5eb0e24740 10 client.26336 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:50.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1b56c4190 con 0x55e1b526a980 2024-09-21T07:15:50.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5e8cc00640 1 -- 172.21.15.27:0/2660546333 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9002f070 con 0x55e1b526a980 2024-09-21T07:15:50.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.133+0000 7f5e8cc00640 1 -- 172.21.15.27:0/2660546333 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5e9403d450 con 0x55e1b526a980 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5eb0e24740 10 client.26336 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5eb0e24740 10 client.26336 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5eb0e24740 20 client.26336 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b5293840 0x55e1b56c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5eb0e24740 4 client.26336 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x55e1b55ae6a0 con 0x55e1b5293840 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b5293840 0x55e1b56c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5ea2a00640 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b5293840 0x55e1b56c69a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2838f803c7622811 server_cookie=28d5dadd3cd10811 in_seq=0 out_seq=0 2024-09-21T07:15:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.134+0000 7f5e8cc00640 10 client.26336 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5e8cc00640 1 -- 172.21.15.27:0/2660546333 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e1b55ae6a0 con 0x55e1b5293840 2024-09-21T07:15:50.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5e8cc00640 10 client.26336 handle_command_reply: tid=0 2024-09-21T07:15:50.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 client.26336 shutdown 2024-09-21T07:15:50.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b526a980 msgr2=0x55e1b564b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b526a980 0x55e1b564b820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5e94005940 tx=0x7f5e9403f000 comp rx=0 tx=0).stop 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 shutdown_connections 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1b5293840 0x55e1b56c69a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1b5272b60 0x55e1b564be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1b5287990 0x55e1b56bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 --2- 172.21.15.27:0/2660546333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1b526a980 0x55e1b564b820 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5e8cc00640 0 client.26336 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 >> 172.21.15.27:0/2660546333 conn(0x55e1b5634570 msgr2=0x55e1b56389f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 shutdown_connections 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 1 -- 172.21.15.27:0/2660546333 wait complete. 2024-09-21T07:15:50.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.147+0000 7f5eb0e24740 20 client.26336 trim_cache size 0 max 16384 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 14, 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "526297b8": { 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 14, 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:50.152 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:15:50.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.148+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e880b6670 msgr2=0x7f5e880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.148+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e880b6670 0x7f5e880b8b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5e94031440 tx=0x7f5e940058d0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.148+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea4114290 msgr2=0x7f5ea41a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.148+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea4114290 0x7f5ea41a60e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009fd0 tx=0x7f5e9c00ef10 comp rx=0 tx=0).stop 2024-09-21T07:15:50.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 shutdown_connections 2024-09-21T07:15:50.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea4077380 0x7f5ea41a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5e880b6670 0x7f5e880b8b80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea4114290 0x7f5ea41a60e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 --2- 172.21.15.27:0/3451153428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea4075dd0 0x7f5ea41a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 >> 172.21.15.27:0/3451153428 conn(0x7f5ea40ff430 msgr2=0x7f5ea4110190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 shutdown_connections 2024-09-21T07:15:50.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:50.149+0000 7f5ea8c00640 1 -- 172.21.15.27:0/3451153428 wait complete. 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 -- 172.21.15.27:0/837672990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 msgr2=0x7fab1010f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 --2- 172.21.15.27:0/837672990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 0x7fab1010f2b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab08009be0 tx=0x7fab0802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 -- 172.21.15.27:0/837672990 shutdown_connections 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 --2- 172.21.15.27:0/837672990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 0x7fab1010f2b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 --2- 172.21.15.27:0/837672990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab10108130 0x7fab10108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 --2- 172.21.15.27:0/837672990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10102800 0x7fab10102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 -- 172.21.15.27:0/837672990 >> 172.21.15.27:0/837672990 conn(0x7fab100fe220 msgr2=0x7fab10100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.732+0000 7fab0fe00640 1 -- 172.21.15.27:0/837672990 shutdown_connections 2024-09-21T07:15:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.733+0000 7fab0fe00640 1 -- 172.21.15.27:0/837672990 wait complete. 2024-09-21T07:15:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 Processor -- start 2024-09-21T07:15:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 -- start start 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab10102800 0x7fab1019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10108130 0x7fab1019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 0x7fab101a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab10112750 con 0x7fab10108130 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab101125d0 con 0x7fab10102800 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab101128d0 con 0x7fab10108a80 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10108130 0x7fab1019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10108130 0x7fab1019c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34730/0 (socket says 172.21.15.27:34730) 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0e000640 1 -- 172.21.15.27:0/3026704868 learned_addr learned my addr 172.21.15.27:0/3026704868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0f400640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 0x7fab101a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.734+0000 7fab0ea00640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab10102800 0x7fab1019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab0e000640 1 -- 172.21.15.27:0/3026704868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 msgr2=0x7fab101a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab0e000640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 0x7fab101a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab0e000640 1 -- 172.21.15.27:0/3026704868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab10102800 msgr2=0x7fab1019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab0e000640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab10102800 0x7fab1019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab0e000640 1 -- 172.21.15.27:0/3026704868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab101a2370 con 0x7fab10108130 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab0f400640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 0x7fab101a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab0e000640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10108130 0x7fab1019c9e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faafc00e9c0 tx=0x7faafc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.735+0000 7fab07400640 1 -- 172.21.15.27:0/3026704868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc00ccc0 con 0x7fab10108130 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.736+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab101a2660 con 0x7fab10108130 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.736+0000 7fab07400640 1 -- 172.21.15.27:0/3026704868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faafc0054d0 con 0x7fab10108130 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.736+0000 7fab07400640 1 -- 172.21.15.27:0/3026704868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc010430 con 0x7fab10108130 2024-09-21T07:15:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.736+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab101a8df0 con 0x7fab10108130 2024-09-21T07:15:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 Processor -- start 2024-09-21T07:15:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 -- start start 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b54869e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b5449c2f0 0x560b5486a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5486a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560b547df6b0 con 0x560b544a44d0 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560b547ceae0 con 0x560b544c51b0 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab1c522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560b547d28f0 con 0x560b5449c2f0 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.739+0000 7fab07400640 1 -- 172.21.15.27:0/3026704868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7faafc005020 con 0x7fab10108130 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b5449c2f0 0x560b5486a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b5449c2f0 0x560b5486a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52678/0 (socket says 172.21.15.27:52678) 2024-09-21T07:15:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b54869e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 -- 172.21.15.27:0/1940615637 learned_addr learned my addr 172.21.15.27:0/1940615637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0e000640 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5486a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 -- 172.21.15.27:0/1940615637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 msgr2=0x560b54869e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b54869e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 -- 172.21.15.27:0/1940615637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 msgr2=0x560b5486a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5486a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 -- 172.21.15.27:0/1940615637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5479b780 con 0x560b5449c2f0 2024-09-21T07:15:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.740+0000 7fab0ea00640 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b5449c2f0 0x560b5486a350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab00009cf0 tx=0x7fab00011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bba4922174792a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7faafbe00640 1 -- 172.21.15.27:0/1940615637 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00005220 con 0x560b5449c2f0 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7faafbe00640 1 -- 172.21.15.27:0/1940615637 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab00004c50 con 0x560b5449c2f0 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7faafbe00640 1 -- 172.21.15.27:0/1940615637 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00019830 con 0x560b5449c2f0 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7fab07400640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faaf40b6680 0x7faaf40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7fab07400640 1 -- 172.21.15.27:0/3026704868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7faafc0e7df0 con 0x7fab10108130 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7fab1c522740 1 -- 172.21.15.27:0/1940615637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b5449c2f0 msgr2=0x560b5486a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7fab1c522740 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b5449c2f0 0x560b5486a350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab00009cf0 tx=0x7fab00011810 comp rx=0 tx=0).stop 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.741+0000 7fab0f400640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faaf40b6680 0x7faaf40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 -- 172.21.15.27:0/1940615637 shutdown_connections 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b5449c2f0 0x560b5486a350 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5486a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 --2- 172.21.15.27:0/1940615637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b54869e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 -- 172.21.15.27:0/1940615637 >> 172.21.15.27:0/1940615637 conn(0x560b544b9300 msgr2=0x560b54867d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 -- 172.21.15.27:0/1940615637 shutdown_connections 2024-09-21T07:15:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 -- 172.21.15.27:0/1940615637 wait complete. 2024-09-21T07:15:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab0f400640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faaf40b6680 0x7faaf40b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab08009370 tx=0x7fab08004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.742+0000 7fab1c522740 1 -- start start 2024-09-21T07:15:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 1 Processor -- start 2024-09-21T07:15:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b548753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b54875910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544c51b0 0x560b54875e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560b54706ab0 con 0x560b544c51b0 2024-09-21T07:15:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560b54775570 con 0x560b5449c2f0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560b54878590 con 0x560b544a44d0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab1c522740 10 client.0 _ll_register_callbacks cb 0x560b54784100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab0f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544c51b0 0x560b54875e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b548753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab0f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544c51b0 0x560b54875e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34746/0 (socket says 172.21.15.27:34746) 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.743+0000 7fab0f400640 1 -- 172.21.15.27:0/731102447 learned_addr learned my addr 172.21.15.27:0/731102447 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7fab0e000640 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b54875910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7fab0ea00640 1 -- 172.21.15.27:0/731102447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 msgr2=0x560b54875910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7fab0ea00640 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b54875910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7fab0ea00640 1 -- 172.21.15.27:0/731102447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544c51b0 msgr2=0x560b54875e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7fab0ea00640 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544c51b0 0x560b54875e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7fab0ea00640 1 -- 172.21.15.27:0/731102447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b54530790 con 0x560b5449c2f0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7fab0ea00640 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b548753d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fab0001b040 tx=0x7fab00005470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7e6dc90cba9d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.744+0000 7faaf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.745+0000 7faaf8c00640 1 -- 172.21.15.27:0/731102447 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00029020 con 0x560b5449c2f0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.745+0000 7faaf8c00640 1 -- 172.21.15.27:0/731102447 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab00019730 con 0x560b5449c2f0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.745+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab00004c50 con 0x560b5449c2f0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.745+0000 7faaf8c00640 1 -- 172.21.15.27:0/731102447 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab00019970 con 0x560b5449c2f0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.745+0000 7fab1c522740 10 client.26348 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.745+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b545a9620 con 0x560b5449c2f0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.745+0000 7faaf8c00640 1 -- 172.21.15.27:0/731102447 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab00004c50 con 0x560b5449c2f0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7faaf8c00640 1 -- 172.21.15.27:0/731102447 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab000255a0 con 0x560b5449c2f0 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7fab1c522740 10 client.26348 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7fab1c522740 10 client.26348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7fab1c522740 20 client.26348 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7fab1c522740 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b54483d80 0x560b54887490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7fab1c522740 4 client.26348 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b547e1600 con 0x560b54483d80 2024-09-21T07:15:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.746+0000 7fab0e000640 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b54483d80 0x560b54887490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.747+0000 7fab0e000640 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b54483d80 0x560b54887490 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d266e2a977bda9ce server_cookie=4e4d4f68b1fcc49f in_seq=0 out_seq=0 2024-09-21T07:15:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.747+0000 7faaf8c00640 10 client.26348 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7faaf8c00640 1 -- 172.21.15.27:0/731102447 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b547e1600 con 0x560b54483d80 2024-09-21T07:15:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7faaf8c00640 10 client.26348 handle_command_reply: tid=0 2024-09-21T07:15:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 client.26348 shutdown 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 msgr2=0x560b548753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b548753d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fab0001b040 tx=0x7fab00005470 comp rx=0 tx=0).stop 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 shutdown_connections 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b54483d80 0x560b54887490 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b54875910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544c51b0 0x560b54875e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 --2- 172.21.15.27:0/731102447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b548753d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7faaf8c00640 0 client.26348 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 >> 172.21.15.27:0/731102447 conn(0x560b544b9300 msgr2=0x560b5486ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 shutdown_connections 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 1 -- 172.21.15.27:0/731102447 wait complete. 2024-09-21T07:15:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.748+0000 7fab1c522740 20 client.26348 trim_cache size 0 max 16384 2024-09-21T07:15:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.763+0000 7fab1c522740 1 Processor -- start 2024-09-21T07:15:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab1c522740 1 -- start start 2024-09-21T07:15:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab1c522740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b5449c2f0 0x560b54866d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab1c522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b5486b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab1c522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b548725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab1c522740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b56d0 con 0x560b5449c2f0 2024-09-21T07:15:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab1c522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b2fa0 con 0x560b544c51b0 2024-09-21T07:15:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab1c522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b0df0 con 0x560b544a44d0 2024-09-21T07:15:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b5449c2f0 0x560b54866d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b5449c2f0 0x560b54866d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34754/0 (socket says 172.21.15.27:34754) 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab0ea00640 1 -- 172.21.15.27:0/1062415016 learned_addr learned my addr 172.21.15.27:0/1062415016 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab0f400640 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b548725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.764+0000 7fab0e000640 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b5486b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7fab0ea00640 1 -- 172.21.15.27:0/1062415016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 msgr2=0x560b5486b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7fab0ea00640 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b5486b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7fab0ea00640 1 -- 172.21.15.27:0/1062415016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 msgr2=0x560b548725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7fab0ea00640 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b548725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7fab0ea00640 1 -- 172.21.15.27:0/1062415016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b54868080 con 0x560b5449c2f0 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7fab0f400640 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b548725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7fab0ea00640 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b5449c2f0 0x560b54866d20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fab000259c0 tx=0x7fab00011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aea45a9895f32f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7faafb400640 1 -- 172.21.15.27:0/1062415016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00029040 con 0x560b5449c2f0 2024-09-21T07:15:50.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7faafb400640 1 -- 172.21.15.27:0/1062415016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab00025c30 con 0x560b5449c2f0 2024-09-21T07:15:50.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.765+0000 7faafb400640 1 -- 172.21.15.27:0/1062415016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab000256b0 con 0x560b5449c2f0 2024-09-21T07:15:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 -- 172.21.15.27:0/1062415016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b5449c2f0 msgr2=0x560b54866d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b5449c2f0 0x560b54866d20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fab000259c0 tx=0x7fab00011d10 comp rx=0 tx=0).stop 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 -- 172.21.15.27:0/1062415016 shutdown_connections 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544a44d0 0x560b5486b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b544c51b0 0x560b548725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 --2- 172.21.15.27:0/1062415016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b5449c2f0 0x560b54866d20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 -- 172.21.15.27:0/1062415016 >> 172.21.15.27:0/1062415016 conn(0x560b544b9300 msgr2=0x560b5486ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 -- 172.21.15.27:0/1062415016 shutdown_connections 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.766+0000 7fab1c522740 1 -- 172.21.15.27:0/1062415016 wait complete. 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 -- start start 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 Processor -- start 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b5487ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5487b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544c51b0 0x560b5487b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b1460 con 0x560b544a44d0 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b28d0 con 0x560b5449c2f0 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b1b30 con 0x560b544c51b0 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.767+0000 7fab1c522740 10 client.0 _ll_register_callbacks cb 0x7fab00005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5487b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5487b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34766/0 (socket says 172.21.15.27:34766) 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 -- 172.21.15.27:0/2534640866 learned_addr learned my addr 172.21.15.27:0/2534640866 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0f400640 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544c51b0 0x560b5487b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0ea00640 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b5487ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 -- 172.21.15.27:0/2534640866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544c51b0 msgr2=0x560b5487b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544c51b0 0x560b5487b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 -- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 msgr2=0x560b5487ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b5487ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 -- 172.21.15.27:0/2534640866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5487bd40 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7fab0e000640 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5487b290 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faafc009770 tx=0x7faafc0ad300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1bba79c4716a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7faaf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.768+0000 7faaf8c00640 1 -- 172.21.15.27:0/2534640866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc0a9060 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7faaf8c00640 1 -- 172.21.15.27:0/2534640866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7faafc0a9b60 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab00025c30 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7faaf8c00640 1 -- 172.21.15.27:0/2534640866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faafc005310 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 10 client.16320 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b548f1a90 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7faaf8c00640 1 -- 172.21.15.27:0/2534640866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab00025c30 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7faaf8c00640 1 -- 172.21.15.27:0/2534640866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faafc0a9240 con 0x560b544a44d0 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 10 client.16320 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 20 client.16320 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b544591c0 0x560b548f4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 4 client.16320 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:15:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.769+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b547dd0b0 con 0x560b544591c0 2024-09-21T07:15:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.770+0000 7fab0ea00640 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b544591c0 0x560b548f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.770+0000 7fab0ea00640 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b544591c0 0x560b548f4250 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac6bb3d2f445e76 server_cookie=412fb718c27dc00b in_seq=0 out_seq=0 2024-09-21T07:15:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.770+0000 7faaf8c00640 10 client.16320 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7faaf8c00640 1 -- 172.21.15.27:0/2534640866 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b547dd0b0 con 0x560b544591c0 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7faaf8c00640 10 client.16320 handle_command_reply: tid=0 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 client.16320 shutdown 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 msgr2=0x560b5487b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5487b290 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faafc009770 tx=0x7faafc0ad300 comp rx=0 tx=0).stop 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 shutdown_connections 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560b544591c0 0x560b548f4250 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b544c51b0 0x560b5487b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560b544a44d0 0x560b5487b290 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 --2- 172.21.15.27:0/2534640866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b5449c2f0 0x560b5487ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7faaf8c00640 0 client.16320 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 >> 172.21.15.27:0/2534640866 conn(0x560b544b9300 msgr2=0x560b54866af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 shutdown_connections 2024-09-21T07:15:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.781+0000 7fab1c522740 1 -- 172.21.15.27:0/2534640866 wait complete. 2024-09-21T07:15:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab1c522740 20 client.16320 trim_cache size 0 max 16384 2024-09-21T07:15:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:15:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:15:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:15:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faaf40b6680 msgr2=0x7faaf40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faaf40b6680 0x7faaf40b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab08009370 tx=0x7fab08004a40 comp rx=0 tx=0).stop 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10108130 msgr2=0x7fab1019c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10108130 0x7fab1019c9e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faafc00e9c0 tx=0x7faafc00ee80 comp rx=0 tx=0).stop 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 shutdown_connections 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab10108a80 0x7fab101a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faaf40b6680 0x7faaf40b8b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab10108130 0x7fab1019c9e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 --2- 172.21.15.27:0/3026704868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab10102800 0x7fab1019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 >> 172.21.15.27:0/3026704868 conn(0x7fab100fe220 msgr2=0x7fab101076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.782+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 shutdown_connections 2024-09-21T07:15:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:15:50.783+0000 7fab0fe00640 1 -- 172.21.15.27:0/3026704868 wait complete. 2024-09-21T07:15:50.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:50 smithi027 ceph-mon[30779]: pgmap v462: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-21T07:15:51.000 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '526297b8', quiesce took 4.1 seconds. Will release after: 23.700000000000003 2024-09-21T07:15:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:50 smithi153 ceph-mon[34747]: pgmap v462: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-21T07:15:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:50 smithi119 ceph-mon[34519]: pgmap v462: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-21T07:15:52.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:15:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:52 smithi027 ceph-mon[30779]: pgmap v463: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-21T07:15:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:52 smithi153 ceph-mon[34747]: pgmap v463: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-21T07:15:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:52 smithi119 ceph-mon[34519]: pgmap v463: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-21T07:15:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.090+0000 7fa774000640 1 -- 172.21.15.27:0/3058537020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 msgr2=0x7fa76c10fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.090+0000 7fa774000640 1 --2- 172.21.15.27:0/3058537020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c10fb80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa758009900 tx=0x7fa75802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.090+0000 7fa774000640 1 -- 172.21.15.27:0/3058537020 shutdown_connections 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.090+0000 7fa774000640 1 --2- 172.21.15.27:0/3058537020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa76c1100c0 0x7fa76c112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.090+0000 7fa774000640 1 --2- 172.21.15.27:0/3058537020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c10fb80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.090+0000 7fa774000640 1 --2- 172.21.15.27:0/3058537020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa76c0757b0 0x7fa76c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.090+0000 7fa774000640 1 -- 172.21.15.27:0/3058537020 >> 172.21.15.27:0/3058537020 conn(0x7fa76c07b850 msgr2=0x7fa76c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.091+0000 7fa774000640 1 -- 172.21.15.27:0/3058537020 shutdown_connections 2024-09-21T07:15:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.091+0000 7fa774000640 1 -- 172.21.15.27:0/3058537020 wait complete. 2024-09-21T07:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 Processor -- start 2024-09-21T07:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 -- start start 2024-09-21T07:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa76c0757b0 0x7fa76c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa76c1100c0 0x7fa76c1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c115cc0 con 0x7fa76c1100c0 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c115b40 con 0x7fa76c076100 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.092+0000 7fa774000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c115e40 con 0x7fa76c0757b0 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33386/0 (socket says 172.21.15.27:33386) 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 -- 172.21.15.27:0/4147491316 learned_addr learned my addr 172.21.15.27:0/4147491316 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa772000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa76c1100c0 0x7fa76c1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa771600640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa76c0757b0 0x7fa76c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 -- 172.21.15.27:0/4147491316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa76c0757b0 msgr2=0x7fa76c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa76c0757b0 0x7fa76c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 -- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa76c1100c0 msgr2=0x7fa76c1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa76c1100c0 0x7fa76c1a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa770c00640 1 -- 172.21.15.27:0/4147491316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c1a2600 con 0x7fa76c076100 2024-09-21T07:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.093+0000 7fa772000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa76c1100c0 0x7fa76c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:54.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.094+0000 7fa770c00640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c19cc30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa758005ab0 tx=0x7fa758003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:54.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.094+0000 7fa76a000640 1 -- 172.21.15.27:0/4147491316 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75803f070 con 0x7fa76c076100 2024-09-21T07:15:54.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.094+0000 7fa76a000640 1 -- 172.21.15.27:0/4147491316 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa758031b50 con 0x7fa76c076100 2024-09-21T07:15:54.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.094+0000 7fa76a000640 1 -- 172.21.15.27:0/4147491316 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758035030 con 0x7fa76c076100 2024-09-21T07:15:54.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.094+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c1a2890 con 0x7fa76c076100 2024-09-21T07:15:54.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.094+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c1a9050 con 0x7fa76c076100 2024-09-21T07:15:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.097+0000 7fa77ac89740 1 Processor -- start 2024-09-21T07:15:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.097+0000 7fa77ac89740 1 -- start start 2024-09-21T07:15:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd96ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa77ac89740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5613dd8e1f70 con 0x5613dd5c4070 2024-09-21T07:15:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa77ac89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5613dd8d13a0 con 0x5613dd59b1b0 2024-09-21T07:15:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa77ac89740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613dd8d51b0 con 0x5613dd5a3390 2024-09-21T07:15:54.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa76a000640 1 -- 172.21.15.27:0/4147491316 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa758042070 con 0x7fa76c076100 2024-09-21T07:15:54.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa772000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd96ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa772000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd96ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34794/0 (socket says 172.21.15.27:34794) 2024-09-21T07:15:54.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa772000640 1 -- 172.21.15.27:0/4117607259 learned_addr learned my addr 172.21.15.27:0/4117607259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa771600640 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.098+0000 7fa770c00640 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa772000640 1 -- 172.21.15.27:0/4117607259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 msgr2=0x5613dd96b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa772000640 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa772000640 1 -- 172.21.15.27:0/4117607259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 msgr2=0x5613dd96b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa772000640 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa772000640 1 -- 172.21.15.27:0/4117607259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613dd89bc90 con 0x5613dd5c4070 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa770c00640 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa772000640 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd96ab60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa7640117e0 tx=0x7fa764011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a684a1bc0127f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa75e600640 1 -- 172.21.15.27:0/4117607259 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764011310 con 0x5613dd5c4070 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa75e600640 1 -- 172.21.15.27:0/4117607259 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa764003e20 con 0x5613dd5c4070 2024-09-21T07:15:54.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.099+0000 7fa75e600640 1 -- 172.21.15.27:0/4117607259 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640044f0 con 0x5613dd5c4070 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa76a000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa7540b6680 0x7fa7540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa76a000640 1 -- 172.21.15.27:0/4147491316 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa758108090 con 0x7fa76c076100 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa771600640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa7540b6680 0x7fa7540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa77ac89740 1 -- 172.21.15.27:0/4117607259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 msgr2=0x5613dd96ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa77ac89740 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd96ab60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa7640117e0 tx=0x7fa764011ca0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa77ac89740 1 -- 172.21.15.27:0/4117607259 shutdown_connections 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa77ac89740 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa77ac89740 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa77ac89740 1 --2- 172.21.15.27:0/4117607259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd96ab60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.100+0000 7fa77ac89740 1 -- 172.21.15.27:0/4117607259 >> 172.21.15.27:0/4117607259 conn(0x5613dd5b8460 msgr2=0x5613dd968a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.101+0000 7fa77ac89740 1 -- 172.21.15.27:0/4117607259 shutdown_connections 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.101+0000 7fa77ac89740 1 -- 172.21.15.27:0/4117607259 wait complete. 2024-09-21T07:15:54.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.101+0000 7fa771600640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa7540b6680 0x7fa7540b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5613dd9697e0 tx=0x7fa760005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:54.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.101+0000 7fa77ac89740 1 -- start start 2024-09-21T07:15:54.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 1 Processor -- start 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd976790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5a3390 0x5613dd976cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd982630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5613dd732700 con 0x5613dd5a3390 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5613dd86c540 con 0x5613dd59b1b0 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613dd979340 con 0x5613dd5c4070 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.102+0000 7fa77ac89740 10 client.0 _ll_register_callbacks cb 0x5613dd87c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5a3390 0x5613dd976cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5a3390 0x5613dd976cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34802/0 (socket says 172.21.15.27:34802) 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa771600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd976790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 -- 172.21.15.27:0/2786669460 learned_addr learned my addr 172.21.15.27:0/2786669460 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa772000640 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd982630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 -- 172.21.15.27:0/2786669460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 msgr2=0x5613dd982630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd982630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 -- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 msgr2=0x5613dd976790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd976790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 -- 172.21.15.27:0/2786669460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613dd62f650 con 0x5613dd5a3390 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa771600640 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd976790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa770c00640 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5a3390 0x5613dd976cd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa758006010 tx=0x7fa758003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3668f41f6f78800 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.103+0000 7fa752a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.104+0000 7fa752a00640 1 -- 172.21.15.27:0/2786669460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758049030 con 0x5613dd5a3390 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.104+0000 7fa752a00640 1 -- 172.21.15.27:0/2786669460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa75810a110 con 0x5613dd5a3390 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.104+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5613dd86eb50 con 0x5613dd5a3390 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.104+0000 7fa752a00640 1 -- 172.21.15.27:0/2786669460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7580cd530 con 0x5613dd5a3390 2024-09-21T07:15:54.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.104+0000 7fa77ac89740 10 client.16332 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.104+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613dd889910 con 0x5613dd5a3390 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.104+0000 7fa752a00640 1 -- 172.21.15.27:0/2786669460 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5613dd86eb50 con 0x5613dd5a3390 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa752a00640 1 -- 172.21.15.27:0/2786669460 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa7581056b0 con 0x5613dd5a3390 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa77ac89740 10 client.16332 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa77ac89740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa77ac89740 20 client.16332 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd5829f0 0x5613dd9886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa77ac89740 4 client.16332 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613dd8e3ec0 con 0x5613dd5829f0 2024-09-21T07:15:54.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.105+0000 7fa771600640 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd5829f0 0x5613dd9886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.106+0000 7fa771600640 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd5829f0 0x5613dd9886d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d5a0f809202e4f server_cookie=d5aad7022f6bd9e2 in_seq=0 out_seq=0 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.106+0000 7fa752a00640 10 client.16332 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa752a00640 1 -- 172.21.15.27:0/2786669460 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613dd8e3ec0 con 0x5613dd5829f0 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa752a00640 10 client.16332 handle_command_reply: tid=0 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 client.16332 shutdown 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5a3390 msgr2=0x5613dd976cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5a3390 0x5613dd976cd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa758006010 tx=0x7fa758003f90 comp rx=0 tx=0).stop 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 shutdown_connections 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd5829f0 0x5613dd9886d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd982630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5a3390 0x5613dd976cd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2786669460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd976790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 >> 172.21.15.27:0/2786669460 conn(0x5613dd5b8460 msgr2=0x5613dd96dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.107+0000 7fa752a00640 0 client.16332 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.108+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 shutdown_connections 2024-09-21T07:15:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.108+0000 7fa77ac89740 1 -- 172.21.15.27:0/2786669460 wait complete. 2024-09-21T07:15:54.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.108+0000 7fa77ac89740 20 client.16332 trim_cache size 0 max 16384 2024-09-21T07:15:54.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.126+0000 7fa77ac89740 1 Processor -- start 2024-09-21T07:15:54.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.126+0000 7fa77ac89740 1 -- start start 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd970180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa77ac89740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b57f0 con 0x5613dd5c4070 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa77ac89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b30c0 con 0x5613dd59b1b0 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa77ac89740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b0f10 con 0x5613dd5a3390 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd970180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd970180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34808/0 (socket says 172.21.15.27:34808) 2024-09-21T07:15:54.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 -- 172.21.15.27:0/3523435107 learned_addr learned my addr 172.21.15.27:0/3523435107 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:54.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa770c00640 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa771600640 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 -- 172.21.15.27:0/3523435107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 msgr2=0x5613dd96fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 -- 172.21.15.27:0/3523435107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 msgr2=0x5613dd96e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.127+0000 7fa772000640 1 -- 172.21.15.27:0/3523435107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613dd977850 con 0x5613dd5c4070 2024-09-21T07:15:54.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.128+0000 7fa772000640 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd970180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa76400f790 tx=0x7fa764011350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd17520397de845 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:54.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.128+0000 7fa75dc00640 1 -- 172.21.15.27:0/3523435107 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76400fba0 con 0x5613dd5c4070 2024-09-21T07:15:54.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.128+0000 7fa75dc00640 1 -- 172.21.15.27:0/3523435107 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa76402a070 con 0x5613dd5c4070 2024-09-21T07:15:54.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.128+0000 7fa75dc00640 1 -- 172.21.15.27:0/3523435107 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640055b0 con 0x5613dd5c4070 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 -- 172.21.15.27:0/3523435107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 msgr2=0x5613dd970180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd970180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa76400f790 tx=0x7fa764011350 comp rx=0 tx=0).stop 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 -- 172.21.15.27:0/3523435107 shutdown_connections 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5a3390 0x5613dd96fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd5c4070 0x5613dd970180 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 --2- 172.21.15.27:0/3523435107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd59b1b0 0x5613dd96e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 -- 172.21.15.27:0/3523435107 >> 172.21.15.27:0/3523435107 conn(0x5613dd5b8460 msgr2=0x5613dd96dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 -- 172.21.15.27:0/3523435107 shutdown_connections 2024-09-21T07:15:54.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.129+0000 7fa77ac89740 1 -- 172.21.15.27:0/3523435107 wait complete. 2024-09-21T07:15:54.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 -- start start 2024-09-21T07:15:54.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 Processor -- start 2024-09-21T07:15:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd59b1b0 0x5613dd97a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd5a3390 0x5613dd97ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd9f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b1580 con 0x5613dd59b1b0 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b29f0 con 0x5613dd5a3390 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b1c50 con 0x5613dd5c4070 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.130+0000 7fa77ac89740 10 client.0 _ll_register_callbacks cb 0x7fa75810b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa771600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd59b1b0 0x5613dd97a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa770c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd5a3390 0x5613dd97ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa771600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd59b1b0 0x5613dd97a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34814/0 (socket says 172.21.15.27:34814) 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa771600640 1 -- 172.21.15.27:0/2605292377 learned_addr learned my addr 172.21.15.27:0/2605292377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa772000640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd9f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa770c00640 1 -- 172.21.15.27:0/2605292377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 msgr2=0x5613dd9f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa770c00640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd9f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa770c00640 1 -- 172.21.15.27:0/2605292377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd59b1b0 msgr2=0x5613dd97a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa770c00640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd59b1b0 0x5613dd97a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa770c00640 1 -- 172.21.15.27:0/2605292377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613dd9f25d0 con 0x5613dd5a3390 2024-09-21T07:15:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa771600640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd59b1b0 0x5613dd97a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa772000640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd9f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa770c00640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd5a3390 0x5613dd97ae80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7580096c0 tx=0x7fa7580ca060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c00fb8f1905918c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.131+0000 7fa752a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa752a00640 1 -- 172.21.15.27:0/2605292377 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7580cee60 con 0x5613dd5a3390 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa752a00640 1 -- 172.21.15.27:0/2605292377 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa75803bd90 con 0x5613dd5a3390 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76402a070 con 0x5613dd5a3390 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa752a00640 1 -- 172.21.15.27:0/2605292377 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa758049030 con 0x5613dd5a3390 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 10 client.26372 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613dd9f3700 con 0x5613dd5a3390 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa752a00640 1 -- 172.21.15.27:0/2605292377 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa76402a070 con 0x5613dd5a3390 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa752a00640 1 -- 172.21.15.27:0/2605292377 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa7580e3ce0 con 0x5613dd5a3390 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 10 client.26372 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 10 client.26372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:15:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 20 client.26372 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:54.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd557da0 0x5613dd9f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:54.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 4 client.26372 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:15:54.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.132+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613dd8df970 con 0x5613dd557da0 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.133+0000 7fa771600640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd557da0 0x5613dd9f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.133+0000 7fa771600640 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd557da0 0x5613dd9f5d30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51b41354c430785 server_cookie=ce188b63226f5782 in_seq=0 out_seq=0 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.133+0000 7fa752a00640 10 client.26372 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa752a00640 1 -- 172.21.15.27:0/2605292377 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x5613dd8df970 con 0x5613dd557da0 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa752a00640 10 client.26372 handle_command_reply: tid=0 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 client.26372 shutdown 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd5a3390 msgr2=0x5613dd97ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd5a3390 0x5613dd97ae80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7580096c0 tx=0x7fa7580ca060 comp rx=0 tx=0).stop 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 shutdown_connections 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5613dd557da0 0x5613dd9f5d30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613dd5c4070 0x5613dd9f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613dd5a3390 0x5613dd97ae80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 --2- 172.21.15.27:0/2605292377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5613dd59b1b0 0x5613dd97a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa752a00640 0 client.26372 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 >> 172.21.15.27:0/2605292377 conn(0x5613dd5b8460 msgr2=0x5613dd977a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 shutdown_connections 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 1 -- 172.21.15.27:0/2605292377 wait complete. 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.134+0000 7fa77ac89740 20 client.26372 trim_cache size 0 max 16384 2024-09-21T07:15:54.139 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:54.140 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:15:54.141 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa7540b6680 msgr2=0x7fa7540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa7540b6680 0x7fa7540b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5613dd9697e0 tx=0x7fa760005f90 comp rx=0 tx=0).stop 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 msgr2=0x7fa76c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c19cc30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa758005ab0 tx=0x7fa758003b90 comp rx=0 tx=0).stop 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 shutdown_connections 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa7540b6680 0x7fa7540b8b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa76c1100c0 0x7fa76c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa76c076100 0x7fa76c19cc30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 --2- 172.21.15.27:0/4147491316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa76c0757b0 0x7fa76c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 >> 172.21.15.27:0/4147491316 conn(0x7fa76c07b850 msgr2=0x7fa76c107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 shutdown_connections 2024-09-21T07:15:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:54.135+0000 7fa774000640 1 -- 172.21.15.27:0/4147491316 wait complete. 2024-09-21T07:15:55.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:54 smithi027 ceph-mon[30779]: pgmap v464: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 7 op/s 2024-09-21T07:15:55.048 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:15:55.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:15:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:54 smithi153 ceph-mon[34747]: pgmap v464: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 7 op/s 2024-09-21T07:15:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:54 smithi119 ceph-mon[34519]: pgmap v464: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 7 op/s 2024-09-21T07:15:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.076+0000 7fdf31a00640 1 -- 172.21.15.27:0/3844872331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c10b4f0 msgr2=0x7fdf2c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.076+0000 7fdf31a00640 1 --2- 172.21.15.27:0/3844872331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c10b4f0 0x7fdf2c10b900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009900 tx=0x7fdf1402a410 comp rx=0 tx=0).stop 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.077+0000 7fdf31a00640 1 -- 172.21.15.27:0/3844872331 shutdown_connections 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.077+0000 7fdf31a00640 1 --2- 172.21.15.27:0/3844872331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf2c10be40 0x7fdf2c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.077+0000 7fdf31a00640 1 --2- 172.21.15.27:0/3844872331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c10b4f0 0x7fdf2c10b900 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.077+0000 7fdf31a00640 1 --2- 172.21.15.27:0/3844872331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c105bb0 0x7fdf2c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.077+0000 7fdf31a00640 1 -- 172.21.15.27:0/3844872331 >> 172.21.15.27:0/3844872331 conn(0x7fdf2c101680 msgr2=0x7fdf2c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:56.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.077+0000 7fdf31a00640 1 -- 172.21.15.27:0/3844872331 shutdown_connections 2024-09-21T07:15:56.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.077+0000 7fdf31a00640 1 -- 172.21.15.27:0/3844872331 wait complete. 2024-09-21T07:15:56.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 Processor -- start 2024-09-21T07:15:56.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 -- start start 2024-09-21T07:15:56.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c105bb0 0x7fdf2c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:56.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf2c10b4f0 0x7fdf2c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:56.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c10be40 0x7fdf2c1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:56.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c115b50 con 0x7fdf2c105bb0 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1159d0 con 0x7fdf2c10be40 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.078+0000 7fdf31a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c115cd0 con 0x7fdf2c10b4f0 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c105bb0 0x7fdf2c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c105bb0 0x7fdf2c19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34820/0 (socket says 172.21.15.27:34820) 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 -- 172.21.15.27:0/4097334852 learned_addr learned my addr 172.21.15.27:0/4097334852 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf23e00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf2c10b4f0 0x7fdf2c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf31000640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c10be40 0x7fdf2c1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 -- 172.21.15.27:0/4097334852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf2c10b4f0 msgr2=0x7fdf2c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf2c10b4f0 0x7fdf2c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 -- 172.21.15.27:0/4097334852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c10be40 msgr2=0x7fdf2c1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c10be40 0x7fdf2c1a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.079+0000 7fdf2be00640 1 -- 172.21.15.27:0/4097334852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c1a24b0 con 0x7fdf2c105bb0 2024-09-21T07:15:56.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.080+0000 7fdf2be00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c105bb0 0x7fdf2c19c540 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9f0 tx=0x7fdf1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:56.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.080+0000 7fdf29600640 1 -- 172.21.15.27:0/4097334852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00cd80 con 0x7fdf2c105bb0 2024-09-21T07:15:56.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.080+0000 7fdf29600640 1 -- 172.21.15.27:0/4097334852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c0108e0 con 0x7fdf2c105bb0 2024-09-21T07:15:56.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.080+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1a27a0 con 0x7fdf2c105bb0 2024-09-21T07:15:56.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.080+0000 7fdf29600640 1 -- 172.21.15.27:0/4097334852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c018980 con 0x7fdf2c105bb0 2024-09-21T07:15:56.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.080+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c1a8f00 con 0x7fdf2c105bb0 2024-09-21T07:15:56.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.081+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2c105fc0 con 0x7fdf2c105bb0 2024-09-21T07:15:56.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.084+0000 7fdf29600640 1 -- 172.21.15.27:0/4097334852 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdf1c010430 con 0x7fdf2c105bb0 2024-09-21T07:15:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.085+0000 7fdf29600640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdf100b6680 0x7fdf100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.085+0000 7fdf29600640 1 -- 172.21.15.27:0/4097334852 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fdf1c0e8280 con 0x7fdf2c105bb0 2024-09-21T07:15:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.085+0000 7fdf23e00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdf100b6680 0x7fdf100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:56.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.086+0000 7fdf23e00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdf100b6680 0x7fdf100b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf1402aa00 tx=0x7fdf14005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:56.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.088+0000 7fdf29600640 1 -- 172.21.15.27:0/4097334852 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf1c0b2800 con 0x7fdf2c105bb0 2024-09-21T07:15:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.325+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf2c02e640 con 0x7fdf2c105bb0 2024-09-21T07:15:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.327+0000 7fdf29600640 1 -- 172.21.15.27:0/4097334852 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fdf1c0b7740 con 0x7fdf2c105bb0 2024-09-21T07:15:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:15:56.329 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:15:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:15:56.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdf100b6680 msgr2=0x7fdf100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:56.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdf100b6680 0x7fdf100b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf1402aa00 tx=0x7fdf14005a20 comp rx=0 tx=0).stop 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c105bb0 msgr2=0x7fdf2c19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c105bb0 0x7fdf2c19c540 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9f0 tx=0x7fdf1c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 shutdown_connections 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf2c10b4f0 0x7fdf2c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdf100b6680 0x7fdf100b8b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c10be40 0x7fdf2c1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 --2- 172.21.15.27:0/4097334852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf2c105bb0 0x7fdf2c19c540 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.329+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 >> 172.21.15.27:0/4097334852 conn(0x7fdf2c101680 msgr2=0x7fdf2c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.330+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 shutdown_connections 2024-09-21T07:15:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:56.330+0000 7fdf31a00640 1 -- 172.21.15.27:0/4097334852 wait complete. 2024-09-21T07:15:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:56 smithi027 ceph-mon[30779]: pgmap v465: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:15:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:56 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4097334852' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:57.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:15:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:56 smithi153 ceph-mon[34747]: pgmap v465: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:15:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:56 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4097334852' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:56 smithi119 ceph-mon[34519]: pgmap v465: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:15:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:56 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4097334852' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:15:57.566 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 -- 172.21.15.27:0/2562756888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 msgr2=0x7fdd3810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 --2- 172.21.15.27:0/2562756888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3810bad0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009900 tx=0x7fdd2002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 -- 172.21.15.27:0/2562756888 shutdown_connections 2024-09-21T07:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 --2- 172.21.15.27:0/2562756888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd3810c010 0x7fdd38112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 --2- 172.21.15.27:0/2562756888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3810bad0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 --2- 172.21.15.27:0/2562756888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd38105d80 0x7fdd38106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 -- 172.21.15.27:0/2562756888 >> 172.21.15.27:0/2562756888 conn(0x7fdd38101810 msgr2=0x7fdd38103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 -- 172.21.15.27:0/2562756888 shutdown_connections 2024-09-21T07:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.214+0000 7fdd3f000640 1 -- 172.21.15.27:0/2562756888 wait complete. 2024-09-21T07:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.215+0000 7fdd3f000640 1 Processor -- start 2024-09-21T07:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3f000640 1 -- start start 2024-09-21T07:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3f000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd38105d80 0x7fdd3819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3f000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd3810c010 0x7fdd381a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3f000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd38115d20 con 0x7fdd3810b6c0 2024-09-21T07:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3f000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd38115ba0 con 0x7fdd3810c010 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3f000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd38115ea0 con 0x7fdd38105d80 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd2f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd37e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd38105d80 0x7fdd3819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd2f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39946/0 (socket says 172.21.15.27:39946) 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd2f400640 1 -- 172.21.15.27:0/1092483912 learned_addr learned my addr 172.21.15.27:0/1092483912 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.216+0000 7fdd3d000640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd3810c010 0x7fdd381a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.217+0000 7fdd37e00640 1 -- 172.21.15.27:0/1092483912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd3810c010 msgr2=0x7fdd381a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.217+0000 7fdd37e00640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd3810c010 0x7fdd381a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.217+0000 7fdd37e00640 1 -- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 msgr2=0x7fdd3819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.217+0000 7fdd37e00640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.217+0000 7fdd37e00640 1 -- 172.21.15.27:0/1092483912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd381a2660 con 0x7fdd38105d80 2024-09-21T07:15:58.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.217+0000 7fdd2f400640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:58.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.217+0000 7fdd37e00640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd38105d80 0x7fdd3819c6f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800e9f0 tx=0x7fdd2800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:58.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.218+0000 7fdd35600640 1 -- 172.21.15.27:0/1092483912 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2800ccc0 con 0x7fdd38105d80 2024-09-21T07:15:58.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.218+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd381a2950 con 0x7fdd38105d80 2024-09-21T07:15:58.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.218+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd381a90b0 con 0x7fdd38105d80 2024-09-21T07:15:58.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.218+0000 7fdd35600640 1 -- 172.21.15.27:0/1092483912 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd280108e0 con 0x7fdd38105d80 2024-09-21T07:15:58.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.218+0000 7fdd35600640 1 -- 172.21.15.27:0/1092483912 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd28018680 con 0x7fdd38105d80 2024-09-21T07:15:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 Processor -- start 2024-09-21T07:15:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 -- start start 2024-09-21T07:15:58.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c2314070 0x5630c26baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22f3390 0x5630c26bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5630c2631fb0 con 0x5630c22eb1b0 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5630c26213e0 con 0x5630c22f3390 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.221+0000 7fdd45c18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5630c26251f0 con 0x5630c2314070 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c2314070 0x5630c26baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c2314070 0x5630c26baba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45618/0 (socket says 172.21.15.27:45618) 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22f3390 0x5630c26bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 -- 172.21.15.27:0/2177694522 learned_addr learned my addr 172.21.15.27:0/2177694522 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 -- 172.21.15.27:0/2177694522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22f3390 msgr2=0x5630c26bb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 --2- 172.21.15.27:0/2177694522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22f3390 0x5630c26bb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 -- 172.21.15.27:0/2177694522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 msgr2=0x5630c26bb0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:15:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 --2- 172.21.15.27:0/2177694522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26bb0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd3d000640 1 -- 172.21.15.27:0/2177694522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c25ebcd0 con 0x5630c2314070 2024-09-21T07:15:58.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.222+0000 7fdd35600640 1 -- 172.21.15.27:0/1092483912 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdd28010430 con 0x7fdd38105d80 2024-09-21T07:15:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.223+0000 7fdd3d000640 1 --2- 172.21.15.27:0/2177694522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c2314070 0x5630c26baba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009fc0 tx=0x7fdd300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f8944cc5be4749 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.224+0000 7fdd2d600640 1 -- 172.21.15.27:0/2177694522 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3001e070 con 0x5630c2314070 2024-09-21T07:15:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.224+0000 7fdd2d600640 1 -- 172.21.15.27:0/2177694522 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3000fa50 con 0x5630c2314070 2024-09-21T07:15:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.224+0000 7fdd2d600640 1 -- 172.21.15.27:0/2177694522 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30019aa0 con 0x5630c2314070 2024-09-21T07:15:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.224+0000 7fdd35600640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdd1c0b6700 0x7fdd1c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.224+0000 7fdd35600640 1 -- 172.21.15.27:0/1092483912 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fdd280e7a40 con 0x7fdd38105d80 2024-09-21T07:15:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.224+0000 7fdd37e00640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdd1c0b6700 0x7fdd1c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 -- 172.21.15.27:0/2177694522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c2314070 msgr2=0x5630c26baba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2177694522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c2314070 0x5630c26baba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009fc0 tx=0x7fdd300118d0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 -- 172.21.15.27:0/2177694522 shutdown_connections 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2177694522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22f3390 0x5630c26bb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2177694522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26bb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2177694522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c2314070 0x5630c26baba0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 -- 172.21.15.27:0/2177694522 >> 172.21.15.27:0/2177694522 conn(0x5630c2308460 msgr2=0x5630c26b8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 -- 172.21.15.27:0/2177694522 shutdown_connections 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd37e00640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdd1c0b6700 0x7fdd1c0b8c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5630c26b9820 tx=0x7fdd28009640 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.225+0000 7fdd45c18740 1 -- 172.21.15.27:0/2177694522 wait complete. 2024-09-21T07:15:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.226+0000 7fdd45c18740 1 -- start start 2024-09-21T07:15:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.226+0000 7fdd45c18740 1 Processor -- start 2024-09-21T07:15:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd45c18740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5630c2482700 con 0x5630c2314070 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd45c18740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5630c25bc580 con 0x5630c22eb1b0 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd45c18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5630c26c9380 con 0x5630c22f3390 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd45c18740 10 client.0 _ll_register_callbacks cb 0x5630c25cc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26d2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39952/0 (socket says 172.21.15.27:39952) 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd3d000640 1 -- 172.21.15.27:0/3589203054 learned_addr learned my addr 172.21.15.27:0/3589203054 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd2f400640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.227+0000 7fdd3d000640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd2f400640 1 -- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 msgr2=0x5630c26c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd2f400640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd2f400640 1 -- 172.21.15.27:0/3589203054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 msgr2=0x5630c26d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd2f400640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd2f400640 1 -- 172.21.15.27:0/3589203054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c237f650 con 0x5630c22f3390 2024-09-21T07:15:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd3d000640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:15:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd2f400640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c6cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdd20005ab0 tx=0x7fdd20003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57e123c79ca08f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd26000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd26000640 1 -- 172.21.15.27:0/3589203054 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20035040 con 0x5630c22f3390 2024-09-21T07:15:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd26000640 1 -- 172.21.15.27:0/3589203054 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fdd20005020 con 0x5630c22f3390 2024-09-21T07:15:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.228+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630c25beb90 con 0x5630c22f3390 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd26000640 1 -- 172.21.15.27:0/3589203054 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd2004e050 con 0x5630c22f3390 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 10 client.26374 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630c25d9950 con 0x5630c22f3390 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd26000640 1 -- 172.21.15.27:0/3589203054 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5630c25beb90 con 0x5630c22f3390 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd26000640 1 -- 172.21.15.27:0/3589203054 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd200314a0 con 0x5630c22f3390 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 10 client.26374 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 10 client.26374 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:15:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 20 client.26374 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22d29f0 0x5630c26d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 4 client.26374 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.229+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630c2633f00 con 0x5630c22d29f0 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.230+0000 7fdd3d000640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22d29f0 0x5630c26d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.230+0000 7fdd3d000640 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22d29f0 0x5630c26d87d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91facdcc09d14240 server_cookie=17d7ac3c417b1919 in_seq=0 out_seq=0 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.230+0000 7fdd26000640 10 client.26374 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd26000640 1 -- 172.21.15.27:0/3589203054 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5630c2633f00 con 0x5630c22d29f0 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd26000640 10 client.26374 handle_command_reply: tid=0 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 client.26374 shutdown 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 msgr2=0x5630c26c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c6cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdd20005ab0 tx=0x7fdd20003820 comp rx=0 tx=0).stop 2024-09-21T07:15:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 shutdown_connections 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22d29f0 0x5630c26d87d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c6cb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3589203054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd26000640 0 client.26374 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 >> 172.21.15.27:0/3589203054 conn(0x5630c2308460 msgr2=0x5630c26bdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.231+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 shutdown_connections 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.232+0000 7fdd45c18740 1 -- 172.21.15.27:0/3589203054 wait complete. 2024-09-21T07:15:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.232+0000 7fdd45c18740 20 client.26374 trim_cache size 0 max 16384 2024-09-21T07:15:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 Processor -- start 2024-09-21T07:15:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 -- start start 2024-09-21T07:15:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c2314070 0x5630c26bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14012700 con 0x5630c22eb1b0 2024-09-21T07:15:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1400ffd0 con 0x5630c2314070 2024-09-21T07:15:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd45c18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1400de20 con 0x5630c22f3390 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39954/0 (socket says 172.21.15.27:39954) 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.246+0000 7fdd3d000640 1 -- 172.21.15.27:0/2043077489 learned_addr learned my addr 172.21.15.27:0/2043077489 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd2f400640 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd3d000640 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c2314070 0x5630c26bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd3d000640 1 -- 172.21.15.27:0/2043077489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 msgr2=0x5630c26c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd3d000640 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd3d000640 1 -- 172.21.15.27:0/2043077489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c2314070 msgr2=0x5630c26bfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd3d000640 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c2314070 0x5630c26bfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd3d000640 1 -- 172.21.15.27:0/2043077489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c26b4240 con 0x5630c22eb1b0 2024-09-21T07:15:58.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd3d000640 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26c4df0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009dd0 tx=0x7fdd30019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66e2f191df73723 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd2cc00640 1 -- 172.21.15.27:0/2043077489 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3001e040 con 0x5630c22eb1b0 2024-09-21T07:15:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd2cc00640 1 -- 172.21.15.27:0/2043077489 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3001f040 con 0x5630c22eb1b0 2024-09-21T07:15:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.247+0000 7fdd2cc00640 1 -- 172.21.15.27:0/2043077489 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300048e0 con 0x5630c22eb1b0 2024-09-21T07:15:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 -- 172.21.15.27:0/2043077489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 msgr2=0x5630c26c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26c4df0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009dd0 tx=0x7fdd30019600 comp rx=0 tx=0).stop 2024-09-21T07:15:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 -- 172.21.15.27:0/2043077489 shutdown_connections 2024-09-21T07:15:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c2314070 0x5630c26bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 --2- 172.21.15.27:0/2043077489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c22eb1b0 0x5630c26c4df0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 -- 172.21.15.27:0/2043077489 >> 172.21.15.27:0/2043077489 conn(0x5630c2308460 msgr2=0x5630c26c16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 -- 172.21.15.27:0/2043077489 shutdown_connections 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.248+0000 7fdd45c18740 1 -- 172.21.15.27:0/2043077489 wait complete. 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.249+0000 7fdd45c18740 1 -- start start 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.249+0000 7fdd45c18740 1 Processor -- start 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd45c18740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd45c18740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1400e490 con 0x5630c2314070 2024-09-21T07:15:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd45c18740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1400f900 con 0x5630c22eb1b0 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd45c18740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1400eb60 con 0x5630c22f3390 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd45c18740 10 client.0 _ll_register_callbacks cb 0x7fdd20004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd2f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd3d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36442/0 (socket says 172.21.15.27:36442) 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd3d000640 1 -- 172.21.15.27:0/3794152527 learned_addr learned my addr 172.21.15.27:0/3794152527 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.250+0000 7fdd3d000640 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd2f400640 1 -- 172.21.15.27:0/3794152527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 msgr2=0x5630c26ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd2f400640 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd2f400640 1 -- 172.21.15.27:0/3794152527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 msgr2=0x5630c26cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd2f400640 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd2f400640 1 -- 172.21.15.27:0/3794152527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c27422a0 con 0x5630c22f3390 2024-09-21T07:15:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd2f400640 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26cacf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd200318a0 tx=0x7fdd20005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc2018076dc932a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:15:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd26000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd26000640 1 -- 172.21.15.27:0/3794152527 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20057060 con 0x5630c22f3390 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3001f040 con 0x5630c22f3390 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.251+0000 7fdd26000640 1 -- 172.21.15.27:0/3794152527 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fdd20031a90 con 0x5630c22f3390 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.252+0000 7fdd26000640 1 -- 172.21.15.27:0/3794152527 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd2005e050 con 0x5630c22f3390 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.252+0000 7fdd45c18740 10 client.26380 fetch_fsmap learned FSMap version 27 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.252+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630c2743520 con 0x5630c22f3390 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.252+0000 7fdd26000640 1 -- 172.21.15.27:0/3794152527 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3001f040 con 0x5630c22f3390 2024-09-21T07:15:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd26000640 1 -- 172.21.15.27:0/3794152527 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd20005040 con 0x5630c22f3390 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd45c18740 10 client.26380 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd45c18740 10 client.26380 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd45c18740 20 client.26380 populate_metadata read hostname 'smithi027' 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22a7da0 0x5630c2745d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd45c18740 4 client.26380 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630c262f9b0 con 0x5630c22a7da0 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.253+0000 7fdd3d000640 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22a7da0 0x5630c2745d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.254+0000 7fdd3d000640 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22a7da0 0x5630c2745d10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9d6a6494f271daf server_cookie=af28f368166754ec in_seq=0 out_seq=0 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.254+0000 7fdd26000640 10 client.26380 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.254+0000 7fdd26000640 1 -- 172.21.15.27:0/3794152527 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630c262f9b0 con 0x5630c22a7da0 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.254+0000 7fdd26000640 10 client.26380 handle_command_reply: tid=0 2024-09-21T07:15:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 client.26380 shutdown 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 msgr2=0x5630c26cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26cacf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd200318a0 tx=0x7fdd20005fd0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 shutdown_connections 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5630c22a7da0 0x5630c2745d10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630c22f3390 0x5630c26cacf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630c2314070 0x5630c26cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 --2- 172.21.15.27:0/3794152527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630c22eb1b0 0x5630c26ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd26000640 0 client.26380 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 >> 172.21.15.27:0/3794152527 conn(0x5630c2308460 msgr2=0x5630c26c16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 shutdown_connections 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 1 -- 172.21.15.27:0/3794152527 wait complete. 2024-09-21T07:15:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.255+0000 7fdd45c18740 20 client.26380 trim_cache size 0 max 16384 2024-09-21T07:15:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:15:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdd1c0b6700 msgr2=0x7fdd1c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdd1c0b6700 0x7fdd1c0b8c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5630c26b9820 tx=0x7fdd28009640 comp rx=0 tx=0).stop 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd38105d80 msgr2=0x7fdd3819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd38105d80 0x7fdd3819c6f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800e9f0 tx=0x7fdd2800edf0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 shutdown_connections 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdd1c0b6700 0x7fdd1c0b8c10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd3810c010 0x7fdd381a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd3810b6c0 0x7fdd3819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 --2- 172.21.15.27:0/1092483912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd38105d80 0x7fdd3819c6f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 >> 172.21.15.27:0/1092483912 conn(0x7fdd38101810 msgr2=0x7fdd38103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 shutdown_connections 2024-09-21T07:15:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:15:58.256+0000 7fdd3f000640 1 -- 172.21.15.27:0/1092483912 wait complete. 2024-09-21T07:15:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:15:58 smithi027 ceph-mon[30779]: pgmap v466: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:15:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:15:58 smithi153 ceph-mon[34747]: pgmap v466: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:15:59.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:15:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:15:58 smithi119 ceph-mon[34519]: pgmap v466: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:15:59.603 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:00.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 -- 172.21.15.27:0/2207619496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e4103b60 msgr2=0x7f85e4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 --2- 172.21.15.27:0/2207619496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e4103b60 0x7f85e4103f70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009900 tx=0x7f85d802a410 comp rx=0 tx=0).stop 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 -- 172.21.15.27:0/2207619496 shutdown_connections 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 --2- 172.21.15.27:0/2207619496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4109df0 0x7f85e4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 --2- 172.21.15.27:0/2207619496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e41094a0 0x7f85e41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 --2- 172.21.15.27:0/2207619496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e4103b60 0x7f85e4103f70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 -- 172.21.15.27:0/2207619496 >> 172.21.15.27:0/2207619496 conn(0x7f85e407b850 msgr2=0x7f85e407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 -- 172.21.15.27:0/2207619496 shutdown_connections 2024-09-21T07:16:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.199+0000 7f85eb800640 1 -- 172.21.15.27:0/2207619496 wait complete. 2024-09-21T07:16:00.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.201+0000 7f85eb800640 1 Processor -- start 2024-09-21T07:16:00.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.201+0000 7f85eb800640 1 -- start start 2024-09-21T07:16:00.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85eb800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e4103b60 0x7f85e419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85eb800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e41094a0 0x7f85e419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85eb800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4109df0 0x7f85e41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85eb800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113ab0 con 0x7f85e41094a0 2024-09-21T07:16:00.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85eb800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113930 con 0x7f85e4103b60 2024-09-21T07:16:00.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85eb800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113c30 con 0x7f85e4109df0 2024-09-21T07:16:00.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e4103b60 0x7f85e419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e4103b60 0x7f85e419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36462/0 (socket says 172.21.15.27:36462) 2024-09-21T07:16:00.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85e8e00640 1 -- 172.21.15.27:0/4253397082 learned_addr learned my addr 172.21.15.27:0/4253397082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:00.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85e3e00640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e41094a0 0x7f85e419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.202+0000 7f85e9800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4109df0 0x7f85e41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.203+0000 7f85e8e00640 1 -- 172.21.15.27:0/4253397082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4109df0 msgr2=0x7f85e41a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.203+0000 7f85e8e00640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4109df0 0x7f85e41a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.203+0000 7f85e8e00640 1 -- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e41094a0 msgr2=0x7f85e419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.203+0000 7f85e8e00640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e41094a0 0x7f85e419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.203+0000 7f85e8e00640 1 -- 172.21.15.27:0/4253397082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e41a2630 con 0x7f85e4103b60 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.203+0000 7f85e8e00640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e4103b60 0x7f85e419c6c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f85d8005940 tx=0x7f85d8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.203+0000 7f85e1600640 1 -- 172.21.15.27:0/4253397082 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d803f030 con 0x7f85e4103b60 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.204+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e41a28c0 con 0x7f85e4103b60 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.204+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41a9080 con 0x7f85e4103b60 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.204+0000 7f85e1600640 1 -- 172.21.15.27:0/4253397082 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d8003aa0 con 0x7f85e4103b60 2024-09-21T07:16:00.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.204+0000 7f85e1600640 1 -- 172.21.15.27:0/4253397082 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8049060 con 0x7f85e4103b60 2024-09-21T07:16:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 Processor -- start 2024-09-21T07:16:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 -- start start 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350fd390 0x55cf354c5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cf3543bfb0 con 0x55cf350f51b0 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cf3542b3e0 con 0x55cf350fd390 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85f24b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf3542f1f0 con 0x55cf3511e070 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.207+0000 7f85e1600640 1 -- 172.21.15.27:0/4253397082 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f85d8042030 con 0x7f85e4103b60 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350fd390 0x55cf354c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350fd390 0x55cf354c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36472/0 (socket says 172.21.15.27:36472) 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 -- 172.21.15.27:0/1049700524 learned_addr learned my addr 172.21.15.27:0/1049700524 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e8e00640 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e9800640 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 -- 172.21.15.27:0/1049700524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 msgr2=0x55cf354c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 -- 172.21.15.27:0/1049700524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 msgr2=0x55cf354c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e3e00640 1 -- 172.21.15.27:0/1049700524 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf353f5cd0 con 0x55cf350fd390 2024-09-21T07:16:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.208+0000 7f85e8e00640 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.209+0000 7f85e3e00640 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350fd390 0x55cf354c5620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85d000bca0 tx=0x7f85d00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196b9effeb7951ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.209+0000 7f85d6000640 1 -- 172.21.15.27:0/1049700524 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d001c680 con 0x55cf350fd390 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.209+0000 7f85d6000640 1 -- 172.21.15.27:0/1049700524 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d000be50 con 0x55cf350fd390 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.209+0000 7f85d6000640 1 -- 172.21.15.27:0/1049700524 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0011c30 con 0x55cf350fd390 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.209+0000 7f85e1600640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85cc0b6700 0x7f85cc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 -- 172.21.15.27:0/1049700524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350fd390 msgr2=0x55cf354c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350fd390 0x55cf354c5620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85d000bca0 tx=0x7f85d00107a0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85e1600640 1 -- 172.21.15.27:0/4253397082 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f85d8107e00 con 0x7f85e4103b60 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85e9800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85cc0b6700 0x7f85cc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 -- 172.21.15.27:0/1049700524 shutdown_connections 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350fd390 0x55cf354c5620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1049700524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 -- 172.21.15.27:0/1049700524 >> 172.21.15.27:0/1049700524 conn(0x55cf35112460 msgr2=0x55cf354c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:00.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 -- 172.21.15.27:0/1049700524 shutdown_connections 2024-09-21T07:16:00.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.210+0000 7f85f24b1740 1 -- 172.21.15.27:0/1049700524 wait complete. 2024-09-21T07:16:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.211+0000 7f85e9800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85cc0b6700 0x7f85cc0b8c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85dc005fd0 tx=0x7f85dc009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.211+0000 7f85f24b1740 1 -- start start 2024-09-21T07:16:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.211+0000 7f85f24b1740 1 Processor -- start 2024-09-21T07:16:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85f24b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cf3528c700 con 0x55cf350fd390 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85f24b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cf353c6580 con 0x55cf350f51b0 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85f24b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf354d3380 con 0x55cf3511e070 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85f24b1740 10 client.0 _ll_register_callbacks cb 0x55cf353d6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36474/0 (socket says 172.21.15.27:36474) 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40002/0 (socket says 172.21.15.27:40002) 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85e8e00640 1 -- 172.21.15.27:0/2960304592 learned_addr learned my addr 172.21.15.27:0/2960304592 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.212+0000 7f85e9800640 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85e8e00640 1 -- 172.21.15.27:0/2960304592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 msgr2=0x55cf354dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85e8e00640 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85e8e00640 1 -- 172.21.15.27:0/2960304592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 msgr2=0x55cf354d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85e8e00640 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85e8e00640 1 -- 172.21.15.27:0/2960304592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf35189650 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85e9800640 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85e8e00640 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d07f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f85d8006010 tx=0x7f85d8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3acd31d47fd0e48a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85caa00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85caa00640 1 -- 172.21.15.27:0/2960304592 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8035070 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf353c8b90 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85caa00640 1 -- 172.21.15.27:0/2960304592 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f85d8003c50 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85caa00640 1 -- 172.21.15.27:0/2960304592 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85d80cd720 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85f24b1740 10 client.26408 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.213+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf353e3950 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85caa00640 1 -- 172.21.15.27:0/2960304592 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cf353c8b90 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85caa00640 1 -- 172.21.15.27:0/2960304592 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f85d804c060 con 0x55cf350f51b0 2024-09-21T07:16:00.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85f24b1740 10 client.26408 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:00.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85f24b1740 10 client.26408 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:16:00.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85f24b1740 20 client.26408 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:00.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350dc9f0 0x55cf354e2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85f24b1740 4 client.26408 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:00.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf3543df00 con 0x55cf350dc9f0 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.214+0000 7f85e3e00640 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350dc9f0 0x55cf354e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.215+0000 7f85e3e00640 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350dc9f0 0x55cf354e2710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3913125841e754ed server_cookie=7f45c9ae806262bc in_seq=0 out_seq=0 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.215+0000 7f85caa00640 10 client.26408 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85caa00640 1 -- 172.21.15.27:0/2960304592 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf3543df00 con 0x55cf350dc9f0 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85caa00640 10 client.26408 handle_command_reply: tid=0 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 client.26408 shutdown 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 msgr2=0x55cf354d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d07f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f85d8006010 tx=0x7f85d8009620 comp rx=0 tx=0).stop 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 shutdown_connections 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350dc9f0 0x55cf354e2710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2960304592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d07f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85caa00640 0 client.26408 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 >> 172.21.15.27:0/2960304592 conn(0x55cf35112460 msgr2=0x55cf354c7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.216+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 shutdown_connections 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.217+0000 7f85f24b1740 1 -- 172.21.15.27:0/2960304592 wait complete. 2024-09-21T07:16:00.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.217+0000 7f85f24b1740 20 client.26408 trim_cache size 0 max 16384 2024-09-21T07:16:00.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 Processor -- start 2024-09-21T07:16:00.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 -- start start 2024-09-21T07:16:00.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354c2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b57f0 con 0x55cf350fd390 2024-09-21T07:16:00.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b30c0 con 0x55cf350f51b0 2024-09-21T07:16:00.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.232+0000 7f85f24b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b0f10 con 0x55cf3511e070 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36476/0 (socket says 172.21.15.27:36476) 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 -- 172.21.15.27:0/2075183289 learned_addr learned my addr 172.21.15.27:0/2075183289 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e9800640 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e3e00640 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 -- 172.21.15.27:0/2075183289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 msgr2=0x55cf354c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 -- 172.21.15.27:0/2075183289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 msgr2=0x55cf354c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.233+0000 7f85e8e00640 1 -- 172.21.15.27:0/2075183289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf354c72f0 con 0x55cf350f51b0 2024-09-21T07:16:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.234+0000 7f85e8e00640 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d2220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f85d80c9f20 tx=0x7f85d80ccbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe9fc6cadced2eff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:00.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.234+0000 7f85d5600640 1 -- 172.21.15.27:0/2075183289 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8035070 con 0x55cf350f51b0 2024-09-21T07:16:00.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.234+0000 7f85d5600640 1 -- 172.21.15.27:0/2075183289 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d80cd460 con 0x55cf350f51b0 2024-09-21T07:16:00.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.234+0000 7f85d5600640 1 -- 172.21.15.27:0/2075183289 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80cd0c0 con 0x55cf350f51b0 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 -- 172.21.15.27:0/2075183289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 msgr2=0x55cf354d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d2220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f85d80c9f20 tx=0x7f85d80ccbb0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 -- 172.21.15.27:0/2075183289 shutdown_connections 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf3511e070 0x55cf354c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350fd390 0x55cf354c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 --2- 172.21.15.27:0/2075183289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf350f51b0 0x55cf354d2220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 -- 172.21.15.27:0/2075183289 >> 172.21.15.27:0/2075183289 conn(0x55cf35112460 msgr2=0x55cf354d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 -- 172.21.15.27:0/2075183289 shutdown_connections 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.235+0000 7f85f24b1740 1 -- 172.21.15.27:0/2075183289 wait complete. 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 -- start start 2024-09-21T07:16:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 Processor -- start 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354d4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf350fd390 0x55cf354d54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf3511e070 0x55cf35546110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b1580 con 0x55cf350f51b0 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b29f0 con 0x55cf3511e070 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.236+0000 7f85f24b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b1c50 con 0x55cf350fd390 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85f24b1740 10 client.0 _ll_register_callbacks cb 0x7f85d803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354d4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354d4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40028/0 (socket says 172.21.15.27:40028) 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e8e00640 1 -- 172.21.15.27:0/1171855744 learned_addr learned my addr 172.21.15.27:0/1171855744 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e3e00640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf350fd390 0x55cf354d54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e9800640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf3511e070 0x55cf35546110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e3e00640 1 -- 172.21.15.27:0/1171855744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf3511e070 msgr2=0x55cf35546110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e3e00640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf3511e070 0x55cf35546110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e3e00640 1 -- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 msgr2=0x55cf354d4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e3e00640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354d4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e3e00640 1 -- 172.21.15.27:0/1171855744 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf354c59e0 con 0x55cf350fd390 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.237+0000 7f85e8e00640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354d4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85e3e00640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf350fd390 0x55cf354d54d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85d0000c00 tx=0x7f85d0010de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=191ec76182ece21a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85caa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85caa00640 1 -- 172.21.15.27:0/1171855744 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0011440 con 0x55cf350fd390 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d80cd460 con 0x55cf350fd390 2024-09-21T07:16:00.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85caa00640 1 -- 172.21.15.27:0/1171855744 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f85d0004020 con 0x55cf350fd390 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85caa00640 1 -- 172.21.15.27:0/1171855744 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85d003d050 con 0x55cf350fd390 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85f24b1740 10 client.26398 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.238+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf3554d8c0 con 0x55cf350fd390 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85caa00640 1 -- 172.21.15.27:0/1171855744 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d80cd460 con 0x55cf350fd390 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85caa00640 1 -- 172.21.15.27:0/1171855744 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f85d00119d0 con 0x55cf350fd390 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85f24b1740 10 client.26398 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85f24b1740 10 client.26398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85f24b1740 20 client.26398 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350b1da0 0x55cf3554fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85f24b1740 4 client.26398 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf354399b0 con 0x55cf350b1da0 2024-09-21T07:16:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.239+0000 7f85e8e00640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350b1da0 0x55cf3554fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.240+0000 7f85e8e00640 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350b1da0 0x55cf3554fda0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10e20eae0edb9b44 server_cookie=b35cdbecdd649eae in_seq=0 out_seq=0 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.240+0000 7f85caa00640 10 client.26398 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.240+0000 7f85caa00640 1 -- 172.21.15.27:0/1171855744 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf354399b0 con 0x55cf350b1da0 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.240+0000 7f85caa00640 10 client.26398 handle_command_reply: tid=0 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.240+0000 7f85f24b1740 1 client.26398 shutdown 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf350fd390 msgr2=0x55cf354d54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf350fd390 0x55cf354d54d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85d0000c00 tx=0x7f85d0010de0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 shutdown_connections 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55cf350b1da0 0x55cf3554fda0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf350fd390 0x55cf354d54d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf3511e070 0x55cf35546110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 --2- 172.21.15.27:0/1171855744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf350f51b0 0x55cf354d4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85caa00640 0 client.26398 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 >> 172.21.15.27:0/1171855744 conn(0x55cf35112460 msgr2=0x55cf354c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 shutdown_connections 2024-09-21T07:16:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 1 -- 172.21.15.27:0/1171855744 wait complete. 2024-09-21T07:16:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.241+0000 7f85f24b1740 20 client.26398 trim_cache size 0 max 16384 2024-09-21T07:16:00.245 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:16:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85cc0b6700 msgr2=0x7f85cc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85cc0b6700 0x7f85cc0b8c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85dc005fd0 tx=0x7f85dc009500 comp rx=0 tx=0).stop 2024-09-21T07:16:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e4103b60 msgr2=0x7f85e419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e4103b60 0x7f85e419c6c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f85d8005940 tx=0x7f85d8004270 comp rx=0 tx=0).stop 2024-09-21T07:16:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 shutdown_connections 2024-09-21T07:16:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4109df0 0x7f85e41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f85cc0b6700 0x7f85cc0b8c10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85e41094a0 0x7f85e419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.242+0000 7f85eb800640 1 --2- 172.21.15.27:0/4253397082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85e4103b60 0x7f85e419c6c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.243+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 >> 172.21.15.27:0/4253397082 conn(0x7f85e407b850 msgr2=0x7f85e41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.243+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 shutdown_connections 2024-09-21T07:16:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:00.243+0000 7f85eb800640 1 -- 172.21.15.27:0/4253397082 wait complete. 2024-09-21T07:16:01.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:00 smithi027 ceph-mon[30779]: pgmap v467: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:16:01.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:00 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:16:01.071 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:16:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:00 smithi153 ceph-mon[34747]: pgmap v467: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:16:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:00 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:16:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:00 smithi119 ceph-mon[34519]: pgmap v467: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:16:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:00 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:16:01.500 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:01.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:16:01.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:16:01.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:16:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 -- 172.21.15.27:0/3899816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec114570 msgr2=0x7fc3ec116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/3899816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec114570 0x7fc3ec116990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 -- 172.21.15.27:0/3899816 shutdown_connections 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/3899816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ec078290 0x7fc3ec114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/3899816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec114570 0x7fc3ec116990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/3899816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 0x7fc3ec077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 -- 172.21.15.27:0/3899816 >> 172.21.15.27:0/3899816 conn(0x7fc3ec101680 msgr2=0x7fc3ec103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 -- 172.21.15.27:0/3899816 shutdown_connections 2024-09-21T07:16:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.153+0000 7fc3f2e00640 1 -- 172.21.15.27:0/3899816 wait complete. 2024-09-21T07:16:02.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.155+0000 7fc3f2e00640 1 Processor -- start 2024-09-21T07:16:02.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.155+0000 7fc3f2e00640 1 -- start start 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.155+0000 7fc3f2e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 0x7fc3ec1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.155+0000 7fc3f2e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec078290 0x7fc3ec1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.155+0000 7fc3f2e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ec114570 0x7fc3ec1a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f2e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec119f40 con 0x7fc3ec078290 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f2e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec119dc0 con 0x7fc3ec077940 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f2e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec11a0c0 con 0x7fc3ec114570 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 0x7fc3ec1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 0x7fc3ec1a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36504/0 (socket says 172.21.15.27:36504) 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 -- 172.21.15.27:0/2728939955 learned_addr learned my addr 172.21.15.27:0/2728939955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1000640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec078290 0x7fc3ec1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f2400640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ec114570 0x7fc3ec1a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 -- 172.21.15.27:0/2728939955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ec114570 msgr2=0x7fc3ec1a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ec114570 0x7fc3ec1a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 -- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec078290 msgr2=0x7fc3ec1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec078290 0x7fc3ec1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1a00640 1 -- 172.21.15.27:0/2728939955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1a68c0 con 0x7fc3ec077940 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.156+0000 7fc3f1000640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec078290 0x7fc3ec1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.157+0000 7fc3f1a00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 0x7fc3ec1a0950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000ea60 tx=0x7fc3e000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.157+0000 7fc3ea000640 1 -- 172.21.15.27:0/2728939955 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000cde0 con 0x7fc3ec077940 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.157+0000 7fc3ea000640 1 -- 172.21.15.27:0/2728939955 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e00035f0 con 0x7fc3ec077940 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.157+0000 7fc3ea000640 1 -- 172.21.15.27:0/2728939955 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0010690 con 0x7fc3ec077940 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.157+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1a6bb0 con 0x7fc3ec077940 2024-09-21T07:16:02.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.157+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1ad310 con 0x7fc3ec077940 2024-09-21T07:16:02.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.160+0000 7fc3fb043740 1 Processor -- start 2024-09-21T07:16:02.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.160+0000 7fc3fb043740 1 -- start start 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bb3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bb44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3fb043740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556684b2afb0 con 0x5566847ec390 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3fb043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556684b1a3e0 con 0x5566847e41b0 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3fb043740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556684b1e1f0 con 0x55668480d070 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3ea000640 1 -- 172.21.15.27:0/2728939955 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc3e00037a0 con 0x7fc3ec077940 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bb3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3f1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bb3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45724/0 (socket says 172.21.15.27:45724) 2024-09-21T07:16:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3f2400640 1 -- 172.21.15.27:0/214915297 learned_addr learned my addr 172.21.15.27:0/214915297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.161+0000 7fc3f1a00640 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bb44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3f2400640 1 -- 172.21.15.27:0/214915297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 msgr2=0x556684bb44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3f2400640 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bb44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3f2400640 1 -- 172.21.15.27:0/214915297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 msgr2=0x556684bb49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3f2400640 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3f2400640 1 -- 172.21.15.27:0/214915297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556684ae4cd0 con 0x55668480d070 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3f1000640 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3f2400640 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bb3f70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e4009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85d2e2a498f205dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3dea00640 1 -- 172.21.15.27:0/214915297 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4031cf0 con 0x55668480d070 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3dea00640 1 -- 172.21.15.27:0/214915297 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e4036030 con 0x55668480d070 2024-09-21T07:16:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.162+0000 7fc3dea00640 1 -- 172.21.15.27:0/214915297 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40042a0 con 0x55668480d070 2024-09-21T07:16:02.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.163+0000 7fc3ea000640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3d40b6700 0x7fc3d40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.163+0000 7fc3ea000640 1 -- 172.21.15.27:0/2728939955 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc3e00e7dd0 con 0x7fc3ec077940 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.163+0000 7fc3f1a00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3d40b6700 0x7fc3d40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.163+0000 7fc3fb043740 1 -- 172.21.15.27:0/214915297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 msgr2=0x556684bb3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.163+0000 7fc3fb043740 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bb3f70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e4009430 comp rx=0 tx=0).stop 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.163+0000 7fc3fb043740 1 -- 172.21.15.27:0/214915297 shutdown_connections 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3fb043740 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3fb043740 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bb44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3fb043740 1 --2- 172.21.15.27:0/214915297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bb3f70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3fb043740 1 -- 172.21.15.27:0/214915297 >> 172.21.15.27:0/214915297 conn(0x556684801460 msgr2=0x556684bb1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3fb043740 1 -- 172.21.15.27:0/214915297 shutdown_connections 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3fb043740 1 -- 172.21.15.27:0/214915297 wait complete. 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3f1a00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3d40b6700 0x7fc3d40b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556684bb2bf0 tx=0x7fc3e00eb880 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.164+0000 7fc3fb043740 1 -- start start 2024-09-21T07:16:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 1 Processor -- start 2024-09-21T07:16:02.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bbf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847ec390 0x556684bbfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55668480d070 0x556684bcb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55668497b700 con 0x55668480d070 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556684ab5580 con 0x5566847e41b0 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556684bc2570 con 0x5566847ec390 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3fb043740 10 client.0 _ll_register_callbacks cb 0x556684ac5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3f1000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847ec390 0x556684bbfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.165+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bbf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bbf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36520/0 (socket says 172.21.15.27:36520) 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f2400640 1 -- 172.21.15.27:0/4080748126 learned_addr learned my addr 172.21.15.27:0/4080748126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f2400640 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55668480d070 0x556684bcb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f1000640 1 -- 172.21.15.27:0/4080748126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 msgr2=0x556684bbf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f1000640 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bbf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f1000640 1 -- 172.21.15.27:0/4080748126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55668480d070 msgr2=0x556684bcb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f1000640 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55668480d070 0x556684bcb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f1000640 1 -- 172.21.15.27:0/4080748126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556684878650 con 0x5566847ec390 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.166+0000 7fc3f2400640 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55668480d070 0x556684bcb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:02.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3f1000640 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847ec390 0x556684bbfc10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec06c630 tx=0x7fc3d8012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84cdde93bcac4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3d3400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3d3400640 1 -- 172.21.15.27:0/4080748126 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8011c40 con 0x5566847ec390 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3d3400640 1 -- 172.21.15.27:0/4080748126 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc3d8011de0 con 0x5566847ec390 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556684ab7b90 con 0x5566847ec390 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3d3400640 1 -- 172.21.15.27:0/4080748126 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3d8025050 con 0x5566847ec390 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3fb043740 10 client.26410 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556684ad2950 con 0x5566847ec390 2024-09-21T07:16:02.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.167+0000 7fc3d3400640 1 -- 172.21.15.27:0/4080748126 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556684ab7b90 con 0x5566847ec390 2024-09-21T07:16:02.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3d3400640 1 -- 172.21.15.27:0/4080748126 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc3d8010440 con 0x5566847ec390 2024-09-21T07:16:02.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3fb043740 10 client.26410 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:02.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3fb043740 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3fb043740 20 client.26410 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3fb043740 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847cb9f0 0x556684bd17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3fb043740 4 client.26410 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556684b2cf00 con 0x5566847cb9f0 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.168+0000 7fc3f2400640 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847cb9f0 0x556684bd17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.169+0000 7fc3f2400640 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847cb9f0 0x556684bd17a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418cae4aefdbcade server_cookie=1bca09406b26a37 in_seq=0 out_seq=0 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.169+0000 7fc3d3400640 10 client.26410 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3d3400640 1 -- 172.21.15.27:0/4080748126 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556684b2cf00 con 0x5566847cb9f0 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3d3400640 10 client.26410 handle_command_reply: tid=0 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 client.26410 shutdown 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847ec390 msgr2=0x556684bbfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847ec390 0x556684bbfc10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec06c630 tx=0x7fc3d8012b60 comp rx=0 tx=0).stop 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 shutdown_connections 2024-09-21T07:16:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847cb9f0 0x556684bd17a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847ec390 0x556684bbfc10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55668480d070 0x556684bcb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 --2- 172.21.15.27:0/4080748126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847e41b0 0x556684bbf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 >> 172.21.15.27:0/4080748126 conn(0x556684801460 msgr2=0x556684bb7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3d3400640 0 client.26410 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 shutdown_connections 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.170+0000 7fc3fb043740 1 -- 172.21.15.27:0/4080748126 wait complete. 2024-09-21T07:16:02.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.171+0000 7fc3fb043740 20 client.26410 trim_cache size 0 max 16384 2024-09-21T07:16:02.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:16:02.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:16:02.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:16:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.184+0000 7fc3fb043740 1 Processor -- start 2024-09-21T07:16:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3fb043740 1 -- start start 2024-09-21T07:16:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847e41b0 0x556684bc15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668480d070 0x556684bcb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3fb043740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b9ab0 con 0x5566847ec390 2024-09-21T07:16:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3fb043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b7380 con 0x55668480d070 2024-09-21T07:16:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3fb043740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b51d0 con 0x5566847e41b0 2024-09-21T07:16:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668480d070 0x556684bcb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668480d070 0x556684bcb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36536/0 (socket says 172.21.15.27:36536) 2024-09-21T07:16:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3f2400640 1 -- 172.21.15.27:0/593117980 learned_addr learned my addr 172.21.15.27:0/593117980 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.185+0000 7fc3f2400640 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847e41b0 0x556684bc15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3f2400640 1 -- 172.21.15.27:0/593117980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847e41b0 msgr2=0x556684bc15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3f2400640 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847e41b0 0x556684bc15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3f2400640 1 -- 172.21.15.27:0/593117980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 msgr2=0x556684bb56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3f2400640 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3f2400640 1 -- 172.21.15.27:0/593117980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556684bc2200 con 0x55668480d070 2024-09-21T07:16:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3f2400640 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668480d070 0x556684bcb790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e4005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df637e787c7ef64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3de000640 1 -- 172.21.15.27:0/593117980 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e405c040 con 0x55668480d070 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3de000640 1 -- 172.21.15.27:0/593117980 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e4036040 con 0x55668480d070 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.186+0000 7fc3de000640 1 -- 172.21.15.27:0/593117980 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e404a040 con 0x55668480d070 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 -- 172.21.15.27:0/593117980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668480d070 msgr2=0x556684bcb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668480d070 0x556684bcb790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e4005d80 comp rx=0 tx=0).stop 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 -- 172.21.15.27:0/593117980 shutdown_connections 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668480d070 0x556684bcb790 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847ec390 0x556684bb56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 --2- 172.21.15.27:0/593117980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566847e41b0 0x556684bc15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 -- 172.21.15.27:0/593117980 >> 172.21.15.27:0/593117980 conn(0x556684801460 msgr2=0x556684bb4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 -- 172.21.15.27:0/593117980 shutdown_connections 2024-09-21T07:16:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.187+0000 7fc3fb043740 1 -- 172.21.15.27:0/593117980 wait complete. 2024-09-21T07:16:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.188+0000 7fc3fb043740 1 -- start start 2024-09-21T07:16:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.188+0000 7fc3fb043740 1 Processor -- start 2024-09-21T07:16:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.188+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847e41b0 0x556684bc7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.188+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847ec390 0x556684bc80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3fb043740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bc27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3fb043740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b5840 con 0x5566847e41b0 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3fb043740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b6cb0 con 0x5566847ec390 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3fb043740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b5f10 con 0x55668480d070 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3fb043740 10 client.0 _ll_register_callbacks cb 0x7fc3d8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847e41b0 0x556684bc7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847ec390 0x556684bc80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847e41b0 0x556684bc7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40086/0 (socket says 172.21.15.27:40086) 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f2400640 1 -- 172.21.15.27:0/3483136778 learned_addr learned my addr 172.21.15.27:0/3483136778 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f2400640 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bc27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f1000640 1 -- 172.21.15.27:0/3483136778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 msgr2=0x556684bc27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f1000640 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bc27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f1000640 1 -- 172.21.15.27:0/3483136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847e41b0 msgr2=0x556684bc7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f1000640 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847e41b0 0x556684bc7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f1000640 1 -- 172.21.15.27:0/3483136778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556684bce290 con 0x5566847ec390 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.189+0000 7fc3f2400640 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847e41b0 0x556684bc7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3f1000640 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847ec390 0x556684bc80a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8010890 tx=0x7fc3d8009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb8ea0aa137b9c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3d3400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3d3400640 1 -- 172.21.15.27:0/3483136778 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d802e060 con 0x5566847ec390 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e4036040 con 0x5566847ec390 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3d3400640 1 -- 172.21.15.27:0/3483136778 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc3d80103d0 con 0x5566847ec390 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3d3400640 1 -- 172.21.15.27:0/3483136778 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3d8035050 con 0x5566847ec390 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3fb043740 10 client.26432 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.190+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556684bb23a0 con 0x5566847ec390 2024-09-21T07:16:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3d3400640 1 -- 172.21.15.27:0/3483136778 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e4036040 con 0x5566847ec390 2024-09-21T07:16:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3d3400640 1 -- 172.21.15.27:0/3483136778 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc3d8028030 con 0x5566847ec390 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3fb043740 10 client.26432 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3fb043740 10 client.26432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3fb043740 20 client.26432 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3fb043740 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847a0da0 0x556684c3f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3fb043740 4 client.26432 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556684b289b0 con 0x5566847a0da0 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.191+0000 7fc3f2400640 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847a0da0 0x556684c3f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3f2400640 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847a0da0 0x556684c3f000 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be45095153bc366b server_cookie=b03f5a505d8a0ebf in_seq=0 out_seq=0 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3d3400640 10 client.26432 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3d3400640 1 -- 172.21.15.27:0/3483136778 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556684b289b0 con 0x5566847a0da0 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3d3400640 10 client.26432 handle_command_reply: tid=0 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 client.26432 shutdown 2024-09-21T07:16:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847ec390 msgr2=0x556684bc80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847ec390 0x556684bc80a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8010890 tx=0x7fc3d8009600 comp rx=0 tx=0).stop 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 shutdown_connections 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5566847a0da0 0x556684c3f000 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55668480d070 0x556684bc27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3d3400640 0 client.26432 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5566847ec390 0x556684bc80a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 --2- 172.21.15.27:0/3483136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566847e41b0 0x556684bc7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 >> 172.21.15.27:0/3483136778 conn(0x556684801460 msgr2=0x556684bc1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 shutdown_connections 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.192+0000 7fc3fb043740 1 -- 172.21.15.27:0/3483136778 wait complete. 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3fb043740 20 client.26432 trim_cache size 0 max 16384 2024-09-21T07:16:02.197 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:16:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3d40b6700 msgr2=0x7fc3d40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3d40b6700 0x7fc3d40b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556684bb2bf0 tx=0x7fc3e00eb880 comp rx=0 tx=0).stop 2024-09-21T07:16:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 msgr2=0x7fc3ec1a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 0x7fc3ec1a0950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000ea60 tx=0x7fc3e000ef20 comp rx=0 tx=0).stop 2024-09-21T07:16:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 shutdown_connections 2024-09-21T07:16:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ec114570 0x7fc3ec1a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3d40b6700 0x7fc3d40b8c10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ec078290 0x7fc3ec1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 --2- 172.21.15.27:0/2728939955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ec077940 0x7fc3ec1a0950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.193+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 >> 172.21.15.27:0/2728939955 conn(0x7fc3ec101680 msgr2=0x7fc3ec116580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.194+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 shutdown_connections 2024-09-21T07:16:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:02.194+0000 7fc3f2e00640 1 -- 172.21.15.27:0/2728939955 wait complete. 2024-09-21T07:16:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:16:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:16:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:16:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:02 smithi153 ceph-mon[34747]: pgmap v468: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-21T07:16:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:02 smithi119 ceph-mon[34519]: pgmap v468: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-21T07:16:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:02 smithi027 ceph-mon[30779]: pgmap v468: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 -- 172.21.15.27:0/1117697341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc077940 msgr2=0x7fe7bc077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/1117697341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc077940 0x7fe7bc077d50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009900 tx=0x7fe7ac02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 -- 172.21.15.27:0/1117697341 shutdown_connections 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/1117697341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc1144d0 0x7fe7bc1168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/1117697341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7bc078290 0x7fe7bc113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/1117697341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc077940 0x7fe7bc077d50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 -- 172.21.15.27:0/1117697341 >> 172.21.15.27:0/1117697341 conn(0x7fe7bc101560 msgr2=0x7fe7bc1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.987+0000 7fe7bbe00640 1 -- 172.21.15.27:0/1117697341 shutdown_connections 2024-09-21T07:16:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.988+0000 7fe7bbe00640 1 -- 172.21.15.27:0/1117697341 wait complete. 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 Processor -- start 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 -- start start 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc077940 0x7fe7bc1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc078290 0x7fe7bc1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7bc1144d0 0x7fe7bc1a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc119e80 con 0x7fe7bc078290 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc119d00 con 0x7fe7bc1144d0 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7bbe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc11a000 con 0x7fe7bc077940 2024-09-21T07:16:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.989+0000 7fe7b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc078290 0x7fe7bc1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc077940 0x7fe7bc1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc078290 0x7fe7bc1a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40108/0 (socket says 172.21.15.27:40108) 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2520286098 learned_addr learned my addr 172.21.15.27:0/2520286098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc077940 0x7fe7bc1a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45760/0 (socket says 172.21.15.27:45760) 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7bb400640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7bc1144d0 0x7fe7bc1a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2520286098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc077940 msgr2=0x7fe7bc1a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc077940 0x7fe7bc1a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2520286098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7bc1144d0 msgr2=0x7fe7bc1a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7bc1144d0 0x7fe7bc1a6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2520286098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc1a6810 con 0x7fe7bc078290 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc077940 0x7fe7bc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.990+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc078290 0x7fe7bc1a0e80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400e9f0 tx=0x7fe7a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.991+0000 7fe7b3400640 1 -- 172.21.15.27:0/2520286098 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400ccc0 con 0x7fe7bc078290 2024-09-21T07:16:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.991+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc1a6b00 con 0x7fe7bc078290 2024-09-21T07:16:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.991+0000 7fe7b3400640 1 -- 172.21.15.27:0/2520286098 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a40108e0 con 0x7fe7bc078290 2024-09-21T07:16:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.991+0000 7fe7b3400640 1 -- 172.21.15.27:0/2520286098 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4018820 con 0x7fe7bc078290 2024-09-21T07:16:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.991+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc1ad290 con 0x7fe7bc078290 2024-09-21T07:16:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 Processor -- start 2024-09-21T07:16:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 -- start start 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7b3400640 1 -- 172.21.15.27:0/2520286098 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe7a4010430 con 0x7fe7bc078290 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5647a782c6b0 con 0x5647a74e92f0 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5647a781bae0 con 0x5647a75121b0 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.994+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647a781f8f0 con 0x5647a74f14d0 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40110/0 (socket says 172.21.15.27:40110) 2024-09-21T07:16:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 -- 172.21.15.27:0/3379506428 learned_addr learned my addr 172.21.15.27:0/3379506428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 -- 172.21.15.27:0/3379506428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 msgr2=0x5647a78b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 -- 172.21.15.27:0/3379506428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 msgr2=0x5647a78b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 -- 172.21.15.27:0/3379506428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a77e8780 con 0x5647a74e92f0 2024-09-21T07:16:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7bb400640 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.995+0000 7fe7baa00640 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b7350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009230 tx=0x7fe7ac004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451c02ab4f7793ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.996+0000 7fe7abe00640 1 -- 172.21.15.27:0/3379506428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac035040 con 0x5647a74e92f0 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.996+0000 7fe7abe00640 1 -- 172.21.15.27:0/3379506428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac004770 con 0x5647a74e92f0 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.996+0000 7fe7b3400640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7a00b6680 0x7fe7a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.996+0000 7fe7abe00640 1 -- 172.21.15.27:0/3379506428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0039b0 con 0x5647a74e92f0 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.996+0000 7fe7b3400640 1 -- 172.21.15.27:0/2520286098 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe7a40e7de0 con 0x7fe7bc078290 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.996+0000 7fe7bb400640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7a00b6680 0x7fe7a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/3379506428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 msgr2=0x5647a78b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b7350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009230 tx=0x7fe7ac004470 comp rx=0 tx=0).stop 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/3379506428 shutdown_connections 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b7350 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/3379506428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/3379506428 >> 172.21.15.27:0/3379506428 conn(0x5647a7506300 msgr2=0x5647a78b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7bb400640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7a00b6680 0x7fe7a00b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4005fd0 tx=0x7fe7b4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/3379506428 shutdown_connections 2024-09-21T07:16:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.997+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/3379506428 wait complete. 2024-09-21T07:16:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 -- start start 2024-09-21T07:16:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 Processor -- start 2024-09-21T07:16:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74e92f0 0x5647a78c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74f14d0 0x5647a78c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5647a7753ab0 con 0x5647a74f14d0 2024-09-21T07:16:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5647a77c2570 con 0x5647a75121b0 2024-09-21T07:16:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.998+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647a78c5530 con 0x5647a74e92f0 2024-09-21T07:16:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7c8bcd740 10 client.0 _ll_register_callbacks cb 0x5647a77d1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74f14d0 0x5647a78c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74f14d0 0x5647a78c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40114/0 (socket says 172.21.15.27:40114) 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2463024223 learned_addr learned my addr 172.21.15.27:0/2463024223 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7bb400640 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74e92f0 0x5647a78c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2463024223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74e92f0 msgr2=0x5647a78c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74e92f0 0x5647a78c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 msgr2=0x5647a78c2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78c2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 -- 172.21.15.27:0/2463024223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a757d790 con 0x5647a74f14d0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74f14d0 0x5647a78c2920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40a9e50 tx=0x7fe7a40ad1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f180d5f8c07bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:03.999+0000 7fe7a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2463024223 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40e5610 con 0x5647a74f14d0 2024-09-21T07:16:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2463024223 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe7a40a91d0 con 0x5647a74f14d0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7ac004770 con 0x5647a74f14d0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2463024223 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7a4010720 con 0x5647a74f14d0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 10 client.16404 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647a75f6620 con 0x5647a74f14d0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2463024223 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac004770 con 0x5647a74f14d0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2463024223 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7a40a93d0 con 0x5647a74f14d0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 10 client.16404 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 20 client.16404 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74d0d80 0x5647a78d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 4 client.16404 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.000+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647a782e600 con 0x5647a74d0d80 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.001+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74d0d80 0x5647a78d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.001+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74d0d80 0x5647a78d44e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be41ab9499e0afa server_cookie=fdf843ec74897e82 in_seq=0 out_seq=0 2024-09-21T07:16:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.001+0000 7fe7a8c00640 10 client.16404 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.002+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2463024223 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5647a782e600 con 0x5647a74d0d80 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.002+0000 7fe7a8c00640 10 client.16404 handle_command_reply: tid=0 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.002+0000 7fe7c8bcd740 1 client.16404 shutdown 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74f14d0 msgr2=0x5647a78c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74f14d0 0x5647a78c2920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40a9e50 tx=0x7fe7a40ad1d0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 shutdown_connections 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74d0d80 0x5647a78d44e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78c2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74f14d0 0x5647a78c2920 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2463024223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74e92f0 0x5647a78c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7a8c00640 0 client.16404 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 >> 172.21.15.27:0/2463024223 conn(0x5647a7506300 msgr2=0x5647a78b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 shutdown_connections 2024-09-21T07:16:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2463024223 wait complete. 2024-09-21T07:16:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.003+0000 7fe7c8bcd740 20 client.16404 trim_cache size 0 max 16384 2024-09-21T07:16:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.018+0000 7fe7c8bcd740 1 Processor -- start 2024-09-21T07:16:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.018+0000 7fe7c8bcd740 1 -- start start 2024-09-21T07:16:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.018+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1b9a30 con 0x5647a74e92f0 2024-09-21T07:16:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1b7300 con 0x5647a75121b0 2024-09-21T07:16:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1b5150 con 0x5647a74f14d0 2024-09-21T07:16:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40130/0 (socket says 172.21.15.27:40130) 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 -- 172.21.15.27:0/2325457920 learned_addr learned my addr 172.21.15.27:0/2325457920 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7bb400640 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 -- 172.21.15.27:0/2325457920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 msgr2=0x5647a78bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78bf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 -- 172.21.15.27:0/2325457920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 msgr2=0x5647a78bfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78bfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.019+0000 7fe7baa00640 1 -- 172.21.15.27:0/2325457920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a78b9110 con 0x5647a74e92f0 2024-09-21T07:16:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.020+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b8550 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009900 tx=0x7fe7ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7c5f12fe9bff81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.020+0000 7fe7ab400640 1 -- 172.21.15.27:0/2325457920 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac05b070 con 0x5647a74e92f0 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.020+0000 7fe7ab400640 1 -- 172.21.15.27:0/2325457920 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac0055b0 con 0x5647a74e92f0 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.020+0000 7fe7ab400640 1 -- 172.21.15.27:0/2325457920 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac035040 con 0x5647a74e92f0 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2325457920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 msgr2=0x5647a78b8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b8550 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009900 tx=0x7fe7ac009260 comp rx=0 tx=0).stop 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2325457920 shutdown_connections 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78bf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a75121b0 0x5647a78bfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2325457920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a74e92f0 0x5647a78b8550 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2325457920 >> 172.21.15.27:0/2325457920 conn(0x5647a7506300 msgr2=0x5647a78b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2325457920 shutdown_connections 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2325457920 wait complete. 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.021+0000 7fe7c8bcd740 1 -- start start 2024-09-21T07:16:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 1 Processor -- start 2024-09-21T07:16:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a74e92f0 0x5647a78caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a75121b0 0x5647a78c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1b57c0 con 0x5647a75121b0 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1b6c30 con 0x5647a74e92f0 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1b5e90 con 0x5647a74f14d0 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7c8bcd740 10 client.0 _ll_register_callbacks cb 0x7fe7a40096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a74e92f0 0x5647a78caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a74e92f0 0x5647a78caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36576/0 (socket says 172.21.15.27:36576) 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7baa00640 1 -- 172.21.15.27:0/2870194748 learned_addr learned my addr 172.21.15.27:0/2870194748 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.022+0000 7fe7bb400640 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a75121b0 0x5647a78c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7baa00640 1 -- 172.21.15.27:0/2870194748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 msgr2=0x5647a78c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7baa00640 1 -- 172.21.15.27:0/2870194748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a75121b0 msgr2=0x5647a78c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a75121b0 0x5647a78c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7baa00640 1 -- 172.21.15.27:0/2870194748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a78c8fb0 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7baa00640 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a74e92f0 0x5647a78caec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac02a430 tx=0x7fe7ac005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728da05115fb8829 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2870194748 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac035070 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.023+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7ac0055b0 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.024+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2870194748 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe7ac005310 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.024+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2870194748 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7ac068050 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.024+0000 7fe7c8bcd740 10 client.26450 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.024+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647a793ed10 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.024+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2870194748 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac0055b0 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.024+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2870194748 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7ac031510 con 0x5647a74e92f0 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7c8bcd740 10 client.26450 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7c8bcd740 10 client.26450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7c8bcd740 20 client.26450 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74a61c0 0x5647a7940ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7c8bcd740 4 client.26450 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:16:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647a782a0b0 con 0x5647a74a61c0 2024-09-21T07:16:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74a61c0 0x5647a7940ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7a8c00640 10 client.26450 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.025+0000 7fe7b3e00640 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74a61c0 0x5647a7940ff0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270ed92ae36c3125 server_cookie=a6fcd2f9250d1f74 in_seq=0 out_seq=0 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.034+0000 7fe7a8c00640 1 -- 172.21.15.27:0/2870194748 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647a782a0b0 con 0x5647a74a61c0 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.034+0000 7fe7a8c00640 10 client.26450 handle_command_reply: tid=0 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.034+0000 7fe7c8bcd740 1 client.26450 shutdown 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a74e92f0 msgr2=0x5647a78caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a74e92f0 0x5647a78caec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac02a430 tx=0x7fe7ac005780 comp rx=0 tx=0).stop 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 shutdown_connections 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647a74a61c0 0x5647a7940ff0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a74f14d0 0x5647a78c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647a75121b0 0x5647a78c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 --2- 172.21.15.27:0/2870194748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647a74e92f0 0x5647a78caec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7a8c00640 0 client.26450 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 >> 172.21.15.27:0/2870194748 conn(0x5647a7506300 msgr2=0x5647a78b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 shutdown_connections 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 1 -- 172.21.15.27:0/2870194748 wait complete. 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.035+0000 7fe7c8bcd740 20 client.26450 trim_cache size 0 max 16384 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:16:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:16:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:16:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7a00b6680 msgr2=0x7fe7a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7a00b6680 0x7fe7a00b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4005fd0 tx=0x7fe7b4009500 comp rx=0 tx=0).stop 2024-09-21T07:16:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc078290 msgr2=0x7fe7bc1a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc078290 0x7fe7bc1a0e80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400e9f0 tx=0x7fe7a400edf0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 shutdown_connections 2024-09-21T07:16:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7a00b6680 0x7fe7a00b8b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7bc1144d0 0x7fe7bc1a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7bc078290 0x7fe7bc1a0e80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 --2- 172.21.15.27:0/2520286098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7bc077940 0x7fe7bc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 >> 172.21.15.27:0/2520286098 conn(0x7fe7bc101560 msgr2=0x7fe7bc116600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 shutdown_connections 2024-09-21T07:16:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:04.036+0000 7fe7bbe00640 1 -- 172.21.15.27:0/2520286098 wait complete. 2024-09-21T07:16:04.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:16:04.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:16:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.066+0000 7fa20b600640 1 -- 172.21.15.27:0/1976980060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 msgr2=0x7fa2041061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.066+0000 7fa20b600640 1 --2- 172.21.15.27:0/1976980060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 0x7fa2041061a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009900 tx=0x7fa1f402a410 comp rx=0 tx=0).stop 2024-09-21T07:16:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.066+0000 7fa20b600640 1 -- 172.21.15.27:0/1976980060 shutdown_connections 2024-09-21T07:16:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.066+0000 7fa20b600640 1 --2- 172.21.15.27:0/1976980060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 0x7fa20410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.066+0000 7fa20b600640 1 --2- 172.21.15.27:0/1976980060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa20410c020 0x7fa204112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.066+0000 7fa20b600640 1 --2- 172.21.15.27:0/1976980060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 0x7fa2041061a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.066+0000 7fa20b600640 1 -- 172.21.15.27:0/1976980060 >> 172.21.15.27:0/1976980060 conn(0x7fa204101860 msgr2=0x7fa204103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.067+0000 7fa20b600640 1 -- 172.21.15.27:0/1976980060 shutdown_connections 2024-09-21T07:16:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.067+0000 7fa20b600640 1 -- 172.21.15.27:0/1976980060 wait complete. 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 Processor -- start 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 -- start start 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 0x7fa20419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 0x7fa20419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa20410c020 0x7fa2041a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa204115ce0 con 0x7fa204105d90 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa204115b60 con 0x7fa20410c020 2024-09-21T07:16:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.068+0000 7fa20b600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa204115e60 con 0x7fa20410b6d0 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa208c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 0x7fa20419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 0x7fa20419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 0x7fa20419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45826/0 (socket says 172.21.15.27:45826) 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 -- 172.21.15.27:0/1436453918 learned_addr learned my addr 172.21.15.27:0/1436453918 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa209600640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa20410c020 0x7fa2041a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 -- 172.21.15.27:0/1436453918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa20410c020 msgr2=0x7fa2041a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa20410c020 0x7fa2041a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 -- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 msgr2=0x7fa20419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 0x7fa20419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.069+0000 7fa1fbe00640 1 -- 172.21.15.27:0/1436453918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2041a2600 con 0x7fa20410b6d0 2024-09-21T07:16:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.070+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 0x7fa20419cc30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9f0 tx=0x7fa1ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.070+0000 7fa202000640 1 -- 172.21.15.27:0/1436453918 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00ccc0 con 0x7fa20410b6d0 2024-09-21T07:16:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.070+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2041a28f0 con 0x7fa20410b6d0 2024-09-21T07:16:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.070+0000 7fa202000640 1 -- 172.21.15.27:0/1436453918 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ec0108e0 con 0x7fa20410b6d0 2024-09-21T07:16:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.070+0000 7fa202000640 1 -- 172.21.15.27:0/1436453918 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec018860 con 0x7fa20410b6d0 2024-09-21T07:16:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.070+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2041a9050 con 0x7fa20410b6d0 2024-09-21T07:16:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.073+0000 7fa21227a740 1 Processor -- start 2024-09-21T07:16:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.073+0000 7fa21227a740 1 -- start start 2024-09-21T07:16:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa202000640 1 -- 172.21.15.27:0/1436453918 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa1ec010430 con 0x7fa20410b6d0 2024-09-21T07:16:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa21227a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa21227a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781a6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa21227a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa21227a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55767811be90 con 0x557677dfe350 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa21227a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55767810b2c0 con 0x557677ddd670 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa21227a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55767810f0d0 con 0x557677dd5490 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa209600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa209600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40172/0 (socket says 172.21.15.27:40172) 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa209600640 1 -- 172.21.15.27:0/2580917946 learned_addr learned my addr 172.21.15.27:0/2580917946 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa208c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.074+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.075+0000 7fa209600640 1 -- 172.21.15.27:0/2580917946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 msgr2=0x5576781a6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.075+0000 7fa209600640 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781a6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.075+0000 7fa209600640 1 -- 172.21.15.27:0/2580917946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 msgr2=0x5576781a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.075+0000 7fa209600640 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.075+0000 7fa209600640 1 -- 172.21.15.27:0/2580917946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557677e69930 con 0x557677dfe350 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.075+0000 7fa209600640 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781a5d10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaeb31dfc2858c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.075+0000 7fa1f9c00640 1 -- 172.21.15.27:0/2580917946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc004d20 con 0x557677dfe350 2024-09-21T07:16:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.076+0000 7fa1f9c00640 1 -- 172.21.15.27:0/2580917946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1fc004050 con 0x557677dfe350 2024-09-21T07:16:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.076+0000 7fa1f9c00640 1 -- 172.21.15.27:0/2580917946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc005280 con 0x557677dfe350 2024-09-21T07:16:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.076+0000 7fa202000640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1e80b6680 0x7fa1e80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.076+0000 7fa202000640 1 -- 172.21.15.27:0/1436453918 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa1ec0e7d50 con 0x7fa20410b6d0 2024-09-21T07:16:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.076+0000 7fa208c00640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1e80b6680 0x7fa1e80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 -- 172.21.15.27:0/2580917946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 msgr2=0x5576781a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781a5d10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc0118d0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 -- 172.21.15.27:0/2580917946 shutdown_connections 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 --2- 172.21.15.27:0/2580917946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781a5d10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 -- 172.21.15.27:0/2580917946 >> 172.21.15.27:0/2580917946 conn(0x557677df2740 msgr2=0x5576781a3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 -- 172.21.15.27:0/2580917946 shutdown_connections 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa21227a740 1 -- 172.21.15.27:0/2580917946 wait complete. 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.077+0000 7fa208c00640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1e80b6680 0x7fa1e80b8b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4031440 tx=0x7fa1f4009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 -- start start 2024-09-21T07:16:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 Processor -- start 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677ddd670 0x5576781b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x5576781bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557677f6c890 con 0x557677ddd670 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5576780a6160 con 0x557677dfe350 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.078+0000 7fa21227a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5576781b44f0 con 0x557677dd5490 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa21227a740 10 client.0 _ll_register_callbacks cb 0x5576780c2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa208c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa208c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45840/0 (socket says 172.21.15.27:45840) 2024-09-21T07:16:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa208c00640 1 -- 172.21.15.27:0/1792321692 learned_addr learned my addr 172.21.15.27:0/1792321692 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677ddd670 0x5576781b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa209600640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x5576781bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa209600640 1 -- 172.21.15.27:0/1792321692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 msgr2=0x5576781b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa209600640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa209600640 1 -- 172.21.15.27:0/1792321692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677ddd670 msgr2=0x5576781b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa209600640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677ddd670 0x5576781b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa209600640 1 -- 172.21.15.27:0/1792321692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576781bdd20 con 0x557677dfe350 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.079+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677ddd670 0x5576781b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.080+0000 7fa209600640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x5576781bd7e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc01b040 tx=0x7fa1fc004850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0a045eb122a052 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.080+0000 7fa1f2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.080+0000 7fa1f2000640 1 -- 172.21.15.27:0/1792321692 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc028020 con 0x557677dfe350 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.080+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc004050 con 0x557677dfe350 2024-09-21T07:16:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.080+0000 7fa1f2000640 1 -- 172.21.15.27:0/1792321692 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa1fc004510 con 0x557677dfe350 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.080+0000 7fa1f2000640 1 -- 172.21.15.27:0/1792321692 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc033050 con 0x557677dfe350 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.080+0000 7fa21227a740 10 client.26462 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576781c0650 con 0x557677dfe350 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa1f2000640 1 -- 172.21.15.27:0/1792321692 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1fc004050 con 0x557677dfe350 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa1f2000640 1 -- 172.21.15.27:0/1792321692 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa1fc005760 con 0x557677dfe350 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa21227a740 10 client.26462 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa21227a740 10 client.26462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa21227a740 20 client.26462 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa21227a740 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677dbccd0 0x5576781c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa21227a740 4 client.26462 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.081+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55767811dde0 con 0x557677dbccd0 2024-09-21T07:16:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.082+0000 7fa208c00640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677dbccd0 0x5576781c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.082+0000 7fa208c00640 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677dbccd0 0x5576781c3a70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67dc36fcb5cccd44 server_cookie=1b1911a729482e5e in_seq=0 out_seq=0 2024-09-21T07:16:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.082+0000 7fa1f2000640 10 client.26462 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa1f2000640 1 -- 172.21.15.27:0/1792321692 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55767811dde0 con 0x557677dbccd0 2024-09-21T07:16:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa1f2000640 10 client.26462 handle_command_reply: tid=0 2024-09-21T07:16:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 client.26462 shutdown 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 msgr2=0x5576781bd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x5576781bd7e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc01b040 tx=0x7fa1fc004850 comp rx=0 tx=0).stop 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 shutdown_connections 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677dbccd0 0x5576781c3a70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x5576781bd7e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677ddd670 0x5576781b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 --2- 172.21.15.27:0/1792321692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.083+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 >> 172.21.15.27:0/1792321692 conn(0x557677df2740 msgr2=0x5576781a2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.084+0000 7fa1f2000640 0 client.26462 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.084+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 shutdown_connections 2024-09-21T07:16:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.084+0000 7fa21227a740 1 -- 172.21.15.27:0/1792321692 wait complete. 2024-09-21T07:16:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.084+0000 7fa21227a740 20 client.26462 trim_cache size 0 max 16384 2024-09-21T07:16:05.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.103+0000 7fa21227a740 1 Processor -- start 2024-09-21T07:16:05.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.103+0000 7fa21227a740 1 -- start start 2024-09-21T07:16:05.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:04 smithi027 ceph-mon[30779]: pgmap v469: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-09-21T07:16:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.103+0000 7fa21227a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa21227a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa21227a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa21227a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b58b0 con 0x557677dfe350 2024-09-21T07:16:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa21227a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b3180 con 0x557677ddd670 2024-09-21T07:16:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa21227a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b0fd0 con 0x557677dd5490 2024-09-21T07:16:05.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45842/0 (socket says 172.21.15.27:45842) 2024-09-21T07:16:05.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 -- 172.21.15.27:0/58358252 learned_addr learned my addr 172.21.15.27:0/58358252 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:05.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa209600640 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 -- 172.21.15.27:0/58358252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 msgr2=0x5576781b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 -- 172.21.15.27:0/58358252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 msgr2=0x5576781b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.104+0000 7fa208c00640 1 -- 172.21.15.27:0/58358252 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576781b4020 con 0x557677dd5490 2024-09-21T07:16:05.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.105+0000 7fa208c00640 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4006010 tx=0x7fa1f403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af97d91275fbf827 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:05.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.105+0000 7fa1f9200640 1 -- 172.21.15.27:0/58358252 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4035040 con 0x557677dd5490 2024-09-21T07:16:05.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.105+0000 7fa1f9200640 1 -- 172.21.15.27:0/58358252 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1f4056030 con 0x557677dd5490 2024-09-21T07:16:05.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.105+0000 7fa1f9200640 1 -- 172.21.15.27:0/58358252 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4004c50 con 0x557677dd5490 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 -- 172.21.15.27:0/58358252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 msgr2=0x5576781b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4006010 tx=0x7fa1f403f000 comp rx=0 tx=0).stop 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 -- 172.21.15.27:0/58358252 shutdown_connections 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dfe350 0x5576781b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677ddd670 0x5576781b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 --2- 172.21.15.27:0/58358252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677dd5490 0x5576781b1ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 -- 172.21.15.27:0/58358252 >> 172.21.15.27:0/58358252 conn(0x557677df2740 msgr2=0x5576781b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 -- 172.21.15.27:0/58358252 shutdown_connections 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.106+0000 7fa21227a740 1 -- 172.21.15.27:0/58358252 wait complete. 2024-09-21T07:16:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.107+0000 7fa21227a740 1 -- start start 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.107+0000 7fa21227a740 1 Processor -- start 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa21227a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dd5490 0x5576781b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa21227a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677ddd670 0x5576781b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa21227a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x557678226380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa21227a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b1640 con 0x557677dd5490 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa21227a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b2ab0 con 0x557677dfe350 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa21227a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b1d10 con 0x557677ddd670 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa21227a740 10 client.0 _ll_register_callbacks cb 0x7fa1ec0eb6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa1fbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677ddd670 0x5576781b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa1fbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677ddd670 0x5576781b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45854/0 (socket says 172.21.15.27:45854) 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa208c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dd5490 0x5576781b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa209600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x557678226380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.108+0000 7fa1fbe00640 1 -- 172.21.15.27:0/2157686440 learned_addr learned my addr 172.21.15.27:0/2157686440 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa209600640 1 -- 172.21.15.27:0/2157686440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677ddd670 msgr2=0x5576781b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa209600640 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677ddd670 0x5576781b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa209600640 1 -- 172.21.15.27:0/2157686440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dd5490 msgr2=0x5576781b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa209600640 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dd5490 0x5576781b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa209600640 1 -- 172.21.15.27:0/2157686440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576782269f0 con 0x557677dfe350 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa1fbe00640 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677ddd670 0x5576781b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa209600640 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x557678226380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00f6a0 tx=0x7fa1fc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eba85523425dda6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa1f2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa1f2000640 1 -- 172.21.15.27:0/2157686440 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc03c060 con 0x557677dfe350 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f4056030 con 0x557677dfe350 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.109+0000 7fa1f2000640 1 -- 172.21.15.27:0/2157686440 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa1fc00f720 con 0x557677dfe350 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa1f2000640 1 -- 172.21.15.27:0/2157686440 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc043050 con 0x557677dfe350 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 10 client.26468 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55767822de20 con 0x557677dfe350 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa1f2000640 1 -- 172.21.15.27:0/2157686440 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1f4056030 con 0x557677dfe350 2024-09-21T07:16:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa1f2000640 1 -- 172.21.15.27:0/2157686440 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa1fc005040 con 0x557677dfe350 2024-09-21T07:16:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 10 client.26468 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 10 client.26468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 20 client.26468 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677d92080 0x557678230430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 4 client.26468 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:16:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.110+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557678119890 con 0x557677d92080 2024-09-21T07:16:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.111+0000 7fa208c00640 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677d92080 0x557678230430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.111+0000 7fa208c00640 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677d92080 0x557678230430 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b078614b1c9842e2 server_cookie=4e806fc96a55c3de in_seq=0 out_seq=0 2024-09-21T07:16:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.111+0000 7fa1f2000640 10 client.26468 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.113+0000 7fa1f2000640 1 -- 172.21.15.27:0/2157686440 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557678119890 con 0x557677d92080 2024-09-21T07:16:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.113+0000 7fa1f2000640 10 client.26468 handle_command_reply: tid=0 2024-09-21T07:16:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.113+0000 7fa21227a740 1 client.26468 shutdown 2024-09-21T07:16:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 msgr2=0x557678226380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x557678226380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00f6a0 tx=0x7fa1fc0047c0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 shutdown_connections 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557677d92080 0x557678230430 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557677ddd670 0x5576781b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557677dfe350 0x557678226380 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 --2- 172.21.15.27:0/2157686440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557677dd5490 0x5576781b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa1f2000640 0 client.26468 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 >> 172.21.15.27:0/2157686440 conn(0x557677df2740 msgr2=0x5576781a3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 shutdown_connections 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 1 -- 172.21.15.27:0/2157686440 wait complete. 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.114+0000 7fa21227a740 20 client.26468 trim_cache size 0 max 16384 2024-09-21T07:16:05.117 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:16:05.118 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:16:05.118 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "93e8af8f-b26b-4c09-ad37-673f408ca863", 2024-09-21T07:16:05.118 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:16:05.118 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:16:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1e80b6680 msgr2=0x7fa1e80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1e80b6680 0x7fa1e80b8b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4031440 tx=0x7fa1f4009560 comp rx=0 tx=0).stop 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 msgr2=0x7fa20419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 0x7fa20419cc30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9f0 tx=0x7fa1ec00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 shutdown_connections 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa20410b6d0 0x7fa20419cc30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa1e80b6680 0x7fa1e80b8b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa20410c020 0x7fa2041a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 --2- 172.21.15.27:0/1436453918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa204105d90 0x7fa20419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 >> 172.21.15.27:0/1436453918 conn(0x7fa204101860 msgr2=0x7fa204103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.115+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 shutdown_connections 2024-09-21T07:16:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:05.116+0000 7fa20b600640 1 -- 172.21.15.27:0/1436453918 wait complete. 2024-09-21T07:16:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:04 smithi153 ceph-mon[34747]: pgmap v469: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-09-21T07:16:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:04 smithi119 ceph-mon[34519]: pgmap v469: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 23 op/s 2024-09-21T07:16:05.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:16:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:05 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:16:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:05 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:05 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:16:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:05 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:05 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:16:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:05 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:05 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:16:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:05 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:06.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:05 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:16:06.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:05 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:06.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:05 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:16:06.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:05 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:06.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.897+0000 7fb136600640 1 -- 172.21.15.27:0/2743698054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1301039b0 msgr2=0x7fb130103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.897+0000 7fb136600640 1 --2- 172.21.15.27:0/2743698054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1301039b0 0x7fb130103dc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb124009900 tx=0x7fb12402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.898+0000 7fb136600640 1 -- 172.21.15.27:0/2743698054 shutdown_connections 2024-09-21T07:16:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.898+0000 7fb136600640 1 --2- 172.21.15.27:0/2743698054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb130109c40 0x7fb130110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.898+0000 7fb136600640 1 --2- 172.21.15.27:0/2743698054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1301092f0 0x7fb130109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.898+0000 7fb136600640 1 --2- 172.21.15.27:0/2743698054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1301039b0 0x7fb130103dc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.898+0000 7fb136600640 1 -- 172.21.15.27:0/2743698054 >> 172.21.15.27:0/2743698054 conn(0x7fb1300ff400 msgr2=0x7fb130101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.898+0000 7fb136600640 1 -- 172.21.15.27:0/2743698054 shutdown_connections 2024-09-21T07:16:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.898+0000 7fb136600640 1 -- 172.21.15.27:0/2743698054 wait complete. 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.899+0000 7fb136600640 1 Processor -- start 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.899+0000 7fb136600640 1 -- start start 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.899+0000 7fb136600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1301039b0 0x7fb13019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.899+0000 7fb136600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1301092f0 0x7fb13019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.899+0000 7fb136600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb130109c40 0x7fb1301a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb136600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb130113950 con 0x7fb130109c40 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb136600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301137d0 con 0x7fb1301039b0 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb136600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb130113ad0 con 0x7fb1301092f0 2024-09-21T07:16:06.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1301039b0 0x7fb13019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1301039b0 0x7fb13019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36660/0 (socket says 172.21.15.27:36660) 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb130109c40 0x7fb1301a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135200640 1 -- 172.21.15.27:0/137403231 learned_addr learned my addr 172.21.15.27:0/137403231 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb12fe00640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1301092f0 0x7fb13019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135c00640 1 -- 172.21.15.27:0/137403231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1301092f0 msgr2=0x7fb13019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135c00640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1301092f0 0x7fb13019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135c00640 1 -- 172.21.15.27:0/137403231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1301039b0 msgr2=0x7fb13019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135c00640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1301039b0 0x7fb13019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.900+0000 7fb135c00640 1 -- 172.21.15.27:0/137403231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1301a2470 con 0x7fb130109c40 2024-09-21T07:16:06.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.901+0000 7fb135c00640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb130109c40 0x7fb1301a1d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb128009fd0 tx=0x7fb12800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:06.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.901+0000 7fb12d600640 1 -- 172.21.15.27:0/137403231 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12800ce20 con 0x7fb130109c40 2024-09-21T07:16:06.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.901+0000 7fb12d600640 1 -- 172.21.15.27:0/137403231 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb128005020 con 0x7fb130109c40 2024-09-21T07:16:06.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.901+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1301a2760 con 0x7fb130109c40 2024-09-21T07:16:06.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.901+0000 7fb12d600640 1 -- 172.21.15.27:0/137403231 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128010690 con 0x7fb130109c40 2024-09-21T07:16:06.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.902+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a8ec0 con 0x7fb130109c40 2024-09-21T07:16:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.905+0000 7fb13e81f740 1 Processor -- start 2024-09-21T07:16:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb12d600640 1 -- 172.21.15.27:0/137403231 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb1280054d0 con 0x7fb130109c40 2024-09-21T07:16:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb13e81f740 1 -- start start 2024-09-21T07:16:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab06bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab06c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab06c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb13e81f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564aafe2f70 con 0x5564aac9c1b0 2024-09-21T07:16:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb13e81f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5564aafd23a0 con 0x5564aacc5070 2024-09-21T07:16:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb13e81f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5564aafd61b0 con 0x5564aaca4390 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.906+0000 7fb135200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab06c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab06bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab06c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40222/0 (socket says 172.21.15.27:40222) 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab06bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36674/0 (socket says 172.21.15.27:36674) 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 -- 172.21.15.27:0/699972224 learned_addr learned my addr 172.21.15.27:0/699972224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb12fe00640 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab06c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 -- 172.21.15.27:0/699972224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 msgr2=0x5564ab06c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab06c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 -- 172.21.15.27:0/699972224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 msgr2=0x5564ab06bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab06bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 -- 172.21.15.27:0/699972224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564aaf9cc90 con 0x5564aac9c1b0 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb12fe00640 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab06c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb135200640 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab06c470 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb124004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22a69be34629575 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.907+0000 7fb12d600640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb1180b67d0 0x7fb1180b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.908+0000 7fb12d600640 1 -- 172.21.15.27:0/137403231 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb1280e80b0 con 0x7fb130109c40 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.908+0000 7fb122000640 1 -- 172.21.15.27:0/699972224 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124040070 con 0x5564aac9c1b0 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.908+0000 7fb12fe00640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb1180b67d0 0x7fb1180b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.908+0000 7fb122000640 1 -- 172.21.15.27:0/699972224 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1240054d0 con 0x5564aac9c1b0 2024-09-21T07:16:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.908+0000 7fb122000640 1 -- 172.21.15.27:0/699972224 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124035040 con 0x5564aac9c1b0 2024-09-21T07:16:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 -- 172.21.15.27:0/699972224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 msgr2=0x5564ab06c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab06c470 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb124004100 comp rx=0 tx=0).stop 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb12fe00640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb1180b67d0 0x7fb1180b8ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097e0 tx=0x7fb11c009770 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 -- 172.21.15.27:0/699972224 shutdown_connections 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab06c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab06c470 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 --2- 172.21.15.27:0/699972224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 -- 172.21.15.27:0/699972224 >> 172.21.15.27:0/699972224 conn(0x5564aacb9460 msgr2=0x5564ab069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 -- 172.21.15.27:0/699972224 shutdown_connections 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.909+0000 7fb13e81f740 1 -- 172.21.15.27:0/699972224 wait complete. 2024-09-21T07:16:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 -- start start 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 Processor -- start 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab0776c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab077c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab0839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564aae33700 con 0x5564aac9c1b0 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5564aaf6d540 con 0x5564aacc5070 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.910+0000 7fb13e81f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5564ab07a6b0 con 0x5564aaca4390 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb13e81f740 10 client.0 _ll_register_callbacks cb 0x5564aaf7d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb135200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab0776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb135200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab0776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40224/0 (socket says 172.21.15.27:40224) 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab077c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45882/0 (socket says 172.21.15.27:45882) 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb135200640 1 -- 172.21.15.27:0/712714027 learned_addr learned my addr 172.21.15.27:0/712714027 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb135c00640 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab0839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb12fe00640 1 -- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 msgr2=0x5564ab0839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb12fe00640 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab0839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb12fe00640 1 -- 172.21.15.27:0/712714027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 msgr2=0x5564ab0776c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb12fe00640 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab0776c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb12fe00640 1 -- 172.21.15.27:0/712714027 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564aad30650 con 0x5564aaca4390 2024-09-21T07:16:06.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.911+0000 7fb135200640 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab0776c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.912+0000 7fb12fe00640 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab077c00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb13006b700 tx=0x7fb11c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5c0c1aa0d127da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.912+0000 7fb116a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.912+0000 7fb116a00640 1 -- 172.21.15.27:0/712714027 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c022640 con 0x5564aaca4390 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.912+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564aaf6fb50 con 0x5564aaca4390 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.912+0000 7fb116a00640 1 -- 172.21.15.27:0/712714027 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb11c0227e0 con 0x5564aaca4390 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.912+0000 7fb116a00640 1 -- 172.21.15.27:0/712714027 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb11c0229e0 con 0x5564aaca4390 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.912+0000 7fb13e81f740 10 client.26464 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564aaf8a910 con 0x5564aaca4390 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb116a00640 1 -- 172.21.15.27:0/712714027 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5564aaf6fb50 con 0x5564aaca4390 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb116a00640 1 -- 172.21.15.27:0/712714027 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb11c02bbf0 con 0x5564aaca4390 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb13e81f740 10 client.26464 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb13e81f740 10 client.26464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb13e81f740 20 client.26464 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb13e81f740 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac839f0 0x5564ab0897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb13e81f740 4 client.26464 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.913+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564aafe4ec0 con 0x5564aac839f0 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.914+0000 7fb135200640 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac839f0 0x5564ab0897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.914+0000 7fb135200640 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac839f0 0x5564ab0897f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31142150fcdeca0e server_cookie=cae976eb1f5a018e in_seq=0 out_seq=0 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.914+0000 7fb116a00640 10 client.26464 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb116a00640 1 -- 172.21.15.27:0/712714027 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5564aafe4ec0 con 0x5564aac839f0 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb116a00640 10 client.26464 handle_command_reply: tid=0 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 client.26464 shutdown 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 msgr2=0x5564ab077c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab077c00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb13006b700 tx=0x7fb11c004730 comp rx=0 tx=0).stop 2024-09-21T07:16:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 shutdown_connections 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac839f0 0x5564ab0897f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aaca4390 0x5564ab077c00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb116a00640 0 client.26464 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab0839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 --2- 172.21.15.27:0/712714027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aac9c1b0 0x5564ab0776c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.915+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 >> 172.21.15.27:0/712714027 conn(0x5564aacb9460 msgr2=0x5564ab06f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.916+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 shutdown_connections 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.916+0000 7fb13e81f740 1 -- 172.21.15.27:0/712714027 wait complete. 2024-09-21T07:16:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.916+0000 7fb13e81f740 20 client.26464 trim_cache size 0 max 16384 2024-09-21T07:16:06.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 Processor -- start 2024-09-21T07:16:06.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 -- start start 2024-09-21T07:16:06.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab0765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aaca4390 0x5564ab06d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aacc5070 0x5564ab0792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b5610 con 0x5564aacc5070 2024-09-21T07:16:06.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b2ee0 con 0x5564aaca4390 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.934+0000 7fb13e81f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b0d30 con 0x5564aac9c1b0 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab0765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aacc5070 0x5564ab0792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aaca4390 0x5564ab06d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab0765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45898/0 (socket says 172.21.15.27:45898) 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 -- 172.21.15.27:0/2799713306 learned_addr learned my addr 172.21.15.27:0/2799713306 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 -- 172.21.15.27:0/2799713306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aaca4390 msgr2=0x5564ab06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 --2- 172.21.15.27:0/2799713306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aaca4390 0x5564ab06d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 -- 172.21.15.27:0/2799713306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aacc5070 msgr2=0x5564ab0792f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 --2- 172.21.15.27:0/2799713306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aacc5070 0x5564ab0792f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.935+0000 7fb135200640 1 -- 172.21.15.27:0/2799713306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ab07a5f0 con 0x5564aac9c1b0 2024-09-21T07:16:06.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.936+0000 7fb135200640 1 --2- 172.21.15.27:0/2799713306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab0765b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb124004c80 tx=0x7fb124004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c4e42c35adc7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:06.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.936+0000 7fb121600640 1 -- 172.21.15.27:0/2799713306 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124040070 con 0x5564aac9c1b0 2024-09-21T07:16:06.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.936+0000 7fb121600640 1 -- 172.21.15.27:0/2799713306 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb124005290 con 0x5564aac9c1b0 2024-09-21T07:16:06.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.936+0000 7fb121600640 1 -- 172.21.15.27:0/2799713306 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124035040 con 0x5564aac9c1b0 2024-09-21T07:16:06.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 -- 172.21.15.27:0/2799713306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 msgr2=0x5564ab0765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2799713306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab0765b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb124004c80 tx=0x7fb124004cb0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 -- 172.21.15.27:0/2799713306 shutdown_connections 2024-09-21T07:16:06.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2799713306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aacc5070 0x5564ab0792f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2799713306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aaca4390 0x5564ab06d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2799713306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab0765b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 -- 172.21.15.27:0/2799713306 >> 172.21.15.27:0/2799713306 conn(0x5564aacb9460 msgr2=0x5564ab078be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 -- 172.21.15.27:0/2799713306 shutdown_connections 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.937+0000 7fb13e81f740 1 -- 172.21.15.27:0/2799713306 wait complete. 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 -- start start 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 Processor -- start 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab07fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aaca4390 0x5564ab0802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab07aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b13a0 con 0x5564aaca4390 2024-09-21T07:16:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b2810 con 0x5564aacc5070 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b1a70 con 0x5564aac9c1b0 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.938+0000 7fb13e81f740 10 client.0 _ll_register_callbacks cb 0x7fb11c004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab07fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aaca4390 0x5564ab0802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab07fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:45910/0 (socket says 172.21.15.27:45910) 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 -- 172.21.15.27:0/2839388083 learned_addr learned my addr 172.21.15.27:0/2839388083 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135c00640 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab07aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 -- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 msgr2=0x5564ab07aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab07aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 -- 172.21.15.27:0/2839388083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aaca4390 msgr2=0x5564ab0802d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aaca4390 0x5564ab0802d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.939+0000 7fb135200640 1 -- 172.21.15.27:0/2839388083 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ab07b1c0 con 0x5564aac9c1b0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb135200640 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab07fd10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb124005ec0 tx=0x7fb124004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee6714972958ff53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb116a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb116a00640 1 -- 172.21.15.27:0/2839388083 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12405b070 con 0x5564aac9c1b0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb116a00640 1 -- 172.21.15.27:0/2839388083 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fb124005440 con 0x5564aac9c1b0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124005290 con 0x5564aac9c1b0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb116a00640 1 -- 172.21.15.27:0/2839388083 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb124040070 con 0x5564aac9c1b0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb13e81f740 10 client.26470 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564ab0f4d00 con 0x5564aac9c1b0 2024-09-21T07:16:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.940+0000 7fb116a00640 1 -- 172.21.15.27:0/2839388083 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb124005290 con 0x5564aac9c1b0 2024-09-21T07:16:06.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb116a00640 1 -- 172.21.15.27:0/2839388083 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb1240315c0 con 0x5564aac9c1b0 2024-09-21T07:16:06.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb13e81f740 10 client.26470 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:06.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb13e81f740 10 client.26470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:06.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb13e81f740 20 client.26470 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:06.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac58da0 0x5564ab0f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb13e81f740 4 client.26470 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:16:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564ab073270 con 0x5564aac58da0 2024-09-21T07:16:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.941+0000 7fb12fe00640 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac58da0 0x5564ab0f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.942+0000 7fb12fe00640 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac58da0 0x5564ab0f6fe0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f3bcd234145fcb server_cookie=fab15667367dc3b4 in_seq=0 out_seq=0 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.942+0000 7fb116a00640 10 client.26470 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb116a00640 1 -- 172.21.15.27:0/2839388083 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x5564ab073270 con 0x5564aac58da0 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb116a00640 10 client.26470 handle_command_reply: tid=0 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 client.26470 shutdown 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 msgr2=0x5564ab07fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab07fd10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb124005ec0 tx=0x7fb124004c10 comp rx=0 tx=0).stop 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 shutdown_connections 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5564aac58da0 0x5564ab0f6fe0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564aacc5070 0x5564ab07aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564aaca4390 0x5564ab0802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 --2- 172.21.15.27:0/2839388083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564aac9c1b0 0x5564ab07fd10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb116a00640 0 client.26470 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 >> 172.21.15.27:0/2839388083 conn(0x5564aacb9460 msgr2=0x5564ab079480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 shutdown_connections 2024-09-21T07:16:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 1 -- 172.21.15.27:0/2839388083 wait complete. 2024-09-21T07:16:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.943+0000 7fb13e81f740 20 client.26470 trim_cache size 0 max 16384 2024-09-21T07:16:06.951 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "93e8af8f-b26b-4c09-ad37-673f408ca863", 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:16:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:16:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:06.956 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:16:06.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb1180b67d0 msgr2=0x7fb1180b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb1180b67d0 0x7fb1180b8ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097e0 tx=0x7fb11c009770 comp rx=0 tx=0).stop 2024-09-21T07:16:06.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb130109c40 msgr2=0x7fb1301a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:06.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb130109c40 0x7fb1301a1d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb128009fd0 tx=0x7fb12800ef10 comp rx=0 tx=0).stop 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 shutdown_connections 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1301092f0 0x7fb13019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb1180b67d0 0x7fb1180b8ce0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb130109c40 0x7fb1301a1d40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 --2- 172.21.15.27:0/137403231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1301039b0 0x7fb13019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 >> 172.21.15.27:0/137403231 conn(0x7fb1300ff400 msgr2=0x7fb13010e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 shutdown_connections 2024-09-21T07:16:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:06.944+0000 7fb136600640 1 -- 172.21.15.27:0/137403231 wait complete. 2024-09-21T07:16:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:06 smithi153 ceph-mon[34747]: pgmap v470: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 24 op/s 2024-09-21T07:16:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:06 smithi119 ceph-mon[34519]: pgmap v470: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 24 op/s 2024-09-21T07:16:07.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:06 smithi027 ceph-mon[30779]: pgmap v470: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 24 op/s 2024-09-21T07:16:07.750 INFO:tasks.cephfs.filesystem:scrub status for tag:93e8af8f-b26b-4c09-ad37-673f408ca863 - {'path': '/', 'tag': '93e8af8f-b26b-4c09-ad37-673f408ca863', 'options': 'recursive,force'} 2024-09-21T07:16:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:08 smithi153 ceph-mon[34747]: pgmap v471: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-09-21T07:16:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:08 smithi119 ceph-mon[34519]: pgmap v471: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-09-21T07:16:09.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:08 smithi027 ceph-mon[30779]: pgmap v471: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-09-21T07:16:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:10 smithi153 ceph-mon[34747]: pgmap v472: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-09-21T07:16:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:10 smithi119 ceph-mon[34519]: pgmap v472: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-09-21T07:16:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:10 smithi027 ceph-mon[30779]: pgmap v472: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-09-21T07:16:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:12 smithi153 ceph-mon[34747]: pgmap v473: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-21T07:16:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:12 smithi119 ceph-mon[34519]: pgmap v473: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-21T07:16:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:12 smithi027 ceph-mon[30779]: pgmap v473: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-21T07:16:14.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:16:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:14 smithi027 ceph-mon[30779]: pgmap v474: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-09-21T07:16:15.103 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:14 smithi153 ceph-mon[34747]: pgmap v474: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-09-21T07:16:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:14 smithi119 ceph-mon[34519]: pgmap v474: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-09-21T07:16:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 -- 172.21.15.27:0/856648753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0114740 msgr2=0x7f33d0116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 --2- 172.21.15.27:0/856648753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0114740 0x7f33d0116b60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009ca0 tx=0x7f33c802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:16:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 -- 172.21.15.27:0/856648753 shutdown_connections 2024-09-21T07:16:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 --2- 172.21.15.27:0/856648753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d0078290 0x7f33d0114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 --2- 172.21.15.27:0/856648753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0114740 0x7f33d0116b60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 --2- 172.21.15.27:0/856648753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d0077940 0x7f33d0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:15.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 -- 172.21.15.27:0/856648753 >> 172.21.15.27:0/856648753 conn(0x7f33d01017d0 msgr2=0x7f33d0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:15.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.743+0000 7f33d5e00640 1 -- 172.21.15.27:0/856648753 shutdown_connections 2024-09-21T07:16:15.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.744+0000 7f33d5e00640 1 -- 172.21.15.27:0/856648753 wait complete. 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 Processor -- start 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 -- start start 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0077940 0x7f33d01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d0078290 0x7f33d01a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d0114740 0x7f33d01a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f33d011a100 con 0x7f33d0077940 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0119f80 con 0x7f33d0114740 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.745+0000 7f33d5e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33d011a280 con 0x7f33d0078290 2024-09-21T07:16:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0077940 0x7f33d01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33c7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d0078290 0x7f33d01a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0077940 0x7f33d01a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55874/0 (socket says 172.21.15.27:55874) 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 -- 172.21.15.27:0/2663978866 learned_addr learned my addr 172.21.15.27:0/2663978866 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cf400640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d0114740 0x7f33d01a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 -- 172.21.15.27:0/2663978866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d0078290 msgr2=0x7f33d01a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d0078290 0x7f33d01a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 -- 172.21.15.27:0/2663978866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d0114740 msgr2=0x7f33d01a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d0114740 0x7f33d01a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.746+0000 7f33cea00640 1 -- 172.21.15.27:0/2663978866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d01a6a80 con 0x7f33d0077940 2024-09-21T07:16:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.747+0000 7f33cea00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0077940 0x7f33d01a0b10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f33c000e9f0 tx=0x7f33c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.747+0000 7f33d4a00640 1 -- 172.21.15.27:0/2663978866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000cd80 con 0x7f33d0077940 2024-09-21T07:16:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.747+0000 7f33d4a00640 1 -- 172.21.15.27:0/2663978866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c00108e0 con 0x7f33d0077940 2024-09-21T07:16:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.747+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d01a6d70 con 0x7f33d0077940 2024-09-21T07:16:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.747+0000 7f33d4a00640 1 -- 172.21.15.27:0/2663978866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0018980 con 0x7f33d0077940 2024-09-21T07:16:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.747+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d01ad4d0 con 0x7f33d0077940 2024-09-21T07:16:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.748+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d0077d50 con 0x7f33d0077940 2024-09-21T07:16:15.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.751+0000 7f33d4a00640 1 -- 172.21.15.27:0/2663978866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f33c0010430 con 0x7f33d0077940 2024-09-21T07:16:15.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.752+0000 7f33d4a00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33b40b6680 0x7f33b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:15.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.752+0000 7f33d4a00640 1 -- 172.21.15.27:0/2663978866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f33c00e8280 con 0x7f33d0077940 2024-09-21T07:16:15.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.752+0000 7f33c7e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33b40b6680 0x7f33b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:15.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.753+0000 7f33c7e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33b40b6680 0x7f33b40b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33d007b2a0 tx=0x7f33b800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:15.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.754+0000 7f33d4a00640 1 -- 172.21.15.27:0/2663978866 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33c00b2800 con 0x7f33d0077940 2024-09-21T07:16:15.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.994+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33d002e640 con 0x7f33d0077940 2024-09-21T07:16:15.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.995+0000 7f33d4a00640 1 -- 172.21.15.27:0/2663978866 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f33c00b7740 con 0x7f33d0077940 2024-09-21T07:16:15.997 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:16:15.997 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:16:15.997 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.997+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33b40b6680 msgr2=0x7f33b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.997+0000 7f33d5e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33b40b6680 0x7f33b40b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33d007b2a0 tx=0x7f33b800b410 comp rx=0 tx=0).stop 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0077940 msgr2=0x7f33d01a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0077940 0x7f33d01a0b10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f33c000e9f0 tx=0x7f33c000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 shutdown_connections 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d0078290 0x7f33d01a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f33b40b6680 0x7f33b40b8b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33d0114740 0x7f33d01a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 --2- 172.21.15.27:0/2663978866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d0077940 0x7f33d01a0b10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 >> 172.21.15.27:0/2663978866 conn(0x7f33d01017d0 msgr2=0x7f33d0116780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 shutdown_connections 2024-09-21T07:16:16.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:15.998+0000 7f33d5e00640 1 -- 172.21.15.27:0/2663978866 wait complete. 2024-09-21T07:16:16.872 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 526297b8 --release --await' 2024-09-21T07:16:16.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 526297b8 --release --await 2024-09-21T07:16:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:16 smithi027 ceph-mon[30779]: pgmap v475: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-21T07:16:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:16 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2663978866' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:16:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:16 smithi153 ceph-mon[34747]: pgmap v475: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-21T07:16:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:16 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2663978866' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.241+0000 7f0ac7800640 1 -- 172.21.15.27:0/294266546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0105270 msgr2=0x7f0ac010d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.241+0000 7f0ac7800640 1 --2- 172.21.15.27:0/294266546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0105270 0x7f0ac010d7b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009900 tx=0x7f0aac02a410 comp rx=0 tx=0).stop 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.242+0000 7f0ac7800640 1 -- 172.21.15.27:0/294266546 shutdown_connections 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.242+0000 7f0ac7800640 1 --2- 172.21.15.27:0/294266546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac010dd70 0x7f0ac0110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.242+0000 7f0ac7800640 1 --2- 172.21.15.27:0/294266546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0105270 0x7f0ac010d7b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.242+0000 7f0ac7800640 1 --2- 172.21.15.27:0/294266546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac0104920 0x7f0ac0104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.242+0000 7f0ac7800640 1 -- 172.21.15.27:0/294266546 >> 172.21.15.27:0/294266546 conn(0x7f0ac007a6f0 msgr2=0x7f0ac007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.242+0000 7f0ac7800640 1 -- 172.21.15.27:0/294266546 shutdown_connections 2024-09-21T07:16:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.242+0000 7f0ac7800640 1 -- 172.21.15.27:0/294266546 wait complete. 2024-09-21T07:16:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 Processor -- start 2024-09-21T07:16:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 -- start start 2024-09-21T07:16:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0104920 0x7f0ac019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac0105270 0x7f0ac019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac010dd70 0x7f0ac019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01139e0 con 0x7f0ac0104920 2024-09-21T07:16:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0113860 con 0x7f0ac010dd70 2024-09-21T07:16:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.243+0000 7f0ac7800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0113b60 con 0x7f0ac0105270 2024-09-21T07:16:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac010dd70 0x7f0ac019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0104920 0x7f0ac019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac010dd70 0x7f0ac019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47322/0 (socket says 172.21.15.27:47322) 2024-09-21T07:16:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac5800640 1 -- 172.21.15.27:0/1600941215 learned_addr learned my addr 172.21.15.27:0/1600941215 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0abfe00640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac0105270 0x7f0ac019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac4e00640 1 -- 172.21.15.27:0/1600941215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac0105270 msgr2=0x7f0ac019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac0105270 0x7f0ac019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac4e00640 1 -- 172.21.15.27:0/1600941215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac010dd70 msgr2=0x7f0ac019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac010dd70 0x7f0ac019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac4e00640 1 -- 172.21.15.27:0/1600941215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac01a0310 con 0x7f0ac0104920 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.244+0000 7f0ac5800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac010dd70 0x7f0ac019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.245+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0104920 0x7f0ac019a3e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400e9f0 tx=0x7f0ab400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.245+0000 7f0abd600640 1 -- 172.21.15.27:0/1600941215 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400ccc0 con 0x7f0ac0104920 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.245+0000 7f0abd600640 1 -- 172.21.15.27:0/1600941215 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab40108e0 con 0x7f0ac0104920 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.245+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac01a0600 con 0x7f0ac0104920 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.245+0000 7f0abd600640 1 -- 172.21.15.27:0/1600941215 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4018880 con 0x7f0ac0104920 2024-09-21T07:16:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.245+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac0102530 con 0x7f0ac0104920 2024-09-21T07:16:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 Processor -- start 2024-09-21T07:16:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 -- start start 2024-09-21T07:16:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d4e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 0x55865d4e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55865d45b6b0 con 0x55865d1204d0 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55865d44aae0 con 0x55865d1411b0 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.249+0000 7f0ace844740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55865d44e8f0 con 0x55865d1182f0 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d4e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d4e5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47334/0 (socket says 172.21.15.27:47334) 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0abfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 -- 172.21.15.27:0/3517824583 learned_addr learned my addr 172.21.15.27:0/3517824583 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 -- 172.21.15.27:0/3517824583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 msgr2=0x55865d4e6170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:16:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 --2- 172.21.15.27:0/3517824583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 0x55865d4e6170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 -- 172.21.15.27:0/3517824583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 msgr2=0x55865d4e66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 --2- 172.21.15.27:0/3517824583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4e66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0ac5800640 1 -- 172.21.15.27:0/3517824583 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865d417780 con 0x55865d1411b0 2024-09-21T07:16:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.250+0000 7f0abd600640 1 -- 172.21.15.27:0/1600941215 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0ab4010430 con 0x7f0ac0104920 2024-09-21T07:16:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.251+0000 7f0ac5800640 1 --2- 172.21.15.27:0/3517824583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d4e5c30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8011810 tx=0x7f0ab8011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31b2a102c631d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.251+0000 7f0ab2000640 1 -- 172.21.15.27:0/3517824583 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800fb40 con 0x55865d1411b0 2024-09-21T07:16:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.251+0000 7f0ab2000640 1 -- 172.21.15.27:0/3517824583 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab800fce0 con 0x55865d1411b0 2024-09-21T07:16:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.251+0000 7f0ab2000640 1 -- 172.21.15.27:0/3517824583 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80126b0 con 0x55865d1411b0 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 -- 172.21.15.27:0/3517824583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 msgr2=0x55865d4e5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 --2- 172.21.15.27:0/3517824583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d4e5c30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8011810 tx=0x7f0ab8011cd0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 -- 172.21.15.27:0/3517824583 shutdown_connections 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0abd600640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0aa80b6680 0x7f0aa80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 --2- 172.21.15.27:0/3517824583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 0x55865d4e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 --2- 172.21.15.27:0/3517824583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 --2- 172.21.15.27:0/3517824583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d4e5c30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 -- 172.21.15.27:0/3517824583 >> 172.21.15.27:0/3517824583 conn(0x55865d135300 msgr2=0x55865d4e3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0abd600640 1 -- 172.21.15.27:0/1600941215 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0ab40e7d70 con 0x7f0ac0104920 2024-09-21T07:16:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 -- 172.21.15.27:0/3517824583 shutdown_connections 2024-09-21T07:16:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ac5800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0aa80b6680 0x7f0aa80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.252+0000 7f0ace844740 1 -- 172.21.15.27:0/3517824583 wait complete. 2024-09-21T07:16:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.253+0000 7f0ac5800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0aa80b6680 0x7f0aa80b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80094a0 tx=0x7f0ab8000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.253+0000 7f0ace844740 1 -- start start 2024-09-21T07:16:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.254+0000 7f0ace844740 1 Processor -- start 2024-09-21T07:16:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.254+0000 7f0ace844740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.254+0000 7f0ace844740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1204d0 0x55865d4f1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.254+0000 7f0ace844740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1411b0 0x55865d4fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.254+0000 7f0ace844740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55865d382ab0 con 0x55865d1411b0 2024-09-21T07:16:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.254+0000 7f0ace844740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55865d3f1570 con 0x55865d1182f0 2024-09-21T07:16:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.254+0000 7f0ace844740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55865d4f43b0 con 0x55865d1204d0 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ace844740 10 client.0 _ll_register_callbacks cb 0x55865d400100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1411b0 0x55865d4fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1411b0 0x55865d4fd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55918/0 (socket says 172.21.15.27:55918) 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 -- 172.21.15.27:0/421996517 learned_addr learned my addr 172.21.15.27:0/421996517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0abfe00640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1204d0 0x55865d4f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 -- 172.21.15.27:0/421996517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1204d0 msgr2=0x55865d4f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1204d0 0x55865d4f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 -- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 msgr2=0x55865d4f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac5800640 1 -- 172.21.15.27:0/421996517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865d1ac790 con 0x55865d1411b0 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.255+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0ac5800640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1411b0 0x55865d4fd6d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0ab801a000 tx=0x7f0ab801bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bd70377b318aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0aa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0aa6a00640 1 -- 172.21.15.27:0/421996517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8036060 con 0x55865d1411b0 2024-09-21T07:16:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab800fce0 con 0x55865d1411b0 2024-09-21T07:16:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0aa6a00640 1 -- 172.21.15.27:0/421996517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0ab8005450 con 0x55865d1411b0 2024-09-21T07:16:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0aa6a00640 1 -- 172.21.15.27:0/421996517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ab803d050 con 0x55865d1411b0 2024-09-21T07:16:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0ace844740 10 client.16464 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.256+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55865d225620 con 0x55865d1411b0 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0aa6a00640 1 -- 172.21.15.27:0/421996517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab800fce0 con 0x55865d1411b0 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0aa6a00640 1 -- 172.21.15.27:0/421996517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ab800c070 con 0x55865d1411b0 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0ace844740 10 client.16464 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0ace844740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0ace844740 20 client.16464 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0ace844740 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0ffd80 0x55865d5032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0ace844740 4 client.16464 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865d45d600 con 0x55865d0ffd80 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.257+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0ffd80 0x55865d5032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.258+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0ffd80 0x55865d5032e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55225bea30f8ef36 server_cookie=8bbb3b44b3758a2c in_seq=0 out_seq=0 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.258+0000 7f0aa6a00640 10 client.16464 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0aa6a00640 1 -- 172.21.15.27:0/421996517 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55865d45d600 con 0x55865d0ffd80 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0aa6a00640 10 client.16464 handle_command_reply: tid=0 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 client.16464 shutdown 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1411b0 msgr2=0x55865d4fd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1411b0 0x55865d4fd6d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0ab801a000 tx=0x7f0ab801bd90 comp rx=0 tx=0).stop 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 shutdown_connections 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0ffd80 0x55865d5032e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1204d0 0x55865d4f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1411b0 0x55865d4fd6d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 --2- 172.21.15.27:0/421996517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.259+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 >> 172.21.15.27:0/421996517 conn(0x55865d135300 msgr2=0x55865d4e6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.260+0000 7f0aa6a00640 0 client.16464 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.260+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 shutdown_connections 2024-09-21T07:16:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.260+0000 7f0ace844740 1 -- 172.21.15.27:0/421996517 wait complete. 2024-09-21T07:16:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.260+0000 7f0ace844740 20 client.16464 trim_cache size 0 max 16384 2024-09-21T07:16:17.278 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.279+0000 7f0ace844740 1 Processor -- start 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ace844740 1 -- start start 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ace844740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ace844740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ace844740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1411b0 0x55865d4fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ace844740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01b58d0 con 0x55865d1204d0 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ace844740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01b31a0 con 0x55865d1182f0 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ace844740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01b0ff0 con 0x55865d1411b0 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0abfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0abfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55934/0 (socket says 172.21.15.27:55934) 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0abfe00640 1 -- 172.21.15.27:0/1464060238 learned_addr learned my addr 172.21.15.27:0/1464060238 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.280+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0ac5800640 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1411b0 0x55865d4fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0abfe00640 1 -- 172.21.15.27:0/1464060238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1411b0 msgr2=0x55865d4fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0abfe00640 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1411b0 0x55865d4fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0abfe00640 1 -- 172.21.15.27:0/1464060238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 msgr2=0x55865d4f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0abfe00640 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0abfe00640 1 -- 172.21.15.27:0/1464060238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865d0fffb0 con 0x55865d1204d0 2024-09-21T07:16:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:16 smithi119 ceph-mon[34519]: pgmap v475: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-21T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:16 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2663978866' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:16:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0abfe00640 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f2f10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0aac005ab0 tx=0x7f0aac004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f29bbbbbd4d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.281+0000 7f0ab1600640 1 -- 172.21.15.27:0/1464060238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac035040 con 0x55865d1204d0 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.282+0000 7f0ab1600640 1 -- 172.21.15.27:0/1464060238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aac004750 con 0x55865d1204d0 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.282+0000 7f0ab1600640 1 -- 172.21.15.27:0/1464060238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003850 con 0x55865d1204d0 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.282+0000 7f0ace844740 1 -- 172.21.15.27:0/1464060238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 msgr2=0x55865d4f2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.282+0000 7f0ace844740 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f2f10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0aac005ab0 tx=0x7f0aac004350 comp rx=0 tx=0).stop 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 -- 172.21.15.27:0/1464060238 shutdown_connections 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1411b0 0x55865d4fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f2f10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 --2- 172.21.15.27:0/1464060238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1182f0 0x55865d4f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 -- 172.21.15.27:0/1464060238 >> 172.21.15.27:0/1464060238 conn(0x55865d135300 msgr2=0x55865d4f2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 -- 172.21.15.27:0/1464060238 shutdown_connections 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 -- 172.21.15.27:0/1464060238 wait complete. 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 -- start start 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.283+0000 7f0ace844740 1 Processor -- start 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ace844740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 0x55865d4f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ace844740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ace844740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d56ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ace844740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01b1660 con 0x55865d1204d0 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ace844740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01b2ad0 con 0x55865d1411b0 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ace844740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01b1d30 con 0x55865d1182f0 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ace844740 10 client.0 _ll_register_callbacks cb 0x7f0aac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0abfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0abfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55938/0 (socket says 172.21.15.27:55938) 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0abfe00640 1 -- 172.21.15.27:0/3345113322 learned_addr learned my addr 172.21.15.27:0/3345113322 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 0x55865d4f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.284+0000 7f0ac5800640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d56ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0abfe00640 1 -- 172.21.15.27:0/3345113322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 msgr2=0x55865d4f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0abfe00640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 0x55865d4f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0abfe00640 1 -- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 msgr2=0x55865d56ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0abfe00640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d56ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0abfe00640 1 -- 172.21.15.27:0/3345113322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865d4f5780 con 0x55865d1204d0 2024-09-21T07:16:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0ac5800640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d56ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0abfe00640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f4f70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009850 tx=0x7f0aac004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aa79531f5f421d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0aa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0aa6a00640 1 -- 172.21.15.27:0/3345113322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac035030 con 0x55865d1204d0 2024-09-21T07:16:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aac004750 con 0x55865d1204d0 2024-09-21T07:16:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.285+0000 7f0aa6a00640 1 -- 172.21.15.27:0/3345113322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0aac005460 con 0x55865d1204d0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0aa6a00640 1 -- 172.21.15.27:0/3345113322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0aac059050 con 0x55865d1204d0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 10 client.16470 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55865d56cf80 con 0x55865d1204d0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0aa6a00640 1 -- 172.21.15.27:0/3345113322 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aac004750 con 0x55865d1204d0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0aa6a00640 1 -- 172.21.15.27:0/3345113322 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0aac03d450 con 0x55865d1204d0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 10 client.16470 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 20 client.16470 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0d51c0 0x55865d56f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 4 client.16470 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.286+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55865d4590b0 con 0x55865d0d51c0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.287+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0d51c0 0x55865d56f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.287+0000 7f0ac4e00640 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0d51c0 0x55865d56f680 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4530cecff9f185b2 server_cookie=47968edbbca0edfa in_seq=0 out_seq=0 2024-09-21T07:16:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.287+0000 7f0aa6a00640 10 client.16470 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.293+0000 7f0aa6a00640 1 -- 172.21.15.27:0/3345113322 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55865d4590b0 con 0x55865d0d51c0 2024-09-21T07:16:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.293+0000 7f0aa6a00640 10 client.16470 handle_command_reply: tid=0 2024-09-21T07:16:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.293+0000 7f0ace844740 1 client.16470 shutdown 2024-09-21T07:16:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 msgr2=0x55865d4f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f4f70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009850 tx=0x7f0aac004bb0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 shutdown_connections 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55865d0d51c0 0x55865d56f680 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55865d1411b0 0x55865d56ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0aa6a00640 0 client.16470 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55865d1204d0 0x55865d4f4f70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 --2- 172.21.15.27:0/3345113322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55865d1182f0 0x55865d4f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 >> 172.21.15.27:0/3345113322 conn(0x55865d135300 msgr2=0x55865d4e72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 shutdown_connections 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 1 -- 172.21.15.27:0/3345113322 wait complete. 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.294+0000 7f0ace844740 20 client.16470 trim_cache size 0 max 16384 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:16:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:16:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:16:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:16:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0aa80b6680 msgr2=0x7f0aa80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0aa80b6680 0x7f0aa80b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80094a0 tx=0x7f0ab8000f40 comp rx=0 tx=0).stop 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0104920 msgr2=0x7f0ac019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0104920 0x7f0ac019a3e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400e9f0 tx=0x7f0ab400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 shutdown_connections 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ac0105270 0x7f0ac019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0aa80b6680 0x7f0aa80b8b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac010dd70 0x7f0ac019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 --2- 172.21.15.27:0/1600941215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ac0104920 0x7f0ac019a3e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 >> 172.21.15.27:0/1600941215 conn(0x7f0ac007a6f0 msgr2=0x7f0ac0100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.295+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 shutdown_connections 2024-09-21T07:16:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:17.296+0000 7f0ac7800640 1 -- 172.21.15.27:0/1600941215 wait complete. 2024-09-21T07:16:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 -- 172.21.15.27:0/1429269489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f810b6c0 msgr2=0x7ff1f810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 --2- 172.21.15.27:0/1429269489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f810b6c0 0x7ff1f810bad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009900 tx=0x7ff1e002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 -- 172.21.15.27:0/1429269489 shutdown_connections 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 --2- 172.21.15.27:0/1429269489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1f810c010 0x7ff1f8112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 --2- 172.21.15.27:0/1429269489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f810b6c0 0x7ff1f810bad0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 --2- 172.21.15.27:0/1429269489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f8105d80 0x7ff1f8106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 -- 172.21.15.27:0/1429269489 >> 172.21.15.27:0/1429269489 conn(0x7ff1f8101810 msgr2=0x7ff1f8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 -- 172.21.15.27:0/1429269489 shutdown_connections 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.957+0000 7ff1fd800640 1 -- 172.21.15.27:0/1429269489 wait complete. 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.958+0000 7ff1fd800640 1 Processor -- start 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1fd800640 1 -- start start 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1fd800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8105d80 0x7ff1f819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1fd800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1f810b6c0 0x7ff1f819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1fd800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f810c010 0x7ff1f81a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1fd800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8115cd0 con 0x7ff1f8105d80 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1fd800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8115b50 con 0x7ff1f810b6c0 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1fd800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8115e50 con 0x7ff1f810c010 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8105d80 0x7ff1f819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8105d80 0x7ff1f819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53264/0 (socket says 172.21.15.27:53264) 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2541165249 learned_addr learned my addr 172.21.15.27:0/2541165249 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1f7400640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f810c010 0x7ff1f81a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.959+0000 7ff1ee000640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1f810b6c0 0x7ff1f819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2541165249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f810c010 msgr2=0x7ff1f81a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f810c010 0x7ff1f81a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2541165249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1f810b6c0 msgr2=0x7ff1f819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1f810b6c0 0x7ff1f819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2541165249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81a2650 con 0x7ff1f8105d80 2024-09-21T07:16:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8105d80 0x7ff1f819c6e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1efe00640 1 -- 172.21.15.27:0/2541165249 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800cd80 con 0x7ff1f8105d80 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1efe00640 1 -- 172.21.15.27:0/2541165249 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e80054d0 con 0x7ff1f8105d80 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a2940 con 0x7ff1f8105d80 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.960+0000 7ff1efe00640 1 -- 172.21.15.27:0/2541165249 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8010690 con 0x7ff1f8105d80 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.961+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a90a0 con 0x7ff1f8105d80 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.964+0000 7ff1efe00640 1 -- 172.21.15.27:0/2541165249 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff1e8004e70 con 0x7ff1f8105d80 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 Processor -- start 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 -- start start 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50663520 0x561d50a0a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d5063a660 0x561d50a0a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561d50981290 con 0x561d50663520 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561d509706c0 con 0x561d5063a660 2024-09-21T07:16:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff204604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d509744d0 con 0x561d50642840 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff1f7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50663520 0x561d50a0a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.965+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a0ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35464/0 (socket says 172.21.15.27:35464) 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1ee000640 1 -- 172.21.15.27:0/312820763 learned_addr learned my addr 172.21.15.27:0/312820763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d5063a660 0x561d50a0a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1ee000640 1 -- 172.21.15.27:0/312820763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d5063a660 msgr2=0x561d50a0a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1ee000640 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d5063a660 0x561d50a0a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1ee000640 1 -- 172.21.15.27:0/312820763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50663520 msgr2=0x561d50a0a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1ee000640 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50663520 0x561d50a0a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1ee000640 1 -- 172.21.15.27:0/312820763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d508fd460 con 0x561d50642840 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1f7400640 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50663520 0x561d50a0a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1efe00640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1dc0b6680 0x7ff1dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.966+0000 7ff1efe00640 1 -- 172.21.15.27:0/2541165249 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff1e80e7e90 con 0x7ff1f8105d80 2024-09-21T07:16:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.967+0000 7ff1ee000640 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a0ac40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009900 tx=0x7ff1e0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7378a90621c6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.967+0000 7ff1f7400640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1dc0b6680 0x7ff1dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.967+0000 7ff1e7e00640 1 -- 172.21.15.27:0/312820763 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0040070 con 0x561d50642840 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.967+0000 7ff1e7e00640 1 -- 172.21.15.27:0/312820763 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e0004030 con 0x561d50642840 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.967+0000 7ff1e7e00640 1 -- 172.21.15.27:0/312820763 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0035040 con 0x561d50642840 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 -- 172.21.15.27:0/312820763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 msgr2=0x561d50a0ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff1f7400640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1dc0b6680 0x7ff1dc0b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009fd0 tx=0x7ff1f0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a0ac40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009900 tx=0x7ff1e0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 -- 172.21.15.27:0/312820763 shutdown_connections 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a0ac40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d5063a660 0x561d50a0a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 --2- 172.21.15.27:0/312820763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50663520 0x561d50a0a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 -- 172.21.15.27:0/312820763 >> 172.21.15.27:0/312820763 conn(0x561d50657910 msgr2=0x561d50a080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 -- 172.21.15.27:0/312820763 shutdown_connections 2024-09-21T07:16:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.968+0000 7ff204604740 1 -- 172.21.15.27:0/312820763 wait complete. 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.969+0000 7ff204604740 1 -- start start 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.969+0000 7ff204604740 1 Processor -- start 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff204604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a15de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff204604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a16320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff204604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff204604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561d507d1bb0 con 0x561d50642840 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff204604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561d5090b6c0 con 0x561d50663520 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff204604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d50a18940 con 0x561d5063a660 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff204604740 10 client.0 _ll_register_callbacks cb 0x561d50926490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a16320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a15de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a16320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53280/0 (socket says 172.21.15.27:53280) 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 -- 172.21.15.27:0/4042163804 learned_addr learned my addr 172.21.15.27:0/4042163804 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1f7400640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 -- 172.21.15.27:0/4042163804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 msgr2=0x561d50a15de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a15de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 -- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 msgr2=0x561d50a21c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a21c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1ee000640 1 -- 172.21.15.27:0/4042163804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d506ceb00 con 0x561d50642840 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1f7400640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a21c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.970+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a15de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff1ee000640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a16320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0032f80 tx=0x7ff1e0005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa6096f2ad532e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff1e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff1e4c00640 1 -- 172.21.15.27:0/4042163804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e004c020 con 0x561d50642840 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e0004030 con 0x561d50642840 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff1e4c00640 1 -- 172.21.15.27:0/4042163804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff1e00316a0 con 0x561d50642840 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff1e4c00640 1 -- 172.21.15.27:0/4042163804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1e0057050 con 0x561d50642840 2024-09-21T07:16:18.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff204604740 10 client.16482 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.971+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d50745960 con 0x561d50642840 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff1e4c00640 1 -- 172.21.15.27:0/4042163804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e0004030 con 0x561d50642840 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff1e4c00640 1 -- 172.21.15.27:0/4042163804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff1e0004cf0 con 0x561d50642840 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff204604740 10 client.16482 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff204604740 10 client.16482 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff204604740 20 client.16482 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff204604740 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d50621ea0 0x561d50a27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff204604740 4 client.16482 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.972+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d509831e0 con 0x561d50621ea0 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.973+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d50621ea0 0x561d50a27af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.973+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d50621ea0 0x561d50a27af0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1601c064c1f5a46c server_cookie=7d0a10d04b855646 in_seq=0 out_seq=0 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.973+0000 7ff1e4c00640 10 client.16482 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff1e4c00640 1 -- 172.21.15.27:0/4042163804 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d509831e0 con 0x561d50621ea0 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff1e4c00640 10 client.16482 handle_command_reply: tid=0 2024-09-21T07:16:18.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 client.16482 shutdown 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 msgr2=0x561d50a16320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a16320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0032f80 tx=0x7ff1e0005e20 comp rx=0 tx=0).stop 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 shutdown_connections 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d50621ea0 0x561d50a27af0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a21c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a16320 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 --2- 172.21.15.27:0/4042163804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a15de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff1e4c00640 0 client.16482 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 >> 172.21.15.27:0/4042163804 conn(0x561d50657910 msgr2=0x561d50a0d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.974+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 shutdown_connections 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.975+0000 7ff204604740 1 -- 172.21.15.27:0/4042163804 wait complete. 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.975+0000 7ff204604740 20 client.16482 trim_cache size 0 max 16384 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 Processor -- start 2024-09-21T07:16:18.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 -- start start 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a07660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a07ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b5740 con 0x561d50642840 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b3010 con 0x561d50663520 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff204604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b0e60 con 0x561d5063a660 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.993+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a07660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a07660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53282/0 (socket says 172.21.15.27:53282) 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 -- 172.21.15.27:0/900348153 learned_addr learned my addr 172.21.15.27:0/900348153 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1f7400640 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a07ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 -- 172.21.15.27:0/900348153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 msgr2=0x561d50a0b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a0b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 -- 172.21.15.27:0/900348153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 msgr2=0x561d50a07ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a07ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 -- 172.21.15.27:0/900348153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d50a0c4b0 con 0x561d50642840 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1ee000640 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a07660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff1e002a910 tx=0x7ff1e0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e451e9ff6a6edd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.994+0000 7ff1e7400640 1 -- 172.21.15.27:0/900348153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e004c040 con 0x561d50642840 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff1e7400640 1 -- 172.21.15.27:0/900348153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00039b0 con 0x561d50642840 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff1e7400640 1 -- 172.21.15.27:0/900348153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0040070 con 0x561d50642840 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff204604740 1 -- 172.21.15.27:0/900348153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 msgr2=0x561d50a07660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff204604740 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a07660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff1e002a910 tx=0x7ff1e0001e70 comp rx=0 tx=0).stop 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff204604740 1 -- 172.21.15.27:0/900348153 shutdown_connections 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff204604740 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a07ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff204604740 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d50642840 0x561d50a07660 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff204604740 1 --2- 172.21.15.27:0/900348153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d5063a660 0x561d50a0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.995+0000 7ff204604740 1 -- 172.21.15.27:0/900348153 >> 172.21.15.27:0/900348153 conn(0x561d50657910 msgr2=0x561d50a0b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.996+0000 7ff204604740 1 -- 172.21.15.27:0/900348153 shutdown_connections 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.996+0000 7ff204604740 1 -- 172.21.15.27:0/900348153 wait complete. 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.996+0000 7ff204604740 1 -- start start 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.996+0000 7ff204604740 1 Processor -- start 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff204604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d5063a660 0x561d50a28cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff204604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a29230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff204604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff204604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b14d0 con 0x561d5063a660 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff204604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b2940 con 0x561d50663520 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff204604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81b1ba0 con 0x561d50642840 2024-09-21T07:16:18.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff204604740 10 client.0 _ll_register_callbacks cb 0x7ff1f0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d5063a660 0x561d50a28cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d5063a660 0x561d50a28cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53290/0 (socket says 172.21.15.27:53290) 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff1f6a00640 1 -- 172.21.15.27:0/1935287017 learned_addr learned my addr 172.21.15.27:0/1935287017 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff1ee000640 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a29230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.997+0000 7ff1f7400640 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1f6a00640 1 -- 172.21.15.27:0/1935287017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 msgr2=0x561d50a29230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a29230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1f6a00640 1 -- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 msgr2=0x561d50a29770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a29770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1f6a00640 1 -- 172.21.15.27:0/1935287017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d50a91470 con 0x561d5063a660 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d5063a660 0x561d50a28cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80ad690 tx=0x7ff1e8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10674e141ad12042 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:18.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1e4c00640 1 -- 172.21.15.27:0/1935287017 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80ad100 con 0x561d5063a660 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff1e4c00640 1 -- 172.21.15.27:0/1935287017 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff1e8005160 con 0x561d5063a660 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.998+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e00039b0 con 0x561d5063a660 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff1e4c00640 1 -- 172.21.15.27:0/1935287017 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1e80a9100 con 0x561d5063a660 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 10 client.16488 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d50a92710 con 0x561d5063a660 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff1e4c00640 1 -- 172.21.15.27:0/1935287017 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00039b0 con 0x561d5063a660 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff1e4c00640 1 -- 172.21.15.27:0/1935287017 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff1e80acbf0 con 0x561d5063a660 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 10 client.16488 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 10 client.16488 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 20 client.16488 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d505f7250 0x561d50a94ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:18.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 4 client.16488 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "526297b8", "release": true, "await": true}] 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:17.999+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "526297b8", "release": true, "await": true}) -- 0x561d5097ec90 con 0x561d505f7250 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.000+0000 7ff1ee000640 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d505f7250 0x561d50a94ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.000+0000 7ff1e4c00640 10 client.16488 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.000+0000 7ff1ee000640 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d505f7250 0x561d50a94ed0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e866e919d710d2b server_cookie=462600b5395a21df in_seq=0 out_seq=0 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff1e4c00640 1 -- 172.21.15.27:0/1935287017 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561d5097ec90 con 0x561d505f7250 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff1e4c00640 10 client.16488 handle_command_reply: tid=0 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 client.16488 shutdown 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d5063a660 msgr2=0x561d50a28cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d5063a660 0x561d50a28cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80ad690 tx=0x7ff1e8004a90 comp rx=0 tx=0).stop 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 shutdown_connections 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d505f7250 0x561d50a94ed0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d50642840 0x561d50a29230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d50663520 0x561d50a29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 --2- 172.21.15.27:0/1935287017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d5063a660 0x561d50a28cf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff1e4c00640 0 client.16488 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.004+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 >> 172.21.15.27:0/1935287017 conn(0x561d50657910 msgr2=0x561d50a17090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.005+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 shutdown_connections 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.005+0000 7ff204604740 1 -- 172.21.15.27:0/1935287017 wait complete. 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.005+0000 7ff204604740 20 client.16488 trim_cache size 0 max 16384 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 16, 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:16:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "526297b8": { 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 16, 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:16:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.005+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1dc0b6680 msgr2=0x7ff1dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.005+0000 7ff1fd800640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1dc0b6680 0x7ff1dc0b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009fd0 tx=0x7ff1f0009500 comp rx=0 tx=0).stop 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.005+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8105d80 msgr2=0x7ff1f819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.005+0000 7ff1fd800640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8105d80 0x7ff1f819c6e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800edf0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 shutdown_connections 2024-09-21T07:16:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1f810c010 0x7ff1f81a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1dc0b6680 0x7ff1dc0b8b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1f810b6c0 0x7ff1f819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 --2- 172.21.15.27:0/2541165249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8105d80 0x7ff1f819c6e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:18.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 >> 172.21.15.27:0/2541165249 conn(0x7ff1f8101810 msgr2=0x7ff1f8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:18.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 shutdown_connections 2024-09-21T07:16:18.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:18.006+0000 7ff1fd800640 1 -- 172.21.15.27:0/2541165249 wait complete. 2024-09-21T07:16:18.910 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '526297b8', total seconds elapsed: 32.0 2024-09-21T07:16:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:18 smithi153 ceph-mon[34747]: pgmap v476: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T07:16:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:18 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:16:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:18 smithi119 ceph-mon[34519]: pgmap v476: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T07:16:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:18 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:16:19.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:18 smithi027 ceph-mon[30779]: pgmap v476: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T07:16:19.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:18 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:16:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:20 smithi153 ceph-mon[34747]: pgmap v477: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-09-21T07:16:21.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:20 smithi119 ceph-mon[34519]: pgmap v477: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-09-21T07:16:21.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:20 smithi027 ceph-mon[30779]: pgmap v477: 129 pgs: 129 active+clean; 2.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-09-21T07:16:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:22 smithi153 ceph-mon[34747]: pgmap v478: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T07:16:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:22 smithi119 ceph-mon[34519]: pgmap v478: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T07:16:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:22 smithi027 ceph-mon[30779]: pgmap v478: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-21T07:16:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:24 smithi153 ceph-mon[34747]: pgmap v479: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-21T07:16:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:24 smithi119 ceph-mon[34519]: pgmap v479: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-21T07:16:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:24 smithi027 ceph-mon[30779]: pgmap v479: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-21T07:16:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:26 smithi153 ceph-mon[34747]: pgmap v480: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-21T07:16:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:26 smithi119 ceph-mon[34519]: pgmap v480: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-21T07:16:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:26 smithi027 ceph-mon[30779]: pgmap v480: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-21T07:16:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:28 smithi153 ceph-mon[34747]: pgmap v481: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-21T07:16:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:28 smithi119 ceph-mon[34519]: pgmap v481: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-21T07:16:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:28 smithi027 ceph-mon[30779]: pgmap v481: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-21T07:16:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 -- 172.21.15.27:0/1081214023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 msgr2=0x7feb94109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 --2- 172.21.15.27:0/1081214023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 0x7feb94109650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feb80009900 tx=0x7feb8002a410 comp rx=0 tx=0).stop 2024-09-21T07:16:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 -- 172.21.15.27:0/1081214023 shutdown_connections 2024-09-21T07:16:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 --2- 172.21.15.27:0/1081214023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94109b90 0x7feb941103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 --2- 172.21.15.27:0/1081214023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 0x7feb94109650 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 --2- 172.21.15.27:0/1081214023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb94103910 0x7feb94103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 -- 172.21.15.27:0/1081214023 >> 172.21.15.27:0/1081214023 conn(0x7feb940ff330 msgr2=0x7feb94101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 -- 172.21.15.27:0/1081214023 shutdown_connections 2024-09-21T07:16:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.512+0000 7feb99600640 1 -- 172.21.15.27:0/1081214023 wait complete. 2024-09-21T07:16:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.513+0000 7feb99600640 1 Processor -- start 2024-09-21T07:16:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.513+0000 7feb99600640 1 -- start start 2024-09-21T07:16:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb99600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94103910 0x7feb9419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb99600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 0x7feb9419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb99600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb94109b90 0x7feb941a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb99600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb941138a0 con 0x7feb94109240 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb99600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113720 con 0x7feb94103910 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb99600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113a20 con 0x7feb94109b90 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 0x7feb9419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94103910 0x7feb9419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94103910 0x7feb9419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60106/0 (socket says 172.21.15.27:60106) 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93e00640 1 -- 172.21.15.27:0/929165621 learned_addr learned my addr 172.21.15.27:0/929165621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb98c00640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb94109b90 0x7feb941a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93400640 1 -- 172.21.15.27:0/929165621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb94109b90 msgr2=0x7feb941a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93400640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb94109b90 0x7feb941a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93400640 1 -- 172.21.15.27:0/929165621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94103910 msgr2=0x7feb9419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93400640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94103910 0x7feb9419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93400640 1 -- 172.21.15.27:0/929165621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb941a23f0 con 0x7feb94109240 2024-09-21T07:16:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.514+0000 7feb93e00640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94103910 0x7feb9419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.515+0000 7feb93400640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 0x7feb9419ca60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feb80009230 tx=0x7feb80003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.515+0000 7feb90c00640 1 -- 172.21.15.27:0/929165621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8003f070 con 0x7feb94109240 2024-09-21T07:16:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.515+0000 7feb90c00640 1 -- 172.21.15.27:0/929165621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb80031b50 con 0x7feb94109240 2024-09-21T07:16:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.515+0000 7feb90c00640 1 -- 172.21.15.27:0/929165621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80035030 con 0x7feb94109240 2024-09-21T07:16:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.515+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb941a2680 con 0x7feb94109240 2024-09-21T07:16:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.515+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb941a2af0 con 0x7feb94109240 2024-09-21T07:16:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.518+0000 7feba1e08740 1 Processor -- start 2024-09-21T07:16:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.518+0000 7feba1e08740 1 -- start start 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feba1e08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f6ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feba1e08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f6f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feba1e08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f6f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feba1e08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608e1ee46b0 con 0x5608e1bca1b0 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feba1e08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5608e1ed3ae0 con 0x5608e1ba94d0 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feba1e08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5608e1ed78f0 con 0x5608e1ba12f0 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feb90c00640 1 -- 172.21.15.27:0/929165621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7feb80042070 con 0x7feb94109240 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feb98c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f6ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feb98c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f6ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40672/0 (socket says 172.21.15.27:40672) 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feb98c00640 1 -- 172.21.15.27:0/3029340224 learned_addr learned my addr 172.21.15.27:0/3029340224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feb93e00640 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f6f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.519+0000 7feb93400640 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f6f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb98c00640 1 -- 172.21.15.27:0/3029340224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 msgr2=0x5608e1f6f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb98c00640 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f6f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb98c00640 1 -- 172.21.15.27:0/3029340224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 msgr2=0x5608e1f6f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb98c00640 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f6f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb98c00640 1 -- 172.21.15.27:0/3029340224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e1ea0780 con 0x5608e1bca1b0 2024-09-21T07:16:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb93400640 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f6f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb98c00640 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f6ee10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009fd0 tx=0x7feb8c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bde254a5b4c9755 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb89200640 1 -- 172.21.15.27:0/3029340224 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c004d20 con 0x5608e1bca1b0 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.520+0000 7feb89200640 1 -- 172.21.15.27:0/3029340224 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c004050 con 0x5608e1bca1b0 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feb89200640 1 -- 172.21.15.27:0/3029340224 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c005280 con 0x5608e1bca1b0 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feb90c00640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb780b6700 0x7feb780b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feb90c00640 1 -- 172.21.15.27:0/929165621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7feb801078e0 con 0x7feb94109240 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feba1e08740 1 -- 172.21.15.27:0/3029340224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 msgr2=0x5608e1f6ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feba1e08740 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f6ee10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009fd0 tx=0x7feb8c0118d0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feb93e00640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb780b6700 0x7feb780b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feba1e08740 1 -- 172.21.15.27:0/3029340224 shutdown_connections 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feba1e08740 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f6f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feba1e08740 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f6f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feba1e08740 1 --2- 172.21.15.27:0/3029340224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f6ee10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.521+0000 7feba1e08740 1 -- 172.21.15.27:0/3029340224 >> 172.21.15.27:0/3029340224 conn(0x5608e1bbe300 msgr2=0x5608e1f6cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.522+0000 7feba1e08740 1 -- 172.21.15.27:0/3029340224 shutdown_connections 2024-09-21T07:16:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.522+0000 7feba1e08740 1 -- 172.21.15.27:0/3029340224 wait complete. 2024-09-21T07:16:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.522+0000 7feb93e00640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb780b6700 0x7feb780b8c10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb84009fd0 tx=0x7feb84005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.522+0000 7feba1e08740 1 -- start start 2024-09-21T07:16:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 1 Processor -- start 2024-09-21T07:16:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1ba94d0 0x5608e1f7a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1bca1b0 0x5608e1f7ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608e1e0bab0 con 0x5608e1ba94d0 2024-09-21T07:16:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5608e1e7a570 con 0x5608e1bca1b0 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5608e1f7d590 con 0x5608e1ba12f0 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feba1e08740 10 client.0 _ll_register_callbacks cb 0x5608e1e89100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feb93400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1ba94d0 0x5608e1f7a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feb93e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feb93e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41508/0 (socket says 172.21.15.27:41508) 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feb93e00640 1 -- 172.21.15.27:0/3087739839 learned_addr learned my addr 172.21.15.27:0/3087739839 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.523+0000 7feb93400640 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1ba94d0 0x5608e1f7a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40682/0 (socket says 172.21.15.27:40682) 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb98c00640 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1bca1b0 0x5608e1f7ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb93400640 1 -- 172.21.15.27:0/3087739839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 msgr2=0x5608e1f7a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb93400640 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb93400640 1 -- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1bca1b0 msgr2=0x5608e1f7ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb93400640 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1bca1b0 0x5608e1f7ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb93400640 1 -- 172.21.15.27:0/3087739839 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e1c35790 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb93400640 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1ba94d0 0x5608e1f7a930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feb80104c80 tx=0x7feb80003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ce61b95faae596 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb7d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb7d600640 1 -- 172.21.15.27:0/3087739839 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80049070 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feb7d600640 1 -- 172.21.15.27:0/3087739839 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7feb800d6b10 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.524+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c004050 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feb7d600640 1 -- 172.21.15.27:0/3087739839 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb800cd200 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 10 client.16500 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608e1cae620 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feb7d600640 1 -- 172.21.15.27:0/3087739839 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c004050 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feb7d600640 1 -- 172.21.15.27:0/3087739839 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feb800b4b70 con 0x5608e1ba94d0 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 10 client.16500 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 20 client.16500 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b88d80 0x5608e1f8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 4 client.16500 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e1ee6600 con 0x5608e1b88d80 2024-09-21T07:16:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.525+0000 7feb93e00640 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b88d80 0x5608e1f8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.526+0000 7feb93e00640 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b88d80 0x5608e1f8c510 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf2e1a4b7acba154 server_cookie=ae359d67cb0e77f3 in_seq=0 out_seq=0 2024-09-21T07:16:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.526+0000 7feb7d600640 10 client.16500 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feb7d600640 1 -- 172.21.15.27:0/3087739839 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608e1ee6600 con 0x5608e1b88d80 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feb7d600640 10 client.16500 handle_command_reply: tid=0 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 client.16500 shutdown 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1ba94d0 msgr2=0x5608e1f7a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1ba94d0 0x5608e1f7a930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feb80104c80 tx=0x7feb80003f90 comp rx=0 tx=0).stop 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 shutdown_connections 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b88d80 0x5608e1f8c510 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1bca1b0 0x5608e1f7ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1ba94d0 0x5608e1f7a930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 --2- 172.21.15.27:0/3087739839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feb7d600640 0 client.16500 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 >> 172.21.15.27:0/3087739839 conn(0x5608e1bbe300 msgr2=0x5608e1f6fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 shutdown_connections 2024-09-21T07:16:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.527+0000 7feba1e08740 1 -- 172.21.15.27:0/3087739839 wait complete. 2024-09-21T07:16:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.528+0000 7feba1e08740 20 client.16500 trim_cache size 0 max 16384 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 Processor -- start 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 -- start start 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba12f0 0x5608e1f77ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba94d0 0x5608e1f91ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f92220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b55b0 con 0x5608e1bca1b0 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b2e80 con 0x5608e1ba12f0 2024-09-21T07:16:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feba1e08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b0cd0 con 0x5608e1ba94d0 2024-09-21T07:16:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.543+0000 7feb98c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f92220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f92220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40684/0 (socket says 172.21.15.27:40684) 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 -- 172.21.15.27:0/791139626 learned_addr learned my addr 172.21.15.27:0/791139626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb93400640 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba94d0 0x5608e1f91ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb93e00640 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba12f0 0x5608e1f77ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 -- 172.21.15.27:0/791139626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba94d0 msgr2=0x5608e1f91ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba94d0 0x5608e1f91ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 -- 172.21.15.27:0/791139626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba12f0 msgr2=0x5608e1f77ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba12f0 0x5608e1f77ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 -- 172.21.15.27:0/791139626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e1f6da70 con 0x5608e1bca1b0 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb98c00640 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f92220 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009f70 tx=0x7feb8c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e573b8378abb640a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.544+0000 7feb7fe00640 1 -- 172.21.15.27:0/791139626 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c003c60 con 0x5608e1bca1b0 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feb7fe00640 1 -- 172.21.15.27:0/791139626 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c004050 con 0x5608e1bca1b0 2024-09-21T07:16:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feb7fe00640 1 -- 172.21.15.27:0/791139626 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c005280 con 0x5608e1bca1b0 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 -- 172.21.15.27:0/791139626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 msgr2=0x5608e1f92220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f92220 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009f70 tx=0x7feb8c009be0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 -- 172.21.15.27:0/791139626 shutdown_connections 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba94d0 0x5608e1f91ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f92220 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 --2- 172.21.15.27:0/791139626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba12f0 0x5608e1f77ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 -- 172.21.15.27:0/791139626 >> 172.21.15.27:0/791139626 conn(0x5608e1bbe300 msgr2=0x5608e1f777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 -- 172.21.15.27:0/791139626 shutdown_connections 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.545+0000 7feba1e08740 1 -- 172.21.15.27:0/791139626 wait complete. 2024-09-21T07:16:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.546+0000 7feba1e08740 1 -- start start 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.546+0000 7feba1e08740 1 Processor -- start 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.546+0000 7feba1e08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.546+0000 7feba1e08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f7ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.546+0000 7feba1e08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feba1e08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b1340 con 0x5608e1bca1b0 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feba1e08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b27b0 con 0x5608e1ba94d0 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feba1e08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b1a10 con 0x5608e1ba12f0 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feba1e08740 10 client.0 _ll_register_callbacks cb 0x7feb80005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb98c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41536/0 (socket says 172.21.15.27:41536) 2024-09-21T07:16:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 -- 172.21.15.27:0/1057797715 learned_addr learned my addr 172.21.15.27:0/1057797715 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93400640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f7ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 -- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 msgr2=0x5608e1f7ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f7ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 -- 172.21.15.27:0/1057797715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 msgr2=0x5608e1f7f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f7f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93e00640 1 -- 172.21.15.27:0/1057797715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e1ff6110 con 0x5608e1ba12f0 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb98c00640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f7f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.547+0000 7feb93400640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f7ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feb93e00640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7ea10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb8400f5e0 tx=0x7feb840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420bc2439bcffc5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feb7d600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feb7d600640 1 -- 172.21.15.27:0/1057797715 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84005020 con 0x5608e1ba12f0 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c004050 con 0x5608e1ba12f0 2024-09-21T07:16:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feb7d600640 1 -- 172.21.15.27:0/1057797715 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7feb840051c0 con 0x5608e1ba12f0 2024-09-21T07:16:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feb7d600640 1 -- 172.21.15.27:0/1057797715 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb8403c050 con 0x5608e1ba12f0 2024-09-21T07:16:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feba1e08740 10 client.26527 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.548+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608e1ff7400 con 0x5608e1ba12f0 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feb7d600640 1 -- 172.21.15.27:0/1057797715 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c004050 con 0x5608e1ba12f0 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feb7d600640 1 -- 172.21.15.27:0/1057797715 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feb8401f600 con 0x5608e1ba12f0 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feba1e08740 10 client.26527 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feba1e08740 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feba1e08740 20 client.26527 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feba1e08740 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b5e1c0 0x5608e1ff9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feba1e08740 4 client.26527 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608e1ee20b0 con 0x5608e1b5e1c0 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.549+0000 7feb93400640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b5e1c0 0x5608e1ff9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.550+0000 7feb93400640 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b5e1c0 0x5608e1ff9bc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9158168d97034e08 server_cookie=7d0d3280e6aa64ce in_seq=0 out_seq=0 2024-09-21T07:16:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.550+0000 7feb7d600640 10 client.26527 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feb7d600640 1 -- 172.21.15.27:0/1057797715 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608e1ee20b0 con 0x5608e1b5e1c0 2024-09-21T07:16:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feb7d600640 10 client.26527 handle_command_reply: tid=0 2024-09-21T07:16:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 client.26527 shutdown 2024-09-21T07:16:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 msgr2=0x5608e1f7ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7ea10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb8400f5e0 tx=0x7feb840046c0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 shutdown_connections 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608e1b5e1c0 0x5608e1ff9bc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608e1bca1b0 0x5608e1f7f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608e1ba94d0 0x5608e1f7ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 --2- 172.21.15.27:0/1057797715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608e1ba12f0 0x5608e1f7ea10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feb7d600640 0 client.26527 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 >> 172.21.15.27:0/1057797715 conn(0x5608e1bbe300 msgr2=0x5608e1f7a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 shutdown_connections 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 1 -- 172.21.15.27:0/1057797715 wait complete. 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.561+0000 7feba1e08740 20 client.26527 trim_cache size 0 max 16384 2024-09-21T07:16:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb780b6700 msgr2=0x7feb780b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb780b6700 0x7feb780b8c10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb84009fd0 tx=0x7feb84005f90 comp rx=0 tx=0).stop 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 msgr2=0x7feb9419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 0x7feb9419ca60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feb80009230 tx=0x7feb80003b90 comp rx=0 tx=0).stop 2024-09-21T07:16:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 shutdown_connections 2024-09-21T07:16:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb94109b90 0x7feb941a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feb780b6700 0x7feb780b8c10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94109240 0x7feb9419ca60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 --2- 172.21.15.27:0/929165621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb94103910 0x7feb9419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 >> 172.21.15.27:0/929165621 conn(0x7feb940ff330 msgr2=0x7feb94108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 shutdown_connections 2024-09-21T07:16:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:30.562+0000 7feb99600640 1 -- 172.21.15.27:0/929165621 wait complete. 2024-09-21T07:16:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:30 smithi153 ceph-mon[34747]: pgmap v482: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:16:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:30 smithi119 ceph-mon[34519]: pgmap v482: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:16:31.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:30 smithi027 ceph-mon[30779]: pgmap v482: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:16:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:32 smithi153 ceph-mon[34747]: pgmap v483: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-09-21T07:16:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:32 smithi119 ceph-mon[34519]: pgmap v483: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-09-21T07:16:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:32 smithi027 ceph-mon[30779]: pgmap v483: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-09-21T07:16:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:34 smithi153 ceph-mon[34747]: pgmap v484: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:16:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:34 smithi119 ceph-mon[34519]: pgmap v484: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:16:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:34 smithi027 ceph-mon[30779]: pgmap v484: 129 pgs: 129 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:16:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:36 smithi153 ceph-mon[34747]: pgmap v485: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-21T07:16:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:36 smithi119 ceph-mon[34519]: pgmap v485: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-21T07:16:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:36 smithi027 ceph-mon[30779]: pgmap v485: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-21T07:16:37.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:16:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.824+0000 7f43aee00640 1 -- 172.21.15.27:0/3671357170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 msgr2=0x7f43a81061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.824+0000 7f43aee00640 1 --2- 172.21.15.27:0/3671357170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 0x7f43a81061a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4398009900 tx=0x7f439802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.824+0000 7f43aee00640 1 -- 172.21.15.27:0/3671357170 shutdown_connections 2024-09-21T07:16:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.824+0000 7f43aee00640 1 --2- 172.21.15.27:0/3671357170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a810c020 0x7f43a8112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.824+0000 7f43aee00640 1 --2- 172.21.15.27:0/3671357170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 0x7f43a810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.824+0000 7f43aee00640 1 --2- 172.21.15.27:0/3671357170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 0x7f43a81061a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.824+0000 7f43aee00640 1 -- 172.21.15.27:0/3671357170 >> 172.21.15.27:0/3671357170 conn(0x7f43a8101860 msgr2=0x7f43a8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.825+0000 7f43aee00640 1 -- 172.21.15.27:0/3671357170 shutdown_connections 2024-09-21T07:16:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.825+0000 7f43aee00640 1 -- 172.21.15.27:0/3671357170 wait complete. 2024-09-21T07:16:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 Processor -- start 2024-09-21T07:16:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 -- start start 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 0x7f43a819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 0x7f43a819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a810c020 0x7f43a81a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8115ce0 con 0x7f43a8105d90 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8115b60 con 0x7f43a810b6d0 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.826+0000 7f43aee00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8115e60 con 0x7f43a810c020 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 0x7f43a819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 0x7f43a819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 0x7f43a819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35934/0 (socket says 172.21.15.27:35934) 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 -- 172.21.15.27:0/89460148 learned_addr learned my addr 172.21.15.27:0/89460148 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43ace00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a810c020 0x7f43a81a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 -- 172.21.15.27:0/89460148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a810c020 msgr2=0x7f43a81a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a810c020 0x7f43a81a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 -- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 msgr2=0x7f43a819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 0x7f43a819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.827+0000 7f43a7400640 1 -- 172.21.15.27:0/89460148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a81a2620 con 0x7f43a810b6d0 2024-09-21T07:16:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.828+0000 7f43a7400640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 0x7f43a819cc50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f439400e970 tx=0x7f439400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.828+0000 7f43a4c00640 1 -- 172.21.15.27:0/89460148 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439400cd80 con 0x7f43a810b6d0 2024-09-21T07:16:38.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.828+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a81a2910 con 0x7f43a810b6d0 2024-09-21T07:16:38.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.828+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a81a9070 con 0x7f43a810b6d0 2024-09-21T07:16:38.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.828+0000 7f43a4c00640 1 -- 172.21.15.27:0/89460148 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43940035f0 con 0x7f43a810b6d0 2024-09-21T07:16:38.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.828+0000 7f43a4c00640 1 -- 172.21.15.27:0/89460148 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394010690 con 0x7f43a810b6d0 2024-09-21T07:16:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.831+0000 7f43b5a7d740 1 Processor -- start 2024-09-21T07:16:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43b5a7d740 1 -- start start 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784843d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597844741b0 0x5597848442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978447c390 0x5597848447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5597847baf70 con 0x55978447c390 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5597847aa3a0 con 0x5597844741b0 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5597847ae1b0 con 0x55978449d070 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597844741b0 0x5597848442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43ace00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784843d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597844741b0 0x5597848442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35944/0 (socket says 172.21.15.27:35944) 2024-09-21T07:16:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.832+0000 7f43a7e00640 1 -- 172.21.15.27:0/1649529631 learned_addr learned my addr 172.21.15.27:0/1649529631 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.833+0000 7f43a4c00640 1 -- 172.21.15.27:0/89460148 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f43940037a0 con 0x7f43a810b6d0 2024-09-21T07:16:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.833+0000 7f43a7e00640 1 -- 172.21.15.27:0/1649529631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 msgr2=0x559784843d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.833+0000 7f43a7e00640 1 --2- 172.21.15.27:0/1649529631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784843d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.833+0000 7f43a7e00640 1 -- 172.21.15.27:0/1649529631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978447c390 msgr2=0x5597848447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.833+0000 7f43a7e00640 1 --2- 172.21.15.27:0/1649529631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978447c390 0x5597848447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.833+0000 7f43a7e00640 1 -- 172.21.15.27:0/1649529631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559784774c90 con 0x5597844741b0 2024-09-21T07:16:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.833+0000 7f43a7e00640 1 --2- 172.21.15.27:0/1649529631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597844741b0 0x5597848442a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4398009900 tx=0x7f4398032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edde494a01c5d385 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.834+0000 7f439d600640 1 -- 172.21.15.27:0/1649529631 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398040070 con 0x5597844741b0 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.834+0000 7f439d600640 1 -- 172.21.15.27:0/1649529631 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4398004030 con 0x5597844741b0 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.834+0000 7f439d600640 1 -- 172.21.15.27:0/1649529631 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398035040 con 0x5597844741b0 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.834+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1649529631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597844741b0 msgr2=0x5597848442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.834+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1649529631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597844741b0 0x5597848442a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4398009900 tx=0x7f4398032b90 comp rx=0 tx=0).stop 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1649529631 shutdown_connections 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1649529631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978447c390 0x5597848447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1649529631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597844741b0 0x5597848442a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1649529631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784843d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1649529631 >> 172.21.15.27:0/1649529631 conn(0x559784491460 msgr2=0x559784841c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1649529631 shutdown_connections 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1649529631 wait complete. 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43a4c00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f438c0b6610 0x7f438c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.835+0000 7f43b5a7d740 1 -- start start 2024-09-21T07:16:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43a4c00640 1 -- 172.21.15.27:0/89460148 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f43940e7e00 con 0x7f43a810b6d0 2024-09-21T07:16:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43ace00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f438c0b6610 0x7f438c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43b5a7d740 1 Processor -- start 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597844741b0 0x55978484f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x55978484fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978449d070 0x55978485b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43ace00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f438c0b6610 0x7f438c0b8b20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f43a0005fd0 tx=0x7f43a0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55978460b700 con 0x55978449d070 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559784745540 con 0x55978447c390 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.836+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5597848524e0 con 0x5597844741b0 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43b5a7d740 10 client.0 _ll_register_callbacks cb 0x559784755430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597844741b0 0x55978484f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597844741b0 0x55978484f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38438/0 (socket says 172.21.15.27:38438) 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 -- 172.21.15.27:0/1601928427 learned_addr learned my addr 172.21.15.27:0/1601928427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43ace00640 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978449d070 0x55978485b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7400640 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x55978484fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 -- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 msgr2=0x55978484fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x55978484fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 -- 172.21.15.27:0/1601928427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978449d070 msgr2=0x55978485b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978449d070 0x55978485b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.837+0000 7f43a7e00640 1 -- 172.21.15.27:0/1601928427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559784508650 con 0x5597844741b0 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f43a7e00640 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597844741b0 0x55978484f4c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4398032f30 tx=0x7f439802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e09561fb5c70ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f4392000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f4392000640 1 -- 172.21.15.27:0/1601928427 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398035040 con 0x5597844741b0 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559784747b50 con 0x5597844741b0 2024-09-21T07:16:38.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f4392000640 1 -- 172.21.15.27:0/1601928427 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f43980314a0 con 0x5597844741b0 2024-09-21T07:16:38.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f4392000640 1 -- 172.21.15.27:0/1601928427 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4398055050 con 0x5597844741b0 2024-09-21T07:16:38.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f43b5a7d740 10 client.26539 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:38.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.838+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559784762910 con 0x5597844741b0 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f4392000640 1 -- 172.21.15.27:0/1601928427 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559784747b50 con 0x5597844741b0 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f4392000640 1 -- 172.21.15.27:0/1601928427 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4398004a80 con 0x5597844741b0 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f43b5a7d740 10 client.26539 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f43b5a7d740 10 client.26539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f43b5a7d740 20 client.26539 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55978445b9f0 0x559784861480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f43b5a7d740 4 client.26539 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597847bcec0 con 0x55978445b9f0 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.839+0000 7f43a7400640 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55978445b9f0 0x559784861480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.840+0000 7f4392000640 10 client.26539 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:38.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.840+0000 7f43a7400640 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55978445b9f0 0x559784861480 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8eecb9e673c6653 server_cookie=8b9abc6e09291c8a in_seq=0 out_seq=0 2024-09-21T07:16:38.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f4392000640 1 -- 172.21.15.27:0/1601928427 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597847bcec0 con 0x55978445b9f0 2024-09-21T07:16:38.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f4392000640 10 client.26539 handle_command_reply: tid=0 2024-09-21T07:16:38.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 client.26539 shutdown 2024-09-21T07:16:38.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597844741b0 msgr2=0x55978484f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597844741b0 0x55978484f4c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4398032f30 tx=0x7f439802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 shutdown_connections 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55978445b9f0 0x559784861480 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55978449d070 0x55978485b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x55978484fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f4392000640 0 client.26539 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/1601928427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597844741b0 0x55978484f4c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.841+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 >> 172.21.15.27:0/1601928427 conn(0x559784491460 msgr2=0x559784846ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.842+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 shutdown_connections 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.842+0000 7f43b5a7d740 1 -- 172.21.15.27:0/1601928427 wait complete. 2024-09-21T07:16:38.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.842+0000 7f43b5a7d740 20 client.26539 trim_cache size 0 max 16384 2024-09-21T07:16:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 Processor -- start 2024-09-21T07:16:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 -- start start 2024-09-21T07:16:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597844741b0 0x55978484ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978447c390 0x5597848454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978449d070 0x55978484c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b5850 con 0x5597844741b0 2024-09-21T07:16:38.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b3120 con 0x55978449d070 2024-09-21T07:16:38.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.861+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b0f70 con 0x55978447c390 2024-09-21T07:16:38.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597844741b0 0x55978484ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597844741b0 0x55978484ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43680/0 (socket says 172.21.15.27:43680) 2024-09-21T07:16:38.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 -- 172.21.15.27:0/3518100605 learned_addr learned my addr 172.21.15.27:0/3518100605 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43ace00640 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978449d070 0x55978484c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7400640 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978447c390 0x5597848454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 -- 172.21.15.27:0/3518100605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978447c390 msgr2=0x5597848454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978447c390 0x5597848454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 -- 172.21.15.27:0/3518100605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978449d070 msgr2=0x55978484c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978449d070 0x55978484c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.862+0000 7f43a7e00640 1 -- 172.21.15.27:0/3518100605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559784850780 con 0x5597844741b0 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.863+0000 7f43a7e00640 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597844741b0 0x55978484ac90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f439802a910 tx=0x7f4398031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab6f924da6250c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.863+0000 7f439cc00640 1 -- 172.21.15.27:0/3518100605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398035070 con 0x5597844741b0 2024-09-21T07:16:38.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.863+0000 7f439cc00640 1 -- 172.21.15.27:0/3518100605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43980314a0 con 0x5597844741b0 2024-09-21T07:16:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.863+0000 7f439cc00640 1 -- 172.21.15.27:0/3518100605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398040040 con 0x5597844741b0 2024-09-21T07:16:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3518100605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597844741b0 msgr2=0x55978484ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597844741b0 0x55978484ac90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f439802a910 tx=0x7f4398031400 comp rx=0 tx=0).stop 2024-09-21T07:16:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3518100605 shutdown_connections 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978447c390 0x5597848454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978449d070 0x55978484c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3518100605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597844741b0 0x55978484ac90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3518100605 >> 172.21.15.27:0/3518100605 conn(0x559784491460 msgr2=0x559784844d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3518100605 shutdown_connections 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.864+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3518100605 wait complete. 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.865+0000 7f43b5a7d740 1 -- start start 2024-09-21T07:16:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.865+0000 7f43b5a7d740 1 Processor -- start 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.866+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x559784862720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.866+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784862c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.866+0000 7f43b5a7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559784430da0 0x5597848631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.866+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b15e0 con 0x559784430da0 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.866+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b2a50 con 0x55978447c390 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.866+0000 7f43b5a7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b1cb0 con 0x55978449d070 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.866+0000 7f43b5a7d740 10 client.0 _ll_register_callbacks cb 0x7f4398005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43ace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559784430da0 0x5597848631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x559784862720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784862c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43ace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559784430da0 0x5597848631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43688/0 (socket says 172.21.15.27:43688) 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43ace00640 1 -- 172.21.15.27:0/3295443611 learned_addr learned my addr 172.21.15.27:0/3295443611 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x559784862720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35964/0 (socket says 172.21.15.27:35964) 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7400640 1 -- 172.21.15.27:0/3295443611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 msgr2=0x559784862720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7400640 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x559784862720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7400640 1 -- 172.21.15.27:0/3295443611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559784430da0 msgr2=0x5597848631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7400640 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559784430da0 0x5597848631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7400640 1 -- 172.21.15.27:0/3295443611 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597848cae30 con 0x55978449d070 2024-09-21T07:16:38.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f43a7400640 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784862c60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43940e51a0 tx=0x7f4394005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16854472348cb95a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:38.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.867+0000 7f4392000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.868+0000 7f4392000640 1 -- 172.21.15.27:0/3295443611 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940e5620 con 0x55978449d070 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.868+0000 7f4392000640 1 -- 172.21.15.27:0/3295443611 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f43940acb00 con 0x55978449d070 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.868+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43980314a0 con 0x55978449d070 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.868+0000 7f4392000640 1 -- 172.21.15.27:0/3295443611 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43940adfa0 con 0x55978449d070 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.868+0000 7f43b5a7d740 10 client.26545 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.868+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597848cc120 con 0x55978449d070 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.868+0000 7f4392000640 1 -- 172.21.15.27:0/3295443611 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43980314a0 con 0x55978449d070 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f4392000640 1 -- 172.21.15.27:0/3295443611 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f43940ac570 con 0x55978449d070 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f43b5a7d740 10 client.26545 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:38.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f43b5a7d740 10 client.26545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f43b5a7d740 20 client.26545 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597848cc820 0x5597848cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f43b5a7d740 4 client.26545 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597847b8970 con 0x5597848cc820 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.869+0000 7f43a7e00640 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597848cc820 0x5597848cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.870+0000 7f4392000640 10 client.26545 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.870+0000 7f43a7e00640 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597848cc820 0x5597848cec40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39fbc18896cf89bb server_cookie=fb64df41c96b2315 in_seq=0 out_seq=0 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.870+0000 7f4392000640 1 -- 172.21.15.27:0/3295443611 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x5597847b8970 con 0x5597848cc820 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.870+0000 7f4392000640 10 client.26545 handle_command_reply: tid=0 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.870+0000 7f43b5a7d740 1 client.26545 shutdown 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 msgr2=0x559784862c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784862c60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43940e51a0 tx=0x7f4394005f10 comp rx=0 tx=0).stop 2024-09-21T07:16:38.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 shutdown_connections 2024-09-21T07:16:38.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597848cc820 0x5597848cec40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55978449d070 0x559784862c60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559784430da0 0x5597848631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 --2- 172.21.15.27:0/3295443611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55978447c390 0x559784862720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f4392000640 0 client.26545 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 >> 172.21.15.27:0/3295443611 conn(0x559784491460 msgr2=0x559784844d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 shutdown_connections 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 1 -- 172.21.15.27:0/3295443611 wait complete. 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.871+0000 7f43b5a7d740 20 client.26545 trim_cache size 0 max 16384 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:16:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:38.879 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f438c0b6610 msgr2=0x7f438c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f438c0b6610 0x7f438c0b8b20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f43a0005fd0 tx=0x7f43a0009500 comp rx=0 tx=0).stop 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 msgr2=0x7f43a819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 0x7f43a819cc50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f439400e970 tx=0x7f439400ee30 comp rx=0 tx=0).stop 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 shutdown_connections 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a810c020 0x7f43a81a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f438c0b6610 0x7f438c0b8b20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a810b6d0 0x7f43a819cc50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 --2- 172.21.15.27:0/89460148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8105d90 0x7f43a819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 >> 172.21.15.27:0/89460148 conn(0x7f43a8101860 msgr2=0x7f43a810ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 shutdown_connections 2024-09-21T07:16:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:38.872+0000 7f43aee00640 1 -- 172.21.15.27:0/89460148 wait complete. 2024-09-21T07:16:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:38 smithi027 ceph-mon[30779]: pgmap v486: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:16:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:38 smithi153 ceph-mon[34747]: pgmap v486: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:1/600: open fd = 3 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:2/600: open fd = 3 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:3/600: open fd = 3 2024-09-21T07:16:39.195 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:4/600: open fd = 3 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:5/600: open fd = 3 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.196 INFO:tasks.workunit.client.0.smithi027.stdout:6/600: open fd = 3 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:7/600: open fd = 3 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:8/600: open fd = 3 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:9/600: open fd = 3 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.197 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:10/600: open fd = 3 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:11/600: open fd = 3 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.198 INFO:tasks.workunit.client.0.smithi027.stdout:12/600: open fd = 3 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:13/600: open fd = 3 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:14/600: open fd = 3 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:15/600: open fd = 3 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.199 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:16/600: open fd = 3 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:17/600: open fd = 3 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.200 INFO:tasks.workunit.client.0.smithi027.stdout:18/600: open fd = 3 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:19/600: open fd = 3 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:20/600: open fd = 3 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.201 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:21/600: open fd = 3 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:22/600: open fd = 3 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:23/600: open fd = 3 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.202 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:24/600: open fd = 3 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:25/600: open fd = 3 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:26/600: open fd = 3 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.203 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:27/600: open fd = 3 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:28/600: open fd = 3 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:29/600: open fd = 3 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.204 INFO:tasks.workunit.client.0.smithi027.stdout:30/600: open fd = 3 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:31/600: open fd = 3 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.205 INFO:tasks.workunit.client.0.smithi027.stdout:32/600: open fd = 3 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:33/600: open fd = 3 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:34/600: open fd = 3 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.206 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:35/600: open fd = 3 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:36/600: open fd = 3 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.207 INFO:tasks.workunit.client.0.smithi027.stdout:37/600: open fd = 3 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:38/600: open fd = 3 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:39/600: open fd = 3 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:40/600: open fd = 3 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.208 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:41/600: open fd = 3 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:42/600: open fd = 3 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:43/600: open fd = 3 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:44/600: open fd = 3 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.209 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:45/600: open fd = 3 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:46/600: open fd = 3 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:47/600: open fd = 3 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:48/600: open fd = 3 2024-09-21T07:16:39.210 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:49/600: open fd = 3 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:50/600: open fd = 3 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:51/600: open fd = 3 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.211 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:52/600: open fd = 3 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:53/600: open fd = 3 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:54/600: open fd = 3 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:55/600: open fd = 3 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.212 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:56/600: open fd = 3 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:57/600: open fd = 3 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:58/600: open fd = 3 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:59/600: open fd = 3 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.213 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:60/600: open fd = 3 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:61/600: open fd = 3 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:62/600: open fd = 3 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:63/600: open fd = 3 2024-09-21T07:16:39.214 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.215 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:16:39.215 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:16:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:38 smithi119 ceph-mon[34519]: pgmap v486: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:16:39.725 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:16:39.725 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:16:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.737+0000 7f64cfe00640 1 -- 172.21.15.27:0/1770850154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01039b0 msgr2=0x7f64d0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.737+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1770850154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01039b0 0x7f64d0103dc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009900 tx=0x7f64c002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.738+0000 7f64cfe00640 1 -- 172.21.15.27:0/1770850154 shutdown_connections 2024-09-21T07:16:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.738+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1770850154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d0109c40 0x7f64d0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.738+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1770850154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64d01092f0 0x7f64d0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.738+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1770850154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01039b0 0x7f64d0103dc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:40.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.738+0000 7f64cfe00640 1 -- 172.21.15.27:0/1770850154 >> 172.21.15.27:0/1770850154 conn(0x7f64d00ff400 msgr2=0x7f64d0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:40.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.738+0000 7f64cfe00640 1 -- 172.21.15.27:0/1770850154 shutdown_connections 2024-09-21T07:16:40.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.738+0000 7f64cfe00640 1 -- 172.21.15.27:0/1770850154 wait complete. 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 Processor -- start 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 -- start start 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d01039b0 0x7f64d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01092f0 0x7f64d0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64d0109c40 0x7f64d0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f64d0113950 con 0x7f64d0109c40 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01137d0 con 0x7f64d01039b0 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.739+0000 7f64cfe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64d0113ad0 con 0x7f64d01092f0 2024-09-21T07:16:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d01039b0 0x7f64d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:40.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64ce000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01092f0 0x7f64d0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:40.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d01039b0 0x7f64d0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35984/0 (socket says 172.21.15.27:35984) 2024-09-21T07:16:40.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64ce000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01092f0 0x7f64d0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38476/0 (socket says 172.21.15.27:38476) 2024-09-21T07:16:40.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 -- 172.21.15.27:0/1477734543 learned_addr learned my addr 172.21.15.27:0/1477734543 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:40.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cf400640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64d0109c40 0x7f64d0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:40.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 -- 172.21.15.27:0/1477734543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01092f0 msgr2=0x7f64d0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01092f0 0x7f64d0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 -- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64d0109c40 msgr2=0x7f64d0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:40.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64d0109c40 0x7f64d0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64cea00640 1 -- 172.21.15.27:0/1477734543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d01a86c0 con 0x7f64d01039b0 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.740+0000 7f64ce000640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01092f0 0x7f64d0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.741+0000 7f64cea00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d01039b0 0x7f64d0075710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005ab0 tx=0x7f64c0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.741+0000 7f64c7400640 1 -- 172.21.15.27:0/1477734543 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c003f070 con 0x7f64d01039b0 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.741+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d01a8950 con 0x7f64d01039b0 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.741+0000 7f64c7400640 1 -- 172.21.15.27:0/1477734543 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c0031b60 con 0x7f64d01039b0 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.741+0000 7f64c7400640 1 -- 172.21.15.27:0/1477734543 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0035030 con 0x7f64d01039b0 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.741+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d01a8e80 con 0x7f64d01039b0 2024-09-21T07:16:40.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.742+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d0103dc0 con 0x7f64d01039b0 2024-09-21T07:16:40.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.745+0000 7f64c7400640 1 -- 172.21.15.27:0/1477734543 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f64c0042070 con 0x7f64d01039b0 2024-09-21T07:16:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.747+0000 7f64c7400640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f64b40b6680 0x7f64b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.747+0000 7f64c7400640 1 -- 172.21.15.27:0/1477734543 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f64c01088a0 con 0x7f64d01039b0 2024-09-21T07:16:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.747+0000 7f64ce000640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f64b40b6680 0x7f64b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.748+0000 7f64ce000640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f64b40b6680 0x7f64b40b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0095f0 tx=0x7f64bc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:40.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.748+0000 7f64c7400640 1 -- 172.21.15.27:0/1477734543 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64c00d2e20 con 0x7f64d01039b0 2024-09-21T07:16:40.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.994+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64d010ba10 con 0x7f64d01039b0 2024-09-21T07:16:40.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.995+0000 7f64c7400640 1 -- 172.21.15.27:0/1477734543 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f64c00d7d60 con 0x7f64d01039b0 2024-09-21T07:16:40.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:16:40.998 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:16:40.999 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:16:41.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f64b40b6680 msgr2=0x7f64b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:41.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f64b40b6680 0x7f64b40b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0095f0 tx=0x7f64bc005f90 comp rx=0 tx=0).stop 2024-09-21T07:16:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d01039b0 msgr2=0x7f64d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d01039b0 0x7f64d0075710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005ab0 tx=0x7f64c0003c10 comp rx=0 tx=0).stop 2024-09-21T07:16:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 shutdown_connections 2024-09-21T07:16:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64d01092f0 0x7f64d0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f64b40b6680 0x7f64b40b8b90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:41.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64d0109c40 0x7f64d0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 --2- 172.21.15.27:0/1477734543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64d01039b0 0x7f64d0075710 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 >> 172.21.15.27:0/1477734543 conn(0x7f64d00ff400 msgr2=0x7f64d0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 shutdown_connections 2024-09-21T07:16:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:40.998+0000 7f64cfe00640 1 -- 172.21.15.27:0/1477734543 wait complete. 2024-09-21T07:16:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:40 smithi153 ceph-mon[34747]: pgmap v487: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:16:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:40 smithi119 ceph-mon[34519]: pgmap v487: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:16:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:40 smithi027 ceph-mon[30779]: pgmap v487: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:16:41.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:16:42.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1477734543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:16:42.129 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1477734543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:16:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1477734543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:16:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 -- 172.21.15.27:0/1717481325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710077940 msgr2=0x7fa710077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 --2- 172.21.15.27:0/1717481325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710077940 0x7fa710077d50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa700009900 tx=0x7fa70002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 -- 172.21.15.27:0/1717481325 shutdown_connections 2024-09-21T07:16:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 --2- 172.21.15.27:0/1717481325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa710114580 0x7fa7101169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 --2- 172.21.15.27:0/1717481325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa710078290 0x7fa710114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 --2- 172.21.15.27:0/1717481325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710077940 0x7fa710077d50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 -- 172.21.15.27:0/1717481325 >> 172.21.15.27:0/1717481325 conn(0x7fa710101670 msgr2=0x7fa710103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 -- 172.21.15.27:0/1717481325 shutdown_connections 2024-09-21T07:16:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.776+0000 7fa714a00640 1 -- 172.21.15.27:0/1717481325 wait complete. 2024-09-21T07:16:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.777+0000 7fa714a00640 1 Processor -- start 2024-09-21T07:16:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.777+0000 7fa714a00640 1 -- start start 2024-09-21T07:16:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa714a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa710077940 0x7fa7101a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa714a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa710078290 0x7fa7101a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa714a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710114580 0x7fa7101a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa714a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa710119ee0 con 0x7fa710114580 2024-09-21T07:16:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa714a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa710119d60 con 0x7fa710077940 2024-09-21T07:16:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa714a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa71011a060 con 0x7fa710078290 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa70f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710114580 0x7fa7101a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa70f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710114580 0x7fa7101a61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43712/0 (socket says 172.21.15.27:43712) 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa70f200640 1 -- 172.21.15.27:0/2973243423 learned_addr learned my addr 172.21.15.27:0/2973243423 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa70e800640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa710077940 0x7fa7101a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.778+0000 7fa707e00640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa710078290 0x7fa7101a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa70e800640 1 -- 172.21.15.27:0/2973243423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa710078290 msgr2=0x7fa7101a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa70e800640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa710078290 0x7fa7101a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa70e800640 1 -- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710114580 msgr2=0x7fa7101a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa70e800640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710114580 0x7fa7101a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa70e800640 1 -- 172.21.15.27:0/2973243423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7101a6920 con 0x7fa710077940 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa70f200640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710114580 0x7fa7101a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa70e800640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa710077940 0x7fa7101a09b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa700006010 tx=0x7fa700004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa707400640 1 -- 172.21.15.27:0/2973243423 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70003f070 con 0x7fa710077940 2024-09-21T07:16:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa707400640 1 -- 172.21.15.27:0/2973243423 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa700003ca0 con 0x7fa710077940 2024-09-21T07:16:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7101a6bb0 con 0x7fa710077940 2024-09-21T07:16:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa707400640 1 -- 172.21.15.27:0/2973243423 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700035070 con 0x7fa710077940 2024-09-21T07:16:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.779+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7101ad370 con 0x7fa710077940 2024-09-21T07:16:42.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 Processor -- start 2024-09-21T07:16:42.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 -- start start 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f671070 0x563c5fa17f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f650390 0x563c5fa189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563c5f98efb0 con 0x563c5f671070 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563c5f97e3e0 con 0x563c5f6481b0 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.783+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563c5f9821f0 con 0x563c5f650390 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707400640 1 -- 172.21.15.27:0/2973243423 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa70003b460 con 0x7fa710077940 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa70f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f671070 0x563c5fa17f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f650390 0x563c5fa189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa70f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f671070 0x563c5fa17f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43728/0 (socket says 172.21.15.27:43728) 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f650390 0x563c5fa189f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38510/0 (socket says 172.21.15.27:38510) 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa70f200640 1 -- 172.21.15.27:0/3314841033 learned_addr learned my addr 172.21.15.27:0/3314841033 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa70e800640 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707e00640 1 -- 172.21.15.27:0/3314841033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 msgr2=0x563c5fa184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707e00640 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707e00640 1 -- 172.21.15.27:0/3314841033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f671070 msgr2=0x563c5fa17f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707e00640 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f671070 0x563c5fa17f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa707e00640 1 -- 172.21.15.27:0/3314841033 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5f948cd0 con 0x563c5f650390 2024-09-21T07:16:42.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.784+0000 7fa70e800640 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:42.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.785+0000 7fa707e00640 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f650390 0x563c5fa189f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa71006c610 tx=0x7fa6f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6f9c6cbd4f0f5e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:42.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.785+0000 7fa6ff400640 1 -- 172.21.15.27:0/3314841033 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8004050 con 0x563c5f650390 2024-09-21T07:16:42.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.785+0000 7fa6ff400640 1 -- 172.21.15.27:0/3314841033 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f80054d0 con 0x563c5f650390 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.785+0000 7fa6ff400640 1 -- 172.21.15.27:0/3314841033 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8004ba0 con 0x563c5f650390 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.785+0000 7fa707400640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6f40b6680 0x7fa6f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa707400640 1 -- 172.21.15.27:0/2973243423 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa700107f10 con 0x7fa710077940 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa70f200640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6f40b6680 0x7fa6f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3314841033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f650390 msgr2=0x563c5fa189f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f650390 0x563c5fa189f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa71006c610 tx=0x7fa6f8011810 comp rx=0 tx=0).stop 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3314841033 shutdown_connections 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f650390 0x563c5fa189f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3314841033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f671070 0x563c5fa17f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3314841033 >> 172.21.15.27:0/3314841033 conn(0x563c5f665460 msgr2=0x563c5fa15e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.786+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3314841033 shutdown_connections 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.787+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3314841033 wait complete. 2024-09-21T07:16:42.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.787+0000 7fa70f200640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6f40b6680 0x7fa6f40b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa708009a50 tx=0x7fa708009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:42.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.787+0000 7fa71c0c6740 1 -- start start 2024-09-21T07:16:42.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.787+0000 7fa71c0c6740 1 Processor -- start 2024-09-21T07:16:42.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa23c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa2faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563c5f7df700 con 0x563c5f650390 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563c5f919580 con 0x563c5f671070 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563c5fa26750 con 0x563c5f6481b0 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa71c0c6740 10 client.0 _ll_register_callbacks cb 0x563c5f929470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa707e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa23c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa70e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa707e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa23c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43744/0 (socket says 172.21.15.27:43744) 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa70e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa236f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38518/0 (socket says 172.21.15.27:38518) 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa707e00640 1 -- 172.21.15.27:0/1070447176 learned_addr learned my addr 172.21.15.27:0/1070447176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.788+0000 7fa70f200640 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa2faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa70e800640 1 -- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 msgr2=0x563c5fa2faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa70e800640 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa2faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa70e800640 1 -- 172.21.15.27:0/1070447176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 msgr2=0x563c5fa23c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa70e800640 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa23c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa70e800640 1 -- 172.21.15.27:0/1070447176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5f6dc650 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa70e800640 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa236f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa7001052b0 tx=0x7fa700105830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e8dec2614ea33e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa6f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1070447176 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700048070 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1070447176 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa7000cccc0 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.789+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c5f91bb90 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1070447176 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7000cd830 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa71c0c6740 10 client.26563 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c5f936950 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1070447176 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563c5f91bb90 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1070447176 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa7000cece0 con 0x563c5f6481b0 2024-09-21T07:16:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa71c0c6740 10 client.26563 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:42.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa71c0c6740 10 client.26563 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:16:42.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa71c0c6740 20 client.26563 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:42.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.790+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f62f9f0 0x563c5fa35840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.791+0000 7fa71c0c6740 4 client.26563 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:42.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.791+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c5f990f00 con 0x563c5f62f9f0 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.791+0000 7fa707e00640 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f62f9f0 0x563c5fa35840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.792+0000 7fa707e00640 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f62f9f0 0x563c5fa35840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e6ce852465f2946 server_cookie=3d0f6318862c460 in_seq=0 out_seq=0 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.792+0000 7fa6f3e00640 10 client.26563 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1070447176 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c5f990f00 con 0x563c5f62f9f0 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa6f3e00640 10 client.26563 handle_command_reply: tid=0 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 client.26563 shutdown 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 msgr2=0x563c5fa236f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa236f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa7001052b0 tx=0x7fa700105830 comp rx=0 tx=0).stop 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 shutdown_connections 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f62f9f0 0x563c5fa35840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa2faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa23c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1070447176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa236f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa6f3e00640 0 client.26563 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.793+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 >> 172.21.15.27:0/1070447176 conn(0x563c5f665460 msgr2=0x563c5fa1b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.794+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 shutdown_connections 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.794+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1070447176 wait complete. 2024-09-21T07:16:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.794+0000 7fa71c0c6740 20 client.26563 trim_cache size 0 max 16384 2024-09-21T07:16:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 Processor -- start 2024-09-21T07:16:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 -- start start 2024-09-21T07:16:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa14d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f671070 0x563c5fa20e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b9a70 con 0x563c5f650390 2024-09-21T07:16:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b7340 con 0x563c5f6481b0 2024-09-21T07:16:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.808+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b5190 con 0x563c5f671070 2024-09-21T07:16:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa707e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa14d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa707e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa208e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43750/0 (socket says 172.21.15.27:43750) 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa14d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36012/0 (socket says 172.21.15.27:36012) 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa707e00640 1 -- 172.21.15.27:0/3727535404 learned_addr learned my addr 172.21.15.27:0/3727535404 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70f200640 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f671070 0x563c5fa20e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70e800640 1 -- 172.21.15.27:0/3727535404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f671070 msgr2=0x563c5fa20e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70e800640 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f671070 0x563c5fa20e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70e800640 1 -- 172.21.15.27:0/3727535404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 msgr2=0x563c5fa208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70e800640 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa70e800640 1 -- 172.21.15.27:0/3727535404 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5fa163a0 con 0x563c5f6481b0 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.809+0000 7fa707e00640 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:42.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.810+0000 7fa70e800640 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa14d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa70002a910 tx=0x7fa7000c92a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4752f3ba10b3f186 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:42.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.810+0000 7fa6fea00640 1 -- 172.21.15.27:0/3727535404 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700048070 con 0x563c5f6481b0 2024-09-21T07:16:42.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.810+0000 7fa6fea00640 1 -- 172.21.15.27:0/3727535404 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7000d2190 con 0x563c5f6481b0 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.810+0000 7fa6fea00640 1 -- 172.21.15.27:0/3727535404 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7000cd1d0 con 0x563c5f6481b0 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3727535404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 msgr2=0x563c5fa14d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa14d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa70002a910 tx=0x7fa7000c92a0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3727535404 shutdown_connections 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f671070 0x563c5fa20e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/3727535404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f6481b0 0x563c5fa14d10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3727535404 >> 172.21.15.27:0/3727535404 conn(0x563c5f665460 msgr2=0x563c5fa14600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3727535404 shutdown_connections 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 -- 172.21.15.27:0/3727535404 wait complete. 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.811+0000 7fa71c0c6740 1 -- start start 2024-09-21T07:16:42.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 1 Processor -- start 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa2c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa26f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b5800 con 0x563c5f650390 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b6c70 con 0x563c5f671070 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b5ed0 con 0x563c5f6481b0 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa71c0c6740 10 client.0 _ll_register_callbacks cb 0x7fa7000c9f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa707e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa70e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa2c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa707e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43756/0 (socket says 172.21.15.27:43756) 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.812+0000 7fa707e00640 1 -- 172.21.15.27:0/1285770445 learned_addr learned my addr 172.21.15.27:0/1285770445 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa70f200640 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa26f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa707e00640 1 -- 172.21.15.27:0/1285770445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 msgr2=0x563c5fa2c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa707e00640 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa2c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa707e00640 1 -- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 msgr2=0x563c5fa26f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa707e00640 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa26f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa707e00640 1 -- 172.21.15.27:0/1285770445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5fa27540 con 0x563c5f650390 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa70f200640 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa707e00640 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa269c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80099b0 tx=0x7fa6f8009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27666cd74d4e26dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.813+0000 7fa6f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.814+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1285770445 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f801d040 con 0x563c5f650390 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.814+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1285770445 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa6f8004150 con 0x563c5f650390 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.814+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7000d2190 con 0x563c5f650390 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.814+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1285770445 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6f8039050 con 0x563c5f650390 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.814+0000 7fa71c0c6740 10 client.16545 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.814+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c5faa1000 con 0x563c5f650390 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.814+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1285770445 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7000d2190 con 0x563c5f650390 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1285770445 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa6f80354d0 con 0x563c5f650390 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa71c0c6740 10 client.16545 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa71c0c6740 10 client.16545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa71c0c6740 20 client.16545 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f604da0 0x563c5faa3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa71c0c6740 4 client.16545 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c5f98c9b0 con 0x563c5f604da0 2024-09-21T07:16:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.815+0000 7fa70e800640 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f604da0 0x563c5faa3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa70e800640 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f604da0 0x563c5faa3600 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e76cc3f78dde688 server_cookie=a22a7ae99d2fb787 in_seq=0 out_seq=0 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa6f3e00640 10 client.16545 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa6f3e00640 1 -- 172.21.15.27:0/1285770445 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c5f98c9b0 con 0x563c5f604da0 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa6f3e00640 10 client.16545 handle_command_reply: tid=0 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 client.16545 shutdown 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 msgr2=0x563c5fa269c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa269c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80099b0 tx=0x7fa6f8009e00 comp rx=0 tx=0).stop 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 shutdown_connections 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x563c5f604da0 0x563c5faa3600 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5f671070 0x563c5fa26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5f650390 0x563c5fa269c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa6f3e00640 0 client.16545 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 --2- 172.21.15.27:0/1285770445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5f6481b0 0x563c5fa2c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 >> 172.21.15.27:0/1285770445 conn(0x563c5f665460 msgr2=0x563c5fa1b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.816+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 shutdown_connections 2024-09-21T07:16:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.817+0000 7fa71c0c6740 1 -- 172.21.15.27:0/1285770445 wait complete. 2024-09-21T07:16:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.817+0000 7fa71c0c6740 20 client.16545 trim_cache size 0 max 16384 2024-09-21T07:16:42.821 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.817+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6f40b6680 msgr2=0x7fa6f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.817+0000 7fa714a00640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6f40b6680 0x7fa6f40b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa708009a50 tx=0x7fa708009760 comp rx=0 tx=0).stop 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.817+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa710077940 msgr2=0x7fa7101a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.817+0000 7fa714a00640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa710077940 0x7fa7101a09b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa700006010 tx=0x7fa700004060 comp rx=0 tx=0).stop 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 shutdown_connections 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa710078290 0x7fa7101a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa6f40b6680 0x7fa6f40b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa710114580 0x7fa7101a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 --2- 172.21.15.27:0/2973243423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa710077940 0x7fa7101a09b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 >> 172.21.15.27:0/2973243423 conn(0x7fa710101670 msgr2=0x7fa710116690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 shutdown_connections 2024-09-21T07:16:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:42.818+0000 7fa714a00640 1 -- 172.21.15.27:0/2973243423 wait complete. 2024-09-21T07:16:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:42 smithi153 ceph-mon[34747]: pgmap v488: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-21T07:16:43.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:42 smithi027 ceph-mon[30779]: pgmap v488: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-21T07:16:43.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:42 smithi119 ceph-mon[34519]: pgmap v488: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-21T07:16:43.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.760+0000 7fe74b800640 1 -- 172.21.15.27:0/2770793379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744103b00 msgr2=0x7fe744103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.760+0000 7fe74b800640 1 --2- 172.21.15.27:0/2770793379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744103b00 0x7fe744103f10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe738009860 tx=0x7fe73802a340 comp rx=0 tx=0).stop 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.761+0000 7fe74b800640 1 -- 172.21.15.27:0/2770793379 shutdown_connections 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.761+0000 7fe74b800640 1 --2- 172.21.15.27:0/2770793379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744109d80 0x7fe7441105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.761+0000 7fe74b800640 1 --2- 172.21.15.27:0/2770793379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744109430 0x7fe744109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.761+0000 7fe74b800640 1 --2- 172.21.15.27:0/2770793379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744103b00 0x7fe744103f10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.761+0000 7fe74b800640 1 -- 172.21.15.27:0/2770793379 >> 172.21.15.27:0/2770793379 conn(0x7fe7440ff500 msgr2=0x7fe744101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.761+0000 7fe74b800640 1 -- 172.21.15.27:0/2770793379 shutdown_connections 2024-09-21T07:16:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.761+0000 7fe74b800640 1 -- 172.21.15.27:0/2770793379 wait complete. 2024-09-21T07:16:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 Processor -- start 2024-09-21T07:16:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 -- start start 2024-09-21T07:16:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b00 0x7fe74419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744109430 0x7fe74419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744109d80 0x7fe7441a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe744113a40 con 0x7fe744103b00 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441138c0 con 0x7fe744109d80 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.762+0000 7fe74b800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe744113bc0 con 0x7fe744109430 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b00 0x7fe74419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b00 0x7fe74419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43776/0 (socket says 172.21.15.27:43776) 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 -- 172.21.15.27:0/1258259113 learned_addr learned my addr 172.21.15.27:0/1258259113 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe743e00640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744109430 0x7fe74419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe749800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744109d80 0x7fe7441a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 -- 172.21.15.27:0/1258259113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744109430 msgr2=0x7fe74419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744109430 0x7fe74419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 -- 172.21.15.27:0/1258259113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744109d80 msgr2=0x7fe7441a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744109d80 0x7fe7441a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.763+0000 7fe748e00640 1 -- 172.21.15.27:0/1258259113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7441a2650 con 0x7fe744103b00 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.764+0000 7fe748e00640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b00 0x7fe74419c720 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe738035040 tx=0x7fe738032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.764+0000 7fe741600640 1 -- 172.21.15.27:0/1258259113 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738037040 con 0x7fe744103b00 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.764+0000 7fe741600640 1 -- 172.21.15.27:0/1258259113 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe738041070 con 0x7fe744103b00 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.764+0000 7fe741600640 1 -- 172.21.15.27:0/1258259113 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738003850 con 0x7fe744103b00 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.764+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7441a28e0 con 0x7fe744103b00 2024-09-21T07:16:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.764+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7441a2d70 con 0x7fe744103b00 2024-09-21T07:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 Processor -- start 2024-09-21T07:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 -- start start 2024-09-21T07:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad07ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad0800170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad043a4d0 0x55cad08006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cad07756b0 con 0x55cad043a4d0 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cad0764ae0 con 0x55cad04322f0 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.768+0000 7fe75286f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cad07688f0 con 0x55cad045b1b0 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad07ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe743e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad043a4d0 0x55cad08006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad07ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38564/0 (socket says 172.21.15.27:38564) 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 -- 172.21.15.27:0/1352343903 learned_addr learned my addr 172.21.15.27:0/1352343903 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 -- 172.21.15.27:0/1352343903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 msgr2=0x55cad0800170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad0800170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 -- 172.21.15.27:0/1352343903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad043a4d0 msgr2=0x55cad08006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad043a4d0 0x55cad08006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 -- 172.21.15.27:0/1352343903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad0731780 con 0x55cad045b1b0 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe743e00640 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad043a4d0 0x55cad08006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.769+0000 7fe749800640 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad07ffc30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009f80 tx=0x7fe73c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18a06ce6b61a0ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.770+0000 7fe736000640 1 -- 172.21.15.27:0/1352343903 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c004b20 con 0x55cad045b1b0 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.770+0000 7fe736000640 1 -- 172.21.15.27:0/1352343903 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73c004cc0 con 0x55cad045b1b0 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.770+0000 7fe736000640 1 -- 172.21.15.27:0/1352343903 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c005330 con 0x55cad045b1b0 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.770+0000 7fe741600640 1 -- 172.21.15.27:0/1258259113 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe7380039f0 con 0x7fe744103b00 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 -- 172.21.15.27:0/1352343903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 msgr2=0x55cad07ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad07ffc30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009f80 tx=0x7fe73c0118d0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 -- 172.21.15.27:0/1352343903 shutdown_connections 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad043a4d0 0x55cad08006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad0800170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 --2- 172.21.15.27:0/1352343903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad07ffc30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 -- 172.21.15.27:0/1352343903 >> 172.21.15.27:0/1352343903 conn(0x55cad044f300 msgr2=0x55cad07fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 -- 172.21.15.27:0/1352343903 shutdown_connections 2024-09-21T07:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 -- 172.21.15.27:0/1352343903 wait complete. 2024-09-21T07:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.771+0000 7fe75286f740 1 -- start start 2024-09-21T07:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 1 Processor -- start 2024-09-21T07:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad080b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad043a4d0 0x55cad080b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad08176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cad069cab0 con 0x55cad045b1b0 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cad070b570 con 0x55cad04322f0 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cad080e3b0 con 0x55cad043a4d0 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.772+0000 7fe75286f740 10 client.0 _ll_register_callbacks cb 0x55cad071a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe741600640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe72c0b6590 0x7fe72c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe748e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad043a4d0 0x55cad080b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe749800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad080b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe743e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad08176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe749800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad080b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36044/0 (socket says 172.21.15.27:36044) 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe749800640 1 -- 172.21.15.27:0/2171474968 learned_addr learned my addr 172.21.15.27:0/2171474968 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe749800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe72c0b6590 0x7fe72c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe743e00640 1 -- 172.21.15.27:0/2171474968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad043a4d0 msgr2=0x55cad080b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe741600640 1 -- 172.21.15.27:0/1258259113 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe738107480 con 0x7fe744103b00 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe743e00640 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad043a4d0 0x55cad080b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe743e00640 1 -- 172.21.15.27:0/2171474968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 msgr2=0x55cad080b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe743e00640 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad080b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe743e00640 1 -- 172.21.15.27:0/2171474968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad04c6790 con 0x55cad045b1b0 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.773+0000 7fe748e00640 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad043a4d0 0x55cad080b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe743e00640 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad08176d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe730009530 tx=0x7fe730011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5da82e6c23bbd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe72aa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe749800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe72c0b6590 0x7fe72c0b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000f80 tx=0x7fe73c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe72aa00640 1 -- 172.21.15.27:0/2171474968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73001e060 con 0x55cad045b1b0 2024-09-21T07:16:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe72aa00640 1 -- 172.21.15.27:0/2171474968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe73000fbc0 con 0x55cad045b1b0 2024-09-21T07:16:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe73c004cc0 con 0x55cad045b1b0 2024-09-21T07:16:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe72aa00640 1 -- 172.21.15.27:0/2171474968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe73000fdc0 con 0x55cad045b1b0 2024-09-21T07:16:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe75286f740 10 client.16557 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.774+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cad053f620 con 0x55cad045b1b0 2024-09-21T07:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe72aa00640 1 -- 172.21.15.27:0/2171474968 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73c004cc0 con 0x55cad045b1b0 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe72aa00640 1 -- 172.21.15.27:0/2171474968 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe730019070 con 0x55cad045b1b0 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe75286f740 10 client.16557 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe75286f740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe75286f740 20 client.16557 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe75286f740 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad0419d80 0x55cad081d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe75286f740 4 client.16557 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.775+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad0777600 con 0x55cad0419d80 2024-09-21T07:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.776+0000 7fe748e00640 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad0419d80 0x55cad081d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.776+0000 7fe72aa00640 10 client.16557 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.776+0000 7fe748e00640 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad0419d80 0x55cad081d4b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7cd15c6b3703c50 server_cookie=89632c35764bfd3 in_seq=0 out_seq=0 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe72aa00640 1 -- 172.21.15.27:0/2171474968 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cad0777600 con 0x55cad0419d80 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe72aa00640 10 client.16557 handle_command_reply: tid=0 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 client.16557 shutdown 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 msgr2=0x55cad08176d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad08176d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe730009530 tx=0x7fe730011b30 comp rx=0 tx=0).stop 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 shutdown_connections 2024-09-21T07:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad0419d80 0x55cad081d4b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad043a4d0 0x55cad080b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad08176d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 --2- 172.21.15.27:0/2171474968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad04322f0 0x55cad080b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 >> 172.21.15.27:0/2171474968 conn(0x55cad044f300 msgr2=0x55cad0800c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.777+0000 7fe72aa00640 0 client.16557 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.778+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 shutdown_connections 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.778+0000 7fe75286f740 1 -- 172.21.15.27:0/2171474968 wait complete. 2024-09-21T07:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.778+0000 7fe75286f740 20 client.16557 trim_cache size 0 max 16384 2024-09-21T07:16:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 Processor -- start 2024-09-21T07:16:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 -- start start 2024-09-21T07:16:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad04322f0 0x55cad07fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0801370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad08175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b5960 con 0x55cad04322f0 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b3230 con 0x55cad043a4d0 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.795+0000 7fe75286f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b1080 con 0x55cad045b1b0 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe748e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad04322f0 0x55cad07fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0801370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe749800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad08175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe748e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad04322f0 0x55cad07fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43812/0 (socket says 172.21.15.27:43812) 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe748e00640 1 -- 172.21.15.27:0/2840860653 learned_addr learned my addr 172.21.15.27:0/2840860653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe743e00640 1 -- 172.21.15.27:0/2840860653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 msgr2=0x55cad08175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe743e00640 1 --2- 172.21.15.27:0/2840860653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad08175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe743e00640 1 -- 172.21.15.27:0/2840860653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad04322f0 msgr2=0x55cad07fdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe743e00640 1 --2- 172.21.15.27:0/2840860653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad04322f0 0x55cad07fdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.796+0000 7fe743e00640 1 -- 172.21.15.27:0/2840860653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad0809750 con 0x55cad043a4d0 2024-09-21T07:16:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.797+0000 7fe743e00640 1 --2- 172.21.15.27:0/2840860653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0801370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe730003ed0 tx=0x7fe73000fd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9d02b271c45303e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.797+0000 7fe735600640 1 -- 172.21.15.27:0/2840860653 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73002e020 con 0x55cad043a4d0 2024-09-21T07:16:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.797+0000 7fe735600640 1 -- 172.21.15.27:0/2840860653 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe730004480 con 0x55cad043a4d0 2024-09-21T07:16:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.797+0000 7fe735600640 1 -- 172.21.15.27:0/2840860653 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73001a530 con 0x55cad043a4d0 2024-09-21T07:16:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 -- 172.21.15.27:0/2840860653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 msgr2=0x55cad0801370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 --2- 172.21.15.27:0/2840860653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0801370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe730003ed0 tx=0x7fe73000fd50 comp rx=0 tx=0).stop 2024-09-21T07:16:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 -- 172.21.15.27:0/2840860653 shutdown_connections 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 --2- 172.21.15.27:0/2840860653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad045b1b0 0x55cad08175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 --2- 172.21.15.27:0/2840860653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0801370 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 --2- 172.21.15.27:0/2840860653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad04322f0 0x55cad07fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 -- 172.21.15.27:0/2840860653 >> 172.21.15.27:0/2840860653 conn(0x55cad044f300 msgr2=0x55cad0800c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 -- 172.21.15.27:0/2840860653 shutdown_connections 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 -- 172.21.15.27:0/2840860653 wait complete. 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.798+0000 7fe75286f740 1 -- start start 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 1 Processor -- start 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad04322f0 0x55cad08109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0810f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad0885de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b16f0 con 0x55cad045b1b0 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b2b60 con 0x55cad043a4d0 2024-09-21T07:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b1dc0 con 0x55cad04322f0 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.799+0000 7fe75286f740 10 client.0 _ll_register_callbacks cb 0x7fe730009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad04322f0 0x55cad08109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0810f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe749800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad0885de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad04322f0 0x55cad08109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38592/0 (socket says 172.21.15.27:38592) 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 -- 172.21.15.27:0/1475702786 learned_addr learned my addr 172.21.15.27:0/1475702786 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 -- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 msgr2=0x55cad0810f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0810f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 -- 172.21.15.27:0/1475702786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 msgr2=0x55cad0885de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad0885de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 -- 172.21.15.27:0/1475702786 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad0886320 con 0x55cad04322f0 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe743e00640 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0810f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe748e00640 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad04322f0 0x55cad08109e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ca1f0 tx=0x7fe73802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46528c9a91d102e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.800+0000 7fe72aa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.801+0000 7fe72aa00640 1 -- 172.21.15.27:0/1475702786 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7380c8e20 con 0x55cad04322f0 2024-09-21T07:16:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.801+0000 7fe72aa00640 1 -- 172.21.15.27:0/1475702786 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe73810ad30 con 0x55cad04322f0 2024-09-21T07:16:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.801+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cad07730b0 con 0x55cad04322f0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.801+0000 7fe72aa00640 1 -- 172.21.15.27:0/1475702786 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7380ccd50 con 0x55cad04322f0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.801+0000 7fe75286f740 10 client.26587 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.801+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cad08874b0 con 0x55cad04322f0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.801+0000 7fe72aa00640 1 -- 172.21.15.27:0/1475702786 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cad07730b0 con 0x55cad04322f0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe72aa00640 1 -- 172.21.15.27:0/1475702786 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7380b8080 con 0x55cad04322f0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe75286f740 10 client.26587 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe75286f740 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe75286f740 20 client.26587 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe75286f740 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad03ef1c0 0x55cad0889c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe75286f740 4 client.26587 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cad07704e0 con 0x55cad03ef1c0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.802+0000 7fe743e00640 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad03ef1c0 0x55cad0889c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.803+0000 7fe743e00640 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad03ef1c0 0x55cad0889c20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a2ae97cc6050dc server_cookie=11a9da1e85b6fc16 in_seq=0 out_seq=0 2024-09-21T07:16:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.803+0000 7fe72aa00640 10 client.26587 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe72aa00640 1 -- 172.21.15.27:0/1475702786 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cad07704e0 con 0x55cad03ef1c0 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe72aa00640 10 client.26587 handle_command_reply: tid=0 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 client.26587 shutdown 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad04322f0 msgr2=0x55cad08109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad04322f0 0x55cad08109e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ca1f0 tx=0x7fe73802a340 comp rx=0 tx=0).stop 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 shutdown_connections 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cad03ef1c0 0x55cad0889c20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cad045b1b0 0x55cad0885de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cad043a4d0 0x55cad0810f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe72aa00640 0 client.26587 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 --2- 172.21.15.27:0/1475702786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cad04322f0 0x55cad08109e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 >> 172.21.15.27:0/1475702786 conn(0x55cad044f300 msgr2=0x55cad07fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.816+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 shutdown_connections 2024-09-21T07:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe75286f740 1 -- 172.21.15.27:0/1475702786 wait complete. 2024-09-21T07:16:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe75286f740 20 client.26587 trim_cache size 0 max 16384 2024-09-21T07:16:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:16:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:16:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:16:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe72c0b6590 msgr2=0x7fe72c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe72c0b6590 0x7fe72c0b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000f80 tx=0x7fe73c009500 comp rx=0 tx=0).stop 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b00 msgr2=0x7fe74419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b00 0x7fe74419c720 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe738035040 tx=0x7fe738032d50 comp rx=0 tx=0).stop 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 shutdown_connections 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe744109430 0x7fe74419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe72c0b6590 0x7fe72c0b8aa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744109d80 0x7fe7441a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 --2- 172.21.15.27:0/1258259113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b00 0x7fe74419c720 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.817+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 >> 172.21.15.27:0/1258259113 conn(0x7fe7440ff500 msgr2=0x7fe744108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.818+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 shutdown_connections 2024-09-21T07:16:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:43.818+0000 7fe74b800640 1 -- 172.21.15.27:0/1258259113 wait complete. 2024-09-21T07:16:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:44.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 -- 172.21.15.27:0/1727527546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 msgr2=0x7f9b0010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1727527546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 0x7f9b0010bb00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009900 tx=0x7f9ae802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 -- 172.21.15.27:0/1727527546 shutdown_connections 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1727527546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b0010c040 0x7f9b00112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1727527546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 0x7f9b0010bb00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1727527546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 0x7f9b001061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 -- 172.21.15.27:0/1727527546 >> 172.21.15.27:0/1727527546 conn(0x7f9b001017e0 msgr2=0x7f9b00103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 -- 172.21.15.27:0/1727527546 shutdown_connections 2024-09-21T07:16:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.577+0000 7f9b04a00640 1 -- 172.21.15.27:0/1727527546 wait complete. 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.578+0000 7f9b04a00640 1 Processor -- start 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.578+0000 7f9b04a00640 1 -- start start 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9b04a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 0x7f9b0019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9b04a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 0x7f9b0019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9b04a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b0010c040 0x7f9b001a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9b04a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00115d00 con 0x7f9b0010b6f0 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9b04a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00115b80 con 0x7f9b00105db0 2024-09-21T07:16:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9b04a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00115e80 con 0x7f9b0010c040 2024-09-21T07:16:44.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9afde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 0x7f9b0019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9af5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 0x7f9b0019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9afde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 0x7f9b0019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36084/0 (socket says 172.21.15.27:36084) 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.579+0000 7f9afde00640 1 -- 172.21.15.27:0/1421731358 learned_addr learned my addr 172.21.15.27:0/1421731358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.580+0000 7f9afe800640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b0010c040 0x7f9b001a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.580+0000 7f9afde00640 1 -- 172.21.15.27:0/1421731358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b0010c040 msgr2=0x7f9b001a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.580+0000 7f9afde00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b0010c040 0x7f9b001a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.580+0000 7f9afde00640 1 -- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 msgr2=0x7f9b0019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.580+0000 7f9afde00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 0x7f9b0019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.580+0000 7f9afde00640 1 -- 172.21.15.27:0/1421731358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b001a2680 con 0x7f9b00105db0 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.580+0000 7f9afde00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 0x7f9b0019c710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9af000e9f0 tx=0x7f9af000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.581+0000 7f9af7400640 1 -- 172.21.15.27:0/1421731358 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af000cd80 con 0x7f9b00105db0 2024-09-21T07:16:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.581+0000 7f9af7400640 1 -- 172.21.15.27:0/1421731358 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af00108e0 con 0x7f9b00105db0 2024-09-21T07:16:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.581+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b001a2970 con 0x7f9b00105db0 2024-09-21T07:16:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.581+0000 7f9af7400640 1 -- 172.21.15.27:0/1421731358 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0018980 con 0x7f9b00105db0 2024-09-21T07:16:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.581+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b001a90d0 con 0x7f9b00105db0 2024-09-21T07:16:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.584+0000 7f9b0b619740 1 Processor -- start 2024-09-21T07:16:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.584+0000 7f9b0b619740 1 -- start start 2024-09-21T07:16:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.584+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad263da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.584+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad2642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.584+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad264820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.584+0000 7f9b0b619740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605ad1dafb0 con 0x5605ace9c390 2024-09-21T07:16:44.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.584+0000 7f9b0b619740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5605ad1ca3e0 con 0x5605ace941b0 2024-09-21T07:16:44.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9b0b619740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605ad1ce1f0 con 0x5605acebd070 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9af5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad264820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9afe800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad263da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9af5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad264820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43840/0 (socket says 172.21.15.27:43840) 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9af5400640 1 -- 172.21.15.27:0/1434160733 learned_addr learned my addr 172.21.15.27:0/1434160733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9afe800640 1 -- 172.21.15.27:0/1434160733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 msgr2=0x5605ad2642e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9afe800640 1 --2- 172.21.15.27:0/1434160733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad2642e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9afe800640 1 -- 172.21.15.27:0/1434160733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 msgr2=0x5605ad264820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9afe800640 1 --2- 172.21.15.27:0/1434160733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad264820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.585+0000 7f9afe800640 1 -- 172.21.15.27:0/1434160733 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605ad194cd0 con 0x5605acebd070 2024-09-21T07:16:44.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.586+0000 7f9af7400640 1 -- 172.21.15.27:0/1421731358 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9af0010430 con 0x7f9b00105db0 2024-09-21T07:16:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.586+0000 7f9afe800640 1 --2- 172.21.15.27:0/1434160733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad263da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009f00 tx=0x7f9af8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8c7cdfea82eda09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.586+0000 7f9aef000640 1 -- 172.21.15.27:0/1434160733 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8004d20 con 0x5605acebd070 2024-09-21T07:16:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.586+0000 7f9aef000640 1 -- 172.21.15.27:0/1434160733 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af8004050 con 0x5605acebd070 2024-09-21T07:16:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.586+0000 7f9aef000640 1 -- 172.21.15.27:0/1434160733 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8005280 con 0x5605acebd070 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 -- 172.21.15.27:0/1434160733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 msgr2=0x5605ad263da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1434160733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad263da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009f00 tx=0x7f9af8011810 comp rx=0 tx=0).stop 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 -- 172.21.15.27:0/1434160733 shutdown_connections 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1434160733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad264820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1434160733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad2642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1434160733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad263da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 -- 172.21.15.27:0/1434160733 >> 172.21.15.27:0/1434160733 conn(0x5605aceb1460 msgr2=0x5605ad261c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 -- 172.21.15.27:0/1434160733 shutdown_connections 2024-09-21T07:16:44.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9b0b619740 1 -- 172.21.15.27:0/1434160733 wait complete. 2024-09-21T07:16:44.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.587+0000 7f9af7400640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9ae40b6610 0x7f9ae40b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.588+0000 7f9af7400640 1 -- 172.21.15.27:0/1421731358 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9af00e7d80 con 0x7f9b00105db0 2024-09-21T07:16:44.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.588+0000 7f9af5400640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9ae40b6610 0x7f9ae40b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.588+0000 7f9b0b619740 1 -- start start 2024-09-21T07:16:44.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.588+0000 7f9b0b619740 1 Processor -- start 2024-09-21T07:16:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9af5400640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9ae40b6610 0x7f9ae40b8b20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8005f70 tx=0x7f9ae8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace941b0 0x5605ad26f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace9c390 0x5605ad26fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad27b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9b0b619740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605ad02b700 con 0x5605ace941b0 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9b0b619740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5605ad165580 con 0x5605acebd070 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9b0b619740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605ad272580 con 0x5605ace9c390 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9b0b619740 10 client.0 _ll_register_callbacks cb 0x5605ad175470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9afde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace941b0 0x5605ad26f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9afde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace941b0 0x5605ad26f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43844/0 (socket says 172.21.15.27:43844) 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9afde00640 1 -- 172.21.15.27:0/2586210456 learned_addr learned my addr 172.21.15.27:0/2586210456 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9af5400640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace9c390 0x5605ad26fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.589+0000 7f9afe800640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad27b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9afde00640 1 -- 172.21.15.27:0/2586210456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace9c390 msgr2=0x5605ad26fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9afde00640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace9c390 0x5605ad26fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9afde00640 1 -- 172.21.15.27:0/2586210456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 msgr2=0x5605ad27b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9afde00640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad27b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9afde00640 1 -- 172.21.15.27:0/2586210456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605acf28650 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9afe800640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad27b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9afde00640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace941b0 0x5605ad26f530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9af00ad5d0 tx=0x7f9af00ad760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f64d9adae9244b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9ae3400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9ae3400640 1 -- 172.21.15.27:0/2586210456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0010720 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9ae3400640 1 -- 172.21.15.27:0/2586210456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9af00aeb50 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.590+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605ad167b90 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9ae3400640 1 -- 172.21.15.27:0/2586210456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9af00ad3e0 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 10 client.16575 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605ad182950 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9ae3400640 1 -- 172.21.15.27:0/2586210456 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5605ad167b90 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9ae3400640 1 -- 172.21.15.27:0/2586210456 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9af00ac5a0 con 0x5605ace941b0 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 10 client.16575 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 10 client.16575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:16:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 20 client.16575 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:44.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace7b9f0 0x5605ad2816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 4 client.16575 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:44.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.591+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605ad1dcf00 con 0x5605ace7b9f0 2024-09-21T07:16:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.592+0000 7f9af5400640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace7b9f0 0x5605ad2816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.592+0000 7f9af5400640 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace7b9f0 0x5605ad2816d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ac5f4ba29776d2 server_cookie=ce4a612fb13eca52 in_seq=0 out_seq=0 2024-09-21T07:16:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.592+0000 7f9ae3400640 10 client.16575 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9ae3400640 1 -- 172.21.15.27:0/2586210456 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605ad1dcf00 con 0x5605ace7b9f0 2024-09-21T07:16:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9ae3400640 10 client.16575 handle_command_reply: tid=0 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 client.16575 shutdown 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace941b0 msgr2=0x5605ad26f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace941b0 0x5605ad26f530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9af00ad5d0 tx=0x7f9af00ad760 comp rx=0 tx=0).stop 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 shutdown_connections 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace7b9f0 0x5605ad2816d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace9c390 0x5605ad26fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad27b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2586210456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace941b0 0x5605ad26f530 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.593+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 >> 172.21.15.27:0/2586210456 conn(0x5605aceb1460 msgr2=0x5605ad266f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.594+0000 7f9ae3400640 0 client.16575 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.594+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 shutdown_connections 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.594+0000 7f9b0b619740 1 -- 172.21.15.27:0/2586210456 wait complete. 2024-09-21T07:16:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.594+0000 7f9b0b619740 20 client.16575 trim_cache size 0 max 16384 2024-09-21T07:16:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 Processor -- start 2024-09-21T07:16:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 -- start start 2024-09-21T07:16:44.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace941b0 0x5605ad271160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad26c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad26cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001b58b0 con 0x5605ace9c390 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001b3180 con 0x5605acebd070 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.608+0000 7f9b0b619740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001b0fd0 con 0x5605ace941b0 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad26cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9af5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad26c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad26cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36098/0 (socket says 172.21.15.27:36098) 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afe800640 1 -- 172.21.15.27:0/1820255504 learned_addr learned my addr 172.21.15.27:0/1820255504 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:44.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace941b0 0x5605ad271160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afde00640 1 -- 172.21.15.27:0/1820255504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 msgr2=0x5605ad26cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:44.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afde00640 1 --2- 172.21.15.27:0/1820255504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad26cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afde00640 1 -- 172.21.15.27:0/1820255504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 msgr2=0x5605ad26c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:44.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afde00640 1 --2- 172.21.15.27:0/1820255504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad26c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afde00640 1 -- 172.21.15.27:0/1820255504 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605ad266c00 con 0x5605ace941b0 2024-09-21T07:16:44.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.609+0000 7f9afde00640 1 --2- 172.21.15.27:0/1820255504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace941b0 0x5605ad271160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9af00ad860 tx=0x7f9af00ad7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2197bf0d5ca18683 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:44.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.610+0000 7f9aee600640 1 -- 172.21.15.27:0/1820255504 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b2300 con 0x5605ace941b0 2024-09-21T07:16:44.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.610+0000 7f9aee600640 1 -- 172.21.15.27:0/1820255504 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af00b7240 con 0x5605ace941b0 2024-09-21T07:16:44.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.610+0000 7f9aee600640 1 -- 172.21.15.27:0/1820255504 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b36f0 con 0x5605ace941b0 2024-09-21T07:16:44.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 -- 172.21.15.27:0/1820255504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace941b0 msgr2=0x5605ad271160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1820255504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace941b0 0x5605ad271160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9af00ad860 tx=0x7f9af00ad7d0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 -- 172.21.15.27:0/1820255504 shutdown_connections 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1820255504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605acebd070 0x5605ad26cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1820255504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad26c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 --2- 172.21.15.27:0/1820255504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ace941b0 0x5605ad271160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 -- 172.21.15.27:0/1820255504 >> 172.21.15.27:0/1820255504 conn(0x5605aceb1460 msgr2=0x5605ad270a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 -- 172.21.15.27:0/1820255504 shutdown_connections 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.611+0000 7f9b0b619740 1 -- 172.21.15.27:0/1820255504 wait complete. 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 -- start start 2024-09-21T07:16:44.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 Processor -- start 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad275270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad275850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad2e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001b1640 con 0x5605ace9c390 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001b2ab0 con 0x5605ace941b0 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001b1d10 con 0x5605acebd070 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.612+0000 7f9b0b619740 10 client.0 _ll_register_callbacks cb 0x7f9af00a97e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad275270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad275270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36102/0 (socket says 172.21.15.27:36102) 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 -- 172.21.15.27:0/2669585811 learned_addr learned my addr 172.21.15.27:0/2669585811 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9af5400640 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad275850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afe800640 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad2e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 -- 172.21.15.27:0/2669585811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 msgr2=0x5605ad2e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad2e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 -- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 msgr2=0x5605ad275850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad275850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 -- 172.21.15.27:0/2669585811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605ad2e3690 con 0x5605ace941b0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9afde00640 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad275270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9af00097b0 tx=0x7f9af00aed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ee2d3ea6025f5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.613+0000 7f9ae3400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.614+0000 7f9ae3400640 1 -- 172.21.15.27:0/2669585811 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b2300 con 0x5605ace941b0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.614+0000 7f9ae3400640 1 -- 172.21.15.27:0/2669585811 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9af00b3490 con 0x5605ace941b0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.614+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af00b7240 con 0x5605ace941b0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.614+0000 7f9ae3400640 1 -- 172.21.15.27:0/2669585811 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9af00b3690 con 0x5605ace941b0 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.614+0000 7f9b0b619740 10 client.26618 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.614+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605ad2e3690 con 0x5605ace941b0 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.614+0000 7f9ae3400640 1 -- 172.21.15.27:0/2669585811 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af00b7240 con 0x5605ace941b0 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9ae3400640 1 -- 172.21.15.27:0/2669585811 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9af00c3c40 con 0x5605ace941b0 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9b0b619740 10 client.26618 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9b0b619740 10 client.26618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9b0b619740 20 client.26618 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace50da0 0x5605ad2edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9b0b619740 4 client.26618 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:16:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605ad1d89b0 con 0x5605ace50da0 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.615+0000 7f9af5400640 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace50da0 0x5605ad2edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9ae3400640 10 client.26618 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9af5400640 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace50da0 0x5605ad2edeb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e04df2b8b9e224b server_cookie=dec2f94aa9f0e5ab in_seq=0 out_seq=0 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9ae3400640 1 -- 172.21.15.27:0/2669585811 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605ad1d89b0 con 0x5605ace50da0 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9ae3400640 10 client.26618 handle_command_reply: tid=0 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 client.26618 shutdown 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 msgr2=0x5605ad275270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad275270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9af00097b0 tx=0x7f9af00aed60 comp rx=0 tx=0).stop 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 shutdown_connections 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5605ace50da0 0x5605ad2edeb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605acebd070 0x5605ad2e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605ace9c390 0x5605ad275850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 --2- 172.21.15.27:0/2669585811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605ace941b0 0x5605ad275270 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9ae3400640 0 client.26618 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:16:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 >> 172.21.15.27:0/2669585811 conn(0x5605aceb1460 msgr2=0x5605ad270900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 shutdown_connections 2024-09-21T07:16:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.616+0000 7f9b0b619740 1 -- 172.21.15.27:0/2669585811 wait complete. 2024-09-21T07:16:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b0b619740 20 client.26618 trim_cache size 0 max 16384 2024-09-21T07:16:44.621 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:16:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9ae40b6610 msgr2=0x7f9ae40b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9ae40b6610 0x7f9ae40b8b20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8005f70 tx=0x7f9ae8031400 comp rx=0 tx=0).stop 2024-09-21T07:16:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 msgr2=0x7f9b0019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 0x7f9b0019c710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9af000e9f0 tx=0x7f9af000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 shutdown_connections 2024-09-21T07:16:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b0010c040 0x7f9b001a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9ae40b6610 0x7f9ae40b8b20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b0010b6f0 0x7f9b0019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 --2- 172.21.15.27:0/1421731358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b00105db0 0x7f9b0019c710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.617+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 >> 172.21.15.27:0/1421731358 conn(0x7f9b001017e0 msgr2=0x7f9b00108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.618+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 shutdown_connections 2024-09-21T07:16:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:44.618+0000 7f9b04a00640 1 -- 172.21.15.27:0/1421731358 wait complete. 2024-09-21T07:16:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:44 smithi119 ceph-mon[34519]: pgmap v489: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-21T07:16:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:44 smithi027 ceph-mon[30779]: pgmap v489: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-21T07:16:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:44 smithi153 ceph-mon[34747]: pgmap v489: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-21T07:16:45.458 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:16:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 -- 172.21.15.27:0/124091635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10c020 msgr2=0x7f5b9c112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 --2- 172.21.15.27:0/124091635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10c020 0x7f5b9c112850 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009900 tx=0x7f5b9402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 -- 172.21.15.27:0/124091635 shutdown_connections 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 --2- 172.21.15.27:0/124091635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10c020 0x7f5b9c112850 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 --2- 172.21.15.27:0/124091635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c10b6d0 0x7f5b9c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 --2- 172.21.15.27:0/124091635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c105d90 0x7f5b9c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 -- 172.21.15.27:0/124091635 >> 172.21.15.27:0/124091635 conn(0x7f5b9c101860 msgr2=0x7f5b9c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 -- 172.21.15.27:0/124091635 shutdown_connections 2024-09-21T07:16:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.461+0000 7f5ba4600640 1 -- 172.21.15.27:0/124091635 wait complete. 2024-09-21T07:16:46.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.462+0000 7f5ba4600640 1 Processor -- start 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba4600640 1 -- start start 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba4600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c105d90 0x7f5b9c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba4600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10b6d0 0x7f5b9c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba4600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c10c020 0x7f5b9c1a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba4600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c115ce0 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba4600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c115b60 con 0x7f5b9c105d90 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba4600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c115e60 con 0x7f5b9c10c020 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10b6d0 0x7f5b9c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c105d90 0x7f5b9c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.463+0000 7f5ba1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c105d90 0x7f5b9c19c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36122/0 (socket says 172.21.15.27:36122) 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1c00640 1 -- 172.21.15.27:0/2341354483 learned_addr learned my addr 172.21.15.27:0/2341354483 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba2600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c10c020 0x7f5b9c1a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1200640 1 -- 172.21.15.27:0/2341354483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c10c020 msgr2=0x7f5b9c1a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c10c020 0x7f5b9c1a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1200640 1 -- 172.21.15.27:0/2341354483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c105d90 msgr2=0x7f5b9c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c105d90 0x7f5b9c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1200640 1 -- 172.21.15.27:0/2341354483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1a26e0 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c105d90 0x7f5b9c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10b6d0 0x7f5b9c19cd10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c06ca80 tx=0x7f5b8800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.464+0000 7f5b9a000640 1 -- 172.21.15.27:0/2341354483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800cc40 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.465+0000 7f5b9a000640 1 -- 172.21.15.27:0/2341354483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b8800cde0 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.465+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1a29d0 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.465+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a9130 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.465+0000 7f5b9a000640 1 -- 172.21.15.27:0/2341354483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88018780 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.468+0000 7f5bab3be740 1 Processor -- start 2024-09-21T07:16:46.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.468+0000 7f5bab3be740 1 -- start start 2024-09-21T07:16:46.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.468+0000 7f5b9a000640 1 -- 172.21.15.27:0/2341354483 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5b88010a90 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693655070 0x5586939fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55869362c1b0 0x5586939fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693634390 0x5586939fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5bab3be740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558693972fb0 con 0x558693655070 2024-09-21T07:16:46.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5bab3be740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5586939623e0 con 0x558693634390 2024-09-21T07:16:46.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5bab3be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5586939661f0 con 0x55869362c1b0 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5ba2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693655070 0x5586939fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5ba1c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55869362c1b0 0x5586939fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5ba2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693655070 0x5586939fbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43894/0 (socket says 172.21.15.27:43894) 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5ba2600640 1 -- 172.21.15.27:0/1371696453 learned_addr learned my addr 172.21.15.27:0/1371696453 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.469+0000 7f5ba1200640 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693634390 0x5586939fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5ba1c00640 1 -- 172.21.15.27:0/1371696453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693634390 msgr2=0x5586939fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693634390 0x5586939fc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5ba1c00640 1 -- 172.21.15.27:0/1371696453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693655070 msgr2=0x5586939fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693655070 0x5586939fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5ba1c00640 1 -- 172.21.15.27:0/1371696453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869392ccd0 con 0x55869362c1b0 2024-09-21T07:16:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5ba1200640 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693634390 0x5586939fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:46.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55869362c1b0 0x5586939fc0e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b90011870 tx=0x7f5b90011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39a74227417057c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5b8ea00640 1 -- 172.21.15.27:0/1371696453 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b900112f0 con 0x55869362c1b0 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5b8ea00640 1 -- 172.21.15.27:0/1371696453 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b90003e20 con 0x55869362c1b0 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5b8ea00640 1 -- 172.21.15.27:0/1371696453 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b900043d0 con 0x55869362c1b0 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.470+0000 7f5b9a000640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b840b6680 0x7f5b840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.471+0000 7f5b9a000640 1 -- 172.21.15.27:0/2341354483 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5b880e7ce0 con 0x7f5b9c10b6d0 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.471+0000 7f5ba2600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b840b6680 0x7f5b840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.471+0000 7f5bab3be740 1 -- 172.21.15.27:0/1371696453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55869362c1b0 msgr2=0x5586939fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.471+0000 7f5bab3be740 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55869362c1b0 0x5586939fc0e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b90011870 tx=0x7f5b90011d30 comp rx=0 tx=0).stop 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 -- 172.21.15.27:0/1371696453 shutdown_connections 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55869362c1b0 0x5586939fc0e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693634390 0x5586939fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 --2- 172.21.15.27:0/1371696453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693655070 0x5586939fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 -- 172.21.15.27:0/1371696453 >> 172.21.15.27:0/1371696453 conn(0x558693649460 msgr2=0x5586939f9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:46.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5ba2600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b840b6680 0x7f5b840b8b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005f70 tx=0x7f5b940058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:46.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 -- 172.21.15.27:0/1371696453 shutdown_connections 2024-09-21T07:16:46.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 -- 172.21.15.27:0/1371696453 wait complete. 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.472+0000 7f5bab3be740 1 -- start start 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 1 Processor -- start 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a07700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a07c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a13670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5586937c3700 con 0x55869362c1b0 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5586938fd580 con 0x558693655070 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558693a0a380 con 0x558693634390 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5bab3be740 10 client.0 _ll_register_callbacks cb 0x55869390d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5ba1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a07700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5ba1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a07c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5ba1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a07700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43910/0 (socket says 172.21.15.27:43910) 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.473+0000 7f5ba1c00640 1 -- 172.21.15.27:0/2267814225 learned_addr learned my addr 172.21.15.27:0/2267814225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:46.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5ba2600640 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a13670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5ba1200640 1 -- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 msgr2=0x558693a13670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a13670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5ba1200640 1 -- 172.21.15.27:0/2267814225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 msgr2=0x558693a07700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a07700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5ba1200640 1 -- 172.21.15.27:0/2267814225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586936c0650 con 0x558693634390 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a07c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b880e5080 tx=0x7f5b88009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb251911f4b76e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5b83400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.474+0000 7f5b83400640 1 -- 172.21.15.27:0/2267814225 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880a96d0 con 0x558693634390 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586938ffb90 con 0x558693634390 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5b83400640 1 -- 172.21.15.27:0/2267814225 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5b880b33f0 con 0x558693634390 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5b83400640 1 -- 172.21.15.27:0/2267814225 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b880acfa0 con 0x558693634390 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 10 client.26617 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:46.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55869391a950 con 0x558693634390 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5b83400640 1 -- 172.21.15.27:0/2267814225 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5586938ffb90 con 0x558693634390 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5b83400640 1 -- 172.21.15.27:0/2267814225 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b880b35f0 con 0x558693634390 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 10 client.26617 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 10 client.26617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 20 client.26617 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586936139f0 0x558693a19640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 4 client.26617 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.475+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558693974f00 con 0x5586936139f0 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.476+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586936139f0 0x558693a19640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.476+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586936139f0 0x558693a19640 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676a059d2586a8b4 server_cookie=b108748900168d62 in_seq=0 out_seq=0 2024-09-21T07:16:46.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.476+0000 7f5b83400640 10 client.26617 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.477+0000 7f5b83400640 1 -- 172.21.15.27:0/2267814225 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558693974f00 con 0x5586936139f0 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.477+0000 7f5b83400640 10 client.26617 handle_command_reply: tid=0 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.477+0000 7f5bab3be740 1 client.26617 shutdown 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 msgr2=0x558693a07c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a07c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b880e5080 tx=0x7f5b88009700 comp rx=0 tx=0).stop 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 shutdown_connections 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586936139f0 0x558693a19640 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a07c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a13670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2267814225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a07700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5b83400640 0 client.26617 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 >> 172.21.15.27:0/2267814225 conn(0x558693649460 msgr2=0x5586939fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 shutdown_connections 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 1 -- 172.21.15.27:0/2267814225 wait complete. 2024-09-21T07:16:46.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.478+0000 7f5bab3be740 20 client.26617 trim_cache size 0 max 16384 2024-09-21T07:16:46.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.492+0000 7f5bab3be740 1 Processor -- start 2024-09-21T07:16:46.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.492+0000 7f5bab3be740 1 -- start start 2024-09-21T07:16:46.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.492+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869362c1b0 0x558693a09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.492+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693634390 0x5586939f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.492+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693655070 0x5586939f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5bab3be740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b59c0 con 0x558693634390 2024-09-21T07:16:46.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5bab3be740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b3290 con 0x55869362c1b0 2024-09-21T07:16:46.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5bab3be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b10e0 con 0x558693655070 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693634390 0x5586939f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693634390 0x5586939f9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43912/0 (socket says 172.21.15.27:43912) 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869362c1b0 0x558693a09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 -- 172.21.15.27:0/3395450969 learned_addr learned my addr 172.21.15.27:0/3395450969 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba2600640 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693655070 0x5586939f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 -- 172.21.15.27:0/3395450969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693655070 msgr2=0x5586939f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693655070 0x5586939f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 -- 172.21.15.27:0/3395450969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869362c1b0 msgr2=0x558693a09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869362c1b0 0x558693a09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1200640 1 -- 172.21.15.27:0/3395450969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558693a087c0 con 0x558693634390 2024-09-21T07:16:46.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.493+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869362c1b0 0x558693a09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:46.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.494+0000 7f5ba1200640 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693634390 0x5586939f9040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009560 tx=0x7f5b880ea1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d061fd92612f7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:46.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.494+0000 7f5b8e000640 1 -- 172.21.15.27:0/3395450969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880ad180 con 0x558693634390 2024-09-21T07:16:46.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.494+0000 7f5b8e000640 1 -- 172.21.15.27:0/3395450969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b880d3070 con 0x558693634390 2024-09-21T07:16:46.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.494+0000 7f5b8e000640 1 -- 172.21.15.27:0/3395450969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880ad600 con 0x558693634390 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 -- 172.21.15.27:0/3395450969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693634390 msgr2=0x5586939f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693634390 0x5586939f9040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009560 tx=0x7f5b880ea1a0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 -- 172.21.15.27:0/3395450969 shutdown_connections 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693655070 0x5586939f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558693634390 0x5586939f9040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 --2- 172.21.15.27:0/3395450969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869362c1b0 0x558693a09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 -- 172.21.15.27:0/3395450969 >> 172.21.15.27:0/3395450969 conn(0x558693649460 msgr2=0x558693a08a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 -- 172.21.15.27:0/3395450969 shutdown_connections 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 -- 172.21.15.27:0/3395450969 wait complete. 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.495+0000 7f5bab3be740 1 -- start start 2024-09-21T07:16:46.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 1 Processor -- start 2024-09-21T07:16:46.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a0fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a0c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b1750 con 0x55869362c1b0 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b2bc0 con 0x558693655070 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b1e20 con 0x558693634390 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5bab3be740 10 client.0 _ll_register_callbacks cb 0x7f5b880a9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.496+0000 7f5ba1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a0fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a0c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a0c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a0c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36158/0 (socket says 172.21.15.27:36158) 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba2600640 1 -- 172.21.15.27:0/2971774212 learned_addr learned my addr 172.21.15.27:0/2971774212 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a0c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38678/0 (socket says 172.21.15.27:38678) 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1c00640 1 -- 172.21.15.27:0/2971774212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 msgr2=0x558693a0c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a0c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1c00640 1 -- 172.21.15.27:0/2971774212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 msgr2=0x558693a0c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:46.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a0c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1c00640 1 -- 172.21.15.27:0/2971774212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558693a0ca90 con 0x55869362c1b0 2024-09-21T07:16:46.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5ba1c00640 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a0fb50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x558693a19da0 tx=0x7f5b900112a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8527e258e65d0b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:46.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.497+0000 7f5b83400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:46.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.498+0000 7f5b83400640 1 -- 172.21.15.27:0/2971774212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90033060 con 0x55869362c1b0 2024-09-21T07:16:46.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.498+0000 7f5b83400640 1 -- 172.21.15.27:0/2971774212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5b90003b70 con 0x55869362c1b0 2024-09-21T07:16:46.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.498+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b880d3070 con 0x55869362c1b0 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.498+0000 7f5b83400640 1 -- 172.21.15.27:0/2971774212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b9003a050 con 0x55869362c1b0 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.498+0000 7f5bab3be740 10 client.16599 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.498+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558693a84ce0 con 0x55869362c1b0 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.498+0000 7f5b83400640 1 -- 172.21.15.27:0/2971774212 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b880d3070 con 0x55869362c1b0 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5b83400640 1 -- 172.21.15.27:0/2971774212 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b900056e0 con 0x55869362c1b0 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5bab3be740 10 client.16599 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5bab3be740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5bab3be740 20 client.16599 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586935e8da0 0x558693a872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5bab3be740 4 client.16599 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:16:46.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586939709b0 con 0x5586935e8da0 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.499+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586935e8da0 0x558693a872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5ba1200640 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586935e8da0 0x558693a872e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcedfc10e4ca5ee6 server_cookie=f508e80e63564d2c in_seq=0 out_seq=0 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5b83400640 10 client.16599 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5b83400640 1 -- 172.21.15.27:0/2971774212 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586939709b0 con 0x5586935e8da0 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5b83400640 10 client.16599 handle_command_reply: tid=0 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 client.16599 shutdown 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 msgr2=0x558693a0fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a0fb50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x558693a19da0 tx=0x7f5b900112a0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 shutdown_connections 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586935e8da0 0x558693a872e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558693634390 0x558693a0c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558693655070 0x558693a0c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 --2- 172.21.15.27:0/2971774212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55869362c1b0 0x558693a0fb50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 >> 172.21.15.27:0/2971774212 conn(0x558693649460 msgr2=0x5586939fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:46.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5b83400640 0 client.16599 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:46.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.500+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 shutdown_connections 2024-09-21T07:16:46.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5bab3be740 1 -- 172.21.15.27:0/2971774212 wait complete. 2024-09-21T07:16:46.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5bab3be740 20 client.16599 trim_cache size 0 max 16384 2024-09-21T07:16:46.505 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b840b6680 msgr2=0x7f5b840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b840b6680 0x7f5b840b8b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005f70 tx=0x7f5b940058d0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10b6d0 msgr2=0x7f5b9c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10b6d0 0x7f5b9c19cd10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c06ca80 tx=0x7f5b8800ee50 comp rx=0 tx=0).stop 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 shutdown_connections 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b9c10c020 0x7f5b9c1a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b840b6680 0x7f5b840b8b90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b9c10b6d0 0x7f5b9c19cd10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.501+0000 7f5ba4600640 1 --2- 172.21.15.27:0/2341354483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c105d90 0x7f5b9c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.502+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 >> 172.21.15.27:0/2341354483 conn(0x7f5b9c101860 msgr2=0x7f5b9c108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.502+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 shutdown_connections 2024-09-21T07:16:46.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:46.502+0000 7f5ba4600640 1 -- 172.21.15.27:0/2341354483 wait complete. 2024-09-21T07:16:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:46 smithi119 ceph-mon[34519]: pgmap v490: 129 pgs: 129 active+clean; 3.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:16:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:46 smithi027 ceph-mon[30779]: pgmap v490: 129 pgs: 129 active+clean; 3.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:16:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:46 smithi153 ceph-mon[34747]: pgmap v490: 129 pgs: 129 active+clean; 3.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:16:48.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:16:48.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:16:48.805 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:49.246 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:48 smithi027 ceph-mon[30779]: pgmap v491: 129 pgs: 129 active+clean; 3.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:16:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:48 smithi119 ceph-mon[34519]: pgmap v491: 129 pgs: 129 active+clean; 3.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:16:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:48 smithi153 ceph-mon[34747]: pgmap v491: 129 pgs: 129 active+clean; 3.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:16:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 -- 172.21.15.27:0/964958920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 msgr2=0x7fc2bc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/964958920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc1098d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009900 tx=0x7fc2a802a410 comp rx=0 tx=0).stop 2024-09-21T07:16:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 -- 172.21.15.27:0/964958920 shutdown_connections 2024-09-21T07:16:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/964958920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2bc109e10 0x7fc2bc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/964958920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc1098d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/964958920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2bc103b80 0x7fc2bc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 -- 172.21.15.27:0/964958920 >> 172.21.15.27:0/964958920 conn(0x7fc2bc0ff5d0 msgr2=0x7fc2bc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 -- 172.21.15.27:0/964958920 shutdown_connections 2024-09-21T07:16:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.432+0000 7fc2c0c00640 1 -- 172.21.15.27:0/964958920 wait complete. 2024-09-21T07:16:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.433+0000 7fc2c0c00640 1 Processor -- start 2024-09-21T07:16:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.433+0000 7fc2c0c00640 1 -- start start 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2c0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2bc103b80 0x7fc2bc19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2c0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2c0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2bc109e10 0x7fc2bc1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2c0c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc113ad0 con 0x7fc2bc1094c0 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2c0c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc113950 con 0x7fc2bc103b80 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2c0c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc113c50 con 0x7fc2bc109e10 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2b9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2b9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56176/0 (socket says 172.21.15.27:56176) 2024-09-21T07:16:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2b9400640 1 -- 172.21.15.27:0/3927107879 learned_addr learned my addr 172.21.15.27:0/3927107879 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2bc103b80 0x7fc2bc19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2ba800640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2bc109e10 0x7fc2bc1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2b9400640 1 -- 172.21.15.27:0/3927107879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2bc109e10 msgr2=0x7fc2bc1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2bc109e10 0x7fc2bc1a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.434+0000 7fc2b9400640 1 -- 172.21.15.27:0/3927107879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2bc103b80 msgr2=0x7fc2bc19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2bc103b80 0x7fc2bc19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2b9400640 1 -- 172.21.15.27:0/3927107879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc1a26b0 con 0x7fc2bc1094c0 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2bc103b80 0x7fc2bc19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc19cce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802a930 tx=0x7fc2a8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2b2a00640 1 -- 172.21.15.27:0/3927107879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a803f030 con 0x7fc2bc1094c0 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc1a2940 con 0x7fc2bc1094c0 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2b2a00640 1 -- 172.21.15.27:0/3927107879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a8004490 con 0x7fc2bc1094c0 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2b2a00640 1 -- 172.21.15.27:0/3927107879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8035070 con 0x7fc2bc1094c0 2024-09-21T07:16:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.435+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc1a9100 con 0x7fc2bc1094c0 2024-09-21T07:16:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.438+0000 7fc2c7805740 1 Processor -- start 2024-09-21T07:16:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2c7805740 1 -- start start 2024-09-21T07:16:49.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebb1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebb2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebb2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2c7805740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5571eeb27e90 con 0x5571ee80a350 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2c7805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5571eeb172c0 con 0x5571ee7e1490 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2c7805740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5571eeb1b0d0 con 0x5571ee7e9670 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2b2a00640 1 -- 172.21.15.27:0/3927107879 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc2a8042030 con 0x7fc2bc1094c0 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebb1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.439+0000 7fc2ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebb1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56188/0 (socket says 172.21.15.27:56188) 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2ba800640 1 -- 172.21.15.27:0/400881519 learned_addr learned my addr 172.21.15.27:0/400881519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebb2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9400640 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebb2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9e00640 1 -- 172.21.15.27:0/400881519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 msgr2=0x5571eebb2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebb2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9e00640 1 -- 172.21.15.27:0/400881519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 msgr2=0x5571eebb1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebb1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9e00640 1 -- 172.21.15.27:0/400881519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ee875930 con 0x5571ee7e1490 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9400640 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebb2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2ba800640 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebb1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.440+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebb2250 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009fd0 tx=0x7fc2ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc797178a02f8d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.441+0000 7fc2a7000640 1 -- 172.21.15.27:0/400881519 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004310 con 0x5571ee7e1490 2024-09-21T07:16:49.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.441+0000 7fc2a7000640 1 -- 172.21.15.27:0/400881519 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac0044b0 con 0x5571ee7e1490 2024-09-21T07:16:49.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.441+0000 7fc2a7000640 1 -- 172.21.15.27:0/400881519 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0035f0 con 0x5571ee7e1490 2024-09-21T07:16:49.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.441+0000 7fc2c7805740 1 -- 172.21.15.27:0/400881519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 msgr2=0x5571eebb2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.441+0000 7fc2c7805740 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebb2250 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009fd0 tx=0x7fc2ac0118d0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2b2a00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2a00b6680 0x7fc2a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2c7805740 1 -- 172.21.15.27:0/400881519 shutdown_connections 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2c7805740 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebb2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2c7805740 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebb2250 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2c7805740 1 --2- 172.21.15.27:0/400881519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebb1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2c7805740 1 -- 172.21.15.27:0/400881519 >> 172.21.15.27:0/400881519 conn(0x5571ee7fe740 msgr2=0x5571eebafc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2b2a00640 1 -- 172.21.15.27:0/3927107879 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc2a8107df0 con 0x7fc2bc1094c0 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2ba800640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2a00b6680 0x7fc2a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2c7805740 1 -- 172.21.15.27:0/400881519 shutdown_connections 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.442+0000 7fc2c7805740 1 -- 172.21.15.27:0/400881519 wait complete. 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.443+0000 7fc2ba800640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2a00b6680 0x7fc2a00b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005fd0 tx=0x7fc2b4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.443+0000 7fc2c7805740 1 -- start start 2024-09-21T07:16:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.443+0000 7fc2c7805740 1 Processor -- start 2024-09-21T07:16:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.443+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebbd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.443+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e9670 0x5571eebbde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.443+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2c7805740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5571ee978890 con 0x5571ee80a350 2024-09-21T07:16:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2c7805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5571eeab2160 con 0x5571ee7e9670 2024-09-21T07:16:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2c7805740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5571eebc04f0 con 0x5571ee7e1490 2024-09-21T07:16:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2c7805740 10 client.0 _ll_register_callbacks cb 0x5571eeacea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56204/0 (socket says 172.21.15.27:56204) 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2ba800640 1 -- 172.21.15.27:0/3897293462 learned_addr learned my addr 172.21.15.27:0/3897293462 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2b9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebbd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e9670 0x5571eebbde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2b9400640 1 -- 172.21.15.27:0/3897293462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 msgr2=0x5571eebbd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebbd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2b9400640 1 -- 172.21.15.27:0/3897293462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 msgr2=0x5571eebc9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2b9400640 1 -- 172.21.15.27:0/3897293462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eebc9d50 con 0x5571ee7e9670 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.444+0000 7fc2ba800640 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e9670 0x5571eebbde00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8105190 tx=0x7fc2a8003f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e2f4bfaedc1539 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc29b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc29b400640 1 -- 172.21.15.27:0/3897293462 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8049040 con 0x5571ee7e9670 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac0044b0 con 0x5571ee7e9670 2024-09-21T07:16:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc29b400640 1 -- 172.21.15.27:0/3897293462 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc2a80d6e20 con 0x5571ee7e9670 2024-09-21T07:16:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc29b400640 1 -- 172.21.15.27:0/3897293462 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2a80b8050 con 0x5571ee7e9670 2024-09-21T07:16:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc2c7805740 10 client.26648 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.445+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571eebcc610 con 0x5571ee7e9670 2024-09-21T07:16:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc29b400640 1 -- 172.21.15.27:0/3897293462 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac0044b0 con 0x5571ee7e9670 2024-09-21T07:16:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc29b400640 1 -- 172.21.15.27:0/3897293462 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2a80c9290 con 0x5571ee7e9670 2024-09-21T07:16:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc2c7805740 10 client.26648 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc2c7805740 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc2c7805740 20 client.26648 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee7c8cd0 0x5571eebcf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc2c7805740 4 client.26648 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571eeb29de0 con 0x5571ee7c8cd0 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.446+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee7c8cd0 0x5571eebcf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.447+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee7c8cd0 0x5571eebcf9c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc579ddbc74439a server_cookie=4fec74a11a09721e in_seq=0 out_seq=0 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.447+0000 7fc29b400640 10 client.26648 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc29b400640 1 -- 172.21.15.27:0/3897293462 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571eeb29de0 con 0x5571ee7c8cd0 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc29b400640 10 client.26648 handle_command_reply: tid=0 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 client.26648 shutdown 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e9670 msgr2=0x5571eebbde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e9670 0x5571eebbde00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8105190 tx=0x7fc2a8003f40 comp rx=0 tx=0).stop 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 shutdown_connections 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee7c8cd0 0x5571eebcf9c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e9670 0x5571eebbde00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc29b400640 0 client.26648 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3897293462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebbd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 >> 172.21.15.27:0/3897293462 conn(0x5571ee7fe740 msgr2=0x5571eebae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 shutdown_connections 2024-09-21T07:16:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 1 -- 172.21.15.27:0/3897293462 wait complete. 2024-09-21T07:16:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.448+0000 7fc2c7805740 20 client.26648 trim_cache size 0 max 16384 2024-09-21T07:16:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 Processor -- start 2024-09-21T07:16:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 -- start start 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee7e9670 0x5571eebb7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee80a350 0x5571eebaf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1b58c0 con 0x5571ee7e9670 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1b3190 con 0x5571ee80a350 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.467+0000 7fc2c7805740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1b0fe0 con 0x5571ee7e1490 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee7e9670 0x5571eebb7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee7e9670 0x5571eebb7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56220/0 (socket says 172.21.15.27:56220) 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 -- 172.21.15.27:0/3814554852 learned_addr learned my addr 172.21.15.27:0/3814554852 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2ba800640 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee80a350 0x5571eebaf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 -- 172.21.15.27:0/3814554852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 msgr2=0x5571eebae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 -- 172.21.15.27:0/3814554852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee80a350 msgr2=0x5571eebaf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee80a350 0x5571eebaf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 -- 172.21.15.27:0/3814554852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571eebb4b70 con 0x5571ee7e9670 2024-09-21T07:16:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2ba800640 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee80a350 0x5571eebaf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.468+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee7e9670 0x5571eebb7e30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8032af0 tx=0x7fc2a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cd9613433bcead server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.469+0000 7fc2a6600640 1 -- 172.21.15.27:0/3814554852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8049070 con 0x5571ee7e9670 2024-09-21T07:16:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.469+0000 7fc2a6600640 1 -- 172.21.15.27:0/3814554852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a80cd450 con 0x5571ee7e9670 2024-09-21T07:16:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.469+0000 7fc2a6600640 1 -- 172.21.15.27:0/3814554852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8105410 con 0x5571ee7e9670 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 -- 172.21.15.27:0/3814554852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee7e9670 msgr2=0x5571eebb7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee7e9670 0x5571eebb7e30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8032af0 tx=0x7fc2a8000bc0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 -- 172.21.15.27:0/3814554852 shutdown_connections 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee80a350 0x5571eebaf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee7e9670 0x5571eebb7e30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3814554852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e1490 0x5571eebae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 -- 172.21.15.27:0/3814554852 >> 172.21.15.27:0/3814554852 conn(0x5571ee7fe740 msgr2=0x5571eebae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 -- 172.21.15.27:0/3814554852 shutdown_connections 2024-09-21T07:16:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 -- 172.21.15.27:0/3814554852 wait complete. 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 -- start start 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.470+0000 7fc2c7805740 1 Processor -- start 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebc5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebc5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2c7805740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2c7805740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1b1650 con 0x5571ee80a350 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2c7805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1b2ac0 con 0x5571ee7e1490 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2c7805740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1b1d20 con 0x5571ee7e9670 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2c7805740 10 client.0 _ll_register_callbacks cb 0x7fc2a8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56222/0 (socket says 172.21.15.27:56222) 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2ba800640 1 -- 172.21.15.27:0/3272668770 learned_addr learned my addr 172.21.15.27:0/3272668770 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.471+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebc5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebc5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2b9e00640 1 -- 172.21.15.27:0/3272668770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 msgr2=0x5571eebc5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebc5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2b9e00640 1 -- 172.21.15.27:0/3272668770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 msgr2=0x5571eebc0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2b9e00640 1 -- 172.21.15.27:0/3272668770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eebc0cf0 con 0x5571ee7e1490 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2b9e00640 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebc5980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005890 tx=0x7fc2ac005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc50ef0319e32de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc29b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc29b400640 1 -- 172.21.15.27:0/3272668770 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0037f0 con 0x5571ee7e1490 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.472+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a80cd450 con 0x5571ee7e1490 2024-09-21T07:16:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc29b400640 1 -- 172.21.15.27:0/3272668770 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc2ac003990 con 0x5571ee7e1490 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc29b400640 1 -- 172.21.15.27:0/3272668770 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac040050 con 0x5571ee7e1490 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc2c7805740 10 client.26654 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571eec3ac20 con 0x5571ee7e1490 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc29b400640 1 -- 172.21.15.27:0/3272668770 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a80cd450 con 0x5571ee7e1490 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc29b400640 1 -- 172.21.15.27:0/3272668770 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2ac00fa90 con 0x5571ee7e1490 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc2c7805740 10 client.26654 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc2c7805740 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.473+0000 7fc2c7805740 20 client.26654 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.474+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee79e080 0x5571eec3d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.474+0000 7fc2c7805740 4 client.26654 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.474+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571eeb25890 con 0x5571ee79e080 2024-09-21T07:16:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.474+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee79e080 0x5571eec3d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.474+0000 7fc2b9400640 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee79e080 0x5571eec3d210 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9c88b193341183 server_cookie=1ab569a10d7e4896 in_seq=0 out_seq=0 2024-09-21T07:16:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.474+0000 7fc29b400640 10 client.26654 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc29b400640 1 -- 172.21.15.27:0/3272668770 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571eeb25890 con 0x5571ee79e080 2024-09-21T07:16:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc29b400640 10 client.26654 handle_command_reply: tid=0 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 client.26654 shutdown 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 msgr2=0x5571eebc5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebc5980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005890 tx=0x7fc2ac005c50 comp rx=0 tx=0).stop 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 shutdown_connections 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571ee79e080 0x5571eec3d210 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571ee7e9670 0x5571eebc5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571ee80a350 0x5571eebc0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 --2- 172.21.15.27:0/3272668770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571ee7e1490 0x5571eebc5980 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc29b400640 0 client.26654 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.476+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 >> 172.21.15.27:0/3272668770 conn(0x5571ee7fe740 msgr2=0x5571eebb2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.477+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 shutdown_connections 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.477+0000 7fc2c7805740 1 -- 172.21.15.27:0/3272668770 wait complete. 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.477+0000 7fc2c7805740 20 client.26654 trim_cache size 0 max 16384 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "860f11a5-87ac-4626-9267-1ad21477cbcd", 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:16:49.480 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2a00b6680 msgr2=0x7fc2a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2a00b6680 0x7fc2a00b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005fd0 tx=0x7fc2b4009500 comp rx=0 tx=0).stop 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 msgr2=0x7fc2bc19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc19cce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802a930 tx=0x7fc2a8031a20 comp rx=0 tx=0).stop 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 shutdown_connections 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2bc109e10 0x7fc2bc1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2a00b6680 0x7fc2a00b8b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2bc1094c0 0x7fc2bc19cce0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 --2- 172.21.15.27:0/3927107879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2bc103b80 0x7fc2bc19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 >> 172.21.15.27:0/3927107879 conn(0x7fc2bc0ff5d0 msgr2=0x7fc2bc106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 shutdown_connections 2024-09-21T07:16:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:49.478+0000 7fc2c0c00640 1 -- 172.21.15.27:0/3927107879 wait complete. 2024-09-21T07:16:50.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:16:50.832 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:16:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:50 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:16:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:50 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:50 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:16:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:50 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:50 smithi119 ceph-mon[34519]: pgmap v492: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-21T07:16:51.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:50 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:16:51.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:50 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:51.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:50 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:16:51.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:50 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:51.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:50 smithi027 ceph-mon[30779]: pgmap v492: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-21T07:16:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:50 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:16:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:50 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:50 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:16:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:50 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:16:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:50 smithi153 ceph-mon[34747]: pgmap v492: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.476+0000 7f4174200640 1 -- 172.21.15.27:0/2173516971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10c010 msgr2=0x7f416c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.476+0000 7f4174200640 1 --2- 172.21.15.27:0/2173516971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10c010 0x7f416c112840 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4164009ca0 tx=0x7f416402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.477+0000 7f4174200640 1 -- 172.21.15.27:0/2173516971 shutdown_connections 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.477+0000 7f4174200640 1 --2- 172.21.15.27:0/2173516971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10c010 0x7f416c112840 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.477+0000 7f4174200640 1 --2- 172.21.15.27:0/2173516971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416c10b6c0 0x7f416c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.477+0000 7f4174200640 1 --2- 172.21.15.27:0/2173516971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 0x7f416c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.477+0000 7f4174200640 1 -- 172.21.15.27:0/2173516971 >> 172.21.15.27:0/2173516971 conn(0x7f416c101810 msgr2=0x7f416c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.477+0000 7f4174200640 1 -- 172.21.15.27:0/2173516971 shutdown_connections 2024-09-21T07:16:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.477+0000 7f4174200640 1 -- 172.21.15.27:0/2173516971 wait complete. 2024-09-21T07:16:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 Processor -- start 2024-09-21T07:16:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 -- start start 2024-09-21T07:16:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 0x7f416c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10b6c0 0x7f416c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416c10c010 0x7f416c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f416c115cd0 con 0x7f416c10b6c0 2024-09-21T07:16:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f416c115b50 con 0x7f416c10c010 2024-09-21T07:16:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.478+0000 7f4174200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f416c115e50 con 0x7f416c105d80 2024-09-21T07:16:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 0x7f416c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 0x7f416c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:43376/0 (socket says 172.21.15.27:43376) 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4170e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10b6c0 0x7f416c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 -- 172.21.15.27:0/1412221864 learned_addr learned my addr 172.21.15.27:0/1412221864 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4172200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416c10c010 0x7f416c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 -- 172.21.15.27:0/1412221864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416c10c010 msgr2=0x7f416c1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416c10c010 0x7f416c1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 -- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10b6c0 msgr2=0x7f416c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10b6c0 0x7f416c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 -- 172.21.15.27:0/1412221864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1a25d0 con 0x7f416c105d80 2024-09-21T07:16:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.479+0000 7f4171800640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 0x7f416c19c660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f416000e9f0 tx=0x7f416000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.480+0000 7f416a000640 1 -- 172.21.15.27:0/1412221864 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416000cd80 con 0x7f416c105d80 2024-09-21T07:16:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.480+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a28c0 con 0x7f416c105d80 2024-09-21T07:16:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.480+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a9020 con 0x7f416c105d80 2024-09-21T07:16:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.480+0000 7f416a000640 1 -- 172.21.15.27:0/1412221864 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41600218f0 con 0x7f416c105d80 2024-09-21T07:16:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.480+0000 7f416a000640 1 -- 172.21.15.27:0/1412221864 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160010900 con 0x7f416c105d80 2024-09-21T07:16:51.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.483+0000 7f417ae8d740 1 Processor -- start 2024-09-21T07:16:51.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.483+0000 7f417ae8d740 1 -- start start 2024-09-21T07:16:51.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.483+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b800413b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.483+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 0x55b8004140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.483+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b8004145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f417ae8d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b80038af70 con 0x55b80006d070 2024-09-21T07:16:51.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f417ae8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b80037a3a0 con 0x55b8000441b0 2024-09-21T07:16:51.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f417ae8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b80037e1b0 con 0x55b80004c390 2024-09-21T07:16:51.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b800413b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b800413b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56242/0 (socket says 172.21.15.27:56242) 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 -- 172.21.15.27:0/747059068 learned_addr learned my addr 172.21.15.27:0/747059068 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4170e00640 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b8004145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 -- 172.21.15.27:0/747059068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 msgr2=0x55b8004145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b8004145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 -- 172.21.15.27:0/747059068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 msgr2=0x55b8004140a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 0x55b8004140a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.484+0000 7f4172200640 1 -- 172.21.15.27:0/747059068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b800344c90 con 0x55b80006d070 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.485+0000 7f416a000640 1 -- 172.21.15.27:0/1412221864 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4160021440 con 0x7f416c105d80 2024-09-21T07:16:51.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.485+0000 7f4172200640 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b800413b60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f416402f410 tx=0x7f416402f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bacf01f7f0a013 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:51.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.485+0000 7f415ea00640 1 -- 172.21.15.27:0/747059068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164031cf0 con 0x55b80006d070 2024-09-21T07:16:51.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.485+0000 7f415ea00640 1 -- 172.21.15.27:0/747059068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4164036030 con 0x55b80006d070 2024-09-21T07:16:51.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.485+0000 7f415ea00640 1 -- 172.21.15.27:0/747059068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640042a0 con 0x55b80006d070 2024-09-21T07:16:51.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.486+0000 7f417ae8d740 1 -- 172.21.15.27:0/747059068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 msgr2=0x55b800413b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.486+0000 7f417ae8d740 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b800413b60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f416402f410 tx=0x7f416402f440 comp rx=0 tx=0).stop 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.486+0000 7f417ae8d740 1 -- 172.21.15.27:0/747059068 shutdown_connections 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.486+0000 7f417ae8d740 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b8004145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.486+0000 7f417ae8d740 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 0x55b8004140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.486+0000 7f417ae8d740 1 --2- 172.21.15.27:0/747059068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b800413b60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.486+0000 7f417ae8d740 1 -- 172.21.15.27:0/747059068 >> 172.21.15.27:0/747059068 conn(0x55b800061460 msgr2=0x55b800411a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.487+0000 7f417ae8d740 1 -- 172.21.15.27:0/747059068 shutdown_connections 2024-09-21T07:16:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.487+0000 7f417ae8d740 1 -- 172.21.15.27:0/747059068 wait complete. 2024-09-21T07:16:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.487+0000 7f416a000640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41540b65a0 0x7f41540b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.487+0000 7f417ae8d740 1 -- start start 2024-09-21T07:16:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.487+0000 7f416a000640 1 -- 172.21.15.27:0/1412221864 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f41600e7d40 con 0x7f416c105d80 2024-09-21T07:16:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.487+0000 7f4172200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41540b65a0 0x7f41540b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 1 Processor -- start 2024-09-21T07:16:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8000441b0 0x55b80041f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f4172200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41540b65a0 0x7f41540b8ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4164005c90 tx=0x7f416402a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80004c390 0x55b80041fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80006d070 0x55b80042b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b8001db700 con 0x55b80004c390 2024-09-21T07:16:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b800315540 con 0x55b80006d070 2024-09-21T07:16:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b800422160 con 0x55b8000441b0 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.488+0000 7f417ae8d740 10 client.0 _ll_register_callbacks cb 0x55b800325430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80004c390 0x55b80041fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80004c390 0x55b80041fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56252/0 (socket says 172.21.15.27:56252) 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 -- 172.21.15.27:0/2665519643 learned_addr learned my addr 172.21.15.27:0/2665519643 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4172200640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80006d070 0x55b80042b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4171800640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8000441b0 0x55b80041f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 -- 172.21.15.27:0/2665519643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8000441b0 msgr2=0x55b80041f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8000441b0 0x55b80041f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 -- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80006d070 msgr2=0x55b80042b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80006d070 0x55b80042b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4170e00640 1 -- 172.21.15.27:0/2665519643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8000d8650 con 0x55b80004c390 2024-09-21T07:16:51.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.489+0000 7f4171800640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8000441b0 0x55b80041f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f4170e00640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80004c390 0x55b80041fc60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f41580126e0 tx=0x7f4158012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d2b60a7810a79b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f4153400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f4153400640 1 -- 172.21.15.27:0/2665519643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158012190 con 0x55b80004c390 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f4153400640 1 -- 172.21.15.27:0/2665519643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f415800f920 con 0x55b80004c390 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b800317b50 con 0x55b80004c390 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f4153400640 1 -- 172.21.15.27:0/2665519643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f415800fb20 con 0x55b80004c390 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f417ae8d740 10 client.16629 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.490+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b800332910 con 0x55b80004c390 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f4153400640 1 -- 172.21.15.27:0/2665519643 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b800317b50 con 0x55b80004c390 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f4153400640 1 -- 172.21.15.27:0/2665519643 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f415801c420 con 0x55b80004c390 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f417ae8d740 10 client.16629 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f417ae8d740 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f417ae8d740 20 client.16629 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b80002b9f0 0x55b8004315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f417ae8d740 4 client.16629 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b80038cec0 con 0x55b80002b9f0 2024-09-21T07:16:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.491+0000 7f4171800640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b80002b9f0 0x55b8004315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.492+0000 7f4171800640 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b80002b9f0 0x55b8004315a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80d596768a0d6e server_cookie=ed5fee85b4cd08b6 in_seq=0 out_seq=0 2024-09-21T07:16:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.492+0000 7f4153400640 10 client.16629 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f4153400640 1 -- 172.21.15.27:0/2665519643 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b80038cec0 con 0x55b80002b9f0 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f4153400640 10 client.16629 handle_command_reply: tid=0 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 client.16629 shutdown 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80004c390 msgr2=0x55b80041fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80004c390 0x55b80041fc60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f41580126e0 tx=0x7f4158012ba0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 shutdown_connections 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b80002b9f0 0x55b8004315a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80006d070 0x55b80042b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80004c390 0x55b80041fc60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2665519643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8000441b0 0x55b80041f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 >> 172.21.15.27:0/2665519643 conn(0x55b800061460 msgr2=0x55b800416cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f4153400640 0 client.16629 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.493+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 shutdown_connections 2024-09-21T07:16:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.494+0000 7f417ae8d740 1 -- 172.21.15.27:0/2665519643 wait complete. 2024-09-21T07:16:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.494+0000 7f417ae8d740 20 client.16629 trim_cache size 0 max 16384 2024-09-21T07:16:51.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.512+0000 7f417ae8d740 1 Processor -- start 2024-09-21T07:16:51.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.512+0000 7f417ae8d740 1 -- start start 2024-09-21T07:16:51.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 0x55b8004113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b80042b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b80042b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f417ae8d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f414c012700 con 0x55b80006d070 2024-09-21T07:16:51.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f417ae8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f414c00ffd0 con 0x55b8000441b0 2024-09-21T07:16:51.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f417ae8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f414c00de20 con 0x55b80004c390 2024-09-21T07:16:51.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f4172200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b80042b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f4172200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b80042b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56264/0 (socket says 172.21.15.27:56264) 2024-09-21T07:16:51.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f4172200640 1 -- 172.21.15.27:0/2908866254 learned_addr learned my addr 172.21.15.27:0/2908866254 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:51.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f4170e00640 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b80042b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.513+0000 7f4171800640 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 0x55b8004113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f4172200640 1 -- 172.21.15.27:0/2908866254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 msgr2=0x55b80042b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f4172200640 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b80042b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f4172200640 1 -- 172.21.15.27:0/2908866254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 msgr2=0x55b8004113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f4172200640 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 0x55b8004113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f4172200640 1 -- 172.21.15.27:0/2908866254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b80041add0 con 0x55b80006d070 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f4172200640 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b80042b920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4164003c40 tx=0x7f416402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5375ed2379b579 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f415e000640 1 -- 172.21.15.27:0/2908866254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164031c70 con 0x55b80006d070 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f415e000640 1 -- 172.21.15.27:0/2908866254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4164060070 con 0x55b80006d070 2024-09-21T07:16:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.514+0000 7f415e000640 1 -- 172.21.15.27:0/2908866254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640308b0 con 0x55b80006d070 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 -- 172.21.15.27:0/2908866254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 msgr2=0x55b80042b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b80042b920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4164003c40 tx=0x7f416402f3a0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 -- 172.21.15.27:0/2908866254 shutdown_connections 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80004c390 0x55b80042b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b80006d070 0x55b80042b920 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 --2- 172.21.15.27:0/2908866254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8000441b0 0x55b8004113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 -- 172.21.15.27:0/2908866254 >> 172.21.15.27:0/2908866254 conn(0x55b800061460 msgr2=0x55b80041ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 -- 172.21.15.27:0/2908866254 shutdown_connections 2024-09-21T07:16:51.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.515+0000 7f417ae8d740 1 -- 172.21.15.27:0/2908866254 wait complete. 2024-09-21T07:16:51.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.516+0000 7f417ae8d740 1 -- start start 2024-09-21T07:16:51.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.516+0000 7f417ae8d740 1 Processor -- start 2024-09-21T07:16:51.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b8000441b0 0x55b800423da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80004c390 0x55b800424340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f417ae8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80006d070 0x55b800494fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f417ae8d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f414c00e490 con 0x55b8000441b0 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f417ae8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f414c00f900 con 0x55b80004c390 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f417ae8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f414c00eb60 con 0x55b80006d070 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f417ae8d740 10 client.0 _ll_register_callbacks cb 0x7f4158009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b8000441b0 0x55b800423da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b8000441b0 0x55b800423da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56278/0 (socket says 172.21.15.27:56278) 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 -- 172.21.15.27:0/3788742735 learned_addr learned my addr 172.21.15.27:0/3788742735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4172200640 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80006d070 0x55b800494fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4170e00640 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80004c390 0x55b800424340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 -- 172.21.15.27:0/3788742735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80006d070 msgr2=0x55b800494fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80006d070 0x55b800494fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 -- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80004c390 msgr2=0x55b800424340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80004c390 0x55b800424340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.517+0000 7f4171800640 1 -- 172.21.15.27:0/3788742735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b800495650 con 0x55b8000441b0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f4171800640 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b8000441b0 0x55b800423da0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4160009250 tx=0x7f41600acfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2666237bddaf1ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f4153400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f4153400640 1 -- 172.21.15.27:0/3788742735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160021730 con 0x55b8000441b0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f4153400640 1 -- 172.21.15.27:0/3788742735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f41600aeb10 con 0x55b8000441b0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4164060070 con 0x55b8000441b0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f4153400640 1 -- 172.21.15.27:0/3788742735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41600ad2f0 con 0x55b8000441b0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f417ae8d740 10 client.16635 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.518+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b80049ca00 con 0x55b8000441b0 2024-09-21T07:16:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f4153400640 1 -- 172.21.15.27:0/3788742735 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4164060070 con 0x55b8000441b0 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f4153400640 1 -- 172.21.15.27:0/3788742735 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f41600ac4b0 con 0x55b8000441b0 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f417ae8d740 10 client.16635 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f417ae8d740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f417ae8d740 20 client.16635 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f417ae8d740 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b800000da0 0x55b80049f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f417ae8d740 4 client.16635 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b800388970 con 0x55b800000da0 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.519+0000 7f4170e00640 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b800000da0 0x55b80049f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f4170e00640 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b800000da0 0x55b80049f140 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77cf2276a0845ce4 server_cookie=c3502a687ae3148a in_seq=0 out_seq=0 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f4153400640 10 client.16635 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f4153400640 1 -- 172.21.15.27:0/3788742735 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x55b800388970 con 0x55b800000da0 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f4153400640 10 client.16635 handle_command_reply: tid=0 2024-09-21T07:16:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f417ae8d740 1 client.16635 shutdown 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b8000441b0 msgr2=0x55b800423da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f417ae8d740 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b8000441b0 0x55b800423da0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4160009250 tx=0x7f41600acfc0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 shutdown_connections 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.520+0000 7f417ae8d740 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b800000da0 0x55b80049f140 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f417ae8d740 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b80006d070 0x55b800494fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f417ae8d740 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b80004c390 0x55b800424340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f417ae8d740 1 --2- 172.21.15.27:0/3788742735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b8000441b0 0x55b800423da0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 >> 172.21.15.27:0/3788742735 conn(0x55b800061460 msgr2=0x55b800418c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f4153400640 0 client.16635 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 shutdown_connections 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f417ae8d740 1 -- 172.21.15.27:0/3788742735 wait complete. 2024-09-21T07:16:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f417ae8d740 20 client.16635 trim_cache size 0 max 16384 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "860f11a5-87ac-4626-9267-1ad21477cbcd", 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:16:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:16:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:16:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41540b65a0 msgr2=0x7f41540b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f4174200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41540b65a0 0x7f41540b8ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4164005c90 tx=0x7f416402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 msgr2=0x7f416c19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.521+0000 7f4174200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 0x7f416c19c660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f416000e9f0 tx=0x7f416000edf0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 shutdown_connections 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41540b65a0 0x7f41540b8ab0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416c10c010 0x7f416c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416c10b6c0 0x7f416c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 --2- 172.21.15.27:0/1412221864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f416c105d80 0x7f416c19c660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 >> 172.21.15.27:0/1412221864 conn(0x7f416c101810 msgr2=0x7f416c108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 shutdown_connections 2024-09-21T07:16:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:16:51.522+0000 7f4174200640 1 -- 172.21.15.27:0/1412221864 wait complete. 2024-09-21T07:16:52.336 INFO:tasks.cephfs.filesystem:scrub status for tag:860f11a5-87ac-4626-9267-1ad21477cbcd - {'path': '/', 'tag': '860f11a5-87ac-4626-9267-1ad21477cbcd', 'options': 'recursive,force'} 2024-09-21T07:16:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:52 smithi119 ceph-mon[34519]: pgmap v493: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T07:16:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:52 smithi027 ceph-mon[30779]: pgmap v493: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T07:16:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:52 smithi153 ceph-mon[34747]: pgmap v493: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T07:16:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:53 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:16:54.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:53 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:16:54.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:53 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:16:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:54 smithi119 ceph-mon[34519]: pgmap v494: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T07:16:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:54 smithi027 ceph-mon[30779]: pgmap v494: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T07:16:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:54 smithi153 ceph-mon[34747]: pgmap v494: 129 pgs: 129 active+clean; 3.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 -- 172.21.15.27:0/1609425691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 msgr2=0x7f23801124d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 --2- 172.21.15.27:0/1609425691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 0x7f23801124d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ca0 tx=0x7f237802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 -- 172.21.15.27:0/1609425691 shutdown_connections 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 --2- 172.21.15.27:0/1609425691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23800760a0 0x7f238010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 --2- 172.21.15.27:0/1609425691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 0x7f23801124d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 --2- 172.21.15.27:0/1609425691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 0x7f2380075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 -- 172.21.15.27:0/1609425691 >> 172.21.15.27:0/1609425691 conn(0x7f23800ff500 msgr2=0x7f2380101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 -- 172.21.15.27:0/1609425691 shutdown_connections 2024-09-21T07:16:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.029+0000 7f2387400640 1 -- 172.21.15.27:0/1609425691 wait complete. 2024-09-21T07:16:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 Processor -- start 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 -- start start 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 0x7f238019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23800760a0 0x7f238019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 0x7f23801a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2380115c60 con 0x7f23801100b0 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2380115ae0 con 0x7f2380075750 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2387400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2380115de0 con 0x7f23800760a0 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2384a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 0x7f238019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2385400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 0x7f23801a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2384a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 0x7f238019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53338/0 (socket says 172.21.15.27:53338) 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.031+0000 7f2384a00640 1 -- 172.21.15.27:0/1138602326 learned_addr learned my addr 172.21.15.27:0/1138602326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.032+0000 7f237fe00640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23800760a0 0x7f238019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.032+0000 7f2384a00640 1 -- 172.21.15.27:0/1138602326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23800760a0 msgr2=0x7f238019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.032+0000 7f2384a00640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23800760a0 0x7f238019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.032+0000 7f2384a00640 1 -- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 msgr2=0x7f23801a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.032+0000 7f2384a00640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 0x7f23801a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.032+0000 7f2384a00640 1 -- 172.21.15.27:0/1138602326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23801a2570 con 0x7f2380075750 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.032+0000 7f2384a00640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 0x7f238019c640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f237000e9f0 tx=0x7f237000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.033+0000 7f237d600640 1 -- 172.21.15.27:0/1138602326 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237000ccc0 con 0x7f2380075750 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.033+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23801a2860 con 0x7f2380075750 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.033+0000 7f237d600640 1 -- 172.21.15.27:0/1138602326 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23700108e0 con 0x7f2380075750 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.033+0000 7f237d600640 1 -- 172.21.15.27:0/1138602326 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370018880 con 0x7f2380075750 2024-09-21T07:16:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.033+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23801a8ff0 con 0x7f2380075750 2024-09-21T07:16:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.036+0000 7f238e3ec740 1 Processor -- start 2024-09-21T07:16:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.036+0000 7f238e3ec740 1 -- start start 2024-09-21T07:16:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.036+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205bc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f238e3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d9205316b0 con 0x55d9201ee2f0 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f238e3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d920520ae0 con 0x55d9201f64d0 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f238e3ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d9205248f0 con 0x55d9202171b0 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f237d600640 1 -- 172.21.15.27:0/1138602326 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2370010430 con 0x7f2380075750 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f237fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205bc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f2385400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f237fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205bc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53344/0 (socket says 172.21.15.27:53344) 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f237fe00640 1 -- 172.21.15.27:0/3578351248 learned_addr learned my addr 172.21.15.27:0/3578351248 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f2384a00640 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f2385400640 1 -- 172.21.15.27:0/3578351248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 msgr2=0x55d9205bc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f2385400640 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205bc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f2385400640 1 -- 172.21.15.27:0/3578351248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 msgr2=0x55d9205bc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f2385400640 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205bc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f2385400640 1 -- 172.21.15.27:0/3578351248 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9204ed780 con 0x55d9202171b0 2024-09-21T07:16:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.037+0000 7f237fe00640 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:16:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.038+0000 7f2385400640 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205bbc30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f237802ade0 tx=0x7f23780098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b183262fec765d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.038+0000 7f2375c00640 1 -- 172.21.15.27:0/3578351248 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378031db0 con 0x55d9202171b0 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.038+0000 7f2375c00640 1 -- 172.21.15.27:0/3578351248 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378036070 con 0x55d9202171b0 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.038+0000 7f2375c00640 1 -- 172.21.15.27:0/3578351248 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780042a0 con 0x55d9202171b0 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.038+0000 7f237d600640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23640b6680 0x7f23640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f237d600640 1 -- 172.21.15.27:0/1138602326 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f23700e7e90 con 0x7f2380075750 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f2384a00640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23640b6680 0x7f23640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 -- 172.21.15.27:0/3578351248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 msgr2=0x55d9205bbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205bbc30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f237802ade0 tx=0x7f23780098b0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 -- 172.21.15.27:0/3578351248 shutdown_connections 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205bc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3578351248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205bbc30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 -- 172.21.15.27:0/3578351248 >> 172.21.15.27:0/3578351248 conn(0x55d92020b300 msgr2=0x55d9205b9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 -- 172.21.15.27:0/3578351248 shutdown_connections 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f238e3ec740 1 -- 172.21.15.27:0/3578351248 wait complete. 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.039+0000 7f2384a00640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23640b6680 0x7f23640b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f23700e5340 tx=0x7f23700eb9e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.040+0000 7f238e3ec740 1 -- start start 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.040+0000 7f238e3ec740 1 Processor -- start 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201ee2f0 0x55d9205c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9201f64d0 0x55d9205c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9202171b0 0x55d9205d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f238e3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d920458ab0 con 0x55d9202171b0 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f238e3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d9204c7570 con 0x55d9201ee2f0 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f238e3ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d9205ca1d0 con 0x55d9201f64d0 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f238e3ec740 10 client.0 _ll_register_callbacks cb 0x55d9204d6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9201f64d0 0x55d9205c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9201f64d0 0x55d9205c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:43466/0 (socket says 172.21.15.27:43466) 2024-09-21T07:16:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 -- 172.21.15.27:0/3542527606 learned_addr learned my addr 172.21.15.27:0/3542527606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f2385400640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9202171b0 0x55d9205d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f2385400640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201ee2f0 0x55d9205c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 -- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201ee2f0 msgr2=0x55d9205c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201ee2f0 0x55d9205c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 -- 172.21.15.27:0/3542527606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9202171b0 msgr2=0x55d9205d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9202171b0 0x55d9205d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f237fe00640 1 -- 172.21.15.27:0/3542527606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d920282790 con 0x55d9201f64d0 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.041+0000 7f2385400640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201ee2f0 0x55d9205c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f237fe00640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9201f64d0 0x55d9205c77f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f236c0115e0 tx=0x7f236c011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0a658a8afa4c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f236a000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:16:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f236a000640 1 -- 172.21.15.27:0/3542527606 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c00f750 con 0x55d9201f64d0 2024-09-21T07:16:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f236a000640 1 -- 172.21.15.27:0/3542527606 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f236c00f8f0 con 0x55d9201f64d0 2024-09-21T07:16:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2378036070 con 0x55d9201f64d0 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f236a000640 1 -- 172.21.15.27:0/3542527606 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f236c026050 con 0x55d9201f64d0 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f238e3ec740 10 client.26668 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.042+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9202fb620 con 0x55d9201f64d0 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f236a000640 1 -- 172.21.15.27:0/3542527606 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378036070 con 0x55d9201f64d0 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f236a000640 1 -- 172.21.15.27:0/3542527606 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f236c014c10 con 0x55d9201f64d0 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f238e3ec740 10 client.26668 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f238e3ec740 10 client.26668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f238e3ec740 20 client.26668 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201d5d80 0x55d9205d92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f238e3ec740 4 client.26668 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d920533600 con 0x55d9201d5d80 2024-09-21T07:16:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.043+0000 7f2385400640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201d5d80 0x55d9205d92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.044+0000 7f2385400640 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201d5d80 0x55d9205d92d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2367ec966ccc99 server_cookie=e3506cf95d06a647 in_seq=0 out_seq=0 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.044+0000 7f236a000640 10 client.26668 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.045+0000 7f236a000640 1 -- 172.21.15.27:0/3542527606 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d920533600 con 0x55d9201d5d80 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.045+0000 7f236a000640 10 client.26668 handle_command_reply: tid=0 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.045+0000 7f238e3ec740 1 client.26668 shutdown 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.045+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9201f64d0 msgr2=0x55d9205c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.045+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9201f64d0 0x55d9205c77f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f236c0115e0 tx=0x7f236c011aa0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 shutdown_connections 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201d5d80 0x55d9205d92d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9201f64d0 0x55d9205c77f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9202171b0 0x55d9205d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 --2- 172.21.15.27:0/3542527606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201ee2f0 0x55d9205c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f236a000640 0 client.26668 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 >> 172.21.15.27:0/3542527606 conn(0x55d92020b300 msgr2=0x55d9205bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 shutdown_connections 2024-09-21T07:16:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 1 -- 172.21.15.27:0/3542527606 wait complete. 2024-09-21T07:16:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.046+0000 7f238e3ec740 20 client.26668 trim_cache size 0 max 16384 2024-09-21T07:16:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.061+0000 7f238e3ec740 1 Processor -- start 2024-09-21T07:16:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.061+0000 7f238e3ec740 1 -- start start 2024-09-21T07:16:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.061+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.061+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f238e3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b5770 con 0x55d9201ee2f0 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f238e3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b3040 con 0x55d9201f64d0 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f238e3ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b0e90 con 0x55d9202171b0 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56322/0 (socket says 172.21.15.27:56322) 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 -- 172.21.15.27:0/1680083610 learned_addr learned my addr 172.21.15.27:0/1680083610 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f237fe00640 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 -- 172.21.15.27:0/1680083610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 msgr2=0x55d9205d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 -- 172.21.15.27:0/1680083610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 msgr2=0x55d9205b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.062+0000 7f2385400640 1 -- 172.21.15.27:0/1680083610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9205d5dc0 con 0x55d9201ee2f0 2024-09-21T07:16:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.063+0000 7f2385400640 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205c7790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2378000f80 tx=0x7f2378003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d47e9ca036dbb24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.063+0000 7f2375200640 1 -- 172.21.15.27:0/1680083610 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237805b070 con 0x55d9201ee2f0 2024-09-21T07:16:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.063+0000 7f2375200640 1 -- 172.21.15.27:0/1680083610 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378036040 con 0x55d9201ee2f0 2024-09-21T07:16:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.063+0000 7f2375200640 1 -- 172.21.15.27:0/1680083610 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378004650 con 0x55d9201ee2f0 2024-09-21T07:16:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 -- 172.21.15.27:0/1680083610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 msgr2=0x55d9205c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205c7790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2378000f80 tx=0x7f2378003db0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 -- 172.21.15.27:0/1680083610 shutdown_connections 2024-09-21T07:16:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d9205d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 --2- 172.21.15.27:0/1680083610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205c7790 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 -- 172.21.15.27:0/1680083610 >> 172.21.15.27:0/1680083610 conn(0x55d92020b300 msgr2=0x55d9205b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 -- 172.21.15.27:0/1680083610 shutdown_connections 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 -- 172.21.15.27:0/1680083610 wait complete. 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 -- start start 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.064+0000 7f238e3ec740 1 Processor -- start 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f238e3ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d92063bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f238e3ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b1500 con 0x55d9201ee2f0 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f238e3ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b2970 con 0x55d9201f64d0 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f238e3ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b1bd0 con 0x55d9202171b0 2024-09-21T07:16:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f238e3ec740 10 client.0 _ll_register_callbacks cb 0x7f236c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:16:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f2385400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f2385400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56332/0 (socket says 172.21.15.27:56332) 2024-09-21T07:16:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f2385400640 1 -- 172.21.15.27:0/2281536029 learned_addr learned my addr 172.21.15.27:0/2281536029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:16:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f237fe00640 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.065+0000 7f2385400640 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d92063bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f2385400640 1 -- 172.21.15.27:0/2281536029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 msgr2=0x55d92063bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f2385400640 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d92063bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f2385400640 1 -- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 msgr2=0x55d9205cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f2385400640 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f2385400640 1 -- 172.21.15.27:0/2281536029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92063c360 con 0x55d9201ee2f0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f2385400640 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205caed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2378006010 tx=0x7f2378004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91803d65ae7f10a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f236a000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f236a000640 1 -- 172.21.15.27:0/2281536029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237805b070 con 0x55d9201ee2f0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f236a000640 1 -- 172.21.15.27:0/2281536029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f2378031db0 con 0x55d9201ee2f0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.066+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2378036040 con 0x55d9201ee2f0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f236a000640 1 -- 172.21.15.27:0/2281536029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2378069050 con 0x55d9201ee2f0 2024-09-21T07:16:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 10 client.16653 fetch_fsmap learned FSMap version 27 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d920643780 con 0x55d9201ee2f0 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f236a000640 1 -- 172.21.15.27:0/2281536029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378036040 con 0x55d9201ee2f0 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f236a000640 1 -- 172.21.15.27:0/2281536029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2378043020 con 0x55d9201ee2f0 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 10 client.16653 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 20 client.16653 populate_metadata read hostname 'smithi027' 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201ab1c0 0x55d920645ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 4 client.16653 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.067+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d92052f0b0 con 0x55d9201ab1c0 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.068+0000 7f237fe00640 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201ab1c0 0x55d920645ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.068+0000 7f237fe00640 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201ab1c0 0x55d920645ad0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae0c5edca43649d server_cookie=be1763666aa2ff4e in_seq=0 out_seq=0 2024-09-21T07:16:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.068+0000 7f236a000640 10 client.16653 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.081+0000 7f236a000640 1 -- 172.21.15.27:0/2281536029 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d92052f0b0 con 0x55d9201ab1c0 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.081+0000 7f236a000640 10 client.16653 handle_command_reply: tid=0 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.081+0000 7f238e3ec740 1 client.16653 shutdown 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 msgr2=0x55d9205caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205caed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2378006010 tx=0x7f2378004230 comp rx=0 tx=0).stop 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 shutdown_connections 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9201ab1c0 0x55d920645ad0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9202171b0 0x55d92063bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9201f64d0 0x55d9205cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 --2- 172.21.15.27:0/2281536029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9201ee2f0 0x55d9205caed0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 >> 172.21.15.27:0/2281536029 conn(0x55d92020b300 msgr2=0x55d9205bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f236a000640 0 client.16653 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 shutdown_connections 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 1 -- 172.21.15.27:0/2281536029 wait complete. 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.082+0000 7f238e3ec740 20 client.16653 trim_cache size 0 max 16384 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:16:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:16:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23640b6680 msgr2=0x7f23640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23640b6680 0x7f23640b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f23700e5340 tx=0x7f23700eb9e0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 msgr2=0x7f238019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:16:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 0x7f238019c640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f237000e9f0 tx=0x7f237000edf0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 shutdown_connections 2024-09-21T07:16:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23800760a0 0x7f238019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23640b6680 0x7f23640b8b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23801100b0 0x7f23801a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 --2- 172.21.15.27:0/1138602326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380075750 0x7f238019c640 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:16:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 >> 172.21.15.27:0/1138602326 conn(0x7f23800ff500 msgr2=0x7f2380109bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:16:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 shutdown_connections 2024-09-21T07:16:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:16:57.083+0000 7f2387400640 1 -- 172.21.15.27:0/1138602326 wait complete. 2024-09-21T07:16:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:56 smithi119 ceph-mon[34519]: pgmap v495: 129 pgs: 129 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-09-21T07:16:57.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:56 smithi027 ceph-mon[30779]: pgmap v495: 129 pgs: 129 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-09-21T07:16:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:56 smithi153 ceph-mon[34747]: pgmap v495: 129 pgs: 129 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-09-21T07:16:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:16:58 smithi119 ceph-mon[34519]: pgmap v496: 129 pgs: 129 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-21T07:16:59.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:16:58 smithi027 ceph-mon[30779]: pgmap v496: 129 pgs: 129 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-21T07:16:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:16:58 smithi153 ceph-mon[34747]: pgmap v496: 129 pgs: 129 active+clean; 3.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-21T07:17:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:00 smithi119 ceph-mon[34519]: pgmap v497: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-09-21T07:17:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:00 smithi027 ceph-mon[30779]: pgmap v497: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-09-21T07:17:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:00 smithi153 ceph-mon[34747]: pgmap v497: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-09-21T07:17:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:01 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:17:02.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:01 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:17:02.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:01 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:17:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:02 smithi119 ceph-mon[34519]: pgmap v498: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:17:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:17:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:17:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:02 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:17:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:02 smithi027 ceph-mon[30779]: pgmap v498: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:17:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:17:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:17:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:02 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:17:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:02 smithi153 ceph-mon[34747]: pgmap v498: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-21T07:17:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:17:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:17:03.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:02 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:17:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:04 smithi119 ceph-mon[34519]: pgmap v499: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-21T07:17:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:04 smithi027 ceph-mon[30779]: pgmap v499: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-21T07:17:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:04 smithi153 ceph-mon[34747]: pgmap v499: 129 pgs: 129 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-21T07:17:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:06 smithi119 ceph-mon[34519]: pgmap v500: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-09-21T07:17:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:06 smithi027 ceph-mon[30779]: pgmap v500: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-09-21T07:17:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:06 smithi153 ceph-mon[34747]: pgmap v500: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-09-21T07:17:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:08 smithi119 ceph-mon[34519]: pgmap v501: 129 pgs: 129 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-21T07:17:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:08 smithi027 ceph-mon[30779]: pgmap v501: 129 pgs: 129 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-21T07:17:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:08 smithi153 ceph-mon[34747]: pgmap v501: 129 pgs: 129 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 -- 172.21.15.27:0/2183417674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c109240 msgr2=0x7f413c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 --2- 172.21.15.27:0/2183417674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c109240 0x7f413c109650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4128009900 tx=0x7f412802a410 comp rx=0 tx=0).stop 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 -- 172.21.15.27:0/2183417674 shutdown_connections 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 --2- 172.21.15.27:0/2183417674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f413c109b90 0x7f413c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 --2- 172.21.15.27:0/2183417674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c109240 0x7f413c109650 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 --2- 172.21.15.27:0/2183417674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413c103910 0x7f413c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 -- 172.21.15.27:0/2183417674 >> 172.21.15.27:0/2183417674 conn(0x7f413c0ff330 msgr2=0x7f413c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 -- 172.21.15.27:0/2183417674 shutdown_connections 2024-09-21T07:17:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.291+0000 7f413b200640 1 -- 172.21.15.27:0/2183417674 wait complete. 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 Processor -- start 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 -- start start 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c103910 0x7f413c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f413c109240 0x7f413c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413c109b90 0x7f413c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f413c113850 con 0x7f413c103910 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1136d0 con 0x7f413c109240 2024-09-21T07:17:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.292+0000 7f413b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1139d0 con 0x7f413c109b90 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c103910 0x7f413c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c103910 0x7f413c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40674/0 (socket says 172.21.15.27:40674) 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 -- 172.21.15.27:0/1155108659 learned_addr learned my addr 172.21.15.27:0/1155108659 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139400640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f413c109240 0x7f413c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f413a800640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413c109b90 0x7f413c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 -- 172.21.15.27:0/1155108659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413c109b90 msgr2=0x7f413c1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413c109b90 0x7f413c1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 -- 172.21.15.27:0/1155108659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f413c109240 msgr2=0x7f413c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f413c109240 0x7f413c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 -- 172.21.15.27:0/1155108659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c1a2460 con 0x7f413c103910 2024-09-21T07:17:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139400640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f413c109240 0x7f413c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.293+0000 7f4139e00640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c103910 0x7f413c19c530 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f412c00e9f0 tx=0x7f412c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.294+0000 7f4132a00640 1 -- 172.21.15.27:0/1155108659 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c00cd80 con 0x7f413c103910 2024-09-21T07:17:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.294+0000 7f4132a00640 1 -- 172.21.15.27:0/1155108659 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412c0108e0 con 0x7f413c103910 2024-09-21T07:17:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.294+0000 7f4132a00640 1 -- 172.21.15.27:0/1155108659 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c018980 con 0x7f413c103910 2024-09-21T07:17:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.294+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c1a2750 con 0x7f413c103910 2024-09-21T07:17:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.294+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c1a8ee0 con 0x7f413c103910 2024-09-21T07:17:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.298+0000 7f4147bdc740 1 Processor -- start 2024-09-21T07:17:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.298+0000 7f4147bdc740 1 -- start start 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578000fe2f0 0x5578004cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001064d0 0x5578004cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4147bdc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578004416b0 con 0x5578001271b0 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4147bdc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557800430ae0 con 0x5578001064d0 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4147bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578004348f0 con 0x5578000fe2f0 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4132a00640 1 -- 172.21.15.27:0/1155108659 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f412c010430 con 0x7f413c103910 2024-09-21T07:17:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40678/0 (socket says 172.21.15.27:40678) 2024-09-21T07:17:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f413a800640 1 -- 172.21.15.27:0/653896377 learned_addr learned my addr 172.21.15.27:0/653896377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.299+0000 7f4139400640 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001064d0 0x5578004cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f413a800640 1 -- 172.21.15.27:0/653896377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578000fe2f0 msgr2=0x5578004cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f413a800640 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578000fe2f0 0x5578004cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f413a800640 1 -- 172.21.15.27:0/653896377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001064d0 msgr2=0x5578004cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f413a800640 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001064d0 0x5578004cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f413a800640 1 -- 172.21.15.27:0/653896377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578003fd780 con 0x5578001271b0 2024-09-21T07:17:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f413a800640 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004cbe10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4134009f40 tx=0x7f4134010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed339c7dfa037bde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f4127000640 1 -- 172.21.15.27:0/653896377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134003b70 con 0x5578001271b0 2024-09-21T07:17:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f4127000640 1 -- 172.21.15.27:0/653896377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4134003d10 con 0x5578001271b0 2024-09-21T07:17:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.300+0000 7f4127000640 1 -- 172.21.15.27:0/653896377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41340119c0 con 0x5578001271b0 2024-09-21T07:17:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.301+0000 7f4132a00640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41200b6680 0x7f41200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.301+0000 7f4147bdc740 1 -- 172.21.15.27:0/653896377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 msgr2=0x5578004cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.301+0000 7f4147bdc740 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004cbe10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4134009f40 tx=0x7f4134010860 comp rx=0 tx=0).stop 2024-09-21T07:17:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.301+0000 7f4132a00640 1 -- 172.21.15.27:0/1155108659 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f412c0e7eb0 con 0x7f413c103910 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.301+0000 7f4139e00640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41200b6680 0x7f41200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 -- 172.21.15.27:0/653896377 shutdown_connections 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578000fe2f0 0x5578004cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001064d0 0x5578004cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 --2- 172.21.15.27:0/653896377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004cbe10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 -- 172.21.15.27:0/653896377 >> 172.21.15.27:0/653896377 conn(0x55780011b300 msgr2=0x5578004c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 -- 172.21.15.27:0/653896377 shutdown_connections 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 -- 172.21.15.27:0/653896377 wait complete. 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4139e00640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41200b6680 0x7f41200b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5578004caa90 tx=0x7f412c024310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 -- start start 2024-09-21T07:17:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.302+0000 7f4147bdc740 1 Processor -- start 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578000fe2f0 0x5578004d7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004d7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001271b0 0x5578004d7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4147bdc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557800368ab0 con 0x5578000fe2f0 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4147bdc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5578003d7570 con 0x5578001271b0 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4147bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578004da590 con 0x5578001064d0 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4147bdc740 10 client.0 _ll_register_callbacks cb 0x5578003e6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578000fe2f0 0x5578004d7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4139400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004d7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578000fe2f0 0x5578004d7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40692/0 (socket says 172.21.15.27:40692) 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f4139400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004d7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54738/0 (socket says 172.21.15.27:54738) 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f413a800640 1 -- 172.21.15.27:0/148598781 learned_addr learned my addr 172.21.15.27:0/148598781 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.303+0000 7f413a800640 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001271b0 0x5578004d7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f4139400640 1 -- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001271b0 msgr2=0x5578004d7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f4139400640 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001271b0 0x5578004d7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f4139400640 1 -- 172.21.15.27:0/148598781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578000fe2f0 msgr2=0x5578004d7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f4139400640 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578000fe2f0 0x5578004d7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f4139400640 1 -- 172.21.15.27:0/148598781 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557800192790 con 0x5578001064d0 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f413a800640 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001271b0 0x5578004d7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f4139400640 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004d7970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f412802a930 tx=0x7f41280316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0db557d5c29b7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f411b400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f411b400640 1 -- 172.21.15.27:0/148598781 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128035040 con 0x5578001064d0 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4134003d10 con 0x5578001064d0 2024-09-21T07:17:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.304+0000 7f411b400640 1 -- 172.21.15.27:0/148598781 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4128004240 con 0x5578001064d0 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f411b400640 1 -- 172.21.15.27:0/148598781 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f412804f050 con 0x5578001064d0 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f4147bdc740 10 client.26683 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55780020b620 con 0x5578001064d0 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f411b400640 1 -- 172.21.15.27:0/148598781 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4134003d10 con 0x5578001064d0 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f411b400640 1 -- 172.21.15.27:0/148598781 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4128004970 con 0x5578001064d0 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f4147bdc740 10 client.26683 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f4147bdc740 10 client.26683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.305+0000 7f4147bdc740 20 client.26683 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.306+0000 7f4147bdc740 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000e5d80 0x5578004e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.306+0000 7f4147bdc740 4 client.26683 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.306+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557800443600 con 0x5578000e5d80 2024-09-21T07:17:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.306+0000 7f413a800640 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000e5d80 0x5578004e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.306+0000 7f413a800640 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000e5d80 0x5578004e9580 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95e87f574e9bf64 server_cookie=a962a3f3f06bfb78 in_seq=0 out_seq=0 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.306+0000 7f411b400640 10 client.26683 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.307+0000 7f411b400640 1 -- 172.21.15.27:0/148598781 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557800443600 con 0x5578000e5d80 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.307+0000 7f411b400640 10 client.26683 handle_command_reply: tid=0 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.307+0000 7f4147bdc740 1 client.26683 shutdown 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 msgr2=0x5578004d7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004d7970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f412802a930 tx=0x7f41280316d0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 shutdown_connections 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000e5d80 0x5578004e9580 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004d7970 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578001271b0 0x5578004d7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f411b400640 0 client.26683 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 --2- 172.21.15.27:0/148598781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578000fe2f0 0x5578004d7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 >> 172.21.15.27:0/148598781 conn(0x55780011b300 msgr2=0x5578004cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 shutdown_connections 2024-09-21T07:17:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 1 -- 172.21.15.27:0/148598781 wait complete. 2024-09-21T07:17:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.308+0000 7f4147bdc740 20 client.26683 trim_cache size 0 max 16384 2024-09-21T07:17:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 Processor -- start 2024-09-21T07:17:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 -- start start 2024-09-21T07:17:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004d4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1b5770 con 0x5578001271b0 2024-09-21T07:17:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1b3040 con 0x5578000fe2f0 2024-09-21T07:17:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.323+0000 7f4147bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1b0e90 con 0x5578001064d0 2024-09-21T07:17:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004d4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004d4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40698/0 (socket says 172.21.15.27:40698) 2024-09-21T07:17:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f413a800640 1 -- 172.21.15.27:0/990450273 learned_addr learned my addr 172.21.15.27:0/990450273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f4139400640 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f413a800640 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f4139400640 1 -- 172.21.15.27:0/990450273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 msgr2=0x5578004c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f4139400640 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f4139400640 1 -- 172.21.15.27:0/990450273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 msgr2=0x5578004d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f4139400640 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004d4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f4139400640 1 -- 172.21.15.27:0/990450273 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578004cec20 con 0x5578001064d0 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.324+0000 7f413a800640 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004d4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.325+0000 7f4139400640 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004cd550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4128004eb0 tx=0x7f41280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d351cb9cf0d99ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.325+0000 7f4126600640 1 -- 172.21.15.27:0/990450273 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128035070 con 0x5578001064d0 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.325+0000 7f4126600640 1 -- 172.21.15.27:0/990450273 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4128005d90 con 0x5578001064d0 2024-09-21T07:17:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.325+0000 7f4126600640 1 -- 172.21.15.27:0/990450273 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128005620 con 0x5578001064d0 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 -- 172.21.15.27:0/990450273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 msgr2=0x5578004cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004cd550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4128004eb0 tx=0x7f41280046c0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 -- 172.21.15.27:0/990450273 shutdown_connections 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004cd550 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004d4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 --2- 172.21.15.27:0/990450273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 -- 172.21.15.27:0/990450273 >> 172.21.15.27:0/990450273 conn(0x55780011b300 msgr2=0x5578004cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 -- 172.21.15.27:0/990450273 shutdown_connections 2024-09-21T07:17:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 -- 172.21.15.27:0/990450273 wait complete. 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 -- start start 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.326+0000 7f4147bdc740 1 Processor -- start 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004dd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4147bdc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004ddb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4147bdc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1b1500 con 0x5578001271b0 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4147bdc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1b2970 con 0x5578000fe2f0 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4147bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1b1bd0 con 0x5578001064d0 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4147bdc740 10 client.0 _ll_register_callbacks cb 0x7f4128009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004ddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4139400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004dd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f413a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004ddb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40702/0 (socket says 172.21.15.27:40702) 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f4139400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004dd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54750/0 (socket says 172.21.15.27:54750) 2024-09-21T07:17:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f413a800640 1 -- 172.21.15.27:0/4064581599 learned_addr learned my addr 172.21.15.27:0/4064581599 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.327+0000 7f413a800640 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f4139400640 1 -- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 msgr2=0x5578004e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f4139400640 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f4139400640 1 -- 172.21.15.27:0/4064581599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 msgr2=0x5578004ddb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f4139400640 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004ddb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f4139400640 1 -- 172.21.15.27:0/4064581599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578004de280 con 0x5578001064d0 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f4139400640 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004dd520 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4128032af0 tx=0x7f4128003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e159aefc089007b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f411b400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:17:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f411b400640 1 -- 172.21.15.27:0/4064581599 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128035070 con 0x5578001064d0 2024-09-21T07:17:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4128005d90 con 0x5578001064d0 2024-09-21T07:17:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.328+0000 7f411b400640 1 -- 172.21.15.27:0/4064581599 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4128005360 con 0x5578001064d0 2024-09-21T07:17:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f411b400640 1 -- 172.21.15.27:0/4064581599 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f412806b050 con 0x5578001064d0 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 10 client.26689 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557800553f50 con 0x5578001064d0 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f411b400640 1 -- 172.21.15.27:0/4064581599 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4128005d90 con 0x5578001064d0 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f411b400640 1 -- 172.21.15.27:0/4064581599 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4128046960 con 0x5578001064d0 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 10 client.26689 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 10 client.26689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 20 client.26689 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000bb1c0 0x557800556620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 4 client.26689 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.329+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55780043f0b0 con 0x5578000bb1c0 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.330+0000 7f413a800640 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000bb1c0 0x557800556620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.330+0000 7f413a800640 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000bb1c0 0x557800556620 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d6da07ffae6fa0 server_cookie=6403bda35320e8f3 in_seq=0 out_seq=0 2024-09-21T07:17:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.330+0000 7f411b400640 10 client.26689 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.345+0000 7f411b400640 1 -- 172.21.15.27:0/4064581599 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55780043f0b0 con 0x5578000bb1c0 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.345+0000 7f411b400640 10 client.26689 handle_command_reply: tid=0 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.345+0000 7f4147bdc740 1 client.26689 shutdown 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 msgr2=0x5578004dd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004dd520 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4128032af0 tx=0x7f4128003820 comp rx=0 tx=0).stop 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 shutdown_connections 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578000bb1c0 0x557800556620 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578001064d0 0x5578004dd520 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578001271b0 0x5578004ddb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 --2- 172.21.15.27:0/4064581599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578000fe2f0 0x5578004e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f411b400640 0 client.26689 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 >> 172.21.15.27:0/4064581599 conn(0x55780011b300 msgr2=0x5578004c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 shutdown_connections 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 1 -- 172.21.15.27:0/4064581599 wait complete. 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.346+0000 7f4147bdc740 20 client.26689 trim_cache size 0 max 16384 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:17:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41200b6680 msgr2=0x7f41200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41200b6680 0x7f41200b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5578004caa90 tx=0x7f412c024310 comp rx=0 tx=0).stop 2024-09-21T07:17:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c103910 msgr2=0x7f413c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c103910 0x7f413c19c530 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f412c00e9f0 tx=0x7f412c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 shutdown_connections 2024-09-21T07:17:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413c109b90 0x7f413c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f41200b6680 0x7f41200b8b90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f413c109240 0x7f413c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 --2- 172.21.15.27:0/1155108659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f413c103910 0x7f413c19c530 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 >> 172.21.15.27:0/1155108659 conn(0x7f413c0ff330 msgr2=0x7f413c108950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 shutdown_connections 2024-09-21T07:17:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:10.347+0000 7f413b200640 1 -- 172.21.15.27:0/1155108659 wait complete. 2024-09-21T07:17:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:10 smithi119 ceph-mon[34519]: pgmap v502: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 67 KiB/s wr, 30 op/s 2024-09-21T07:17:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:10 smithi027 ceph-mon[30779]: pgmap v502: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 67 KiB/s wr, 30 op/s 2024-09-21T07:17:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:10 smithi153 ceph-mon[34747]: pgmap v502: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 67 KiB/s wr, 30 op/s 2024-09-21T07:17:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:12 smithi119 ceph-mon[34519]: pgmap v503: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-09-21T07:17:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:12 smithi027 ceph-mon[30779]: pgmap v503: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-09-21T07:17:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:12 smithi153 ceph-mon[34747]: pgmap v503: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-09-21T07:17:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:14 smithi119 ceph-mon[34519]: pgmap v504: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-09-21T07:17:15.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:14 smithi027 ceph-mon[30779]: pgmap v504: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-09-21T07:17:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:14 smithi153 ceph-mon[34747]: pgmap v504: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-09-21T07:17:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:17 smithi119 ceph-mon[34519]: pgmap v505: 129 pgs: 129 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-09-21T07:17:17.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:17 smithi027 ceph-mon[30779]: pgmap v505: 129 pgs: 129 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-09-21T07:17:17.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:17 smithi153 ceph-mon[34747]: pgmap v505: 129 pgs: 129 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-09-21T07:17:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:19 smithi027 ceph-mon[30779]: pgmap v506: 129 pgs: 129 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-21T07:17:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:19 smithi119 ceph-mon[34519]: pgmap v506: 129 pgs: 129 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-21T07:17:19.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:19 smithi153 ceph-mon[34747]: pgmap v506: 129 pgs: 129 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-21T07:17:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:21 smithi119 ceph-mon[34519]: pgmap v507: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:17:21.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:21 smithi027 ceph-mon[30779]: pgmap v507: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:17:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:21 smithi153 ceph-mon[34747]: pgmap v507: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:17:22.338 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:17:22.739 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:17:23.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:23 smithi027 ceph-mon[30779]: pgmap v508: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:23 smithi119 ceph-mon[34519]: pgmap v508: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 -- 172.21.15.27:0/2308293178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c109c20 msgr2=0x7f3d7c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2308293178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c109c20 0x7f3d7c110450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009ca0 tx=0x7f3d7402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 -- 172.21.15.27:0/2308293178 shutdown_connections 2024-09-21T07:17:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2308293178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 0x7f3d7c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2308293178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c109c20 0x7f3d7c110450 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2308293178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d7c103990 0x7f3d7c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 -- 172.21.15.27:0/2308293178 >> 172.21.15.27:0/2308293178 conn(0x7f3d7c0ff440 msgr2=0x7f3d7c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 -- 172.21.15.27:0/2308293178 shutdown_connections 2024-09-21T07:17:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.342+0000 7f3d7b200640 1 -- 172.21.15.27:0/2308293178 wait complete. 2024-09-21T07:17:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 Processor -- start 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 -- start start 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c103990 0x7f3d7c077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 0x7f3d7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d7c109c20 0x7f3d7c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1138e0 con 0x7f3d7c103990 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c113760 con 0x7f3d7c109c20 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.344+0000 7f3d7b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c113a60 con 0x7f3d7c1092d0 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d79e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c103990 0x7f3d7c077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 0x7f3d7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d7a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d7c109c20 0x7f3d7c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 0x7f3d7c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46410/0 (socket says 172.21.15.27:46410) 2024-09-21T07:17:23.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 -- 172.21.15.27:0/2612352048 learned_addr learned my addr 172.21.15.27:0/2612352048 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 -- 172.21.15.27:0/2612352048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d7c109c20 msgr2=0x7f3d7c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d7c109c20 0x7f3d7c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 -- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c103990 msgr2=0x7f3d7c077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c103990 0x7f3d7c077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 -- 172.21.15.27:0/2612352048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c076470 con 0x7f3d7c1092d0 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.345+0000 7f3d73e00640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 0x7f3d7c075710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9f0 tx=0x7f3d6400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.346+0000 7f3d72a00640 1 -- 172.21.15.27:0/2612352048 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400ccc0 con 0x7f3d7c1092d0 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.346+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7c1acc50 con 0x7f3d7c1092d0 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.346+0000 7f3d72a00640 1 -- 172.21.15.27:0/2612352048 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d640108e0 con 0x7f3d7c1092d0 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.346+0000 7f3d72a00640 1 -- 172.21.15.27:0/2612352048 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64018860 con 0x7f3d7c1092d0 2024-09-21T07:17:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.346+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7c1ad190 con 0x7f3d7c1092d0 2024-09-21T07:17:23.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.349+0000 7f3d87e0c740 1 Processor -- start 2024-09-21T07:17:23.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.349+0000 7f3d87e0c740 1 -- start start 2024-09-21T07:17:23.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0090c070 0x562f00cb2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cb3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008eb390 0x562f00cb39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562f00c29f70 con 0x562f008eb390 2024-09-21T07:17:23.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562f00c193a0 con 0x562f008e31b0 2024-09-21T07:17:23.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f00c1d1b0 con 0x562f0090c070 2024-09-21T07:17:23.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d72a00640 1 -- 172.21.15.27:0/2612352048 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3d64010430 con 0x7f3d7c1092d0 2024-09-21T07:17:23.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d7a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0090c070 0x562f00cb2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d7a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0090c070 0x562f00cb2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46422/0 (socket says 172.21.15.27:46422) 2024-09-21T07:17:23.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d7a800640 1 -- 172.21.15.27:0/1744099628 learned_addr learned my addr 172.21.15.27:0/1744099628 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d79e00640 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cb3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.350+0000 7f3d73e00640 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008eb390 0x562f00cb39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d7a800640 1 -- 172.21.15.27:0/1744099628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 msgr2=0x562f00cb3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d7a800640 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cb3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d7a800640 1 -- 172.21.15.27:0/1744099628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008eb390 msgr2=0x562f00cb39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d7a800640 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008eb390 0x562f00cb39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d7a800640 1 -- 172.21.15.27:0/1744099628 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f00be3c90 con 0x562f0090c070 2024-09-21T07:17:23.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d73e00640 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008eb390 0x562f00cb39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:17:23.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d7a800640 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0090c070 0x562f00cb2f30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d74035000 tx=0x7f3d7402f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c655f329538c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d6b400640 1 -- 172.21.15.27:0/1744099628 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74031cf0 con 0x562f0090c070 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d6b400640 1 -- 172.21.15.27:0/1744099628 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d74036030 con 0x562f0090c070 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.351+0000 7f3d6b400640 1 -- 172.21.15.27:0/1744099628 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740042a0 con 0x562f0090c070 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d72a00640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3d600b6680 0x7f3d600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d72a00640 1 -- 172.21.15.27:0/2612352048 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3d640e7d70 con 0x7f3d7c1092d0 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d87e0c740 1 -- 172.21.15.27:0/1744099628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0090c070 msgr2=0x562f00cb2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0090c070 0x562f00cb2f30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d74035000 tx=0x7f3d7402f4c0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d79e00640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3d600b6680 0x7f3d600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d87e0c740 1 -- 172.21.15.27:0/1744099628 shutdown_connections 2024-09-21T07:17:23.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008eb390 0x562f00cb39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cb3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/1744099628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0090c070 0x562f00cb2f30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.352+0000 7f3d87e0c740 1 -- 172.21.15.27:0/1744099628 >> 172.21.15.27:0/1744099628 conn(0x562f00900460 msgr2=0x562f00cb0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.353+0000 7f3d87e0c740 1 -- 172.21.15.27:0/1744099628 shutdown_connections 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.353+0000 7f3d87e0c740 1 -- 172.21.15.27:0/1744099628 wait complete. 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.353+0000 7f3d79e00640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3d600b6680 0x7f3d600b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x562f00cb1bb0 tx=0x7f3d6c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.353+0000 7f3d87e0c740 1 -- start start 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 1 Processor -- start 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008e31b0 0x562f00cbe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008eb390 0x562f00cbebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562f00a7a700 con 0x562f0090c070 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562f00bb4540 con 0x562f008eb390 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f00cc1530 con 0x562f008e31b0 2024-09-21T07:17:23.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.354+0000 7f3d87e0c740 10 client.0 _ll_register_callbacks cb 0x562f00bc4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:23.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d7a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008eb390 0x562f00cbebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d79e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008e31b0 0x562f00cbe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008eb390 0x562f00cbebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54688/0 (socket says 172.21.15.27:54688) 2024-09-21T07:17:23.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 -- 172.21.15.27:0/2289089458 learned_addr learned my addr 172.21.15.27:0/2289089458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 -- 172.21.15.27:0/2289089458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008e31b0 msgr2=0x562f00cbe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008e31b0 0x562f00cbe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 -- 172.21.15.27:0/2289089458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 msgr2=0x562f00cca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 -- 172.21.15.27:0/2289089458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f00977650 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d73e00640 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008eb390 0x562f00cbebe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9c0 tx=0x7f3d640096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990193cd091908bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.355+0000 7f3d5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d5fe00640 1 -- 172.21.15.27:0/2289089458 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640ad210 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d5fe00640 1 -- 172.21.15.27:0/2289089458 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3d640aeb40 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f00bb6b50 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d5fe00640 1 -- 172.21.15.27:0/2289089458 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d640ad030 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d87e0c740 10 client.26714 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f00bd1910 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d5fe00640 1 -- 172.21.15.27:0/2289089458 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562f00bb6b50 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d5fe00640 1 -- 172.21.15.27:0/2289089458 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d640b3480 con 0x562f008eb390 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.356+0000 7f3d87e0c740 10 client.26714 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:23.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.357+0000 7f3d87e0c740 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:23.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.357+0000 7f3d87e0c740 20 client.26714 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:23.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.357+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f008ca9f0 0x562f00cd06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.357+0000 7f3d87e0c740 4 client.26714 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:23.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.357+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f00c2bec0 con 0x562f008ca9f0 2024-09-21T07:17:23.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.357+0000 7f3d79e00640 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f008ca9f0 0x562f00cd06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.358+0000 7f3d79e00640 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f008ca9f0 0x562f00cd06d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b1da6e7230fe8e server_cookie=f777c5e652e7f81b in_seq=0 out_seq=0 2024-09-21T07:17:23.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.358+0000 7f3d5fe00640 10 client.26714 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.358+0000 7f3d5fe00640 1 -- 172.21.15.27:0/2289089458 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f00c2bec0 con 0x562f008ca9f0 2024-09-21T07:17:23.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.358+0000 7f3d5fe00640 10 client.26714 handle_command_reply: tid=0 2024-09-21T07:17:23.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.358+0000 7f3d87e0c740 1 client.26714 shutdown 2024-09-21T07:17:23.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008eb390 msgr2=0x562f00cbebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008eb390 0x562f00cbebe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9c0 tx=0x7f3d640096a0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 shutdown_connections 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f008ca9f0 0x562f00cd06d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008eb390 0x562f00cbebe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/2289089458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008e31b0 0x562f00cbe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d5fe00640 0 client.26714 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 >> 172.21.15.27:0/2289089458 conn(0x562f00900460 msgr2=0x562f00cb60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 shutdown_connections 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.359+0000 7f3d87e0c740 1 -- 172.21.15.27:0/2289089458 wait complete. 2024-09-21T07:17:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.360+0000 7f3d87e0c740 20 client.26714 trim_cache size 0 max 16384 2024-09-21T07:17:23.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.381+0000 7f3d87e0c740 1 Processor -- start 2024-09-21T07:17:23.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d87e0c740 1 -- start start 2024-09-21T07:17:23.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008e31b0 0x562f00cbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0090c070 0x562f00cb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1b9ac0 con 0x562f008e31b0 2024-09-21T07:17:23.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1b7390 con 0x562f0090c070 2024-09-21T07:17:23.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1b51e0 con 0x562f008eb390 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.382+0000 7f3d79e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008e31b0 0x562f00cbd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d79e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008e31b0 0x562f00cbd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55546/0 (socket says 172.21.15.27:55546) 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d79e00640 1 -- 172.21.15.27:0/859653580 learned_addr learned my addr 172.21.15.27:0/859653580 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d7a800640 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0090c070 0x562f00cb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d73e00640 1 -- 172.21.15.27:0/859653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0090c070 msgr2=0x562f00cb4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d73e00640 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0090c070 0x562f00cb4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d73e00640 1 -- 172.21.15.27:0/859653580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008e31b0 msgr2=0x562f00cbd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d73e00640 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008e31b0 0x562f00cbd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d73e00640 1 -- 172.21.15.27:0/859653580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f00cb1360 con 0x562f008eb390 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d79e00640 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008e31b0 0x562f00cbd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:23.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.383+0000 7f3d73e00640 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc0580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9c0 tx=0x7f3d640e7ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477e91cfef34e60f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.384+0000 7f3d6aa00640 1 -- 172.21.15.27:0/859653580 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640ad210 con 0x562f008eb390 2024-09-21T07:17:23.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.384+0000 7f3d6aa00640 1 -- 172.21.15.27:0/859653580 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d64010720 con 0x562f008eb390 2024-09-21T07:17:23.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.384+0000 7f3d6aa00640 1 -- 172.21.15.27:0/859653580 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640b7280 con 0x562f008eb390 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 -- 172.21.15.27:0/859653580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 msgr2=0x562f00cc0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc0580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9c0 tx=0x7f3d640e7ff0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 -- 172.21.15.27:0/859653580 shutdown_connections 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc0580 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f0090c070 0x562f00cb4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/859653580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f008e31b0 0x562f00cbd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 -- 172.21.15.27:0/859653580 >> 172.21.15.27:0/859653580 conn(0x562f00900460 msgr2=0x562f00cb3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 -- 172.21.15.27:0/859653580 shutdown_connections 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 -- 172.21.15.27:0/859653580 wait complete. 2024-09-21T07:17:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.385+0000 7f3d87e0c740 1 -- start start 2024-09-21T07:17:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 1 Processor -- start 2024-09-21T07:17:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cc6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cc3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1b5850 con 0x562f0090c070 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1b6cc0 con 0x562f008e31b0 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1b5f20 con 0x562f008eb390 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d87e0c740 10 client.0 _ll_register_callbacks cb 0x7f3d640a9f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d7a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cc3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d79e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cc6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.386+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46448/0 (socket says 172.21.15.27:46448) 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d7a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cc3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55550/0 (socket says 172.21.15.27:55550) 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d79e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cc6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54702/0 (socket says 172.21.15.27:54702) 2024-09-21T07:17:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 -- 172.21.15.27:0/5962529 learned_addr learned my addr 172.21.15.27:0/5962529 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 -- 172.21.15.27:0/5962529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 msgr2=0x562f00cc6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cc6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 -- 172.21.15.27:0/5962529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 msgr2=0x562f00cc3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cc3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 -- 172.21.15.27:0/5962529 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f00cc4050 con 0x562f008eb390 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d7a800640 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cc3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d73e00640 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc3400 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3d640a91a0 tx=0x7f3d6400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a2441349c00d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.387+0000 7f3d5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d5fe00640 1 -- 172.21.15.27:0/5962529 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640ad210 con 0x562f008eb390 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d5fe00640 1 -- 172.21.15.27:0/5962529 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3d640ac790 con 0x562f008eb390 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f00c27970 con 0x562f008eb390 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d5fe00640 1 -- 172.21.15.27:0/5962529 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d640c8e60 con 0x562f008eb390 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d87e0c740 10 client.26707 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f00d3bf90 con 0x562f008eb390 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d5fe00640 1 -- 172.21.15.27:0/5962529 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562f00c27970 con 0x562f008eb390 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d5fe00640 1 -- 172.21.15.27:0/5962529 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d64018a70 con 0x562f008eb390 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d87e0c740 10 client.26707 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d87e0c740 10 client.26707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d87e0c740 20 client.26707 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.388+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f0089fda0 0x562f00d3e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.389+0000 7f3d87e0c740 4 client.26707 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:17:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.389+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f00c24da0 con 0x562f0089fda0 2024-09-21T07:17:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.389+0000 7f3d79e00640 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f0089fda0 0x562f00d3e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d5fe00640 10 client.26707 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d79e00640 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f0089fda0 0x562f00d3e500 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b762172a3444489 server_cookie=f43244e04fca9704 in_seq=0 out_seq=0 2024-09-21T07:17:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d5fe00640 1 -- 172.21.15.27:0/5962529 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x562f00c24da0 con 0x562f0089fda0 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d5fe00640 10 client.26707 handle_command_reply: tid=0 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 client.26707 shutdown 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 msgr2=0x562f00cc3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc3400 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3d640a91a0 tx=0x7f3d6400cef0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 shutdown_connections 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562f0089fda0 0x562f00d3e500 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f008eb390 0x562f00cc3400 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f0090c070 0x562f00cc3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d5fe00640 0 client.26707 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 --2- 172.21.15.27:0/5962529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f008e31b0 0x562f00cc6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.390+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 >> 172.21.15.27:0/5962529 conn(0x562f00900460 msgr2=0x562f00cb4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.391+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 shutdown_connections 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.391+0000 7f3d87e0c740 1 -- 172.21.15.27:0/5962529 wait complete. 2024-09-21T07:17:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.391+0000 7f3d87e0c740 20 client.26707 trim_cache size 0 max 16384 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:17:23.398 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.391+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3d600b6680 msgr2=0x7f3d600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.391+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3d600b6680 0x7f3d600b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x562f00cb1bb0 tx=0x7f3d6c00b390 comp rx=0 tx=0).stop 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.391+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 msgr2=0x7f3d7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.391+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 0x7f3d7c075710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400e9f0 tx=0x7f3d6400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 shutdown_connections 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c1092d0 0x7f3d7c075710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3d600b6680 0x7f3d600b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d7c109c20 0x7f3d7c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 --2- 172.21.15.27:0/2612352048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d7c103990 0x7f3d7c077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 >> 172.21.15.27:0/2612352048 conn(0x7f3d7c0ff440 msgr2=0x7f3d7c108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 shutdown_connections 2024-09-21T07:17:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:23.392+0000 7f3d7b200640 1 -- 172.21.15.27:0/2612352048 wait complete. 2024-09-21T07:17:23.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:23 smithi153 ceph-mon[34747]: pgmap v508: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 -- 172.21.15.27:0/1406849006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 msgr2=0x7f9414110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 --2- 172.21.15.27:0/1406849006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 0x7f9414110280 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f940c009ca0 tx=0x7f940c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 -- 172.21.15.27:0/1406849006 shutdown_connections 2024-09-21T07:17:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 --2- 172.21.15.27:0/1406849006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9414105280 0x7f941410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 --2- 172.21.15.27:0/1406849006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 0x7f9414110280 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 --2- 172.21.15.27:0/1406849006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 0x7f9414104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 -- 172.21.15.27:0/1406849006 >> 172.21.15.27:0/1406849006 conn(0x7f94140fe3c0 msgr2=0x7f9414100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 -- 172.21.15.27:0/1406849006 shutdown_connections 2024-09-21T07:17:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.556+0000 7f941b400640 1 -- 172.21.15.27:0/1406849006 wait complete. 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.557+0000 7f941b400640 1 Processor -- start 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f941b400640 1 -- start start 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f941b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 0x7f941419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f941b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9414105280 0x7f941419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f941b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 0x7f94141a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f941b400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9414113a10 con 0x7f941410dd90 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f941b400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9414113890 con 0x7f9414104930 2024-09-21T07:17:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f941b400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9414113b90 con 0x7f9414105280 2024-09-21T07:17:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f9418a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 0x7f941419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f9418a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 0x7f941419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54716/0 (socket says 172.21.15.27:54716) 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f9418a00640 1 -- 172.21.15.27:0/315108212 learned_addr learned my addr 172.21.15.27:0/315108212 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.558+0000 7f9419400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 0x7f94141a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9413e00640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9414105280 0x7f941419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9418a00640 1 -- 172.21.15.27:0/315108212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9414105280 msgr2=0x7f941419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9418a00640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9414105280 0x7f941419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9418a00640 1 -- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 msgr2=0x7f94141a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9418a00640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 0x7f94141a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9418a00640 1 -- 172.21.15.27:0/315108212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94141a2520 con 0x7f9414104930 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9419400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 0x7f94141a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9418a00640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 0x7f941419c5f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f940800ea60 tx=0x7f940800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.559+0000 7f9411600640 1 -- 172.21.15.27:0/315108212 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940800cde0 con 0x7f9414104930 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.560+0000 7f9411600640 1 -- 172.21.15.27:0/315108212 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94080035f0 con 0x7f9414104930 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.560+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94141a2810 con 0x7f9414104930 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.560+0000 7f9411600640 1 -- 172.21.15.27:0/315108212 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408010690 con 0x7f9414104930 2024-09-21T07:17:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.560+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94141a8fa0 con 0x7f9414104930 2024-09-21T07:17:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.563+0000 7f9422475740 1 Processor -- start 2024-09-21T07:17:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.563+0000 7f9422475740 1 -- start start 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9422475740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9422475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21ea1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9422475740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9422475740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55df21e166b0 con 0x55df21afc1b0 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9422475740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55df21e05ae0 con 0x55df21adb4d0 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9422475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55df21e098f0 con 0x55df21ad32f0 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9419400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9413e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9413e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54726/0 (socket says 172.21.15.27:54726) 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9413e00640 1 -- 172.21.15.27:0/3829768271 learned_addr learned my addr 172.21.15.27:0/3829768271 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.564+0000 7f9411600640 1 -- 172.21.15.27:0/315108212 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f94080037a0 con 0x7f9414104930 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9418a00640 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21ea1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9419400640 1 -- 172.21.15.27:0/3829768271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 msgr2=0x55df21ea1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9419400640 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21ea1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9419400640 1 -- 172.21.15.27:0/3829768271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 msgr2=0x55df21ea16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9419400640 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9419400640 1 -- 172.21.15.27:0/3829768271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df21dd2780 con 0x55df21afc1b0 2024-09-21T07:17:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9419400640 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea0c30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f940c02ade0 tx=0x7f940c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9678e9775e6e08ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.565+0000 7f9405c00640 1 -- 172.21.15.27:0/3829768271 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c031cf0 con 0x55df21afc1b0 2024-09-21T07:17:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.566+0000 7f9405c00640 1 -- 172.21.15.27:0/3829768271 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940c036040 con 0x55df21afc1b0 2024-09-21T07:17:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.566+0000 7f9405c00640 1 -- 172.21.15.27:0/3829768271 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c0042a0 con 0x55df21afc1b0 2024-09-21T07:17:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.566+0000 7f9422475740 1 -- 172.21.15.27:0/3829768271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 msgr2=0x55df21ea0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.566+0000 7f9422475740 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea0c30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f940c02ade0 tx=0x7f940c009440 comp rx=0 tx=0).stop 2024-09-21T07:17:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 -- 172.21.15.27:0/3829768271 shutdown_connections 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21ea1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 --2- 172.21.15.27:0/3829768271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea0c30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 -- 172.21.15.27:0/3829768271 >> 172.21.15.27:0/3829768271 conn(0x55df21af0300 msgr2=0x55df21e9eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9411600640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93fc0b6700 0x7f93fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 -- 172.21.15.27:0/3829768271 shutdown_connections 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 -- 172.21.15.27:0/3829768271 wait complete. 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9411600640 1 -- 172.21.15.27:0/315108212 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f94080e7d90 con 0x7f9414104930 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9413e00640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93fc0b6700 0x7f93fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.567+0000 7f9422475740 1 -- start start 2024-09-21T07:17:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9413e00640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93fc0b6700 0x7f93fc0b8c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9400009d40 tx=0x7f9400005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 1 Processor -- start 2024-09-21T07:17:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21eac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21eac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21eacc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55df21d3dab0 con 0x55df21afc1b0 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55df21dac570 con 0x55df21adb4d0 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55df21eaf350 con 0x55df21ad32f0 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9422475740 10 client.0 _ll_register_callbacks cb 0x55df21dbb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9418a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21eac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.568+0000 7f9419400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21eacc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9418a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21eac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46490/0 (socket says 172.21.15.27:46490) 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9418a00640 1 -- 172.21.15.27:0/540275045 learned_addr learned my addr 172.21.15.27:0/540275045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9413e00640 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21eac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9419400640 1 -- 172.21.15.27:0/540275045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 msgr2=0x55df21eac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9419400640 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21eac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9419400640 1 -- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 msgr2=0x55df21eac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9419400640 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21eac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9419400640 1 -- 172.21.15.27:0/540275045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df21b67790 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f9419400640 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21eacc60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f940c004af0 tx=0x7f940c02afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5a77f95b8bbc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.569+0000 7f93fa000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f93fa000640 1 -- 172.21.15.27:0/540275045 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c052020 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f940c036040 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f93fa000640 1 -- 172.21.15.27:0/540275045 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f940c031af0 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f93fa000640 1 -- 172.21.15.27:0/540275045 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f940c05d050 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f9422475740 10 client.16701 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df21be0620 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f93fa000640 1 -- 172.21.15.27:0/540275045 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940c036040 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.570+0000 7f93fa000640 1 -- 172.21.15.27:0/540275045 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f940c04d030 con 0x55df21afc1b0 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9422475740 10 client.16701 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9422475740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9422475740 20 client.16701 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9422475740 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21abad80 0x55df21ebe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9422475740 4 client.16701 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df21e18600 con 0x55df21abad80 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9418a00640 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21abad80 0x55df21ebe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f9418a00640 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21abad80 0x55df21ebe0f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87bd92713ebe7e71 server_cookie=bdd50c3e64dc557a in_seq=0 out_seq=0 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.571+0000 7f93fa000640 10 client.16701 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.572+0000 7f93fa000640 1 -- 172.21.15.27:0/540275045 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df21e18600 con 0x55df21abad80 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.572+0000 7f93fa000640 10 client.16701 handle_command_reply: tid=0 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.572+0000 7f9422475740 1 client.16701 shutdown 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 msgr2=0x55df21eacc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21eacc60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f940c004af0 tx=0x7f940c02afd0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 shutdown_connections 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21abad80 0x55df21ebe0f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21eacc60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21eac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f93fa000640 0 client.16701 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 --2- 172.21.15.27:0/540275045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21eac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 >> 172.21.15.27:0/540275045 conn(0x55df21af0300 msgr2=0x55df21ea1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 shutdown_connections 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 1 -- 172.21.15.27:0/540275045 wait complete. 2024-09-21T07:17:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.573+0000 7f9422475740 20 client.16701 trim_cache size 0 max 16384 2024-09-21T07:17:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.588+0000 7f9422475740 1 Processor -- start 2024-09-21T07:17:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.588+0000 7f9422475740 1 -- start start 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9422475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21e9d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9422475740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9422475740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9422475740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94141b57c0 con 0x55df21afc1b0 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9422475740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94141b3090 con 0x55df21adb4d0 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9422475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94141b0ee0 con 0x55df21ad32f0 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9419400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9419400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55602/0 (socket says 172.21.15.27:55602) 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9419400640 1 -- 172.21.15.27:0/2018528187 learned_addr learned my addr 172.21.15.27:0/2018528187 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9413e00640 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.589+0000 7f9418a00640 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21e9d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9419400640 1 -- 172.21.15.27:0/2018528187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 msgr2=0x55df21e9d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9419400640 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21e9d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9419400640 1 -- 172.21.15.27:0/2018528187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 msgr2=0x55df21ea93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9419400640 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9419400640 1 -- 172.21.15.27:0/2018528187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df21ea2f30 con 0x55df21afc1b0 2024-09-21T07:17:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9413e00640 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9419400640 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea98f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f940c02afa0 tx=0x7f940c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37921761d232a4a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9405200640 1 -- 172.21.15.27:0/2018528187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c052070 con 0x55df21afc1b0 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9405200640 1 -- 172.21.15.27:0/2018528187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940c036040 con 0x55df21afc1b0 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.590+0000 7f9405200640 1 -- 172.21.15.27:0/2018528187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c04a040 con 0x55df21afc1b0 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 -- 172.21.15.27:0/2018528187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 msgr2=0x55df21ea98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea98f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f940c02afa0 tx=0x7f940c000f40 comp rx=0 tx=0).stop 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 -- 172.21.15.27:0/2018528187 shutdown_connections 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21afc1b0 0x55df21ea98f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ea93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 --2- 172.21.15.27:0/2018528187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21ad32f0 0x55df21e9d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 -- 172.21.15.27:0/2018528187 >> 172.21.15.27:0/2018528187 conn(0x55df21af0300 msgr2=0x55df21e9d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 -- 172.21.15.27:0/2018528187 shutdown_connections 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.591+0000 7f9422475740 1 -- 172.21.15.27:0/2018528187 wait complete. 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.592+0000 7f9422475740 1 -- start start 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.592+0000 7f9422475740 1 Processor -- start 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.592+0000 7f9422475740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21ad32f0 0x55df21ec0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.592+0000 7f9422475740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ec0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.592+0000 7f9422475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21afc1b0 0x55df21f26870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9422475740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94141b1550 con 0x55df21ad32f0 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9422475740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94141b29c0 con 0x55df21adb4d0 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9422475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94141b1c20 con 0x55df21afc1b0 2024-09-21T07:17:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9422475740 10 client.0 _ll_register_callbacks cb 0x7f9400009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21ad32f0 0x55df21ec0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21ad32f0 0x55df21ec0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55608/0 (socket says 172.21.15.27:55608) 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 -- 172.21.15.27:0/208765155 learned_addr learned my addr 172.21.15.27:0/208765155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9413e00640 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ec0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9419400640 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21afc1b0 0x55df21f26870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 -- 172.21.15.27:0/208765155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21afc1b0 msgr2=0x55df21f26870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21afc1b0 0x55df21f26870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 -- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 msgr2=0x55df21ec0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ec0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.593+0000 7f9418a00640 1 -- 172.21.15.27:0/208765155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df21f26f10 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f9418a00640 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21ad32f0 0x55df21ec0a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f94080ad570 tx=0x7f9408003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e15a9a28812351a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f93fa000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f93fa000640 1 -- 172.21.15.27:0/208765155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94080ad1c0 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f93fa000640 1 -- 172.21.15.27:0/208765155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f94080e53f0 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f940c036040 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f93fa000640 1 -- 172.21.15.27:0/208765155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94080acfe0 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f9422475740 10 client.16707 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df21f28180 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.594+0000 7f93fa000640 1 -- 172.21.15.27:0/208765155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940c036040 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f93fa000640 1 -- 172.21.15.27:0/208765155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f94080aeba0 con 0x55df21ad32f0 2024-09-21T07:17:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f9422475740 10 client.16707 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f9422475740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f9422475740 20 client.16707 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f9422475740 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21a901c0 0x55df21f2a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f9422475740 4 client.16707 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df21e140b0 con 0x55df21a901c0 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.595+0000 7f9413e00640 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21a901c0 0x55df21f2a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.596+0000 7f9413e00640 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21a901c0 0x55df21f2a920 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2287631142ff0b0 server_cookie=8715b7ada94736c2 in_seq=0 out_seq=0 2024-09-21T07:17:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.596+0000 7f93fa000640 10 client.16707 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f93fa000640 1 -- 172.21.15.27:0/208765155 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df21e140b0 con 0x55df21a901c0 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f93fa000640 10 client.16707 handle_command_reply: tid=0 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 client.16707 shutdown 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21ad32f0 msgr2=0x55df21ec0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21ad32f0 0x55df21ec0a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f94080ad570 tx=0x7f9408003cf0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 shutdown_connections 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55df21a901c0 0x55df21f2a920 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df21afc1b0 0x55df21f26870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df21adb4d0 0x55df21ec0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f93fa000640 0 client.16707 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.609+0000 7f9422475740 1 --2- 172.21.15.27:0/208765155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55df21ad32f0 0x55df21ec0a10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 >> 172.21.15.27:0/208765155 conn(0x55df21af0300 msgr2=0x55df21eb75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 shutdown_connections 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f9422475740 1 -- 172.21.15.27:0/208765155 wait complete. 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f9422475740 20 client.16707 trim_cache size 0 max 16384 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:17:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:17:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93fc0b6700 msgr2=0x7f93fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f941b400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93fc0b6700 0x7f93fc0b8c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9400009d40 tx=0x7f9400005f90 comp rx=0 tx=0).stop 2024-09-21T07:17:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 msgr2=0x7f941419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.610+0000 7f941b400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 0x7f941419c5f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f940800ea60 tx=0x7f940800ef20 comp rx=0 tx=0).stop 2024-09-21T07:17:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 shutdown_connections 2024-09-21T07:17:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9414105280 0x7f941419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93fc0b6700 0x7f93fc0b8c10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f941410dd90 0x7f94141a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 --2- 172.21.15.27:0/315108212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9414104930 0x7f941419c5f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 >> 172.21.15.27:0/315108212 conn(0x7f94140fe3c0 msgr2=0x7f941410f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 shutdown_connections 2024-09-21T07:17:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:23.611+0000 7f941b400640 1 -- 172.21.15.27:0/315108212 wait complete. 2024-09-21T07:17:24.254 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:17:24.254 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:17:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:17:25.062 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:25 smithi027 ceph-mon[30779]: pgmap v509: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:17:25.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 -- 172.21.15.27:0/1841389056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f0105d80 msgr2=0x7f74f0106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 --2- 172.21.15.27:0/1841389056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f0105d80 0x7f74f0106190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009900 tx=0x7f74e002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 -- 172.21.15.27:0/1841389056 shutdown_connections 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 --2- 172.21.15.27:0/1841389056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f010b6c0 0x7f74f010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 --2- 172.21.15.27:0/1841389056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f010c010 0x7f74f0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 --2- 172.21.15.27:0/1841389056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f0105d80 0x7f74f0106190 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 -- 172.21.15.27:0/1841389056 >> 172.21.15.27:0/1841389056 conn(0x7f74f0101810 msgr2=0x7f74f0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 -- 172.21.15.27:0/1841389056 shutdown_connections 2024-09-21T07:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.242+0000 7f74f4a00640 1 -- 172.21.15.27:0/1841389056 wait complete. 2024-09-21T07:17:25.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.243+0000 7f74f4a00640 1 Processor -- start 2024-09-21T07:17:25.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.243+0000 7f74f4a00640 1 -- start start 2024-09-21T07:17:25.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74f4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f0105d80 0x7f74f019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74f4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f010b6c0 0x7f74f019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74f4a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f010c010 0x7f74f01a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74f4a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0115cd0 con 0x7f74f010b6c0 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74f4a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0115b50 con 0x7f74f0105d80 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74f4a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0115e50 con 0x7f74f010c010 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f010b6c0 0x7f74f019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f010b6c0 0x7f74f019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55626/0 (socket says 172.21.15.27:55626) 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 -- 172.21.15.27:0/3460043374 learned_addr learned my addr 172.21.15.27:0/3460043374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74ede00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f0105d80 0x7f74f019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74ee800640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f010c010 0x7f74f01a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 -- 172.21.15.27:0/3460043374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f010c010 msgr2=0x7f74f01a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f010c010 0x7f74f01a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 -- 172.21.15.27:0/3460043374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f0105d80 msgr2=0x7f74f019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f0105d80 0x7f74f019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74e7e00640 1 -- 172.21.15.27:0/3460043374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f01a2650 con 0x7f74f010b6c0 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.244+0000 7f74ee800640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f010c010 0x7f74f01a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.245+0000 7f74e7e00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f010b6c0 0x7f74f019cc80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9c0 tx=0x7f74d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.245+0000 7f74e6a00640 1 -- 172.21.15.27:0/3460043374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d800ccc0 con 0x7f74f010b6c0 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.245+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f01a2940 con 0x7f74f010b6c0 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.245+0000 7f74e6a00640 1 -- 172.21.15.27:0/3460043374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d80054d0 con 0x7f74f010b6c0 2024-09-21T07:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.245+0000 7f74e6a00640 1 -- 172.21.15.27:0/3460043374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8010430 con 0x7f74f010b6c0 2024-09-21T07:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.245+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a90a0 con 0x7f74f010b6c0 2024-09-21T07:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.247+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f0106190 con 0x7f74f010b6c0 2024-09-21T07:17:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.249+0000 7f74e6a00640 1 -- 172.21.15.27:0/3460043374 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f74d8005020 con 0x7f74f010b6c0 2024-09-21T07:17:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.251+0000 7f74e6a00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f74d40b6680 0x7f74d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.251+0000 7f74e6a00640 1 -- 172.21.15.27:0/3460043374 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f74d80e7f00 con 0x7f74f010b6c0 2024-09-21T07:17:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.251+0000 7f74ede00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f74d40b6680 0x7f74d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:25.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.252+0000 7f74ede00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f74d40b6680 0x7f74d40b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f74e0031440 tx=0x7f74e0009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:25.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.255+0000 7f74e6a00640 1 -- 172.21.15.27:0/3460043374 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74d80ae0a0 con 0x7f74f010b6c0 2024-09-21T07:17:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:25 smithi119 ceph-mon[34519]: pgmap v509: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:17:25.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:25 smithi153 ceph-mon[34747]: pgmap v509: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:17:25.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.494+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74f002e640 con 0x7f74f010b6c0 2024-09-21T07:17:25.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.495+0000 7f74e6a00640 1 -- 172.21.15.27:0/3460043374 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f74d80b2480 con 0x7f74f010b6c0 2024-09-21T07:17:25.497 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:17:25.497 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:17:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:17:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f74d40b6680 msgr2=0x7f74d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f74d40b6680 0x7f74d40b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f74e0031440 tx=0x7f74e0009560 comp rx=0 tx=0).stop 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f010b6c0 msgr2=0x7f74f019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f010b6c0 0x7f74f019cc80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9c0 tx=0x7f74d800ee80 comp rx=0 tx=0).stop 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 shutdown_connections 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74f010c010 0x7f74f01a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f74d40b6680 0x7f74d40b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74f010b6c0 0x7f74f019cc80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 --2- 172.21.15.27:0/3460043374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f0105d80 0x7f74f019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 >> 172.21.15.27:0/3460043374 conn(0x7f74f0101810 msgr2=0x7f74f0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 shutdown_connections 2024-09-21T07:17:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:25.498+0000 7f74f4a00640 1 -- 172.21.15.27:0/3460043374 wait complete. 2024-09-21T07:17:26.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:26 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3460043374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:17:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:26 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3460043374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:17:26.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:26 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3460043374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:17:26.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:17:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:27 smithi119 ceph-mon[34519]: pgmap v510: 129 pgs: 129 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:17:27.331 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:27 smithi027 ceph-mon[30779]: pgmap v510: 129 pgs: 129 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:17:27.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:27 smithi153 ceph-mon[34747]: pgmap v510: 129 pgs: 129 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:17:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.516+0000 7ff107800640 1 -- 172.21.15.27:0/3032252831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001101a0 msgr2=0x7ff1001125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.516+0000 7ff107800640 1 --2- 172.21.15.27:0/3032252831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001101a0 0x7ff1001125c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ca0 tx=0x7ff0f802a950 comp rx=0 tx=0).stop 2024-09-21T07:17:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.517+0000 7ff107800640 1 -- 172.21.15.27:0/3032252831 shutdown_connections 2024-09-21T07:17:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.517+0000 7ff107800640 1 --2- 172.21.15.27:0/3032252831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001101a0 0x7ff1001125c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.517+0000 7ff107800640 1 --2- 172.21.15.27:0/3032252831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100102f40 0x7ff10010fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.517+0000 7ff107800640 1 --2- 172.21.15.27:0/3032252831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1001025f0 0x7ff100102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.517+0000 7ff107800640 1 -- 172.21.15.27:0/3032252831 >> 172.21.15.27:0/3032252831 conn(0x7ff100079740 msgr2=0x7ff100079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.517+0000 7ff107800640 1 -- 172.21.15.27:0/3032252831 shutdown_connections 2024-09-21T07:17:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.517+0000 7ff107800640 1 -- 172.21.15.27:0/3032252831 wait complete. 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.518+0000 7ff107800640 1 Processor -- start 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.518+0000 7ff107800640 1 -- start start 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.518+0000 7ff107800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001025f0 0x7ff10019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff107800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100102f40 0x7ff10019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff107800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1001101a0 0x7ff1001a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff107800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff100115c90 con 0x7ff1001025f0 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff107800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff100115b10 con 0x7ff100102f40 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff107800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff100115e10 con 0x7ff1001101a0 2024-09-21T07:17:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff104e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001025f0 0x7ff10019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff104e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001025f0 0x7ff10019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55638/0 (socket says 172.21.15.27:55638) 2024-09-21T07:17:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff104e00640 1 -- 172.21.15.27:0/643631047 learned_addr learned my addr 172.21.15.27:0/643631047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100102f40 0x7ff10019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff105800640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1001101a0 0x7ff1001a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff104e00640 1 -- 172.21.15.27:0/643631047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1001101a0 msgr2=0x7ff1001a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff104e00640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1001101a0 0x7ff1001a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff104e00640 1 -- 172.21.15.27:0/643631047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100102f40 msgr2=0x7ff10019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.519+0000 7ff104e00640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100102f40 0x7ff10019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.520+0000 7ff104e00640 1 -- 172.21.15.27:0/643631047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1001a2510 con 0x7ff1001025f0 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.520+0000 7ff104e00640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001025f0 0x7ff10019c5a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400e9f0 tx=0x7ff0f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.520+0000 7ff0fd600640 1 -- 172.21.15.27:0/643631047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400cd80 con 0x7ff1001025f0 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.520+0000 7ff0fd600640 1 -- 172.21.15.27:0/643631047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f40108e0 con 0x7ff1001025f0 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.520+0000 7ff0fd600640 1 -- 172.21.15.27:0/643631047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4018980 con 0x7ff1001025f0 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.520+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1001a2800 con 0x7ff1001025f0 2024-09-21T07:17:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.520+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1001a8f60 con 0x7ff1001025f0 2024-09-21T07:17:27.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 Processor -- start 2024-09-21T07:17:27.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 -- start start 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da38b070 0x55e1da731da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da7322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da36a390 0x55e1da732820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e1da6a8fb0 con 0x55e1da3621b0 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1da6983e0 con 0x55e1da38b070 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.524+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1da69c1f0 con 0x55e1da36a390 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da38b070 0x55e1da731da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da38b070 0x55e1da731da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54808/0 (socket says 172.21.15.27:54808) 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 -- 172.21.15.27:0/1746522540 learned_addr learned my addr 172.21.15.27:0/1746522540 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff0fd600640 1 -- 172.21.15.27:0/643631047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff0f4010430 con 0x7ff1001025f0 2024-09-21T07:17:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da36a390 0x55e1da732820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 -- 172.21.15.27:0/1746522540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da36a390 msgr2=0x55e1da732820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da36a390 0x55e1da732820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff104e00640 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da7322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 -- 172.21.15.27:0/1746522540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 msgr2=0x55e1da7322e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da7322e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff105800640 1 -- 172.21.15.27:0/1746522540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1da662cd0 con 0x55e1da38b070 2024-09-21T07:17:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.525+0000 7ff104e00640 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da7322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:17:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.526+0000 7ff105800640 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da38b070 0x55e1da731da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8035000 tx=0x7ff0f80093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d6cc99fec9fea8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.526+0000 7ff0f2000640 1 -- 172.21.15.27:0/1746522540 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8048070 con 0x55e1da38b070 2024-09-21T07:17:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.526+0000 7ff0f2000640 1 -- 172.21.15.27:0/1746522540 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f8003e20 con 0x55e1da38b070 2024-09-21T07:17:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.526+0000 7ff0f2000640 1 -- 172.21.15.27:0/1746522540 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8034040 con 0x55e1da38b070 2024-09-21T07:17:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff0fd600640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff0e80b6700 0x7ff0e80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 -- 172.21.15.27:0/1746522540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da38b070 msgr2=0x55e1da731da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da38b070 0x55e1da731da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8035000 tx=0x7ff0f80093a0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff104e00640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff0e80b6700 0x7ff0e80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 -- 172.21.15.27:0/1746522540 shutdown_connections 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff0fd600640 1 -- 172.21.15.27:0/643631047 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff0f40e7d80 con 0x7ff1001025f0 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da36a390 0x55e1da732820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da7322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/1746522540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da38b070 0x55e1da731da0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 -- 172.21.15.27:0/1746522540 >> 172.21.15.27:0/1746522540 conn(0x55e1da37f460 msgr2=0x55e1da72fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 -- 172.21.15.27:0/1746522540 shutdown_connections 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.527+0000 7ff10e3ff740 1 -- 172.21.15.27:0/1746522540 wait complete. 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.528+0000 7ff104e00640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff0e80b6700 0x7ff0e80b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55e1da730a20 tx=0x7ff0f4000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.528+0000 7ff10e3ff740 1 -- start start 2024-09-21T07:17:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.528+0000 7ff10e3ff740 1 Processor -- start 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da73d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da73dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da749840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e1da4f9700 con 0x55e1da3621b0 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1da633580 con 0x55e1da36a390 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1da740520 con 0x55e1da38b070 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff10e3ff740 10 client.0 _ll_register_callbacks cb 0x55e1da643470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da73dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff105800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da73d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da73dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54824/0 (socket says 172.21.15.27:54824) 2024-09-21T07:17:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff0ffe00640 1 -- 172.21.15.27:0/3773429229 learned_addr learned my addr 172.21.15.27:0/3773429229 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.529+0000 7ff105800640 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da749840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0ffe00640 1 -- 172.21.15.27:0/3773429229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 msgr2=0x55e1da749840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da749840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0ffe00640 1 -- 172.21.15.27:0/3773429229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 msgr2=0x55e1da73d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da73d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0ffe00640 1 -- 172.21.15.27:0/3773429229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1da3f6650 con 0x55e1da36a390 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff105800640 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da73d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da73dae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec012740 tx=0x7ff0ec012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd44fe530418cc53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.530+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3773429229 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00f920 con 0x55e1da36a390 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.531+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3773429229 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff0ec00fac0 con 0x55e1da36a390 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.531+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1da635b90 con 0x55e1da36a390 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.531+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3773429229 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0ec00fcc0 con 0x55e1da36a390 2024-09-21T07:17:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.531+0000 7ff10e3ff740 10 client.26756 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:27.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.531+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1da650950 con 0x55e1da36a390 2024-09-21T07:17:27.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.531+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3773429229 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e1da635b90 con 0x55e1da36a390 2024-09-21T07:17:27.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3773429229 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff0ec01c420 con 0x55e1da36a390 2024-09-21T07:17:27.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff10e3ff740 10 client.26756 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:27.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff10e3ff740 10 client.26756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:17:27.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff10e3ff740 20 client.26756 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da3499f0 0x55e1da74f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff10e3ff740 4 client.26756 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1da6aaf00 con 0x55e1da3499f0 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.532+0000 7ff105800640 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da3499f0 0x55e1da74f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.533+0000 7ff105800640 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da3499f0 0x55e1da74f6b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3e9b5f3b9ad8c55 server_cookie=ab4d41250cad8099 in_seq=0 out_seq=0 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.533+0000 7ff0e6a00640 10 client.26756 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3773429229 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e1da6aaf00 con 0x55e1da3499f0 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff0e6a00640 10 client.26756 handle_command_reply: tid=0 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 client.26756 shutdown 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 msgr2=0x55e1da73dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da73dae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec012740 tx=0x7ff0ec012c00 comp rx=0 tx=0).stop 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 shutdown_connections 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da3499f0 0x55e1da74f6b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da749840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da73dae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff0e6a00640 0 client.26756 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:17:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3773429229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da73d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 >> 172.21.15.27:0/3773429229 conn(0x55e1da37f460 msgr2=0x55e1da734f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 shutdown_connections 2024-09-21T07:17:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3773429229 wait complete. 2024-09-21T07:17:27.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.534+0000 7ff10e3ff740 20 client.26756 trim_cache size 0 max 16384 2024-09-21T07:17:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 Processor -- start 2024-09-21T07:17:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 -- start start 2024-09-21T07:17:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da3621b0 0x55e1da734ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da7353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da38b070 0x55e1da73a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b57b0 con 0x55e1da38b070 2024-09-21T07:17:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b3080 con 0x55e1da36a390 2024-09-21T07:17:27.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.549+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b0ed0 con 0x55e1da3621b0 2024-09-21T07:17:27.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff105800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da38b070 0x55e1da73a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da7353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff105800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da38b070 0x55e1da73a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55668/0 (socket says 172.21.15.27:55668) 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff105800640 1 -- 172.21.15.27:0/700710796 learned_addr learned my addr 172.21.15.27:0/700710796 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff105800640 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da3621b0 0x55e1da734ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff0ffe00640 1 -- 172.21.15.27:0/700710796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da3621b0 msgr2=0x55e1da734ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da3621b0 0x55e1da734ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff0ffe00640 1 -- 172.21.15.27:0/700710796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da38b070 msgr2=0x55e1da73a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da38b070 0x55e1da73a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff0ffe00640 1 -- 172.21.15.27:0/700710796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1da7345a0 con 0x55e1da36a390 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff105800640 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da38b070 0x55e1da73a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.550+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da7353e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec01c890 tx=0x7ff0ec0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b570695ef93563 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff0f1600640 1 -- 172.21.15.27:0/700710796 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00f920 con 0x55e1da36a390 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff0f1600640 1 -- 172.21.15.27:0/700710796 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ec012e50 con 0x55e1da36a390 2024-09-21T07:17:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff0f1600640 1 -- 172.21.15.27:0/700710796 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec005470 con 0x55e1da36a390 2024-09-21T07:17:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff10e3ff740 1 -- 172.21.15.27:0/700710796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 msgr2=0x55e1da7353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da7353e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec01c890 tx=0x7ff0ec0096e0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff10e3ff740 1 -- 172.21.15.27:0/700710796 shutdown_connections 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da38b070 0x55e1da73a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da7353e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/700710796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da3621b0 0x55e1da734ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.551+0000 7ff10e3ff740 1 -- 172.21.15.27:0/700710796 >> 172.21.15.27:0/700710796 conn(0x55e1da37f460 msgr2=0x55e1da73be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.552+0000 7ff10e3ff740 1 -- 172.21.15.27:0/700710796 shutdown_connections 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.552+0000 7ff10e3ff740 1 -- 172.21.15.27:0/700710796 wait complete. 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.552+0000 7ff10e3ff740 1 -- start start 2024-09-21T07:17:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.552+0000 7ff10e3ff740 1 Processor -- start 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da741f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da742470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff10e3ff740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da7b9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b1540 con 0x55e1da3621b0 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b29b0 con 0x55e1da36a390 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff10e3ff740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b1c10 con 0x55e1da38b070 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff10e3ff740 10 client.0 _ll_register_callbacks cb 0x7ff0ec009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff105800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da741f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff105800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da741f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55672/0 (socket says 172.21.15.27:55672) 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff105800640 1 -- 172.21.15.27:0/3147050658 learned_addr learned my addr 172.21.15.27:0/3147050658 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.553+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da742470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff105800640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da7b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0ffe00640 1 -- 172.21.15.27:0/3147050658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 msgr2=0x55e1da7b9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da7b9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0ffe00640 1 -- 172.21.15.27:0/3147050658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 msgr2=0x55e1da741f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da741f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0ffe00640 1 -- 172.21.15.27:0/3147050658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1da7b95e0 con 0x55e1da36a390 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff105800640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da741f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0ffe00640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da742470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec01c9b0 tx=0x7ff0ec005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbc1515232a31f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.554+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3147050658 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec01cd10 con 0x55e1da36a390 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3147050658 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff0ec018070 con 0x55e1da36a390 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0ec012e50 con 0x55e1da36a390 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3147050658 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0ec03c050 con 0x55e1da36a390 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff10e3ff740 10 client.26762 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1da7ba860 con 0x55e1da36a390 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3147050658 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ec012e50 con 0x55e1da36a390 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3147050658 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff0ec029030 con 0x55e1da36a390 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.555+0000 7ff10e3ff740 10 client.26762 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.556+0000 7ff10e3ff740 10 client.26762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.556+0000 7ff10e3ff740 20 client.26762 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.556+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da31eda0 0x55e1da7bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.556+0000 7ff10e3ff740 4 client.26762 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:17:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.556+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1da6a69b0 con 0x55e1da31eda0 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.556+0000 7ff105800640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da31eda0 0x55e1da7bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff105800640 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da31eda0 0x55e1da7bcef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7848d362893a4599 server_cookie=920288618087b1ff in_seq=0 out_seq=0 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff0e6a00640 10 client.26762 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff0e6a00640 1 -- 172.21.15.27:0/3147050658 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1da6a69b0 con 0x55e1da31eda0 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff0e6a00640 10 client.26762 handle_command_reply: tid=0 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 client.26762 shutdown 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 msgr2=0x55e1da742470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da742470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec01c9b0 tx=0x7ff0ec005f90 comp rx=0 tx=0).stop 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 shutdown_connections 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55e1da31eda0 0x55e1da7bcef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1da38b070 0x55e1da7b9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1da36a390 0x55e1da742470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 --2- 172.21.15.27:0/3147050658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1da3621b0 0x55e1da741f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 >> 172.21.15.27:0/3147050658 conn(0x55e1da37f460 msgr2=0x55e1da734ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff0e6a00640 0 client.26762 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:17:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 shutdown_connections 2024-09-21T07:17:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 1 -- 172.21.15.27:0/3147050658 wait complete. 2024-09-21T07:17:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.557+0000 7ff10e3ff740 20 client.26762 trim_cache size 0 max 16384 2024-09-21T07:17:27.562 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff0e80b6700 msgr2=0x7ff0e80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff0e80b6700 0x7ff0e80b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55e1da730a20 tx=0x7ff0f4000bc0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001025f0 msgr2=0x7ff10019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001025f0 0x7ff10019c5a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400e9f0 tx=0x7ff0f400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 shutdown_connections 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1001101a0 0x7ff1001a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff0e80b6700 0x7ff0e80b8c10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100102f40 0x7ff10019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 --2- 172.21.15.27:0/643631047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1001025f0 0x7ff10019c5a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 >> 172.21.15.27:0/643631047 conn(0x7ff100079740 msgr2=0x7ff100109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 shutdown_connections 2024-09-21T07:17:27.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:27.558+0000 7ff107800640 1 -- 172.21.15.27:0/643631047 wait complete. 2024-09-21T07:17:28.526 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:17:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:17:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:29 smithi119 ceph-mon[34519]: pgmap v511: 129 pgs: 129 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:17:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:29 smithi027 ceph-mon[30779]: pgmap v511: 129 pgs: 129 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:17:29.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:29 smithi153 ceph-mon[34747]: pgmap v511: 129 pgs: 129 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.554+0000 7fe9ed600640 1 -- 172.21.15.27:0/2718739007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 msgr2=0x7fe9e8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.554+0000 7fe9ed600640 1 --2- 172.21.15.27:0/2718739007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8103dc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009900 tx=0x7fe9d802a410 comp rx=0 tx=0).stop 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.554+0000 7fe9ed600640 1 -- 172.21.15.27:0/2718739007 shutdown_connections 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.554+0000 7fe9ed600640 1 --2- 172.21.15.27:0/2718739007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e81092f0 0x7fe9e8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.554+0000 7fe9ed600640 1 --2- 172.21.15.27:0/2718739007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e8109c40 0x7fe9e8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.554+0000 7fe9ed600640 1 --2- 172.21.15.27:0/2718739007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8103dc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.554+0000 7fe9ed600640 1 -- 172.21.15.27:0/2718739007 >> 172.21.15.27:0/2718739007 conn(0x7fe9e80ff400 msgr2=0x7fe9e8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.555+0000 7fe9ed600640 1 -- 172.21.15.27:0/2718739007 shutdown_connections 2024-09-21T07:17:29.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.555+0000 7fe9ed600640 1 -- 172.21.15.27:0/2718739007 wait complete. 2024-09-21T07:17:29.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 Processor -- start 2024-09-21T07:17:29.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 -- start start 2024-09-21T07:17:29.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e81092f0 0x7fe9e8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e8109c40 0x7fe9e8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8113900 con 0x7fe9e81039b0 2024-09-21T07:17:29.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8113780 con 0x7fe9e81092f0 2024-09-21T07:17:29.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.556+0000 7fe9ed600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8113a80 con 0x7fe9e8109c40 2024-09-21T07:17:29.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46818/0 (socket says 172.21.15.27:46818) 2024-09-21T07:17:29.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 -- 172.21.15.27:0/175285626 learned_addr learned my addr 172.21.15.27:0/175285626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7400640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e81092f0 0x7fe9e8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e8109c40 0x7fe9e8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 -- 172.21.15.27:0/175285626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e8109c40 msgr2=0x7fe9e8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e8109c40 0x7fe9e8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 -- 172.21.15.27:0/175285626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e81092f0 msgr2=0x7fe9e8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e81092f0 0x7fe9e8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.557+0000 7fe9e7e00640 1 -- 172.21.15.27:0/175285626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e81a85b0 con 0x7fe9e81039b0 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.558+0000 7fe9e7e00640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8075710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005940 tx=0x7fe9d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.558+0000 7fe9e4c00640 1 -- 172.21.15.27:0/175285626 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d803f030 con 0x7fe9e81039b0 2024-09-21T07:17:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.558+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e81a8840 con 0x7fe9e81039b0 2024-09-21T07:17:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.558+0000 7fe9e4c00640 1 -- 172.21.15.27:0/175285626 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d8003aa0 con 0x7fe9e81039b0 2024-09-21T07:17:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.558+0000 7fe9e4c00640 1 -- 172.21.15.27:0/175285626 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8035070 con 0x7fe9e81039b0 2024-09-21T07:17:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.558+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e81a8d70 con 0x7fe9e81039b0 2024-09-21T07:17:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.561+0000 7fe9f5836740 1 Processor -- start 2024-09-21T07:17:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.561+0000 7fe9f5836740 1 -- start start 2024-09-21T07:17:29.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9f5836740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a1f145fb0 con 0x561a1ee07390 2024-09-21T07:17:29.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9f5836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561a1f1353e0 con 0x561a1ee28070 2024-09-21T07:17:29.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9f5836740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a1f1391f0 con 0x561a1edff1b0 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9e4c00640 1 -- 172.21.15.27:0/175285626 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe9d8042030 con 0x7fe9e81039b0 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9e7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9ecc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9e7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46830/0 (socket says 172.21.15.27:46830) 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.562+0000 7fe9e7400640 1 -- 172.21.15.27:0/3584705246 learned_addr learned my addr 172.21.15.27:0/3584705246 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9e7e00640 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9ecc00640 1 -- 172.21.15.27:0/3584705246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 msgr2=0x561a1f1cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9ecc00640 1 -- 172.21.15.27:0/3584705246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 msgr2=0x561a1f1cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9ecc00640 1 -- 172.21.15.27:0/3584705246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1f0ffcd0 con 0x561a1ee28070 2024-09-21T07:17:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1cef70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009f30 tx=0x7fe9e0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d2a9bec7c59a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.563+0000 7fe9dd200640 1 -- 172.21.15.27:0/3584705246 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0016070 con 0x561a1ee28070 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.564+0000 7fe9dd200640 1 -- 172.21.15.27:0/3584705246 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e0004d20 con 0x561a1ee28070 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.564+0000 7fe9dd200640 1 -- 172.21.15.27:0/3584705246 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0012740 con 0x561a1ee28070 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.564+0000 7fe9e4c00640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9cc0b6680 0x7fe9cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.564+0000 7fe9e4c00640 1 -- 172.21.15.27:0/175285626 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe9d81080d0 con 0x7fe9e81039b0 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.564+0000 7fe9e7e00640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9cc0b6680 0x7fe9cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 -- 172.21.15.27:0/3584705246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 msgr2=0x561a1f1cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1cef70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009f30 tx=0x7fe9e0010800 comp rx=0 tx=0).stop 2024-09-21T07:17:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 -- 172.21.15.27:0/3584705246 shutdown_connections 2024-09-21T07:17:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 --2- 172.21.15.27:0/3584705246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1cef70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 -- 172.21.15.27:0/3584705246 >> 172.21.15.27:0/3584705246 conn(0x561a1ee1c460 msgr2=0x561a1f1cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 -- 172.21.15.27:0/3584705246 shutdown_connections 2024-09-21T07:17:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9e7e00640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9cc0b6680 0x7fe9cc0b8b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8105580 tx=0x7fe9d802a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.565+0000 7fe9f5836740 1 -- 172.21.15.27:0/3584705246 wait complete. 2024-09-21T07:17:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.566+0000 7fe9f5836740 1 -- start start 2024-09-21T07:17:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.566+0000 7fe9f5836740 1 Processor -- start 2024-09-21T07:17:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.566+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.566+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1dabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.566+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.566+0000 7fe9f5836740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a1ef96700 con 0x561a1ee07390 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.566+0000 7fe9f5836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561a1f0d0580 con 0x561a1ee28070 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9f5836740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a1f1dd750 con 0x561a1edff1b0 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9f5836740 10 client.0 _ll_register_callbacks cb 0x561a1f0e0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1da670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52800/0 (socket says 172.21.15.27:52800) 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 -- 172.21.15.27:0/2863053691 learned_addr learned my addr 172.21.15.27:0/2863053691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1dabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 -- 172.21.15.27:0/2863053691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 msgr2=0x561a1f1e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1e6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 -- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 msgr2=0x561a1f1dabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1dabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9ecc00640 1 -- 172.21.15.27:0/2863053691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1ee93650 con 0x561a1edff1b0 2024-09-21T07:17:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.567+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1dabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1da670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009880 tx=0x7fe9e0005a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b73c7f18ec9b44d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9d1600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9d1600640 1 -- 172.21.15.27:0/2863053691 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0016070 con 0x561a1edff1b0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a1f0d2b90 con 0x561a1edff1b0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9d1600640 1 -- 172.21.15.27:0/2863053691 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe9e0003b70 con 0x561a1edff1b0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9d1600640 1 -- 172.21.15.27:0/2863053691 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9e0030050 con 0x561a1edff1b0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9f5836740 10 client.26761 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.568+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a1f0ed950 con 0x561a1edff1b0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9d1600640 1 -- 172.21.15.27:0/2863053691 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a1f0d2b90 con 0x561a1edff1b0 2024-09-21T07:17:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9d1600640 1 -- 172.21.15.27:0/2863053691 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9e000f040 con 0x561a1edff1b0 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9f5836740 10 client.26761 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9f5836740 10 client.26761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9f5836740 20 client.26761 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1ede69f0 0x561a1f1ec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9f5836740 4 client.26761 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a1f147f00 con 0x561a1ede69f0 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.569+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1ede69f0 0x561a1f1ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.570+0000 7fe9d1600640 10 client.26761 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:17:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.570+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1ede69f0 0x561a1f1ec710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e954844ab2241e82 server_cookie=f348f75629ac4c8b in_seq=0 out_seq=0 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9d1600640 1 -- 172.21.15.27:0/2863053691 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561a1f147f00 con 0x561a1ede69f0 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9d1600640 10 client.26761 handle_command_reply: tid=0 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 client.26761 shutdown 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 msgr2=0x561a1f1da670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1da670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009880 tx=0x7fe9e0005a10 comp rx=0 tx=0).stop 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 shutdown_connections 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1ede69f0 0x561a1f1ec710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee28070 0x561a1f1e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1dabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2863053691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1edff1b0 0x561a1f1da670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9d1600640 0 client.26761 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:17:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 >> 172.21.15.27:0/2863053691 conn(0x561a1ee1c460 msgr2=0x561a1f1d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 shutdown_connections 2024-09-21T07:17:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.571+0000 7fe9f5836740 1 -- 172.21.15.27:0/2863053691 wait complete. 2024-09-21T07:17:29.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.572+0000 7fe9f5836740 20 client.26761 trim_cache size 0 max 16384 2024-09-21T07:17:29.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 Processor -- start 2024-09-21T07:17:29.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 -- start start 2024-09-21T07:17:29.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1edff1b0 0x561a1f1d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee07390 0x561a1f1cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 0x561a1f1e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b5640 con 0x561a1edff1b0 2024-09-21T07:17:29.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b2f10 con 0x561a1ee07390 2024-09-21T07:17:29.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9f5836740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b0d60 con 0x561a1ee28070 2024-09-21T07:17:29.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9e7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee07390 0x561a1f1cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9e7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee07390 0x561a1f1cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:49520/0 (socket says 172.21.15.27:49520) 2024-09-21T07:17:29.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9e7400640 1 -- 172.21.15.27:0/2444960523 learned_addr learned my addr 172.21.15.27:0/2444960523 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:29.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.586+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1edff1b0 0x561a1f1d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9e7400640 1 -- 172.21.15.27:0/2444960523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 msgr2=0x561a1f1e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 0x561a1f1e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9e7400640 1 -- 172.21.15.27:0/2444960523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1edff1b0 msgr2=0x561a1f1d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1edff1b0 0x561a1f1d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9e7400640 1 -- 172.21.15.27:0/2444960523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1f1eca80 con 0x561a1ee07390 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee07390 0x561a1f1cbd10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561a1f1eccb0 tx=0x7fe9d4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147863b205d13e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9d3e00640 1 -- 172.21.15.27:0/2444960523 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4005020 con 0x561a1ee07390 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.587+0000 7fe9d3e00640 1 -- 172.21.15.27:0/2444960523 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d400f820 con 0x561a1ee07390 2024-09-21T07:17:29.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9d3e00640 1 -- 172.21.15.27:0/2444960523 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4004a00 con 0x561a1ee07390 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 -- 172.21.15.27:0/2444960523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee07390 msgr2=0x561a1f1cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee07390 0x561a1f1cbd10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561a1f1eccb0 tx=0x7fe9d4005f90 comp rx=0 tx=0).stop 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 -- 172.21.15.27:0/2444960523 shutdown_connections 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 0x561a1f1e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1ee07390 0x561a1f1cbd10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2444960523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1edff1b0 0x561a1f1d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 -- 172.21.15.27:0/2444960523 >> 172.21.15.27:0/2444960523 conn(0x561a1ee1c460 msgr2=0x561a1f1cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:29.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.588+0000 7fe9f5836740 1 -- 172.21.15.27:0/2444960523 shutdown_connections 2024-09-21T07:17:29.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.589+0000 7fe9f5836740 1 -- 172.21.15.27:0/2444960523 wait complete. 2024-09-21T07:17:29.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.589+0000 7fe9f5836740 1 -- start start 2024-09-21T07:17:29.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.589+0000 7fe9f5836740 1 Processor -- start 2024-09-21T07:17:29.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1edff1b0 0x561a1f1f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1f2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9f5836740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 0x561a1f1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9f5836740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b13d0 con 0x561a1ee07390 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9f5836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b2840 con 0x561a1edff1b0 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9f5836740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b1aa0 con 0x561a1ee28070 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9f5836740 10 client.0 _ll_register_callbacks cb 0x7fe9e0013b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9e7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9e7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46850/0 (socket says 172.21.15.27:46850) 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9e7400640 1 -- 172.21.15.27:0/2484804888 learned_addr learned my addr 172.21.15.27:0/2484804888 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1edff1b0 0x561a1f1f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 0x561a1f1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9e7400640 1 -- 172.21.15.27:0/2484804888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 msgr2=0x561a1f1f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.590+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 0x561a1f1f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9e7400640 1 -- 172.21.15.27:0/2484804888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1edff1b0 msgr2=0x561a1f1f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1edff1b0 0x561a1f1f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9e7400640 1 -- 172.21.15.27:0/2484804888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1f1e2060 con 0x561a1ee07390 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1edff1b0 0x561a1f1f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9e7400640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1f2170 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4014010 tx=0x7fe9d4012330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe86358ac0357265 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9d1600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9d1600640 1 -- 172.21.15.27:0/2484804888 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4036060 con 0x561a1ee07390 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9d1600640 1 -- 172.21.15.27:0/2484804888 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe9d4005020 con 0x561a1ee07390 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.591+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d400f820 con 0x561a1ee07390 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9d1600640 1 -- 172.21.15.27:0/2484804888 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9d403d050 con 0x561a1ee07390 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 10 client.16746 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a1f257120 con 0x561a1ee07390 2024-09-21T07:17:29.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9d1600640 1 -- 172.21.15.27:0/2484804888 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d400f820 con 0x561a1ee07390 2024-09-21T07:17:29.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9d1600640 1 -- 172.21.15.27:0/2484804888 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9d4030030 con 0x561a1ee07390 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 10 client.16746 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 10 client.16746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 20 client.16746 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1edbbda0 0x561a1f2598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 4 client.16746 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.592+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a1f1439b0 con 0x561a1edbbda0 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.593+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1edbbda0 0x561a1f2598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.593+0000 7fe9d1600640 10 client.16746 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.593+0000 7fe9ecc00640 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1edbbda0 0x561a1f2598a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ecf3ef56520037b2 server_cookie=d9477ae6e2912b23 in_seq=0 out_seq=0 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.593+0000 7fe9d1600640 1 -- 172.21.15.27:0/2484804888 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a1f1439b0 con 0x561a1edbbda0 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.593+0000 7fe9d1600640 10 client.16746 handle_command_reply: tid=0 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.593+0000 7fe9f5836740 1 client.16746 shutdown 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 msgr2=0x561a1f1f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1f2170 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4014010 tx=0x7fe9d4012330 comp rx=0 tx=0).stop 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 shutdown_connections 2024-09-21T07:17:29.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x561a1edbbda0 0x561a1f2598a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1ee28070 0x561a1f1f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1ee07390 0x561a1f1f2170 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 --2- 172.21.15.27:0/2484804888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a1edff1b0 0x561a1f1f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9d1600640 0 client.16746 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 >> 172.21.15.27:0/2484804888 conn(0x561a1ee1c460 msgr2=0x561a1f1d9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 shutdown_connections 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 1 -- 172.21.15.27:0/2484804888 wait complete. 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.594+0000 7fe9f5836740 20 client.16746 trim_cache size 0 max 16384 2024-09-21T07:17:29.598 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9cc0b6680 msgr2=0x7fe9cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9cc0b6680 0x7fe9cc0b8b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8105580 tx=0x7fe9d802a410 comp rx=0 tx=0).stop 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 msgr2=0x7fe9e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8075710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005940 tx=0x7fe9d8004270 comp rx=0 tx=0).stop 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 shutdown_connections 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e8109c40 0x7fe9e8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9cc0b6680 0x7fe9cc0b8b90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e81092f0 0x7fe9e8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 --2- 172.21.15.27:0/175285626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e81039b0 0x7fe9e8075710 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:29.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 >> 172.21.15.27:0/175285626 conn(0x7fe9e80ff400 msgr2=0x7fe9e8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:29.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 shutdown_connections 2024-09-21T07:17:29.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:29.595+0000 7fe9ed600640 1 -- 172.21.15.27:0/175285626 wait complete. 2024-09-21T07:17:30.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:17:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:17:31.286 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:31 smithi027 ceph-mon[30779]: pgmap v512: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-09-21T07:17:31.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:31 smithi153 ceph-mon[34747]: pgmap v512: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-09-21T07:17:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 -- 172.21.15.27:0/2680676216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 msgr2=0x7fd9ac106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 --2- 172.21.15.27:0/2680676216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac106190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009900 tx=0x7fd9a002a410 comp rx=0 tx=0).stop 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 -- 172.21.15.27:0/2680676216 shutdown_connections 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 --2- 172.21.15.27:0/2680676216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9ac10c010 0x7fd9ac112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 --2- 172.21.15.27:0/2680676216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ac10b6c0 0x7fd9ac10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 --2- 172.21.15.27:0/2680676216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac106190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 -- 172.21.15.27:0/2680676216 >> 172.21.15.27:0/2680676216 conn(0x7fd9ac101810 msgr2=0x7fd9ac103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 -- 172.21.15.27:0/2680676216 shutdown_connections 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.472+0000 7fd9b4800640 1 -- 172.21.15.27:0/2680676216 wait complete. 2024-09-21T07:17:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.473+0000 7fd9b4800640 1 Processor -- start 2024-09-21T07:17:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b4800640 1 -- start start 2024-09-21T07:17:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b4800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b4800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ac10b6c0 0x7fd9ac19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b4800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9ac10c010 0x7fd9ac1a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b4800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac115d20 con 0x7fd9ac105d80 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b4800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac115ba0 con 0x7fd9ac10b6c0 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b4800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac115ea0 con 0x7fd9ac10c010 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.474+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9ac10c010 0x7fd9ac1a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ac10b6c0 0x7fd9ac19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46862/0 (socket says 172.21.15.27:46862) 2024-09-21T07:17:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 -- 172.21.15.27:0/1244647042 learned_addr learned my addr 172.21.15.27:0/1244647042 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 -- 172.21.15.27:0/1244647042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9ac10c010 msgr2=0x7fd9ac1a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9ac10c010 0x7fd9ac1a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 -- 172.21.15.27:0/1244647042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ac10b6c0 msgr2=0x7fd9ac19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ac10b6c0 0x7fd9ac19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 -- 172.21.15.27:0/1244647042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac1a2680 con 0x7fd9ac105d80 2024-09-21T07:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9ac10c010 0x7fd9ac1a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1400640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ac10b6c0 0x7fd9ac19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.475+0000 7fd9b1e00640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac19c710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005ab0 tx=0x7fd9a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.476+0000 7fd9a6a00640 1 -- 172.21.15.27:0/1244647042 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a003f070 con 0x7fd9ac105d80 2024-09-21T07:17:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.476+0000 7fd9a6a00640 1 -- 172.21.15.27:0/1244647042 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a0031b60 con 0x7fd9ac105d80 2024-09-21T07:17:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.476+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac1a2910 con 0x7fd9ac105d80 2024-09-21T07:17:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.476+0000 7fd9a6a00640 1 -- 172.21.15.27:0/1244647042 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0035030 con 0x7fd9ac105d80 2024-09-21T07:17:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.476+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac1a90d0 con 0x7fd9ac105d80 2024-09-21T07:17:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.479+0000 7fd9bb474740 1 Processor -- start 2024-09-21T07:17:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.479+0000 7fd9bb474740 1 -- start start 2024-09-21T07:17:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9a6a00640 1 -- 172.21.15.27:0/1244647042 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd9a0042070 con 0x7fd9ac105d80 2024-09-21T07:17:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f6e070 0x562af7314da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af73152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7315820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9bb474740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562af728bfb0 con 0x562af6f4d390 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9bb474740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562af727b3e0 con 0x562af6f6e070 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9bb474740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562af727f1f0 con 0x562af6f451b0 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7315820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f6e070 0x562af7314da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7315820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46876/0 (socket says 172.21.15.27:46876) 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9b1400640 1 -- 172.21.15.27:0/3402598798 learned_addr learned my addr 172.21.15.27:0/3402598798 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.480+0000 7fd9b1e00640 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af73152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd9b2800640 1 -- 172.21.15.27:0/3402598798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 msgr2=0x562af73152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd9b2800640 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af73152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd9b2800640 1 -- 172.21.15.27:0/3402598798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 msgr2=0x562af7315820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd9b2800640 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7315820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd9b2800640 1 -- 172.21.15.27:0/3402598798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562af7245cd0 con 0x562af6f6e070 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd9b1e00640 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af73152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd9b2800640 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f6e070 0x562af7314da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8011760 tx=0x7fd9a8011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=820f5db875689023 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.481+0000 7fd99b000640 1 -- 172.21.15.27:0/3402598798 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8004d20 con 0x562af6f6e070 2024-09-21T07:17:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.482+0000 7fd9a6a00640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9940b6700 0x7fd9940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.482+0000 7fd99b000640 1 -- 172.21.15.27:0/3402598798 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8003970 con 0x562af6f6e070 2024-09-21T07:17:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.482+0000 7fd99b000640 1 -- 172.21.15.27:0/3402598798 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80043a0 con 0x562af6f6e070 2024-09-21T07:17:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.482+0000 7fd9a6a00640 1 -- 172.21.15.27:0/1244647042 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd9a01080d0 con 0x7fd9ac105d80 2024-09-21T07:17:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.482+0000 7fd9b1e00640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9940b6700 0x7fd9940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 -- 172.21.15.27:0/3402598798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f6e070 msgr2=0x562af7314da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f6e070 0x562af7314da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8011760 tx=0x7fd9a8011c20 comp rx=0 tx=0).stop 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 -- 172.21.15.27:0/3402598798 shutdown_connections 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af73152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9b1e00640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9940b6700 0x7fd9940b8c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562af7313a20 tx=0x7fd9a003b790 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7315820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 --2- 172.21.15.27:0/3402598798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f6e070 0x562af7314da0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 -- 172.21.15.27:0/3402598798 >> 172.21.15.27:0/3402598798 conn(0x562af6f62460 msgr2=0x562af7312c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 -- 172.21.15.27:0/3402598798 shutdown_connections 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 -- 172.21.15.27:0/3402598798 wait complete. 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.483+0000 7fd9bb474740 1 -- start start 2024-09-21T07:17:31.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 1 Processor -- start 2024-09-21T07:17:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7320520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7320a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f6e070 0x562af732c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562af70dc700 con 0x562af6f4d390 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562af7216580 con 0x562af6f451b0 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562af7323580 con 0x562af6f6e070 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.484+0000 7fd9bb474740 10 client.0 _ll_register_callbacks cb 0x562af7226470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7320520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7320520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:49570/0 (socket says 172.21.15.27:49570) 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7320a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 -- 172.21.15.27:0/1565822570 learned_addr learned my addr 172.21.15.27:0/1565822570 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f6e070 0x562af732c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 -- 172.21.15.27:0/1565822570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f6e070 msgr2=0x562af732c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f6e070 0x562af732c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 -- 172.21.15.27:0/1565822570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 msgr2=0x562af7320a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7320a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 -- 172.21.15.27:0/1565822570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562af6fd9650 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7320520 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80115a0 tx=0x7fd9a8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8ee0bd6a4e9e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.485+0000 7fd98f400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd98f400640 1 -- 172.21.15.27:0/1565822570 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a802d060 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd98f400640 1 -- 172.21.15.27:0/1565822570 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd9a8011230 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562af7218b90 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd98f400640 1 -- 172.21.15.27:0/1565822570 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9a801d070 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd9bb474740 10 client.26792 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562af7233950 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd98f400640 1 -- 172.21.15.27:0/1565822570 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562af7218b90 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd98f400640 1 -- 172.21.15.27:0/1565822570 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a80105c0 con 0x562af6f451b0 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd9bb474740 10 client.26792 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd9bb474740 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd9bb474740 20 client.26792 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.486+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f2c9f0 0x562af7332630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.487+0000 7fd9bb474740 4 client.26792 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:31.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.487+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562af728df00 con 0x562af6f2c9f0 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.487+0000 7fd9b1400640 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f2c9f0 0x562af7332630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.488+0000 7fd9b1400640 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f2c9f0 0x562af7332630 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d630274c8e10507 server_cookie=2e6cc24412d9a693 in_seq=0 out_seq=0 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.488+0000 7fd98f400640 10 client.26792 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.488+0000 7fd98f400640 1 -- 172.21.15.27:0/1565822570 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562af728df00 con 0x562af6f2c9f0 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.488+0000 7fd98f400640 10 client.26792 handle_command_reply: tid=0 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.488+0000 7fd9bb474740 1 client.26792 shutdown 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 msgr2=0x562af7320520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7320520 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80115a0 tx=0x7fd9a8005bd0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 shutdown_connections 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f2c9f0 0x562af7332630 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f6e070 0x562af732c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f4d390 0x562af7320a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd98f400640 0 client.26792 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:31.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1565822570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7320520 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 >> 172.21.15.27:0/1565822570 conn(0x562af6f62460 msgr2=0x562af7317f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:31.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 shutdown_connections 2024-09-21T07:17:31.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 1 -- 172.21.15.27:0/1565822570 wait complete. 2024-09-21T07:17:31.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.489+0000 7fd9bb474740 20 client.26792 trim_cache size 0 max 16384 2024-09-21T07:17:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.503+0000 7fd9bb474740 1 Processor -- start 2024-09-21T07:17:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.503+0000 7fd9bb474740 1 -- start start 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.503+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af7322150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f4d390 0x562af732c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af732cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9bb474740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1b5850 con 0x562af6f6e070 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9bb474740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1b3120 con 0x562af6f4d390 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9bb474740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1b0f70 con 0x562af6f451b0 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af732cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af732cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46888/0 (socket says 172.21.15.27:46888) 2024-09-21T07:17:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 -- 172.21.15.27:0/1006519992 learned_addr learned my addr 172.21.15.27:0/1006519992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b1400640 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f4d390 0x562af732c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af7322150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 -- 172.21.15.27:0/1006519992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 msgr2=0x562af7322150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af7322150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 -- 172.21.15.27:0/1006519992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f4d390 msgr2=0x562af732c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f4d390 0x562af732c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b2800640 1 -- 172.21.15.27:0/1006519992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562af7317340 con 0x562af6f6e070 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.504+0000 7fd9b1400640 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f4d390 0x562af732c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.505+0000 7fd9b2800640 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af732cce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80109e0 tx=0x7fd9a8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb82113e426fcbb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.505+0000 7fd99a600640 1 -- 172.21.15.27:0/1006519992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a802d040 con 0x562af6f6e070 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.505+0000 7fd99a600640 1 -- 172.21.15.27:0/1006519992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a801e040 con 0x562af6f6e070 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.505+0000 7fd99a600640 1 -- 172.21.15.27:0/1006519992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8003970 con 0x562af6f6e070 2024-09-21T07:17:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 -- 172.21.15.27:0/1006519992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 msgr2=0x562af732cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af732cce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80109e0 tx=0x7fd9a8011c20 comp rx=0 tx=0).stop 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 -- 172.21.15.27:0/1006519992 shutdown_connections 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af732cce0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f4d390 0x562af732c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 --2- 172.21.15.27:0/1006519992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f451b0 0x562af7322150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 -- 172.21.15.27:0/1006519992 >> 172.21.15.27:0/1006519992 conn(0x562af6f62460 msgr2=0x562af7321a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 -- 172.21.15.27:0/1006519992 shutdown_connections 2024-09-21T07:17:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 -- 172.21.15.27:0/1006519992 wait complete. 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.506+0000 7fd9bb474740 1 -- start start 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 1 Processor -- start 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7324b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f4d390 0x562af7325060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af739bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1b15e0 con 0x562af6f6e070 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1b2a50 con 0x562af6f451b0 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1b1cb0 con 0x562af6f4d390 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9bb474740 10 client.0 _ll_register_callbacks cb 0x7fd9a8004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7324b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f4d390 0x562af7325060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7324b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:49592/0 (socket says 172.21.15.27:49592) 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.507+0000 7fd9b2800640 1 -- 172.21.15.27:0/2876965857 learned_addr learned my addr 172.21.15.27:0/2876965857 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b2800640 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af739bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b1400640 1 -- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 msgr2=0x562af7324b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b1400640 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7324b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b1400640 1 -- 172.21.15.27:0/2876965857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 msgr2=0x562af739bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b1400640 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af739bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b1400640 1 -- 172.21.15.27:0/2876965857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562af73258e0 con 0x562af6f4d390 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b2800640 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7324b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd9b1400640 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f4d390 0x562af7325060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x562af7321640 tx=0x7fd99c012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bba779caf1db18b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.508+0000 7fd98f400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.509+0000 7fd98f400640 1 -- 172.21.15.27:0/2876965857 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011b80 con 0x562af6f4d390 2024-09-21T07:17:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.509+0000 7fd98f400640 1 -- 172.21.15.27:0/2876965857 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd99c011d20 con 0x562af6f4d390 2024-09-21T07:17:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.509+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a801e040 con 0x562af6f4d390 2024-09-21T07:17:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.509+0000 7fd98f400640 1 -- 172.21.15.27:0/2876965857 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd99c032050 con 0x562af6f4d390 2024-09-21T07:17:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.509+0000 7fd9bb474740 10 client.26782 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.509+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562af739d3b0 con 0x562af6f4d390 2024-09-21T07:17:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.509+0000 7fd98f400640 1 -- 172.21.15.27:0/2876965857 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a801e040 con 0x562af6f4d390 2024-09-21T07:17:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd98f400640 1 -- 172.21.15.27:0/2876965857 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd99c01f400 con 0x562af6f4d390 2024-09-21T07:17:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd9bb474740 10 client.26782 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd9bb474740 10 client.26782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd9bb474740 20 client.26782 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd9bb474740 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f01da0 0x562af739fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd9bb474740 4 client.26782 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562af72899b0 con 0x562af6f01da0 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.510+0000 7fd9b2800640 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f01da0 0x562af739fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9b2800640 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f01da0 0x562af739fb70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9b3a2e1fa6007b server_cookie=4f1e385cd59539a3 in_seq=0 out_seq=0 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd98f400640 10 client.26782 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd98f400640 1 -- 172.21.15.27:0/2876965857 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562af72899b0 con 0x562af6f01da0 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd98f400640 10 client.26782 handle_command_reply: tid=0 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 client.26782 shutdown 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f4d390 msgr2=0x562af7325060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f4d390 0x562af7325060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x562af7321640 tx=0x7fd99c012bf0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 shutdown_connections 2024-09-21T07:17:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562af6f01da0 0x562af739fb70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562af6f4d390 0x562af7325060 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562af6f6e070 0x562af739bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 --2- 172.21.15.27:0/2876965857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562af6f451b0 0x562af7324b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd98f400640 0 client.26782 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.511+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 >> 172.21.15.27:0/2876965857 conn(0x562af6f62460 msgr2=0x562af7312c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.512+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 shutdown_connections 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.512+0000 7fd9bb474740 1 -- 172.21.15.27:0/2876965857 wait complete. 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.512+0000 7fd9bb474740 20 client.26782 trim_cache size 0 max 16384 2024-09-21T07:17:31.516 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.512+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9940b6700 msgr2=0x7fd9940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.512+0000 7fd9b4800640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9940b6700 0x7fd9940b8c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562af7313a20 tx=0x7fd9a003b790 comp rx=0 tx=0).stop 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.512+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 msgr2=0x7fd9ac19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.512+0000 7fd9b4800640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac19c710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005ab0 tx=0x7fd9a0003c10 comp rx=0 tx=0).stop 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 shutdown_connections 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9ac10c010 0x7fd9ac1a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9940b6700 0x7fd9940b8c10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ac10b6c0 0x7fd9ac19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 --2- 172.21.15.27:0/1244647042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9ac105d80 0x7fd9ac19c710 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 >> 172.21.15.27:0/1244647042 conn(0x7fd9ac101810 msgr2=0x7fd9ac103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 shutdown_connections 2024-09-21T07:17:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:31.513+0000 7fd9b4800640 1 -- 172.21.15.27:0/1244647042 wait complete. 2024-09-21T07:17:31.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:31 smithi119 ceph-mon[34519]: pgmap v512: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-09-21T07:17:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:33 smithi027 ceph-mon[30779]: pgmap v513: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-21T07:17:33.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:33 smithi153 ceph-mon[34747]: pgmap v513: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-21T07:17:33.472 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:17:33.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:17:33.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:33 smithi119 ceph-mon[34519]: pgmap v513: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-21T07:17:33.871 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 -- 172.21.15.27:0/2904722379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c10b6d0 msgr2=0x7f007c10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 --2- 172.21.15.27:0/2904722379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c10b6d0 0x7f007c10bae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0068009900 tx=0x7f006802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 -- 172.21.15.27:0/2904722379 shutdown_connections 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 --2- 172.21.15.27:0/2904722379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c10b6d0 0x7f007c10bae0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 --2- 172.21.15.27:0/2904722379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10c020 0x7f007c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 --2- 172.21.15.27:0/2904722379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007c105d90 0x7f007c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 -- 172.21.15.27:0/2904722379 >> 172.21.15.27:0/2904722379 conn(0x7f007c101860 msgr2=0x7f007c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 -- 172.21.15.27:0/2904722379 shutdown_connections 2024-09-21T07:17:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.493+0000 7f0083600640 1 -- 172.21.15.27:0/2904722379 wait complete. 2024-09-21T07:17:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.494+0000 7f0083600640 1 Processor -- start 2024-09-21T07:17:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.494+0000 7f0083600640 1 -- start start 2024-09-21T07:17:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0083600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c105d90 0x7f007c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0083600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10b6d0 0x7f007c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0083600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007c10c020 0x7f007c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0083600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f007c115d30 con 0x7f007c10b6d0 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0083600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f007c115bb0 con 0x7f007c10c020 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0083600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f007c115eb0 con 0x7f007c105d90 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f007be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10b6d0 0x7f007c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f007be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10b6d0 0x7f007c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46912/0 (socket says 172.21.15.27:46912) 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f007be00640 1 -- 172.21.15.27:0/3350520685 learned_addr learned my addr 172.21.15.27:0/3350520685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0080c00640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c105d90 0x7f007c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.495+0000 7f0081600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007c10c020 0x7f007c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.496+0000 7f007be00640 1 -- 172.21.15.27:0/3350520685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c105d90 msgr2=0x7f007c19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.496+0000 7f007be00640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c105d90 0x7f007c19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.496+0000 7f007be00640 1 -- 172.21.15.27:0/3350520685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007c10c020 msgr2=0x7f007c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.496+0000 7f007be00640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007c10c020 0x7f007c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.496+0000 7f007be00640 1 -- 172.21.15.27:0/3350520685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c1a2690 con 0x7f007c10b6d0 2024-09-21T07:17:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.496+0000 7f007be00640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10b6d0 0x7f007c19ccc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0068005ab0 tx=0x7f0068003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.496+0000 7f0079600640 1 -- 172.21.15.27:0/3350520685 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006803f070 con 0x7f007c10b6d0 2024-09-21T07:17:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.497+0000 7f0079600640 1 -- 172.21.15.27:0/3350520685 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0068031b50 con 0x7f007c10b6d0 2024-09-21T07:17:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.497+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c1a2920 con 0x7f007c10b6d0 2024-09-21T07:17:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.497+0000 7f0079600640 1 -- 172.21.15.27:0/3350520685 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068035030 con 0x7f007c10b6d0 2024-09-21T07:17:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.497+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c1a90e0 con 0x7f007c10b6d0 2024-09-21T07:17:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.500+0000 7f008a3fc740 1 Processor -- start 2024-09-21T07:17:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.500+0000 7f008a3fc740 1 -- start start 2024-09-21T07:17:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc85751d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85752250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f0079600640 1 -- 172.21.15.27:0/3350520685 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0068042070 con 0x7f007c10b6d0 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc85752790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f008a3fc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cc856c7e90 con 0x55cc853aa350 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f008a3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cc856b72c0 con 0x55cc85389670 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f008a3fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cc856bb0d0 con 0x55cc85381490 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f0081600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc85751d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f0080c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85752250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f0081600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc85751d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46922/0 (socket says 172.21.15.27:46922) 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f0081600640 1 -- 172.21.15.27:0/2450510095 learned_addr learned my addr 172.21.15.27:0/2450510095 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.501+0000 7f007be00640 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc85752790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.502+0000 7f0081600640 1 -- 172.21.15.27:0/2450510095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 msgr2=0x55cc85752250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.502+0000 7f0081600640 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85752250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.502+0000 7f0081600640 1 -- 172.21.15.27:0/2450510095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 msgr2=0x55cc85752790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.502+0000 7f0081600640 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc85752790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.502+0000 7f0081600640 1 -- 172.21.15.27:0/2450510095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc85415930 con 0x55cc853aa350 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.502+0000 7f0081600640 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc85751d10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0074011440 tx=0x7f0074011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17beb5f4743d102f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.502+0000 7f006dc00640 1 -- 172.21.15.27:0/2450510095 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074003c30 con 0x55cc853aa350 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.503+0000 7f006dc00640 1 -- 172.21.15.27:0/2450510095 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0074003dd0 con 0x55cc853aa350 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.503+0000 7f006dc00640 1 -- 172.21.15.27:0/2450510095 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007400fae0 con 0x55cc853aa350 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.503+0000 7f0079600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00640b6680 0x7f00640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.503+0000 7f0079600640 1 -- 172.21.15.27:0/3350520685 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0068108060 con 0x7f007c10b6d0 2024-09-21T07:17:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.503+0000 7f0080c00640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00640b6680 0x7f00640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 -- 172.21.15.27:0/2450510095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 msgr2=0x55cc85751d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc85751d10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0074011440 tx=0x7f0074011900 comp rx=0 tx=0).stop 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 -- 172.21.15.27:0/2450510095 shutdown_connections 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85752250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc85752790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 --2- 172.21.15.27:0/2450510095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc85751d10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 -- 172.21.15.27:0/2450510095 >> 172.21.15.27:0/2450510095 conn(0x55cc8539e740 msgr2=0x55cc8574fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f0080c00640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00640b6680 0x7f00640b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55cc85750990 tx=0x7f0070005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 -- 172.21.15.27:0/2450510095 shutdown_connections 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.504+0000 7f008a3fc740 1 -- 172.21.15.27:0/2450510095 wait complete. 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 -- start start 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 Processor -- start 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85381490 0x55cc8575d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85389670 0x55cc8575ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc853aa350 0x55cc85769810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cc85518890 con 0x55cc85381490 2024-09-21T07:17:34.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cc85652160 con 0x55cc853aa350 2024-09-21T07:17:34.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cc857604f0 con 0x55cc85389670 2024-09-21T07:17:34.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.505+0000 7f008a3fc740 10 client.0 _ll_register_callbacks cb 0x55cc8566ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:34.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85381490 0x55cc8575d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85381490 0x55cc8575d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46928/0 (socket says 172.21.15.27:46928) 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 -- 172.21.15.27:0/3356512520 learned_addr learned my addr 172.21.15.27:0/3356512520 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f007be00640 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85389670 0x55cc8575ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0081600640 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc853aa350 0x55cc85769810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 -- 172.21.15.27:0/3356512520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85389670 msgr2=0x55cc8575ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85389670 0x55cc8575ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 -- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc853aa350 msgr2=0x55cc85769810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc853aa350 0x55cc85769810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.506+0000 7f0080c00640 1 -- 172.21.15.27:0/3356512520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc85769d50 con 0x55cc85381490 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f0080c00640 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85381490 0x55cc8575d880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f007001e9f0 tx=0x7f0070004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed69df19cdc8d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f0062000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f0062000640 1 -- 172.21.15.27:0/3356512520 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007002e060 con 0x55cc85381490 2024-09-21T07:17:34.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f0062000640 1 -- 172.21.15.27:0/3356512520 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0070018030 con 0x55cc85381490 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0074003dd0 con 0x55cc85381490 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f0062000640 1 -- 172.21.15.27:0/3356512520 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0070035050 con 0x55cc85381490 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f008a3fc740 10 client.16776 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.507+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc8576c6b0 con 0x55cc85381490 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f0062000640 1 -- 172.21.15.27:0/3356512520 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0074003dd0 con 0x55cc85381490 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f0062000640 1 -- 172.21.15.27:0/3356512520 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f007001c3a0 con 0x55cc85381490 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f008a3fc740 10 client.16776 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f008a3fc740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f008a3fc740 20 client.16776 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f008a3fc740 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc85368cd0 0x55cc8576fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f008a3fc740 4 client.16776 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.508+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc856c9de0 con 0x55cc85368cd0 2024-09-21T07:17:34.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.509+0000 7f007be00640 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc85368cd0 0x55cc8576fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.509+0000 7f007be00640 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc85368cd0 0x55cc8576fbe0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701e060ec5c92896 server_cookie=c62eced3bd615f69 in_seq=0 out_seq=0 2024-09-21T07:17:34.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.509+0000 7f0062000640 10 client.16776 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f0062000640 1 -- 172.21.15.27:0/3356512520 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc856c9de0 con 0x55cc85368cd0 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f0062000640 10 client.16776 handle_command_reply: tid=0 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 client.16776 shutdown 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85381490 msgr2=0x55cc8575d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85381490 0x55cc8575d880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f007001e9f0 tx=0x7f0070004730 comp rx=0 tx=0).stop 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 shutdown_connections 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc85368cd0 0x55cc8576fbe0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85389670 0x55cc8575ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc853aa350 0x55cc85769810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 --2- 172.21.15.27:0/3356512520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85381490 0x55cc8575d880 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 >> 172.21.15.27:0/3356512520 conn(0x55cc8539e740 msgr2=0x55cc8574e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.510+0000 7f0062000640 0 client.16776 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.511+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 shutdown_connections 2024-09-21T07:17:34.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.511+0000 7f008a3fc740 1 -- 172.21.15.27:0/3356512520 wait complete. 2024-09-21T07:17:34.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.511+0000 7f008a3fc740 20 client.16776 trim_cache size 0 max 16384 2024-09-21T07:17:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.529+0000 7f008a3fc740 1 Processor -- start 2024-09-21T07:17:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f008a3fc740 1 -- start start 2024-09-21T07:17:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85757e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc8574e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc8574e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f008a3fc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1b5890 con 0x55cc853aa350 2024-09-21T07:17:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f008a3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1b3160 con 0x55cc85389670 2024-09-21T07:17:34.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f008a3fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1b0fb0 con 0x55cc85381490 2024-09-21T07:17:34.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f0081600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc8574e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f0081600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc8574e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46930/0 (socket says 172.21.15.27:46930) 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f0080c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85757e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f0081600640 1 -- 172.21.15.27:0/174848840 learned_addr learned my addr 172.21.15.27:0/174848840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.530+0000 7f007be00640 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc8574e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f0081600640 1 -- 172.21.15.27:0/174848840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 msgr2=0x55cc85757e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f0081600640 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85757e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f0081600640 1 -- 172.21.15.27:0/174848840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 msgr2=0x55cc8574e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f0081600640 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc8574e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f0081600640 1 -- 172.21.15.27:0/174848840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc85758f10 con 0x55cc853aa350 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f0080c00640 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85757e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f007be00640 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc8574e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.531+0000 7f0081600640 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc8574e7c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f007400f5b0 tx=0x7f007400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66eb42ec2379f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f006d200640 1 -- 172.21.15.27:0/174848840 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074003cf0 con 0x55cc853aa350 2024-09-21T07:17:34.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f006d200640 1 -- 172.21.15.27:0/174848840 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007400fc60 con 0x55cc853aa350 2024-09-21T07:17:34.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f006d200640 1 -- 172.21.15.27:0/174848840 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00740057b0 con 0x55cc853aa350 2024-09-21T07:17:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f008a3fc740 1 -- 172.21.15.27:0/174848840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 msgr2=0x55cc8574e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f008a3fc740 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc8574e7c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f007400f5b0 tx=0x7f007400f680 comp rx=0 tx=0).stop 2024-09-21T07:17:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f008a3fc740 1 -- 172.21.15.27:0/174848840 shutdown_connections 2024-09-21T07:17:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f008a3fc740 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc853aa350 0x55cc8574e7c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f008a3fc740 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85389670 0x55cc8574e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f008a3fc740 1 --2- 172.21.15.27:0/174848840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc85381490 0x55cc85757e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.532+0000 7f008a3fc740 1 -- 172.21.15.27:0/174848840 >> 172.21.15.27:0/174848840 conn(0x55cc8539e740 msgr2=0x55cc8575bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.533+0000 7f008a3fc740 1 -- 172.21.15.27:0/174848840 shutdown_connections 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.533+0000 7f008a3fc740 1 -- 172.21.15.27:0/174848840 wait complete. 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.533+0000 7f008a3fc740 1 -- start start 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.533+0000 7f008a3fc740 1 Processor -- start 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85381490 0x55cc85765780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85389670 0x55cc85765cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f008a3fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc853aa350 0x55cc85760730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f008a3fc740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1b1620 con 0x55cc85389670 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f008a3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1b2a90 con 0x55cc85381490 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f008a3fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1b1cf0 con 0x55cc853aa350 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f008a3fc740 10 client.0 _ll_register_callbacks cb 0x7f0070009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f007be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85389670 0x55cc85765cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f0080c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85381490 0x55cc85765780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f007be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85389670 0x55cc85765cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46932/0 (socket says 172.21.15.27:46932) 2024-09-21T07:17:34.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f007be00640 1 -- 172.21.15.27:0/1132957810 learned_addr learned my addr 172.21.15.27:0/1132957810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:34.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.534+0000 7f0081600640 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc853aa350 0x55cc85760730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f007be00640 1 -- 172.21.15.27:0/1132957810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc853aa350 msgr2=0x55cc85760730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f007be00640 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc853aa350 0x55cc85760730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f007be00640 1 -- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85381490 msgr2=0x55cc85765780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f007be00640 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85381490 0x55cc85765780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f007be00640 1 -- 172.21.15.27:0/1132957810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc857662d0 con 0x55cc85389670 2024-09-21T07:17:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f0080c00640 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85381490 0x55cc85765780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f007be00640 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85389670 0x55cc85765cc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0068006010 tx=0x7f0068000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabe7b8794496c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f0062000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f0062000640 1 -- 172.21.15.27:0/1132957810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068049040 con 0x55cc85389670 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f0062000640 1 -- 172.21.15.27:0/1132957810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0068105680 con 0x55cc85389670 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.535+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f007400fc60 con 0x55cc85389670 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f0062000640 1 -- 172.21.15.27:0/1132957810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00680cd8d0 con 0x55cc85389670 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 10 client.16782 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc857daa20 con 0x55cc85389670 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f0062000640 1 -- 172.21.15.27:0/1132957810 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007400fc60 con 0x55cc85389670 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f0062000640 1 -- 172.21.15.27:0/1132957810 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f006810bc30 con 0x55cc85389670 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 10 client.16782 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 20 client.16782 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc8533e080 0x55cc857dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 4 client.16782 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:17:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.536+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cc856c5890 con 0x55cc8533e080 2024-09-21T07:17:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.537+0000 7f0080c00640 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc8533e080 0x55cc857dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.537+0000 7f0080c00640 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc8533e080 0x55cc857dd160 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b23eb92f9f787e server_cookie=9ca8255b53bb0b2f in_seq=0 out_seq=0 2024-09-21T07:17:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.537+0000 7f0062000640 10 client.16782 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f0062000640 1 -- 172.21.15.27:0/1132957810 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cc856c5890 con 0x55cc8533e080 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f0062000640 10 client.16782 handle_command_reply: tid=0 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 client.16782 shutdown 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85389670 msgr2=0x55cc85765cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85389670 0x55cc85765cc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0068006010 tx=0x7f0068000bc0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 shutdown_connections 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cc8533e080 0x55cc857dd160 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc853aa350 0x55cc85760730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cc85389670 0x55cc85765cc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 --2- 172.21.15.27:0/1132957810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc85381490 0x55cc85765780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 >> 172.21.15.27:0/1132957810 conn(0x55cc8539e740 msgr2=0x55cc8574e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.539+0000 7f0062000640 0 client.16782 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.540+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 shutdown_connections 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.540+0000 7f008a3fc740 1 -- 172.21.15.27:0/1132957810 wait complete. 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.540+0000 7f008a3fc740 20 client.16782 trim_cache size 0 max 16384 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "c896104d-ae00-459e-a046-11dd33f800e9", 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:17:34.543 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:17:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.540+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00640b6680 msgr2=0x7f00640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.540+0000 7f0083600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00640b6680 0x7f00640b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55cc85750990 tx=0x7f0070005f90 comp rx=0 tx=0).stop 2024-09-21T07:17:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.540+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10b6d0 msgr2=0x7f007c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.540+0000 7f0083600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10b6d0 0x7f007c19ccc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0068005ab0 tx=0x7f0068003b90 comp rx=0 tx=0).stop 2024-09-21T07:17:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 shutdown_connections 2024-09-21T07:17:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00640b6680 0x7f00640b8b90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007c10c020 0x7f007c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c10b6d0 0x7f007c19ccc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 --2- 172.21.15.27:0/3350520685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f007c105d90 0x7f007c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 >> 172.21.15.27:0/3350520685 conn(0x7f007c101860 msgr2=0x7f007c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 shutdown_connections 2024-09-21T07:17:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:34.541+0000 7f0083600640 1 -- 172.21.15.27:0/3350520685 wait complete. 2024-09-21T07:17:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:35 smithi027 ceph-mon[30779]: pgmap v514: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-09-21T07:17:35.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:35 smithi153 ceph-mon[34747]: pgmap v514: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-09-21T07:17:35.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:17:35.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:35 smithi119 ceph-mon[34519]: pgmap v514: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-09-21T07:17:35.894 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:17:36.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:36 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:17:36.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:36 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:17:36.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:36 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:17:36.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:36 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:17:36.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:36 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:17:36.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:36 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:17:36.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:36 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:17:36.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:36 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:17:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:36 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:17:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:36 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:17:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:36 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:17:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:36 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:17:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.586+0000 7f7e99400640 1 -- 172.21.15.27:0/2374469392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e94109c40 msgr2=0x7f7e94110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.586+0000 7f7e99400640 1 --2- 172.21.15.27:0/2374469392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e94109c40 0x7f7e94110470 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009ca0 tx=0x7f7e8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.587+0000 7f7e99400640 1 -- 172.21.15.27:0/2374469392 shutdown_connections 2024-09-21T07:17:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.587+0000 7f7e99400640 1 --2- 172.21.15.27:0/2374469392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e941092f0 0x7f7e94109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.587+0000 7f7e99400640 1 --2- 172.21.15.27:0/2374469392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e94109c40 0x7f7e94110470 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.587+0000 7f7e99400640 1 --2- 172.21.15.27:0/2374469392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e941039b0 0x7f7e94103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.587+0000 7f7e99400640 1 -- 172.21.15.27:0/2374469392 >> 172.21.15.27:0/2374469392 conn(0x7f7e940ff400 msgr2=0x7f7e94101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.587+0000 7f7e99400640 1 -- 172.21.15.27:0/2374469392 shutdown_connections 2024-09-21T07:17:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.587+0000 7f7e99400640 1 -- 172.21.15.27:0/2374469392 wait complete. 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 Processor -- start 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 -- start start 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e941039b0 0x7f7e9419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e941092f0 0x7f7e9419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e94109c40 0x7f7e941a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94113950 con 0x7f7e941039b0 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941137d0 con 0x7f7e941092f0 2024-09-21T07:17:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.588+0000 7f7e99400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94113ad0 con 0x7f7e94109c40 2024-09-21T07:17:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e941039b0 0x7f7e9419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e941092f0 0x7f7e9419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e941039b0 0x7f7e9419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46950/0 (socket says 172.21.15.27:46950) 2024-09-21T07:17:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93e00640 1 -- 172.21.15.27:0/2641530799 learned_addr learned my addr 172.21.15.27:0/2641530799 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e98a00640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e94109c40 0x7f7e941a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93400640 1 -- 172.21.15.27:0/2641530799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e94109c40 msgr2=0x7f7e941a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e94109c40 0x7f7e941a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93400640 1 -- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e941039b0 msgr2=0x7f7e9419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e941039b0 0x7f7e9419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93400640 1 -- 172.21.15.27:0/2641530799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e941a23f0 con 0x7f7e941092f0 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.589+0000 7f7e93e00640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e941039b0 0x7f7e9419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.590+0000 7f7e93400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e941092f0 0x7f7e9419ca20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7e9406b930 tx=0x7f7e8000db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.590+0000 7f7e90c00640 1 -- 172.21.15.27:0/2641530799 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80017070 con 0x7f7e941092f0 2024-09-21T07:17:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.590+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e941a26e0 con 0x7f7e941092f0 2024-09-21T07:17:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.590+0000 7f7e90c00640 1 -- 172.21.15.27:0/2641530799 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e80009820 con 0x7f7e941092f0 2024-09-21T07:17:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.590+0000 7f7e90c00640 1 -- 172.21.15.27:0/2641530799 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80004ba0 con 0x7f7e941092f0 2024-09-21T07:17:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.590+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e941a8e40 con 0x7f7e941092f0 2024-09-21T07:17:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.593+0000 7f7ea1662740 1 Processor -- start 2024-09-21T07:17:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.593+0000 7f7ea1662740 1 -- start start 2024-09-21T07:17:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b50dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b13e1b0 0x563b1b50e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b50e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7ea1662740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563b1b484f70 con 0x563b1b167070 2024-09-21T07:17:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7ea1662740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b1b4743a0 con 0x563b1b146390 2024-09-21T07:17:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7ea1662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563b1b4781b0 con 0x563b1b13e1b0 2024-09-21T07:17:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7e98a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b50dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7e98a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b50dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46966/0 (socket says 172.21.15.27:46966) 2024-09-21T07:17:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7e98a00640 1 -- 172.21.15.27:0/1681413488 learned_addr learned my addr 172.21.15.27:0/1681413488 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.594+0000 7f7e93e00640 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b13e1b0 0x563b1b50e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e90c00640 1 -- 172.21.15.27:0/2641530799 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7e80009c60 con 0x7f7e941092f0 2024-09-21T07:17:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e98a00640 1 -- 172.21.15.27:0/1681413488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b13e1b0 msgr2=0x563b1b50e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e98a00640 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b13e1b0 0x563b1b50e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e98a00640 1 -- 172.21.15.27:0/1681413488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 msgr2=0x563b1b50e770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:17:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e98a00640 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b50e770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e98a00640 1 -- 172.21.15.27:0/1681413488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1b43ec90 con 0x563b1b167070 2024-09-21T07:17:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e98a00640 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b50dcf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009ca0 tx=0x7f7e8c02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78eb59af6328f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.595+0000 7f7e89200640 1 -- 172.21.15.27:0/1681413488 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c031db0 con 0x563b1b167070 2024-09-21T07:17:36.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.596+0000 7f7e89200640 1 -- 172.21.15.27:0/1681413488 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c036030 con 0x563b1b167070 2024-09-21T07:17:36.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.596+0000 7f7e89200640 1 -- 172.21.15.27:0/1681413488 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0042d0 con 0x563b1b167070 2024-09-21T07:17:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.596+0000 7f7ea1662740 1 -- 172.21.15.27:0/1681413488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 msgr2=0x563b1b50dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.596+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b50dcf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009ca0 tx=0x7f7e8c02f310 comp rx=0 tx=0).stop 2024-09-21T07:17:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.596+0000 7f7e90c00640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e780b6680 0x7f7e780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 -- 172.21.15.27:0/1681413488 shutdown_connections 2024-09-21T07:17:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b13e1b0 0x563b1b50e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b50e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1681413488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b50dcf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 -- 172.21.15.27:0/1681413488 >> 172.21.15.27:0/1681413488 conn(0x563b1b15b460 msgr2=0x563b1b50bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7e90c00640 1 -- 172.21.15.27:0/2641530799 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7e800e7330 con 0x7f7e941092f0 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7e93e00640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e780b6680 0x7f7e780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 -- 172.21.15.27:0/1681413488 shutdown_connections 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 -- 172.21.15.27:0/1681413488 wait complete. 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.597+0000 7f7ea1662740 1 -- start start 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7e93e00640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e780b6680 0x7f7e780b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563b1b50c970 tx=0x7f7e84005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 1 Processor -- start 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b13e1b0 0x563b1b5198d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b146390 0x563b1b519e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b5255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563b1b2d5700 con 0x563b1b167070 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b1b40f540 con 0x563b1b13e1b0 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563b1b51c2f0 con 0x563b1b146390 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.598+0000 7f7ea1662740 10 client.0 _ll_register_callbacks cb 0x563b1b41f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b13e1b0 0x563b1b5198d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b13e1b0 0x563b1b5198d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:49690/0 (socket says 172.21.15.27:49690) 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 -- 172.21.15.27:0/116130302 learned_addr learned my addr 172.21.15.27:0/116130302 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93400640 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b146390 0x563b1b519e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e98a00640 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b5255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 -- 172.21.15.27:0/116130302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b146390 msgr2=0x563b1b519e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b146390 0x563b1b519e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 -- 172.21.15.27:0/116130302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 msgr2=0x563b1b5255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b5255e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.599+0000 7f7e93e00640 1 -- 172.21.15.27:0/116130302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1b1d2650 con 0x563b1b13e1b0 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7e93e00640 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b13e1b0 0x563b1b5198d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e84003670 tx=0x7f7e84004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19234553d0ac61b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7e7d600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:17:36.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7e7d600640 1 -- 172.21.15.27:0/116130302 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400f8b0 con 0x563b1b13e1b0 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b1b411b50 con 0x563b1b13e1b0 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7e7d600640 1 -- 172.21.15.27:0/116130302 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7e8400fa50 con 0x563b1b13e1b0 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7e7d600640 1 -- 172.21.15.27:0/116130302 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e8400fc50 con 0x563b1b13e1b0 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7ea1662740 10 client.26825 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.600+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b1b42c910 con 0x563b1b13e1b0 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7e7d600640 1 -- 172.21.15.27:0/116130302 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563b1b411b50 con 0x563b1b13e1b0 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7e7d600640 1 -- 172.21.15.27:0/116130302 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e84027070 con 0x563b1b13e1b0 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7ea1662740 10 client.26825 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7ea1662740 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7ea1662740 20 client.26825 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7ea1662740 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b1259f0 0x563b1b52b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7ea1662740 4 client.26825 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:36.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.601+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b1b486ec0 con 0x563b1b1259f0 2024-09-21T07:17:36.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.602+0000 7f7e93400640 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b1259f0 0x563b1b52b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.602+0000 7f7e93400640 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b1259f0 0x563b1b52b6b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb85c3ca280606c server_cookie=50eb697d5b091061 in_seq=0 out_seq=0 2024-09-21T07:17:36.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.602+0000 7f7e7d600640 10 client.26825 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7e7d600640 1 -- 172.21.15.27:0/116130302 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7e80086e50 con 0x563b1b1259f0 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7e7d600640 10 client.26825 handle_command_reply: tid=0 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 client.26825 shutdown 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b13e1b0 msgr2=0x563b1b5198d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b13e1b0 0x563b1b5198d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e84003670 tx=0x7f7e84004730 comp rx=0 tx=0).stop 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 shutdown_connections 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b1259f0 0x563b1b52b6b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b146390 0x563b1b519e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7e7d600640 0 client.26825 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b167070 0x563b1b5255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 --2- 172.21.15.27:0/116130302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b13e1b0 0x563b1b5198d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 >> 172.21.15.27:0/116130302 conn(0x563b1b15b460 msgr2=0x563b1b510ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.603+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 shutdown_connections 2024-09-21T07:17:36.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.604+0000 7f7ea1662740 1 -- 172.21.15.27:0/116130302 wait complete. 2024-09-21T07:17:36.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.604+0000 7f7ea1662740 20 client.26825 trim_cache size 0 max 16384 2024-09-21T07:17:36.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.622+0000 7f7ea1662740 1 Processor -- start 2024-09-21T07:17:36.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.622+0000 7f7ea1662740 1 -- start start 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b513000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b513540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b5254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7ea1662740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b5660 con 0x563b1b13e1b0 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7ea1662740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b2f30 con 0x563b1b146390 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7ea1662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b0d80 con 0x563b1b167070 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b513000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7e93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b513540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b513000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46986/0 (socket says 172.21.15.27:46986) 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7e93e00640 1 -- 172.21.15.27:0/1782062987 learned_addr learned my addr 172.21.15.27:0/1782062987 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.623+0000 7f7e98a00640 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b5254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e93400640 1 -- 172.21.15.27:0/1782062987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 msgr2=0x563b1b5254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e93400640 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b5254e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e93400640 1 -- 172.21.15.27:0/1782062987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 msgr2=0x563b1b513000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e93400640 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b513000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e93400640 1 -- 172.21.15.27:0/1782062987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1b511e00 con 0x563b1b146390 2024-09-21T07:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e93400640 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b513540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7e800e4730 tx=0x7f7e80094040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e493bc882c31d3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1782062987 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800b1050 con 0x563b1b146390 2024-09-21T07:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1782062987 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563b1b486ec0 con 0x563b1b146390 2024-09-21T07:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.624+0000 7f7e7fe00640 1 -- 172.21.15.27:0/1782062987 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80020040 con 0x563b1b146390 2024-09-21T07:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 -- 172.21.15.27:0/1782062987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 msgr2=0x563b1b513540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b513540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7e800e4730 tx=0x7f7e80094040 comp rx=0 tx=0).stop 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 -- 172.21.15.27:0/1782062987 shutdown_connections 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b5254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b513540 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 --2- 172.21.15.27:0/1782062987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b513000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 -- 172.21.15.27:0/1782062987 >> 172.21.15.27:0/1782062987 conn(0x563b1b15b460 msgr2=0x563b1b517f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 -- 172.21.15.27:0/1782062987 shutdown_connections 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.625+0000 7f7ea1662740 1 -- 172.21.15.27:0/1782062987 wait complete. 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.626+0000 7f7ea1662740 1 -- start start 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.626+0000 7f7ea1662740 1 Processor -- start 2024-09-21T07:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b521ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b51f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7ea1662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b51fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7ea1662740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b13f0 con 0x563b1b13e1b0 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7ea1662740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b2860 con 0x563b1b146390 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7ea1662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b1ac0 con 0x563b1b167070 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7ea1662740 10 client.0 _ll_register_callbacks cb 0x7f7e84004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b521ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b51f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b51f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:49708/0 (socket says 172.21.15.27:49708) 2024-09-21T07:17:36.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b521ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46990/0 (socket says 172.21.15.27:46990) 2024-09-21T07:17:36.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93400640 1 -- 172.21.15.27:0/2570886561 learned_addr learned my addr 172.21.15.27:0/2570886561 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e98a00640 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b51fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93e00640 1 -- 172.21.15.27:0/2570886561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 msgr2=0x563b1b51fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93e00640 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b51fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93e00640 1 -- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 msgr2=0x563b1b51f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93e00640 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b51f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93e00640 1 -- 172.21.15.27:0/2570886561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1b51fff0 con 0x563b1b13e1b0 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.627+0000 7f7e93400640 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b51f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7e93e00640 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b521ac0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009620 tx=0x7f7e84003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898246b969dbb6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7e7d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7e7d600640 1 -- 172.21.15.27:0/2570886561 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8403b060 con 0x563b1b13e1b0 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7e7d600640 1 -- 172.21.15.27:0/2570886561 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f7e84028cc0 con 0x563b1b13e1b0 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b1b486ec0 con 0x563b1b13e1b0 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7e7d600640 1 -- 172.21.15.27:0/2570886561 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e84036440 con 0x563b1b13e1b0 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7ea1662740 10 client.16800 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.628+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b1b596d90 con 0x563b1b13e1b0 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7e7d600640 1 -- 172.21.15.27:0/2570886561 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563b1b486ec0 con 0x563b1b13e1b0 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7e7d600640 1 -- 172.21.15.27:0/2570886561 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e84027070 con 0x563b1b13e1b0 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7ea1662740 10 client.16800 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7ea1662740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7ea1662740 20 client.16800 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7ea1662740 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b0fada0 0x563b1b599550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7ea1662740 4 client.16800 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b1b482970 con 0x563b1b0fada0 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.629+0000 7f7e93400640 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b0fada0 0x563b1b599550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.630+0000 7f7e93400640 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b0fada0 0x563b1b599550 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b82cd0ec2bf05be server_cookie=da79b5656f04d06e in_seq=0 out_seq=0 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.630+0000 7f7e7d600640 10 client.16800 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.630+0000 7f7e7d600640 1 -- 172.21.15.27:0/2570886561 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x563b1b482970 con 0x563b1b0fada0 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.630+0000 7f7e7d600640 10 client.16800 handle_command_reply: tid=0 2024-09-21T07:17:36.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.630+0000 7f7ea1662740 1 client.16800 shutdown 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.630+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 msgr2=0x563b1b521ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.630+0000 7f7ea1662740 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b521ac0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009620 tx=0x7f7e84003f00 comp rx=0 tx=0).stop 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 shutdown_connections 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1b0fada0 0x563b1b599550 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b1b167070 0x563b1b51fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b1b146390 0x563b1b51f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 --2- 172.21.15.27:0/2570886561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b1b13e1b0 0x563b1b521ac0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 >> 172.21.15.27:0/2570886561 conn(0x563b1b15b460 msgr2=0x563b1b50a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7e7d600640 0 client.16800 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 shutdown_connections 2024-09-21T07:17:36.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 1 -- 172.21.15.27:0/2570886561 wait complete. 2024-09-21T07:17:36.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.631+0000 7f7ea1662740 20 client.16800 trim_cache size 0 max 16384 2024-09-21T07:17:36.639 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:17:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:17:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:17:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:17:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "c896104d-ae00-459e-a046-11dd33f800e9", 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:17:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:17:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e780b6680 msgr2=0x7f7e780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e780b6680 0x7f7e780b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563b1b50c970 tx=0x7f7e84005f90 comp rx=0 tx=0).stop 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e941092f0 msgr2=0x7f7e9419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e941092f0 0x7f7e9419ca20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7e9406b930 tx=0x7f7e8000db30 comp rx=0 tx=0).stop 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 shutdown_connections 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e94109c40 0x7f7e941a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e780b6680 0x7f7e780b8b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e941092f0 0x7f7e9419ca20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 --2- 172.21.15.27:0/2641530799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e941039b0 0x7f7e9419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 >> 172.21.15.27:0/2641530799 conn(0x7f7e940ff400 msgr2=0x7f7e94106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 shutdown_connections 2024-09-21T07:17:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:17:36.632+0000 7f7e99400640 1 -- 172.21.15.27:0/2641530799 wait complete. 2024-09-21T07:17:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 -- 172.21.15.27:0/630900415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa41810bd90 msgr2=0x7fa4181125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 --2- 172.21.15.27:0/630900415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa41810bd90 0x7fa4181125c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ca0 tx=0x7fa41002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 -- 172.21.15.27:0/630900415 shutdown_connections 2024-09-21T07:17:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 --2- 172.21.15.27:0/630900415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa41810b440 0x7fa41810b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 --2- 172.21.15.27:0/630900415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa41810bd90 0x7fa4181125c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 --2- 172.21.15.27:0/630900415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa418105b10 0x7fa418105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 -- 172.21.15.27:0/630900415 >> 172.21.15.27:0/630900415 conn(0x7fa418101550 msgr2=0x7fa4181039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 -- 172.21.15.27:0/630900415 shutdown_connections 2024-09-21T07:17:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.810+0000 7fa41cc00640 1 -- 172.21.15.27:0/630900415 wait complete. 2024-09-21T07:17:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.811+0000 7fa41cc00640 1 Processor -- start 2024-09-21T07:17:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.811+0000 7fa41cc00640 1 -- start start 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa418105b10 0x7fa41819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa41810b440 0x7fa41819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa41810bd90 0x7fa4181a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa41cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa418115a50 con 0x7fa418105b10 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa41cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181158d0 con 0x7fa41810bd90 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa41cc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa418115bd0 con 0x7fa41810b440 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa417400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa418105b10 0x7fa41819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa417400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa418105b10 0x7fa41819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47002/0 (socket says 172.21.15.27:47002) 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa417400640 1 -- 172.21.15.27:0/176607355 learned_addr learned my addr 172.21.15.27:0/176607355 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa416a00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa41810b440 0x7fa41819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.812+0000 7fa417e00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa41810bd90 0x7fa4181a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa417400640 1 -- 172.21.15.27:0/176607355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa41810b440 msgr2=0x7fa41819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa417400640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa41810b440 0x7fa41819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa417400640 1 -- 172.21.15.27:0/176607355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa41810bd90 msgr2=0x7fa4181a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa417400640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa41810bd90 0x7fa4181a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa417400640 1 -- 172.21.15.27:0/176607355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4181a2420 con 0x7fa418105b10 2024-09-21T07:17:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa416a00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa41810b440 0x7fa41819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:17:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa417400640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa418105b10 0x7fa41819c4f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa40800e9f0 tx=0x7fa40800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa40fe00640 1 -- 172.21.15.27:0/176607355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40800cd80 con 0x7fa418105b10 2024-09-21T07:17:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4181a2710 con 0x7fa418105b10 2024-09-21T07:17:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa40fe00640 1 -- 172.21.15.27:0/176607355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4080108e0 con 0x7fa418105b10 2024-09-21T07:17:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa40fe00640 1 -- 172.21.15.27:0/176607355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408018980 con 0x7fa418105b10 2024-09-21T07:17:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.813+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181a8ea0 con 0x7fa418105b10 2024-09-21T07:17:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 Processor -- start 2024-09-21T07:17:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 -- start start 2024-09-21T07:17:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca53122f0 0x556ca56e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca531a4d0 0x556ca56e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556ca56556b0 con 0x556ca531a4d0 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556ca5644ae0 con 0x556ca533b1b0 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa42525d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556ca56488f0 con 0x556ca53122f0 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa40fe00640 1 -- 172.21.15.27:0/176607355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa408010430 con 0x7fa418105b10 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.817+0000 7fa416a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca531a4d0 0x556ca56e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa416a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca531a4d0 0x556ca56e0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47018/0 (socket says 172.21.15.27:47018) 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa416a00640 1 -- 172.21.15.27:0/411024923 learned_addr learned my addr 172.21.15.27:0/411024923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417400640 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca53122f0 0x556ca56e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417e00640 1 -- 172.21.15.27:0/411024923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca53122f0 msgr2=0x556ca56e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417e00640 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca53122f0 0x556ca56e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417e00640 1 -- 172.21.15.27:0/411024923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca531a4d0 msgr2=0x556ca56e0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417e00640 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca531a4d0 0x556ca56e0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417e00640 1 -- 172.21.15.27:0/411024923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca5611780 con 0x556ca533b1b0 2024-09-21T07:17:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417400640 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca53122f0 0x556ca56e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:17:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.818+0000 7fa417e00640 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56dfe10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa41002ade0 tx=0x7fa41002f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ab213e237c11bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.819+0000 7fa403e00640 1 -- 172.21.15.27:0/411024923 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410031cf0 con 0x556ca533b1b0 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.819+0000 7fa403e00640 1 -- 172.21.15.27:0/411024923 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa410036070 con 0x556ca533b1b0 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.819+0000 7fa403e00640 1 -- 172.21.15.27:0/411024923 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410004040 con 0x556ca533b1b0 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa40fe00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3fc0b6680 0x7fa3fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa417400640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3fc0b6680 0x7fa3fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa40fe00640 1 -- 172.21.15.27:0/176607355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa4080e7eb0 con 0x7fa418105b10 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 -- 172.21.15.27:0/411024923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 msgr2=0x556ca56dfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56dfe10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa41002ade0 tx=0x7fa41002f410 comp rx=0 tx=0).stop 2024-09-21T07:17:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 -- 172.21.15.27:0/411024923 shutdown_connections 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca53122f0 0x556ca56e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca531a4d0 0x556ca56e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 --2- 172.21.15.27:0/411024923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56dfe10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 -- 172.21.15.27:0/411024923 >> 172.21.15.27:0/411024923 conn(0x556ca532f300 msgr2=0x556ca56ddd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 -- 172.21.15.27:0/411024923 shutdown_connections 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.820+0000 7fa42525d740 1 -- 172.21.15.27:0/411024923 wait complete. 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.821+0000 7fa417400640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3fc0b6680 0x7fa3fc0b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556ca56dea90 tx=0x7fa408000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.821+0000 7fa42525d740 1 -- start start 2024-09-21T07:17:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.821+0000 7fa42525d740 1 Processor -- start 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa42525d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa42525d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa42525d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa42525d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556ca557cab0 con 0x556ca53122f0 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa42525d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556ca55eb570 con 0x556ca533b1b0 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa42525d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556ca56ee3b0 con 0x556ca531a4d0 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa42525d740 10 client.0 _ll_register_callbacks cb 0x556ca55fa100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56eb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47022/0 (socket says 172.21.15.27:47022) 2024-09-21T07:17:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 -- 172.21.15.27:0/3330177070 learned_addr learned my addr 172.21.15.27:0/3330177070 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa416a00640 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 -- 172.21.15.27:0/3330177070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 msgr2=0x556ca56eb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56eb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 -- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 msgr2=0x556ca56ebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56ebf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.822+0000 7fa417e00640 1 -- 172.21.15.27:0/3330177070 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca53a6790 con 0x556ca53122f0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa417e00640 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56eb440 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa41002a900 tx=0x7fa410003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42557c5ceb8c946 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa400c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa400c00640 1 -- 172.21.15.27:0/3330177070 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41004e040 con 0x556ca53122f0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa400c00640 1 -- 172.21.15.27:0/3330177070 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa41002f860 con 0x556ca53122f0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa410036070 con 0x556ca53122f0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa400c00640 1 -- 172.21.15.27:0/3330177070 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa410031910 con 0x556ca53122f0 2024-09-21T07:17:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa42525d740 10 client.16812 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ca541f620 con 0x556ca53122f0 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.823+0000 7fa400c00640 1 -- 172.21.15.27:0/3330177070 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa410036070 con 0x556ca53122f0 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa400c00640 1 -- 172.21.15.27:0/3330177070 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa410043020 con 0x556ca53122f0 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa42525d740 10 client.16812 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa42525d740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa42525d740 20 client.16812 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa42525d740 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52f9d80 0x556ca56fd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa42525d740 4 client.16812 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ca5657600 con 0x556ca52f9d80 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.824+0000 7fa416a00640 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52f9d80 0x556ca56fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.825+0000 7fa416a00640 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52f9d80 0x556ca56fd180 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695086af26342fdb server_cookie=fb449e573ee05fab in_seq=0 out_seq=0 2024-09-21T07:17:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.825+0000 7fa400c00640 10 client.16812 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa400c00640 1 -- 172.21.15.27:0/3330177070 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ca5657600 con 0x556ca52f9d80 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa400c00640 10 client.16812 handle_command_reply: tid=0 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 client.16812 shutdown 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 msgr2=0x556ca56eb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56eb440 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa41002a900 tx=0x7fa410003c20 comp rx=0 tx=0).stop 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 shutdown_connections 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52f9d80 0x556ca56fd180 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56eb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 --2- 172.21.15.27:0/3330177070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56eb440 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 >> 172.21.15.27:0/3330177070 conn(0x556ca532f300 msgr2=0x556ca56e0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa400c00640 0 client.16812 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 shutdown_connections 2024-09-21T07:17:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 1 -- 172.21.15.27:0/3330177070 wait complete. 2024-09-21T07:17:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.826+0000 7fa42525d740 20 client.16812 trim_cache size 0 max 16384 2024-09-21T07:17:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.841+0000 7fa42525d740 1 Processor -- start 2024-09-21T07:17:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.841+0000 7fa42525d740 1 -- start start 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa42525d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa42525d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa42525d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa42525d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b5580 con 0x556ca53122f0 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa42525d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b2e50 con 0x556ca533b1b0 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa42525d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b0ca0 con 0x556ca531a4d0 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa417e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa417e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56dcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47034/0 (socket says 172.21.15.27:47034) 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa417e00640 1 -- 172.21.15.27:0/3981813856 learned_addr learned my addr 172.21.15.27:0/3981813856 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa416a00640 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.842+0000 7fa417e00640 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa417e00640 1 -- 172.21.15.27:0/3981813856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 msgr2=0x556ca56ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa417e00640 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa417e00640 1 -- 172.21.15.27:0/3981813856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 msgr2=0x556ca56e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa417e00640 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa417e00640 1 -- 172.21.15.27:0/3981813856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca56eda20 con 0x556ca53122f0 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa417e00640 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56dcd20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa410002230 tx=0x7fa410000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a0b367c4438702 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa403400640 1 -- 172.21.15.27:0/3981813856 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410031910 con 0x556ca53122f0 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa403400640 1 -- 172.21.15.27:0/3981813856 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa410036070 con 0x556ca53122f0 2024-09-21T07:17:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.843+0000 7fa403400640 1 -- 172.21.15.27:0/3981813856 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41002f8c0 con 0x556ca53122f0 2024-09-21T07:17:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 -- 172.21.15.27:0/3981813856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 msgr2=0x556ca56dcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56dcd20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa410002230 tx=0x7fa410000f40 comp rx=0 tx=0).stop 2024-09-21T07:17:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 -- 172.21.15.27:0/3981813856 shutdown_connections 2024-09-21T07:17:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca56ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca56e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 --2- 172.21.15.27:0/3981813856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca56dcd20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 -- 172.21.15.27:0/3981813856 >> 172.21.15.27:0/3981813856 conn(0x556ca532f300 msgr2=0x556ca56ec960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 -- 172.21.15.27:0/3981813856 shutdown_connections 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 -- 172.21.15.27:0/3981813856 wait complete. 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.844+0000 7fa42525d740 1 -- start start 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 1 Processor -- start 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca575b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca575bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca575c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b1310 con 0x556ca53122f0 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b2780 con 0x556ca533b1b0 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b19e0 con 0x556ca531a4d0 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa42525d740 10 client.0 _ll_register_callbacks cb 0x7fa404009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa416a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca575bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa417e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca575b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.845+0000 7fa416a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca575bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52988/0 (socket says 172.21.15.27:52988) 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa416a00640 1 -- 172.21.15.27:0/155299563 learned_addr learned my addr 172.21.15.27:0/155299563 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa417e00640 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca575c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa416a00640 1 -- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 msgr2=0x556ca575c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa416a00640 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca575c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa416a00640 1 -- 172.21.15.27:0/155299563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 msgr2=0x556ca575b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa416a00640 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca575b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa416a00640 1 -- 172.21.15.27:0/155299563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca5766910 con 0x556ca531a4d0 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa417e00640 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca575b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa416a00640 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca575bdd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa404000c00 tx=0x7fa404012970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cedc8536a40febd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.846+0000 7fa400c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.847+0000 7fa400c00640 1 -- 172.21.15.27:0/155299563 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404029020 con 0x556ca531a4d0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.847+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa410036070 con 0x556ca531a4d0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.847+0000 7fa400c00640 1 -- 172.21.15.27:0/155299563 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa40401f9f0 con 0x556ca531a4d0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.847+0000 7fa400c00640 1 -- 172.21.15.27:0/155299563 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa404034050 con 0x556ca531a4d0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.847+0000 7fa42525d740 10 client.26833 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.847+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ca5767c50 con 0x556ca531a4d0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.847+0000 7fa400c00640 1 -- 172.21.15.27:0/155299563 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa410036070 con 0x556ca531a4d0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa400c00640 1 -- 172.21.15.27:0/155299563 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa40401d410 con 0x556ca531a4d0 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa42525d740 10 client.26833 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa42525d740 10 client.26833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa42525d740 20 client.26833 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa42525d740 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52cf1c0 0x556ca576a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa42525d740 4 client.26833 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:17:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ca56530b0 con 0x556ca52cf1c0 2024-09-21T07:17:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.848+0000 7fa417e00640 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52cf1c0 0x556ca576a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.849+0000 7fa400c00640 10 client.26833 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.849+0000 7fa417e00640 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52cf1c0 0x556ca576a440 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1547b776d99d44e1 server_cookie=e9e7befdb3bcafd7 in_seq=0 out_seq=0 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa400c00640 1 -- 172.21.15.27:0/155299563 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ca56530b0 con 0x556ca52cf1c0 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa400c00640 10 client.26833 handle_command_reply: tid=0 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 client.26833 shutdown 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 msgr2=0x556ca575bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca575bdd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa404000c00 tx=0x7fa404012970 comp rx=0 tx=0).stop 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 shutdown_connections 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556ca52cf1c0 0x556ca576a440 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ca531a4d0 0x556ca575bdd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ca533b1b0 0x556ca575c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 --2- 172.21.15.27:0/155299563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556ca53122f0 0x556ca575b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa400c00640 0 client.26833 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 >> 172.21.15.27:0/155299563 conn(0x556ca532f300 msgr2=0x556ca56ed2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 shutdown_connections 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 1 -- 172.21.15.27:0/155299563 wait complete. 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.862+0000 7fa42525d740 20 client.26833 trim_cache size 0 max 16384 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:17:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3fc0b6680 msgr2=0x7fa3fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3fc0b6680 0x7fa3fc0b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556ca56dea90 tx=0x7fa408000bc0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa418105b10 msgr2=0x7fa41819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa418105b10 0x7fa41819c4f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa40800e9f0 tx=0x7fa40800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 shutdown_connections 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa41810b440 0x7fa41819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3fc0b6680 0x7fa3fc0b8b90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa41810bd90 0x7fa4181a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 --2- 172.21.15.27:0/176607355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa418105b10 0x7fa41819c4f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 >> 172.21.15.27:0/176607355 conn(0x7fa418101550 msgr2=0x7fa418103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 shutdown_connections 2024-09-21T07:17:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:36.863+0000 7fa41cc00640 1 -- 172.21.15.27:0/176607355 wait complete. 2024-09-21T07:17:37.329 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:37 smithi153 ceph-mon[34747]: pgmap v515: 129 pgs: 129 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 16 op/s 2024-09-21T07:17:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:37 smithi027 ceph-mon[30779]: pgmap v515: 129 pgs: 129 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 16 op/s 2024-09-21T07:17:37.522 INFO:tasks.cephfs.filesystem:scrub status for tag:c896104d-ae00-459e-a046-11dd33f800e9 - {'path': '/', 'tag': 'c896104d-ae00-459e-a046-11dd33f800e9', 'options': 'recursive,force'} 2024-09-21T07:17:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:37 smithi119 ceph-mon[34519]: pgmap v515: 129 pgs: 129 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 16 op/s 2024-09-21T07:17:39.199 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:39 smithi027 ceph-mon[30779]: pgmap v516: 129 pgs: 129 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:39.199 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:39 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:17:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:39 smithi153 ceph-mon[34747]: pgmap v516: 129 pgs: 129 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:39 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:17:39.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:39 smithi119 ceph-mon[34519]: pgmap v516: 129 pgs: 129 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:39.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:39 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:17:41.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:41 smithi119 ceph-mon[34519]: pgmap v517: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-21T07:17:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:41 smithi027 ceph-mon[30779]: pgmap v517: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-21T07:17:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:41 smithi153 ceph-mon[34747]: pgmap v517: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-21T07:17:42.722 INFO:tasks.workunit.client.0.smithi027.stdout:64/600: open fd = 3 2024-09-21T07:17:42.722 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:65/600: open fd = 3 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:66/600: open fd = 3 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:67/600: open fd = 3 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.723 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:68/600: open fd = 3 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:69/600: open fd = 3 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:70/600: open fd = 3 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:71/600: open fd = 3 2024-09-21T07:17:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:72/600: open fd = 3 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:73/600: open fd = 3 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:74/600: open fd = 3 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:75/600: open fd = 3 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:76/600: open fd = 3 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:77/600: open fd = 3 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:78/600: open fd = 3 2024-09-21T07:17:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:79/600: open fd = 3 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:80/600: open fd = 3 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:81/600: open fd = 3 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.727 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:82/600: open fd = 3 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:83/600: open fd = 3 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:84/600: open fd = 3 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:85/600: open fd = 3 2024-09-21T07:17:42.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:86/600: open fd = 3 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:87/600: open fd = 3 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:88/600: open fd = 3 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.729 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:89/600: open fd = 3 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:90/600: open fd = 3 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:91/600: open fd = 3 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:92/600: open fd = 3 2024-09-21T07:17:42.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:93/600: open fd = 3 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:94/600: open fd = 3 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:95/600: open fd = 3 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.731 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:96/600: open fd = 3 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:97/600: open fd = 3 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:98/600: open fd = 3 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:99/600: open fd = 3 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:100/600: open fd = 3 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:101/600: open fd = 3 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:102/600: open fd = 3 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.733 INFO:tasks.workunit.client.0.smithi027.stdout:103/600: open fd = 3 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:104/600: open fd = 3 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:105/600: open fd = 3 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:106/600: open fd = 3 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:107/600: open fd = 3 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:108/600: open fd = 3 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:109/600: open fd = 3 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:110/600: open fd = 3 2024-09-21T07:17:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:111/600: open fd = 3 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:112/600: open fd = 3 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:113/600: open fd = 3 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:114/600: open fd = 3 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:115/600: open fd = 3 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:116/600: open fd = 3 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:117/600: open fd = 3 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:118/600: open fd = 3 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:119/600: open fd = 3 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:120/600: open fd = 3 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:121/600: open fd = 3 2024-09-21T07:17:42.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:122/600: open fd = 3 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:123/600: open fd = 3 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:124/600: open fd = 3 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.739 INFO:tasks.workunit.client.0.smithi027.stdout:125/600: open fd = 3 2024-09-21T07:17:42.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:42.740 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:17:42.740 INFO:tasks.workunit.client.0.smithi027.stdout:126/600: open fd = 3 2024-09-21T07:17:42.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:17:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:43 smithi027 ceph-mon[30779]: pgmap v518: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-21T07:17:43.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:43 smithi153 ceph-mon[34747]: pgmap v518: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-21T07:17:43.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:43 smithi119 ceph-mon[34519]: pgmap v518: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-21T07:17:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:45 smithi027 ceph-mon[30779]: pgmap v519: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:45 smithi153 ceph-mon[34747]: pgmap v519: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:45.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:45 smithi119 ceph-mon[34519]: pgmap v519: 129 pgs: 129 active+clean; 4.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:17:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:47 smithi027 ceph-mon[30779]: pgmap v520: 129 pgs: 129 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-09-21T07:17:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:47 smithi153 ceph-mon[34747]: pgmap v520: 129 pgs: 129 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-09-21T07:17:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:47 smithi119 ceph-mon[34519]: pgmap v520: 129 pgs: 129 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-09-21T07:17:49.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:49 smithi027 ceph-mon[30779]: pgmap v521: 129 pgs: 129 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-09-21T07:17:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:49 smithi153 ceph-mon[34747]: pgmap v521: 129 pgs: 129 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-09-21T07:17:49.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:49 smithi119 ceph-mon[34519]: pgmap v521: 129 pgs: 129 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-09-21T07:17:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 -- 172.21.15.27:0/4049231599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 msgr2=0x7f36a8109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 --2- 172.21.15.27:0/4049231599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a8109660 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3690009900 tx=0x7f369002a410 comp rx=0 tx=0).stop 2024-09-21T07:17:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 -- 172.21.15.27:0/4049231599 shutdown_connections 2024-09-21T07:17:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 --2- 172.21.15.27:0/4049231599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a8109ba0 0x7f36a81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 --2- 172.21.15.27:0/4049231599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a8109660 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 --2- 172.21.15.27:0/4049231599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a8103920 0x7f36a8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 -- 172.21.15.27:0/4049231599 >> 172.21.15.27:0/4049231599 conn(0x7f36a80ff320 msgr2=0x7f36a8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 -- 172.21.15.27:0/4049231599 shutdown_connections 2024-09-21T07:17:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.067+0000 7f36ad800640 1 -- 172.21.15.27:0/4049231599 wait complete. 2024-09-21T07:17:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.068+0000 7f36ad800640 1 Processor -- start 2024-09-21T07:17:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.068+0000 7f36ad800640 1 -- start start 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36ad800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a8103920 0x7f36a819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36ad800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36ad800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a8109ba0 0x7f36a81a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36ad800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8113860 con 0x7f36a8109250 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36ad800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81136e0 con 0x7f36a8103920 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36ad800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81139e0 con 0x7f36a8109ba0 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f369fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f369fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48992/0 (socket says 172.21.15.27:48992) 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f369fe00640 1 -- 172.21.15.27:0/1440155696 learned_addr learned my addr 172.21.15.27:0/1440155696 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36a7e00640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a8103920 0x7f36a819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.069+0000 7f36ace00640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a8109ba0 0x7f36a81a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f369fe00640 1 -- 172.21.15.27:0/1440155696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a8109ba0 msgr2=0x7f36a81a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f369fe00640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a8109ba0 0x7f36a81a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f369fe00640 1 -- 172.21.15.27:0/1440155696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a8103920 msgr2=0x7f36a819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f369fe00640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a8103920 0x7f36a819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f369fe00640 1 -- 172.21.15.27:0/1440155696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a81a2410 con 0x7f36a8109250 2024-09-21T07:17:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f369fe00640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a819ca80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f369002a930 tx=0x7f3690031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f36a5600640 1 -- 172.21.15.27:0/1440155696 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369003f030 con 0x7f36a8109250 2024-09-21T07:17:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f36a5600640 1 -- 172.21.15.27:0/1440155696 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690004490 con 0x7f36a8109250 2024-09-21T07:17:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f36a5600640 1 -- 172.21.15.27:0/1440155696 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690035070 con 0x7f36a8109250 2024-09-21T07:17:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a81a26a0 con 0x7f36a8109250 2024-09-21T07:17:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.070+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a81a2b40 con 0x7f36a8109250 2024-09-21T07:17:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 Processor -- start 2024-09-21T07:17:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 -- start start 2024-09-21T07:17:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd7de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5577ebcf36b0 con 0x5577eb9d91b0 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5577ebce2ae0 con 0x5577eb9b84d0 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5577ebce68f0 con 0x5577eb9b02f0 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36a5600640 1 -- 172.21.15.27:0/1440155696 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3690042030 con 0x7f36a8109250 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36a7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36ace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd7de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36a7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd7e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55888/0 (socket says 172.21.15.27:55888) 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.074+0000 7f36a7e00640 1 -- 172.21.15.27:0/2040560182 learned_addr learned my addr 172.21.15.27:0/2040560182 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.075+0000 7f369fe00640 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.075+0000 7f36a7e00640 1 -- 172.21.15.27:0/2040560182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 msgr2=0x5577ebd7e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.075+0000 7f36a7e00640 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd7e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.075+0000 7f36a7e00640 1 -- 172.21.15.27:0/2040560182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 msgr2=0x5577ebd7de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.075+0000 7f36a7e00640 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd7de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.075+0000 7f36a7e00640 1 -- 172.21.15.27:0/2040560182 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577ebcaf780 con 0x5577eb9b02f0 2024-09-21T07:17:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.075+0000 7f36a7e00640 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd7e350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3698009d20 tx=0x7f3698011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2684ea07265a016 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.076+0000 7f36a5600640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f368c0b6700 0x7f368c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.076+0000 7f36a5600640 1 -- 172.21.15.27:0/1440155696 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3690107d70 con 0x7f36a8109250 2024-09-21T07:17:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.076+0000 7f36ace00640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f368c0b6700 0x7f368c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.077+0000 7f36ace00640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f368c0b6700 0x7f368c0b8c10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005fd0 tx=0x7f36a0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.079+0000 7f369d600640 1 -- 172.21.15.27:0/2040560182 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698004230 con 0x5577eb9b02f0 2024-09-21T07:17:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.079+0000 7f369d600640 1 -- 172.21.15.27:0/2040560182 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36980043d0 con 0x5577eb9b02f0 2024-09-21T07:17:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.079+0000 7f369d600640 1 -- 172.21.15.27:0/2040560182 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980198c0 con 0x5577eb9b02f0 2024-09-21T07:17:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.080+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2040560182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 msgr2=0x5577ebd7e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.080+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd7e350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3698009d20 tx=0x7f3698011810 comp rx=0 tx=0).stop 2024-09-21T07:17:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.080+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2040560182 shutdown_connections 2024-09-21T07:17:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.080+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd7e350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.080+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.080+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2040560182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd7de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.080+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2040560182 >> 172.21.15.27:0/2040560182 conn(0x5577eb9cd300 msgr2=0x5577ebd7bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.081+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2040560182 shutdown_connections 2024-09-21T07:17:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.081+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2040560182 wait complete. 2024-09-21T07:17:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.081+0000 7f36b5ea8740 1 -- start start 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.081+0000 7f36b5ea8740 1 Processor -- start 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd89420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd89960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd89ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5577ebc1aab0 con 0x5577eb9d91b0 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5577ebc89570 con 0x5577eb9b84d0 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5577ebd8c590 con 0x5577eb9b02f0 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36b5ea8740 10 client.0 _ll_register_callbacks cb 0x5577ebc98100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36ace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd89ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f369fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd89960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36ace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd89ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49006/0 (socket says 172.21.15.27:49006) 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36ace00640 1 -- 172.21.15.27:0/2673260047 learned_addr learned my addr 172.21.15.27:0/2673260047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.082+0000 7f36a7e00640 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd89420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36ace00640 1 -- 172.21.15.27:0/2673260047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 msgr2=0x5577ebd89420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36ace00640 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd89420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36ace00640 1 -- 172.21.15.27:0/2673260047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 msgr2=0x5577ebd89960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36ace00640 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd89960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36ace00640 1 -- 172.21.15.27:0/2673260047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577eba44790 con 0x5577eb9d91b0 2024-09-21T07:17:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36a7e00640 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd89420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36ace00640 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd89ec0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f36a0003fd0 tx=0x7f36a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83075917a2b9193d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f3696000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f3696000640 1 -- 172.21.15.27:0/2673260047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a000f8b0 con 0x5577eb9d91b0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36980043d0 con 0x5577eb9d91b0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.083+0000 7f3696000640 1 -- 172.21.15.27:0/2673260047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f36a000fa50 con 0x5577eb9d91b0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f3696000640 1 -- 172.21.15.27:0/2673260047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36a000fc50 con 0x5577eb9d91b0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 10 client.16830 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5577ebabd620 con 0x5577eb9d91b0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f3696000640 1 -- 172.21.15.27:0/2673260047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36980043d0 con 0x5577eb9d91b0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f3696000640 1 -- 172.21.15.27:0/2673260047 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36a0026070 con 0x5577eb9d91b0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 10 client.16830 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 20 client.16830 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb997d80 0x5577ebd9b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 4 client.16830 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.084+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577ebcf5600 con 0x5577eb997d80 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.085+0000 7f36a7e00640 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb997d80 0x5577ebd9b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.085+0000 7f36a7e00640 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb997d80 0x5577ebd9b6f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3160e48adaf5e721 server_cookie=d5a961f6f01b220d in_seq=0 out_seq=0 2024-09-21T07:17:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.085+0000 7f3696000640 10 client.16830 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.086+0000 7f3696000640 1 -- 172.21.15.27:0/2673260047 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577ebcf5600 con 0x5577eb997d80 2024-09-21T07:17:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.086+0000 7f3696000640 10 client.16830 handle_command_reply: tid=0 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.086+0000 7f36b5ea8740 1 client.16830 shutdown 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 msgr2=0x5577ebd89ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd89ec0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f36a0003fd0 tx=0x7f36a0004ab0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 shutdown_connections 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb997d80 0x5577ebd9b6f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd89ec0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd89960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f3696000640 0 client.16830 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/2673260047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b02f0 0x5577ebd89420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 >> 172.21.15.27:0/2673260047 conn(0x5577eb9cd300 msgr2=0x5577ebd7ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 shutdown_connections 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 1 -- 172.21.15.27:0/2673260047 wait complete. 2024-09-21T07:17:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.087+0000 7f36b5ea8740 20 client.16830 trim_cache size 0 max 16384 2024-09-21T07:17:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.102+0000 7f36b5ea8740 1 Processor -- start 2024-09-21T07:17:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36b5ea8740 1 -- start start 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b02f0 0x5577ebd7ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b84d0 0x5577ebd7f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81b5740 con 0x5577eb9d91b0 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81b3010 con 0x5577eb9b02f0 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81b0e60 con 0x5577eb9b84d0 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36ace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36ace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd865f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49014/0 (socket says 172.21.15.27:49014) 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36ace00640 1 -- 172.21.15.27:0/1697510310 learned_addr learned my addr 172.21.15.27:0/1697510310 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f369fe00640 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b84d0 0x5577ebd7f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.103+0000 7f36a7e00640 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b02f0 0x5577ebd7ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f36ace00640 1 -- 172.21.15.27:0/1697510310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b84d0 msgr2=0x5577ebd7f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f36ace00640 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b84d0 0x5577ebd7f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f36ace00640 1 -- 172.21.15.27:0/1697510310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b02f0 msgr2=0x5577ebd7ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f36ace00640 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b02f0 0x5577ebd7ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f36ace00640 1 -- 172.21.15.27:0/1697510310 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577ebd87900 con 0x5577eb9d91b0 2024-09-21T07:17:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f36a7e00640 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b02f0 0x5577ebd7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:17:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f36ace00640 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd865f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f36a0011e90 tx=0x7f36a0003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f13ae2f2dd7764 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f369cc00640 1 -- 172.21.15.27:0/1697510310 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a000f8b0 con 0x5577eb9d91b0 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.104+0000 7f369cc00640 1 -- 172.21.15.27:0/1697510310 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a000fa50 con 0x5577eb9d91b0 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f369cc00640 1 -- 172.21.15.27:0/1697510310 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a001d860 con 0x5577eb9d91b0 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f36b5ea8740 1 -- 172.21.15.27:0/1697510310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 msgr2=0x5577ebd865f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd865f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f36a0011e90 tx=0x7f36a0003df0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f36b5ea8740 1 -- 172.21.15.27:0/1697510310 shutdown_connections 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9b84d0 0x5577ebd7f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9d91b0 0x5577ebd865f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/1697510310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b02f0 0x5577ebd7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.105+0000 7f36b5ea8740 1 -- 172.21.15.27:0/1697510310 >> 172.21.15.27:0/1697510310 conn(0x5577eb9cd300 msgr2=0x5577ebd7ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.106+0000 7f36b5ea8740 1 -- 172.21.15.27:0/1697510310 shutdown_connections 2024-09-21T07:17:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.106+0000 7f36b5ea8740 1 -- 172.21.15.27:0/1697510310 wait complete. 2024-09-21T07:17:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.106+0000 7f36b5ea8740 1 -- start start 2024-09-21T07:17:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.106+0000 7f36b5ea8740 1 Processor -- start 2024-09-21T07:17:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9b02f0 0x5577ebd8f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd8f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36b5ea8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9d91b0 0x5577ebdfe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81b14d0 con 0x5577eb9b02f0 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81b2940 con 0x5577eb9b84d0 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36b5ea8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81b1ba0 con 0x5577eb9d91b0 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36b5ea8740 10 client.0 _ll_register_callbacks cb 0x7f36a0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9b02f0 0x5577ebd8f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36a7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9b02f0 0x5577ebd8f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49020/0 (socket says 172.21.15.27:49020) 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36a7e00640 1 -- 172.21.15.27:0/966635583 learned_addr learned my addr 172.21.15.27:0/966635583 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f36ace00640 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9d91b0 0x5577ebdfe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.107+0000 7f369fe00640 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f36a7e00640 1 -- 172.21.15.27:0/966635583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9d91b0 msgr2=0x5577ebdfe1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f36a7e00640 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9d91b0 0x5577ebdfe1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f36a7e00640 1 -- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 msgr2=0x5577ebd8f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:17:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f36a7e00640 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd8f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f36a7e00640 1 -- 172.21.15.27:0/966635583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577ebdfe810 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f36a7e00640 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9b02f0 0x5577ebd8f1d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3698009250 tx=0x7f3698019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c06e5ed01ad74c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f3696000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f3696000640 1 -- 172.21.15.27:0/966635583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698003a00 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a000fa50 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.108+0000 7f3696000640 1 -- 172.21.15.27:0/966635583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3698005040 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f3696000640 1 -- 172.21.15.27:0/966635583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f369803b050 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 10 client.16836 fetch_fsmap learned FSMap version 27 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5577ebe05b80 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f3696000640 1 -- 172.21.15.27:0/966635583 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a000fa50 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f3696000640 1 -- 172.21.15.27:0/966635583 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3698019830 con 0x5577eb9b02f0 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 10 client.16836 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:17:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 20 client.16836 populate_metadata read hostname 'smithi027' 2024-09-21T07:17:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb96d1c0 0x5577ebe08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:17:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 4 client.16836 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:17:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.109+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577ebcf10b0 con 0x5577eb96d1c0 2024-09-21T07:17:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.110+0000 7f369fe00640 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb96d1c0 0x5577ebe08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:17:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.110+0000 7f3696000640 10 client.16836 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.110+0000 7f369fe00640 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb96d1c0 0x5577ebe08120 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de96e0e48cb5a602 server_cookie=16ff00c9a31daffc in_seq=0 out_seq=0 2024-09-21T07:17:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.125+0000 7f3696000640 1 -- 172.21.15.27:0/966635583 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577ebcf10b0 con 0x5577eb96d1c0 2024-09-21T07:17:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.125+0000 7f3696000640 10 client.16836 handle_command_reply: tid=0 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.125+0000 7f36b5ea8740 1 client.16836 shutdown 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9b02f0 msgr2=0x5577ebd8f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9b02f0 0x5577ebd8f1d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3698009250 tx=0x7f3698019420 comp rx=0 tx=0).stop 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 shutdown_connections 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5577eb96d1c0 0x5577ebe08120 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577eb9d91b0 0x5577ebdfe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577eb9b84d0 0x5577ebd8f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f3696000640 0 client.16836 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 --2- 172.21.15.27:0/966635583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5577eb9b02f0 0x5577ebd8f1d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 >> 172.21.15.27:0/966635583 conn(0x5577eb9cd300 msgr2=0x5577ebd7f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 shutdown_connections 2024-09-21T07:17:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 1 -- 172.21.15.27:0/966635583 wait complete. 2024-09-21T07:17:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.126+0000 7f36b5ea8740 20 client.16836 trim_cache size 0 max 16384 2024-09-21T07:17:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:17:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:17:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:17:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:17:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f368c0b6700 msgr2=0x7f368c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f368c0b6700 0x7f368c0b8c10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005fd0 tx=0x7f36a0009500 comp rx=0 tx=0).stop 2024-09-21T07:17:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 msgr2=0x7f36a819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:17:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a819ca80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f369002a930 tx=0x7f3690031a20 comp rx=0 tx=0).stop 2024-09-21T07:17:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 shutdown_connections 2024-09-21T07:17:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36a8109ba0 0x7f36a81a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f368c0b6700 0x7f368c0b8c10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36a8109250 0x7f36a819ca80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 --2- 172.21.15.27:0/1440155696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a8103920 0x7f36a819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:17:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 >> 172.21.15.27:0/1440155696 conn(0x7f36a80ff320 msgr2=0x7f36a8106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:17:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 shutdown_connections 2024-09-21T07:17:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:17:50.127+0000 7f36ad800640 1 -- 172.21.15.27:0/1440155696 wait complete. 2024-09-21T07:17:51.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:51 smithi027 ceph-mon[30779]: pgmap v522: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-21T07:17:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:51 smithi153 ceph-mon[34747]: pgmap v522: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-21T07:17:51.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:51 smithi119 ceph-mon[34519]: pgmap v522: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-21T07:17:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:53 smithi027 ceph-mon[30779]: pgmap v523: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-21T07:17:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:53 smithi153 ceph-mon[34747]: pgmap v523: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-21T07:17:53.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:53 smithi119 ceph-mon[34519]: pgmap v523: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-21T07:17:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:55 smithi027 ceph-mon[30779]: pgmap v524: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-09-21T07:17:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:55 smithi153 ceph-mon[34747]: pgmap v524: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-09-21T07:17:55.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:55 smithi119 ceph-mon[34519]: pgmap v524: 129 pgs: 129 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-09-21T07:17:57.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:57 smithi027 ceph-mon[30779]: pgmap v525: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:17:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:57 smithi153 ceph-mon[34747]: pgmap v525: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:17:57.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:57 smithi119 ceph-mon[34519]: pgmap v525: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:17:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:17:59 smithi153 ceph-mon[34747]: pgmap v526: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-21T07:17:59.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:17:59 smithi119 ceph-mon[34519]: pgmap v526: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-21T07:17:59.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:17:59 smithi027 ceph-mon[30779]: pgmap v526: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-21T07:18:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:01 smithi153 ceph-mon[34747]: pgmap v527: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-21T07:18:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:01 smithi119 ceph-mon[34519]: pgmap v527: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-21T07:18:01.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:01 smithi027 ceph-mon[30779]: pgmap v527: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-21T07:18:02.168 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T07:18:02.169 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:18:02.583 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:03.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.235+0000 7f828d400640 1 -- 172.21.15.27:0/2451455610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288105d90 msgr2=0x7f82881061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.235+0000 7f828d400640 1 --2- 172.21.15.27:0/2451455610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288105d90 0x7f82881061a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8278009900 tx=0x7f827802a410 comp rx=0 tx=0).stop 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.236+0000 7f828d400640 1 -- 172.21.15.27:0/2451455610 shutdown_connections 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.236+0000 7f828d400640 1 --2- 172.21.15.27:0/2451455610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828810c020 0x7f8288112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.236+0000 7f828d400640 1 --2- 172.21.15.27:0/2451455610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f828810b6d0 0x7f828810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.236+0000 7f828d400640 1 --2- 172.21.15.27:0/2451455610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8288105d90 0x7f82881061a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.236+0000 7f828d400640 1 -- 172.21.15.27:0/2451455610 >> 172.21.15.27:0/2451455610 conn(0x7f8288101860 msgr2=0x7f8288103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.236+0000 7f828d400640 1 -- 172.21.15.27:0/2451455610 shutdown_connections 2024-09-21T07:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.236+0000 7f828d400640 1 -- 172.21.15.27:0/2451455610 wait complete. 2024-09-21T07:18:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.238+0000 7f828d400640 1 Processor -- start 2024-09-21T07:18:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.238+0000 7f828d400640 1 -- start start 2024-09-21T07:18:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.238+0000 7f828d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8288105d90 0x7f828807b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.238+0000 7f828d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f828810b6d0 0x7f8288077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.238+0000 7f828d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828810c020 0x7f8288078040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f828d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8288115ce0 con 0x7f828810b6d0 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f828d400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8288115b60 con 0x7f8288105d90 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f828d400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8288115e60 con 0x7f828810c020 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f8286800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8288105d90 0x7f828807b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f8286800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8288105d90 0x7f828807b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34080/0 (socket says 172.21.15.27:34080) 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f8286800640 1 -- 172.21.15.27:0/3188623451 learned_addr learned my addr 172.21.15.27:0/3188623451 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f8285e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f828810b6d0 0x7f8288077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.239+0000 7f8287200640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828810c020 0x7f8288078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f8285e00640 1 -- 172.21.15.27:0/3188623451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828810c020 msgr2=0x7f8288078040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f8285e00640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828810c020 0x7f8288078040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f8285e00640 1 -- 172.21.15.27:0/3188623451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8288105d90 msgr2=0x7f828807b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f8285e00640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8288105d90 0x7f828807b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f8285e00640 1 -- 172.21.15.27:0/3188623451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82881acc00 con 0x7f828810b6d0 2024-09-21T07:18:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f8286800640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8288105d90 0x7f828807b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f8285e00640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f828810b6d0 0x7f8288077920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f828806ca80 tx=0x7f827400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.240+0000 7f827f400640 1 -- 172.21.15.27:0/3188623451 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827400cc40 con 0x7f828810b6d0 2024-09-21T07:18:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.241+0000 7f827f400640 1 -- 172.21.15.27:0/3188623451 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827400cde0 con 0x7f828810b6d0 2024-09-21T07:18:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.241+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82881ace30 con 0x7f828810b6d0 2024-09-21T07:18:03.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.241+0000 7f827f400640 1 -- 172.21.15.27:0/3188623451 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274018780 con 0x7f828810b6d0 2024-09-21T07:18:03.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.242+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82881ad230 con 0x7f828810b6d0 2024-09-21T07:18:03.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.243+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82881061a0 con 0x7f828810b6d0 2024-09-21T07:18:03.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.245+0000 7f827f400640 1 -- 172.21.15.27:0/3188623451 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8274010a90 con 0x7f828810b6d0 2024-09-21T07:18:03.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.248+0000 7f827f400640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f826c0b6680 0x7f826c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.248+0000 7f8286800640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f826c0b6680 0x7f826c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.248+0000 7f827f400640 1 -- 172.21.15.27:0/3188623451 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f82740e7e20 con 0x7f828810b6d0 2024-09-21T07:18:03.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.249+0000 7f8286800640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f826c0b6680 0x7f826c0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8278009230 tx=0x7f82780058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.256+0000 7f827f400640 1 -- 172.21.15.27:0/3188623451 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82740adfc0 con 0x7f828810b6d0 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 -- 172.21.15.27:0/1213631801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109240 msgr2=0x7fa824109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 --2- 172.21.15.27:0/1213631801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109240 0x7fa824109650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa810009900 tx=0x7fa81002a410 comp rx=0 tx=0).stop 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 -- 172.21.15.27:0/1213631801 shutdown_connections 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 --2- 172.21.15.27:0/1213631801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824109b90 0x7fa8241103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 --2- 172.21.15.27:0/1213631801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109240 0x7fa824109650 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 --2- 172.21.15.27:0/1213631801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa824103910 0x7fa824103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 -- 172.21.15.27:0/1213631801 >> 172.21.15.27:0/1213631801 conn(0x7fa8240ff330 msgr2=0x7fa824101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 -- 172.21.15.27:0/1213631801 shutdown_connections 2024-09-21T07:18:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.333+0000 7fa82a200640 1 -- 172.21.15.27:0/1213631801 wait complete. 2024-09-21T07:18:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.334+0000 7fa82a200640 1 Processor -- start 2024-09-21T07:18:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.334+0000 7fa82a200640 1 -- start start 2024-09-21T07:18:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.334+0000 7fa82a200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824103910 0x7fa82419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa82a200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa824109240 0x7fa82419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa82a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109b90 0x7fa8241a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa82a200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241138a0 con 0x7fa824103910 2024-09-21T07:18:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa82a200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa824113720 con 0x7fa824109b90 2024-09-21T07:18:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa82a200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa824113a20 con 0x7fa824109240 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824103910 0x7fa82419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824103910 0x7fa82419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55382/0 (socket says 172.21.15.27:55382) 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 -- 172.21.15.27:0/997914678 learned_addr learned my addr 172.21.15.27:0/997914678 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa829800640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109b90 0x7fa8241a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa823e00640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa824109240 0x7fa82419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 -- 172.21.15.27:0/997914678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa824109240 msgr2=0x7fa82419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa824109240 0x7fa82419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 -- 172.21.15.27:0/997914678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109b90 msgr2=0x7fa8241a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109b90 0x7fa8241a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.335+0000 7fa828e00640 1 -- 172.21.15.27:0/997914678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8241a2450 con 0x7fa824103910 2024-09-21T07:18:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.336+0000 7fa828e00640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824103910 0x7fa82419c520 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa81800e9f0 tx=0x7fa81800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.336+0000 7fa821600640 1 -- 172.21.15.27:0/997914678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81800cd80 con 0x7fa824103910 2024-09-21T07:18:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.336+0000 7fa821600640 1 -- 172.21.15.27:0/997914678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8180054d0 con 0x7fa824103910 2024-09-21T07:18:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.336+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8241a2740 con 0x7fa824103910 2024-09-21T07:18:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.336+0000 7fa821600640 1 -- 172.21.15.27:0/997914678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818010690 con 0x7fa824103910 2024-09-21T07:18:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.336+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a8ed0 con 0x7fa824103910 2024-09-21T07:18:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.339+0000 7fa832a33740 1 Processor -- start 2024-09-21T07:18:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.339+0000 7fa832a33740 1 -- start start 2024-09-21T07:18:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa832a33740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37ea1b0 0x5603f3b8ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa832a33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 0x5603f3b8f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa832a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3b8f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa832a33740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5603f3b046b0 con 0x5603f37c12f0 2024-09-21T07:18:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa832a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5603f3af3ae0 con 0x5603f37c94d0 2024-09-21T07:18:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa832a33740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5603f3af78f0 con 0x5603f37ea1b0 2024-09-21T07:18:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa821600640 1 -- 172.21.15.27:0/997914678 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa818004e90 con 0x7fa824103910 2024-09-21T07:18:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa823e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3b8f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa823e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3b8f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34104/0 (socket says 172.21.15.27:34104) 2024-09-21T07:18:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa823e00640 1 -- 172.21.15.27:0/3742815504 learned_addr learned my addr 172.21.15.27:0/3742815504 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:03.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:03 smithi027 ceph-mon[30779]: pgmap v528: 129 pgs: 129 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-09-21T07:18:03.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:18:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.340+0000 7fa829800640 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37ea1b0 0x5603f3b8ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.341+0000 7fa823e00640 1 -- 172.21.15.27:0/3742815504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37ea1b0 msgr2=0x5603f3b8ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.341+0000 7fa823e00640 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37ea1b0 0x5603f3b8ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.341+0000 7fa823e00640 1 -- 172.21.15.27:0/3742815504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 msgr2=0x5603f3b8f350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.341+0000 7fa823e00640 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 0x5603f3b8f350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.341+0000 7fa823e00640 1 -- 172.21.15.27:0/3742815504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f3ac0780 con 0x5603f37c94d0 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.341+0000 7fa823e00640 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3b8f890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa810031440 tx=0x7fa810004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a185226bd11ba871 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.341+0000 7fa816000640 1 -- 172.21.15.27:0/3742815504 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810040070 con 0x5603f37c94d0 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.342+0000 7fa816000640 1 -- 172.21.15.27:0/3742815504 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810005500 con 0x5603f37c94d0 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.342+0000 7fa816000640 1 -- 172.21.15.27:0/3742815504 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810035040 con 0x5603f37c94d0 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.342+0000 7fa821600640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa80c0b6680 0x7fa80c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa821600640 1 -- 172.21.15.27:0/997914678 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa8180e7ea0 con 0x7fa824103910 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 -- 172.21.15.27:0/3742815504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 msgr2=0x5603f3b8f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3b8f890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa810031440 tx=0x7fa810004040 comp rx=0 tx=0).stop 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa829800640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa80c0b6680 0x7fa80c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 -- 172.21.15.27:0/3742815504 shutdown_connections 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3b8f890 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 0x5603f3b8f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 --2- 172.21.15.27:0/3742815504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37ea1b0 0x5603f3b8ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 -- 172.21.15.27:0/3742815504 >> 172.21.15.27:0/3742815504 conn(0x5603f37de300 msgr2=0x5603f3b8cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 -- 172.21.15.27:0/3742815504 shutdown_connections 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.343+0000 7fa832a33740 1 -- 172.21.15.27:0/3742815504 wait complete. 2024-09-21T07:18:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.344+0000 7fa829800640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa80c0b6680 0x7fa80c0b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005fd0 tx=0x7fa81c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.344+0000 7fa832a33740 1 -- start start 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.344+0000 7fa832a33740 1 Processor -- start 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa832a33740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3b9a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa832a33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c94d0 0x5603f3b9a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa832a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3ba6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa832a33740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5603f3a2bab0 con 0x5603f37c94d0 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa832a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5603f3a9a570 con 0x5603f37ea1b0 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa832a33740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5603f3b9d530 con 0x5603f37c12f0 2024-09-21T07:18:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa832a33740 10 client.0 _ll_register_callbacks cb 0x5603f3aa9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa828e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3b9a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa828e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3b9a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55988/0 (socket says 172.21.15.27:55988) 2024-09-21T07:18:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa828e00640 1 -- 172.21.15.27:0/1035958662 learned_addr learned my addr 172.21.15.27:0/1035958662 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa823e00640 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c94d0 0x5603f3b9a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.345+0000 7fa829800640 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3ba6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa828e00640 1 -- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 msgr2=0x5603f3ba6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa828e00640 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3ba6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa828e00640 1 -- 172.21.15.27:0/1035958662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c94d0 msgr2=0x5603f3b9a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa828e00640 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c94d0 0x5603f3b9a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa828e00640 1 -- 172.21.15.27:0/1035958662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f3855790 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa828e00640 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3b9a4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa8180e52a0 tx=0x7fa818005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966580536a043873 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa80aa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa80aa00640 1 -- 172.21.15.27:0/1035958662 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180ad0b0 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa80aa00640 1 -- 172.21.15.27:0/1035958662 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa8180acda0 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.346+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa810005500 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa80aa00640 1 -- 172.21.15.27:0/1035958662 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa818005180 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 10 client.26869 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603f38ce620 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa80aa00640 1 -- 172.21.15.27:0/1035958662 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810005500 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa80aa00640 1 -- 172.21.15.27:0/1035958662 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa8180ac610 con 0x5603f37c12f0 2024-09-21T07:18:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 10 client.26869 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 10 client.26869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 20 client.26869 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f37a8d80 0x5603f3bac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 4 client.26869 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.347+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f3b06600 con 0x5603f37a8d80 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.348+0000 7fa823e00640 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f37a8d80 0x5603f3bac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.348+0000 7fa80aa00640 10 client.26869 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.348+0000 7fa823e00640 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f37a8d80 0x5603f3bac5a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b20e0910aa6238 server_cookie=51f93393376e7c2c in_seq=0 out_seq=0 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.349+0000 7fa80aa00640 1 -- 172.21.15.27:0/1035958662 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603f3b06600 con 0x5603f37a8d80 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.349+0000 7fa80aa00640 10 client.26869 handle_command_reply: tid=0 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.349+0000 7fa832a33740 1 client.26869 shutdown 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.349+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 msgr2=0x5603f3b9a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.349+0000 7fa832a33740 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3b9a4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa8180e52a0 tx=0x7fa818005750 comp rx=0 tx=0).stop 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 shutdown_connections 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f37a8d80 0x5603f3bac5a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3ba6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c94d0 0x5603f3b9a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa80aa00640 0 client.26869 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 --2- 172.21.15.27:0/1035958662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3b9a4b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 >> 172.21.15.27:0/1035958662 conn(0x5603f37de300 msgr2=0x5603f3b8fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 shutdown_connections 2024-09-21T07:18:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 1 -- 172.21.15.27:0/1035958662 wait complete. 2024-09-21T07:18:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.350+0000 7fa832a33740 20 client.26869 trim_cache size 0 max 16384 2024-09-21T07:18:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.369+0000 7fa832a33740 1 Processor -- start 2024-09-21T07:18:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.369+0000 7fa832a33740 1 -- start start 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa832a33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 0x5603f3b9a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa832a33740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c94d0 0x5603f3b8bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa832a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3b9c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa832a33740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b56d0 con 0x5603f37c12f0 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa832a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b2fa0 con 0x5603f37ea1b0 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa832a33740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b0df0 con 0x5603f37c94d0 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa823e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c94d0 0x5603f3b8bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa829800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3b9c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa828e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 0x5603f3b9a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa823e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c94d0 0x5603f3b8bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56002/0 (socket says 172.21.15.27:56002) 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa823e00640 1 -- 172.21.15.27:0/828058647 learned_addr learned my addr 172.21.15.27:0/828058647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.370+0000 7fa829800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3b9c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34126/0 (socket says 172.21.15.27:34126) 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa823e00640 1 -- 172.21.15.27:0/828058647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 msgr2=0x5603f3b9c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa823e00640 1 --2- 172.21.15.27:0/828058647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3b9c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa823e00640 1 -- 172.21.15.27:0/828058647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 msgr2=0x5603f3b9a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa823e00640 1 --2- 172.21.15.27:0/828058647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 0x5603f3b9a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa823e00640 1 -- 172.21.15.27:0/828058647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f3b9ca90 con 0x5603f37c94d0 2024-09-21T07:18:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa823e00640 1 --2- 172.21.15.27:0/828058647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c94d0 0x5603f3b8bd20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa81002aa00 tx=0x7fa81002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869d4fc9e94fc404 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa815600640 1 -- 172.21.15.27:0/828058647 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810040040 con 0x5603f37c94d0 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa815600640 1 -- 172.21.15.27:0/828058647 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810005500 con 0x5603f37c94d0 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.371+0000 7fa815600640 1 -- 172.21.15.27:0/828058647 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810035040 con 0x5603f37c94d0 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 -- 172.21.15.27:0/828058647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c94d0 msgr2=0x5603f3b8bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 --2- 172.21.15.27:0/828058647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c94d0 0x5603f3b8bd20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa81002aa00 tx=0x7fa81002a410 comp rx=0 tx=0).stop 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 -- 172.21.15.27:0/828058647 shutdown_connections 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 --2- 172.21.15.27:0/828058647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c94d0 0x5603f3b8bd20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 --2- 172.21.15.27:0/828058647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37ea1b0 0x5603f3b9c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 --2- 172.21.15.27:0/828058647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37c12f0 0x5603f3b9a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 -- 172.21.15.27:0/828058647 >> 172.21.15.27:0/828058647 conn(0x5603f37de300 msgr2=0x5603f3b9b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 -- 172.21.15.27:0/828058647 shutdown_connections 2024-09-21T07:18:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 -- 172.21.15.27:0/828058647 wait complete. 2024-09-21T07:18:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.373+0000 7fa832a33740 1 -- start start 2024-09-21T07:18:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 1 Processor -- start 2024-09-21T07:18:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3ba2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3ba0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37ea1b0 0x5603f3ba06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b1460 con 0x5603f37ea1b0 2024-09-21T07:18:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b28d0 con 0x5603f37c94d0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b1b30 con 0x5603f37c12f0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa832a33740 10 client.0 _ll_register_callbacks cb 0x7fa818009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa823e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3ba0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa828e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3ba2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa823e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3ba0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34134/0 (socket says 172.21.15.27:34134) 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.374+0000 7fa823e00640 1 -- 172.21.15.27:0/540323302 learned_addr learned my addr 172.21.15.27:0/540323302 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa829800640 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37ea1b0 0x5603f3ba06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa823e00640 1 -- 172.21.15.27:0/540323302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 msgr2=0x5603f3ba2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa823e00640 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3ba2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa823e00640 1 -- 172.21.15.27:0/540323302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37ea1b0 msgr2=0x5603f3ba06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa823e00640 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37ea1b0 0x5603f3ba06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa823e00640 1 -- 172.21.15.27:0/540323302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f3ba0c10 con 0x5603f37c94d0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa828e00640 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3ba2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa823e00640 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3ba0190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa81002a930 tx=0x7fa8100042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1688f5151900efd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.375+0000 7fa80aa00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa80aa00640 1 -- 172.21.15.27:0/540323302 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810035070 con 0x5603f37c94d0 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa80aa00640 1 -- 172.21.15.27:0/540323302 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fa810005050 con 0x5603f37c94d0 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa810005500 con 0x5603f37c94d0 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa80aa00640 1 -- 172.21.15.27:0/540323302 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa810066050 con 0x5603f37c94d0 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa832a33740 10 client.26888 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603f3c16bb0 con 0x5603f37c94d0 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa80aa00640 1 -- 172.21.15.27:0/540323302 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810005500 con 0x5603f37c94d0 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.376+0000 7fa80aa00640 1 -- 172.21.15.27:0/540323302 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa810061030 con 0x5603f37c94d0 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa832a33740 10 client.26888 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa832a33740 10 client.26888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa832a33740 20 client.26888 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa832a33740 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f377e1c0 0x5603f3c18f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa832a33740 4 client.26888 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:18:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603f3b020b0 con 0x5603f377e1c0 2024-09-21T07:18:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa828e00640 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f377e1c0 0x5603f3c18f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa828e00640 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f377e1c0 0x5603f3c18f00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389f43855d979278 server_cookie=a8585cf1611d2c2e in_seq=0 out_seq=0 2024-09-21T07:18:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.377+0000 7fa80aa00640 10 client.26888 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.389+0000 7fa80aa00640 1 -- 172.21.15.27:0/540323302 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603f3b020b0 con 0x5603f377e1c0 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.389+0000 7fa80aa00640 10 client.26888 handle_command_reply: tid=0 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.389+0000 7fa832a33740 1 client.26888 shutdown 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 msgr2=0x5603f3ba0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3ba0190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa81002a930 tx=0x7fa8100042f0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 shutdown_connections 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5603f377e1c0 0x5603f3c18f00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f37ea1b0 0x5603f3ba06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603f37c94d0 0x5603f3ba0190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 --2- 172.21.15.27:0/540323302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603f37c12f0 0x5603f3ba2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa80aa00640 0 client.26888 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 >> 172.21.15.27:0/540323302 conn(0x5603f37de300 msgr2=0x5603f3b9c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 shutdown_connections 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 1 -- 172.21.15.27:0/540323302 wait complete. 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.390+0000 7fa832a33740 20 client.26888 trim_cache size 0 max 16384 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:18:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:18:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa80c0b6680 msgr2=0x7fa80c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa80c0b6680 0x7fa80c0b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005fd0 tx=0x7fa81c009500 comp rx=0 tx=0).stop 2024-09-21T07:18:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824103910 msgr2=0x7fa82419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824103910 0x7fa82419c520 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa81800e9f0 tx=0x7fa81800edf0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 shutdown_connections 2024-09-21T07:18:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa824109240 0x7fa82419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa80c0b6680 0x7fa80c0b8b90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa824109b90 0x7fa8241a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 --2- 172.21.15.27:0/997914678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824103910 0x7fa82419c520 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 >> 172.21.15.27:0/997914678 conn(0x7fa8240ff330 msgr2=0x7fa8241068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 shutdown_connections 2024-09-21T07:18:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:03.391+0000 7fa82a200640 1 -- 172.21.15.27:0/997914678 wait complete. 2024-09-21T07:18:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:03 smithi153 ceph-mon[34747]: pgmap v528: 129 pgs: 129 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-09-21T07:18:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:18:03.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.512+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8288078c20 con 0x7f828810b6d0 2024-09-21T07:18:03.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.513+0000 7f827f400640 1 -- 172.21.15.27:0/3188623451 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f82740b23a0 con 0x7f828810b6d0 2024-09-21T07:18:03.515 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:18:03.515 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:18:03.515 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:18:03.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f826c0b6680 msgr2=0x7f826c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f826c0b6680 0x7f826c0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8278009230 tx=0x7f82780058d0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f828810b6d0 msgr2=0x7f8288077920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:03.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f828810b6d0 0x7f8288077920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f828806ca80 tx=0x7f827400ee50 comp rx=0 tx=0).stop 2024-09-21T07:18:03.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 shutdown_connections 2024-09-21T07:18:03.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f828810c020 0x7f8288078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f826c0b6680 0x7f826c0b8b90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f828810b6d0 0x7f8288077920 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 --2- 172.21.15.27:0/3188623451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8288105d90 0x7f828807b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:03.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 >> 172.21.15.27:0/3188623451 conn(0x7f8288101860 msgr2=0x7f8288103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:03.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 shutdown_connections 2024-09-21T07:18:03.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:03.516+0000 7f828d400640 1 -- 172.21.15.27:0/3188623451 wait complete. 2024-09-21T07:18:03.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:03 smithi119 ceph-mon[34519]: pgmap v528: 129 pgs: 129 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-09-21T07:18:03.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:18:04.360 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:18:04.360 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:18:04.360 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:18:04.360 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:04 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3188623451' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:04.398 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T07:18:04.399 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T07:18:04.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:18:04.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:18:04.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:18:04.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:04 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3188623451' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:04.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:18:04.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:18:04.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:18:04.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:04 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3188623451' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:04.785 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:05.221 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:05 smithi027 ceph-mon[30779]: pgmap v529: 129 pgs: 129 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-09-21T07:18:05.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 -- 172.21.15.27:0/3431421406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 msgr2=0x7f88501061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 --2- 172.21.15.27:0/3431421406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f88501061d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8844009900 tx=0x7f884402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 -- 172.21.15.27:0/3431421406 shutdown_connections 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 --2- 172.21.15.27:0/3431421406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f885010c050 0x7f8850112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 --2- 172.21.15.27:0/3431421406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f885010b700 0x7f885010bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 --2- 172.21.15.27:0/3431421406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f88501061d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 -- 172.21.15.27:0/3431421406 >> 172.21.15.27:0/3431421406 conn(0x7f88501017d0 msgr2=0x7f8850103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 -- 172.21.15.27:0/3431421406 shutdown_connections 2024-09-21T07:18:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.415+0000 7f8857400640 1 -- 172.21.15.27:0/3431421406 wait complete. 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.416+0000 7f8857400640 1 Processor -- start 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8857400640 1 -- start start 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8857400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f885019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8857400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f885010b700 0x7f885019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8857400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f885010c050 0x7f88501a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8857400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8850115d10 con 0x7f8850105dc0 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8857400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8850115b90 con 0x7f885010c050 2024-09-21T07:18:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8857400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8850115e90 con 0x7f885010b700 2024-09-21T07:18:05.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8854a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f885019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8854a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f885019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55426/0 (socket says 172.21.15.27:55426) 2024-09-21T07:18:05.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.417+0000 7f8854a00640 1 -- 172.21.15.27:0/1671443792 learned_addr learned my addr 172.21.15.27:0/1671443792 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f884fe00640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f885010b700 0x7f885019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f8855400640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f885010c050 0x7f88501a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f8854a00640 1 -- 172.21.15.27:0/1671443792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f885010b700 msgr2=0x7f885019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f8854a00640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f885010b700 0x7f885019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f8854a00640 1 -- 172.21.15.27:0/1671443792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f885010c050 msgr2=0x7f88501a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f8854a00640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f885010c050 0x7f88501a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f8854a00640 1 -- 172.21.15.27:0/1671443792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88501a2630 con 0x7f8850105dc0 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.418+0000 7f8854a00640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f885019c6c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8844005940 tx=0x7f88440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:05.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.419+0000 7f884d600640 1 -- 172.21.15.27:0/1671443792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884403f030 con 0x7f8850105dc0 2024-09-21T07:18:05.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.419+0000 7f884d600640 1 -- 172.21.15.27:0/1671443792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8844003cb0 con 0x7f8850105dc0 2024-09-21T07:18:05.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.419+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88501a28c0 con 0x7f8850105dc0 2024-09-21T07:18:05.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.419+0000 7f884d600640 1 -- 172.21.15.27:0/1671443792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8844035070 con 0x7f8850105dc0 2024-09-21T07:18:05.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.419+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88501a9080 con 0x7f8850105dc0 2024-09-21T07:18:05.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.423+0000 7f885e091740 1 Processor -- start 2024-09-21T07:18:05.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.424+0000 7f885e091740 1 -- start start 2024-09-21T07:18:05.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.424+0000 7f885e091740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0f4840 0x56110f4a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.424+0000 7f885e091740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.424+0000 7f885e091740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0d3b60 0x56110f4a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.424+0000 7f885e091740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56110f411ca0 con 0x56110f0cb980 2024-09-21T07:18:05.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.424+0000 7f885e091740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56110f4010d0 con 0x56110f0d3b60 2024-09-21T07:18:05.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.424+0000 7f885e091740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56110f404ee0 con 0x56110f0f4840 2024-09-21T07:18:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:05 smithi153 ceph-mon[34747]: pgmap v529: 129 pgs: 129 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-09-21T07:18:05.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.425+0000 7f8855400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0f4840 0x56110f4a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.425+0000 7f8855400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0f4840 0x56110f4a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56038/0 (socket says 172.21.15.27:56038) 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.425+0000 7f884d600640 1 -- 172.21.15.27:0/1671443792 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8844042030 con 0x7f8850105dc0 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.425+0000 7f8855400640 1 -- 172.21.15.27:0/2262774022 learned_addr learned my addr 172.21.15.27:0/2262774022 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.425+0000 7f8854a00640 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f884fe00640 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0d3b60 0x56110f4a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f8855400640 1 -- 172.21.15.27:0/2262774022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0d3b60 msgr2=0x56110f4a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f8855400640 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0d3b60 0x56110f4a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f8855400640 1 -- 172.21.15.27:0/2262774022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 msgr2=0x56110f4a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f8855400640 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f8855400640 1 -- 172.21.15.27:0/2262774022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110f3b3400 con 0x56110f0f4840 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f884fe00640 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0d3b60 0x56110f4a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f8854a00640 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.426+0000 7f8855400640 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0f4840 0x56110f4a4b60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8848010370 tx=0x7f8848010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2968b8a98ec4cb45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:05.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.427+0000 7f8841c00640 1 -- 172.21.15.27:0/2262774022 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848016070 con 0x56110f0f4840 2024-09-21T07:18:05.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.427+0000 7f8841c00640 1 -- 172.21.15.27:0/2262774022 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8848004d20 con 0x56110f0f4840 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.427+0000 7f8841c00640 1 -- 172.21.15.27:0/2262774022 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848005350 con 0x56110f0f4840 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.427+0000 7f884d600640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88380b6680 0x7f88380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f884d600640 1 -- 172.21.15.27:0/1671443792 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f88441082d0 con 0x7f8850105dc0 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f8854a00640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88380b6680 0x7f88380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 -- 172.21.15.27:0/2262774022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0f4840 msgr2=0x56110f4a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0f4840 0x56110f4a4b60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8848010370 tx=0x7f8848010830 comp rx=0 tx=0).stop 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 -- 172.21.15.27:0/2262774022 shutdown_connections 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0d3b60 0x56110f4a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 --2- 172.21.15.27:0/2262774022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0f4840 0x56110f4a4b60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 -- 172.21.15.27:0/2262774022 >> 172.21.15.27:0/2262774022 conn(0x56110f0e8990 msgr2=0x56110f4a2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.428+0000 7f885e091740 1 -- 172.21.15.27:0/2262774022 shutdown_connections 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.429+0000 7f885e091740 1 -- 172.21.15.27:0/2262774022 wait complete. 2024-09-21T07:18:05.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.429+0000 7f8854a00640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88380b6680 0x7f88380b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x56110f4a37e0 tx=0x7f88440045a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:05.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.429+0000 7f885e091740 1 -- start start 2024-09-21T07:18:05.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.429+0000 7f885e091740 1 Processor -- start 2024-09-21T07:18:05.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f885e091740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0cb980 0x56110f4b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f885e091740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f885e091740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0f4840 0x56110f4b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f885e091740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56110f2627a0 con 0x56110f0f4840 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f885e091740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56110f3a6a10 con 0x56110f0cb980 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f885e091740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56110f4b3340 con 0x56110f0d3b60 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f885e091740 10 client.0 _ll_register_callbacks cb 0x56110f3b6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f884fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f8855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0cb980 0x56110f4b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f884fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56040/0 (socket says 172.21.15.27:56040) 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.430+0000 7f884fe00640 1 -- 172.21.15.27:0/1845960532 learned_addr learned my addr 172.21.15.27:0/1845960532 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f8855400640 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0f4840 0x56110f4b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f884fe00640 1 -- 172.21.15.27:0/1845960532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0cb980 msgr2=0x56110f4b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f884fe00640 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0cb980 0x56110f4b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f884fe00640 1 -- 172.21.15.27:0/1845960532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0f4840 msgr2=0x56110f4b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f884fe00640 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0f4840 0x56110f4b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f884fe00640 1 -- 172.21.15.27:0/1845960532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110f16eb90 con 0x56110f0d3b60 2024-09-21T07:18:05.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f8855400640 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0f4840 0x56110f4b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:05.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f884fe00640 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b0740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f883c00b870 tx=0x7f883c00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=962735d4f7e3f422 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:05.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.431+0000 7f8836000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:05.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.432+0000 7f8836000640 1 -- 172.21.15.27:0/1845960532 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c01b640 con 0x56110f0d3b60 2024-09-21T07:18:05.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.432+0000 7f8836000640 1 -- 172.21.15.27:0/1845960532 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f883c01b7e0 con 0x56110f0d3b60 2024-09-21T07:18:05.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.432+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8848004d20 con 0x56110f0d3b60 2024-09-21T07:18:05.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.432+0000 7f8836000640 1 -- 172.21.15.27:0/1845960532 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f883c01b9e0 con 0x56110f0d3b60 2024-09-21T07:18:05.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.432+0000 7f885e091740 10 client.26887 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:05.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.432+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56110f4bf070 con 0x56110f0d3b60 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f8836000640 1 -- 172.21.15.27:0/1845960532 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8848004d20 con 0x56110f0d3b60 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f8836000640 1 -- 172.21.15.27:0/1845960532 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f883c019020 con 0x56110f0d3b60 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f885e091740 10 client.26887 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f885e091740 10 client.26887 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f885e091740 20 client.26887 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f885e091740 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0b31c0 0x56110f4c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f885e091740 4 client.26887 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.433+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56110f413bf0 con 0x56110f0b31c0 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.434+0000 7f8855400640 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0b31c0 0x56110f4c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.435+0000 7f8855400640 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0b31c0 0x56110f4c25e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2196449d9cd688 server_cookie=940e157701107ccf in_seq=0 out_seq=0 2024-09-21T07:18:05.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.435+0000 7f8836000640 10 client.26887 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:05.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.436+0000 7f8836000640 1 -- 172.21.15.27:0/1845960532 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56110f413bf0 con 0x56110f0b31c0 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.436+0000 7f8836000640 10 client.26887 handle_command_reply: tid=0 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.436+0000 7f885e091740 1 client.26887 shutdown 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.436+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 msgr2=0x56110f4b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.436+0000 7f885e091740 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b0740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f883c00b870 tx=0x7f883c00bd30 comp rx=0 tx=0).stop 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 shutdown_connections 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0b31c0 0x56110f4c25e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b0740 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f8836000640 0 client.26887 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0f4840 0x56110f4b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 --2- 172.21.15.27:0/1845960532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0cb980 0x56110f4b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 >> 172.21.15.27:0/1845960532 conn(0x56110f0e8990 msgr2=0x56110f4989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:05.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 shutdown_connections 2024-09-21T07:18:05.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 1 -- 172.21.15.27:0/1845960532 wait complete. 2024-09-21T07:18:05.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.437+0000 7f885e091740 20 client.26887 trim_cache size 0 max 16384 2024-09-21T07:18:05.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.473+0000 7f885e091740 1 Processor -- start 2024-09-21T07:18:05.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.473+0000 7f885e091740 1 -- start start 2024-09-21T07:18:05.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f885e091740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f885e091740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0cb980 0x56110f4ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f885e091740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0d3b60 0x56110f4c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f885e091740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f88501b14a0 con 0x56110f0d3b60 2024-09-21T07:18:05.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f885e091740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88501aed70 con 0x56110f0b31c0 2024-09-21T07:18:05.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f885e091740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88501acbc0 con 0x56110f0cb980 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f884fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0cb980 0x56110f4ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f8855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f884fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0cb980 0x56110f4ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56046/0 (socket says 172.21.15.27:56046) 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f8855400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0d3b60 0x56110f4c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.474+0000 7f884fe00640 1 -- 172.21.15.27:0/64347844 learned_addr learned my addr 172.21.15.27:0/64347844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.475+0000 7f884fe00640 1 -- 172.21.15.27:0/64347844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 msgr2=0x56110f4ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.475+0000 7f884fe00640 1 --2- 172.21.15.27:0/64347844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.475+0000 7f884fe00640 1 -- 172.21.15.27:0/64347844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0d3b60 msgr2=0x56110f4c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.475+0000 7f884fe00640 1 --2- 172.21.15.27:0/64347844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0d3b60 0x56110f4c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.475+0000 7f884fe00640 1 -- 172.21.15.27:0/64347844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110f495c50 con 0x56110f0cb980 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.475+0000 7f884fe00640 1 --2- 172.21.15.27:0/64347844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0cb980 0x56110f4ad930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f883c011b50 tx=0x7f883c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4700c611ebc9106 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.476+0000 7f8841200640 1 -- 172.21.15.27:0/64347844 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c032070 con 0x56110f0cb980 2024-09-21T07:18:05.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.476+0000 7f8841200640 1 -- 172.21.15.27:0/64347844 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883c0039c0 con 0x56110f0cb980 2024-09-21T07:18:05.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.476+0000 7f8841200640 1 -- 172.21.15.27:0/64347844 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c011900 con 0x56110f0cb980 2024-09-21T07:18:05.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 -- 172.21.15.27:0/64347844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0cb980 msgr2=0x56110f4ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 --2- 172.21.15.27:0/64347844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0cb980 0x56110f4ad930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f883c011b50 tx=0x7f883c004360 comp rx=0 tx=0).stop 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 -- 172.21.15.27:0/64347844 shutdown_connections 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 --2- 172.21.15.27:0/64347844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0cb980 0x56110f4ad930 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 --2- 172.21.15.27:0/64347844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0d3b60 0x56110f4c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 --2- 172.21.15.27:0/64347844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 -- 172.21.15.27:0/64347844 >> 172.21.15.27:0/64347844 conn(0x56110f0e8990 msgr2=0x56110f4a5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 -- 172.21.15.27:0/64347844 shutdown_connections 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 -- 172.21.15.27:0/64347844 wait complete. 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.477+0000 7f885e091740 1 -- start start 2024-09-21T07:18:05.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 1 Processor -- start 2024-09-21T07:18:05.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ad230 con 0x56110f0cb980 2024-09-21T07:18:05.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ae6a0 con 0x56110f0b31c0 2024-09-21T07:18:05.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ad900 con 0x56110f0d3b60 2024-09-21T07:18:05.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.478+0000 7f885e091740 10 client.0 _ll_register_callbacks cb 0x7f883c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f8855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55452/0 (socket says 172.21.15.27:55452) 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f8855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4b8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34204/0 (socket says 172.21.15.27:34204) 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 -- 172.21.15.27:0/3325468610 learned_addr learned my addr 172.21.15.27:0/3325468610 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f8855400640 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 -- 172.21.15.27:0/3325468610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 msgr2=0x56110f4b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 -- 172.21.15.27:0/3325468610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 msgr2=0x56110f4b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.479+0000 7f884fe00640 1 -- 172.21.15.27:0/3325468610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110f4b5f40 con 0x56110f0cb980 2024-09-21T07:18:05.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.480+0000 7f884fe00640 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4b5270 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f883c030040 tx=0x7f883c0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7834ab51b2e63f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.480+0000 7f8836000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.480+0000 7f8836000640 1 -- 172.21.15.27:0/3325468610 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c032050 con 0x56110f0cb980 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.480+0000 7f8836000640 1 -- 172.21.15.27:0/3325468610 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f883c01b640 con 0x56110f0cb980 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.480+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f883c0039c0 con 0x56110f0cb980 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.480+0000 7f8836000640 1 -- 172.21.15.27:0/3325468610 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f883c046050 con 0x56110f0cb980 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.480+0000 7f885e091740 10 client.16875 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56110f52dee0 con 0x56110f0cb980 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f8836000640 1 -- 172.21.15.27:0/3325468610 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883c0039c0 con 0x56110f0cb980 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f8836000640 1 -- 172.21.15.27:0/3325468610 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f883c041030 con 0x56110f0cb980 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f885e091740 10 client.16875 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f885e091740 10 client.16875 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f885e091740 20 client.16875 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f885e091740 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0f4840 0x56110f530670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:05.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f885e091740 4 client.16875 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T07:18:05.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.481+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x56110f40f6a0 con 0x56110f0f4840 2024-09-21T07:18:05.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.482+0000 7f8855400640 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0f4840 0x56110f530670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:05.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.482+0000 7f8855400640 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0f4840 0x56110f530670 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb84251a135f180 server_cookie=30cf07028c6f0396 in_seq=0 out_seq=0 2024-09-21T07:18:05.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.482+0000 7f8836000640 10 client.16875 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.494+0000 7f8836000640 1 -- 172.21.15.27:0/3325468610 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56110f40f6a0 con 0x56110f0f4840 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.494+0000 7f8836000640 10 client.16875 handle_command_reply: tid=0 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.494+0000 7f885e091740 1 client.16875 shutdown 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 msgr2=0x56110f4b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4b5270 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f883c030040 tx=0x7f883c0043d0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 shutdown_connections 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56110f0f4840 0x56110f530670 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56110f0d3b60 0x56110f4b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56110f0cb980 0x56110f4b5270 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 --2- 172.21.15.27:0/3325468610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56110f0b31c0 0x56110f4b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f8836000640 0 client.16875 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 >> 172.21.15.27:0/3325468610 conn(0x56110f0e8990 msgr2=0x56110f4a5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 shutdown_connections 2024-09-21T07:18:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 1 -- 172.21.15.27:0/3325468610 wait complete. 2024-09-21T07:18:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.495+0000 7f885e091740 20 client.16875 trim_cache size 0 max 16384 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 18, 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "fa29fba": { 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 18, 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:18:05.499 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 130.0, 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:18:05.500 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.496+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88380b6680 msgr2=0x7f88380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.496+0000 7f8857400640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88380b6680 0x7f88380b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x56110f4a37e0 tx=0x7f88440045a0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.496+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 msgr2=0x7f885019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.496+0000 7f8857400640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f885019c6c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8844005940 tx=0x7f88440042f0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 shutdown_connections 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f885010b700 0x7f885019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f88380b6680 0x7f88380b8b90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f885010c050 0x7f88501a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 --2- 172.21.15.27:0/1671443792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8850105dc0 0x7f885019c6c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 >> 172.21.15.27:0/1671443792 conn(0x7f88501017d0 msgr2=0x7f8850104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 shutdown_connections 2024-09-21T07:18:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:05.497+0000 7f8857400640 1 -- 172.21.15.27:0/1671443792 wait complete. 2024-09-21T07:18:05.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:05 smithi119 ceph-mon[34519]: pgmap v529: 129 pgs: 129 active+clean; 5.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-09-21T07:18:06.253 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fa29fba', quiesce took 4.1 seconds. Will release after: 5.9 2024-09-21T07:18:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:07 smithi153 ceph-mon[34747]: pgmap v530: 129 pgs: 129 active+clean; 1.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-09-21T07:18:07.523 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:18:07.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:07 smithi119 ceph-mon[34519]: pgmap v530: 129 pgs: 129 active+clean; 1.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-09-21T07:18:07.578 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:07 smithi027 ceph-mon[30779]: pgmap v530: 129 pgs: 129 active+clean; 1.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-09-21T07:18:07.918 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:08.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 -- 172.21.15.27:0/25671726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c4105bb0 msgr2=0x7ff6c4105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 --2- 172.21.15.27:0/25671726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c4105bb0 0x7ff6c4105fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009900 tx=0x7ff6b802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 -- 172.21.15.27:0/25671726 shutdown_connections 2024-09-21T07:18:08.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 --2- 172.21.15.27:0/25671726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c410be40 0x7ff6c4112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 --2- 172.21.15.27:0/25671726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c410b4f0 0x7ff6c410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 --2- 172.21.15.27:0/25671726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c4105bb0 0x7ff6c4105fc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 -- 172.21.15.27:0/25671726 >> 172.21.15.27:0/25671726 conn(0x7ff6c4101680 msgr2=0x7ff6c4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:08.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 -- 172.21.15.27:0/25671726 shutdown_connections 2024-09-21T07:18:08.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.509+0000 7ff6ca200640 1 -- 172.21.15.27:0/25671726 wait complete. 2024-09-21T07:18:08.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.511+0000 7ff6ca200640 1 Processor -- start 2024-09-21T07:18:08.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.511+0000 7ff6ca200640 1 -- start start 2024-09-21T07:18:08.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.511+0000 7ff6ca200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c4105bb0 0x7ff6c419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.511+0000 7ff6ca200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c410b4f0 0x7ff6c419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.511+0000 7ff6ca200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c410be40 0x7ff6c41a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6ca200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4115b00 con 0x7ff6c4105bb0 2024-09-21T07:18:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6ca200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4115980 con 0x7ff6c410be40 2024-09-21T07:18:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6ca200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4115c80 con 0x7ff6c410b4f0 2024-09-21T07:18:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c4105bb0 0x7ff6c419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c4105bb0 0x7ff6c419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44840/0 (socket says 172.21.15.27:44840) 2024-09-21T07:18:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c8e00640 1 -- 172.21.15.27:0/610576788 learned_addr learned my addr 172.21.15.27:0/610576788 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c9800640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c410be40 0x7ff6c41a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c410b4f0 0x7ff6c419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c9800640 1 -- 172.21.15.27:0/610576788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c410b4f0 msgr2=0x7ff6c419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c9800640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c410b4f0 0x7ff6c419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c9800640 1 -- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c4105bb0 msgr2=0x7ff6c419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c9800640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c4105bb0 0x7ff6c419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c9800640 1 -- 172.21.15.27:0/610576788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c41a2460 con 0x7ff6c410be40 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.512+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c4105bb0 0x7ff6c419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.513+0000 7ff6c9800640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c410be40 0x7ff6c41a1d30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009fd0 tx=0x7ff6bc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.513+0000 7ff6c1600640 1 -- 172.21.15.27:0/610576788 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00ce20 con 0x7ff6c410be40 2024-09-21T07:18:08.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.513+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c41a2750 con 0x7ff6c410be40 2024-09-21T07:18:08.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.513+0000 7ff6c1600640 1 -- 172.21.15.27:0/610576788 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6bc005020 con 0x7ff6c410be40 2024-09-21T07:18:08.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.513+0000 7ff6c1600640 1 -- 172.21.15.27:0/610576788 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc010690 con 0x7ff6c410be40 2024-09-21T07:18:08.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.513+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a8eb0 con 0x7ff6c410be40 2024-09-21T07:18:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.516+0000 7ff6d23f5740 1 Processor -- start 2024-09-21T07:18:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.517+0000 7ff6c1600640 1 -- 172.21.15.27:0/610576788 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff6bc0054d0 con 0x7ff6c410be40 2024-09-21T07:18:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.517+0000 7ff6d23f5740 1 -- start start 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be7bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be7c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55979bdf2f70 con 0x55979bad5070 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55979bde23a0 con 0x55979baac1b0 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55979bde61b0 con 0x55979bab4390 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be7bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be7bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44850/0 (socket says 172.21.15.27:44850) 2024-09-21T07:18:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3968941120 learned_addr learned my addr 172.21.15.27:0/3968941120 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be7c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3968941120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 msgr2=0x55979be7c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be7c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3968941120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 msgr2=0x55979be7c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.518+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be7c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3968941120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979bdacc90 con 0x55979bad5070 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be7bf30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0013530 tx=0x7ff6b0009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284806f68062d35f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6c1600640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6ac0b6680 0x7ff6ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6b6000640 1 -- 172.21.15.27:0/3968941120 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0004050 con 0x55979bad5070 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6c1600640 1 -- 172.21.15.27:0/610576788 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff6bc0e81b0 con 0x7ff6c410be40 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6b6000640 1 -- 172.21.15.27:0/3968941120 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b00041f0 con 0x55979bad5070 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6b6000640 1 -- 172.21.15.27:0/3968941120 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000f760 con 0x55979bad5070 2024-09-21T07:18:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.519+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6ac0b6680 0x7ff6ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3968941120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 msgr2=0x55979be7bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6ac0b6680 0x7ff6ac0b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8031440 tx=0x7ff6b80058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be7bf30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0013530 tx=0x7ff6b0009370 comp rx=0 tx=0).stop 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3968941120 shutdown_connections 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be7c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be7c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3968941120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be7bf30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.520+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3968941120 >> 172.21.15.27:0/3968941120 conn(0x55979bac9460 msgr2=0x55979be79e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.521+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3968941120 shutdown_connections 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.521+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3968941120 wait complete. 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.521+0000 7ff6d23f5740 1 -- start start 2024-09-21T07:18:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.521+0000 7ff6d23f5740 1 Processor -- start 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be876e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be87c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55979bc43700 con 0x55979bad5070 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55979bd7d540 con 0x55979baac1b0 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55979be8a710 con 0x55979bab4390 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6d23f5740 10 client.0 _ll_register_callbacks cb 0x55979bd8d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6c8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be876e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6c8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be876e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34360/0 (socket says 172.21.15.27:34360) 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1419553007 learned_addr learned my addr 172.21.15.27:0/1419553007 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6c9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.522+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be87c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1419553007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 msgr2=0x55979be87c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be87c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1419553007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 msgr2=0x55979be93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be93a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1419553007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979bb40650 con 0x55979baac1b0 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6c9800640 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be876e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8000c00 tx=0x7ff6b8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b7edbebbd6f14f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.523+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1419553007 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b804d070 con 0x55979baac1b0 2024-09-21T07:18:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.524+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1419553007 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff6b8003cc0 con 0x55979baac1b0 2024-09-21T07:18:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.524+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55979bd7fb50 con 0x55979baac1b0 2024-09-21T07:18:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.524+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1419553007 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6b805a050 con 0x55979baac1b0 2024-09-21T07:18:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.524+0000 7ff6d23f5740 10 client.26918 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.524+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55979bd9a910 con 0x55979baac1b0 2024-09-21T07:18:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.524+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1419553007 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55979bd7fb50 con 0x55979baac1b0 2024-09-21T07:18:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.524+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1419553007 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6b8049420 con 0x55979baac1b0 2024-09-21T07:18:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.525+0000 7ff6d23f5740 10 client.26918 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.525+0000 7ff6d23f5740 10 client.26918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.525+0000 7ff6d23f5740 20 client.26918 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.525+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba939f0 0x55979be99850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.525+0000 7ff6d23f5740 4 client.26918 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.525+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55979bdf4ec0 con 0x55979ba939f0 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.525+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba939f0 0x55979be99850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.526+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba939f0 0x55979be99850 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d31e17796f1b71 server_cookie=66d18137edb0b3e6 in_seq=0 out_seq=0 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.526+0000 7ff6aaa00640 10 client.26918 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.526+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1419553007 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55979bdf4ec0 con 0x55979ba939f0 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.526+0000 7ff6aaa00640 10 client.26918 handle_command_reply: tid=0 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.526+0000 7ff6d23f5740 1 client.26918 shutdown 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 msgr2=0x55979be876e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be876e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8000c00 tx=0x7ff6b8009260 comp rx=0 tx=0).stop 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 shutdown_connections 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba939f0 0x55979be99850 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be87c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1419553007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be876e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6aaa00640 0 client.26918 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 >> 172.21.15.27:0/1419553007 conn(0x55979bac9460 msgr2=0x55979be7f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 shutdown_connections 2024-09-21T07:18:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1419553007 wait complete. 2024-09-21T07:18:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.527+0000 7ff6d23f5740 20 client.26918 trim_cache size 0 max 16384 2024-09-21T07:18:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 Processor -- start 2024-09-21T07:18:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 -- start start 2024-09-21T07:18:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979baac1b0 0x55979be82f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979bab4390 0x55979be848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bad5070 0x55979be84e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b55f0 con 0x55979baac1b0 2024-09-21T07:18:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b2ec0 con 0x55979bab4390 2024-09-21T07:18:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.546+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b0d10 con 0x55979bad5070 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979bab4390 0x55979be848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979bab4390 0x55979be848d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34372/0 (socket says 172.21.15.27:34372) 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3684980180 learned_addr learned my addr 172.21.15.27:0/3684980180 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979baac1b0 0x55979be82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c9800640 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bad5070 0x55979be84e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3684980180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bad5070 msgr2=0x55979be84e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bad5070 0x55979be84e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3684980180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979baac1b0 msgr2=0x55979be82f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979baac1b0 0x55979be82f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 -- 172.21.15.27:0/3684980180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979be7e220 con 0x55979bab4390 2024-09-21T07:18:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979baac1b0 0x55979be82f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.547+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979bab4390 0x55979be848d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0003df0 tx=0x7ff6b0009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127eb036accb27e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6b5600640 1 -- 172.21.15.27:0/3684980180 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00050c0 con 0x55979bab4390 2024-09-21T07:18:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6b5600640 1 -- 172.21.15.27:0/3684980180 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b00035f0 con 0x55979bab4390 2024-09-21T07:18:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6b5600640 1 -- 172.21.15.27:0/3684980180 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000f790 con 0x55979bab4390 2024-09-21T07:18:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3684980180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979bab4390 msgr2=0x55979be848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979bab4390 0x55979be848d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0003df0 tx=0x7ff6b0009f20 comp rx=0 tx=0).stop 2024-09-21T07:18:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3684980180 shutdown_connections 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bad5070 0x55979be84e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979bab4390 0x55979be848d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/3684980180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979baac1b0 0x55979be82f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.548+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3684980180 >> 172.21.15.27:0/3684980180 conn(0x55979bac9460 msgr2=0x55979be85f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.549+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3684980180 shutdown_connections 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.549+0000 7ff6d23f5740 1 -- 172.21.15.27:0/3684980180 wait complete. 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.549+0000 7ff6d23f5740 1 -- start start 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.549+0000 7ff6d23f5740 1 Processor -- start 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be8feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6d23f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be8a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b1380 con 0x55979bad5070 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b27f0 con 0x55979baac1b0 2024-09-21T07:18:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6d23f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b1a50 con 0x55979bab4390 2024-09-21T07:18:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6d23f5740 10 client.0 _ll_register_callbacks cb 0x7ff6b8031820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6c8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6c9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be8a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6c8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be8f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34376/0 (socket says 172.21.15.27:34376) 2024-09-21T07:18:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1758566539 learned_addr learned my addr 172.21.15.27:0/1758566539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.550+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be8feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1758566539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 msgr2=0x55979be8feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be8feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1758566539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 msgr2=0x55979be8a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be8a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6c8e00640 1 -- 172.21.15.27:0/1758566539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979be905f0 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6c8e00640 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be8f970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009230 tx=0x7ff6b80496d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d724aa7df7332f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.551+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1758566539 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8062070 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1758566539 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff6b8003bc0 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b00035f0 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1758566539 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6b806e050 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6d23f5740 10 client.26924 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55979bf04720 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1758566539 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b00035f0 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1758566539 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6b8004800 con 0x55979baac1b0 2024-09-21T07:18:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.552+0000 7ff6d23f5740 10 client.26924 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6d23f5740 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6d23f5740 20 client.26924 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba68da0 0x55979bf06e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6d23f5740 4 client.26924 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55979bdf0970 con 0x55979ba68da0 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba68da0 0x55979bf06e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6c3e00640 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba68da0 0x55979bf06e80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5f350d3fad0846 server_cookie=f6ee2a4dc081f3f9 in_seq=0 out_seq=0 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.553+0000 7ff6aaa00640 10 client.26924 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6aaa00640 1 -- 172.21.15.27:0/1758566539 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x55979bdf0970 con 0x55979ba68da0 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6aaa00640 10 client.26924 handle_command_reply: tid=0 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 client.26924 shutdown 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 msgr2=0x55979be8f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be8f970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009230 tx=0x7ff6b80496d0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 shutdown_connections 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55979ba68da0 0x55979bf06e80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55979bab4390 0x55979be8feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55979bad5070 0x55979be8a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6aaa00640 0 client.26924 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 --2- 172.21.15.27:0/1758566539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55979baac1b0 0x55979be8f970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 >> 172.21.15.27:0/1758566539 conn(0x55979bac9460 msgr2=0x55979be7f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 shutdown_connections 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 1 -- 172.21.15.27:0/1758566539 wait complete. 2024-09-21T07:18:08.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.554+0000 7ff6d23f5740 20 client.26924 trim_cache size 0 max 16384 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6ac0b6680 msgr2=0x7ff6ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6ac0b6680 0x7ff6ac0b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8031440 tx=0x7ff6b80058d0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c410be40 msgr2=0x7ff6c41a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c410be40 0x7ff6c41a1d30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009fd0 tx=0x7ff6bc00ef10 comp rx=0 tx=0).stop 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 shutdown_connections 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c410b4f0 0x7ff6c419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff6ac0b6680 0x7ff6ac0b8b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6c410be40 0x7ff6c41a1d30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 --2- 172.21.15.27:0/610576788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c4105bb0 0x7ff6c419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 >> 172.21.15.27:0/610576788 conn(0x7ff6c4101680 msgr2=0x7ff6c410abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.555+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 shutdown_connections 2024-09-21T07:18:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:08.556+0000 7ff6ca200640 1 -- 172.21.15.27:0/610576788 wait complete. 2024-09-21T07:18:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:09 smithi027 ceph-mon[30779]: pgmap v531: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 68 KiB/s wr, 30 op/s 2024-09-21T07:18:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:09 smithi153 ceph-mon[34747]: pgmap v531: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 68 KiB/s wr, 30 op/s 2024-09-21T07:18:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:09 smithi119 ceph-mon[34519]: pgmap v531: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 68 KiB/s wr, 30 op/s 2024-09-21T07:18:09.536 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:18:09.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:18:09.924 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 -- 172.21.15.27:0/3289641018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 msgr2=0x7f151010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 --2- 172.21.15.27:0/3289641018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 0x7f151010d850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009900 tx=0x7f14f802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 -- 172.21.15.27:0/3289641018 shutdown_connections 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 --2- 172.21.15.27:0/3289641018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f151010de10 0x7f1510110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 --2- 172.21.15.27:0/3289641018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 0x7f151010d850 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 --2- 172.21.15.27:0/3289641018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 0x7f1510104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 -- 172.21.15.27:0/3289641018 >> 172.21.15.27:0/3289641018 conn(0x7f1510079760 msgr2=0x7f1510079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 -- 172.21.15.27:0/3289641018 shutdown_connections 2024-09-21T07:18:10.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.575+0000 7f1515600640 1 -- 172.21.15.27:0/3289641018 wait complete. 2024-09-21T07:18:10.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.576+0000 7f1515600640 1 Processor -- start 2024-09-21T07:18:10.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1515600640 1 -- start start 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1515600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 0x7f15101026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1515600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 0x7f1510102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1515600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f151010de10 0x7f1510103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1515600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1510113ac0 con 0x7f1510105300 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1515600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1510113940 con 0x7f15101049b0 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1515600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1510113c40 con 0x7f151010de10 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f1506000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 0x7f1510102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f150f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f151010de10 0x7f1510103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.577+0000 7f150ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 0x7f15101026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f1506000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 0x7f1510102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44894/0 (socket says 172.21.15.27:44894) 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f1506000640 1 -- 172.21.15.27:0/2099825131 learned_addr learned my addr 172.21.15.27:0/2099825131 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:10.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f150ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 0x7f15101026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34404/0 (socket says 172.21.15.27:34404) 2024-09-21T07:18:10.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f150ea00640 1 -- 172.21.15.27:0/2099825131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f151010de10 msgr2=0x7f1510103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:10.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f150ea00640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f151010de10 0x7f1510103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f150ea00640 1 -- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 msgr2=0x7f1510102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:10.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f150ea00640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 0x7f1510102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f150ea00640 1 -- 172.21.15.27:0/2099825131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15101a8830 con 0x7f15101049b0 2024-09-21T07:18:10.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.578+0000 7f150ea00640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 0x7f15101026f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f150000e9f0 tx=0x7f150000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:10.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.579+0000 7f1507e00640 1 -- 172.21.15.27:0/2099825131 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150000cd80 con 0x7f15101049b0 2024-09-21T07:18:10.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.579+0000 7f1507e00640 1 -- 172.21.15.27:0/2099825131 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15000035f0 con 0x7f15101049b0 2024-09-21T07:18:10.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.579+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15101a8b20 con 0x7f15101049b0 2024-09-21T07:18:10.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.579+0000 7f1507e00640 1 -- 172.21.15.27:0/2099825131 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500010670 con 0x7f15101049b0 2024-09-21T07:18:10.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.579+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15101a9030 con 0x7f15101049b0 2024-09-21T07:18:10.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.580+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151006c2c0 con 0x7f15101049b0 2024-09-21T07:18:10.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.583+0000 7f1507e00640 1 -- 172.21.15.27:0/2099825131 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f15000037a0 con 0x7f15101049b0 2024-09-21T07:18:10.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.584+0000 7f1507e00640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14f40b6680 0x7f14f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:10.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.584+0000 7f1507e00640 1 -- 172.21.15.27:0/2099825131 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f15000e7f40 con 0x7f15101049b0 2024-09-21T07:18:10.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.584+0000 7f1506000640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14f40b6680 0x7f14f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:10.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.585+0000 7f1506000640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14f40b6680 0x7f14f40b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006010 tx=0x7f14f8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:10.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.586+0000 7f1507e00640 1 -- 172.21.15.27:0/2099825131 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1500056710 con 0x7f15101049b0 2024-09-21T07:18:10.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.825+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1510104dc0 con 0x7f15101049b0 2024-09-21T07:18:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.826+0000 7f1507e00640 1 -- 172.21.15.27:0/2099825131 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f1510104dc0 con 0x7f15101049b0 2024-09-21T07:18:10.828 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:18:10.828 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:18:10.829 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.828+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14f40b6680 msgr2=0x7f14f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.828+0000 7f1515600640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14f40b6680 0x7f14f40b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006010 tx=0x7f14f8005a20 comp rx=0 tx=0).stop 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.828+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 msgr2=0x7f15101026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.828+0000 7f1515600640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 0x7f15101026f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f150000e9f0 tx=0x7f150000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 shutdown_connections 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f151010de10 0x7f1510103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14f40b6680 0x7f14f40b8b90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510105300 0x7f1510102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 --2- 172.21.15.27:0/2099825131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15101049b0 0x7f15101026f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 >> 172.21.15.27:0/2099825131 conn(0x7f1510079760 msgr2=0x7f1510106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 shutdown_connections 2024-09-21T07:18:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:10.829+0000 7f1515600640 1 -- 172.21.15.27:0/2099825131 wait complete. 2024-09-21T07:18:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:11 smithi153 ceph-mon[34747]: pgmap v532: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 61 KiB/s wr, 29 op/s 2024-09-21T07:18:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:11 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2099825131' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:11.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:11 smithi119 ceph-mon[34519]: pgmap v532: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 61 KiB/s wr, 29 op/s 2024-09-21T07:18:11.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:11 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2099825131' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:11 smithi027 ceph-mon[30779]: pgmap v532: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 61 KiB/s wr, 29 op/s 2024-09-21T07:18:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:11 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2099825131' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:11.629 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:18:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:12.154 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 -- 172.21.15.27:0/429757251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae8075750 msgr2=0x7f4ae8075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 --2- 172.21.15.27:0/429757251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae8075750 0x7f4ae8075b60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009900 tx=0x7f4ad802a410 comp rx=0 tx=0).stop 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 -- 172.21.15.27:0/429757251 shutdown_connections 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 --2- 172.21.15.27:0/429757251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ae8110120 0x7f4ae8112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 --2- 172.21.15.27:0/429757251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ae80760a0 0x7f4ae810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 --2- 172.21.15.27:0/429757251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae8075750 0x7f4ae8075b60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 -- 172.21.15.27:0/429757251 >> 172.21.15.27:0/429757251 conn(0x7f4ae807b850 msgr2=0x7f4ae807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 -- 172.21.15.27:0/429757251 shutdown_connections 2024-09-21T07:18:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.652+0000 7f4aee800640 1 -- 172.21.15.27:0/429757251 wait complete. 2024-09-21T07:18:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.654+0000 7f4aee800640 1 Processor -- start 2024-09-21T07:18:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.654+0000 7f4aee800640 1 -- start start 2024-09-21T07:18:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4aee800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ae8075750 0x7f4ae819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4aee800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae80760a0 0x7f4ae819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4aee800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ae8110120 0x7f4ae81a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4aee800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8115c90 con 0x7f4ae8075750 2024-09-21T07:18:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4aee800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8115b10 con 0x7f4ae8110120 2024-09-21T07:18:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4aee800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8115e10 con 0x7f4ae80760a0 2024-09-21T07:18:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4ae7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ae8075750 0x7f4ae819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4ae7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ae8075750 0x7f4ae819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44920/0 (socket says 172.21.15.27:44920) 2024-09-21T07:18:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4ae7400640 1 -- 172.21.15.27:0/1184237476 learned_addr learned my addr 172.21.15.27:0/1184237476 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4ae7e00640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ae8110120 0x7f4ae81a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.655+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae80760a0 0x7f4ae819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.656+0000 7f4ae7e00640 1 -- 172.21.15.27:0/1184237476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae80760a0 msgr2=0x7f4ae819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.656+0000 7f4ae7e00640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae80760a0 0x7f4ae819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.656+0000 7f4ae7e00640 1 -- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ae8075750 msgr2=0x7f4ae819c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.656+0000 7f4ae7e00640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ae8075750 0x7f4ae819c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.656+0000 7f4ae7e00640 1 -- 172.21.15.27:0/1184237476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae81a25f0 con 0x7f4ae8110120 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.656+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae80760a0 0x7f4ae819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.656+0000 7f4ae7e00640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ae8110120 0x7f4ae81a1ec0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009fd0 tx=0x7f4ae000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.657+0000 7f4adfe00640 1 -- 172.21.15.27:0/1184237476 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000ce20 con 0x7f4ae8110120 2024-09-21T07:18:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.657+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae81a28e0 con 0x7f4ae8110120 2024-09-21T07:18:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.657+0000 7f4adfe00640 1 -- 172.21.15.27:0/1184237476 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae0005020 con 0x7f4ae8110120 2024-09-21T07:18:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.657+0000 7f4adfe00640 1 -- 172.21.15.27:0/1184237476 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0010690 con 0x7f4ae8110120 2024-09-21T07:18:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.657+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae81a9040 con 0x7f4ae8110120 2024-09-21T07:18:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.660+0000 7f4af53c2740 1 Processor -- start 2024-09-21T07:18:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4adfe00640 1 -- 172.21.15.27:0/1184237476 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4ae00054d0 con 0x7f4ae8110120 2024-09-21T07:18:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4af53c2740 1 -- start start 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dafda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e01b0 0x55c223db02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223db0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4af53c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c223d26fb0 con 0x55c2239e01b0 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4af53c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c223d163e0 con 0x55c223a09070 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.661+0000 7f4af53c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c223d1a1f0 con 0x55c2239e8390 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dafda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dafda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34428/0 (socket says 172.21.15.27:34428) 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 -- 172.21.15.27:0/1682634433 learned_addr learned my addr 172.21.15.27:0/1682634433 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae7400640 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e01b0 0x55c223db02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223db0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 -- 172.21.15.27:0/1682634433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 msgr2=0x55c223db0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223db0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 -- 172.21.15.27:0/1682634433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e01b0 msgr2=0x55c223db02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e01b0 0x55c223db02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.662+0000 7f4ae6a00640 1 -- 172.21.15.27:0/1682634433 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c223ce0cd0 con 0x55c223a09070 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.663+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dafda0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011b70 tx=0x7f4ad4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2e31b7decba311b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.663+0000 7f4adfe00640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4acc0b6700 0x7f4acc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.663+0000 7f4adcc00640 1 -- 172.21.15.27:0/1682634433 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400fc50 con 0x55c223a09070 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.663+0000 7f4adcc00640 1 -- 172.21.15.27:0/1682634433 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad400fdf0 con 0x55c223a09070 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.663+0000 7f4adcc00640 1 -- 172.21.15.27:0/1682634433 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4005440 con 0x55c223a09070 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.663+0000 7f4adfe00640 1 -- 172.21.15.27:0/1184237476 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4ae00e8130 con 0x7f4ae8110120 2024-09-21T07:18:12.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.663+0000 7f4ae7400640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4acc0b6700 0x7f4acc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4af53c2740 1 -- 172.21.15.27:0/1682634433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 msgr2=0x55c223dafda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4af53c2740 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dafda0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011b70 tx=0x7f4ad4004730 comp rx=0 tx=0).stop 2024-09-21T07:18:12.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4ae7400640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4acc0b6700 0x7f4acc0b8c10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8031440 tx=0x7f4ad80058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:12.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4af53c2740 1 -- 172.21.15.27:0/1682634433 shutdown_connections 2024-09-21T07:18:12.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4af53c2740 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223db0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4af53c2740 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e01b0 0x55c223db02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4af53c2740 1 --2- 172.21.15.27:0/1682634433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dafda0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.664+0000 7f4af53c2740 1 -- 172.21.15.27:0/1682634433 >> 172.21.15.27:0/1682634433 conn(0x55c2239fd460 msgr2=0x55c223dadc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:12.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.665+0000 7f4af53c2740 1 -- 172.21.15.27:0/1682634433 shutdown_connections 2024-09-21T07:18:12.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.665+0000 7f4af53c2740 1 -- 172.21.15.27:0/1682634433 wait complete. 2024-09-21T07:18:12.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.665+0000 7f4af53c2740 1 -- start start 2024-09-21T07:18:12.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.665+0000 7f4af53c2740 1 Processor -- start 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223dbb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dbba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4af53c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c223b77700 con 0x55c223a09070 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4af53c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c223cb1580 con 0x55c2239e01b0 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4af53c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c223dbe580 con 0x55c2239e8390 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4af53c2740 10 client.0 _ll_register_callbacks cb 0x55c223cc1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4ae7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223dbb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4ae7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223dbb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34432/0 (socket says 172.21.15.27:34432) 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4ae7400640 1 -- 172.21.15.27:0/2802353507 learned_addr learned my addr 172.21.15.27:0/2802353507 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dbba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.666+0000 7f4ae7e00640 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.667+0000 7f4ae6a00640 1 -- 172.21.15.27:0/2802353507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 msgr2=0x55c223dbb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.667+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223dbb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.667+0000 7f4ae6a00640 1 -- 172.21.15.27:0/2802353507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 msgr2=0x55c223dc78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.667+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.667+0000 7f4ae6a00640 1 -- 172.21.15.27:0/2802353507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c223a74650 con 0x55c2239e8390 2024-09-21T07:18:12.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.668+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dbba60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011b40 tx=0x7f4ad4009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1173f6e165a710 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:12.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.668+0000 7f4ad1600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:12.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.668+0000 7f4ad1600640 1 -- 172.21.15.27:0/2802353507 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40053c0 con 0x55c2239e8390 2024-09-21T07:18:12.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.668+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c223cb3b90 con 0x55c2239e8390 2024-09-21T07:18:12.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.668+0000 7f4ad1600640 1 -- 172.21.15.27:0/2802353507 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4ad4005560 con 0x55c2239e8390 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.669+0000 7f4ad1600640 1 -- 172.21.15.27:0/2802353507 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ad40047a0 con 0x55c2239e8390 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.669+0000 7f4af53c2740 10 client.26929 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.669+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c223cce950 con 0x55c2239e8390 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.669+0000 7f4ad1600640 1 -- 172.21.15.27:0/2802353507 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c223cb3b90 con 0x55c2239e8390 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4ad1600640 1 -- 172.21.15.27:0/2802353507 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4ad4026070 con 0x55c2239e8390 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4af53c2740 10 client.26929 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4af53c2740 10 client.26929 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4af53c2740 20 client.26929 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c2239c79f0 0x55c223dcd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4af53c2740 4 client.26929 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:12.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c223d28f00 con 0x55c2239c79f0 2024-09-21T07:18:12.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.670+0000 7f4ae7400640 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c2239c79f0 0x55c223dcd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.671+0000 7f4ad1600640 10 client.26929 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:12.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.671+0000 7f4ae7400640 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c2239c79f0 0x55c223dcd4b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad63700b811704de server_cookie=c64fe4ad351ffcea in_seq=0 out_seq=0 2024-09-21T07:18:12.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.672+0000 7f4ad1600640 1 -- 172.21.15.27:0/2802353507 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c223d28f00 con 0x55c2239c79f0 2024-09-21T07:18:12.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.672+0000 7f4ad1600640 10 client.26929 handle_command_reply: tid=0 2024-09-21T07:18:12.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.672+0000 7f4af53c2740 1 client.26929 shutdown 2024-09-21T07:18:12.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.672+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 msgr2=0x55c223dbba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.672+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dbba60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011b40 tx=0x7f4ad4009370 comp rx=0 tx=0).stop 2024-09-21T07:18:12.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.672+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 shutdown_connections 2024-09-21T07:18:12.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.672+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c2239c79f0 0x55c223dcd4b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dbba60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2802353507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223dbb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4ad1600640 0 client.26929 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 >> 172.21.15.27:0/2802353507 conn(0x55c2239fd460 msgr2=0x55c223db2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 shutdown_connections 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4af53c2740 1 -- 172.21.15.27:0/2802353507 wait complete. 2024-09-21T07:18:12.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.673+0000 7f4af53c2740 20 client.26929 trim_cache size 0 max 16384 2024-09-21T07:18:12.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.691+0000 7f4af53c2740 1 Processor -- start 2024-09-21T07:18:12.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.691+0000 7f4af53c2740 1 -- start start 2024-09-21T07:18:12.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223db35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dc77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4af53c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b5850 con 0x55c223a09070 2024-09-21T07:18:12.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4af53c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b3120 con 0x55c2239e01b0 2024-09-21T07:18:12.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4af53c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b0f70 con 0x55c2239e8390 2024-09-21T07:18:12.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4ae7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dc77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dc77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57560/0 (socket says 172.21.15.27:57560) 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4ae7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223db35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.692+0000 7f4ae6a00640 1 -- 172.21.15.27:0/2731339468 learned_addr learned my addr 172.21.15.27:0/2731339468 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ae6a00640 1 -- 172.21.15.27:0/2731339468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 msgr2=0x55c223db35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2731339468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223db35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ae6a00640 1 -- 172.21.15.27:0/2731339468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 msgr2=0x55c223dc7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2731339468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ae6a00640 1 -- 172.21.15.27:0/2731339468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c223db2380 con 0x55c2239e8390 2024-09-21T07:18:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2731339468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dc77a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004ff0 tx=0x7f4ad4005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0dba84b66d6db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ad3e00640 1 -- 172.21.15.27:0/2731339468 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4039070 con 0x55c2239e8390 2024-09-21T07:18:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ad3e00640 1 -- 172.21.15.27:0/2731339468 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad401fe50 con 0x55c2239e8390 2024-09-21T07:18:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.693+0000 7f4ad3e00640 1 -- 172.21.15.27:0/2731339468 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40052f0 con 0x55c2239e8390 2024-09-21T07:18:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 -- 172.21.15.27:0/2731339468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 msgr2=0x55c223dc77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2731339468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dc77a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004ff0 tx=0x7f4ad4005d30 comp rx=0 tx=0).stop 2024-09-21T07:18:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 -- 172.21.15.27:0/2731339468 shutdown_connections 2024-09-21T07:18:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2731339468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e8390 0x55c223dc77a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2731339468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c223a09070 0x55c223dc7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2731339468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2239e01b0 0x55c223db35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 -- 172.21.15.27:0/2731339468 >> 172.21.15.27:0/2731339468 conn(0x55c2239fd460 msgr2=0x55c223db2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 -- 172.21.15.27:0/2731339468 shutdown_connections 2024-09-21T07:18:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 -- 172.21.15.27:0/2731339468 wait complete. 2024-09-21T07:18:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.695+0000 7f4af53c2740 1 -- start start 2024-09-21T07:18:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.696+0000 7f4af53c2740 1 Processor -- start 2024-09-21T07:18:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.696+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e01b0 0x55c223dc4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e8390 0x55c223dbf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4af53c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dbfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4af53c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b15e0 con 0x55c2239e8390 2024-09-21T07:18:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4af53c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b2a50 con 0x55c223a09070 2024-09-21T07:18:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4af53c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b1cb0 con 0x55c2239e01b0 2024-09-21T07:18:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4af53c2740 10 client.0 _ll_register_callbacks cb 0x7f4ad4005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e01b0 0x55c223dc4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e01b0 0x55c223dc4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57570/0 (socket says 172.21.15.27:57570) 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 -- 172.21.15.27:0/2882873656 learned_addr learned my addr 172.21.15.27:0/2882873656 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e8390 0x55c223dbf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7e00640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dbfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 -- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 msgr2=0x55c223dbfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dbfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 -- 172.21.15.27:0/2882873656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e8390 msgr2=0x55c223dbf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e8390 0x55c223dbf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae7400640 1 -- 172.21.15.27:0/2882873656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c223dc03d0 con 0x55c2239e01b0 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.697+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e8390 0x55c223dbf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.698+0000 7f4ae7400640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e01b0 0x55c223dc4110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8004770 tx=0x7f4ad8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e769e29e152669 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.698+0000 7f4ad1600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:12.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.698+0000 7f4ad1600640 1 -- 172.21.15.27:0/2882873656 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8035040 con 0x55c2239e01b0 2024-09-21T07:18:12.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.698+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad401fe50 con 0x55c2239e01b0 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.698+0000 7f4ad1600640 1 -- 172.21.15.27:0/2882873656 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4ad80035f0 con 0x55c2239e01b0 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4ad1600640 1 -- 172.21.15.27:0/2882873656 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ad80037f0 con 0x55c2239e01b0 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 10 client.26935 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c223e37f00 con 0x55c2239e01b0 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4ad1600640 1 -- 172.21.15.27:0/2882873656 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad401fe50 con 0x55c2239e01b0 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4ad1600640 1 -- 172.21.15.27:0/2882873656 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4ad803c9e0 con 0x55c2239e01b0 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 10 client.26935 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 10 client.26935 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 20 client.26935 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c22399cda0 0x55c223e3a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 4 client.26935 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.699+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c223d249b0 con 0x55c22399cda0 2024-09-21T07:18:12.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.700+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c22399cda0 0x55c223e3a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.700+0000 7f4ae6a00640 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c22399cda0 0x55c223e3a6a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7366c65af337a5ec server_cookie=d876f0666f695fa8 in_seq=0 out_seq=0 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.700+0000 7f4ad1600640 10 client.26935 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4ad1600640 1 -- 172.21.15.27:0/2882873656 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c223d249b0 con 0x55c22399cda0 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4ad1600640 10 client.26935 handle_command_reply: tid=0 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 client.26935 shutdown 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e01b0 msgr2=0x55c223dc4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e01b0 0x55c223dc4110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8004770 tx=0x7f4ad8009260 comp rx=0 tx=0).stop 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 shutdown_connections 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55c22399cda0 0x55c223e3a6a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4ad1600640 0 client.26935 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c223a09070 0x55c223dbfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2239e8390 0x55c223dbf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 --2- 172.21.15.27:0/2882873656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2239e01b0 0x55c223dc4110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 >> 172.21.15.27:0/2882873656 conn(0x55c2239fd460 msgr2=0x55c223db0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 shutdown_connections 2024-09-21T07:18:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 1 -- 172.21.15.27:0/2882873656 wait complete. 2024-09-21T07:18:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.701+0000 7f4af53c2740 20 client.26935 trim_cache size 0 max 16384 2024-09-21T07:18:12.705 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.702+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4acc0b6700 msgr2=0x7f4acc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.702+0000 7f4aee800640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4acc0b6700 0x7f4acc0b8c10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8031440 tx=0x7f4ad80058d0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.702+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ae8110120 msgr2=0x7f4ae81a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.702+0000 7f4aee800640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ae8110120 0x7f4ae81a1ec0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009fd0 tx=0x7f4ae000ef10 comp rx=0 tx=0).stop 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 shutdown_connections 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ae80760a0 0x7f4ae819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4acc0b6700 0x7f4acc0b8c10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ae8110120 0x7f4ae81a1ec0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 --2- 172.21.15.27:0/1184237476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ae8075750 0x7f4ae819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 >> 172.21.15.27:0/1184237476 conn(0x7f4ae807b850 msgr2=0x7f4ae8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 shutdown_connections 2024-09-21T07:18:12.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:12.703+0000 7f4aee800640 1 -- 172.21.15.27:0/1184237476 wait complete. 2024-09-21T07:18:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:13 smithi153 ceph-mon[34747]: pgmap v533: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 61 KiB/s wr, 29 op/s 2024-09-21T07:18:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:13 smithi119 ceph-mon[34519]: pgmap v533: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 61 KiB/s wr, 29 op/s 2024-09-21T07:18:13.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:13 smithi027 ceph-mon[30779]: pgmap v533: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 61 KiB/s wr, 29 op/s 2024-09-21T07:18:13.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:18:14.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 -- 172.21.15.27:0/2540873152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810c040 msgr2=0x7f82a8112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:14.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 --2- 172.21.15.27:0/2540873152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810c040 0x7f82a8112870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009ca0 tx=0x7f82a002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 -- 172.21.15.27:0/2540873152 shutdown_connections 2024-09-21T07:18:14.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 --2- 172.21.15.27:0/2540873152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810c040 0x7f82a8112870 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 --2- 172.21.15.27:0/2540873152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a810b6f0 0x7f82a810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 --2- 172.21.15.27:0/2540873152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a8105db0 0x7f82a81061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 -- 172.21.15.27:0/2540873152 >> 172.21.15.27:0/2540873152 conn(0x7f82a81017e0 msgr2=0x7f82a8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:14.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 -- 172.21.15.27:0/2540873152 shutdown_connections 2024-09-21T07:18:14.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.286+0000 7f82af200640 1 -- 172.21.15.27:0/2540873152 wait complete. 2024-09-21T07:18:14.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 Processor -- start 2024-09-21T07:18:14.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 -- start start 2024-09-21T07:18:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a8105db0 0x7f82a807b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810b6f0 0x7f82a8077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a810c040 0x7f82a8078040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8115d50 con 0x7f82a810b6f0 2024-09-21T07:18:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8115bd0 con 0x7f82a8105db0 2024-09-21T07:18:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.288+0000 7f82af200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8115ed0 con 0x7f82a810c040 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82ad200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a810c040 0x7f82a8078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a8105db0 0x7f82a807b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82ad200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a810c040 0x7f82a8078040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57586/0 (socket says 172.21.15.27:57586) 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82ad200640 1 -- 172.21.15.27:0/825562263 learned_addr learned my addr 172.21.15.27:0/825562263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82a7400640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810b6f0 0x7f82a8077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82a7e00640 1 -- 172.21.15.27:0/825562263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a810c040 msgr2=0x7f82a8078040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82a7e00640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a810c040 0x7f82a8078040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82a7e00640 1 -- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810b6f0 msgr2=0x7f82a8077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82a7e00640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810b6f0 0x7f82a8077920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.289+0000 7f82a7e00640 1 -- 172.21.15.27:0/825562263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a81acc10 con 0x7f82a8105db0 2024-09-21T07:18:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.290+0000 7f82a7e00640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a8105db0 0x7f82a807b640 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f829800e9f0 tx=0x7f829800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:14.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.290+0000 7f82a4c00640 1 -- 172.21.15.27:0/825562263 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829800ccc0 con 0x7f82a8105db0 2024-09-21T07:18:14.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.290+0000 7f82a4c00640 1 -- 172.21.15.27:0/825562263 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8298005020 con 0x7f82a8105db0 2024-09-21T07:18:14.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.290+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a81ace40 con 0x7f82a8105db0 2024-09-21T07:18:14.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.290+0000 7f82a4c00640 1 -- 172.21.15.27:0/825562263 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298004a00 con 0x7f82a8105db0 2024-09-21T07:18:14.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.290+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a81ad240 con 0x7f82a8105db0 2024-09-21T07:18:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.292+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a81061c0 con 0x7f82a8105db0 2024-09-21T07:18:14.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.294+0000 7f82a4c00640 1 -- 172.21.15.27:0/825562263 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8298004ba0 con 0x7f82a8105db0 2024-09-21T07:18:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.297+0000 7f82a4c00640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f828c0b6680 0x7f828c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.297+0000 7f82a4c00640 1 -- 172.21.15.27:0/825562263 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f82980e7d60 con 0x7f82a8105db0 2024-09-21T07:18:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.297+0000 7f82a7400640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f828c0b6680 0x7f828c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.298+0000 7f82a4c00640 1 -- 172.21.15.27:0/825562263 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82980adf00 con 0x7f82a8105db0 2024-09-21T07:18:14.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.298+0000 7f82a7400640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f828c0b6680 0x7f828c0b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f82a8104ee0 tx=0x7f8294005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:14.315 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:14.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.543+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82a8078c20 con 0x7f82a8105db0 2024-09-21T07:18:14.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.544+0000 7f82a4c00640 1 -- 172.21.15.27:0/825562263 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f82a8078c20 con 0x7f82a8105db0 2024-09-21T07:18:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:18:14.546 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:18:14.547 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f828c0b6680 msgr2=0x7f828c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f828c0b6680 0x7f828c0b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f82a8104ee0 tx=0x7f8294005f90 comp rx=0 tx=0).stop 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a8105db0 msgr2=0x7f82a807b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a8105db0 0x7f82a807b640 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f829800e9f0 tx=0x7f829800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 shutdown_connections 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a810c040 0x7f82a8078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f828c0b6680 0x7f828c0b8b90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82a810b6f0 0x7f82a8077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 --2- 172.21.15.27:0/825562263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a8105db0 0x7f82a807b640 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.546+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 >> 172.21.15.27:0/825562263 conn(0x7f82a81017e0 msgr2=0x7f82a8108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.547+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 shutdown_connections 2024-09-21T07:18:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:14.547+0000 7f82af200640 1 -- 172.21.15.27:0/825562263 wait complete. 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.092+0000 7fd475800640 1 -- 172.21.15.27:0/17236397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd47010c010 msgr2=0x7fd470112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.092+0000 7fd475800640 1 --2- 172.21.15.27:0/17236397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd47010c010 0x7fd470112840 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd468009ca0 tx=0x7fd46802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.092+0000 7fd475800640 1 -- 172.21.15.27:0/17236397 shutdown_connections 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.092+0000 7fd475800640 1 --2- 172.21.15.27:0/17236397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd47010b6c0 0x7fd47010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.092+0000 7fd475800640 1 --2- 172.21.15.27:0/17236397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd47010c010 0x7fd470112840 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.092+0000 7fd475800640 1 --2- 172.21.15.27:0/17236397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd470105d80 0x7fd470106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.092+0000 7fd475800640 1 -- 172.21.15.27:0/17236397 >> 172.21.15.27:0/17236397 conn(0x7fd470101810 msgr2=0x7fd470103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.093+0000 7fd475800640 1 -- 172.21.15.27:0/17236397 shutdown_connections 2024-09-21T07:18:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.093+0000 7fd475800640 1 -- 172.21.15.27:0/17236397 wait complete. 2024-09-21T07:18:15.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.094+0000 7fd475800640 1 Processor -- start 2024-09-21T07:18:15.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.094+0000 7fd475800640 1 -- start start 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.094+0000 7fd475800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd470105d80 0x7fd47019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.094+0000 7fd475800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47010b6c0 0x7fd47019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.094+0000 7fd475800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd47010c010 0x7fd4701a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd475800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd470115d20 con 0x7fd470105d80 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd475800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd470115ba0 con 0x7fd47010b6c0 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd475800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd470115ea0 con 0x7fd47010c010 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd470105d80 0x7fd47019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd470105d80 0x7fd47019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44978/0 (socket says 172.21.15.27:44978) 2024-09-21T07:18:15.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd467e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47010b6c0 0x7fd47019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd46ea00640 1 -- 172.21.15.27:0/1139540504 learned_addr learned my addr 172.21.15.27:0/1139540504 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:15.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd46f400640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd47010c010 0x7fd4701a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd467e00640 1 -- 172.21.15.27:0/1139540504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd47010c010 msgr2=0x7fd4701a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd467e00640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd47010c010 0x7fd4701a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd467e00640 1 -- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd470105d80 msgr2=0x7fd47019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd467e00640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd470105d80 0x7fd47019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.095+0000 7fd467e00640 1 -- 172.21.15.27:0/1139540504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4701a2680 con 0x7fd47010b6c0 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.096+0000 7fd46ea00640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd470105d80 0x7fd47019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.096+0000 7fd467e00640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47010b6c0 0x7fd47019ccb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd47006caa0 tx=0x7fd45800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.096+0000 7fd467400640 1 -- 172.21.15.27:0/1139540504 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45800cd00 con 0x7fd47010b6c0 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.096+0000 7fd467400640 1 -- 172.21.15.27:0/1139540504 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4580035f0 con 0x7fd47010b6c0 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.096+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701a2970 con 0x7fd47010b6c0 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.096+0000 7fd467400640 1 -- 172.21.15.27:0/1139540504 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458010690 con 0x7fd47010b6c0 2024-09-21T07:18:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.096+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701a90d0 con 0x7fd47010b6c0 2024-09-21T07:18:15.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.099+0000 7fd47c47c740 1 Processor -- start 2024-09-21T07:18:15.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd47c47c740 1 -- start start 2024-09-21T07:18:15.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd47c47c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b522433fb0 con 0x55b5220f5390 2024-09-21T07:18:15.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd47c47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b5224233e0 con 0x55b522116070 2024-09-21T07:18:15.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd47c47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b5224271f0 con 0x55b5220ed1b0 2024-09-21T07:18:15.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd46f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd46f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224bcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34484/0 (socket says 172.21.15.27:34484) 2024-09-21T07:18:15.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd46f400640 1 -- 172.21.15.27:0/1146939998 learned_addr learned my addr 172.21.15.27:0/1146939998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:15.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.100+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.101+0000 7fd467400640 1 -- 172.21.15.27:0/1139540504 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd4580037a0 con 0x7fd47010b6c0 2024-09-21T07:18:15.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.101+0000 7fd46f400640 1 -- 172.21.15.27:0/1146939998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 msgr2=0x55b5224bd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.101+0000 7fd46f400640 1 --2- 172.21.15.27:0/1146939998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224bd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.101+0000 7fd46f400640 1 -- 172.21.15.27:0/1146939998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 msgr2=0x55b5224bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.101+0000 7fd46f400640 1 --2- 172.21.15.27:0/1146939998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.101+0000 7fd46f400640 1 -- 172.21.15.27:0/1146939998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5223edcd0 con 0x55b522116070 2024-09-21T07:18:15.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.101+0000 7fd46f400640 1 --2- 172.21.15.27:0/1146939998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224bcda0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd468009ca0 tx=0x7fd468030fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a9b3b42404cffe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.102+0000 7fd45fe00640 1 -- 172.21.15.27:0/1146939998 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468030800 con 0x55b522116070 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.102+0000 7fd45fe00640 1 -- 172.21.15.27:0/1146939998 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd468036030 con 0x55b522116070 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.102+0000 7fd45fe00640 1 -- 172.21.15.27:0/1146939998 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4680043a0 con 0x55b522116070 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.102+0000 7fd467400640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4540b6680 0x7fd4540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.102+0000 7fd467400640 1 -- 172.21.15.27:0/1139540504 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd4580e7cb0 con 0x7fd47010b6c0 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 -- 172.21.15.27:0/1146939998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 msgr2=0x55b5224bcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd46ea00640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4540b6680 0x7fd4540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 --2- 172.21.15.27:0/1146939998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224bcda0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd468009ca0 tx=0x7fd468030fd0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 -- 172.21.15.27:0/1146939998 shutdown_connections 2024-09-21T07:18:15.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 --2- 172.21.15.27:0/1146939998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 --2- 172.21.15.27:0/1146939998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 --2- 172.21.15.27:0/1146939998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224bcda0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 -- 172.21.15.27:0/1146939998 >> 172.21.15.27:0/1146939998 conn(0x55b52210a460 msgr2=0x55b5224bac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:15.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 -- 172.21.15.27:0/1146939998 shutdown_connections 2024-09-21T07:18:15.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.103+0000 7fd47c47c740 1 -- 172.21.15.27:0/1146939998 wait complete. 2024-09-21T07:18:15.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 -- start start 2024-09-21T07:18:15.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd46ea00640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4540b6680 0x7fd4540b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55b5224bba20 tx=0x7fd460005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:15.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 Processor -- start 2024-09-21T07:18:15.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220ed1b0 0x55b5224c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220f5390 0x55b5224c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b5224d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b522284700 con 0x55b5220ed1b0 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b5223be580 con 0x55b5220f5390 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b5224cb520 con 0x55b522116070 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.104+0000 7fd47c47c740 10 client.0 _ll_register_callbacks cb 0x55b5223ce470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220f5390 0x55b5224c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220ed1b0 0x55b5224c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220f5390 0x55b5224c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34492/0 (socket says 172.21.15.27:34492) 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220ed1b0 0x55b5224c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44994/0 (socket says 172.21.15.27:44994) 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 -- 172.21.15.27:0/415600015 learned_addr learned my addr 172.21.15.27:0/415600015 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd46f400640 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b5224d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 -- 172.21.15.27:0/415600015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 msgr2=0x55b5224d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b5224d4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 -- 172.21.15.27:0/415600015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220ed1b0 msgr2=0x55b5224c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220ed1b0 0x55b5224c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 -- 172.21.15.27:0/415600015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b522181650 con 0x55b5220f5390 2024-09-21T07:18:15.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd467e00640 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220f5390 0x55b5224c8b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4580a9d20 tx=0x7fd4580a9700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac967d82e5cdfbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:15.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.105+0000 7fd45cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:15.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd45cc00640 1 -- 172.21.15.27:0/415600015 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580e54e0 con 0x55b5220f5390 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd45cc00640 1 -- 172.21.15.27:0/415600015 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd4580a90a0 con 0x55b5220f5390 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5223c0b90 con 0x55b5220f5390 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd45cc00640 1 -- 172.21.15.27:0/415600015 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4580ad260 con 0x55b5220f5390 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd47c47c740 10 client.26966 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5223db950 con 0x55b5220f5390 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd45cc00640 1 -- 172.21.15.27:0/415600015 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b5223c0b90 con 0x55b5220f5390 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.106+0000 7fd45cc00640 1 -- 172.21.15.27:0/415600015 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd4580b33c0 con 0x55b5220f5390 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.107+0000 7fd47c47c740 10 client.26966 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.107+0000 7fd47c47c740 10 client.26966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.107+0000 7fd47c47c740 20 client.26966 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.107+0000 7fd47c47c740 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220d49f0 0x55b5224da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.107+0000 7fd47c47c740 4 client.26966 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:15.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.107+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b522435f00 con 0x55b5220d49f0 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.107+0000 7fd46ea00640 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220d49f0 0x55b5224da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.108+0000 7fd46ea00640 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220d49f0 0x55b5224da630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=231fcbf36c9101d server_cookie=b70f27d2abc544f5 in_seq=0 out_seq=0 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.108+0000 7fd45cc00640 10 client.26966 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.108+0000 7fd45cc00640 1 -- 172.21.15.27:0/415600015 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b522435f00 con 0x55b5220d49f0 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.108+0000 7fd45cc00640 10 client.26966 handle_command_reply: tid=0 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.108+0000 7fd47c47c740 1 client.26966 shutdown 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220f5390 msgr2=0x55b5224c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220f5390 0x55b5224c8b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4580a9d20 tx=0x7fd4580a9700 comp rx=0 tx=0).stop 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 shutdown_connections 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220d49f0 0x55b5224da630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b5224d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220f5390 0x55b5224c8b30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 --2- 172.21.15.27:0/415600015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220ed1b0 0x55b5224c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd45cc00640 0 client.26966 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:18:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 >> 172.21.15.27:0/415600015 conn(0x55b52210a460 msgr2=0x55b5224bff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:15.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 shutdown_connections 2024-09-21T07:18:15.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 1 -- 172.21.15.27:0/415600015 wait complete. 2024-09-21T07:18:15.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.109+0000 7fd47c47c740 20 client.26966 trim_cache size 0 max 16384 2024-09-21T07:18:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.123+0000 7fd47c47c740 1 Processor -- start 2024-09-21T07:18:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd47c47c740 1 -- start start 2024-09-21T07:18:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd47c47c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b5890 con 0x55b5220f5390 2024-09-21T07:18:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd47c47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b3160 con 0x55b522116070 2024-09-21T07:18:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd47c47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b0fb0 con 0x55b5220ed1b0 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224c05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57634/0 (socket says 172.21.15.27:57634) 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd467e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd46ea00640 1 -- 172.21.15.27:0/892207416 learned_addr learned my addr 172.21.15.27:0/892207416 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.124+0000 7fd46f400640 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.125+0000 7fd46ea00640 1 -- 172.21.15.27:0/892207416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 msgr2=0x55b5224e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.125+0000 7fd46ea00640 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.125+0000 7fd46ea00640 1 -- 172.21.15.27:0/892207416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 msgr2=0x55b5224dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.125+0000 7fd46ea00640 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.125+0000 7fd46ea00640 1 -- 172.21.15.27:0/892207416 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5224bf380 con 0x55b5220ed1b0 2024-09-21T07:18:15.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.125+0000 7fd46ea00640 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224c05b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd460013010 tx=0x7fd46001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674095fae56c7db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.125+0000 7fd45f400640 1 -- 172.21.15.27:0/892207416 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460011890 con 0x55b5220ed1b0 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.126+0000 7fd45f400640 1 -- 172.21.15.27:0/892207416 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd460005020 con 0x55b5220ed1b0 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.126+0000 7fd45f400640 1 -- 172.21.15.27:0/892207416 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4600049e0 con 0x55b5220ed1b0 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.126+0000 7fd47c47c740 1 -- 172.21.15.27:0/892207416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 msgr2=0x55b5224c05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.126+0000 7fd47c47c740 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224c05b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd460013010 tx=0x7fd46001b000 comp rx=0 tx=0).stop 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 -- 172.21.15.27:0/892207416 shutdown_connections 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b522116070 0x55b5224e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 --2- 172.21.15.27:0/892207416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5220ed1b0 0x55b5224c05b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 -- 172.21.15.27:0/892207416 >> 172.21.15.27:0/892207416 conn(0x55b52210a460 msgr2=0x55b5224bfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 -- 172.21.15.27:0/892207416 shutdown_connections 2024-09-21T07:18:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 -- 172.21.15.27:0/892207416 wait complete. 2024-09-21T07:18:15.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.127+0000 7fd47c47c740 1 -- start start 2024-09-21T07:18:15.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 1 Processor -- start 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220ed1b0 0x55b5224ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b522543fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b1620 con 0x55b5220f5390 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b2a90 con 0x55b5220ed1b0 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b1cf0 con 0x55b522116070 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd47c47c740 10 client.0 _ll_register_callbacks cb 0x7fd4580ad0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd467e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220ed1b0 0x55b5224ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd467e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45010/0 (socket says 172.21.15.27:45010) 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220ed1b0 0x55b5224ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34502/0 (socket says 172.21.15.27:34502) 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.128+0000 7fd467e00640 1 -- 172.21.15.27:0/592432172 learned_addr learned my addr 172.21.15.27:0/592432172 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:15.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd46f400640 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b522543fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd467e00640 1 -- 172.21.15.27:0/592432172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 msgr2=0x55b522543fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd467e00640 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b522543fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd467e00640 1 -- 172.21.15.27:0/592432172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220ed1b0 msgr2=0x55b5224ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd467e00640 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220ed1b0 0x55b5224ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd467e00640 1 -- 172.21.15.27:0/592432172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5224b60c0 con 0x55b5220f5390 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd46ea00640 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220ed1b0 0x55b5224ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd467e00640 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224cd420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd458003e40 tx=0x7fd4580a92d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f829bc8517b0035 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.129+0000 7fd45cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd45cc00640 1 -- 172.21.15.27:0/592432172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580e54e0 con 0x55b5220f5390 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd460005020 con 0x55b5220f5390 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd45cc00640 1 -- 172.21.15.27:0/592432172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd4580ac630 con 0x55b5220f5390 2024-09-21T07:18:15.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd45cc00640 1 -- 172.21.15.27:0/592432172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4580ade50 con 0x55b5220f5390 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd47c47c740 10 client.16938 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b522545610 con 0x55b5220f5390 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd45cc00640 1 -- 172.21.15.27:0/592432172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd460005020 con 0x55b5220f5390 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.130+0000 7fd45cc00640 1 -- 172.21.15.27:0/592432172 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd4580aea80 con 0x55b5220f5390 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.131+0000 7fd47c47c740 10 client.16938 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.131+0000 7fd47c47c740 10 client.16938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.131+0000 7fd47c47c740 20 client.16938 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.131+0000 7fd47c47c740 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220a9da0 0x55b522547c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.131+0000 7fd47c47c740 4 client.16938 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.131+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5224319b0 con 0x55b5220a9da0 2024-09-21T07:18:15.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.131+0000 7fd46ea00640 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220a9da0 0x55b522547c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd45cc00640 10 client.16938 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:18:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd46ea00640 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220a9da0 0x55b522547c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bebbb558301185eb server_cookie=bfa045f96825b85f in_seq=0 out_seq=0 2024-09-21T07:18:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd45cc00640 1 -- 172.21.15.27:0/592432172 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5224319b0 con 0x55b5220a9da0 2024-09-21T07:18:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd45cc00640 10 client.16938 handle_command_reply: tid=0 2024-09-21T07:18:15.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 client.16938 shutdown 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 msgr2=0x55b5224cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224cd420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd458003e40 tx=0x7fd4580a92d0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 shutdown_connections 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55b5220a9da0 0x55b522547c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b522116070 0x55b522543fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5220f5390 0x55b5224cd420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 --2- 172.21.15.27:0/592432172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5220ed1b0 0x55b5224ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 >> 172.21.15.27:0/592432172 conn(0x55b52210a460 msgr2=0x55b5224bddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.132+0000 7fd45cc00640 0 client.16938 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.133+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 shutdown_connections 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.133+0000 7fd47c47c740 1 -- 172.21.15.27:0/592432172 wait complete. 2024-09-21T07:18:15.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.133+0000 7fd47c47c740 20 client.16938 trim_cache size 0 max 16384 2024-09-21T07:18:15.137 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:18:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.133+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4540b6680 msgr2=0x7fd4540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.133+0000 7fd475800640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4540b6680 0x7fd4540b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55b5224bba20 tx=0x7fd460005f90 comp rx=0 tx=0).stop 2024-09-21T07:18:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.133+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47010b6c0 msgr2=0x7fd47019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.133+0000 7fd475800640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47010b6c0 0x7fd47019ccb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd47006caa0 tx=0x7fd45800ee50 comp rx=0 tx=0).stop 2024-09-21T07:18:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 shutdown_connections 2024-09-21T07:18:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd47010c010 0x7fd4701a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4540b6680 0x7fd4540b8b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47010b6c0 0x7fd47019ccb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 --2- 172.21.15.27:0/1139540504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd470105d80 0x7fd47019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 >> 172.21.15.27:0/1139540504 conn(0x7fd470101810 msgr2=0x7fd470108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 shutdown_connections 2024-09-21T07:18:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:15.134+0000 7fd475800640 1 -- 172.21.15.27:0/1139540504 wait complete. 2024-09-21T07:18:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:15 smithi153 ceph-mon[34747]: pgmap v534: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 23 op/s 2024-09-21T07:18:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:15 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/825562263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:15.496 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:15 smithi027 ceph-mon[30779]: pgmap v534: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 23 op/s 2024-09-21T07:18:15.496 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:15 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/825562263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:15.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:15 smithi119 ceph-mon[34519]: pgmap v534: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 23 op/s 2024-09-21T07:18:15.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:15 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/825562263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:16.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:18:16.508 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id fa29fba --release --await' 2024-09-21T07:18:16.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id fa29fba --release --await 2024-09-21T07:18:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 -- 172.21.15.27:0/3304938169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c105090 msgr2=0x7fd23c10d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 --2- 172.21.15.27:0/3304938169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c105090 0x7fd23c10d5d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd228009900 tx=0x7fd22802a410 comp rx=0 tx=0).stop 2024-09-21T07:18:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 -- 172.21.15.27:0/3304938169 shutdown_connections 2024-09-21T07:18:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 --2- 172.21.15.27:0/3304938169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c10dba0 0x7fd23c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 --2- 172.21.15.27:0/3304938169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c105090 0x7fd23c10d5d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 --2- 172.21.15.27:0/3304938169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd23c104740 0x7fd23c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 -- 172.21.15.27:0/3304938169 >> 172.21.15.27:0/3304938169 conn(0x7fd23c0fe230 msgr2=0x7fd23c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 -- 172.21.15.27:0/3304938169 shutdown_connections 2024-09-21T07:18:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.598+0000 7fd23b200640 1 -- 172.21.15.27:0/3304938169 wait complete. 2024-09-21T07:18:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.599+0000 7fd23b200640 1 Processor -- start 2024-09-21T07:18:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.599+0000 7fd23b200640 1 -- start start 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd23c104740 0x7fd23c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105090 0x7fd23c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c10dba0 0x7fd23c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1137e0 con 0x7fd23c10dba0 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c113660 con 0x7fd23c104740 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c113960 con 0x7fd23c105090 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c10dba0 0x7fd23c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c10dba0 0x7fd23c102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45022/0 (socket says 172.21.15.27:45022) 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd23a800640 1 -- 172.21.15.27:0/2146381746 learned_addr learned my addr 172.21.15.27:0/2146381746 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd239400640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105090 0x7fd23c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.600+0000 7fd239e00640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd23c104740 0x7fd23c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd23a800640 1 -- 172.21.15.27:0/2146381746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105090 msgr2=0x7fd23c1029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd23a800640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105090 0x7fd23c1029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd23a800640 1 -- 172.21.15.27:0/2146381746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd23c104740 msgr2=0x7fd23c102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd23a800640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd23c104740 0x7fd23c102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd23a800640 1 -- 172.21.15.27:0/2146381746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c1a85d0 con 0x7fd23c10dba0 2024-09-21T07:18:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd23a800640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c10dba0 0x7fd23c102ef0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd23400eb10 tx=0x7fd23400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd232a00640 1 -- 172.21.15.27:0/2146381746 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234004b20 con 0x7fd23c10dba0 2024-09-21T07:18:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd232a00640 1 -- 172.21.15.27:0/2146381746 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd234004cc0 con 0x7fd23c10dba0 2024-09-21T07:18:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23c1a88c0 con 0x7fd23c10dba0 2024-09-21T07:18:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.601+0000 7fd232a00640 1 -- 172.21.15.27:0/2146381746 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234010670 con 0x7fd23c10dba0 2024-09-21T07:18:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.602+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23c1a8e30 con 0x7fd23c10dba0 2024-09-21T07:18:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.605+0000 7fd247ed3740 1 Processor -- start 2024-09-21T07:18:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.607+0000 7fd232a00640 1 -- 172.21.15.27:0/2146381746 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd234005020 con 0x7fd23c10dba0 2024-09-21T07:18:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.607+0000 7fd247ed3740 1 -- start start 2024-09-21T07:18:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.607+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa352e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.607+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa353350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.607+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa353890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.607+0000 7fd247ed3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d1fa2c86b0 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd247ed3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d1fa2b7ae0 con 0x55d1f9fae1b0 2024-09-21T07:18:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd247ed3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d1fa2bb8f0 con 0x55d1f9f852f0 2024-09-21T07:18:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa353890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa353890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45032/0 (socket says 172.21.15.27:45032) 2024-09-21T07:18:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239400640 1 -- 172.21.15.27:0/3998918137 learned_addr learned my addr 172.21.15.27:0/3998918137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239e00640 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa353350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239400640 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa352e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239e00640 1 -- 172.21.15.27:0/3998918137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 msgr2=0x55d1fa352e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239e00640 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa352e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239e00640 1 -- 172.21.15.27:0/3998918137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 msgr2=0x55d1fa353890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.608+0000 7fd239e00640 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa353890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd239e00640 1 -- 172.21.15.27:0/3998918137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fa284780 con 0x55d1f9f852f0 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd239e00640 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa353350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009c20 tx=0x7fd22c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d6ad88f2db467cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd232a00640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2200b6730 0x7fd2200b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd227400640 1 -- 172.21.15.27:0/3998918137 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c01e070 con 0x55d1f9f852f0 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd227400640 1 -- 172.21.15.27:0/3998918137 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22c0035f0 con 0x55d1f9f852f0 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd227400640 1 -- 172.21.15.27:0/3998918137 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c019690 con 0x55d1f9f852f0 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd239400640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2200b6730 0x7fd2200b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.609+0000 7fd232a00640 1 -- 172.21.15.27:0/2146381746 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd2340e7f30 con 0x7fd23c10dba0 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.610+0000 7fd247ed3740 1 -- 172.21.15.27:0/3998918137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 msgr2=0x55d1fa353350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.610+0000 7fd239400640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2200b6730 0x7fd2200b8c40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd2280096c0 tx=0x7fd228000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.610+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa353350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009c20 tx=0x7fd22c011810 comp rx=0 tx=0).stop 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.611+0000 7fd247ed3740 1 -- 172.21.15.27:0/3998918137 shutdown_connections 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.611+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa353350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.611+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa353890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.611+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3998918137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa352e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.611+0000 7fd247ed3740 1 -- 172.21.15.27:0/3998918137 >> 172.21.15.27:0/3998918137 conn(0x55d1f9fa2300 msgr2=0x55d1fa350d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.611+0000 7fd247ed3740 1 -- 172.21.15.27:0/3998918137 shutdown_connections 2024-09-21T07:18:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.611+0000 7fd247ed3740 1 -- 172.21.15.27:0/3998918137 wait complete. 2024-09-21T07:18:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 -- start start 2024-09-21T07:18:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 Processor -- start 2024-09-21T07:18:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa35e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa35ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d1fa1efab0 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d1fa25e570 con 0x55d1f9fae1b0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.612+0000 7fd247ed3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d1fa361590 con 0x55d1f9f852f0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd247ed3740 10 client.0 _ll_register_callbacks cb 0x55d1fa26d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa35e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45042/0 (socket says 172.21.15.27:45042) 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 -- 172.21.15.27:0/155974009 learned_addr learned my addr 172.21.15.27:0/155974009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd23a800640 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa35ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 -- 172.21.15.27:0/155974009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 msgr2=0x55d1fa35e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa35e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 -- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 msgr2=0x55d1fa35ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa35ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239400640 1 -- 172.21.15.27:0/155974009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fa019790 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.613+0000 7fd239e00640 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa35e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd239400640 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35e900 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd228009230 tx=0x7fd22802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df03f19f7bcbebda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd21be00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd21be00640 1 -- 172.21.15.27:0/155974009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22804e070 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd21be00640 1 -- 172.21.15.27:0/155974009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd228005020 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd22c0035f0 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd21be00640 1 -- 172.21.15.27:0/155974009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd22805b050 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd247ed3740 10 client.16950 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.614+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1fa092620 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd21be00640 1 -- 172.21.15.27:0/155974009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22c0035f0 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd21be00640 1 -- 172.21.15.27:0/155974009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd2280035f0 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd247ed3740 10 client.16950 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd247ed3740 10 client.16950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd247ed3740 20 client.16950 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd247ed3740 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f6cd80 0x55d1fa3705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd247ed3740 4 client.16950 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.615+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1fa2ca600 con 0x55d1f9f6cd80 2024-09-21T07:18:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.616+0000 7fd239e00640 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f6cd80 0x55d1fa3705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.616+0000 7fd21be00640 10 client.16950 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.616+0000 7fd239e00640 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f6cd80 0x55d1fa3705b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b4c3271e1eb8ae server_cookie=a0977075ea555461 in_seq=0 out_seq=0 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.617+0000 7fd21be00640 1 -- 172.21.15.27:0/155974009 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1fa2ca600 con 0x55d1f9f6cd80 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.617+0000 7fd21be00640 10 client.16950 handle_command_reply: tid=0 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.617+0000 7fd247ed3740 1 client.16950 shutdown 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.617+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 msgr2=0x55d1fa35e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.617+0000 7fd247ed3740 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35e900 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd228009230 tx=0x7fd22802a410 comp rx=0 tx=0).stop 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 shutdown_connections 2024-09-21T07:18:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f6cd80 0x55d1fa3705b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9fae1b0 0x55d1fa35ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35e900 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd21be00640 0 client.16950 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 --2- 172.21.15.27:0/155974009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f852f0 0x55d1fa35e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 >> 172.21.15.27:0/155974009 conn(0x55d1f9fa2300 msgr2=0x55d1fa353e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 shutdown_connections 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 1 -- 172.21.15.27:0/155974009 wait complete. 2024-09-21T07:18:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.618+0000 7fd247ed3740 20 client.16950 trim_cache size 0 max 16384 2024-09-21T07:18:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.640+0000 7fd247ed3740 1 Processor -- start 2024-09-21T07:18:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.640+0000 7fd247ed3740 1 -- start start 2024-09-21T07:18:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.640+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa354800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.640+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.640+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9fae1b0 0x55d1fa375d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.640+0000 7fd247ed3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1b56f0 con 0x55d1f9fae1b0 2024-09-21T07:18:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd247ed3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1b2fc0 con 0x55d1f9f852f0 2024-09-21T07:18:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd247ed3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1b0e10 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa354800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd23a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9fae1b0 0x55d1fa375d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa354800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34544/0 (socket says 172.21.15.27:34544) 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 -- 172.21.15.27:0/3894000685 learned_addr learned my addr 172.21.15.27:0/3894000685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239400640 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 -- 172.21.15.27:0/3894000685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f8d4d0 msgr2=0x55d1fa35fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 -- 172.21.15.27:0/3894000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9fae1b0 msgr2=0x55d1fa375d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9fae1b0 0x55d1fa375d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd239e00640 1 -- 172.21.15.27:0/3894000685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fa36aee0 con 0x55d1f9f852f0 2024-09-21T07:18:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.641+0000 7fd23a800640 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9fae1b0 0x55d1fa375d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.642+0000 7fd239e00640 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa354800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55d1fa351a90 tx=0x7fd22c0093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4434cd4ebc7a89b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.642+0000 7fd226a00640 1 -- 172.21.15.27:0/3894000685 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c01e040 con 0x55d1f9f852f0 2024-09-21T07:18:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.642+0000 7fd226a00640 1 -- 172.21.15.27:0/3894000685 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22c0054d0 con 0x55d1f9f852f0 2024-09-21T07:18:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.642+0000 7fd226a00640 1 -- 172.21.15.27:0/3894000685 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c003850 con 0x55d1f9f852f0 2024-09-21T07:18:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 -- 172.21.15.27:0/3894000685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 msgr2=0x55d1fa354800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa354800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55d1fa351a90 tx=0x7fd22c0093b0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 -- 172.21.15.27:0/3894000685 shutdown_connections 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa35fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9fae1b0 0x55d1fa375d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 --2- 172.21.15.27:0/3894000685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa354800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 -- 172.21.15.27:0/3894000685 >> 172.21.15.27:0/3894000685 conn(0x55d1f9fa2300 msgr2=0x55d1fa35f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 -- 172.21.15.27:0/3894000685 shutdown_connections 2024-09-21T07:18:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.643+0000 7fd247ed3740 1 -- 172.21.15.27:0/3894000685 wait complete. 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.644+0000 7fd247ed3740 1 -- start start 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.644+0000 7fd247ed3740 1 Processor -- start 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa367250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa364790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd247ed3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9fae1b0 0x55d1fa364ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd247ed3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1b1480 con 0x55d1f9f8d4d0 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd247ed3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1b28f0 con 0x55d1f9f852f0 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd247ed3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1b1b50 con 0x55d1f9fae1b0 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd247ed3740 10 client.0 _ll_register_callbacks cb 0x7fd228032dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa364790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa367250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa364790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45062/0 (socket says 172.21.15.27:45062) 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239400640 1 -- 172.21.15.27:0/1419598088 learned_addr learned my addr 172.21.15.27:0/1419598088 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd23a800640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9fae1b0 0x55d1fa364ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239e00640 1 -- 172.21.15.27:0/1419598088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9fae1b0 msgr2=0x55d1fa364ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239e00640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9fae1b0 0x55d1fa364ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239e00640 1 -- 172.21.15.27:0/1419598088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 msgr2=0x55d1fa364790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239e00640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa364790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239e00640 1 -- 172.21.15.27:0/1419598088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fa3e2790 con 0x55d1f9f852f0 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd23a800640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9fae1b0 0x55d1fa364ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.645+0000 7fd239400640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa364790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.646+0000 7fd239e00640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa367250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009440 tx=0x7fd22c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7802b67014fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.646+0000 7fd21be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.646+0000 7fd21be00640 1 -- 172.21.15.27:0/1419598088 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c01e070 con 0x55d1f9f852f0 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.646+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd22c0054d0 con 0x55d1f9f852f0 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.646+0000 7fd21be00640 1 -- 172.21.15.27:0/1419598088 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd22c004500 con 0x55d1f9f852f0 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.646+0000 7fd21be00640 1 -- 172.21.15.27:0/1419598088 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd22c004700 con 0x55d1f9f852f0 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.646+0000 7fd247ed3740 10 client.26990 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1fa3e3930 con 0x55d1f9f852f0 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd21be00640 1 -- 172.21.15.27:0/1419598088 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22c0054d0 con 0x55d1f9f852f0 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd21be00640 1 -- 172.21.15.27:0/1419598088 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd22c003aa0 con 0x55d1f9f852f0 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd247ed3740 10 client.26990 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd247ed3740 10 client.26990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd247ed3740 20 client.26990 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd247ed3740 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f421c0 0x55d1fa3e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd247ed3740 4 client.26990 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:18:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1fa2c60b0 con 0x55d1f9f421c0 2024-09-21T07:18:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.647+0000 7fd239400640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f421c0 0x55d1fa3e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.648+0000 7fd239400640 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f421c0 0x55d1fa3e5df0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c39a43ca7b71c6d server_cookie=d0227eb150d0d9cb in_seq=0 out_seq=0 2024-09-21T07:18:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.648+0000 7fd21be00640 10 client.26990 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd21be00640 1 -- 172.21.15.27:0/1419598088 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1fa2c60b0 con 0x55d1f9f421c0 2024-09-21T07:18:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd21be00640 10 client.26990 handle_command_reply: tid=0 2024-09-21T07:18:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 client.26990 shutdown 2024-09-21T07:18:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 msgr2=0x55d1fa367250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa367250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009440 tx=0x7fd22c005d30 comp rx=0 tx=0).stop 2024-09-21T07:18:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 shutdown_connections 2024-09-21T07:18:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1f9f421c0 0x55d1fa3e5df0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1f9fae1b0 0x55d1fa364ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1f9f8d4d0 0x55d1fa364790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 --2- 172.21.15.27:0/1419598088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1f9f852f0 0x55d1fa367250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd21be00640 0 client.26990 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 >> 172.21.15.27:0/1419598088 conn(0x55d1f9fa2300 msgr2=0x55d1fa360230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 shutdown_connections 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 1 -- 172.21.15.27:0/1419598088 wait complete. 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.662+0000 7fd247ed3740 20 client.26990 trim_cache size 0 max 16384 2024-09-21T07:18:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:18:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:18:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:18:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:18:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2200b6730 msgr2=0x7fd2200b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2200b6730 0x7fd2200b8c40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd2280096c0 tx=0x7fd228000bc0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c10dba0 msgr2=0x7fd23c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c10dba0 0x7fd23c102ef0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd23400eb10 tx=0x7fd23400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 shutdown_connections 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105090 0x7fd23c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd2200b6730 0x7fd2200b8c40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd23c10dba0 0x7fd23c102ef0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 --2- 172.21.15.27:0/2146381746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd23c104740 0x7fd23c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.663+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 >> 172.21.15.27:0/2146381746 conn(0x7fd23c0fe230 msgr2=0x7fd23c1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.664+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 shutdown_connections 2024-09-21T07:18:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:16.664+0000 7fd23b200640 1 -- 172.21.15.27:0/2146381746 wait complete. 2024-09-21T07:18:17.328 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 -- 172.21.15.27:0/1935418502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 msgr2=0x7f4fd8103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/1935418502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8103f90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009900 tx=0x7f4fcc02a400 comp rx=0 tx=0).stop 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 -- 172.21.15.27:0/1935418502 shutdown_connections 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/1935418502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd8109e10 0x7f4fd8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/1935418502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd81094c0 0x7f4fd81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/1935418502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8103f90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 -- 172.21.15.27:0/1935418502 >> 172.21.15.27:0/1935418502 conn(0x7f4fd80ff590 msgr2=0x7f4fd81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.337+0000 7f4fdfa00640 1 -- 172.21.15.27:0/1935418502 shutdown_connections 2024-09-21T07:18:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.338+0000 7f4fdfa00640 1 -- 172.21.15.27:0/1935418502 wait complete. 2024-09-21T07:18:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 Processor -- start 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 -- start start 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd81094c0 0x7f4fd8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8109e10 0x7f4fd8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8113b20 con 0x7f4fd8103b80 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81139a0 con 0x7f4fd8109e10 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.339+0000 7f4fdfa00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8113ca0 con 0x7f4fd81094c0 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45080/0 (socket says 172.21.15.27:45080) 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fd7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd81094c0 0x7f4fd8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 -- 172.21.15.27:0/445525668 learned_addr learned my addr 172.21.15.27:0/445525668 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdda00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8109e10 0x7f4fd8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 -- 172.21.15.27:0/445525668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd81094c0 msgr2=0x7f4fd8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd81094c0 0x7f4fd8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 -- 172.21.15.27:0/445525668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8109e10 msgr2=0x7f4fd8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8109e10 0x7f4fd8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.340+0000 7f4fdd000640 1 -- 172.21.15.27:0/445525668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81a8790 con 0x7f4fd8103b80 2024-09-21T07:18:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.341+0000 7f4fdd000640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005ab0 tx=0x7f4fcc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.341+0000 7f4fd5600640 1 -- 172.21.15.27:0/445525668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc03f070 con 0x7f4fd8103b80 2024-09-21T07:18:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.341+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81a8a20 con 0x7f4fd8103b80 2024-09-21T07:18:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.341+0000 7f4fd5600640 1 -- 172.21.15.27:0/445525668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc031b60 con 0x7f4fd8103b80 2024-09-21T07:18:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.341+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a8f50 con 0x7f4fd8103b80 2024-09-21T07:18:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.342+0000 7f4fd5600640 1 -- 172.21.15.27:0/445525668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc049060 con 0x7f4fd8103b80 2024-09-21T07:18:17.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.345+0000 7f4fd5600640 1 -- 172.21.15.27:0/445525668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4fcc042070 con 0x7f4fd8103b80 2024-09-21T07:18:17.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.346+0000 7f4fe668e740 1 Processor -- start 2024-09-21T07:18:17.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.346+0000 7f4fe668e740 1 -- start start 2024-09-21T07:18:17.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e383a070 0x5588e3be0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3be12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e3819390 0x5588e3be1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fe668e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588e3b57fb0 con 0x5588e38111b0 2024-09-21T07:18:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fe668e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5588e3b473e0 con 0x5588e383a070 2024-09-21T07:18:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fe668e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5588e3b4b1f0 con 0x5588e3819390 2024-09-21T07:18:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fdda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e383a070 0x5588e3be0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fdda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e383a070 0x5588e3be0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34578/0 (socket says 172.21.15.27:34578) 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fdda00640 1 -- 172.21.15.27:0/3935101389 learned_addr learned my addr 172.21.15.27:0/3935101389 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fdd000640 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3be12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.347+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e3819390 0x5588e3be1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdda00640 1 -- 172.21.15.27:0/3935101389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e3819390 msgr2=0x5588e3be1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdda00640 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e3819390 0x5588e3be1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdda00640 1 -- 172.21.15.27:0/3935101389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 msgr2=0x5588e3be12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdda00640 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3be12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdda00640 1 -- 172.21.15.27:0/3935101389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e3b11cd0 con 0x5588e383a070 2024-09-21T07:18:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fd5600640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc00b6680 0x7f4fc00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdd000640 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3be12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fd5600640 1 -- 172.21.15.27:0/445525668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4fcc1085b0 con 0x7f4fd8103b80 2024-09-21T07:18:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdda00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc00b6680 0x7f4fc00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.348+0000 7f4fdda00640 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e383a070 0x5588e3be0da0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009fd0 tx=0x7f4fd00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a44f5118199e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.349+0000 7f4fca000640 1 -- 172.21.15.27:0/3935101389 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0004250 con 0x5588e383a070 2024-09-21T07:18:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.349+0000 7f4fca000640 1 -- 172.21.15.27:0/3935101389 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd000fe60 con 0x5588e383a070 2024-09-21T07:18:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.349+0000 7f4fca000640 1 -- 172.21.15.27:0/3935101389 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd001bae0 con 0x5588e383a070 2024-09-21T07:18:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 -- 172.21.15.27:0/3935101389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e383a070 msgr2=0x5588e3be0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e383a070 0x5588e3be0da0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009fd0 tx=0x7f4fd00118d0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fdda00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc00b6680 0x7f4fc00b8b90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0020000 tx=0x7f4fd0021000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 -- 172.21.15.27:0/3935101389 shutdown_connections 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e3819390 0x5588e3be1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3be12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 --2- 172.21.15.27:0/3935101389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e383a070 0x5588e3be0da0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 -- 172.21.15.27:0/3935101389 >> 172.21.15.27:0/3935101389 conn(0x5588e382e460 msgr2=0x5588e3bdec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 -- 172.21.15.27:0/3935101389 shutdown_connections 2024-09-21T07:18:17.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.350+0000 7f4fe668e740 1 -- 172.21.15.27:0/3935101389 wait complete. 2024-09-21T07:18:17.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.351+0000 7f4fe668e740 1 -- start start 2024-09-21T07:18:17.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.351+0000 7f4fe668e740 1 Processor -- start 2024-09-21T07:18:17.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e38111b0 0x5588e3bec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3becb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e383a070 0x5588e3bf88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fe668e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588e39a8700 con 0x5588e383a070 2024-09-21T07:18:17.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fe668e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5588e3ae2580 con 0x5588e3819390 2024-09-21T07:18:17.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fe668e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5588e3bef580 con 0x5588e38111b0 2024-09-21T07:18:17.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fe668e740 10 client.0 _ll_register_callbacks cb 0x5588e3af2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:17.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fdda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e383a070 0x5588e3bf88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fdda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e383a070 0x5588e3bf88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45106/0 (socket says 172.21.15.27:45106) 2024-09-21T07:18:17.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fdda00640 1 -- 172.21.15.27:0/1371494330 learned_addr learned my addr 172.21.15.27:0/1371494330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:17.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.352+0000 7f4fdd000640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e38111b0 0x5588e3bec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3becb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fdd000640 1 -- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 msgr2=0x5588e3becb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fdd000640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3becb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fdd000640 1 -- 172.21.15.27:0/1371494330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e383a070 msgr2=0x5588e3bf88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fdd000640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e383a070 0x5588e3bf88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fdd000640 1 -- 172.21.15.27:0/1371494330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e38a5650 con 0x5588e38111b0 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fdda00640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e383a070 0x5588e3bf88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3becb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fdd000640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e38111b0 0x5588e3bec5e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc105950 tx=0x7f4fcc0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c487d860531a276 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.353+0000 7f4fbea00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.354+0000 7f4fbea00640 1 -- 172.21.15.27:0/1371494330 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0cdd60 con 0x5588e38111b0 2024-09-21T07:18:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.354+0000 7f4fbea00640 1 -- 172.21.15.27:0/1371494330 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4fcc03b460 con 0x5588e38111b0 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.354+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588e3ae4b90 con 0x5588e38111b0 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.354+0000 7f4fbea00640 1 -- 172.21.15.27:0/1371494330 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc0d7a70 con 0x5588e38111b0 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.354+0000 7f4fe668e740 10 client.26989 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.354+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588e3aff950 con 0x5588e38111b0 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.354+0000 7f4fbea00640 1 -- 172.21.15.27:0/1371494330 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5588e3ae4b90 con 0x5588e38111b0 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fbea00640 1 -- 172.21.15.27:0/1371494330 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4fcc108230 con 0x5588e38111b0 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fe668e740 10 client.26989 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fe668e740 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fe668e740 20 client.26989 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37f89f0 0x5588e3bfe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fe668e740 4 client.26989 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588e3b59f00 con 0x5588e37f89f0 2024-09-21T07:18:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.355+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37f89f0 0x5588e3bfe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.356+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37f89f0 0x5588e3bfe710 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4f2bc4f24b444b server_cookie=fc1a101535c77b72 in_seq=0 out_seq=0 2024-09-21T07:18:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.356+0000 7f4fbea00640 10 client.26989 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fbea00640 1 -- 172.21.15.27:0/1371494330 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588e3b59f00 con 0x5588e37f89f0 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fbea00640 10 client.26989 handle_command_reply: tid=0 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 client.26989 shutdown 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e38111b0 msgr2=0x5588e3bec5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e38111b0 0x5588e3bec5e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc105950 tx=0x7f4fcc0045e0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 shutdown_connections 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37f89f0 0x5588e3bfe710 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e383a070 0x5588e3bf88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3becb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1371494330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e38111b0 0x5588e3bec5e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fbea00640 0 client.26989 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.357+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 >> 172.21.15.27:0/1371494330 conn(0x5588e382e460 msgr2=0x5588e3be3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.358+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 shutdown_connections 2024-09-21T07:18:17.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.358+0000 7f4fe668e740 1 -- 172.21.15.27:0/1371494330 wait complete. 2024-09-21T07:18:17.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.358+0000 7f4fe668e740 20 client.26989 trim_cache size 0 max 16384 2024-09-21T07:18:17.376 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:17 smithi027 ceph-mon[30779]: pgmap v535: 129 pgs: 129 active+clean; 1.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-21T07:18:17.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.373+0000 7f4fe668e740 1 Processor -- start 2024-09-21T07:18:17.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fe668e740 1 -- start start 2024-09-21T07:18:17.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e38111b0 0x5588e3bee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e3819390 0x5588e3be97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3be9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fe668e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b58a0 con 0x5588e3819390 2024-09-21T07:18:17.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fe668e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b3170 con 0x5588e38111b0 2024-09-21T07:18:17.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fe668e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b0fc0 con 0x5588e383a070 2024-09-21T07:18:17.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fd7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e3819390 0x5588e3be97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.374+0000 7f4fdd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e38111b0 0x5588e3bee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e3819390 0x5588e3be97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45108/0 (socket says 172.21.15.27:45108) 2024-09-21T07:18:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fdda00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3be9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 -- 172.21.15.27:0/1282263742 learned_addr learned my addr 172.21.15.27:0/1282263742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 -- 172.21.15.27:0/1282263742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 msgr2=0x5588e3be9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3be9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 -- 172.21.15.27:0/1282263742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e38111b0 msgr2=0x5588e3bee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e38111b0 0x5588e3bee210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 -- 172.21.15.27:0/1282263742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e3becfa0 con 0x5588e3819390 2024-09-21T07:18:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fdda00640 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3be9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:18:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.375+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e3819390 0x5588e3be97d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8101d60 tx=0x7f4fc4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbaa82b100602ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.376+0000 7f4fc9600640 1 -- 172.21.15.27:0/1282263742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4020040 con 0x5588e3819390 2024-09-21T07:18:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.376+0000 7f4fc9600640 1 -- 172.21.15.27:0/1282263742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fc40054d0 con 0x5588e3819390 2024-09-21T07:18:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.376+0000 7f4fc9600640 1 -- 172.21.15.27:0/1282263742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4004dd0 con 0x5588e3819390 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 -- 172.21.15.27:0/1282263742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e3819390 msgr2=0x5588e3be97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e3819390 0x5588e3be97d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8101d60 tx=0x7f4fc4005f90 comp rx=0 tx=0).stop 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 -- 172.21.15.27:0/1282263742 shutdown_connections 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3be9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e3819390 0x5588e3be97d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 --2- 172.21.15.27:0/1282263742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e38111b0 0x5588e3bee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 -- 172.21.15.27:0/1282263742 >> 172.21.15.27:0/1282263742 conn(0x5588e382e460 msgr2=0x5588e3bedb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 -- 172.21.15.27:0/1282263742 shutdown_connections 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.377+0000 7f4fe668e740 1 -- 172.21.15.27:0/1282263742 wait complete. 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.378+0000 7f4fe668e740 1 -- start start 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.378+0000 7f4fe668e740 1 Processor -- start 2024-09-21T07:18:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3bf1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3bf15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fe668e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3c67fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fe668e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b1630 con 0x5588e38111b0 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fe668e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b2aa0 con 0x5588e3819390 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fe668e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b1d00 con 0x5588e383a070 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fe668e740 10 client.0 _ll_register_callbacks cb 0x7f4fcc0ca990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fdd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3bf1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fd7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3bf15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fdd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3bf1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45112/0 (socket says 172.21.15.27:45112) 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fdd000640 1 -- 172.21.15.27:0/241529771 learned_addr learned my addr 172.21.15.27:0/241529771 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.379+0000 7f4fdda00640 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3c67fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.380+0000 7f4fd7e00640 1 -- 172.21.15.27:0/241529771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 msgr2=0x5588e3c67fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.380+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3c67fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.380+0000 7f4fd7e00640 1 -- 172.21.15.27:0/241529771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 msgr2=0x5588e3bf1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.380+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3bf1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.380+0000 7f4fd7e00640 1 -- 172.21.15.27:0/241529771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e3c685b0 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.380+0000 7f4fd7e00640 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3bf15a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5588e3acd3c0 tx=0x7f4fc4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba83a18145bb29b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.380+0000 7f4fbea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.381+0000 7f4fbea00640 1 -- 172.21.15.27:0/241529771 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4020050 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.381+0000 7f4fbea00640 1 -- 172.21.15.27:0/241529771 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f4fc4004310 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.381+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc40054d0 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.381+0000 7f4fbea00640 1 -- 172.21.15.27:0/241529771 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fc4036050 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.381+0000 7f4fe668e740 10 client.27008 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.381+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588e3c697d0 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.381+0000 7f4fbea00640 1 -- 172.21.15.27:0/241529771 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fc40054d0 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fbea00640 1 -- 172.21.15.27:0/241529771 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4fc4004b90 con 0x5588e3819390 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fe668e740 10 client.27008 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fe668e740 10 client.27008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fe668e740 20 client.27008 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fe668e740 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37cdda0 0x5588e3c6bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fe668e740 4 client.27008 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:18:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588e3b559b0 con 0x5588e37cdda0 2024-09-21T07:18:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.382+0000 7f4fdd000640 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37cdda0 0x5588e3c6bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.383+0000 7f4fdd000640 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37cdda0 0x5588e3c6bf60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175526485134840b server_cookie=16b418a938ae3670 in_seq=0 out_seq=0 2024-09-21T07:18:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.383+0000 7f4fbea00640 10 client.27008 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.383+0000 7f4fbea00640 1 -- 172.21.15.27:0/241529771 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588e3b559b0 con 0x5588e37cdda0 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.383+0000 7f4fbea00640 10 client.27008 handle_command_reply: tid=0 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.383+0000 7f4fe668e740 1 client.27008 shutdown 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.383+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 msgr2=0x5588e3bf15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.383+0000 7f4fe668e740 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3bf15a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5588e3acd3c0 tx=0x7f4fc4004730 comp rx=0 tx=0).stop 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 shutdown_connections 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5588e37cdda0 0x5588e3c6bf60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588e383a070 0x5588e3c67fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588e3819390 0x5588e3bf15a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 --2- 172.21.15.27:0/241529771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5588e38111b0 0x5588e3bf1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fbea00640 0 client.27008 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 >> 172.21.15.27:0/241529771 conn(0x5588e382e460 msgr2=0x5588e3be1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 shutdown_connections 2024-09-21T07:18:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 1 -- 172.21.15.27:0/241529771 wait complete. 2024-09-21T07:18:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.384+0000 7f4fe668e740 20 client.27008 trim_cache size 0 max 16384 2024-09-21T07:18:17.390 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:18:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc00b6680 msgr2=0x7f4fc00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc00b6680 0x7f4fc00b8b90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0020000 tx=0x7f4fd0021000 comp rx=0 tx=0).stop 2024-09-21T07:18:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 msgr2=0x7f4fd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005ab0 tx=0x7f4fcc003c10 comp rx=0 tx=0).stop 2024-09-21T07:18:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 shutdown_connections 2024-09-21T07:18:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd81094c0 0x7f4fd8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fc00b6680 0x7f4fc00b8b90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8109e10 0x7f4fd8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 --2- 172.21.15.27:0/445525668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fd8103b80 0x7f4fd8075710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.385+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 >> 172.21.15.27:0/445525668 conn(0x7f4fd80ff590 msgr2=0x7f4fd81029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.386+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 shutdown_connections 2024-09-21T07:18:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:17.386+0000 7f4fdfa00640 1 -- 172.21.15.27:0/445525668 wait complete. 2024-09-21T07:18:17.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:17 smithi153 ceph-mon[34747]: pgmap v535: 129 pgs: 129 active+clean; 1.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-21T07:18:17.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:17 smithi119 ceph-mon[34519]: pgmap v535: 129 pgs: 129 active+clean; 1.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.288+0000 7ff499600640 1 -- 172.21.15.27:0/1299166216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494114280 msgr2=0x7ff494116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.288+0000 7ff499600640 1 --2- 172.21.15.27:0/1299166216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494114280 0x7ff494116750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009900 tx=0x7ff48c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.289+0000 7ff499600640 1 -- 172.21.15.27:0/1299166216 shutdown_connections 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.289+0000 7ff499600640 1 --2- 172.21.15.27:0/1299166216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff494075dd0 0x7ff4940761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.289+0000 7ff499600640 1 --2- 172.21.15.27:0/1299166216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494114280 0x7ff494116750 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.289+0000 7ff499600640 1 --2- 172.21.15.27:0/1299166216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff494077380 0x7ff4940757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.289+0000 7ff499600640 1 -- 172.21.15.27:0/1299166216 >> 172.21.15.27:0/1299166216 conn(0x7ff4940ff440 msgr2=0x7ff494101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:18.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.289+0000 7ff499600640 1 -- 172.21.15.27:0/1299166216 shutdown_connections 2024-09-21T07:18:18.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.289+0000 7ff499600640 1 -- 172.21.15.27:0/1299166216 wait complete. 2024-09-21T07:18:18.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.290+0000 7ff499600640 1 Processor -- start 2024-09-21T07:18:18.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.290+0000 7ff499600640 1 -- start start 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff499600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494075dd0 0x7ff4941a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff499600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff494077380 0x7ff4941a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff499600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff494114280 0x7ff4941a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff499600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff494119ec0 con 0x7ff494075dd0 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff499600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff494119d40 con 0x7ff494114280 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff499600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff49411a040 con 0x7ff494077380 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff493e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494075dd0 0x7ff4941a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff493e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494075dd0 0x7ff4941a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44992/0 (socket says 172.21.15.27:44992) 2024-09-21T07:18:18.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff493e00640 1 -- 172.21.15.27:0/1534101089 learned_addr learned my addr 172.21.15.27:0/1534101089 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:18.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.291+0000 7ff48b400640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff494077380 0x7ff4941a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff498c00640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff494114280 0x7ff4941a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff48b400640 1 -- 172.21.15.27:0/1534101089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff494114280 msgr2=0x7ff4941a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff48b400640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff494114280 0x7ff4941a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff48b400640 1 -- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494075dd0 msgr2=0x7ff4941a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff48b400640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494075dd0 0x7ff4941a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff48b400640 1 -- 172.21.15.27:0/1534101089 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4941a6800 con 0x7ff494077380 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff493e00640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494075dd0 0x7ff4941a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff48b400640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff494077380 0x7ff4941a0e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00e9c0 tx=0x7ff47c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.292+0000 7ff491600640 1 -- 172.21.15.27:0/1534101089 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00ccc0 con 0x7ff494077380 2024-09-21T07:18:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.293+0000 7ff491600640 1 -- 172.21.15.27:0/1534101089 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff47c005020 con 0x7ff494077380 2024-09-21T07:18:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.293+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4941a6af0 con 0x7ff494077380 2024-09-21T07:18:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.293+0000 7ff491600640 1 -- 172.21.15.27:0/1534101089 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c004a00 con 0x7ff494077380 2024-09-21T07:18:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.293+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4941ad250 con 0x7ff494077380 2024-09-21T07:18:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 Processor -- start 2024-09-21T07:18:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 -- start start 2024-09-21T07:18:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da79d520 0x55f5dab44340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab44880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab44dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5daabb290 con 0x55f5da79d520 2024-09-21T07:18:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5daaaa6c0 con 0x55f5da77c840 2024-09-21T07:18:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.296+0000 7ff4a19df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5daaae4d0 con 0x55f5da774660 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff491600640 1 -- 172.21.15.27:0/1534101089 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff47c0051d0 con 0x7ff494077380 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab44880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff498c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da79d520 0x55f5dab44340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab44880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36066/0 (socket says 172.21.15.27:36066) 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 -- 172.21.15.27:0/1523658244 learned_addr learned my addr 172.21.15.27:0/1523658244 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff48b400640 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab44dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 -- 172.21.15.27:0/1523658244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 msgr2=0x55f5dab44dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab44dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 -- 172.21.15.27:0/1523658244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da79d520 msgr2=0x55f5dab44340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da79d520 0x55f5dab44340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff493e00640 1 -- 172.21.15.27:0/1523658244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5daa37460 con 0x55f5da774660 2024-09-21T07:18:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff498c00640 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da79d520 0x55f5dab44340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.297+0000 7ff48b400640 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab44dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:18:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.298+0000 7ff493e00640 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab44880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff48400f8d0 tx=0x7ff48400fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aae08c4c4cbb4f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.298+0000 7ff489200640 1 -- 172.21.15.27:0/1523658244 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484011c20 con 0x55f5da774660 2024-09-21T07:18:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.298+0000 7ff489200640 1 -- 172.21.15.27:0/1523658244 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff484011dc0 con 0x55f5da774660 2024-09-21T07:18:18.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.298+0000 7ff489200640 1 -- 172.21.15.27:0/1523658244 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484018700 con 0x55f5da774660 2024-09-21T07:18:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 -- 172.21.15.27:0/1523658244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 msgr2=0x55f5dab44880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab44880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff48400f8d0 tx=0x7ff48400fd90 comp rx=0 tx=0).stop 2024-09-21T07:18:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 -- 172.21.15.27:0/1523658244 shutdown_connections 2024-09-21T07:18:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab44880 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab44dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1523658244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da79d520 0x55f5dab44340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 -- 172.21.15.27:0/1523658244 >> 172.21.15.27:0/1523658244 conn(0x55f5da791910 msgr2=0x55f5dab42230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff491600640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff4780b6700 0x7ff4780b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 -- 172.21.15.27:0/1523658244 shutdown_connections 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff4a19df740 1 -- 172.21.15.27:0/1523658244 wait complete. 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff491600640 1 -- 172.21.15.27:0/1534101089 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff47c0e7a10 con 0x7ff494077380 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.299+0000 7ff493e00640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff4780b6700 0x7ff4780b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.300+0000 7ff4a19df740 1 -- start start 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.300+0000 7ff4a19df740 1 Processor -- start 2024-09-21T07:18:18.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.300+0000 7ff493e00640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff4780b6700 0x7ff4780b8c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff48400f690 tx=0x7ff484000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da774660 0x55f5dab4fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da79d520 0x55f5dab5be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff4a19df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5da90bbb0 con 0x55f5da774660 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff4a19df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5daa456c0 con 0x55f5da77c840 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff4a19df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5dab52b20 con 0x55f5da79d520 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff4a19df740 10 client.0 _ll_register_callbacks cb 0x55f5daa60490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff493e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da774660 0x55f5dab4fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff498c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da79d520 0x55f5dab5be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff493e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da774660 0x55f5dab4fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45008/0 (socket says 172.21.15.27:45008) 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff498c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da79d520 0x55f5dab5be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36074/0 (socket says 172.21.15.27:36074) 2024-09-21T07:18:18.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff493e00640 1 -- 172.21.15.27:0/3997600744 learned_addr learned my addr 172.21.15.27:0/3997600744 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.301+0000 7ff48b400640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff48b400640 1 -- 172.21.15.27:0/3997600744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da79d520 msgr2=0x55f5dab5be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff48b400640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da79d520 0x55f5dab5be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff48b400640 1 -- 172.21.15.27:0/3997600744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da774660 msgr2=0x55f5dab4fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff48b400640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da774660 0x55f5dab4fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff48b400640 1 -- 172.21.15.27:0/3997600744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5da808b00 con 0x55f5da77c840 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff498c00640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da79d520 0x55f5dab5be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff493e00640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da774660 0x55f5dab4fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff48b400640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab503d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0e4db0 tx=0x7ff47c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=145784d2aa4472dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff481600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff481600640 1 -- 172.21.15.27:0/3997600744 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c01c040 con 0x55f5da77c840 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff481600640 1 -- 172.21.15.27:0/3997600744 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff47c0a7dc0 con 0x55f5da77c840 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff484011dc0 con 0x55f5da77c840 2024-09-21T07:18:18.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.302+0000 7ff481600640 1 -- 172.21.15.27:0/3997600744 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff47c0adbb0 con 0x55f5da77c840 2024-09-21T07:18:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 10 client.27017 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5da87f960 con 0x55f5da77c840 2024-09-21T07:18:18.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff481600640 1 -- 172.21.15.27:0/3997600744 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff484011dc0 con 0x55f5da77c840 2024-09-21T07:18:18.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff481600640 1 -- 172.21.15.27:0/3997600744 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff47c0a8e00 con 0x55f5da77c840 2024-09-21T07:18:18.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 10 client.27017 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:18.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 10 client.27017 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:18:18.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 20 client.27017 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:18.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da75bea0 0x55f5dab61d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 4 client.27017 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.303+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5daabd1e0 con 0x55f5da75bea0 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.304+0000 7ff493e00640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da75bea0 0x55f5dab61d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.304+0000 7ff493e00640 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da75bea0 0x55f5dab61d50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac3d0c5f0721587 server_cookie=f1e8c6b60afc5ce2 in_seq=0 out_seq=0 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.304+0000 7ff481600640 10 client.27017 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.305+0000 7ff481600640 1 -- 172.21.15.27:0/3997600744 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5daabd1e0 con 0x55f5da75bea0 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.305+0000 7ff481600640 10 client.27017 handle_command_reply: tid=0 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.305+0000 7ff4a19df740 1 client.27017 shutdown 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 msgr2=0x55f5dab503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab503d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0e4db0 tx=0x7ff47c005720 comp rx=0 tx=0).stop 2024-09-21T07:18:18.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 shutdown_connections 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da75bea0 0x55f5dab61d50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da79d520 0x55f5dab5be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da77c840 0x55f5dab503d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 --2- 172.21.15.27:0/3997600744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da774660 0x55f5dab4fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff481600640 0 client.27017 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 >> 172.21.15.27:0/3997600744 conn(0x55f5da791910 msgr2=0x55f5dab47520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 shutdown_connections 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 1 -- 172.21.15.27:0/3997600744 wait complete. 2024-09-21T07:18:18.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.306+0000 7ff4a19df740 20 client.27017 trim_cache size 0 max 16384 2024-09-21T07:18:18.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.324+0000 7ff4a19df740 1 Processor -- start 2024-09-21T07:18:18.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.324+0000 7ff4a19df740 1 -- start start 2024-09-21T07:18:18.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab51b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab49420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dab49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff4a19df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b9a50 con 0x55f5da77c840 2024-09-21T07:18:18.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff4a19df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b7320 con 0x55f5da79d520 2024-09-21T07:18:18.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff4a19df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b5170 con 0x55f5da774660 2024-09-21T07:18:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff48b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab49420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff48b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab49420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45024/0 (socket says 172.21.15.27:45024) 2024-09-21T07:18:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff48b400640 1 -- 172.21.15.27:0/1955087543 learned_addr learned my addr 172.21.15.27:0/1955087543 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff493e00640 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab51b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.325+0000 7ff498c00640 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dab49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff48b400640 1 -- 172.21.15.27:0/1955087543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 msgr2=0x55f5dab51b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff48b400640 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab51b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff48b400640 1 -- 172.21.15.27:0/1955087543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 msgr2=0x55f5dab49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff48b400640 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dab49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff48b400640 1 -- 172.21.15.27:0/1955087543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5dab42750 con 0x55f5da77c840 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff48b400640 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab49420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0b0040 tx=0x7ff47c0e52b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f67c1c48c4739 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff483e00640 1 -- 172.21.15.27:0/1955087543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c01c070 con 0x55f5da77c840 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff483e00640 1 -- 172.21.15.27:0/1955087543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff47c0b1f90 con 0x55f5da77c840 2024-09-21T07:18:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.326+0000 7ff483e00640 1 -- 172.21.15.27:0/1955087543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c004e00 con 0x55f5da77c840 2024-09-21T07:18:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 -- 172.21.15.27:0/1955087543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 msgr2=0x55f5dab49420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab49420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0b0040 tx=0x7ff47c0e52b0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 -- 172.21.15.27:0/1955087543 shutdown_connections 2024-09-21T07:18:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dab49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab49420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 --2- 172.21.15.27:0/1955087543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab51b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 -- 172.21.15.27:0/1955087543 >> 172.21.15.27:0/1955087543 conn(0x55f5da791910 msgr2=0x55f5dab47ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 -- 172.21.15.27:0/1955087543 shutdown_connections 2024-09-21T07:18:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.327+0000 7ff4a19df740 1 -- 172.21.15.27:0/1955087543 wait complete. 2024-09-21T07:18:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.328+0000 7ff4a19df740 1 -- start start 2024-09-21T07:18:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.328+0000 7ff4a19df740 1 Processor -- start 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.328+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.328+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab54d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff4a19df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dabc5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff4a19df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b57e0 con 0x55f5da77c840 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff4a19df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b6c50 con 0x55f5da79d520 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff4a19df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b5eb0 con 0x55f5da774660 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff4a19df740 10 client.0 _ll_register_callbacks cb 0x7ff47c00c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:18.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff48b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab54d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff493e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff48b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab54d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45030/0 (socket says 172.21.15.27:45030) 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff493e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36104/0 (socket says 172.21.15.27:36104) 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff48b400640 1 -- 172.21.15.27:0/2558827791 learned_addr learned my addr 172.21.15.27:0/2558827791 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff498c00640 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dabc5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff493e00640 1 -- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 msgr2=0x55f5dabc5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff493e00640 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dabc5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff493e00640 1 -- 172.21.15.27:0/2558827791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 msgr2=0x55f5dab54d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff493e00640 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab54d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.329+0000 7ff493e00640 1 -- 172.21.15.27:0/2558827791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5dabc5e00 con 0x55f5da774660 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff493e00640 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab547a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f5dab62420 tx=0x7ff4840096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dfabc6438e3ac05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff481600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff481600640 1 -- 172.21.15.27:0/2558827791 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484019c70 con 0x55f5da774660 2024-09-21T07:18:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff47c0b1f90 con 0x55f5da774660 2024-09-21T07:18:18.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff481600640 1 -- 172.21.15.27:0/2558827791 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7ff484011c90 con 0x55f5da774660 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff481600640 1 -- 172.21.15.27:0/2558827791 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff484043050 con 0x55f5da774660 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff4a19df740 10 client.27010 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.330+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5dabcd230 con 0x55f5da774660 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.331+0000 7ff481600640 1 -- 172.21.15.27:0/2558827791 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff47c0b1f90 con 0x55f5da774660 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.331+0000 7ff481600640 1 -- 172.21.15.27:0/2558827791 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff484031770 con 0x55f5da774660 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff4a19df740 10 client.27010 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff4a19df740 10 client.27010 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff4a19df740 20 client.27010 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff4a19df740 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da731250 0x55f5dabcf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff4a19df740 4 client.27010 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fa29fba", "release": true, "await": true}] 2024-09-21T07:18:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fa29fba", "release": true, "await": true}) -- 0x55f5daab8c90 con 0x55f5da731250 2024-09-21T07:18:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff48b400640 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da731250 0x55f5dabcf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff48b400640 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da731250 0x55f5dabcf770 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12657ba6bf9c87cc server_cookie=caa8a56de16a7c2a in_seq=0 out_seq=0 2024-09-21T07:18:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.332+0000 7ff481600640 10 client.27010 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.335+0000 7ff481600640 1 -- 172.21.15.27:0/2558827791 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f5daab8c90 con 0x55f5da731250 2024-09-21T07:18:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.335+0000 7ff481600640 10 client.27010 handle_command_reply: tid=0 2024-09-21T07:18:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.335+0000 7ff4a19df740 1 client.27010 shutdown 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.335+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 msgr2=0x55f5dab547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.335+0000 7ff4a19df740 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab547a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f5dab62420 tx=0x7ff4840096c0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 shutdown_connections 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f5da731250 0x55f5dabcf770 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff481600640 0 client.27010 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5da79d520 0x55f5dabc5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5da77c840 0x55f5dab54d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 --2- 172.21.15.27:0/2558827791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5da774660 0x55f5dab547a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 >> 172.21.15.27:0/2558827791 conn(0x55f5da791910 msgr2=0x55f5dab47ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 shutdown_connections 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 1 -- 172.21.15.27:0/2558827791 wait complete. 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.336+0000 7ff4a19df740 20 client.27010 trim_cache size 0 max 16384 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 20, 2024-09-21T07:18:18.339 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "fa29fba": { 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 20, 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 130.0, 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:18:18.340 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:18:18.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:18.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:18.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:18.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:18.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:18.341 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff4780b6700 msgr2=0x7ff4780b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff4780b6700 0x7ff4780b8c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff48400f690 tx=0x7ff484000f40 comp rx=0 tx=0).stop 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff494077380 msgr2=0x7ff4941a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff494077380 0x7ff4941a0e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00e9c0 tx=0x7ff47c00ee80 comp rx=0 tx=0).stop 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 shutdown_connections 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff494077380 0x7ff4941a0e30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff4780b6700 0x7ff4780b8c10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff494114280 0x7ff4941a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 --2- 172.21.15.27:0/1534101089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff494075dd0 0x7ff4941a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 >> 172.21.15.27:0/1534101089 conn(0x7ff4940ff440 msgr2=0x7ff494110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 shutdown_connections 2024-09-21T07:18:18.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:18.337+0000 7ff499600640 1 -- 172.21.15.27:0/1534101089 wait complete. 2024-09-21T07:18:19.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:19 smithi027 ceph-mon[30779]: pgmap v536: 129 pgs: 129 active+clean; 1.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:18:19.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:19 smithi153 ceph-mon[34747]: pgmap v536: 129 pgs: 129 active+clean; 1.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:18:19.518 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fa29fba', total seconds elapsed: 17.3 2024-09-21T07:18:19.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:19 smithi119 ceph-mon[34519]: pgmap v536: 129 pgs: 129 active+clean; 1.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-09-21T07:18:19.712 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:18:19.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:18:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 -- 172.21.15.27:0/2135760831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10b6c0 msgr2=0x7fc08c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 --2- 172.21.15.27:0/2135760831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10b6c0 0x7fc08c10bad0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc074009900 tx=0x7fc07402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 -- 172.21.15.27:0/2135760831 shutdown_connections 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 --2- 172.21.15.27:0/2135760831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc08c10c010 0x7fc08c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 --2- 172.21.15.27:0/2135760831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10b6c0 0x7fc08c10bad0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 --2- 172.21.15.27:0/2135760831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08c105d80 0x7fc08c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 -- 172.21.15.27:0/2135760831 >> 172.21.15.27:0/2135760831 conn(0x7fc08c1017d0 msgr2=0x7fc08c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 -- 172.21.15.27:0/2135760831 shutdown_connections 2024-09-21T07:18:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.711+0000 7fc093c00640 1 -- 172.21.15.27:0/2135760831 wait complete. 2024-09-21T07:18:20.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.712+0000 7fc093c00640 1 Processor -- start 2024-09-21T07:18:20.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc093c00640 1 -- start start 2024-09-21T07:18:20.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc093c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc08c105d80 0x7fc08c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc093c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08c10b6c0 0x7fc08c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc093c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10c010 0x7fc08c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc093c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c115d20 con 0x7fc08c10c010 2024-09-21T07:18:20.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc093c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c115ba0 con 0x7fc08c105d80 2024-09-21T07:18:20.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc093c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c115ea0 con 0x7fc08c10b6c0 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc091c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10c010 0x7fc08c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc091200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc08c105d80 0x7fc08c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc091c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10c010 0x7fc08c1a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45048/0 (socket says 172.21.15.27:45048) 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc091c00640 1 -- 172.21.15.27:0/3232407598 learned_addr learned my addr 172.21.15.27:0/3232407598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.713+0000 7fc083e00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08c10b6c0 0x7fc08c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc091c00640 1 -- 172.21.15.27:0/3232407598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08c10b6c0 msgr2=0x7fc08c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc091c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08c10b6c0 0x7fc08c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc091c00640 1 -- 172.21.15.27:0/3232407598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc08c105d80 msgr2=0x7fc08c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc091c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc08c105d80 0x7fc08c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc091c00640 1 -- 172.21.15.27:0/3232407598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c1a2640 con 0x7fc08c10c010 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc091c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10c010 0x7fc08c1a1f10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc08400eb10 tx=0x7fc08400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc08a000640 1 -- 172.21.15.27:0/3232407598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084004b20 con 0x7fc08c10c010 2024-09-21T07:18:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc08a000640 1 -- 172.21.15.27:0/3232407598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084004cc0 con 0x7fc08c10c010 2024-09-21T07:18:20.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.714+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c1a2930 con 0x7fc08c10c010 2024-09-21T07:18:20.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.715+0000 7fc08a000640 1 -- 172.21.15.27:0/3232407598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084010670 con 0x7fc08c10c010 2024-09-21T07:18:20.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.715+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a9090 con 0x7fc08c10c010 2024-09-21T07:18:20.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.718+0000 7fc09a834740 1 Processor -- start 2024-09-21T07:18:20.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.718+0000 7fc08a000640 1 -- 172.21.15.27:0/3232407598 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc084005020 con 0x7fc08c10c010 2024-09-21T07:18:20.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc09a834740 1 -- start start 2024-09-21T07:18:20.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc09a834740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfab9350 0x55eadfe60d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc09a834740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfa90490 0x55eadfe61250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc09a834740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe61790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc09a834740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eadfdd6e90 con 0x55eadfa90490 2024-09-21T07:18:20.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc09a834740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eadfdc62c0 con 0x55eadfa98670 2024-09-21T07:18:20.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc09a834740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eadfdca0d0 con 0x55eadfab9350 2024-09-21T07:18:20.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc083e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfab9350 0x55eadfe60d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc091200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfa90490 0x55eadfe61250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc083e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfab9350 0x55eadfe60d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36136/0 (socket says 172.21.15.27:36136) 2024-09-21T07:18:20.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.719+0000 7fc083e00640 1 -- 172.21.15.27:0/3864328817 learned_addr learned my addr 172.21.15.27:0/3864328817 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc083e00640 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe61790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc091200640 1 -- 172.21.15.27:0/3864328817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfab9350 msgr2=0x55eadfe60d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc091200640 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfab9350 0x55eadfe60d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc091200640 1 -- 172.21.15.27:0/3864328817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 msgr2=0x55eadfe61790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc091200640 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe61790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc091200640 1 -- 172.21.15.27:0/3864328817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadfb24930 con 0x55eadfa90490 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc083e00640 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfab9350 0x55eadfe60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.720+0000 7fc091200640 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfa90490 0x55eadfe61250 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009c00 tx=0x7fc07c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e559f98156b2dfae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.721+0000 7fc082000640 1 -- 172.21.15.27:0/3864328817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004230 con 0x55eadfa90490 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.721+0000 7fc082000640 1 -- 172.21.15.27:0/3864328817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c0043d0 con 0x55eadfa90490 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.721+0000 7fc082000640 1 -- 172.21.15.27:0/3864328817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004b90 con 0x55eadfa90490 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.721+0000 7fc08a000640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0700b6700 0x7fc0700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.721+0000 7fc08a000640 1 -- 172.21.15.27:0/3232407598 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc0840e7fc0 con 0x7fc08c10c010 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.721+0000 7fc083e00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0700b6700 0x7fc0700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 -- 172.21.15.27:0/3864328817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfa90490 msgr2=0x55eadfe61250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfa90490 0x55eadfe61250 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009c00 tx=0x7fc07c011810 comp rx=0 tx=0).stop 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 -- 172.21.15.27:0/3864328817 shutdown_connections 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe61790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfa90490 0x55eadfe61250 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 --2- 172.21.15.27:0/3864328817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfab9350 0x55eadfe60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 -- 172.21.15.27:0/3864328817 >> 172.21.15.27:0/3864328817 conn(0x55eadfaad740 msgr2=0x55eadfe5ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 -- 172.21.15.27:0/3864328817 shutdown_connections 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc083e00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0700b6700 0x7fc0700b8c10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc074031440 tx=0x7fc074005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.722+0000 7fc09a834740 1 -- 172.21.15.27:0/3864328817 wait complete. 2024-09-21T07:18:20.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 -- start start 2024-09-21T07:18:20.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 Processor -- start 2024-09-21T07:18:20.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eadfc27890 con 0x55eadfab9350 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eadfd61160 con 0x55eadfa98670 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eadfe6f4f0 con 0x55eadfa90490 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.723+0000 7fc09a834740 10 client.0 _ll_register_callbacks cb 0x55eadfd7da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe6c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36152/0 (socket says 172.21.15.27:36152) 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 -- 172.21.15.27:0/554828358 learned_addr learned my addr 172.21.15.27:0/554828358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091c00640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc083e00640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 -- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 msgr2=0x55eadfe6cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 -- 172.21.15.27:0/554828358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 msgr2=0x55eadfe78810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe78810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091200640 1 -- 172.21.15.27:0/554828358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadfe78d50 con 0x55eadfa90490 2024-09-21T07:18:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.724+0000 7fc091c00640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc091200640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe6c8a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005890 tx=0x7fc07c019af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ffb066db3122e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc07aa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc07aa00640 1 -- 172.21.15.27:0/554828358 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c028020 con 0x55eadfa90490 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc07c0043d0 con 0x55eadfa90490 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc07aa00640 1 -- 172.21.15.27:0/554828358 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc07c024620 con 0x55eadfa90490 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc07aa00640 1 -- 172.21.15.27:0/554828358 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc07c033050 con 0x55eadfa90490 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc09a834740 10 client.27022 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.725+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eadfe7b680 con 0x55eadfa90490 2024-09-21T07:18:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc07aa00640 1 -- 172.21.15.27:0/554828358 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c0043d0 con 0x55eadfa90490 2024-09-21T07:18:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc07aa00640 1 -- 172.21.15.27:0/554828358 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc07c018070 con 0x55eadfa90490 2024-09-21T07:18:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc09a834740 10 client.27022 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc09a834740 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc09a834740 20 client.27022 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc09a834740 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa77cd0 0x55eadfe7ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc09a834740 4 client.27022 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.726+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eadfdd8de0 con 0x55eadfa77cd0 2024-09-21T07:18:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.727+0000 7fc083e00640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa77cd0 0x55eadfe7ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.727+0000 7fc083e00640 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa77cd0 0x55eadfe7ea40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18807862d2f2626 server_cookie=5673cd563f8dfdc8 in_seq=0 out_seq=0 2024-09-21T07:18:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.727+0000 7fc07aa00640 10 client.27022 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.728+0000 7fc07aa00640 1 -- 172.21.15.27:0/554828358 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eadfdd8de0 con 0x55eadfa77cd0 2024-09-21T07:18:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.728+0000 7fc07aa00640 10 client.27022 handle_command_reply: tid=0 2024-09-21T07:18:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.728+0000 7fc09a834740 1 client.27022 shutdown 2024-09-21T07:18:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 msgr2=0x55eadfe6c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe6c8a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005890 tx=0x7fc07c019af0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 shutdown_connections 2024-09-21T07:18:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa77cd0 0x55eadfe7ea40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 --2- 172.21.15.27:0/554828358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe6c8a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc07aa00640 0 client.27022 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 >> 172.21.15.27:0/554828358 conn(0x55eadfaad740 msgr2=0x55eadfe5d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 shutdown_connections 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 1 -- 172.21.15.27:0/554828358 wait complete. 2024-09-21T07:18:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.729+0000 7fc09a834740 20 client.27022 trim_cache size 0 max 16384 2024-09-21T07:18:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 Processor -- start 2024-09-21T07:18:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 -- start start 2024-09-21T07:18:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe66e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe6e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1b5830 con 0x55eadfab9350 2024-09-21T07:18:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1b3100 con 0x55eadfa98670 2024-09-21T07:18:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc09a834740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1b0f50 con 0x55eadfa90490 2024-09-21T07:18:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc091c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe6e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc091c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe6e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45078/0 (socket says 172.21.15.27:45078) 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.748+0000 7fc091c00640 1 -- 172.21.15.27:0/3025983437 learned_addr learned my addr 172.21.15.27:0/3025983437 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc083e00640 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc091200640 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe66e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc091c00640 1 -- 172.21.15.27:0/3025983437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 msgr2=0x55eadfe66e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc091c00640 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe66e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc091c00640 1 -- 172.21.15.27:0/3025983437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 msgr2=0x55eadfe6be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc091c00640 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc091c00640 1 -- 172.21.15.27:0/3025983437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadfe6f050 con 0x55eadfab9350 2024-09-21T07:18:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc091c00640 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe6e2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc0840ad7c0 tx=0x7fc084004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81d36d32b82ddaf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:20.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc081600640 1 -- 172.21.15.27:0/3025983437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840ad230 con 0x55eadfab9350 2024-09-21T07:18:20.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.749+0000 7fc081600640 1 -- 172.21.15.27:0/3025983437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0840ad5d0 con 0x55eadfab9350 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc081600640 1 -- 172.21.15.27:0/3025983437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840b2200 con 0x55eadfab9350 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 -- 172.21.15.27:0/3025983437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 msgr2=0x55eadfe6e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe6e2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc0840ad7c0 tx=0x7fc084004a40 comp rx=0 tx=0).stop 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 -- 172.21.15.27:0/3025983437 shutdown_connections 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe6e2d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa98670 0x55eadfe6be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 --2- 172.21.15.27:0/3025983437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa90490 0x55eadfe66e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 -- 172.21.15.27:0/3025983437 >> 172.21.15.27:0/3025983437 conn(0x55eadfaad740 msgr2=0x55eadfe6dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.750+0000 7fc09a834740 1 -- 172.21.15.27:0/3025983437 shutdown_connections 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.751+0000 7fc09a834740 1 -- 172.21.15.27:0/3025983437 wait complete. 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.751+0000 7fc09a834740 1 -- start start 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.751+0000 7fc09a834740 1 Processor -- start 2024-09-21T07:18:20.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc09a834740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa90490 0x55eadfe74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc09a834740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa98670 0x55eadfe70120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc09a834740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe70660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc09a834740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1b15c0 con 0x55eadfab9350 2024-09-21T07:18:20.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc09a834740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1b2a30 con 0x55eadfa90490 2024-09-21T07:18:20.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc09a834740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1b1c90 con 0x55eadfa98670 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc09a834740 10 client.0 _ll_register_callbacks cb 0x7fc074009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc091200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa90490 0x55eadfe74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc091200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa90490 0x55eadfe74c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50870/0 (socket says 172.21.15.27:50870) 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc083e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa98670 0x55eadfe70120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc091c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe70660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.752+0000 7fc091200640 1 -- 172.21.15.27:0/594591398 learned_addr learned my addr 172.21.15.27:0/594591398 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc083e00640 1 -- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa90490 msgr2=0x55eadfe74c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc083e00640 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa90490 0x55eadfe74c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc083e00640 1 -- 172.21.15.27:0/594591398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 msgr2=0x55eadfe70660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc083e00640 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe70660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc083e00640 1 -- 172.21.15.27:0/594591398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadfe70ba0 con 0x55eadfa98670 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc091200640 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa90490 0x55eadfe74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc091c00640 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe70660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:20.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc083e00640 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa98670 0x55eadfe70120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc074000c00 tx=0x7fc074005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f942f907d2cee621 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc07aa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc07aa00640 1 -- 172.21.15.27:0/594591398 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074031af0 con 0x55eadfa98670 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0840ad5d0 con 0x55eadfa98670 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc07aa00640 1 -- 172.21.15.27:0/594591398 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fc074031c90 con 0x55eadfa98670 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc07aa00640 1 -- 172.21.15.27:0/594591398 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc074003f00 con 0x55eadfa98670 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.753+0000 7fc09a834740 10 client.27028 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eadfee9010 con 0x55eadfa98670 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc07aa00640 1 -- 172.21.15.27:0/594591398 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0840ad5d0 con 0x55eadfa98670 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc07aa00640 1 -- 172.21.15.27:0/594591398 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc07403c4c0 con 0x55eadfa98670 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc09a834740 10 client.27028 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc09a834740 10 client.27028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc09a834740 20 client.27028 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc09a834740 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa4d080 0x55eadfeeb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc09a834740 4 client.27028 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:18:20.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.754+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eadfdd4890 con 0x55eadfa4d080 2024-09-21T07:18:20.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.755+0000 7fc091200640 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa4d080 0x55eadfeeb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.755+0000 7fc091200640 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa4d080 0x55eadfeeb7d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69733d7b73c7ac9c server_cookie=91b6ee503ad675c8 in_seq=0 out_seq=0 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.755+0000 7fc07aa00640 10 client.27028 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.756+0000 7fc07aa00640 1 -- 172.21.15.27:0/594591398 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eadfdd4890 con 0x55eadfa4d080 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.756+0000 7fc07aa00640 10 client.27028 handle_command_reply: tid=0 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 client.27028 shutdown 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa98670 msgr2=0x55eadfe70120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa98670 0x55eadfe70120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc074000c00 tx=0x7fc074005780 comp rx=0 tx=0).stop 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 shutdown_connections 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eadfa4d080 0x55eadfeeb7d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eadfa98670 0x55eadfe70120 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eadfab9350 0x55eadfe70660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 --2- 172.21.15.27:0/594591398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eadfa90490 0x55eadfe74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc07aa00640 0 client.27028 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 >> 172.21.15.27:0/594591398 conn(0x55eadfaad740 msgr2=0x55eadfe5e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 shutdown_connections 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 1 -- 172.21.15.27:0/594591398 wait complete. 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.757+0000 7fc09a834740 20 client.27028 trim_cache size 0 max 16384 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "e2a1fe8e-18f3-445f-b7e7-67885d99e650", 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:18:20.761 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:18:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0700b6700 msgr2=0x7fc0700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0700b6700 0x7fc0700b8c10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc074031440 tx=0x7fc074005a20 comp rx=0 tx=0).stop 2024-09-21T07:18:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10c010 msgr2=0x7fc08c1a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10c010 0x7fc08c1a1f10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc08400eb10 tx=0x7fc08400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 shutdown_connections 2024-09-21T07:18:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08c10b6c0 0x7fc08c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc0700b6700 0x7fc0700b8c10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc08c10c010 0x7fc08c1a1f10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 --2- 172.21.15.27:0/3232407598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc08c105d80 0x7fc08c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 >> 172.21.15.27:0/3232407598 conn(0x7fc08c1017d0 msgr2=0x7fc08c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 shutdown_connections 2024-09-21T07:18:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:20.758+0000 7fc093c00640 1 -- 172.21.15.27:0/3232407598 wait complete. 2024-09-21T07:18:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:21 smithi153 ceph-mon[34747]: pgmap v537: 129 pgs: 129 active+clean; 1.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-21T07:18:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:21 smithi119 ceph-mon[34519]: pgmap v537: 129 pgs: 129 active+clean; 1.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-21T07:18:21.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:21 smithi027 ceph-mon[30779]: pgmap v537: 129 pgs: 129 active+clean; 1.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-21T07:18:21.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:18:21.980 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:22.410 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:22 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:18:22.410 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:22 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:18:22.410 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:22 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:18:22.410 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:22 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:18:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:22 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:18:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:22 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:18:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:22 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:18:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:22 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:18:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:22 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:18:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:22 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:18:22.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:22 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:18:22.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:22 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:18:22.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 -- 172.21.15.27:0/3921352883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e4109c40 msgr2=0x7fe9e4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/3921352883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e4109c40 0x7fe9e4110470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009ca0 tx=0x7fe9dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 -- 172.21.15.27:0/3921352883 shutdown_connections 2024-09-21T07:18:22.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/3921352883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e4109c40 0x7fe9e4110470 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/3921352883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e41092f0 0x7fe9e4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/3921352883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e41039b0 0x7fe9e4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 -- 172.21.15.27:0/3921352883 >> 172.21.15.27:0/3921352883 conn(0x7fe9e40ff400 msgr2=0x7fe9e4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:22.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 -- 172.21.15.27:0/3921352883 shutdown_connections 2024-09-21T07:18:22.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.592+0000 7fe9e9c00640 1 -- 172.21.15.27:0/3921352883 wait complete. 2024-09-21T07:18:22.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.593+0000 7fe9e9c00640 1 Processor -- start 2024-09-21T07:18:22.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.593+0000 7fe9e9c00640 1 -- start start 2024-09-21T07:18:22.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e41039b0 0x7fe9e419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e41092f0 0x7fe9e419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4109c40 0x7fe9e41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e9c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4113950 con 0x7fe9e41092f0 2024-09-21T07:18:22.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e9c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41137d0 con 0x7fe9e4109c40 2024-09-21T07:18:22.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e9c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4113ad0 con 0x7fe9e41039b0 2024-09-21T07:18:22.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e41092f0 0x7fe9e419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e41092f0 0x7fe9e419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45110/0 (socket says 172.21.15.27:45110) 2024-09-21T07:18:22.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3400640 1 -- 172.21.15.27:0/864677151 learned_addr learned my addr 172.21.15.27:0/864677151 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:22.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e41039b0 0x7fe9e419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e9200640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4109c40 0x7fe9e41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3400640 1 -- 172.21.15.27:0/864677151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e41039b0 msgr2=0x7fe9e419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3400640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e41039b0 0x7fe9e419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3400640 1 -- 172.21.15.27:0/864677151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4109c40 msgr2=0x7fe9e41a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.594+0000 7fe9e3400640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4109c40 0x7fe9e41a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e3400640 1 -- 172.21.15.27:0/864677151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e41a2410 con 0x7fe9e41092f0 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e41039b0 0x7fe9e419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e3400640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e41092f0 0x7fe9e419ca40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000d950 tx=0x7fe9d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e0c00640 1 -- 172.21.15.27:0/864677151 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0017070 con 0x7fe9e41092f0 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e0c00640 1 -- 172.21.15.27:0/864677151 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d0009c30 con 0x7fe9e41092f0 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e0c00640 1 -- 172.21.15.27:0/864677151 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00047a0 con 0x7fe9e41092f0 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e41a2700 con 0x7fe9e41092f0 2024-09-21T07:18:22.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.595+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e41a8e60 con 0x7fe9e41092f0 2024-09-21T07:18:22.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.598+0000 7fe9f1dd6740 1 Processor -- start 2024-09-21T07:18:22.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9f1dd6740 1 -- start start 2024-09-21T07:18:22.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9e0c00640 1 -- 172.21.15.27:0/864677151 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe9d0004020 con 0x7fe9e41092f0 2024-09-21T07:18:22.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56de230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5559e5654f70 con 0x5559e5337070 2024-09-21T07:18:22.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5559e56443a0 con 0x5559e5316390 2024-09-21T07:18:22.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5559e56481b0 con 0x5559e530e1b0 2024-09-21T07:18:22.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.599+0000 7fe9e9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56ddcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45112/0 (socket says 172.21.15.27:45112) 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e9200640 1 -- 172.21.15.27:0/809344073 learned_addr learned my addr 172.21.15.27:0/809344073 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e3400640 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56de230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e9200640 1 -- 172.21.15.27:0/809344073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 msgr2=0x5559e56de230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e9200640 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56de230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e9200640 1 -- 172.21.15.27:0/809344073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 msgr2=0x5559e56de770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e9200640 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56de770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e9200640 1 -- 172.21.15.27:0/809344073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559e560ec90 con 0x5559e5337070 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.600+0000 7fe9e3400640 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56de770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:22.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.601+0000 7fe9e9200640 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56ddcf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009ca0 tx=0x7fe9dc02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a3427fd439c97c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:22.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.601+0000 7fe9d9600640 1 -- 172.21.15.27:0/809344073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc031db0 con 0x5559e5337070 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.601+0000 7fe9d9600640 1 -- 172.21.15.27:0/809344073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9dc036030 con 0x5559e5337070 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.601+0000 7fe9d9600640 1 -- 172.21.15.27:0/809344073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc004330 con 0x5559e5337070 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.601+0000 7fe9e0c00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9c80b6680 0x7fe9c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.601+0000 7fe9e0c00640 1 -- 172.21.15.27:0/864677151 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe9d00e76f0 con 0x7fe9e41092f0 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.601+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9c80b6680 0x7fe9c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/809344073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 msgr2=0x5559e56ddcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56ddcf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009ca0 tx=0x7fe9dc02f310 comp rx=0 tx=0).stop 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/809344073 shutdown_connections 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56de230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56de770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/809344073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56ddcf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/809344073 >> 172.21.15.27:0/809344073 conn(0x5559e532b460 msgr2=0x5559e56dbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/809344073 shutdown_connections 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/809344073 wait complete. 2024-09-21T07:18:22.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.602+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9c80b6680 0x7fe9c80b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5559e56dc970 tx=0x7fe9d4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:22.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.603+0000 7fe9f1dd6740 1 -- start start 2024-09-21T07:18:22.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.603+0000 7fe9f1dd6740 1 Processor -- start 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56f5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5559e54a5700 con 0x5559e5337070 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5559e55df540 con 0x5559e5316390 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5559e56ec2f0 con 0x5559e530e1b0 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9f1dd6740 10 client.0 _ll_register_callbacks cb 0x5559e55ef430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9e9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9e3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9e9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45120/0 (socket says 172.21.15.27:45120) 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9e9200640 1 -- 172.21.15.27:0/1766495443 learned_addr learned my addr 172.21.15.27:0/1766495443 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.604+0000 7fe9e9200640 1 -- 172.21.15.27:0/1766495443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 msgr2=0x5559e56e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9e9200640 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9e9200640 1 -- 172.21.15.27:0/1766495443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 msgr2=0x5559e56e9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9e9200640 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56e9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9e9200640 1 -- 172.21.15.27:0/1766495443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559e53a2650 con 0x5559e5337070 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9e9200640 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56f5610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009bf0 tx=0x7fe9dc02f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9756e437b5e906f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9ce000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9ce000640 1 -- 172.21.15.27:0/1766495443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc031450 con 0x5559e5337070 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9ce000640 1 -- 172.21.15.27:0/1766495443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe9dc0315f0 con 0x5559e5337070 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.605+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5559e55e1b50 con 0x5559e5337070 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9ce000640 1 -- 172.21.15.27:0/1766495443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9dc0317f0 con 0x5559e5337070 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 10 client.17022 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559e55fc910 con 0x5559e5337070 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9ce000640 1 -- 172.21.15.27:0/1766495443 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5559e55e1b50 con 0x5559e5337070 2024-09-21T07:18:22.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9ce000640 1 -- 172.21.15.27:0/1766495443 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9dc04e070 con 0x5559e5337070 2024-09-21T07:18:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 10 client.17022 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 20 client.17022 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52f59f0 0x5559e56fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 4 client.17022 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.606+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559e5656ec0 con 0x5559e52f59f0 2024-09-21T07:18:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.607+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52f59f0 0x5559e56fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.607+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52f59f0 0x5559e56fb4d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec5b5f651448071 server_cookie=f856c039ad750fa3 in_seq=0 out_seq=0 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.607+0000 7fe9ce000640 10 client.17022 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9ce000640 1 -- 172.21.15.27:0/1766495443 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5559e5656ec0 con 0x5559e52f59f0 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9ce000640 10 client.17022 handle_command_reply: tid=0 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 client.17022 shutdown 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 msgr2=0x5559e56f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56f5610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009bf0 tx=0x7fe9dc02f500 comp rx=0 tx=0).stop 2024-09-21T07:18:22.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 shutdown_connections 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52f59f0 0x5559e56fb4d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5337070 0x5559e56f5610 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56e9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/1766495443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e530e1b0 0x5559e56e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 >> 172.21.15.27:0/1766495443 conn(0x5559e532b460 msgr2=0x5559e56e0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.608+0000 7fe9ce000640 0 client.17022 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.609+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 shutdown_connections 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.609+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/1766495443 wait complete. 2024-09-21T07:18:22.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.609+0000 7fe9f1dd6740 20 client.17022 trim_cache size 0 max 16384 2024-09-21T07:18:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.627+0000 7fe9f1dd6740 1 Processor -- start 2024-09-21T07:18:22.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.627+0000 7fe9f1dd6740 1 -- start start 2024-09-21T07:18:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e530e1b0 0x5559e56e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56db1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56db710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41b5610 con 0x5559e530e1b0 2024-09-21T07:18:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41b2ee0 con 0x5559e5316390 2024-09-21T07:18:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41b0d30 con 0x5559e5337070 2024-09-21T07:18:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e530e1b0 0x5559e56e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e530e1b0 0x5559e56e48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45132/0 (socket says 172.21.15.27:45132) 2024-09-21T07:18:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 -- 172.21.15.27:0/2391285406 learned_addr learned my addr 172.21.15.27:0/2391285406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e9200640 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3400640 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56db1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 -- 172.21.15.27:0/2391285406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 msgr2=0x5559e56db710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56db710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 -- 172.21.15.27:0/2391285406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 msgr2=0x5559e56db1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56db1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.628+0000 7fe9e3e00640 1 -- 172.21.15.27:0/2391285406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559e56e0b50 con 0x5559e530e1b0 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.629+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e530e1b0 0x5559e56e48c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5559e56fbaa0 tx=0x7fe9d401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf688b3533a7d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.629+0000 7fe9d8c00640 1 -- 172.21.15.27:0/2391285406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4005020 con 0x5559e530e1b0 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.629+0000 7fe9d8c00640 1 -- 172.21.15.27:0/2391285406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d40047a0 con 0x5559e530e1b0 2024-09-21T07:18:22.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.629+0000 7fe9d8c00640 1 -- 172.21.15.27:0/2391285406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400f940 con 0x5559e530e1b0 2024-09-21T07:18:22.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/2391285406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e530e1b0 msgr2=0x5559e56e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e530e1b0 0x5559e56e48c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5559e56fbaa0 tx=0x7fe9d401b000 comp rx=0 tx=0).stop 2024-09-21T07:18:22.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/2391285406 shutdown_connections 2024-09-21T07:18:22.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56db710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e5316390 0x5559e56db1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/2391285406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e530e1b0 0x5559e56e48c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/2391285406 >> 172.21.15.27:0/2391285406 conn(0x5559e532b460 msgr2=0x5559e56e7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:22.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/2391285406 shutdown_connections 2024-09-21T07:18:22.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.630+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/2391285406 wait complete. 2024-09-21T07:18:22.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 -- start start 2024-09-21T07:18:22.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 Processor -- start 2024-09-21T07:18:22.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e530e1b0 0x5559e56f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5316390 0x5559e56ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41b13a0 con 0x5559e5316390 2024-09-21T07:18:22.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41b2810 con 0x5559e530e1b0 2024-09-21T07:18:22.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41b1a70 con 0x5559e5337070 2024-09-21T07:18:22.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.631+0000 7fe9f1dd6740 10 client.0 _ll_register_callbacks cb 0x7fe9dc009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:22.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e530e1b0 0x5559e56f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5316390 0x5559e56ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e530e1b0 0x5559e56f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50936/0 (socket says 172.21.15.27:50936) 2024-09-21T07:18:22.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 -- 172.21.15.27:0/750013603 learned_addr learned my addr 172.21.15.27:0/750013603 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:22.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e9200640 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 -- 172.21.15.27:0/750013603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 msgr2=0x5559e56eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 -- 172.21.15.27:0/750013603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5316390 msgr2=0x5559e56ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:22.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5316390 0x5559e56ee390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e3e00640 1 -- 172.21.15.27:0/750013603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559e5765d10 con 0x5559e530e1b0 2024-09-21T07:18:22.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.632+0000 7fe9e9200640 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:22.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9e3e00640 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e530e1b0 0x5559e56f1ed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4005f60 tx=0x7fe9d4004030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c84c1f82da6316 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:22.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9ce000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:22.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9ce000640 1 -- 172.21.15.27:0/750013603 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400f7e0 con 0x5559e530e1b0 2024-09-21T07:18:22.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d40047a0 con 0x5559e530e1b0 2024-09-21T07:18:22.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9ce000640 1 -- 172.21.15.27:0/750013603 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fe9d4018050 con 0x5559e530e1b0 2024-09-21T07:18:22.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9ce000640 1 -- 172.21.15.27:0/750013603 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9d400f980 con 0x5559e530e1b0 2024-09-21T07:18:22.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9f1dd6740 10 client.27059 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:22.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559e5766de0 con 0x5559e530e1b0 2024-09-21T07:18:22.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.633+0000 7fe9ce000640 1 -- 172.21.15.27:0/750013603 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d40047a0 con 0x5559e530e1b0 2024-09-21T07:18:22.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9ce000640 1 -- 172.21.15.27:0/750013603 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9d403d420 con 0x5559e530e1b0 2024-09-21T07:18:22.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9f1dd6740 10 client.27059 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:22.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9f1dd6740 10 client.27059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:22.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9f1dd6740 20 client.27059 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:22.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52cada0 0x5559e57695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:22.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9f1dd6740 4 client.27059 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:18:22.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559e5652970 con 0x5559e52cada0 2024-09-21T07:18:22.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.634+0000 7fe9e3400640 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52cada0 0x5559e57695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:22.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9e3400640 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52cada0 0x5559e57695a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe5879accbe383c server_cookie=4df1101ce2eed5ee in_seq=0 out_seq=0 2024-09-21T07:18:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9ce000640 10 client.27059 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9ce000640 1 -- 172.21.15.27:0/750013603 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x5559e5652970 con 0x5559e52cada0 2024-09-21T07:18:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9ce000640 10 client.27059 handle_command_reply: tid=0 2024-09-21T07:18:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 client.27059 shutdown 2024-09-21T07:18:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e530e1b0 msgr2=0x5559e56f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e530e1b0 0x5559e56f1ed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4005f60 tx=0x7fe9d4004030 comp rx=0 tx=0).stop 2024-09-21T07:18:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 shutdown_connections 2024-09-21T07:18:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5559e52cada0 0x5559e57695a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559e5337070 0x5559e56eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5559e5316390 0x5559e56ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 --2- 172.21.15.27:0/750013603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559e530e1b0 0x5559e56f1ed0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 >> 172.21.15.27:0/750013603 conn(0x5559e532b460 msgr2=0x5559e56e48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:22.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9ce000640 0 client.27059 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:22.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.635+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 shutdown_connections 2024-09-21T07:18:22.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9f1dd6740 1 -- 172.21.15.27:0/750013603 wait complete. 2024-09-21T07:18:22.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9f1dd6740 20 client.27059 trim_cache size 0 max 16384 2024-09-21T07:18:22.649 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:18:22.649 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:18:22.649 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:18:22.649 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:18:22.649 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "e2a1fe8e-18f3-445f-b7e7-67885d99e650", 2024-09-21T07:18:22.650 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:18:22.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:22.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.650 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:18:22.650 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:18:22.650 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:18:22.651 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.652 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:18:22.652 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:18:22.652 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.652 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.652 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.652 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.653 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.653 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:18:22.653 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:18:22.653 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.653 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.654 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.654 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.654 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.654 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:18:22.654 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:18:22.654 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.655 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.655 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.655 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.655 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.655 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:18:22.656 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:18:22.656 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.656 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.656 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.656 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.656 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.657 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:18:22.657 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:18:22.657 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.657 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.657 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.657 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.657 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.658 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:18:22.658 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:18:22.658 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.658 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.658 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.659 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.659 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:22.659 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:18:22.659 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:18:22.659 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:22.659 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:22.660 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:22.660 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:22.660 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:22.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:22.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:22.660 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:18:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9c80b6680 msgr2=0x7fe9c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9c80b6680 0x7fe9c80b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5559e56dc970 tx=0x7fe9d4005f90 comp rx=0 tx=0).stop 2024-09-21T07:18:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e41092f0 msgr2=0x7fe9e419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e41092f0 0x7fe9e419ca40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000d950 tx=0x7fe9d000de10 comp rx=0 tx=0).stop 2024-09-21T07:18:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 shutdown_connections 2024-09-21T07:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9c80b6680 0x7fe9c80b8b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4109c40 0x7fe9e41a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9e41092f0 0x7fe9e419ca40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 --2- 172.21.15.27:0/864677151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9e41039b0 0x7fe9e419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.636+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 >> 172.21.15.27:0/864677151 conn(0x7fe9e40ff400 msgr2=0x7fe9e4106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.637+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 shutdown_connections 2024-09-21T07:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:22.637+0000 7fe9e9c00640 1 -- 172.21.15.27:0/864677151 wait complete. 2024-09-21T07:18:23.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:23 smithi153 ceph-mon[34747]: pgmap v538: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-21T07:18:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:23 smithi027 ceph-mon[30779]: pgmap v538: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-21T07:18:23.524 INFO:tasks.cephfs.filesystem:scrub status for tag:e2a1fe8e-18f3-445f-b7e7-67885d99e650 - {'path': '/', 'tag': 'e2a1fe8e-18f3-445f-b7e7-67885d99e650', 'options': 'recursive,force'} 2024-09-21T07:18:23.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:23 smithi119 ceph-mon[34519]: pgmap v538: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-21T07:18:24.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:24 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:18:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:24 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:18:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:24 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:18:25.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:25 smithi153 ceph-mon[34747]: pgmap v539: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-21T07:18:25.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:25 smithi119 ceph-mon[34519]: pgmap v539: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-21T07:18:25.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:25 smithi027 ceph-mon[30779]: pgmap v539: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-21T07:18:26.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:26 smithi153 ceph-mon[34747]: pgmap v540: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-21T07:18:26.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:26 smithi119 ceph-mon[34519]: pgmap v540: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-21T07:18:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:26 smithi027 ceph-mon[30779]: pgmap v540: 129 pgs: 129 active+clean; 2.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-21T07:18:28.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:28 smithi027 ceph-mon[30779]: pgmap v541: 129 pgs: 129 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-21T07:18:28.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:28 smithi153 ceph-mon[34747]: pgmap v541: 129 pgs: 129 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-21T07:18:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:28 smithi119 ceph-mon[34519]: pgmap v541: 129 pgs: 129 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-09-21T07:18:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 -- 172.21.15.27:0/757864863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c103900 msgr2=0x7f0e8c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 --2- 172.21.15.27:0/757864863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c103900 0x7f0e8c103d10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009900 tx=0x7f0e7c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 -- 172.21.15.27:0/757864863 shutdown_connections 2024-09-21T07:18:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 --2- 172.21.15.27:0/757864863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8c109230 0x7f0e8c109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 --2- 172.21.15.27:0/757864863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8c109b80 0x7f0e8c1103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 --2- 172.21.15.27:0/757864863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c103900 0x7f0e8c103d10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 -- 172.21.15.27:0/757864863 >> 172.21.15.27:0/757864863 conn(0x7f0e8c0ff360 msgr2=0x7f0e8c1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 -- 172.21.15.27:0/757864863 shutdown_connections 2024-09-21T07:18:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.883+0000 7f0e8be00640 1 -- 172.21.15.27:0/757864863 wait complete. 2024-09-21T07:18:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.884+0000 7f0e8be00640 1 Processor -- start 2024-09-21T07:18:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.884+0000 7f0e8be00640 1 -- start start 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8c103900 0x7f0e8c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c109230 0x7f0e8c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8c109b80 0x7f0e8c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c113890 con 0x7f0e8c109230 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c113710 con 0x7f0e8c109b80 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c113a10 con 0x7f0e8c103900 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c109230 0x7f0e8c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c109230 0x7f0e8c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33624/0 (socket says 172.21.15.27:33624) 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 -- 172.21.15.27:0/2567657466 learned_addr learned my addr 172.21.15.27:0/2567657466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8c103900 0x7f0e8c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8b400640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8c109b80 0x7f0e8c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 -- 172.21.15.27:0/2567657466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8c103900 msgr2=0x7f0e8c19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8c103900 0x7f0e8c19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 -- 172.21.15.27:0/2567657466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8c109b80 msgr2=0x7f0e8c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8c109b80 0x7f0e8c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.885+0000 7f0e8a000640 1 -- 172.21.15.27:0/2567657466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c1a2420 con 0x7f0e8c109230 2024-09-21T07:18:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.886+0000 7f0e8a000640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c109230 0x7f0e8c19ca90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e9c0 tx=0x7f0e7800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.886+0000 7f0e83400640 1 -- 172.21.15.27:0/2567657466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800ccc0 con 0x7f0e8c109230 2024-09-21T07:18:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.886+0000 7f0e83400640 1 -- 172.21.15.27:0/2567657466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e780054d0 con 0x7f0e8c109230 2024-09-21T07:18:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.886+0000 7f0e83400640 1 -- 172.21.15.27:0/2567657466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78010430 con 0x7f0e8c109230 2024-09-21T07:18:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.886+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c1a2710 con 0x7f0e8c109230 2024-09-21T07:18:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.886+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1a8ea0 con 0x7f0e8c109230 2024-09-21T07:18:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 Processor -- start 2024-09-21T07:18:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 -- start start 2024-09-21T07:18:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e83400640 1 -- 172.21.15.27:0/2567657466 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0e78004f40 con 0x7f0e8c109230 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde373e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde374350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde374890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bde2e96b0 con 0x555bddfae4d0 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555bde2d8ae0 con 0x555bddfa62f0 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e98a03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555bde2dc8f0 con 0x555bddfcf1b0 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e8b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde373e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde374350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e8a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde374890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e8b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde373e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:60250/0 (socket says 172.21.15.27:60250) 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.890+0000 7f0e8b400640 1 -- 172.21.15.27:0/3344277564 learned_addr learned my addr 172.21.15.27:0/3344277564 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.891+0000 7f0e8aa00640 1 -- 172.21.15.27:0/3344277564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 msgr2=0x555bde373e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.891+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/3344277564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde373e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.891+0000 7f0e8aa00640 1 -- 172.21.15.27:0/3344277564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 msgr2=0x555bde374890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.891+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/3344277564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde374890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.891+0000 7f0e8aa00640 1 -- 172.21.15.27:0/3344277564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bde2a5780 con 0x555bddfa62f0 2024-09-21T07:18:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.891+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/3344277564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde374350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02a910 tx=0x7f0e7c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e547ab1f734c52ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.892+0000 7f0e77e00640 1 -- 172.21.15.27:0/3344277564 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c040070 con 0x555bddfa62f0 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.892+0000 7f0e77e00640 1 -- 172.21.15.27:0/3344277564 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c005500 con 0x555bddfa62f0 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.892+0000 7f0e77e00640 1 -- 172.21.15.27:0/3344277564 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c035040 con 0x555bddfa62f0 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.892+0000 7f0e83400640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e700b6700 0x7f0e700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.892+0000 7f0e83400640 1 -- 172.21.15.27:0/2567657466 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0e780e7ec0 con 0x7f0e8c109230 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.892+0000 7f0e8b400640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e700b6700 0x7f0e700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 -- 172.21.15.27:0/3344277564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 msgr2=0x555bde374350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 --2- 172.21.15.27:0/3344277564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde374350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02a910 tx=0x7f0e7c004040 comp rx=0 tx=0).stop 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 -- 172.21.15.27:0/3344277564 shutdown_connections 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 --2- 172.21.15.27:0/3344277564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde374890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 --2- 172.21.15.27:0/3344277564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde374350 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 --2- 172.21.15.27:0/3344277564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde373e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 -- 172.21.15.27:0/3344277564 >> 172.21.15.27:0/3344277564 conn(0x555bddfc3300 msgr2=0x555bde371d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 -- 172.21.15.27:0/3344277564 shutdown_connections 2024-09-21T07:18:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e98a03740 1 -- 172.21.15.27:0/3344277564 wait complete. 2024-09-21T07:18:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.893+0000 7f0e8b400640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e700b6700 0x7f0e700b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0e84012680 tx=0x7f0e84009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 -- start start 2024-09-21T07:18:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 Processor -- start 2024-09-21T07:18:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde37f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde37f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde37ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bde210ab0 con 0x555bddfae4d0 2024-09-21T07:18:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555bde27f570 con 0x555bddfa62f0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555bde382530 con 0x555bddfcf1b0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.894+0000 7f0e98a03740 10 client.0 _ll_register_callbacks cb 0x555bde28e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde37f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde37f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde37f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34392/0 (socket says 172.21.15.27:34392) 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 -- 172.21.15.27:0/4175099733 learned_addr learned my addr 172.21.15.27:0/4175099733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8b400640 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde37ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 -- 172.21.15.27:0/4175099733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 msgr2=0x555bde37ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde37ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 -- 172.21.15.27:0/4175099733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 msgr2=0x555bde37f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde37f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 -- 172.21.15.27:0/4175099733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bde03a790 con 0x555bddfa62f0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde37f4b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005ab0 tx=0x7f0e7c004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23d7ac01b99133b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.895+0000 7f0e74c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e74c00640 1 -- 172.21.15.27:0/4175099733 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c04d020 con 0x555bddfa62f0 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c005500 con 0x555bddfa62f0 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e74c00640 1 -- 172.21.15.27:0/4175099733 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0e7c0048c0 con 0x555bddfa62f0 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e74c00640 1 -- 172.21.15.27:0/4175099733 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c058050 con 0x555bddfa62f0 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e98a03740 10 client.27071 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bde0b3620 con 0x555bddfa62f0 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e74c00640 1 -- 172.21.15.27:0/4175099733 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c005500 con 0x555bddfa62f0 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e74c00640 1 -- 172.21.15.27:0/4175099733 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0e7c003c00 con 0x555bddfa62f0 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e98a03740 10 client.27071 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e98a03740 10 client.27071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.896+0000 7f0e98a03740 20 client.27071 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.897+0000 7f0e98a03740 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf8dd80 0x555bde391360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.897+0000 7f0e98a03740 4 client.27071 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.897+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bde2eb600 con 0x555bddf8dd80 2024-09-21T07:18:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.897+0000 7f0e8a000640 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf8dd80 0x555bde391360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.897+0000 7f0e8a000640 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf8dd80 0x555bde391360 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5fa5a239686948 server_cookie=e939223ca70c9c5a in_seq=0 out_seq=0 2024-09-21T07:18:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.897+0000 7f0e74c00640 10 client.27071 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.898+0000 7f0e74c00640 1 -- 172.21.15.27:0/4175099733 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bde2eb600 con 0x555bddf8dd80 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.898+0000 7f0e74c00640 10 client.27071 handle_command_reply: tid=0 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.898+0000 7f0e98a03740 1 client.27071 shutdown 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.898+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 msgr2=0x555bde37f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.898+0000 7f0e98a03740 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde37f4b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005ab0 tx=0x7f0e7c004f90 comp rx=0 tx=0).stop 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 shutdown_connections 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf8dd80 0x555bde391360 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde37ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfae4d0 0x555bde37f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 --2- 172.21.15.27:0/4175099733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfa62f0 0x555bde37f4b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e74c00640 0 client.27071 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 >> 172.21.15.27:0/4175099733 conn(0x555bddfc3300 msgr2=0x555bde374e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 shutdown_connections 2024-09-21T07:18:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 1 -- 172.21.15.27:0/4175099733 wait complete. 2024-09-21T07:18:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.899+0000 7f0e98a03740 20 client.27071 trim_cache size 0 max 16384 2024-09-21T07:18:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 Processor -- start 2024-09-21T07:18:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 -- start start 2024-09-21T07:18:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfa62f0 0x555bde375550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde37f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfcf1b0 0x555bde37fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b5600 con 0x555bddfcf1b0 2024-09-21T07:18:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b2ed0 con 0x555bddfae4d0 2024-09-21T07:18:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.914+0000 7f0e98a03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b0d20 con 0x555bddfa62f0 2024-09-21T07:18:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfcf1b0 0x555bde37fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfcf1b0 0x555bde37fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33642/0 (socket says 172.21.15.27:33642) 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 -- 172.21.15.27:0/1847216150 learned_addr learned my addr 172.21.15.27:0/1847216150 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfa62f0 0x555bde375550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8a000640 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde37f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 -- 172.21.15.27:0/1847216150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfa62f0 msgr2=0x555bde375550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfa62f0 0x555bde375550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 -- 172.21.15.27:0/1847216150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 msgr2=0x555bde37f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde37f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8b400640 1 -- 172.21.15.27:0/1847216150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bde379a10 con 0x555bddfcf1b0 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.915+0000 7f0e8a000640 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde37f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.916+0000 7f0e8b400640 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfcf1b0 0x555bde37fed0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e84003cf0 tx=0x7f0e8401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87f64e9e57aaa43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.916+0000 7f0e77400640 1 -- 172.21.15.27:0/1847216150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84012020 con 0x555bddfcf1b0 2024-09-21T07:18:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.916+0000 7f0e77400640 1 -- 172.21.15.27:0/1847216150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e840121c0 con 0x555bddfcf1b0 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.916+0000 7f0e77400640 1 -- 172.21.15.27:0/1847216150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84011760 con 0x555bddfcf1b0 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 -- 172.21.15.27:0/1847216150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfcf1b0 msgr2=0x555bde37fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfcf1b0 0x555bde37fed0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e84003cf0 tx=0x7f0e8401b000 comp rx=0 tx=0).stop 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 -- 172.21.15.27:0/1847216150 shutdown_connections 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfcf1b0 0x555bde37fed0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde37f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1847216150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfa62f0 0x555bde375550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 -- 172.21.15.27:0/1847216150 >> 172.21.15.27:0/1847216150 conn(0x555bddfc3300 msgr2=0x555bde374e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 -- 172.21.15.27:0/1847216150 shutdown_connections 2024-09-21T07:18:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 -- 172.21.15.27:0/1847216150 wait complete. 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 -- start start 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.917+0000 7f0e98a03740 1 Processor -- start 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfa62f0 0x555bde3f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde3f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e98a03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde383020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e98a03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b1390 con 0x555bddfa62f0 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e98a03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b2800 con 0x555bddfae4d0 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e98a03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b1a60 con 0x555bddfcf1b0 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e98a03740 10 client.0 _ll_register_callbacks cb 0x7f0e780eb840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfa62f0 0x555bde3f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfa62f0 0x555bde3f3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33654/0 (socket says 172.21.15.27:33654) 2024-09-21T07:18:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e8aa00640 1 -- 172.21.15.27:0/1874667947 learned_addr learned my addr 172.21.15.27:0/1874667947 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e8b400640 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde383020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.918+0000 7f0e8a000640 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde3f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e8aa00640 1 -- 172.21.15.27:0/1874667947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 msgr2=0x555bde383020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde383020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e8aa00640 1 -- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 msgr2=0x555bde3f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde3f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e8aa00640 1 -- 172.21.15.27:0/1874667947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bde383790 con 0x555bddfa62f0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e8aa00640 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfa62f0 0x555bde3f3ee0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003fa0 tx=0x7f0e7c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da92a374b80b8f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e74c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e74c00640 1 -- 172.21.15.27:0/1874667947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c060020 con 0x555bddfa62f0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e840121c0 con 0x555bddfa62f0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.919+0000 7f0e74c00640 1 -- 172.21.15.27:0/1874667947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f0e7c005200 con 0x555bddfa62f0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e74c00640 1 -- 172.21.15.27:0/1874667947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c06b050 con 0x555bddfa62f0 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 10 client.17046 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bde3fb6b0 con 0x555bddfa62f0 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e74c00640 1 -- 172.21.15.27:0/1874667947 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e840121c0 con 0x555bddfa62f0 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e74c00640 1 -- 172.21.15.27:0/1874667947 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0e7c003c30 con 0x555bddfa62f0 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 10 client.17046 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 20 client.17046 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf631c0 0x555bde3fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 4 client.17046 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bde2e70b0 con 0x555bddf631c0 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.920+0000 7f0e8a000640 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf631c0 0x555bde3fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.921+0000 7f0e8a000640 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf631c0 0x555bde3fddd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12cc3ebd73257036 server_cookie=81d9e63f89d15d98 in_seq=0 out_seq=0 2024-09-21T07:18:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.921+0000 7f0e74c00640 10 client.17046 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e74c00640 1 -- 172.21.15.27:0/1874667947 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bde2e70b0 con 0x555bddf631c0 2024-09-21T07:18:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e74c00640 10 client.17046 handle_command_reply: tid=0 2024-09-21T07:18:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 client.17046 shutdown 2024-09-21T07:18:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfa62f0 msgr2=0x555bde3f3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfa62f0 0x555bde3f3ee0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003fa0 tx=0x7f0e7c001e70 comp rx=0 tx=0).stop 2024-09-21T07:18:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 shutdown_connections 2024-09-21T07:18:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555bddf631c0 0x555bde3fddd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bddfcf1b0 0x555bde383020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bddfae4d0 0x555bde3f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e74c00640 0 client.17046 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 --2- 172.21.15.27:0/1874667947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bddfa62f0 0x555bde3f3ee0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 >> 172.21.15.27:0/1874667947 conn(0x555bddfc3300 msgr2=0x555bde374e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 shutdown_connections 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 1 -- 172.21.15.27:0/1874667947 wait complete. 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.934+0000 7f0e98a03740 20 client.17046 trim_cache size 0 max 16384 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:18:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:18:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:18:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:18:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e700b6700 msgr2=0x7f0e700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e700b6700 0x7f0e700b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0e84012680 tx=0x7f0e84009500 comp rx=0 tx=0).stop 2024-09-21T07:18:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c109230 msgr2=0x7f0e8c19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c109230 0x7f0e8c19ca90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e9c0 tx=0x7f0e7800edc0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 shutdown_connections 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e700b6700 0x7f0e700b8c10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e8c109b80 0x7f0e8c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e8c109230 0x7f0e8c19ca90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 --2- 172.21.15.27:0/2567657466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8c103900 0x7f0e8c19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 >> 172.21.15.27:0/2567657466 conn(0x7f0e8c0ff360 msgr2=0x7f0e8c1088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 shutdown_connections 2024-09-21T07:18:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:29.935+0000 7f0e8be00640 1 -- 172.21.15.27:0/2567657466 wait complete. 2024-09-21T07:18:30.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:30 smithi153 ceph-mon[34747]: pgmap v542: 129 pgs: 129 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-21T07:18:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:30 smithi119 ceph-mon[34519]: pgmap v542: 129 pgs: 129 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-21T07:18:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:30 smithi027 ceph-mon[30779]: pgmap v542: 129 pgs: 129 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-21T07:18:32.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:32 smithi153 ceph-mon[34747]: pgmap v543: 129 pgs: 129 active+clean; 2.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T07:18:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:32 smithi119 ceph-mon[34519]: pgmap v543: 129 pgs: 129 active+clean; 2.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T07:18:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:32 smithi027 ceph-mon[30779]: pgmap v543: 129 pgs: 129 active+clean; 2.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T07:18:34.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:34 smithi153 ceph-mon[34747]: pgmap v544: 129 pgs: 129 active+clean; 2.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:18:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:34 smithi119 ceph-mon[34519]: pgmap v544: 129 pgs: 129 active+clean; 2.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:18:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:34 smithi027 ceph-mon[30779]: pgmap v544: 129 pgs: 129 active+clean; 2.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:18:36.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:36 smithi153 ceph-mon[34747]: pgmap v545: 129 pgs: 129 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-09-21T07:18:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:36 smithi119 ceph-mon[34519]: pgmap v545: 129 pgs: 129 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-09-21T07:18:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:36 smithi027 ceph-mon[30779]: pgmap v545: 129 pgs: 129 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-09-21T07:18:38.090 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:18:38.090 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T07:18:38.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:18:38.490 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:38.847 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:38 smithi027 ceph-mon[30779]: pgmap v546: 129 pgs: 129 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T07:18:38.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:38 smithi153 ceph-mon[34747]: pgmap v546: 129 pgs: 129 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T07:18:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:38 smithi119 ceph-mon[34519]: pgmap v546: 129 pgs: 129 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T07:18:39.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.154+0000 7f78ac200640 1 -- 172.21.15.27:0/907397793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 msgr2=0x7f78a410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.154+0000 7f78ac200640 1 --2- 172.21.15.27:0/907397793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 0x7f78a410fc00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7890009900 tx=0x7f789002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.154+0000 7f78ac200640 1 -- 172.21.15.27:0/907397793 shutdown_connections 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.154+0000 7f78ac200640 1 --2- 172.21.15.27:0/907397793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78a4110140 0x7f78a4112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.154+0000 7f78ac200640 1 --2- 172.21.15.27:0/907397793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 0x7f78a410fc00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.154+0000 7f78ac200640 1 --2- 172.21.15.27:0/907397793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4075730 0x7f78a4075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.154+0000 7f78ac200640 1 -- 172.21.15.27:0/907397793 >> 172.21.15.27:0/907397793 conn(0x7f78a407b850 msgr2=0x7f78a407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.155+0000 7f78ac200640 1 -- 172.21.15.27:0/907397793 shutdown_connections 2024-09-21T07:18:39.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.155+0000 7f78ac200640 1 -- 172.21.15.27:0/907397793 wait complete. 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.156+0000 7f78ac200640 1 Processor -- start 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.156+0000 7f78ac200640 1 -- start start 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.156+0000 7f78ac200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78a4075730 0x7f78a419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.156+0000 7f78ac200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 0x7f78a419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78ac200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4110140 0x7f78a41a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78ac200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4115c50 con 0x7f78a4076080 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78ac200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4115ad0 con 0x7f78a4075730 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78ac200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4115dd0 con 0x7f78a4110140 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 0x7f78a419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 0x7f78a419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39536/0 (socket says 172.21.15.27:39536) 2024-09-21T07:18:39.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a8e00640 1 -- 172.21.15.27:0/4114488213 learned_addr learned my addr 172.21.15.27:0/4114488213 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a9800640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78a4075730 0x7f78a419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78aa200640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4110140 0x7f78a41a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a9800640 1 -- 172.21.15.27:0/4114488213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4110140 msgr2=0x7f78a41a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a9800640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4110140 0x7f78a41a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a9800640 1 -- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 msgr2=0x7f78a419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.157+0000 7f78a9800640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 0x7f78a419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.158+0000 7f78a9800640 1 -- 172.21.15.27:0/4114488213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a41a2610 con 0x7f78a4075730 2024-09-21T07:18:39.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.158+0000 7f78a9800640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78a4075730 0x7f78a419c6a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f789800e9f0 tx=0x7f789800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:39.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.158+0000 7f78a2000640 1 -- 172.21.15.27:0/4114488213 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789800ccc0 con 0x7f78a4075730 2024-09-21T07:18:39.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.158+0000 7f78a2000640 1 -- 172.21.15.27:0/4114488213 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78980108e0 con 0x7f78a4075730 2024-09-21T07:18:39.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.158+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a41a2900 con 0x7f78a4075730 2024-09-21T07:18:39.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.158+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41a9060 con 0x7f78a4075730 2024-09-21T07:18:39.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.158+0000 7f78a2000640 1 -- 172.21.15.27:0/4114488213 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898018840 con 0x7f78a4075730 2024-09-21T07:18:39.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.160+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78840052c0 con 0x7f78a4075730 2024-09-21T07:18:39.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.162+0000 7f78a2000640 1 -- 172.21.15.27:0/4114488213 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7898010430 con 0x7f78a4075730 2024-09-21T07:18:39.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.164+0000 7f78a2000640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f788c0b6680 0x7f788c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:39.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.164+0000 7f78a2000640 1 -- 172.21.15.27:0/4114488213 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f78980e7c90 con 0x7f78a4075730 2024-09-21T07:18:39.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.164+0000 7f78a8e00640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f788c0b6680 0x7f788c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:39.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.165+0000 7f78a8e00640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f788c0b6680 0x7f788c0b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f78a41119d0 tx=0x7f7890009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.165+0000 7f78a2000640 1 -- 172.21.15.27:0/4114488213 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78980ade30 con 0x7f78a4075730 2024-09-21T07:18:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.403+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78840055b0 con 0x7f78a4075730 2024-09-21T07:18:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.404+0000 7f78a2000640 1 -- 172.21.15.27:0/4114488213 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f78980b2210 con 0x7f78a4075730 2024-09-21T07:18:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:18:39.407 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:18:39.407 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:18:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f788c0b6680 msgr2=0x7f788c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f788c0b6680 0x7f788c0b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f78a41119d0 tx=0x7f7890009560 comp rx=0 tx=0).stop 2024-09-21T07:18:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78a4075730 msgr2=0x7f78a419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78a4075730 0x7f78a419c6a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f789800e9f0 tx=0x7f789800edf0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 shutdown_connections 2024-09-21T07:18:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78a4110140 0x7f78a41a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f788c0b6680 0x7f788c0b8b90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78a4076080 0x7f78a419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 --2- 172.21.15.27:0/4114488213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78a4075730 0x7f78a419c6a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 >> 172.21.15.27:0/4114488213 conn(0x7f78a407b850 msgr2=0x7f78a41019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 shutdown_connections 2024-09-21T07:18:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:39.409+0000 7f78ac200640 1 -- 172.21.15.27:0/4114488213 wait complete. 2024-09-21T07:18:39.779 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:39 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4114488213' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:39.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:39 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4114488213' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:40.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:39 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4114488213' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:40.311 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T07:18:40.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T07:18:40.718 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:40.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:40 smithi027 ceph-mon[30779]: pgmap v547: 129 pgs: 129 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-21T07:18:40.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:40 smithi153 ceph-mon[34747]: pgmap v547: 129 pgs: 129 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-21T07:18:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:40 smithi119 ceph-mon[34519]: pgmap v547: 129 pgs: 129 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 -- 172.21.15.27:0/421149120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c105dc0 msgr2=0x7f3a3c1061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 --2- 172.21.15.27:0/421149120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c105dc0 0x7f3a3c1061d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009900 tx=0x7f3a2c02a410 comp rx=0 tx=0).stop 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 -- 172.21.15.27:0/421149120 shutdown_connections 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 --2- 172.21.15.27:0/421149120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a3c10c050 0x7f3a3c112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 --2- 172.21.15.27:0/421149120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a3c10b700 0x7f3a3c10bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 --2- 172.21.15.27:0/421149120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c105dc0 0x7f3a3c1061d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 -- 172.21.15.27:0/421149120 >> 172.21.15.27:0/421149120 conn(0x7f3a3c1017d0 msgr2=0x7f3a3c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 -- 172.21.15.27:0/421149120 shutdown_connections 2024-09-21T07:18:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.378+0000 7f3a42200640 1 -- 172.21.15.27:0/421149120 wait complete. 2024-09-21T07:18:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.379+0000 7f3a42200640 1 Processor -- start 2024-09-21T07:18:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.379+0000 7f3a42200640 1 -- start start 2024-09-21T07:18:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a42200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a3c105dc0 0x7f3a3c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a42200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c10b700 0x7f3a3c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a42200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a3c10c050 0x7f3a3c1a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a42200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c115d10 con 0x7f3a3c10b700 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a42200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c115b90 con 0x7f3a3c105dc0 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a42200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c115e90 con 0x7f3a3c10c050 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a32a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c10b700 0x7f3a3c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a32a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c10b700 0x7f3a3c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39552/0 (socket says 172.21.15.27:39552) 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a32a00640 1 -- 172.21.15.27:0/3622559079 learned_addr learned my addr 172.21.15.27:0/3622559079 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a3c10c050 0x7f3a3c1a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.380+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a3c105dc0 0x7f3a3c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.381+0000 7f3a32a00640 1 -- 172.21.15.27:0/3622559079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a3c10c050 msgr2=0x7f3a3c1a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.381+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a3c10c050 0x7f3a3c1a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.381+0000 7f3a32a00640 1 -- 172.21.15.27:0/3622559079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a3c105dc0 msgr2=0x7f3a3c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.381+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a3c105dc0 0x7f3a3c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.381+0000 7f3a32a00640 1 -- 172.21.15.27:0/3622559079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c1a2710 con 0x7f3a3c10b700 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.381+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c10b700 0x7f3a3c19cd40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400e9f0 tx=0x7f3a2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.381+0000 7f3a38c00640 1 -- 172.21.15.27:0/3622559079 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400ccc0 con 0x7f3a3c10b700 2024-09-21T07:18:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.382+0000 7f3a38c00640 1 -- 172.21.15.27:0/3622559079 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a240108e0 con 0x7f3a3c10b700 2024-09-21T07:18:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.382+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3c1a2a00 con 0x7f3a3c10b700 2024-09-21T07:18:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.382+0000 7f3a38c00640 1 -- 172.21.15.27:0/3622559079 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24018880 con 0x7f3a3c10b700 2024-09-21T07:18:41.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.382+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3c1a9160 con 0x7f3a3c10b700 2024-09-21T07:18:41.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.385+0000 7f3a48eb7740 1 Processor -- start 2024-09-21T07:18:41.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.385+0000 7f3a48eb7740 1 -- start start 2024-09-21T07:18:41.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a38c00640 1 -- 172.21.15.27:0/3622559079 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3a24010430 con 0x7f3a3c10b700 2024-09-21T07:18:41.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf591ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e2f990 0x55bf591ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e37b70 0x55bf59200430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf59175a90 con 0x55bf58e58850 2024-09-21T07:18:41.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bf59164ec0 con 0x55bf58e37b70 2024-09-21T07:18:41.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf59168cd0 con 0x55bf58e2f990 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a3be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf591ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a3be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf591ff9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39558/0 (socket says 172.21.15.27:39558) 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a32a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e37b70 0x55bf59200430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a3be00640 1 -- 172.21.15.27:0/3189550837 learned_addr learned my addr 172.21.15.27:0/3189550837 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.386+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e2f990 0x55bf591ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a3be00640 1 -- 172.21.15.27:0/3189550837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e2f990 msgr2=0x55bf591ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e2f990 0x55bf591ffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a3be00640 1 -- 172.21.15.27:0/3189550837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e37b70 msgr2=0x55bf59200430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e37b70 0x55bf59200430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a3be00640 1 -- 172.21.15.27:0/3189550837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5910a9d0 con 0x55bf58e58850 2024-09-21T07:18:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e37b70 0x55bf59200430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:41.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf591ff9b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009f60 tx=0x7f3a34009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e05fb2dbd448c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a2be00640 1 -- 172.21.15.27:0/3189550837 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3401e070 con 0x55bf58e58850 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.387+0000 7f3a2be00640 1 -- 172.21.15.27:0/3189550837 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a34003c30 con 0x55bf58e58850 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.388+0000 7f3a2be00640 1 -- 172.21.15.27:0/3189550837 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400f840 con 0x55bf58e58850 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.388+0000 7f3a38c00640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3a200b6680 0x7f3a200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.388+0000 7f3a38c00640 1 -- 172.21.15.27:0/3622559079 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3a240e7e30 con 0x7f3a3c10b700 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.388+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3a200b6680 0x7f3a200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.388+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3189550837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 msgr2=0x55bf591ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.388+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf591ff9b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009f60 tx=0x7f3a34009f90 comp rx=0 tx=0).stop 2024-09-21T07:18:41.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3189550837 shutdown_connections 2024-09-21T07:18:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e2f990 0x55bf591ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e37b70 0x55bf59200430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3189550837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf591ff9b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3189550837 >> 172.21.15.27:0/3189550837 conn(0x55bf58e4cc40 msgr2=0x55bf591fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3189550837 shutdown_connections 2024-09-21T07:18:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3189550837 wait complete. 2024-09-21T07:18:41.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.389+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3a200b6680 0x7f3a200b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c031440 tx=0x7f3a2c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 -- start start 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 Processor -- start 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf5920b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf592174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf58fc67b0 con 0x55bf58e2f990 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bf59109650 con 0x55bf58e58850 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.390+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf5920e190 con 0x55bf58e37b70 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a48eb7740 10 client.0 _ll_register_callbacks cb 0x55bf58e700a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:41.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf5920b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a32a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf5920b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39570/0 (socket says 172.21.15.27:39570) 2024-09-21T07:18:41.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a3b400640 1 -- 172.21.15.27:0/3252885996 learned_addr learned my addr 172.21.15.27:0/3252885996 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:41.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf592174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a32a00640 1 -- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 msgr2=0x55bf592174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf592174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a32a00640 1 -- 172.21.15.27:0/3252885996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 msgr2=0x55bf5920b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf5920b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.391+0000 7f3a32a00640 1 -- 172.21.15.27:0/3252885996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf58ed2ba0 con 0x55bf58e37b70 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920b9d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a240e51d0 tx=0x7f3a24010cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc412ed5c607311 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a28c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a28c00640 1 -- 172.21.15.27:0/3252885996 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a240ad220 con 0x55bf58e37b70 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a34003c30 con 0x55bf58e37b70 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a28c00640 1 -- 172.21.15.27:0/3252885996 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3a240a9420 con 0x55bf58e37b70 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a28c00640 1 -- 172.21.15.27:0/3252885996 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a240e5660 con 0x55bf58e37b70 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a48eb7740 10 client.27082 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf5921a0e0 con 0x55bf58e37b70 2024-09-21T07:18:41.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.392+0000 7f3a28c00640 1 -- 172.21.15.27:0/3252885996 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a34003c30 con 0x55bf58e37b70 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a28c00640 1 -- 172.21.15.27:0/3252885996 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3a240b3540 con 0x55bf58e37b70 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a48eb7740 10 client.27082 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a48eb7740 10 client.27082 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a48eb7740 20 client.27082 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58e171d0 0x55bf5921d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a48eb7740 4 client.27082 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf591779e0 con 0x55bf58e171d0 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.393+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58e171d0 0x55bf5921d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.394+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58e171d0 0x55bf5921d540 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df52a3cad7b7af3e server_cookie=546bf45ba540d224 in_seq=0 out_seq=0 2024-09-21T07:18:41.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.394+0000 7f3a28c00640 10 client.27082 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:41.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.394+0000 7f3a28c00640 1 -- 172.21.15.27:0/3252885996 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf591779e0 con 0x55bf58e171d0 2024-09-21T07:18:41.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a28c00640 10 client.27082 handle_command_reply: tid=0 2024-09-21T07:18:41.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 client.27082 shutdown 2024-09-21T07:18:41.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 msgr2=0x55bf5920b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920b9d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a240e51d0 tx=0x7f3a24010cf0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 shutdown_connections 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58e171d0 0x55bf5921d540 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920b9d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf592174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3252885996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf5920b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 >> 172.21.15.27:0/3252885996 conn(0x55bf58e4cc40 msgr2=0x55bf59202b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a28c00640 0 client.27082 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 shutdown_connections 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3252885996 wait complete. 2024-09-21T07:18:41.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.395+0000 7f3a48eb7740 20 client.27082 trim_cache size 0 max 16384 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.414+0000 7f3a48eb7740 1 Processor -- start 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.414+0000 7f3a48eb7740 1 -- start start 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e2f990 0x55bf5920cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf59208480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf592089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b5950 con 0x55bf58e58850 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b3220 con 0x55bf58e2f990 2024-09-21T07:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b1070 con 0x55bf58e37b70 2024-09-21T07:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a3be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf592089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a3be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf592089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39572/0 (socket says 172.21.15.27:39572) 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a3be00640 1 -- 172.21.15.27:0/3730633802 learned_addr learned my addr 172.21.15.27:0/3730633802 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e2f990 0x55bf5920cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.415+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf59208480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.416+0000 7f3a3be00640 1 -- 172.21.15.27:0/3730633802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 msgr2=0x55bf59208480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.416+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf59208480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.416+0000 7f3a3be00640 1 -- 172.21.15.27:0/3730633802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e2f990 msgr2=0x55bf5920cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.416+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e2f990 0x55bf5920cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.416+0000 7f3a3be00640 1 -- 172.21.15.27:0/3730633802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5920c330 con 0x55bf58e58850 2024-09-21T07:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.416+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf592089c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009b80 tx=0x7f3a3400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431d3464d841ddfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.416+0000 7f3a2b400640 1 -- 172.21.15.27:0/3730633802 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3401e040 con 0x55bf58e58850 2024-09-21T07:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a2b400640 1 -- 172.21.15.27:0/3730633802 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3401f040 con 0x55bf58e58850 2024-09-21T07:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a2b400640 1 -- 172.21.15.27:0/3730633802 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400f590 con 0x55bf58e58850 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3730633802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 msgr2=0x55bf592089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf592089c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009b80 tx=0x7f3a3400f520 comp rx=0 tx=0).stop 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3730633802 shutdown_connections 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf59208480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e58850 0x55bf592089c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.417+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3730633802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e2f990 0x55bf5920cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.418+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3730633802 >> 172.21.15.27:0/3730633802 conn(0x55bf58e4cc40 msgr2=0x55bf5920c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.418+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3730633802 shutdown_connections 2024-09-21T07:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.418+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3730633802 wait complete. 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.418+0000 7f3a48eb7740 1 -- start start 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.418+0000 7f3a48eb7740 1 Processor -- start 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf59213b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a48eb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf5920e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b16e0 con 0x55bf58e2f990 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b2b50 con 0x55bf58e58850 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a48eb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b1db0 con 0x55bf58e37b70 2024-09-21T07:18:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a48eb7740 10 client.0 _ll_register_callbacks cb 0x7f3a240eb5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf59213b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf59213b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39582/0 (socket says 172.21.15.27:39582) 2024-09-21T07:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a3b400640 1 -- 172.21.15.27:0/3474224745 learned_addr learned my addr 172.21.15.27:0/3474224745 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.419+0000 7f3a3be00640 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf5920e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a3b400640 1 -- 172.21.15.27:0/3474224745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 msgr2=0x55bf5920e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a3b400640 1 -- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 msgr2=0x55bf5920e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf5920e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a3b400640 1 -- 172.21.15.27:0/3474224745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5920efb0 con 0x55bf58e2f990 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a3b400640 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf59213b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009230 tx=0x7f3a2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d9d29ed3c1b92b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a28c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a28c00640 1 -- 172.21.15.27:0/3474224745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c05f060 con 0x55bf58e2f990 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.420+0000 7f3a28c00640 1 -- 172.21.15.27:0/3474224745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f3a2c005040 con 0x55bf58e2f990 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a3401f040 con 0x55bf58e2f990 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a28c00640 1 -- 172.21.15.27:0/3474224745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a2c066050 con 0x55bf58e2f990 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a48eb7740 10 client.17070 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf59288dc0 con 0x55bf58e2f990 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a28c00640 1 -- 172.21.15.27:0/3474224745 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3401f040 con 0x55bf58e2f990 2024-09-21T07:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a28c00640 1 -- 172.21.15.27:0/3474224745 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3a2c0035f0 con 0x55bf58e2f990 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a48eb7740 10 client.17070 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a48eb7740 10 client.17070 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a48eb7740 20 client.17070 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.421+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58dec580 0x55bf5928b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.422+0000 7f3a48eb7740 4 client.17070 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.422+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x55bf59173490 con 0x55bf58dec580 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.422+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58dec580 0x55bf5928b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.422+0000 7f3a32a00640 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58dec580 0x55bf5928b5d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc443f0df801c610 server_cookie=565190dcb0b65606 in_seq=0 out_seq=0 2024-09-21T07:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.422+0000 7f3a28c00640 10 client.17070 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:41.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.434+0000 7f3a28c00640 1 -- 172.21.15.27:0/3474224745 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bf59173490 con 0x55bf58dec580 2024-09-21T07:18:41.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.434+0000 7f3a28c00640 10 client.17070 handle_command_reply: tid=0 2024-09-21T07:18:41.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.434+0000 7f3a48eb7740 1 client.17070 shutdown 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.434+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 msgr2=0x55bf59213b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.434+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf59213b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009230 tx=0x7f3a2c009260 comp rx=0 tx=0).stop 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 shutdown_connections 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf58dec580 0x55bf5928b5d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf58e37b70 0x55bf5920e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf58e58850 0x55bf5920e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a28c00640 0 client.17070 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 --2- 172.21.15.27:0/3474224745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf58e2f990 0x55bf59213b10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 >> 172.21.15.27:0/3474224745 conn(0x55bf58e4cc40 msgr2=0x55bf592009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 shutdown_connections 2024-09-21T07:18:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 1 -- 172.21.15.27:0/3474224745 wait complete. 2024-09-21T07:18:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.435+0000 7f3a48eb7740 20 client.17070 trim_cache size 0 max 16384 2024-09-21T07:18:41.437 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:18:41.437 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:18:41.437 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 22, 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "7b1b6a1a": { 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 22, 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:41.438 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:18:41.439 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T07:18:41.439 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:18:41.439 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:18:41.439 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:18:41.439 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:18:41.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:18:41.440 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:18:41.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:41.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:41.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:41.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:41.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:41.440 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:18:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3a200b6680 msgr2=0x7f3a200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3a200b6680 0x7f3a200b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c031440 tx=0x7f3a2c009560 comp rx=0 tx=0).stop 2024-09-21T07:18:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c10b700 msgr2=0x7f3a3c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c10b700 0x7f3a3c19cd40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400e9f0 tx=0x7f3a2400edf0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 shutdown_connections 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a3c10c050 0x7f3a3c1a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3a200b6680 0x7f3a200b8b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a3c10b700 0x7f3a3c19cd40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 --2- 172.21.15.27:0/3622559079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a3c105dc0 0x7f3a3c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 >> 172.21.15.27:0/3622559079 conn(0x7f3a3c1017d0 msgr2=0x7f3a3c109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.436+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 shutdown_connections 2024-09-21T07:18:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:41.437+0000 7f3a42200640 1 -- 172.21.15.27:0/3622559079 wait complete. 2024-09-21T07:18:42.305 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7b1b6a1a', quiesce took 4.2 seconds. Will release after: 15.05 2024-09-21T07:18:42.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:42 smithi153 ceph-mon[34747]: pgmap v548: 129 pgs: 129 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T07:18:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:42 smithi119 ceph-mon[34519]: pgmap v548: 129 pgs: 129 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T07:18:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:42 smithi027 ceph-mon[30779]: pgmap v548: 129 pgs: 129 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T07:18:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.137+0000 7fd333200640 1 -- 172.21.15.27:0/2784084072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 msgr2=0x7fd334105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.137+0000 7fd333200640 1 --2- 172.21.15.27:0/2784084072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd334105f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd324009900 tx=0x7fd32402a410 comp rx=0 tx=0).stop 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.137+0000 7fd333200640 1 -- 172.21.15.27:0/2784084072 shutdown_connections 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.137+0000 7fd333200640 1 --2- 172.21.15.27:0/2784084072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd33410bda0 0x7fd3341125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.137+0000 7fd333200640 1 --2- 172.21.15.27:0/2784084072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33410b450 0x7fd33410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.137+0000 7fd333200640 1 --2- 172.21.15.27:0/2784084072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd334105f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.138+0000 7fd333200640 1 -- 172.21.15.27:0/2784084072 >> 172.21.15.27:0/2784084072 conn(0x7fd334101560 msgr2=0x7fd3341039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.138+0000 7fd333200640 1 -- 172.21.15.27:0/2784084072 shutdown_connections 2024-09-21T07:18:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.138+0000 7fd333200640 1 -- 172.21.15.27:0/2784084072 wait complete. 2024-09-21T07:18:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 Processor -- start 2024-09-21T07:18:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 -- start start 2024-09-21T07:18:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd33419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33410b450 0x7fd33419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd33410bda0 0x7fd3341a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd334115a60 con 0x7fd334105b20 2024-09-21T07:18:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3341158e0 con 0x7fd33410b450 2024-09-21T07:18:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.139+0000 7fd333200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd334115be0 con 0x7fd33410bda0 2024-09-21T07:18:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd33419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd33419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39598/0 (socket says 172.21.15.27:39598) 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 -- 172.21.15.27:0/1424094291 learned_addr learned my addr 172.21.15.27:0/1424094291 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd332800640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd33410bda0 0x7fd3341a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331400640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33410b450 0x7fd33419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 -- 172.21.15.27:0/1424094291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd33410bda0 msgr2=0x7fd3341a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd33410bda0 0x7fd3341a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 -- 172.21.15.27:0/1424094291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33410b450 msgr2=0x7fd33419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33410b450 0x7fd33419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 -- 172.21.15.27:0/1424094291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3341a23f0 con 0x7fd334105b20 2024-09-21T07:18:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.140+0000 7fd331e00640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd33419c4c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd324005940 tx=0x7fd324004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.141+0000 7fd32aa00640 1 -- 172.21.15.27:0/1424094291 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32403f030 con 0x7fd334105b20 2024-09-21T07:18:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.141+0000 7fd32aa00640 1 -- 172.21.15.27:0/1424094291 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd324003aa0 con 0x7fd334105b20 2024-09-21T07:18:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.141+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3341a2680 con 0x7fd334105b20 2024-09-21T07:18:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.141+0000 7fd32aa00640 1 -- 172.21.15.27:0/1424094291 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324035070 con 0x7fd334105b20 2024-09-21T07:18:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.141+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3341a8e70 con 0x7fd334105b20 2024-09-21T07:18:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.144+0000 7fd33ffe9740 1 Processor -- start 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.144+0000 7fd33ffe9740 1 -- start start 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.144+0000 7fd32aa00640 1 -- 172.21.15.27:0/1424094291 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd324042030 con 0x7fd334105b20 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64b1b1b0 0x564b64ebfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ec0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ec0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564b64e356b0 con 0x564b64b1b1b0 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564b64e24ae0 con 0x564b64afa4d0 2024-09-21T07:18:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564b64e288f0 con 0x564b64af22f0 2024-09-21T07:18:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64b1b1b0 0x564b64ebfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd331e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ec0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64b1b1b0 0x564b64ebfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39612/0 (socket says 172.21.15.27:39612) 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd332800640 1 -- 172.21.15.27:0/1036298149 learned_addr learned my addr 172.21.15.27:0/1036298149 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd331400640 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ec0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd332800640 1 -- 172.21.15.27:0/1036298149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 msgr2=0x564b64ec0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd332800640 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ec0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd332800640 1 -- 172.21.15.27:0/1036298149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 msgr2=0x564b64ec0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.145+0000 7fd332800640 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ec0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd332800640 1 -- 172.21.15.27:0/1036298149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b64df1780 con 0x564b64b1b1b0 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd332800640 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64b1b1b0 0x564b64ebfe10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0039f0 tx=0x7fd32c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3507149df48e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd31f400640 1 -- 172.21.15.27:0/1036298149 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c01a470 con 0x564b64b1b1b0 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd32aa00640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3180b6680 0x7fd3180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd31f400640 1 -- 172.21.15.27:0/1036298149 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd32c015070 con 0x564b64b1b1b0 2024-09-21T07:18:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd31f400640 1 -- 172.21.15.27:0/1036298149 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c010860 con 0x564b64b1b1b0 2024-09-21T07:18:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd32aa00640 1 -- 172.21.15.27:0/1424094291 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd324108160 con 0x7fd334105b20 2024-09-21T07:18:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.146+0000 7fd331e00640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3180b6680 0x7fd3180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1036298149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64b1b1b0 msgr2=0x564b64ebfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64b1b1b0 0x564b64ebfe10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0039f0 tx=0x7fd32c003eb0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1036298149 shutdown_connections 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ec0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ec0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1036298149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64b1b1b0 0x564b64ebfe10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1036298149 >> 172.21.15.27:0/1036298149 conn(0x564b64b0f300 msgr2=0x564b64ebdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd331e00640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3180b6680 0x7fd3180b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x564b64ebea90 tx=0x7fd32410bc10 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1036298149 shutdown_connections 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.147+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1036298149 wait complete. 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.148+0000 7fd33ffe9740 1 -- start start 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.148+0000 7fd33ffe9740 1 Processor -- start 2024-09-21T07:18:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.148+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64af22f0 0x564b64ecb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.148+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ecb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ecbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564b64d5cab0 con 0x564b64afa4d0 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564b64dcb570 con 0x564b64af22f0 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564b64ece590 con 0x564b64b1b1b0 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd33ffe9740 10 client.0 _ll_register_callbacks cb 0x564b64dda100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ecb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ecb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39616/0 (socket says 172.21.15.27:39616) 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 -- 172.21.15.27:0/1070403742 learned_addr learned my addr 172.21.15.27:0/1070403742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd332800640 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64af22f0 0x564b64ecb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd332800640 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ecbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 -- 172.21.15.27:0/1070403742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 msgr2=0x564b64ecbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ecbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 -- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64af22f0 msgr2=0x564b64ecb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64af22f0 0x564b64ecb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 -- 172.21.15.27:0/1070403742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b64b86790 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd331400640 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ecb970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd320012700 tx=0x7fd320012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee56d7015550506 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.149+0000 7fd313e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.150+0000 7fd313e00640 1 -- 172.21.15.27:0/1070403742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32000f920 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.150+0000 7fd313e00640 1 -- 172.21.15.27:0/1070403742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd32000fac0 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.150+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd32c015070 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.150+0000 7fd313e00640 1 -- 172.21.15.27:0/1070403742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd32000fcc0 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.150+0000 7fd33ffe9740 10 client.17082 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.150+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b64bff620 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.150+0000 7fd313e00640 1 -- 172.21.15.27:0/1070403742 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd32c015070 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd313e00640 1 -- 172.21.15.27:0/1070403742 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd32001c420 con 0x564b64afa4d0 2024-09-21T07:18:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd33ffe9740 10 client.17082 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd33ffe9740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd33ffe9740 20 client.17082 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64ad9d80 0x564b64edd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd33ffe9740 4 client.17082 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b64e37600 con 0x564b64ad9d80 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.151+0000 7fd332800640 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64ad9d80 0x564b64edd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.152+0000 7fd332800640 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64ad9d80 0x564b64edd610 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15708d0bcab6411d server_cookie=691baad88a6de024 in_seq=0 out_seq=0 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.152+0000 7fd313e00640 10 client.17082 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd313e00640 1 -- 172.21.15.27:0/1070403742 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564b64e37600 con 0x564b64ad9d80 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd313e00640 10 client.17082 handle_command_reply: tid=0 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 client.17082 shutdown 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 msgr2=0x564b64ecb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ecb970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd320012700 tx=0x7fd320012bc0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 shutdown_connections 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64ad9d80 0x564b64edd610 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ecbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ecb970 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1070403742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64af22f0 0x564b64ecb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd313e00640 0 client.17082 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 >> 172.21.15.27:0/1070403742 conn(0x564b64b0f300 msgr2=0x564b64ec0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 shutdown_connections 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1070403742 wait complete. 2024-09-21T07:18:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.153+0000 7fd33ffe9740 20 client.17082 trim_cache size 0 max 16384 2024-09-21T07:18:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 Processor -- start 2024-09-21T07:18:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 -- start start 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ebcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ec1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64b1b1b0 0x564b64ec8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd3341b56b0 con 0x564b64afa4d0 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3341b2f80 con 0x564b64b1b1b0 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3341b0dd0 con 0x564b64af22f0 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd331400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ec1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd331400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ec1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39624/0 (socket says 172.21.15.27:39624) 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.169+0000 7fd331400640 1 -- 172.21.15.27:0/1807040021 learned_addr learned my addr 172.21.15.27:0/1807040021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd332800640 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ebcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd332800640 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64b1b1b0 0x564b64ec8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd331400640 1 -- 172.21.15.27:0/1807040021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 msgr2=0x564b64ebcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd331400640 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ebcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd331400640 1 -- 172.21.15.27:0/1807040021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64b1b1b0 msgr2=0x564b64ec8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd331400640 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64b1b1b0 0x564b64ec8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd331400640 1 -- 172.21.15.27:0/1807040021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b64ec2c20 con 0x564b64afa4d0 2024-09-21T07:18:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd331400640 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ec1550 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd320001eb0 tx=0x7fd32000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b947a626b8bec784 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd31ea00640 1 -- 172.21.15.27:0/1807040021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32000fe60 con 0x564b64afa4d0 2024-09-21T07:18:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.170+0000 7fd31ea00640 1 -- 172.21.15.27:0/1807040021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd32000f920 con 0x564b64afa4d0 2024-09-21T07:18:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd31ea00640 1 -- 172.21.15.27:0/1807040021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3200055e0 con 0x564b64afa4d0 2024-09-21T07:18:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1807040021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 msgr2=0x564b64ec1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ec1550 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd320001eb0 tx=0x7fd32000f850 comp rx=0 tx=0).stop 2024-09-21T07:18:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1807040021 shutdown_connections 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64b1b1b0 0x564b64ec8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64afa4d0 0x564b64ec1550 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/1807040021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64af22f0 0x564b64ebcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1807040021 >> 172.21.15.27:0/1807040021 conn(0x564b64b0f300 msgr2=0x564b64ec0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1807040021 shutdown_connections 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.171+0000 7fd33ffe9740 1 -- 172.21.15.27:0/1807040021 wait complete. 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 -- start start 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 Processor -- start 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64af22f0 0x564b64ed40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ed1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ed1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd3341b1440 con 0x564b64af22f0 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3341b28b0 con 0x564b64afa4d0 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3341b1b10 con 0x564b64b1b1b0 2024-09-21T07:18:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.172+0000 7fd33ffe9740 10 client.0 _ll_register_callbacks cb 0x7fd320009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64af22f0 0x564b64ed40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64af22f0 0x564b64ed40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39638/0 (socket says 172.21.15.27:39638) 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 -- 172.21.15.27:0/2046433329 learned_addr learned my addr 172.21.15.27:0/2046433329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd331400640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ed1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ed1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 -- 172.21.15.27:0/2046433329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 msgr2=0x564b64ed1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ed1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 -- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 msgr2=0x564b64ed1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ed1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd332800640 1 -- 172.21.15.27:0/2046433329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b64ed2350 con 0x564b64af22f0 2024-09-21T07:18:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.173+0000 7fd331400640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ed1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd332800640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64af22f0 0x564b64ed40b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd32c010510 tx=0x7fd32c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ded88e8f3b9912c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd313e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd313e00640 1 -- 172.21.15.27:0/2046433329 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0049e0 con 0x564b64af22f0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd313e00640 1 -- 172.21.15.27:0/2046433329 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fd32c004b80 con 0x564b64af22f0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd32000f920 con 0x564b64af22f0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd313e00640 1 -- 172.21.15.27:0/2046433329 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd32c004d80 con 0x564b64af22f0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd33ffe9740 10 client.17088 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b64f48060 con 0x564b64af22f0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.174+0000 7fd313e00640 1 -- 172.21.15.27:0/2046433329 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd32000f920 con 0x564b64af22f0 2024-09-21T07:18:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd313e00640 1 -- 172.21.15.27:0/2046433329 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd32c010670 con 0x564b64af22f0 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd33ffe9740 10 client.17088 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd33ffe9740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd33ffe9740 20 client.17088 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64aaf1c0 0x564b64f4a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd33ffe9740 4 client.17088 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b64e330b0 con 0x564b64aaf1c0 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.175+0000 7fd331400640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64aaf1c0 0x564b64f4a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.176+0000 7fd331400640 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64aaf1c0 0x564b64f4a6c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50e07f6a424cb5f server_cookie=1378b454652d3d8 in_seq=0 out_seq=0 2024-09-21T07:18:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.176+0000 7fd313e00640 10 client.17088 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd313e00640 1 -- 172.21.15.27:0/2046433329 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b64e330b0 con 0x564b64aaf1c0 2024-09-21T07:18:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd313e00640 10 client.17088 handle_command_reply: tid=0 2024-09-21T07:18:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 client.17088 shutdown 2024-09-21T07:18:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64af22f0 msgr2=0x564b64ed40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64af22f0 0x564b64ed40b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd32c010510 tx=0x7fd32c011f90 comp rx=0 tx=0).stop 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 shutdown_connections 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564b64aaf1c0 0x564b64f4a6c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564b64b1b1b0 0x564b64ed1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd313e00640 0 client.17088 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564b64afa4d0 0x564b64ed1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 --2- 172.21.15.27:0/2046433329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564b64af22f0 0x564b64ed40b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 >> 172.21.15.27:0/2046433329 conn(0x564b64b0f300 msgr2=0x564b64ec1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 shutdown_connections 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 1 -- 172.21.15.27:0/2046433329 wait complete. 2024-09-21T07:18:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.190+0000 7fd33ffe9740 20 client.17088 trim_cache size 0 max 16384 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3180b6680 msgr2=0x7fd3180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3180b6680 0x7fd3180b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x564b64ebea90 tx=0x7fd32410bc10 comp rx=0 tx=0).stop 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 msgr2=0x7fd33419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd33419c4c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd324005940 tx=0x7fd324004270 comp rx=0 tx=0).stop 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 shutdown_connections 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd33410bda0 0x7fd3341a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3180b6680 0x7fd3180b8b90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33410b450 0x7fd33419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 --2- 172.21.15.27:0/1424094291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd334105b20 0x7fd33419c4c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 >> 172.21.15.27:0/1424094291 conn(0x7fd334101560 msgr2=0x7fd334103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 shutdown_connections 2024-09-21T07:18:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:43.191+0000 7fd333200640 1 -- 172.21.15.27:0/1424094291 wait complete. 2024-09-21T07:18:44.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:44 smithi153 ceph-mon[34747]: pgmap v549: 129 pgs: 129 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-21T07:18:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:44 smithi119 ceph-mon[34519]: pgmap v549: 129 pgs: 129 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-21T07:18:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:44 smithi027 ceph-mon[30779]: pgmap v549: 129 pgs: 129 active+clean; 2.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-21T07:18:46.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:46 smithi153 ceph-mon[34747]: pgmap v550: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-21T07:18:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:46 smithi119 ceph-mon[34519]: pgmap v550: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-21T07:18:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:46 smithi027 ceph-mon[30779]: pgmap v550: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-21T07:18:48.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:48 smithi153 ceph-mon[34747]: pgmap v551: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-21T07:18:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:48 smithi119 ceph-mon[34519]: pgmap v551: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-21T07:18:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:48 smithi027 ceph-mon[30779]: pgmap v551: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-21T07:18:50.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:50 smithi027 ceph-mon[30779]: pgmap v552: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-21T07:18:50.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:50 smithi153 ceph-mon[34747]: pgmap v552: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-21T07:18:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:50 smithi119 ceph-mon[34519]: pgmap v552: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-21T07:18:52.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:52 smithi153 ceph-mon[34747]: pgmap v553: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:18:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:52 smithi119 ceph-mon[34519]: pgmap v553: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:18:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:52 smithi027 ceph-mon[30779]: pgmap v553: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:18:53.525 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:18:53.930 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 -- 172.21.15.27:0/1815159789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10b4f0 msgr2=0x7efecc10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 --2- 172.21.15.27:0/1815159789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10b4f0 0x7efecc10b900 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009900 tx=0x7efeb802a410 comp rx=0 tx=0).stop 2024-09-21T07:18:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 -- 172.21.15.27:0/1815159789 shutdown_connections 2024-09-21T07:18:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 --2- 172.21.15.27:0/1815159789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc10be40 0x7efecc112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 --2- 172.21.15.27:0/1815159789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10b4f0 0x7efecc10b900 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 --2- 172.21.15.27:0/1815159789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efecc105bb0 0x7efecc105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 -- 172.21.15.27:0/1815159789 >> 172.21.15.27:0/1815159789 conn(0x7efecc101680 msgr2=0x7efecc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:54.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 -- 172.21.15.27:0/1815159789 shutdown_connections 2024-09-21T07:18:54.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.528+0000 7efed2600640 1 -- 172.21.15.27:0/1815159789 wait complete. 2024-09-21T07:18:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.529+0000 7efed2600640 1 Processor -- start 2024-09-21T07:18:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.529+0000 7efed2600640 1 -- start start 2024-09-21T07:18:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.529+0000 7efed2600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc105bb0 0x7efecc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.529+0000 7efed2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efecc10b4f0 0x7efecc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10be40 0x7efecc1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed2600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efecc115b00 con 0x7efecc10be40 2024-09-21T07:18:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed2600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efecc115980 con 0x7efecc10b4f0 2024-09-21T07:18:54.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed2600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efecc115c80 con 0x7efecc105bb0 2024-09-21T07:18:54.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10be40 0x7efecc1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10be40 0x7efecc1a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40288/0 (socket says 172.21.15.27:40288) 2024-09-21T07:18:54.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc105bb0 0x7efecc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1c00640 1 -- 172.21.15.27:0/1111999956 learned_addr learned my addr 172.21.15.27:0/1111999956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:54.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efecbe00640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efecc10b4f0 0x7efecc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1200640 1 -- 172.21.15.27:0/1111999956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efecc10b4f0 msgr2=0x7efecc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1200640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efecc10b4f0 0x7efecc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1200640 1 -- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10be40 msgr2=0x7efecc1a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1200640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10be40 0x7efecc1a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1200640 1 -- 172.21.15.27:0/1111999956 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc1a2480 con 0x7efecc105bb0 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.530+0000 7efed1c00640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10be40 0x7efecc1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.531+0000 7efed1200640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc105bb0 0x7efecc19c510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9f0 tx=0x7efec000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.531+0000 7efec9600640 1 -- 172.21.15.27:0/1111999956 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec000ccc0 con 0x7efecc105bb0 2024-09-21T07:18:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.531+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc1a2770 con 0x7efecc105bb0 2024-09-21T07:18:54.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.531+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc1a8ed0 con 0x7efecc105bb0 2024-09-21T07:18:54.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.532+0000 7efec9600640 1 -- 172.21.15.27:0/1111999956 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec00108e0 con 0x7efecc105bb0 2024-09-21T07:18:54.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.532+0000 7efec9600640 1 -- 172.21.15.27:0/1111999956 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0026060 con 0x7efecc105bb0 2024-09-21T07:18:54.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.534+0000 7efeda872740 1 Processor -- start 2024-09-21T07:18:54.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efeda872740 1 -- start start 2024-09-21T07:18:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efeda872740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb8e070 0x55820ff34f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efeda872740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efeda872740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb6d390 0x55820ff359b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efeda872740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55820feabf70 con 0x55820fb6d390 2024-09-21T07:18:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efeda872740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55820fe9b3a0 con 0x55820fb8e070 2024-09-21T07:18:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efeda872740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55820fe9f1b0 con 0x55820fb651b0 2024-09-21T07:18:54.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efecbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb6d390 0x55820ff359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efecbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb6d390 0x55820ff359b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40296/0 (socket says 172.21.15.27:40296) 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efecbe00640 1 -- 172.21.15.27:0/3135878044 learned_addr learned my addr 172.21.15.27:0/3135878044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.535+0000 7efed1c00640 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb8e070 0x55820ff34f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efec9600640 1 -- 172.21.15.27:0/1111999956 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efec0010430 con 0x7efecc105bb0 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efed1c00640 1 -- 172.21.15.27:0/3135878044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 msgr2=0x55820ff35470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efed1c00640 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff35470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efed1c00640 1 -- 172.21.15.27:0/3135878044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb6d390 msgr2=0x55820ff359b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efed1c00640 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb6d390 0x55820ff359b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efed1c00640 1 -- 172.21.15.27:0/3135878044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820fe65c90 con 0x55820fb8e070 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efecbe00640 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb6d390 0x55820ff359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.536+0000 7efed1c00640 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb8e070 0x55820ff34f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efec400e800 tx=0x7efec400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5ce07097b522ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.537+0000 7efebe000640 1 -- 172.21.15.27:0/3135878044 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4011c20 con 0x55820fb8e070 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.537+0000 7efebe000640 1 -- 172.21.15.27:0/3135878044 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec4011dc0 con 0x55820fb8e070 2024-09-21T07:18:54.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.537+0000 7efebe000640 1 -- 172.21.15.27:0/3135878044 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec400f6b0 con 0x55820fb8e070 2024-09-21T07:18:54.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 -- 172.21.15.27:0/3135878044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb8e070 msgr2=0x55820ff34f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb8e070 0x55820ff34f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efec400e800 tx=0x7efec400ecc0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 -- 172.21.15.27:0/3135878044 shutdown_connections 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff35470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb6d390 0x55820ff359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 --2- 172.21.15.27:0/3135878044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb8e070 0x55820ff34f30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 -- 172.21.15.27:0/3135878044 >> 172.21.15.27:0/3135878044 conn(0x55820fb82460 msgr2=0x55820ff32e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 -- 172.21.15.27:0/3135878044 shutdown_connections 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efeda872740 1 -- 172.21.15.27:0/3135878044 wait complete. 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efec9600640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efeb40b6590 0x7efeb40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.538+0000 7efec9600640 1 -- 172.21.15.27:0/1111999956 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7efec00e7c90 con 0x7efecc105bb0 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.539+0000 7efecbe00640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efeb40b6590 0x7efeb40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.539+0000 7efeda872740 1 -- start start 2024-09-21T07:18:54.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.539+0000 7efeda872740 1 Processor -- start 2024-09-21T07:18:54.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.539+0000 7efecbe00640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efeb40b6590 0x7efeb40b8aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009230 tx=0x7efeb8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efeda872740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff40720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efeda872740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff40c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efeda872740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efeda872740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55820fcfc700 con 0x55820fb8e070 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efeda872740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55820fe36540 con 0x55820fb651b0 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efeda872740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55820ff43710 con 0x55820fb6d390 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efeda872740 10 client.0 _ll_register_callbacks cb 0x55820fe46430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff40720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efecbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff40c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff40720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37478/0 (socket says 172.21.15.27:37478) 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1200640 1 -- 172.21.15.27:0/3246379652 learned_addr learned my addr 172.21.15.27:0/3246379652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1200640 1 -- 172.21.15.27:0/3246379652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 msgr2=0x55820ff40c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1200640 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff40c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1200640 1 -- 172.21.15.27:0/3246379652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 msgr2=0x55820ff4ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:54.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.540+0000 7efed1200640 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efed1200640 1 -- 172.21.15.27:0/3246379652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820fbf9650 con 0x55820fb651b0 2024-09-21T07:18:54.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efed1200640 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff40720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efec000cf40 tx=0x7efec00ace10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd14a20b76fb75f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:54.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efeb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:54.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efeb2a00640 1 -- 172.21.15.27:0/3246379652 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0010720 con 0x55820fb651b0 2024-09-21T07:18:54.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efeb2a00640 1 -- 172.21.15.27:0/3246379652 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7efec00ac4b0 con 0x55820fb651b0 2024-09-21T07:18:54.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55820fe38b50 con 0x55820fb651b0 2024-09-21T07:18:54.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efeb2a00640 1 -- 172.21.15.27:0/3246379652 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efec00ad2f0 con 0x55820fb651b0 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efeda872740 10 client.27131 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.541+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55820fe53910 con 0x55820fb651b0 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeb2a00640 1 -- 172.21.15.27:0/3246379652 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55820fe38b50 con 0x55820fb651b0 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeb2a00640 1 -- 172.21.15.27:0/3246379652 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efec00aea60 con 0x55820fb651b0 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeda872740 10 client.27131 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeda872740 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeda872740 20 client.27131 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeda872740 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb4c9f0 0x55820ff52920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeda872740 4 client.27131 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55820feadec0 con 0x55820fb4c9f0 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.542+0000 7efecbe00640 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb4c9f0 0x55820ff52920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.543+0000 7efecbe00640 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb4c9f0 0x55820ff52920 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46519a561591ed96 server_cookie=31b6e2c874eaddc0 in_seq=0 out_seq=0 2024-09-21T07:18:54.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.543+0000 7efeb2a00640 10 client.27131 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:54.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.544+0000 7efeb2a00640 1 -- 172.21.15.27:0/3246379652 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55820feadec0 con 0x55820fb4c9f0 2024-09-21T07:18:54.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.544+0000 7efeb2a00640 10 client.27131 handle_command_reply: tid=0 2024-09-21T07:18:54.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.544+0000 7efeda872740 1 client.27131 shutdown 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 msgr2=0x55820ff40720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff40720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efec000cf40 tx=0x7efec00ace10 comp rx=0 tx=0).stop 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 shutdown_connections 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb4c9f0 0x55820ff52920 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff40c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 --2- 172.21.15.27:0/3246379652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff40720 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeb2a00640 0 client.27131 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 >> 172.21.15.27:0/3246379652 conn(0x55820fb82460 msgr2=0x55820ff315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 shutdown_connections 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 1 -- 172.21.15.27:0/3246379652 wait complete. 2024-09-21T07:18:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.545+0000 7efeda872740 20 client.27131 trim_cache size 0 max 16384 2024-09-21T07:18:54.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.563+0000 7efeda872740 1 Processor -- start 2024-09-21T07:18:54.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.563+0000 7efeda872740 1 -- start start 2024-09-21T07:18:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efeda872740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff42600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efeda872740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff36570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efeda872740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efeda872740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efeac012610 con 0x55820fb8e070 2024-09-21T07:18:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efeda872740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efeac00fee0 con 0x55820fb651b0 2024-09-21T07:18:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efeda872740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efeac00dd30 con 0x55820fb6d390 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efed1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efecbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efed1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40326/0 (socket says 172.21.15.27:40326) 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efecbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff36570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59278/0 (socket says 172.21.15.27:59278) 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efed1c00640 1 -- 172.21.15.27:0/1549591602 learned_addr learned my addr 172.21.15.27:0/1549591602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.564+0000 7efed1200640 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff42600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efecbe00640 1 -- 172.21.15.27:0/1549591602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 msgr2=0x55820ff42600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efecbe00640 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff42600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efecbe00640 1 -- 172.21.15.27:0/1549591602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 msgr2=0x55820ff4c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efecbe00640 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efecbe00640 1 -- 172.21.15.27:0/1549591602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820ff419e0 con 0x55820fb6d390 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efecbe00640 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff36570 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efeb802a930 tx=0x7efeb803f070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d8e2777898694e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efebd600640 1 -- 172.21.15.27:0/1549591602 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8035040 con 0x55820fb6d390 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efebd600640 1 -- 172.21.15.27:0/1549591602 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb8005500 con 0x55820fb6d390 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.565+0000 7efebd600640 1 -- 172.21.15.27:0/1549591602 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8004dc0 con 0x55820fb6d390 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 -- 172.21.15.27:0/1549591602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 msgr2=0x55820ff36570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff36570 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efeb802a930 tx=0x7efeb803f070 comp rx=0 tx=0).stop 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 -- 172.21.15.27:0/1549591602 shutdown_connections 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb6d390 0x55820ff36570 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff4c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 --2- 172.21.15.27:0/1549591602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb651b0 0x55820ff42600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 -- 172.21.15.27:0/1549591602 >> 172.21.15.27:0/1549591602 conn(0x55820fb82460 msgr2=0x55820ff35e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 -- 172.21.15.27:0/1549591602 shutdown_connections 2024-09-21T07:18:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.566+0000 7efeda872740 1 -- 172.21.15.27:0/1549591602 wait complete. 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 -- start start 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 Processor -- start 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff48ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb6d390 0x55820ff453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff45920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efeac00e3a0 con 0x55820fb8e070 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efeac00f810 con 0x55820fb6d390 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efeac00ea70 con 0x55820fb651b0 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.567+0000 7efeda872740 10 client.0 _ll_register_callbacks cb 0x7efec00e54e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb6d390 0x55820ff453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efed1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff48ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb6d390 0x55820ff453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37494/0 (socket says 172.21.15.27:37494) 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 -- 172.21.15.27:0/4175400503 learned_addr learned my addr 172.21.15.27:0/4175400503 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efed1c00640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff45920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 -- 172.21.15.27:0/4175400503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 msgr2=0x55820ff48ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff48ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 -- 172.21.15.27:0/4175400503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 msgr2=0x55820ff45920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff45920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efecbe00640 1 -- 172.21.15.27:0/4175400503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820ff45e60 con 0x55820fb6d390 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efed1200640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff48ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.568+0000 7efed1c00640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff45920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efecbe00640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb6d390 0x55820ff453e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efeb8004a00 tx=0x7efeb8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce822a3acd3fca5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeb2a00640 1 -- 172.21.15.27:0/4175400503 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb806a060 con 0x55820fb6d390 2024-09-21T07:18:54.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb8005500 con 0x55820fb6d390 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeb2a00640 1 -- 172.21.15.27:0/4175400503 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7efeb8005260 con 0x55820fb6d390 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeb2a00640 1 -- 172.21.15.27:0/4175400503 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efeb8071050 con 0x55820fb6d390 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeda872740 10 client.27137 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55820ffbdf60 con 0x55820fb6d390 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.569+0000 7efeb2a00640 1 -- 172.21.15.27:0/4175400503 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb8005500 con 0x55820fb6d390 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efeb2a00640 1 -- 172.21.15.27:0/4175400503 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efeb80035f0 con 0x55820fb6d390 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efeda872740 10 client.27137 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efeda872740 10 client.27137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efeda872740 20 client.27137 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efeda872740 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb21da0 0x55820ffc0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efeda872740 4 client.27137 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55820fea9970 con 0x55820fb21da0 2024-09-21T07:18:54.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.570+0000 7efed1200640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb21da0 0x55820ffc0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:54.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efed1200640 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb21da0 0x55820ffc0730 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b56afeec31685c8 server_cookie=d8009f7841cc3063 in_seq=0 out_seq=0 2024-09-21T07:18:54.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeb2a00640 10 client.27137 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeb2a00640 1 -- 172.21.15.27:0/4175400503 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55820fea9970 con 0x55820fb21da0 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeb2a00640 10 client.27137 handle_command_reply: tid=0 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 client.27137 shutdown 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb6d390 msgr2=0x55820ff453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb6d390 0x55820ff453e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efeb8004a00 tx=0x7efeb8000bc0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 shutdown_connections 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55820fb21da0 0x55820ffc0730 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55820fb8e070 0x55820ff45920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55820fb6d390 0x55820ff453e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 --2- 172.21.15.27:0/4175400503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55820fb651b0 0x55820ff48ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeb2a00640 0 client.27137 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 >> 172.21.15.27:0/4175400503 conn(0x55820fb82460 msgr2=0x55820ff36820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 shutdown_connections 2024-09-21T07:18:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.571+0000 7efeda872740 1 -- 172.21.15.27:0/4175400503 wait complete. 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efeda872740 20 client.27137 trim_cache size 0 max 16384 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.577 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.578 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:18:54.581 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efeb40b6590 msgr2=0x7efeb40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efeb40b6590 0x7efeb40b8aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009230 tx=0x7efeb8031400 comp rx=0 tx=0).stop 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc105bb0 msgr2=0x7efecc19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc105bb0 0x7efecc19c510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9f0 tx=0x7efec000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 shutdown_connections 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efeb40b6590 0x7efeb40b8aa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc10be40 0x7efecc1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efecc10b4f0 0x7efecc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 --2- 172.21.15.27:0/1111999956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc105bb0 0x7efecc19c510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.572+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 >> 172.21.15.27:0/1111999956 conn(0x7efecc101680 msgr2=0x7efecc103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.573+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 shutdown_connections 2024-09-21T07:18:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:54.573+0000 7efed2600640 1 -- 172.21.15.27:0/1111999956 wait complete. 2024-09-21T07:18:54.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:54 smithi153 ceph-mon[34747]: pgmap v554: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.4 KiB/s wr, 8 op/s 2024-09-21T07:18:55.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:54 smithi119 ceph-mon[34519]: pgmap v554: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.4 KiB/s wr, 8 op/s 2024-09-21T07:18:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:54 smithi027 ceph-mon[30779]: pgmap v554: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.4 KiB/s wr, 8 op/s 2024-09-21T07:18:55.506 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:18:55.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:18:55.890 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 -- 172.21.15.27:0/1449473241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc1141d0 msgr2=0x7f89cc1166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 --2- 172.21.15.27:0/1449473241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc1141d0 0x7f89cc1166a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009ca0 tx=0x7f89c402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 -- 172.21.15.27:0/1449473241 shutdown_connections 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 --2- 172.21.15.27:0/1449473241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc075dd0 0x7f89cc0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 --2- 172.21.15.27:0/1449473241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc1141d0 0x7f89cc1166a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 --2- 172.21.15.27:0/1449473241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc077390 0x7f89cc0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 -- 172.21.15.27:0/1449473241 >> 172.21.15.27:0/1449473241 conn(0x7f89cc0ff330 msgr2=0x7f89cc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 -- 172.21.15.27:0/1449473241 shutdown_connections 2024-09-21T07:18:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.388+0000 7f89d1000640 1 -- 172.21.15.27:0/1449473241 wait complete. 2024-09-21T07:18:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 Processor -- start 2024-09-21T07:18:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 -- start start 2024-09-21T07:18:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc075dd0 0x7f89cc1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc077390 0x7f89cc1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc1141d0 0x7f89cc1a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc119e10 con 0x7f89cc077390 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc119c90 con 0x7f89cc075dd0 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.390+0000 7f89d1000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc119f90 con 0x7f89cc1141d0 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc075dd0 0x7f89cc1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc075dd0 0x7f89cc1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37510/0 (socket says 172.21.15.27:37510) 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 -- 172.21.15.27:0/1648566090 learned_addr learned my addr 172.21.15.27:0/1648566090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89caa00640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc077390 0x7f89cc1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cbe00640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc1141d0 0x7f89cc1a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 -- 172.21.15.27:0/1648566090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc1141d0 msgr2=0x7f89cc1a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc1141d0 0x7f89cc1a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 -- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc077390 msgr2=0x7f89cc1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc077390 0x7f89cc1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 -- 172.21.15.27:0/1648566090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc1a6720 con 0x7f89cc075dd0 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89caa00640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc077390 0x7f89cc1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cbe00640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc1141d0 0x7f89cc1a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.391+0000 7f89cb400640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc075dd0 0x7f89cc1a07f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9f0 tx=0x7f89bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.392+0000 7f89c3e00640 1 -- 172.21.15.27:0/1648566090 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00cd80 con 0x7f89cc075dd0 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.392+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc1a6a10 con 0x7f89cc075dd0 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.392+0000 7f89c3e00640 1 -- 172.21.15.27:0/1648566090 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc0108e0 con 0x7f89cc075dd0 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.392+0000 7f89c3e00640 1 -- 172.21.15.27:0/1648566090 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc018980 con 0x7f89cc075dd0 2024-09-21T07:18:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.392+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc1ad1a0 con 0x7f89cc075dd0 2024-09-21T07:18:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.395+0000 7f89d97fa740 1 Processor -- start 2024-09-21T07:18:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.395+0000 7f89d97fa740 1 -- start start 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.395+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5c21b0 0x55c91e966e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.395+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e967350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.395+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e967890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89d97fa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c91e8dc6b0 con 0x55c91e5c21b0 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89d97fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c91e8cbae0 con 0x55c91e5a14d0 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89d97fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c91e8cf8f0 con 0x55c91e5992f0 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89cbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5c21b0 0x55c91e966e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89caa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e967890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89cbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5c21b0 0x55c91e966e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40356/0 (socket says 172.21.15.27:40356) 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89caa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e967890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37522/0 (socket says 172.21.15.27:37522) 2024-09-21T07:18:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89cbe00640 1 -- 172.21.15.27:0/3094739347 learned_addr learned my addr 172.21.15.27:0/3094739347 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89caa00640 1 -- 172.21.15.27:0/3094739347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 msgr2=0x55c91e967350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:18:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89caa00640 1 --2- 172.21.15.27:0/3094739347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e967350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89caa00640 1 -- 172.21.15.27:0/3094739347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5c21b0 msgr2=0x55c91e966e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89caa00640 1 --2- 172.21.15.27:0/3094739347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5c21b0 0x55c91e966e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89c3e00640 1 -- 172.21.15.27:0/1648566090 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f89bc010430 con 0x7f89cc075dd0 2024-09-21T07:18:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.396+0000 7f89caa00640 1 -- 172.21.15.27:0/3094739347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91e898780 con 0x55c91e5a14d0 2024-09-21T07:18:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.397+0000 7f89caa00640 1 --2- 172.21.15.27:0/3094739347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e967890 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009c30 tx=0x7f89b80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783cde08e7388d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.397+0000 7f89c0c00640 1 -- 172.21.15.27:0/3094739347 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8005020 con 0x55c91e5a14d0 2024-09-21T07:18:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.397+0000 7f89c0c00640 1 -- 172.21.15.27:0/3094739347 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b80047a0 con 0x55c91e5a14d0 2024-09-21T07:18:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.397+0000 7f89c0c00640 1 -- 172.21.15.27:0/3094739347 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80126b0 con 0x55c91e5a14d0 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89c3e00640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89b00b6700 0x7f89b00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89d97fa740 1 -- 172.21.15.27:0/3094739347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 msgr2=0x55c91e967890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89d97fa740 1 --2- 172.21.15.27:0/3094739347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e967890 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009c30 tx=0x7f89b80117b0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89c3e00640 1 -- 172.21.15.27:0/1648566090 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f89bc0e7ee0 con 0x7f89cc075dd0 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89d97fa740 1 -- 172.21.15.27:0/3094739347 shutdown_connections 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89d97fa740 1 --2- 172.21.15.27:0/3094739347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e967350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89d97fa740 1 --2- 172.21.15.27:0/3094739347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e967890 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89d97fa740 1 --2- 172.21.15.27:0/3094739347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5c21b0 0x55c91e966e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89d97fa740 1 -- 172.21.15.27:0/3094739347 >> 172.21.15.27:0/3094739347 conn(0x55c91e5b6300 msgr2=0x55c91e964d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.398+0000 7f89cbe00640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89b00b6700 0x7f89b00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.399+0000 7f89d97fa740 1 -- 172.21.15.27:0/3094739347 shutdown_connections 2024-09-21T07:18:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.399+0000 7f89d97fa740 1 -- 172.21.15.27:0/3094739347 wait complete. 2024-09-21T07:18:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.399+0000 7f89cbe00640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89b00b6700 0x7f89b00b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f89c4004af0 tx=0x7f89c4005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.399+0000 7f89d97fa740 1 -- start start 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 1 Processor -- start 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e9724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e9729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e972f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c91e803ab0 con 0x55c91e5a14d0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c91e872570 con 0x55c91e5c21b0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c91e975590 con 0x55c91e5992f0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.400+0000 7f89d97fa740 10 client.0 _ll_register_callbacks cb 0x55c91e881100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e9724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e972f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89caa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e9729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e9724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59320/0 (socket says 172.21.15.27:59320) 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 -- 172.21.15.27:0/1907762402 learned_addr learned my addr 172.21.15.27:0/1907762402 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 -- 172.21.15.27:0/1907762402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 msgr2=0x55c91e972f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e972f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 -- 172.21.15.27:0/1907762402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 msgr2=0x55c91e9729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e9729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 -- 172.21.15.27:0/1907762402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91e62d790 con 0x55c91e5992f0 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89cb400640 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e9724b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c6a0 tx=0x7f89bc0ad390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b340a708e62ca8a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.401+0000 7f89b5600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89b5600640 1 -- 172.21.15.27:0/1907762402 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc010720 con 0x55c91e5992f0 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b80047a0 con 0x55c91e5992f0 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89b5600640 1 -- 172.21.15.27:0/1907762402 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f89bc018d30 con 0x55c91e5992f0 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89b5600640 1 -- 172.21.15.27:0/1907762402 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89bc098070 con 0x55c91e5992f0 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89d97fa740 10 client.27130 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c91e6a6620 con 0x55c91e5992f0 2024-09-21T07:18:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89b5600640 1 -- 172.21.15.27:0/1907762402 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b80047a0 con 0x55c91e5992f0 2024-09-21T07:18:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89b5600640 1 -- 172.21.15.27:0/1907762402 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f89bc0a92d0 con 0x55c91e5992f0 2024-09-21T07:18:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89d97fa740 10 client.27130 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89d97fa740 10 client.27130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.402+0000 7f89d97fa740 20 client.27130 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.403+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e580d80 0x55c91e9845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.403+0000 7f89d97fa740 4 client.27130 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.403+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c91e8de600 con 0x55c91e580d80 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.403+0000 7f89caa00640 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e580d80 0x55c91e9845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.404+0000 7f89b5600640 10 client.27130 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.404+0000 7f89caa00640 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e580d80 0x55c91e9845b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953c04e7d9de87af server_cookie=399c6f923450f63e in_seq=0 out_seq=0 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.404+0000 7f89b5600640 1 -- 172.21.15.27:0/1907762402 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c91e8de600 con 0x55c91e580d80 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.404+0000 7f89b5600640 10 client.27130 handle_command_reply: tid=0 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 client.27130 shutdown 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 msgr2=0x55c91e9724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e9724b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c6a0 tx=0x7f89bc0ad390 comp rx=0 tx=0).stop 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 shutdown_connections 2024-09-21T07:18:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e580d80 0x55c91e9845b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e972f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e9729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1907762402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e9724b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 >> 172.21.15.27:0/1907762402 conn(0x55c91e5b6300 msgr2=0x55c91e967e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89b5600640 0 client.27130 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 shutdown_connections 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 1 -- 172.21.15.27:0/1907762402 wait complete. 2024-09-21T07:18:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.405+0000 7f89d97fa740 20 client.27130 trim_cache size 0 max 16384 2024-09-21T07:18:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.421+0000 7f89d97fa740 1 Processor -- start 2024-09-21T07:18:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.421+0000 7f89d97fa740 1 -- start start 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5992f0 0x55c91e968550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e96f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5c21b0 0x55c91e96fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89d97fa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1b9a40 con 0x55c91e5992f0 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89d97fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1b7310 con 0x55c91e5a14d0 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89d97fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1b5160 con 0x55c91e5c21b0 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89caa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e96f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89cb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5992f0 0x55c91e968550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89caa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e96f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37532/0 (socket says 172.21.15.27:37532) 2024-09-21T07:18:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89caa00640 1 -- 172.21.15.27:0/1887052474 learned_addr learned my addr 172.21.15.27:0/1887052474 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89cbe00640 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5c21b0 0x55c91e96fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89caa00640 1 -- 172.21.15.27:0/1887052474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5c21b0 msgr2=0x55c91e96fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.422+0000 7f89caa00640 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5c21b0 0x55c91e96fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.423+0000 7f89caa00640 1 -- 172.21.15.27:0/1887052474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5992f0 msgr2=0x55c91e968550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.423+0000 7f89caa00640 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5992f0 0x55c91e968550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.423+0000 7f89caa00640 1 -- 172.21.15.27:0/1887052474 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91e969110 con 0x55c91e5a14d0 2024-09-21T07:18:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.423+0000 7f89caa00640 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e96f680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89b8011cb0 tx=0x7f89b8009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b2057a2845ab08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.423+0000 7f89b7e00640 1 -- 172.21.15.27:0/1887052474 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b801f040 con 0x55c91e5a14d0 2024-09-21T07:18:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.423+0000 7f89b7e00640 1 -- 172.21.15.27:0/1887052474 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b8012900 con 0x55c91e5a14d0 2024-09-21T07:18:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.423+0000 7f89b7e00640 1 -- 172.21.15.27:0/1887052474 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8026850 con 0x55c91e5a14d0 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 -- 172.21.15.27:0/1887052474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 msgr2=0x55c91e96f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e96f680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89b8011cb0 tx=0x7f89b8009960 comp rx=0 tx=0).stop 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 -- 172.21.15.27:0/1887052474 shutdown_connections 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5c21b0 0x55c91e96fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5a14d0 0x55c91e96f680 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 --2- 172.21.15.27:0/1887052474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5992f0 0x55c91e968550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 -- 172.21.15.27:0/1887052474 >> 172.21.15.27:0/1887052474 conn(0x55c91e5b6300 msgr2=0x55c91e967e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 -- 172.21.15.27:0/1887052474 shutdown_connections 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.424+0000 7f89d97fa740 1 -- 172.21.15.27:0/1887052474 wait complete. 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.425+0000 7f89d97fa740 1 -- start start 2024-09-21T07:18:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.425+0000 7f89d97fa740 1 Processor -- start 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e97af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e978430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89d97fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e978970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89d97fa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1b57d0 con 0x55c91e5a14d0 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89d97fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1b6c40 con 0x55c91e5c21b0 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89d97fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1b5ea0 con 0x55c91e5992f0 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89d97fa740 10 client.0 _ll_register_callbacks cb 0x7f89bc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89caa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e978430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89caa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e978430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40388/0 (socket says 172.21.15.27:40388) 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89cb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e97af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89cbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e978970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89caa00640 1 -- 172.21.15.27:0/2275583931 learned_addr learned my addr 172.21.15.27:0/2275583931 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89cb400640 1 -- 172.21.15.27:0/2275583931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 msgr2=0x55c91e978970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89cb400640 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e978970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89cb400640 1 -- 172.21.15.27:0/2275583931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 msgr2=0x55c91e978430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89cb400640 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e978430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89cb400640 1 -- 172.21.15.27:0/2275583931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91e9790b0 con 0x55c91e5992f0 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.426+0000 7f89caa00640 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e978430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89cb400640 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e97af70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0aabf0 tx=0x7f89bc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b21a8c011413ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89b5600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89b5600640 1 -- 172.21.15.27:0/2275583931 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0ad750 con 0x55c91e5992f0 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89b5600640 1 -- 172.21.15.27:0/2275583931 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f89bc0c3d30 con 0x55c91e5992f0 2024-09-21T07:18:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b8012900 con 0x55c91e5992f0 2024-09-21T07:18:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89b5600640 1 -- 172.21.15.27:0/2275583931 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89bc010720 con 0x55c91e5992f0 2024-09-21T07:18:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89d97fa740 10 client.27136 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.427+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c91e9eedf0 con 0x55c91e5992f0 2024-09-21T07:18:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89b5600640 1 -- 172.21.15.27:0/2275583931 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b8012900 con 0x55c91e5992f0 2024-09-21T07:18:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89b5600640 1 -- 172.21.15.27:0/2275583931 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f89bc0b68e0 con 0x55c91e5992f0 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89d97fa740 10 client.27136 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89d97fa740 10 client.27136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89d97fa740 20 client.27136 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89d97fa740 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e5561c0 0x55c91e9f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89d97fa740 4 client.27136 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c91e8da0b0 con 0x55c91e5561c0 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.428+0000 7f89caa00640 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e5561c0 0x55c91e9f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.429+0000 7f89caa00640 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e5561c0 0x55c91e9f1630 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f343925fe58c1b5 server_cookie=ee8a2d22a90d7a4a in_seq=0 out_seq=0 2024-09-21T07:18:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.429+0000 7f89b5600640 10 client.27136 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.441+0000 7f89b5600640 1 -- 172.21.15.27:0/2275583931 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c91e8da0b0 con 0x55c91e5561c0 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.441+0000 7f89b5600640 10 client.27136 handle_command_reply: tid=0 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.441+0000 7f89d97fa740 1 client.27136 shutdown 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.441+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 msgr2=0x55c91e97af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.441+0000 7f89d97fa740 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e97af70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0aabf0 tx=0x7f89bc000bc0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.441+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 shutdown_connections 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.441+0000 7f89d97fa740 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c91e5561c0 0x55c91e9f1630 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89d97fa740 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c91e5c21b0 0x55c91e978970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89d97fa740 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c91e5a14d0 0x55c91e978430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89d97fa740 1 --2- 172.21.15.27:0/2275583931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c91e5992f0 0x55c91e97af70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89b5600640 0 client.27136 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 >> 172.21.15.27:0/2275583931 conn(0x55c91e5b6300 msgr2=0x55c91e967e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 shutdown_connections 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89d97fa740 1 -- 172.21.15.27:0/2275583931 wait complete. 2024-09-21T07:18:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.442+0000 7f89d97fa740 20 client.27136 trim_cache size 0 max 16384 2024-09-21T07:18:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:18:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:18:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:18:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89b00b6700 msgr2=0x7f89b00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89b00b6700 0x7f89b00b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f89c4004af0 tx=0x7f89c4005c30 comp rx=0 tx=0).stop 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc075dd0 msgr2=0x7f89cc1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc075dd0 0x7f89cc1a07f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9f0 tx=0x7f89bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 shutdown_connections 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc1141d0 0x7f89cc1a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89b00b6700 0x7f89b00b8c10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc077390 0x7f89cc1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 --2- 172.21.15.27:0/1648566090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc075dd0 0x7f89cc1a07f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 >> 172.21.15.27:0/1648566090 conn(0x7f89cc0ff330 msgr2=0x7f89cc1100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 shutdown_connections 2024-09-21T07:18:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:18:56.443+0000 7f89d1000640 1 -- 172.21.15.27:0/1648566090 wait complete. 2024-09-21T07:18:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 -- 172.21.15.27:0/2217588666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 msgr2=0x7fef1010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 --2- 172.21.15.27:0/2217588666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 0x7fef1010bad0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feefc009900 tx=0x7feefc02a410 comp rx=0 tx=0).stop 2024-09-21T07:18:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 -- 172.21.15.27:0/2217588666 shutdown_connections 2024-09-21T07:18:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 --2- 172.21.15.27:0/2217588666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef1010c010 0x7fef10112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 --2- 172.21.15.27:0/2217588666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 0x7fef1010bad0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 --2- 172.21.15.27:0/2217588666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef10105d80 0x7fef10106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 -- 172.21.15.27:0/2217588666 >> 172.21.15.27:0/2217588666 conn(0x7fef10101810 msgr2=0x7fef10103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 -- 172.21.15.27:0/2217588666 shutdown_connections 2024-09-21T07:18:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.499+0000 7fef18600640 1 -- 172.21.15.27:0/2217588666 wait complete. 2024-09-21T07:18:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.500+0000 7fef18600640 1 Processor -- start 2024-09-21T07:18:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.500+0000 7fef18600640 1 -- start start 2024-09-21T07:18:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef18600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef10105d80 0x7fef1019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef18600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 0x7fef1019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef18600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef1010c010 0x7fef101a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef18600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef10115cd0 con 0x7fef1010b6c0 2024-09-21T07:18:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef18600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef10115b50 con 0x7fef10105d80 2024-09-21T07:18:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef18600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef10115e50 con 0x7fef1010c010 2024-09-21T07:18:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef15200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 0x7fef1019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef15200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 0x7fef1019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40400/0 (socket says 172.21.15.27:40400) 2024-09-21T07:18:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef15200640 1 -- 172.21.15.27:0/2485337338 learned_addr learned my addr 172.21.15.27:0/2485337338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef15c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef10105d80 0x7fef1019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.501+0000 7fef16600640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef1010c010 0x7fef101a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef15c00640 1 -- 172.21.15.27:0/2485337338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef1010c010 msgr2=0x7fef101a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef15c00640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef1010c010 0x7fef101a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef15c00640 1 -- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 msgr2=0x7fef1019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef15c00640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 0x7fef1019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef15c00640 1 -- 172.21.15.27:0/2485337338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef101a2630 con 0x7fef10105d80 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef15c00640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef10105d80 0x7fef1019c6c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef0400e9f0 tx=0x7fef0400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef0e000640 1 -- 172.21.15.27:0/2485337338 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0400ccc0 con 0x7fef10105d80 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef101a2920 con 0x7fef10105d80 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef0e000640 1 -- 172.21.15.27:0/2485337338 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef04005020 con 0x7fef10105d80 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef0e000640 1 -- 172.21.15.27:0/2485337338 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04004a00 con 0x7fef10105d80 2024-09-21T07:18:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.502+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef101a9080 con 0x7fef10105d80 2024-09-21T07:18:56.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.503+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef10106190 con 0x7fef10105d80 2024-09-21T07:18:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.506+0000 7fef0e000640 1 -- 172.21.15.27:0/2485337338 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fef04004ba0 con 0x7fef10105d80 2024-09-21T07:18:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.508+0000 7fef0e000640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feef80b6680 0x7feef80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.508+0000 7fef0e000640 1 -- 172.21.15.27:0/2485337338 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fef040e7b00 con 0x7fef10105d80 2024-09-21T07:18:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.508+0000 7fef15200640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feef80b6680 0x7feef80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.509+0000 7fef15200640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feef80b6680 0x7feef80b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fef10109170 tx=0x7feefc009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.509+0000 7fef0e000640 1 -- 172.21.15.27:0/2485337338 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef040add30 con 0x7fef10105d80 2024-09-21T07:18:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.751+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef1019d7f0 con 0x7fef10105d80 2024-09-21T07:18:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.753+0000 7fef0e000640 1 -- 172.21.15.27:0/2485337338 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fef040b2110 con 0x7fef10105d80 2024-09-21T07:18:56.755 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:18:56.755 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:18:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.755+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feef80b6680 msgr2=0x7feef80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.755+0000 7fef18600640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feef80b6680 0x7feef80b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fef10109170 tx=0x7feefc009560 comp rx=0 tx=0).stop 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef10105d80 msgr2=0x7fef1019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef10105d80 0x7fef1019c6c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef0400e9f0 tx=0x7fef0400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 shutdown_connections 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef1010c010 0x7fef101a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7feef80b6680 0x7feef80b8b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef1010b6c0 0x7fef1019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 --2- 172.21.15.27:0/2485337338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef10105d80 0x7fef1019c6c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 >> 172.21.15.27:0/2485337338 conn(0x7fef10101810 msgr2=0x7fef10103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 shutdown_connections 2024-09-21T07:18:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:56.756+0000 7fef18600640 1 -- 172.21.15.27:0/2485337338 wait complete. 2024-09-21T07:18:56.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:56 smithi153 ceph-mon[34747]: pgmap v555: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-09-21T07:18:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:56 smithi119 ceph-mon[34519]: pgmap v555: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-09-21T07:18:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:56 smithi027 ceph-mon[30779]: pgmap v555: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-09-21T07:18:57.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:18:57.416 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:18:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:57.822 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:18:57.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:57 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2485337338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:57.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:57 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2485337338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:58.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:57 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2485337338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 -- 172.21.15.27:0/1466248275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db4103b80 msgr2=0x7f5db4103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 --2- 172.21.15.27:0/1466248275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db4103b80 0x7f5db4103f90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009900 tx=0x7f5da802a400 comp rx=0 tx=0).stop 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 -- 172.21.15.27:0/1466248275 shutdown_connections 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 --2- 172.21.15.27:0/1466248275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5db41094c0 0x7f5db41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 --2- 172.21.15.27:0/1466248275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db4109e10 0x7f5db4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 --2- 172.21.15.27:0/1466248275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db4103b80 0x7f5db4103f90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 -- 172.21.15.27:0/1466248275 >> 172.21.15.27:0/1466248275 conn(0x7f5db40ff590 msgr2=0x7f5db41019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 -- 172.21.15.27:0/1466248275 shutdown_connections 2024-09-21T07:18:58.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.645+0000 7f5dbb600640 1 -- 172.21.15.27:0/1466248275 wait complete. 2024-09-21T07:18:58.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.646+0000 7f5dbb600640 1 Processor -- start 2024-09-21T07:18:58.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.646+0000 7f5dbb600640 1 -- start start 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5dbb600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5db4103b80 0x7f5db419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5dbb600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db41094c0 0x7f5db419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5dbb600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db4109e10 0x7f5db419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5dbb600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4113ad0 con 0x7f5db4109e10 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5dbb600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4113950 con 0x7f5db41094c0 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5dbb600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4113c50 con 0x7f5db4103b80 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db41094c0 0x7f5db419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db41094c0 0x7f5db419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60792/0 (socket says 172.21.15.27:60792) 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5db3e00640 1 -- 172.21.15.27:0/2790314308 learned_addr learned my addr 172.21.15.27:0/2790314308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5db9600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db4109e10 0x7f5db419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.647+0000 7f5db8c00640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5db4103b80 0x7f5db419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db3e00640 1 -- 172.21.15.27:0/2790314308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5db4103b80 msgr2=0x7f5db419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db3e00640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5db4103b80 0x7f5db419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db3e00640 1 -- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db4109e10 msgr2=0x7f5db419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db3e00640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db4109e10 0x7f5db419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db3e00640 1 -- 172.21.15.27:0/2790314308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db41a0450 con 0x7f5db41094c0 2024-09-21T07:18:58.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db8c00640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5db4103b80 0x7f5db419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:18:58.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db3e00640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db41094c0 0x7f5db419aa80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5da000e9c0 tx=0x7f5da000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db1600640 1 -- 172.21.15.27:0/2790314308 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da000ccc0 con 0x7f5db41094c0 2024-09-21T07:18:58.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db1600640 1 -- 172.21.15.27:0/2790314308 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da00054d0 con 0x7f5db41094c0 2024-09-21T07:18:58.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db41a0740 con 0x7f5db41094c0 2024-09-21T07:18:58.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.648+0000 7f5db1600640 1 -- 172.21.15.27:0/2790314308 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0010430 con 0x7f5db41094c0 2024-09-21T07:18:58.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.649+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db40755f0 con 0x7f5db41094c0 2024-09-21T07:18:58.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.650+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db4104e00 con 0x7f5db41094c0 2024-09-21T07:18:58.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.653+0000 7f5db1600640 1 -- 172.21.15.27:0/2790314308 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5da0005000 con 0x7f5db41094c0 2024-09-21T07:18:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.655+0000 7f5db1600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d9c0b6680 0x7f5d9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.655+0000 7f5db1600640 1 -- 172.21.15.27:0/2790314308 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f5da00e8080 con 0x7f5db41094c0 2024-09-21T07:18:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.655+0000 7f5db8c00640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d9c0b6680 0x7f5d9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.656+0000 7f5db1600640 1 -- 172.21.15.27:0/2790314308 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5da0056710 con 0x7f5db41094c0 2024-09-21T07:18:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.656+0000 7f5db8c00640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d9c0b6680 0x7f5d9c0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5da8031440 tx=0x7f5da8009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:58 smithi027 ceph-mon[30779]: pgmap v556: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 KiB/s wr, 8 op/s 2024-09-21T07:18:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.730+0000 7fab4d400640 1 -- 172.21.15.27:0/1168439802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab481092f0 msgr2=0x7fab48109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.730+0000 7fab4d400640 1 --2- 172.21.15.27:0/1168439802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab481092f0 0x7fab48109700 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fab34009900 tx=0x7fab3402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.731+0000 7fab4d400640 1 -- 172.21.15.27:0/1168439802 shutdown_connections 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.731+0000 7fab4d400640 1 --2- 172.21.15.27:0/1168439802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab48109c40 0x7fab48110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.731+0000 7fab4d400640 1 --2- 172.21.15.27:0/1168439802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab481092f0 0x7fab48109700 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.731+0000 7fab4d400640 1 --2- 172.21.15.27:0/1168439802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab481039b0 0x7fab48103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.731+0000 7fab4d400640 1 -- 172.21.15.27:0/1168439802 >> 172.21.15.27:0/1168439802 conn(0x7fab480ff440 msgr2=0x7fab48101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.731+0000 7fab4d400640 1 -- 172.21.15.27:0/1168439802 shutdown_connections 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.731+0000 7fab4d400640 1 -- 172.21.15.27:0/1168439802 wait complete. 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.732+0000 7fab4d400640 1 Processor -- start 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.732+0000 7fab4d400640 1 -- start start 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab4d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab481039b0 0x7fab4819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab4d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab481092f0 0x7fab4819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab4d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab48109c40 0x7fab481a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab4d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab48113900 con 0x7fab48109c40 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab4d400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab48113780 con 0x7fab481039b0 2024-09-21T07:18:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab4d400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab48113a80 con 0x7fab481092f0 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab4ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab48109c40 0x7fab481a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab481039b0 0x7fab4819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab481092f0 0x7fab4819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab481039b0 0x7fab4819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60798/0 (socket says 172.21.15.27:60798) 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47e00640 1 -- 172.21.15.27:0/1427199032 learned_addr learned my addr 172.21.15.27:0/1427199032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47e00640 1 -- 172.21.15.27:0/1427199032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab481092f0 msgr2=0x7fab4819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47e00640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab481092f0 0x7fab4819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47e00640 1 -- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab48109c40 msgr2=0x7fab481a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.733+0000 7fab47e00640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab48109c40 0x7fab481a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab47e00640 1 -- 172.21.15.27:0/1427199032 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab481a24a0 con 0x7fab481039b0 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab47400640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab481092f0 0x7fab4819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab47e00640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab481039b0 0x7fab4819c530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab3800ea60 tx=0x7fab3800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab44c00640 1 -- 172.21.15.27:0/1427199032 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3800cde0 con 0x7fab481039b0 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab44c00640 1 -- 172.21.15.27:0/1427199032 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab380035f0 con 0x7fab481039b0 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab44c00640 1 -- 172.21.15.27:0/1427199032 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38010690 con 0x7fab481039b0 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab481a2790 con 0x7fab481039b0 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.734+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab481a8ef0 con 0x7fab481039b0 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 Processor -- start 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 -- start start 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fd9070 0x55d37937fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d379380270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb8390 0x55d3793807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d3792f6fb0 con 0x55d378fb8390 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d3792e63e0 con 0x55d378fd9070 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.738+0000 7fab55668740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d3792ea1f0 con 0x55d378fb01b0 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb8390 0x55d3793807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb8390 0x55d3793807b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48382/0 (socket says 172.21.15.27:48382) 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 -- 172.21.15.27:0/1788977022 learned_addr learned my addr 172.21.15.27:0/1788977022 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab4ca00640 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fd9070 0x55d37937fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 -- 172.21.15.27:0/1788977022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 msgr2=0x55d379380270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d379380270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 -- 172.21.15.27:0/1788977022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fd9070 msgr2=0x55d37937fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fd9070 0x55d37937fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.739+0000 7fab47400640 1 -- 172.21.15.27:0/1788977022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3792b0cd0 con 0x55d378fb8390 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.740+0000 7fab47400640 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb8390 0x55d3793807b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab4806b990 tx=0x7fab34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c19b1e825798050 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.740+0000 7fab3d200640 1 -- 172.21.15.27:0/1788977022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34040070 con 0x55d378fb8390 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.740+0000 7fab3d200640 1 -- 172.21.15.27:0/1788977022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab34031720 con 0x55d378fb8390 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.740+0000 7fab3d200640 1 -- 172.21.15.27:0/1788977022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34035040 con 0x55d378fb8390 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 -- 172.21.15.27:0/1788977022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb8390 msgr2=0x55d3793807b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb8390 0x55d3793807b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab4806b990 tx=0x7fab34000bc0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab44c00640 1 -- 172.21.15.27:0/1427199032 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fab380037a0 con 0x7fab481039b0 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 -- 172.21.15.27:0/1788977022 shutdown_connections 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d379380270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb8390 0x55d3793807b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 --2- 172.21.15.27:0/1788977022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fd9070 0x55d37937fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 -- 172.21.15.27:0/1788977022 >> 172.21.15.27:0/1788977022 conn(0x55d378fcd460 msgr2=0x55d37937dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 -- 172.21.15.27:0/1788977022 shutdown_connections 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.741+0000 7fab55668740 1 -- 172.21.15.27:0/1788977022 wait complete. 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.742+0000 7fab55668740 1 -- start start 2024-09-21T07:18:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.742+0000 7fab55668740 1 Processor -- start 2024-09-21T07:18:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab55668740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb01b0 0x55d37938b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab55668740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37938be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab55668740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fd9070 0x55d37939ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab55668740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d379147700 con 0x55d378fb01b0 2024-09-21T07:18:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab55668740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d379281580 con 0x55d378fb8390 2024-09-21T07:18:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab55668740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d37938e4b0 con 0x55d378fd9070 2024-09-21T07:18:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab55668740 10 client.0 _ll_register_callbacks cb 0x55d379291470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab44c00640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab2c0b6590 0x7fab2c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.743+0000 7fab47e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37938be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab47e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37938be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60818/0 (socket says 172.21.15.27:60818) 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab47400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fd9070 0x55d37939ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab47e00640 1 -- 172.21.15.27:0/1184912389 learned_addr learned my addr 172.21.15.27:0/1184912389 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab4ca00640 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb01b0 0x55d37938b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab44c00640 1 -- 172.21.15.27:0/1427199032 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab380e7db0 con 0x7fab481039b0 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab4ca00640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab2c0b6590 0x7fab2c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab4ca00640 1 -- 172.21.15.27:0/1184912389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fd9070 msgr2=0x55d37939ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab4ca00640 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fd9070 0x55d37939ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab4ca00640 1 -- 172.21.15.27:0/1184912389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 msgr2=0x55d37938be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab4ca00640 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37938be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.744+0000 7fab4ca00640 1 -- 172.21.15.27:0/1184912389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d379044650 con 0x55d378fb01b0 2024-09-21T07:18:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.745+0000 7fab4ca00640 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb01b0 0x55d37938b8f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fab40005fd0 tx=0x7fab4000f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b177cd069a8e5a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.745+0000 7fab31600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.745+0000 7fab31600640 1 -- 172.21.15.27:0/1184912389 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40022040 con 0x55d378fb01b0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.745+0000 7fab31600640 1 -- 172.21.15.27:0/1184912389 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab40009040 con 0x55d378fb01b0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.745+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d379283b90 con 0x55d378fb01b0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.745+0000 7fab4ca00640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab2c0b6590 0x7fab2c0b8aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fab40024000 tx=0x7fab4001f000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.746+0000 7fab31600640 1 -- 172.21.15.27:0/1184912389 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab4002e070 con 0x55d378fb01b0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.746+0000 7fab55668740 10 client.17148 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.746+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d37929e950 con 0x55d378fb01b0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.746+0000 7fab31600640 1 -- 172.21.15.27:0/1184912389 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d379283b90 con 0x55d378fb01b0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab31600640 1 -- 172.21.15.27:0/1184912389 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab400314a0 con 0x55d378fb01b0 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab55668740 10 client.17148 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab55668740 10 client.17148 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:18:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab55668740 20 client.17148 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab55668740 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f979f0 0x55d3793a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab55668740 4 client.17148 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3792f8f00 con 0x55d378f979f0 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.747+0000 7fab47e00640 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f979f0 0x55d3793a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.748+0000 7fab47e00640 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f979f0 0x55d3793a6090 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a232b04252db6ca4 server_cookie=36c257ca471abb81 in_seq=0 out_seq=0 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.748+0000 7fab31600640 10 client.17148 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab31600640 1 -- 172.21.15.27:0/1184912389 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3792f8f00 con 0x55d378f979f0 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab31600640 10 client.17148 handle_command_reply: tid=0 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 client.17148 shutdown 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb01b0 msgr2=0x55d37938b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb01b0 0x55d37938b8f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fab40005fd0 tx=0x7fab4000f950 comp rx=0 tx=0).stop 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 shutdown_connections 2024-09-21T07:18:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f979f0 0x55d3793a6090 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fd9070 0x55d37939ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37938be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 --2- 172.21.15.27:0/1184912389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fb01b0 0x55d37938b8f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 >> 172.21.15.27:0/1184912389 conn(0x55d378fcd460 msgr2=0x55d379382f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab31600640 0 client.17148 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 shutdown_connections 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.749+0000 7fab55668740 1 -- 172.21.15.27:0/1184912389 wait complete. 2024-09-21T07:18:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.750+0000 7fab55668740 20 client.17148 trim_cache size 0 max 16384 2024-09-21T07:18:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.765+0000 7fab55668740 1 Processor -- start 2024-09-21T07:18:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.765+0000 7fab55668740 1 -- start start 2024-09-21T07:18:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab55668740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb01b0 0x55d379386ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab55668740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb8390 0x55d379381470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab55668740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d3793888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab55668740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab481b1320 con 0x55d378fd9070 2024-09-21T07:18:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab55668740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab481aebf0 con 0x55d378fb01b0 2024-09-21T07:18:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab55668740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab481acaa0 con 0x55d378fb8390 2024-09-21T07:18:58.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab4ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d3793888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab47e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb01b0 0x55d379386ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab47400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb8390 0x55d379381470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab4ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d3793888e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48410/0 (socket says 172.21.15.27:48410) 2024-09-21T07:18:58.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.766+0000 7fab4ca00640 1 -- 172.21.15.27:0/4095634265 learned_addr learned my addr 172.21.15.27:0/4095634265 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:58.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab4ca00640 1 -- 172.21.15.27:0/4095634265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb8390 msgr2=0x55d379381470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:58.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab4ca00640 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb8390 0x55d379381470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab4ca00640 1 -- 172.21.15.27:0/4095634265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb01b0 msgr2=0x55d379386ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab4ca00640 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb01b0 0x55d379386ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab4ca00640 1 -- 172.21.15.27:0/4095634265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3793a6620 con 0x55d378fd9070 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab47400640 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb8390 0x55d379381470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab47e00640 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb01b0 0x55d379386ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.767+0000 7fab4ca00640 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d3793888e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fab4001e3e0 tx=0x7fab40031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54dd6fb5d85f06d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.768+0000 7fab33e00640 1 -- 172.21.15.27:0/4095634265 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40036070 con 0x55d378fd9070 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.768+0000 7fab33e00640 1 -- 172.21.15.27:0/4095634265 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab40023020 con 0x55d378fd9070 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.768+0000 7fab33e00640 1 -- 172.21.15.27:0/4095634265 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40022040 con 0x55d378fd9070 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.768+0000 7fab55668740 1 -- 172.21.15.27:0/4095634265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 msgr2=0x55d3793888e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.768+0000 7fab55668740 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d3793888e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fab4001e3e0 tx=0x7fab40031f40 comp rx=0 tx=0).stop 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 -- 172.21.15.27:0/4095634265 shutdown_connections 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb8390 0x55d379381470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d3793888e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 --2- 172.21.15.27:0/4095634265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb01b0 0x55d379386ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 -- 172.21.15.27:0/4095634265 >> 172.21.15.27:0/4095634265 conn(0x55d378fcd460 msgr2=0x55d379380d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 -- 172.21.15.27:0/4095634265 shutdown_connections 2024-09-21T07:18:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 -- 172.21.15.27:0/4095634265 wait complete. 2024-09-21T07:18:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 -- start start 2024-09-21T07:18:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.769+0000 7fab55668740 1 Processor -- start 2024-09-21T07:18:58.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab55668740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d37939bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab55668740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37939c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab55668740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d379396f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab55668740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab481ad0f0 con 0x55d378fd9070 2024-09-21T07:18:58.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab55668740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab481ae520 con 0x55d378fb8390 2024-09-21T07:18:58.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab55668740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab481ad780 con 0x55d378fb01b0 2024-09-21T07:18:58.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab55668740 10 client.0 _ll_register_callbacks cb 0x7fab40010c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:18:58.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab4ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d379396f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab47e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d37939bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab4ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d379396f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48418/0 (socket says 172.21.15.27:48418) 2024-09-21T07:18:58.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab4ca00640 1 -- 172.21.15.27:0/807285953 learned_addr learned my addr 172.21.15.27:0/807285953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:18:58.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.770+0000 7fab47e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d37939bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38766/0 (socket says 172.21.15.27:38766) 2024-09-21T07:18:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab47400640 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37939c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab4ca00640 1 -- 172.21.15.27:0/807285953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 msgr2=0x55d37939bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab4ca00640 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d37939bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab4ca00640 1 -- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 msgr2=0x55d37939c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:18:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab4ca00640 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37939c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab4ca00640 1 -- 172.21.15.27:0/807285953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d379397490 con 0x55d378fd9070 2024-09-21T07:18:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab47400640 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37939c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:18:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab4ca00640 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d379396f50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fab40009270 tx=0x7fab4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bbab19aed02bba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:18:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.771+0000 7fab31600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:18:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab31600640 1 -- 172.21.15.27:0/807285953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40041070 con 0x55d378fd9070 2024-09-21T07:18:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab40023020 con 0x55d378fd9070 2024-09-21T07:18:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab31600640 1 -- 172.21.15.27:0/807285953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7fab40031580 con 0x55d378fd9070 2024-09-21T07:18:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab31600640 1 -- 172.21.15.27:0/807285953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab4004d050 con 0x55d378fd9070 2024-09-21T07:18:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab55668740 10 client.17154 fetch_fsmap learned FSMap version 27 2024-09-21T07:18:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d379411180 con 0x55d378fd9070 2024-09-21T07:18:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab31600640 1 -- 172.21.15.27:0/807285953 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab40023020 con 0x55d378fd9070 2024-09-21T07:18:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.772+0000 7fab31600640 1 -- 172.21.15.27:0/807285953 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab4003b760 con 0x55d378fd9070 2024-09-21T07:18:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab55668740 10 client.17154 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:18:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab55668740 10 client.17154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab55668740 20 client.17154 populate_metadata read hostname 'smithi027' 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab55668740 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f6cda0 0x55d3794137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab55668740 4 client.17154 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3792f49b0 con 0x55d378f6cda0 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab47e00640 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f6cda0 0x55d3794137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab47e00640 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f6cda0 0x55d3794137a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0fa431938aff2f3 server_cookie=b493115cf468624c in_seq=0 out_seq=0 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.773+0000 7fab31600640 10 client.17154 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab31600640 1 -- 172.21.15.27:0/807285953 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3792f49b0 con 0x55d378f6cda0 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab31600640 10 client.17154 handle_command_reply: tid=0 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 client.17154 shutdown 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 msgr2=0x55d379396f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d379396f50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fab40009270 tx=0x7fab4000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 shutdown_connections 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d378f6cda0 0x55d3794137a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d378fd9070 0x55d379396f50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d378fb8390 0x55d37939c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 --2- 172.21.15.27:0/807285953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d378fb01b0 0x55d37939bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 >> 172.21.15.27:0/807285953 conn(0x55d378fcd460 msgr2=0x55d379382e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab31600640 0 client.17154 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:18:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 shutdown_connections 2024-09-21T07:18:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.774+0000 7fab55668740 1 -- 172.21.15.27:0/807285953 wait complete. 2024-09-21T07:18:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab55668740 20 client.17154 trim_cache size 0 max 16384 2024-09-21T07:18:58.785 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:18:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab2c0b6590 msgr2=0x7fab2c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab2c0b6590 0x7fab2c0b8aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fab40024000 tx=0x7fab4001f000 comp rx=0 tx=0).stop 2024-09-21T07:18:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab481039b0 msgr2=0x7fab4819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab481039b0 0x7fab4819c530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab3800ea60 tx=0x7fab3800ef20 comp rx=0 tx=0).stop 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 shutdown_connections 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab481092f0 0x7fab4819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab2c0b6590 0x7fab2c0b8aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab48109c40 0x7fab481a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 --2- 172.21.15.27:0/1427199032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab481039b0 0x7fab4819c530 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 >> 172.21.15.27:0/1427199032 conn(0x7fab480ff440 msgr2=0x7fab4810e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.775+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 shutdown_connections 2024-09-21T07:18:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.776+0000 7fab4d400640 1 -- 172.21.15.27:0/1427199032 wait complete. 2024-09-21T07:18:58.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:58 smithi153 ceph-mon[34747]: pgmap v556: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 KiB/s wr, 8 op/s 2024-09-21T07:18:58.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.931+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5db40758b0 con 0x7f5db41094c0 2024-09-21T07:18:58.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.932+0000 7f5db1600640 1 -- 172.21.15.27:0/2790314308 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f5da00052f0 con 0x7f5db41094c0 2024-09-21T07:18:58.935 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:18:58.935 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:18:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:18:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d9c0b6680 msgr2=0x7f5d9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d9c0b6680 0x7f5d9c0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5da8031440 tx=0x7f5da8009680 comp rx=0 tx=0).stop 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db41094c0 msgr2=0x7f5db419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db41094c0 0x7f5db419aa80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5da000e9c0 tx=0x7f5da000ee80 comp rx=0 tx=0).stop 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 shutdown_connections 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5d9c0b6680 0x7f5d9c0b8b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db4109e10 0x7f5db419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5db41094c0 0x7f5db419aa80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 --2- 172.21.15.27:0/2790314308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5db4103b80 0x7f5db419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 >> 172.21.15.27:0/2790314308 conn(0x7f5db40ff590 msgr2=0x7f5db4107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 shutdown_connections 2024-09-21T07:18:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:18:58.935+0000 7f5dbb600640 1 -- 172.21.15.27:0/2790314308 wait complete. 2024-09-21T07:18:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:58 smithi119 ceph-mon[34519]: pgmap v556: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.6 KiB/s wr, 8 op/s 2024-09-21T07:18:59.765 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 7b1b6a1a --release --await' 2024-09-21T07:18:59.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 7b1b6a1a --release --await 2024-09-21T07:18:59.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:18:59 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2790314308' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:00.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:18:59 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2790314308' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:00.063 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:18:59 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2790314308' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:00.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:19:00.195 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:00.845 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 -- 172.21.15.27:0/1471115361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10103b60 msgr2=0x7f9d10103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1471115361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10103b60 0x7f9d10103f70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009860 tx=0x7f9d0402a340 comp rx=0 tx=0).stop 2024-09-21T07:19:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 -- 172.21.15.27:0/1471115361 shutdown_connections 2024-09-21T07:19:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1471115361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d10109df0 0x7f9d10110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1471115361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d101094a0 0x7f9d101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1471115361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10103b60 0x7f9d10103f70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 -- 172.21.15.27:0/1471115361 >> 172.21.15.27:0/1471115361 conn(0x7f9d1007b850 msgr2=0x7f9d1007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 -- 172.21.15.27:0/1471115361 shutdown_connections 2024-09-21T07:19:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.875+0000 7f9d17c00640 1 -- 172.21.15.27:0/1471115361 wait complete. 2024-09-21T07:19:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.876+0000 7f9d17c00640 1 Processor -- start 2024-09-21T07:19:00.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.877+0000 7f9d17c00640 1 -- start start 2024-09-21T07:19:00.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.877+0000 7f9d17c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d10103b60 0x7f9d1010e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.877+0000 7f9d17c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d101094a0 0x7f9d1010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.877+0000 7f9d17c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10109df0 0x7f9d1010acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.877+0000 7f9d17c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10113ab0 con 0x7f9d101094a0 2024-09-21T07:19:00.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.877+0000 7f9d17c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10113930 con 0x7f9d10103b60 2024-09-21T07:19:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.877+0000 7f9d17c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10113c30 con 0x7f9d10109df0 2024-09-21T07:19:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d10103b60 0x7f9d1010e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d10103b60 0x7f9d1010e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60856/0 (socket says 172.21.15.27:60856) 2024-09-21T07:19:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 -- 172.21.15.27:0/1284136392 learned_addr learned my addr 172.21.15.27:0/1284136392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d0fe00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d101094a0 0x7f9d1010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10109df0 0x7f9d1010acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 -- 172.21.15.27:0/1284136392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10109df0 msgr2=0x7f9d1010acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10109df0 0x7f9d1010acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 -- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d101094a0 msgr2=0x7f9d1010a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d101094a0 0x7f9d1010a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.878+0000 7f9d15200640 1 -- 172.21.15.27:0/1284136392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d101acc30 con 0x7f9d10103b60 2024-09-21T07:19:00.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.879+0000 7f9d15200640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d10103b60 0x7f9d1010e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d04035040 tx=0x7f9d04032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:00.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.879+0000 7f9d0d600640 1 -- 172.21.15.27:0/1284136392 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04037040 con 0x7f9d10103b60 2024-09-21T07:19:00.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.879+0000 7f9d0d600640 1 -- 172.21.15.27:0/1284136392 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d04041070 con 0x7f9d10103b60 2024-09-21T07:19:00.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.879+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d101ace00 con 0x7f9d10103b60 2024-09-21T07:19:00.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.879+0000 7f9d0d600640 1 -- 172.21.15.27:0/1284136392 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04003850 con 0x7f9d10103b60 2024-09-21T07:19:00.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.879+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101ad250 con 0x7f9d10103b60 2024-09-21T07:19:00.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 Processor -- start 2024-09-21T07:19:00.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 -- start start 2024-09-21T07:19:00.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6c721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c72700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c72c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5624c6be9290 con 0x5624c68aa840 2024-09-21T07:19:00.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5624c6bd86c0 con 0x5624c68a2660 2024-09-21T07:19:00.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.884+0000 7f9d1e832740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5624c6bdc4d0 con 0x5624c68cb520 2024-09-21T07:19:00.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d0d600640 1 -- 172.21.15.27:0/1284136392 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9d04004050 con 0x7f9d10103b60 2024-09-21T07:19:00.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6c721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6c721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38798/0 (socket says 172.21.15.27:38798) 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 -- 172.21.15.27:0/2821700645 learned_addr learned my addr 172.21.15.27:0/2821700645 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d0fe00640 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c72c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15200640 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c72700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 -- 172.21.15.27:0/2821700645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 msgr2=0x5624c6c72700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c72700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 -- 172.21.15.27:0/2821700645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 msgr2=0x5624c6c72c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c72c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.885+0000 7f9d15c00640 1 -- 172.21.15.27:0/2821700645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c6b65460 con 0x5624c68cb520 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.886+0000 7f9d15c00640 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6c721c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9d080099f0 tx=0x7f9d080095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b7f72074a5d982 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.886+0000 7f9d02000640 1 -- 172.21.15.27:0/2821700645 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0801d070 con 0x5624c68cb520 2024-09-21T07:19:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.886+0000 7f9d02000640 1 -- 172.21.15.27:0/2821700645 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d08003e40 con 0x5624c68cb520 2024-09-21T07:19:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.886+0000 7f9d02000640 1 -- 172.21.15.27:0/2821700645 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800d9d0 con 0x5624c68cb520 2024-09-21T07:19:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 -- 172.21.15.27:0/2821700645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 msgr2=0x5624c6c721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6c721c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9d080099f0 tx=0x7f9d080095c0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 -- 172.21.15.27:0/2821700645 shutdown_connections 2024-09-21T07:19:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c72c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c72700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 --2- 172.21.15.27:0/2821700645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6c721c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 -- 172.21.15.27:0/2821700645 >> 172.21.15.27:0/2821700645 conn(0x5624c68bf910 msgr2=0x5624c6c700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 -- 172.21.15.27:0/2821700645 shutdown_connections 2024-09-21T07:19:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.887+0000 7f9d1e832740 1 -- 172.21.15.27:0/2821700645 wait complete. 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.888+0000 7f9d1e832740 1 -- start start 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.888+0000 7f9d1e832740 1 Processor -- start 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.888+0000 7f9d0d600640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d0d600640 1 -- 172.21.15.27:0/1284136392 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9d04107410 con 0x7f9d10103b60 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d15c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c7dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c7e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c89c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d1e832740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5624c6a39bb0 con 0x5624c68aa840 2024-09-21T07:19:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d1e832740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5624c6b736c0 con 0x5624c68cb520 2024-09-21T07:19:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d1e832740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5624c6c809a0 con 0x5624c68a2660 2024-09-21T07:19:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d1e832740 10 client.0 _ll_register_callbacks cb 0x5624c6b8e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d15200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c7dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d15c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c89c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d15200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c7dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38806/0 (socket says 172.21.15.27:38806) 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d15200640 1 -- 172.21.15.27:0/1581282646 learned_addr learned my addr 172.21.15.27:0/1581282646 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.889+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c7e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d15200640 1 -- 172.21.15.27:0/1581282646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 msgr2=0x5624c6c89c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d15200640 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c89c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d15200640 1 -- 172.21.15.27:0/1581282646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 msgr2=0x5624c6c7e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d15200640 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c7e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d15200640 1 -- 172.21.15.27:0/1581282646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c6936b00 con 0x5624c68a2660 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d0fe00640 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c7e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d15c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009e20 tx=0x7f9d08003dc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9d15200640 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c7dd60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ca220 tx=0x7f9d040303b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d658c1ac60c6adb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.890+0000 7f9cf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.891+0000 7f9cf6a00640 1 -- 172.21.15.27:0/1581282646 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040c8e50 con 0x5624c68a2660 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.891+0000 7f9cf6a00640 1 -- 172.21.15.27:0/1581282646 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9d040c9220 con 0x5624c68a2660 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.891+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5624c6beb1e0 con 0x5624c68a2660 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.891+0000 7f9cf6a00640 1 -- 172.21.15.27:0/1581282646 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d040cc720 con 0x5624c68a2660 2024-09-21T07:19:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.891+0000 7f9d1e832740 10 client.27175 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.891+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624c69ad960 con 0x5624c68a2660 2024-09-21T07:19:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9cf6a00640 1 -- 172.21.15.27:0/1581282646 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5624c6beb1e0 con 0x5624c68a2660 2024-09-21T07:19:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9cf6a00640 1 -- 172.21.15.27:0/1581282646 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9d040b40f0 con 0x5624c68a2660 2024-09-21T07:19:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9d1e832740 10 client.27175 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9d1e832740 10 client.27175 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:19:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9d1e832740 20 client.27175 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9d1e832740 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c6889ea0 0x5624c6c8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9d1e832740 4 client.27175 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.892+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624c6be6c90 con 0x5624c6889ea0 2024-09-21T07:19:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.893+0000 7f9d0fe00640 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c6889ea0 0x5624c6c8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.893+0000 7f9d0fe00640 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c6889ea0 0x5624c6c8fb30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66417a1b906b0f95 server_cookie=2c30952e226259ec in_seq=0 out_seq=0 2024-09-21T07:19:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.893+0000 7f9cf6a00640 10 client.27175 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.894+0000 7f9cf6a00640 1 -- 172.21.15.27:0/1581282646 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624c6be6c90 con 0x5624c6889ea0 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.894+0000 7f9cf6a00640 10 client.27175 handle_command_reply: tid=0 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.894+0000 7f9d1e832740 1 client.27175 shutdown 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.894+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 msgr2=0x5624c6c7dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.894+0000 7f9d1e832740 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c7dd60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ca220 tx=0x7f9d040303b0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 shutdown_connections 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c6889ea0 0x5624c6c8fb30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c89c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c7e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 --2- 172.21.15.27:0/1581282646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c7dd60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 >> 172.21.15.27:0/1581282646 conn(0x5624c68bf910 msgr2=0x5624c6c753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9cf6a00640 0 client.27175 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 shutdown_connections 2024-09-21T07:19:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 1 -- 172.21.15.27:0/1581282646 wait complete. 2024-09-21T07:19:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.895+0000 7f9d1e832740 20 client.27175 trim_cache size 0 max 16384 2024-09-21T07:19:00.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.914+0000 7f9d1e832740 1 Processor -- start 2024-09-21T07:19:00.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.914+0000 7f9d1e832740 1 -- start start 2024-09-21T07:19:00.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c79310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c73bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d1e832740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101b9c10 con 0x5624c68aa840 2024-09-21T07:19:00.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d1e832740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101b74e0 con 0x5624c68cb520 2024-09-21T07:19:00.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d1e832740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101b5330 con 0x5624c68a2660 2024-09-21T07:19:00.917 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:00 smithi027 ceph-mon[30779]: pgmap v557: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c73bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d15c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d15c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c759d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60900/0 (socket says 172.21.15.27:60900) 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d15c00640 1 -- 172.21.15.27:0/3508494701 learned_addr learned my addr 172.21.15.27:0/3508494701 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.915+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c73bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48460/0 (socket says 172.21.15.27:48460) 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d15c00640 1 -- 172.21.15.27:0/3508494701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 msgr2=0x5624c6c79310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d15c00640 1 --2- 172.21.15.27:0/3508494701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c79310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d15c00640 1 -- 172.21.15.27:0/3508494701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 msgr2=0x5624c6c73bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d15c00640 1 --2- 172.21.15.27:0/3508494701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c73bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d15c00640 1 -- 172.21.15.27:0/3508494701 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c6c78770 con 0x5624c68cb520 2024-09-21T07:19:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d15c00640 1 --2- 172.21.15.27:0/3508494701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c759d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9d08000f80 tx=0x7f9d0802d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f49a0cdd8939ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d01600640 1 -- 172.21.15.27:0/3508494701 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080047c0 con 0x5624c68cb520 2024-09-21T07:19:00.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d01600640 1 -- 172.21.15.27:0/3508494701 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0801e050 con 0x5624c68cb520 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.916+0000 7f9d01600640 1 -- 172.21.15.27:0/3508494701 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080055a0 con 0x5624c68cb520 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.917+0000 7f9d1e832740 1 -- 172.21.15.27:0/3508494701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 msgr2=0x5624c6c759d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.917+0000 7f9d1e832740 1 --2- 172.21.15.27:0/3508494701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c759d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9d08000f80 tx=0x7f9d0802d000 comp rx=0 tx=0).stop 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.917+0000 7f9d1e832740 1 -- 172.21.15.27:0/3508494701 shutdown_connections 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.917+0000 7f9d1e832740 1 --2- 172.21.15.27:0/3508494701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68cb520 0x5624c6c759d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.917+0000 7f9d1e832740 1 --2- 172.21.15.27:0/3508494701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c73bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.917+0000 7f9d1e832740 1 --2- 172.21.15.27:0/3508494701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68a2660 0x5624c6c79310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.917+0000 7f9d1e832740 1 -- 172.21.15.27:0/3508494701 >> 172.21.15.27:0/3508494701 conn(0x5624c68bf910 msgr2=0x5624c6c752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.918+0000 7f9d1e832740 1 -- 172.21.15.27:0/3508494701 shutdown_connections 2024-09-21T07:19:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.918+0000 7f9d1e832740 1 -- 172.21.15.27:0/3508494701 wait complete. 2024-09-21T07:19:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.918+0000 7f9d1e832740 1 -- start start 2024-09-21T07:19:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.918+0000 7f9d1e832740 1 Processor -- start 2024-09-21T07:19:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c82490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c82a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d1e832740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6cf34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d1e832740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101b59a0 con 0x5624c68aa840 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d1e832740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101b6e10 con 0x5624c68a2660 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d1e832740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101b6070 con 0x5624c68cb520 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d1e832740 10 client.0 _ll_register_callbacks cb 0x7f9d04005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c82a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d15c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6cf34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c82a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48470/0 (socket says 172.21.15.27:48470) 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d0fe00640 1 -- 172.21.15.27:0/982390292 learned_addr learned my addr 172.21.15.27:0/982390292 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d15200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c82490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.919+0000 7f9d15c00640 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6cf34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38824/0 (socket says 172.21.15.27:38824) 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9d15200640 1 -- 172.21.15.27:0/982390292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 msgr2=0x5624c6cf34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9d15200640 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6cf34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9d15200640 1 -- 172.21.15.27:0/982390292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 msgr2=0x5624c6c82a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9d15200640 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c82a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9d15200640 1 -- 172.21.15.27:0/982390292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c6cf3b30 con 0x5624c68a2660 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9d15c00640 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6cf34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9d15200640 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c82490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ccf40 tx=0x7f9d041073a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412a1945611ae662 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9cf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9cf6a00640 1 -- 172.21.15.27:0/982390292 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040cc900 con 0x5624c68a2660 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.920+0000 7f9cf6a00640 1 -- 172.21.15.27:0/982390292 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f9d040c8a90 con 0x5624c68a2660 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0801e050 con 0x5624c68a2660 2024-09-21T07:19:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9cf6a00640 1 -- 172.21.15.27:0/982390292 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d040ccd80 con 0x5624c68a2660 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 10 client.27200 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624c6cfae70 con 0x5624c68a2660 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9cf6a00640 1 -- 172.21.15.27:0/982390292 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0801e050 con 0x5624c68a2660 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9cf6a00640 1 -- 172.21.15.27:0/982390292 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9d040b8080 con 0x5624c68a2660 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 10 client.27200 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 10 client.27200 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 20 client.27200 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c685f250 0x5624c6cfd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 4 client.27200 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7b1b6a1a", "release": true, "await": true}] 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.921+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7b1b6a1a", "release": true, "await": true}) -- 0x5624c6be40c0 con 0x5624c685f250 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.922+0000 7f9d0fe00640 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c685f250 0x5624c6cfd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.922+0000 7f9d0fe00640 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c685f250 0x5624c6cfd490 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7ebbf376d143f3 server_cookie=3f66e1da7950dcfe in_seq=0 out_seq=0 2024-09-21T07:19:00.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.922+0000 7f9cf6a00640 10 client.27200 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:00.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:00 smithi153 ceph-mon[34747]: pgmap v557: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-21T07:19:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.925+0000 7f9cf6a00640 1 -- 172.21.15.27:0/982390292 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5624c6be40c0 con 0x5624c685f250 2024-09-21T07:19:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.925+0000 7f9cf6a00640 10 client.27200 handle_command_reply: tid=0 2024-09-21T07:19:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 client.27200 shutdown 2024-09-21T07:19:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 msgr2=0x5624c6c82490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c82490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ccf40 tx=0x7f9d041073a0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 shutdown_connections 2024-09-21T07:19:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5624c685f250 0x5624c6cfd490 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624c68cb520 0x5624c6cf34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5624c68aa840 0x5624c6c82a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 --2- 172.21.15.27:0/982390292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624c68a2660 0x5624c6c82490 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9cf6a00640 0 client.27200 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 >> 172.21.15.27:0/982390292 conn(0x5624c68bf910 msgr2=0x5624c6c73990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 shutdown_connections 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 1 -- 172.21.15.27:0/982390292 wait complete. 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.926+0000 7f9d1e832740 20 client.27200 trim_cache size 0 max 16384 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 24, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "7b1b6a1a": { 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 24, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:19:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:00.931 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cf80b6590 msgr2=0x7f9cf80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009e20 tx=0x7f9d08003dc0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d10103b60 msgr2=0x7f9d1010e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d10103b60 0x7f9d1010e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d04035040 tx=0x7f9d04032dc0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 shutdown_connections 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d10109df0 0x7f9d1010acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d101094a0 0x7f9d1010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 --2- 172.21.15.27:0/1284136392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d10103b60 0x7f9d1010e2f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 >> 172.21.15.27:0/1284136392 conn(0x7f9d1007b850 msgr2=0x7f9d101072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 shutdown_connections 2024-09-21T07:19:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:00.927+0000 7f9d17c00640 1 -- 172.21.15.27:0/1284136392 wait complete. 2024-09-21T07:19:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:00 smithi119 ceph-mon[34519]: pgmap v557: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-21T07:19:01.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 -- 172.21.15.27:0/1220898844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f410dc20 msgr2=0x7f47f4110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 --2- 172.21.15.27:0/1220898844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f410dc20 0x7f47f4110110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009ca0 tx=0x7f47ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 -- 172.21.15.27:0/1220898844 shutdown_connections 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 --2- 172.21.15.27:0/1220898844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f410dc20 0x7f47f4110110 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 --2- 172.21.15.27:0/1220898844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f4105110 0x7f47f410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 --2- 172.21.15.27:0/1220898844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f41047c0 0x7f47f4104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 -- 172.21.15.27:0/1220898844 >> 172.21.15.27:0/1220898844 conn(0x7f47f40fe2c0 msgr2=0x7f47f4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 -- 172.21.15.27:0/1220898844 shutdown_connections 2024-09-21T07:19:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.542+0000 7f47fa600640 1 -- 172.21.15.27:0/1220898844 wait complete. 2024-09-21T07:19:01.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.543+0000 7f47fa600640 1 Processor -- start 2024-09-21T07:19:01.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47fa600640 1 -- start start 2024-09-21T07:19:01.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47fa600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f41047c0 0x7f47f419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47fa600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f4105110 0x7f47f419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47fa600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f410dc20 0x7f47f41a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47fa600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41138f0 con 0x7f47f41047c0 2024-09-21T07:19:01.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47fa600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4113770 con 0x7f47f410dc20 2024-09-21T07:19:01.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47fa600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4113a70 con 0x7f47f4105110 2024-09-21T07:19:01.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.544+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f4105110 0x7f47f419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f41047c0 0x7f47f419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f4105110 0x7f47f419c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38854/0 (socket says 172.21.15.27:38854) 2024-09-21T07:19:01.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f3e00640 1 -- 172.21.15.27:0/1835092812 learned_addr learned my addr 172.21.15.27:0/1835092812 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f9c00640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f410dc20 0x7f47f41a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f9200640 1 -- 172.21.15.27:0/1835092812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f4105110 msgr2=0x7f47f419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f9200640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f4105110 0x7f47f419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f9200640 1 -- 172.21.15.27:0/1835092812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f410dc20 msgr2=0x7f47f41a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f9200640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f410dc20 0x7f47f41a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f9200640 1 -- 172.21.15.27:0/1835092812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f41a23b0 con 0x7f47f41047c0 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.545+0000 7f47f3e00640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f4105110 0x7f47f419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.546+0000 7f47f9200640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f41047c0 0x7f47f419c440 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f47e800e9f0 tx=0x7f47e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.546+0000 7f47f1600640 1 -- 172.21.15.27:0/1835092812 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e800ccc0 con 0x7f47f41047c0 2024-09-21T07:19:01.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.546+0000 7f47f1600640 1 -- 172.21.15.27:0/1835092812 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e8005020 con 0x7f47f41047c0 2024-09-21T07:19:01.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.546+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41a26a0 con 0x7f47f41047c0 2024-09-21T07:19:01.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.546+0000 7f47f1600640 1 -- 172.21.15.27:0/1835092812 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8004a00 con 0x7f47f41047c0 2024-09-21T07:19:01.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.546+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41a8e00 con 0x7f47f41047c0 2024-09-21T07:19:01.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.549+0000 7f4802863740 1 Processor -- start 2024-09-21T07:19:01.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f4802863740 1 -- start start 2024-09-21T07:19:01.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f47f1600640 1 -- 172.21.15.27:0/1835092812 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f47e8004ba0 con 0x7f47f41047c0 2024-09-21T07:19:01.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f4802863740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d04430f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f4802863740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d040611b0 0x556d044314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f4802863740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d044319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f4802863740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556d043a7fb0 con 0x556d0408a070 2024-09-21T07:19:01.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f4802863740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556d043973e0 con 0x556d040611b0 2024-09-21T07:19:01.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f4802863740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556d0439b1f0 con 0x556d04069390 2024-09-21T07:19:01.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f47f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d04430f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f47f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d04430f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48504/0 (socket says 172.21.15.27:48504) 2024-09-21T07:19:01.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f47f9c00640 1 -- 172.21.15.27:0/2162187909 learned_addr learned my addr 172.21.15.27:0/2162187909 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:01.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f47f9200640 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d040611b0 0x556d044314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.550+0000 7f47f3e00640 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d044319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.551+0000 7f47f9c00640 1 -- 172.21.15.27:0/2162187909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 msgr2=0x556d044319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.551+0000 7f47f9c00640 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d044319f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.551+0000 7f47f9c00640 1 -- 172.21.15.27:0/2162187909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d040611b0 msgr2=0x556d044314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.551+0000 7f47f9c00640 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d040611b0 0x556d044314b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.551+0000 7f47f9c00640 1 -- 172.21.15.27:0/2162187909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d04361cd0 con 0x556d0408a070 2024-09-21T07:19:01.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.551+0000 7f47f9200640 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d040611b0 0x556d044314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:01.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.551+0000 7f47f9c00640 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d04430f70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009ca0 tx=0x7f47ec009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc06d8b55ae25306 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:01.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.552+0000 7f47e6000640 1 -- 172.21.15.27:0/2162187909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec031cf0 con 0x556d0408a070 2024-09-21T07:19:01.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.552+0000 7f47e6000640 1 -- 172.21.15.27:0/2162187909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec036030 con 0x556d0408a070 2024-09-21T07:19:01.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.552+0000 7f47e6000640 1 -- 172.21.15.27:0/2162187909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0042a0 con 0x556d0408a070 2024-09-21T07:19:01.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.552+0000 7f47f1600640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47dc0b6680 0x7f47dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.552+0000 7f47f1600640 1 -- 172.21.15.27:0/1835092812 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f47e80e7a20 con 0x7f47f41047c0 2024-09-21T07:19:01.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f47f9200640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47dc0b6680 0x7f47dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 -- 172.21.15.27:0/2162187909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 msgr2=0x556d04430f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d04430f70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009ca0 tx=0x7f47ec009430 comp rx=0 tx=0).stop 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 -- 172.21.15.27:0/2162187909 shutdown_connections 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d044319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d040611b0 0x556d044314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 --2- 172.21.15.27:0/2162187909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d04430f70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 -- 172.21.15.27:0/2162187909 >> 172.21.15.27:0/2162187909 conn(0x556d0407e460 msgr2=0x556d0442ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 -- 172.21.15.27:0/2162187909 shutdown_connections 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f4802863740 1 -- 172.21.15.27:0/2162187909 wait complete. 2024-09-21T07:19:01.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.553+0000 7f47f9200640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47dc0b6680 0x7f47dc0b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x556d0442fbf0 tx=0x7f47e80ed530 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.554+0000 7f4802863740 1 -- start start 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.554+0000 7f4802863740 1 Processor -- start 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.554+0000 7f4802863740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d040611b0 0x556d0443c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.554+0000 7f4802863740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d0443cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.554+0000 7f4802863740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d04448890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f4802863740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556d041f8700 con 0x556d040611b0 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f4802863740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556d04332580 con 0x556d0408a070 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f4802863740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556d0443f570 con 0x556d04069390 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f4802863740 10 client.0 _ll_register_callbacks cb 0x556d04342470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d040611b0 0x556d0443c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d040611b0 0x556d0443c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48514/0 (socket says 172.21.15.27:48514) 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 -- 172.21.15.27:0/675499737 learned_addr learned my addr 172.21.15.27:0/675499737 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f3e00640 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d0443cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d04448890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 -- 172.21.15.27:0/675499737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 msgr2=0x556d0443cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d0443cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 -- 172.21.15.27:0/675499737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 msgr2=0x556d04448890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d04448890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.555+0000 7f47f9c00640 1 -- 172.21.15.27:0/675499737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d040f5650 con 0x556d040611b0 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f47f9c00640 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d040611b0 0x556d0443c6c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f47ec031f40 tx=0x7f47ec003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bd908193c176c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f47daa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f47daa00640 1 -- 172.21.15.27:0/675499737 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec057060 con 0x556d040611b0 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f47daa00640 1 -- 172.21.15.27:0/675499737 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f47ec031750 con 0x556d040611b0 2024-09-21T07:19:01.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d04334b90 con 0x556d040611b0 2024-09-21T07:19:01.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f47daa00640 1 -- 172.21.15.27:0/675499737 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47ec05e050 con 0x556d040611b0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f4802863740 10 client.17184 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.556+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d0434f950 con 0x556d040611b0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f47daa00640 1 -- 172.21.15.27:0/675499737 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556d04334b90 con 0x556d040611b0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f47daa00640 1 -- 172.21.15.27:0/675499737 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f47ec043070 con 0x556d040611b0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f4802863740 10 client.17184 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f4802863740 10 client.17184 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f4802863740 20 client.17184 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f4802863740 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d040489f0 0x556d0444e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f4802863740 4 client.17184 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d043a9f00 con 0x556d040489f0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.557+0000 7f47f3e00640 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d040489f0 0x556d0444e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.558+0000 7f47f3e00640 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d040489f0 0x556d0444e470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c82c0b446698794 server_cookie=c6e35442d1d7bf38 in_seq=0 out_seq=0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.558+0000 7f47daa00640 10 client.17184 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f47daa00640 1 -- 172.21.15.27:0/675499737 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d043a9f00 con 0x556d040489f0 2024-09-21T07:19:01.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f47daa00640 10 client.17184 handle_command_reply: tid=0 2024-09-21T07:19:01.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 client.17184 shutdown 2024-09-21T07:19:01.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d040611b0 msgr2=0x556d0443c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d040611b0 0x556d0443c6c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f47ec031f40 tx=0x7f47ec003d40 comp rx=0 tx=0).stop 2024-09-21T07:19:01.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 shutdown_connections 2024-09-21T07:19:01.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d040489f0 0x556d0444e470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d04069390 0x556d0443cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d04448890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 --2- 172.21.15.27:0/675499737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d040611b0 0x556d0443c6c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 >> 172.21.15.27:0/675499737 conn(0x556d0407e460 msgr2=0x556d04434120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f47daa00640 0 client.17184 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 shutdown_connections 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.559+0000 7f4802863740 1 -- 172.21.15.27:0/675499737 wait complete. 2024-09-21T07:19:01.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.560+0000 7f4802863740 20 client.17184 trim_cache size 0 max 16384 2024-09-21T07:19:01.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 Processor -- start 2024-09-21T07:19:01.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 -- start start 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d0442dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d04069390 0x556d04448800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d044398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b5570 con 0x556d04069390 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b2e40 con 0x556d0408a070 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.574+0000 7f4802863740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b0c90 con 0x556d040611b0 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d04069390 0x556d04448800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d04069390 0x556d04448800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48522/0 (socket says 172.21.15.27:48522) 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 -- 172.21.15.27:0/1948486146 learned_addr learned my addr 172.21.15.27:0/1948486146 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f9c00640 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d0442dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f9c00640 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d044398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 -- 172.21.15.27:0/1948486146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 msgr2=0x556d0442dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d0442dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 -- 172.21.15.27:0/1948486146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 msgr2=0x556d044398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d044398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f3e00640 1 -- 172.21.15.27:0/1948486146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0443d760 con 0x556d04069390 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.575+0000 7f47f9c00640 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d0442dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.576+0000 7f47f3e00640 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d04069390 0x556d04448800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f47e0012360 tx=0x7f47e0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04b614e0898d32a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.576+0000 7f47e5600640 1 -- 172.21.15.27:0/1948486146 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0010b30 con 0x556d04069390 2024-09-21T07:19:01.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.576+0000 7f47e5600640 1 -- 172.21.15.27:0/1948486146 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e00054d0 con 0x556d04069390 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.576+0000 7f47e5600640 1 -- 172.21.15.27:0/1948486146 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0004dd0 con 0x556d04069390 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 -- 172.21.15.27:0/1948486146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d04069390 msgr2=0x556d04448800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d04069390 0x556d04448800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f47e0012360 tx=0x7f47e0005f90 comp rx=0 tx=0).stop 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 -- 172.21.15.27:0/1948486146 shutdown_connections 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d0408a070 0x556d044398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d04069390 0x556d04448800 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 --2- 172.21.15.27:0/1948486146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d0442dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 -- 172.21.15.27:0/1948486146 >> 172.21.15.27:0/1948486146 conn(0x556d0407e460 msgr2=0x556d0442d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 -- 172.21.15.27:0/1948486146 shutdown_connections 2024-09-21T07:19:01.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 -- 172.21.15.27:0/1948486146 wait complete. 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.577+0000 7f4802863740 1 -- start start 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 1 Processor -- start 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d04452f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d04069390 0x556d0444e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d0444ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b1300 con 0x556d0408a070 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b2770 con 0x556d04069390 2024-09-21T07:19:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b19d0 con 0x556d040611b0 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.578+0000 7f4802863740 10 client.0 _ll_register_callbacks cb 0x7f47ec003b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d04452f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d04069390 0x556d0444e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d04069390 0x556d0444e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60962/0 (socket says 172.21.15.27:60962) 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d0444ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f3e00640 1 -- 172.21.15.27:0/731090746 learned_addr learned my addr 172.21.15.27:0/731090746 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f9c00640 1 -- 172.21.15.27:0/731090746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 msgr2=0x556d04452f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f9c00640 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d04452f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f9c00640 1 -- 172.21.15.27:0/731090746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d04069390 msgr2=0x556d0444e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f9c00640 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d04069390 0x556d0444e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.579+0000 7f47f9c00640 1 -- 172.21.15.27:0/731090746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d044b7be0 con 0x556d0408a070 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f47f9c00640 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d0444ebb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f47ec030830 tx=0x7f47ec003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0757c9d20e8b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f47daa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f47daa00640 1 -- 172.21.15.27:0/731090746 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec057030 con 0x556d0408a070 2024-09-21T07:19:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f47daa00640 1 -- 172.21.15.27:0/731090746 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9373+0+0 (secure 0 0 0) 0x7f47ec031650 con 0x556d0408a070 2024-09-21T07:19:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e00054d0 con 0x556d0408a070 2024-09-21T07:19:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f47daa00640 1 -- 172.21.15.27:0/731090746 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47ec073050 con 0x556d0408a070 2024-09-21T07:19:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f4802863740 10 client.17190 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.580+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d044b8d60 con 0x556d0408a070 2024-09-21T07:19:01.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f47daa00640 1 -- 172.21.15.27:0/731090746 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e00054d0 con 0x556d0408a070 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f47daa00640 1 -- 172.21.15.27:0/731090746 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f47ec043040 con 0x556d0408a070 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f4802863740 10 client.17190 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f4802863740 10 client.17190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f4802863740 20 client.17190 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f4802863740 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d0401dda0 0x556d044bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f4802863740 4 client.17190 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d043a59b0 con 0x556d0401dda0 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.581+0000 7f47f3e00640 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d0401dda0 0x556d044bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.582+0000 7f47f3e00640 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d0401dda0 0x556d044bb3b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ec9913ecb990382 server_cookie=38b6df65f92e82f5 in_seq=0 out_seq=0 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.582+0000 7f47daa00640 10 client.17190 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.582+0000 7f47daa00640 1 -- 172.21.15.27:0/731090746 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d043a59b0 con 0x556d0401dda0 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.582+0000 7f47daa00640 10 client.17190 handle_command_reply: tid=0 2024-09-21T07:19:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.582+0000 7f4802863740 1 client.17190 shutdown 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.582+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 msgr2=0x556d0444ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.582+0000 7f4802863740 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d0444ebb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f47ec030830 tx=0x7f47ec003a30 comp rx=0 tx=0).stop 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 shutdown_connections 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x556d0401dda0 0x556d044bb3b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0408a070 0x556d0444ebb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d04069390 0x556d0444e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 --2- 172.21.15.27:0/731090746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d040611b0 0x556d04452f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 >> 172.21.15.27:0/731090746 conn(0x556d0407e460 msgr2=0x556d04431e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f47daa00640 0 client.17190 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 shutdown_connections 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 1 -- 172.21.15.27:0/731090746 wait complete. 2024-09-21T07:19:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.583+0000 7f4802863740 20 client.17190 trim_cache size 0 max 16384 2024-09-21T07:19:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:19:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47dc0b6680 msgr2=0x7f47dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47dc0b6680 0x7f47dc0b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x556d0442fbf0 tx=0x7f47e80ed530 comp rx=0 tx=0).stop 2024-09-21T07:19:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f41047c0 msgr2=0x7f47f419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f41047c0 0x7f47f419c440 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f47e800e9f0 tx=0x7f47e800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 shutdown_connections 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f4105110 0x7f47f419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47dc0b6680 0x7f47dc0b8b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47f410dc20 0x7f47f41a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 --2- 172.21.15.27:0/1835092812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f41047c0 0x7f47f419c440 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 >> 172.21.15.27:0/1835092812 conn(0x7f47f40fe2c0 msgr2=0x7f47f41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 shutdown_connections 2024-09-21T07:19:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:01.584+0000 7f47fa600640 1 -- 172.21.15.27:0/1835092812 wait complete. 2024-09-21T07:19:02.584 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7b1b6a1a', total seconds elapsed: 24.5 2024-09-21T07:19:02.585 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T07:19:02.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:19:02.888 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:02 smithi027 ceph-mon[30779]: pgmap v558: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-21T07:19:02.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:19:02.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:02 smithi153 ceph-mon[34747]: pgmap v558: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-21T07:19:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:03.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:02 smithi119 ceph-mon[34519]: pgmap v558: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-21T07:19:03.659 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:03.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.676+0000 7feddd000640 1 -- 172.21.15.27:0/2407542286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 msgr2=0x7fedd8110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:03.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.676+0000 7feddd000640 1 --2- 172.21.15.27:0/2407542286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 0x7fedd8110630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009ca0 tx=0x7fedd002a8e0 comp rx=0 tx=0).stop 2024-09-21T07:19:03.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.677+0000 7feddd000640 1 -- 172.21.15.27:0/2407542286 shutdown_connections 2024-09-21T07:19:03.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.677+0000 7feddd000640 1 --2- 172.21.15.27:0/2407542286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 0x7fedd8110630 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:03.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.677+0000 7feddd000640 1 --2- 172.21.15.27:0/2407542286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd81094b0 0x7fedd81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.677+0000 7feddd000640 1 --2- 172.21.15.27:0/2407542286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 0x7fedd8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.677+0000 7feddd000640 1 -- 172.21.15.27:0/2407542286 >> 172.21.15.27:0/2407542286 conn(0x7fedd80ff5e0 msgr2=0x7fedd8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.677+0000 7feddd000640 1 -- 172.21.15.27:0/2407542286 shutdown_connections 2024-09-21T07:19:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.677+0000 7feddd000640 1 -- 172.21.15.27:0/2407542286 wait complete. 2024-09-21T07:19:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.679+0000 7feddd000640 1 Processor -- start 2024-09-21T07:19:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.679+0000 7feddd000640 1 -- start start 2024-09-21T07:19:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.679+0000 7feddd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 0x7fedd819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.679+0000 7feddd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd81094b0 0x7fedd819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7feddd000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 0x7fedd81a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7feddd000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8113b10 con 0x7fedd81094b0 2024-09-21T07:19:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7feddd000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8113990 con 0x7fedd8103b70 2024-09-21T07:19:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7feddd000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8113c90 con 0x7fedd8109e00 2024-09-21T07:19:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7fedd5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 0x7fedd819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7fedd5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 0x7fedd819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60986/0 (socket says 172.21.15.27:60986) 2024-09-21T07:19:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7fedd5e00640 1 -- 172.21.15.27:0/2432502671 learned_addr learned my addr 172.21.15.27:0/2432502671 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.680+0000 7fedd6800640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 0x7fedd81a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.681+0000 7fedd5e00640 1 -- 172.21.15.27:0/2432502671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 msgr2=0x7fedd81a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.681+0000 7fedd5e00640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 0x7fedd81a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.681+0000 7fedd5e00640 1 -- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd81094b0 msgr2=0x7fedd819cc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:19:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.682+0000 7fedd5400640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd81094b0 0x7fedd819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.683+0000 7fedd5e00640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd81094b0 0x7fedd819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.683+0000 7fedd5e00640 1 -- 172.21.15.27:0/2432502671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd81a25d0 con 0x7fedd8103b70 2024-09-21T07:19:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.683+0000 7fedd5400640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd81094b0 0x7fedd819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:19:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.683+0000 7fedd5e00640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 0x7fedd819c660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fedc800e9f0 tx=0x7fedc800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.685+0000 7fedcea00640 1 -- 172.21.15.27:0/2432502671 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc800cd80 con 0x7fedd8103b70 2024-09-21T07:19:03.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.685+0000 7fedcea00640 1 -- 172.21.15.27:0/2432502671 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedc80108e0 con 0x7fedd8103b70 2024-09-21T07:19:03.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.685+0000 7fedcea00640 1 -- 172.21.15.27:0/2432502671 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8018880 con 0x7fedd8103b70 2024-09-21T07:19:03.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.685+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd81a28c0 con 0x7fedd8103b70 2024-09-21T07:19:03.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.686+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd810a4e0 con 0x7fedd8103b70 2024-09-21T07:19:03.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.688+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd8104df0 con 0x7fedd8103b70 2024-09-21T07:19:03.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.692+0000 7fedcea00640 1 -- 172.21.15.27:0/2432502671 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fedc8010430 con 0x7fedd8103b70 2024-09-21T07:19:03.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.695+0000 7fedcea00640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedbc0b67d0 0x7fedbc0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:03.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.695+0000 7fedcea00640 1 -- 172.21.15.27:0/2432502671 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7fedc80e7f90 con 0x7fedd8103b70 2024-09-21T07:19:03.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.696+0000 7fedd5400640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedbc0b67d0 0x7fedbc0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:03.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.697+0000 7fedd5400640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedbc0b67d0 0x7fedbc0b8ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fedd8102dc0 tx=0x7fedc4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:03.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:03.698+0000 7fedcea00640 1 -- 172.21.15.27:0/2432502671 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fedc8056710 con 0x7fedd8103b70 2024-09-21T07:19:03.813 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:03 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:19:04.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.007+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedd810a800 con 0x7fedd8103b70 2024-09-21T07:19:04.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.008+0000 7fedcea00640 1 -- 172.21.15.27:0/2432502671 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fedc80ae0e0 con 0x7fedd8103b70 2024-09-21T07:19:04.011 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:19:04.011 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:19:04.011 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:19:04.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedbc0b67d0 msgr2=0x7fedbc0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedbc0b67d0 0x7fedbc0b8ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fedd8102dc0 tx=0x7fedc4005f90 comp rx=0 tx=0).stop 2024-09-21T07:19:04.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 msgr2=0x7fedd819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 0x7fedd819c660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fedc800e9f0 tx=0x7fedc800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 shutdown_connections 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedd8109e00 0x7fedd81a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fedbc0b67d0 0x7fedbc0b8ce0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd81094b0 0x7fedd819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 --2- 172.21.15.27:0/2432502671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedd8103b70 0x7fedd819c660 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.011+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 >> 172.21.15.27:0/2432502671 conn(0x7fedd80ff5e0 msgr2=0x7fedd8108b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.012+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 shutdown_connections 2024-09-21T07:19:04.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.012+0000 7feddd000640 1 -- 172.21.15.27:0/2432502671 wait complete. 2024-09-21T07:19:04.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:03 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:19:04.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:03 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 -- 172.21.15.27:0/167315004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf0103b60 msgr2=0x7efdf0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 --2- 172.21.15.27:0/167315004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf0103b60 0x7efdf0103f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efde4009900 tx=0x7efde402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 -- 172.21.15.27:0/167315004 shutdown_connections 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 --2- 172.21.15.27:0/167315004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf01094a0 0x7efdf01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 --2- 172.21.15.27:0/167315004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf0109df0 0x7efdf0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 --2- 172.21.15.27:0/167315004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf0103b60 0x7efdf0103f70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 -- 172.21.15.27:0/167315004 >> 172.21.15.27:0/167315004 conn(0x7efdf007b850 msgr2=0x7efdf007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 -- 172.21.15.27:0/167315004 shutdown_connections 2024-09-21T07:19:04.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.319+0000 7efdf8200640 1 -- 172.21.15.27:0/167315004 wait complete. 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.320+0000 7efdf8200640 1 Processor -- start 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.320+0000 7efdf8200640 1 -- start start 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf8200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf0103b60 0x7efdf019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf8200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf01094a0 0x7efdf019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf8200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf0109df0 0x7efdf019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf8200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0113b00 con 0x7efdf0109df0 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf8200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0113980 con 0x7efdf01094a0 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf8200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0113c80 con 0x7efdf0103b60 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf0109df0 0x7efdf019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf0109df0 0x7efdf019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48560/0 (socket says 172.21.15.27:48560) 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf6200640 1 -- 172.21.15.27:0/3757834288 learned_addr learned my addr 172.21.15.27:0/3757834288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf4e00640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf01094a0 0x7efdf019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.321+0000 7efdf5800640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf0103b60 0x7efdf019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf4e00640 1 -- 172.21.15.27:0/3757834288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf0103b60 msgr2=0x7efdf019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf4e00640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf0103b60 0x7efdf019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf4e00640 1 -- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf0109df0 msgr2=0x7efdf019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf4e00640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf0109df0 0x7efdf019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf4e00640 1 -- 172.21.15.27:0/3757834288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf01a03e0 con 0x7efdf01094a0 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf4e00640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf01094a0 0x7efdf019aa10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdf006c9f0 tx=0x7efddc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdee000640 1 -- 172.21.15.27:0/3757834288 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc00cc40 con 0x7efdf01094a0 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdee000640 1 -- 172.21.15.27:0/3757834288 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc00cde0 con 0x7efdf01094a0 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdee000640 1 -- 172.21.15.27:0/3757834288 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc018780 con 0x7efdf01094a0 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf01a06d0 con 0x7efdf01094a0 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.322+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01a6e30 con 0x7efdf01094a0 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.325+0000 7efdfefe9740 1 Processor -- start 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdfefe9740 1 -- start start 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1749070 0x55b3a1aefda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1af02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1af0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdfefe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1a66fb0 con 0x55b3a1728390 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdfefe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1a563e0 con 0x55b3a17201b0 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdfefe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1a5a1f0 con 0x55b3a1749070 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdee000640 1 -- 172.21.15.27:0/3757834288 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efddc010a90 con 0x7efdf01094a0 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdf5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1af02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdf5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1af02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:32782/0 (socket says 172.21.15.27:32782) 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdf5800640 1 -- 172.21.15.27:0/1405699676 learned_addr learned my addr 172.21.15.27:0/1405699676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.326+0000 7efdf6200640 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1749070 0x55b3a1aefda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efdf4e00640 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1af0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efdf5800640 1 -- 172.21.15.27:0/1405699676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1749070 msgr2=0x55b3a1aefda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efdf5800640 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1749070 0x55b3a1aefda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efdf5800640 1 -- 172.21.15.27:0/1405699676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 msgr2=0x55b3a1af0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efdf5800640 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1af0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efdf5800640 1 -- 172.21.15.27:0/1405699676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a1a20cd0 con 0x55b3a17201b0 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efdf5800640 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1af02e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efde4009230 tx=0x7efde4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14a89ed342047a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efde2a00640 1 -- 172.21.15.27:0/1405699676 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4035040 con 0x55b3a17201b0 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.327+0000 7efde2a00640 1 -- 172.21.15.27:0/1405699676 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde4004430 con 0x55b3a17201b0 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.328+0000 7efde2a00640 1 -- 172.21.15.27:0/1405699676 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde40035f0 con 0x55b3a17201b0 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.328+0000 7efdee000640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdd80b6700 0x7efdd80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.328+0000 7efdee000640 1 -- 172.21.15.27:0/3757834288 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7efddc0e7d80 con 0x7efdf01094a0 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.328+0000 7efdfefe9740 1 -- 172.21.15.27:0/1405699676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 msgr2=0x55b3a1af02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.328+0000 7efdfefe9740 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1af02e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efde4009230 tx=0x7efde4004040 comp rx=0 tx=0).stop 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.328+0000 7efdf6200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdd80b6700 0x7efdd80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 -- 172.21.15.27:0/1405699676 shutdown_connections 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1af0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1af02e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 --2- 172.21.15.27:0/1405699676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1749070 0x55b3a1aefda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 -- 172.21.15.27:0/1405699676 >> 172.21.15.27:0/1405699676 conn(0x55b3a173d460 msgr2=0x55b3a1aedc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 -- 172.21.15.27:0/1405699676 shutdown_connections 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 -- 172.21.15.27:0/1405699676 wait complete. 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdf6200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdd80b6700 0x7efdd80b8c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efde80049d0 tx=0x7efde8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 -- start start 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.329+0000 7efdfefe9740 1 Processor -- start 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1aff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1affe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1749070 0x55b3a1b0bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdfefe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b3a18b7700 con 0x55b3a1749070 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdfefe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3a19f1580 con 0x55b3a17201b0 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdfefe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1b02850 con 0x55b3a1728390 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdfefe9740 10 client.0 _ll_register_callbacks cb 0x55b3a1a01470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdf5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1aff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdf6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1749070 0x55b3a1b0bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdf5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1aff8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:32788/0 (socket says 172.21.15.27:32788) 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdf5800640 1 -- 172.21.15.27:0/4183656517 learned_addr learned my addr 172.21.15.27:0/4183656517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.330+0000 7efdf4e00640 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1affe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdf5800640 1 -- 172.21.15.27:0/4183656517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 msgr2=0x55b3a1affe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdf5800640 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1affe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdf5800640 1 -- 172.21.15.27:0/4183656517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1749070 msgr2=0x55b3a1b0bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdf5800640 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1749070 0x55b3a1b0bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdf5800640 1 -- 172.21.15.27:0/4183656517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a17b4650 con 0x55b3a17201b0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdf5800640 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1aff8e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efde402a910 tx=0x7efde402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287a9b93f10e619a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdd7400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdd7400640 1 -- 172.21.15.27:0/4183656517 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4053060 con 0x55b3a17201b0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3a19f3b90 con 0x55b3a17201b0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.331+0000 7efdd7400640 1 -- 172.21.15.27:0/4183656517 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7efde4004440 con 0x55b3a17201b0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdd7400640 1 -- 172.21.15.27:0/4183656517 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efde4035050 con 0x55b3a17201b0 2024-09-21T07:19:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 10 client.27236 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3a1a0e950 con 0x55b3a17201b0 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdd7400640 1 -- 172.21.15.27:0/4183656517 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b3a19f3b90 con 0x55b3a17201b0 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdd7400640 1 -- 172.21.15.27:0/4183656517 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efde4031810 con 0x55b3a17201b0 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 10 client.27236 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 10 client.27236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 20 client.27236 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a17079f0 0x55b3a1b117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 4 client.27236 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.332+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3a1a68f00 con 0x55b3a17079f0 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.333+0000 7efdf4e00640 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a17079f0 0x55b3a1b117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.333+0000 7efdf4e00640 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a17079f0 0x55b3a1b117b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e7ce245b810fef server_cookie=51c591263c64031b in_seq=0 out_seq=0 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.333+0000 7efdd7400640 10 client.27236 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdd7400640 1 -- 172.21.15.27:0/4183656517 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3a1a68f00 con 0x55b3a17079f0 2024-09-21T07:19:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdd7400640 10 client.27236 handle_command_reply: tid=0 2024-09-21T07:19:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 client.27236 shutdown 2024-09-21T07:19:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 msgr2=0x55b3a1aff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1aff8e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efde402a910 tx=0x7efde402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 shutdown_connections 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a17079f0 0x55b3a1b117b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1affe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1749070 0x55b3a1b0bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 --2- 172.21.15.27:0/4183656517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a17201b0 0x55b3a1aff8e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 >> 172.21.15.27:0/4183656517 conn(0x55b3a173d460 msgr2=0x55b3a1af2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdd7400640 0 client.27236 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 shutdown_connections 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.334+0000 7efdfefe9740 1 -- 172.21.15.27:0/4183656517 wait complete. 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.335+0000 7efdfefe9740 20 client.27236 trim_cache size 0 max 16384 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 Processor -- start 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 -- start start 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a17201b0 0x55b3a1afb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1af14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1afcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b1600 con 0x55b3a17201b0 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01aeed0 con 0x55b3a1749070 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.354+0000 7efdfefe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01acd40 con 0x55b3a1728390 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1afcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a17201b0 0x55b3a1afb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1afcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:32802/0 (socket says 172.21.15.27:32802) 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 -- 172.21.15.27:0/412098368 learned_addr learned my addr 172.21.15.27:0/412098368 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf4e00640 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1af14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 -- 172.21.15.27:0/412098368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 msgr2=0x55b3a1af14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1af14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 -- 172.21.15.27:0/412098368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a17201b0 msgr2=0x55b3a1afb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a17201b0 0x55b3a1afb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf6200640 1 -- 172.21.15.27:0/412098368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a1afa4e0 con 0x55b3a1749070 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf4e00640 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1af14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.355+0000 7efdf5800640 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a17201b0 0x55b3a1afb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.356+0000 7efdf6200640 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1afcad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efde800fb00 tx=0x7efde8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdeccabd123569e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.356+0000 7efde2000640 1 -- 172.21.15.27:0/412098368 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde801d600 con 0x55b3a1749070 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.356+0000 7efde2000640 1 -- 172.21.15.27:0/412098368 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde802d070 con 0x55b3a1749070 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.356+0000 7efde2000640 1 -- 172.21.15.27:0/412098368 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8028c00 con 0x55b3a1749070 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 -- 172.21.15.27:0/412098368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 msgr2=0x55b3a1afcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1afcad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efde800fb00 tx=0x7efde8017000 comp rx=0 tx=0).stop 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 -- 172.21.15.27:0/412098368 shutdown_connections 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a1728390 0x55b3a1af14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1afcad0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 --2- 172.21.15.27:0/412098368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a17201b0 0x55b3a1afb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 -- 172.21.15.27:0/412098368 >> 172.21.15.27:0/412098368 conn(0x55b3a173d460 msgr2=0x55b3a1af0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 -- 172.21.15.27:0/412098368 shutdown_connections 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.357+0000 7efdfefe9740 1 -- 172.21.15.27:0/412098368 wait complete. 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 -- start start 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 Processor -- start 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a17201b0 0x55b3a1b12830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1b12d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1b132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01ad390 con 0x55b3a1728390 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01ae800 con 0x55b3a1749070 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01ada60 con 0x55b3a17201b0 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.358+0000 7efdfefe9740 10 client.0 _ll_register_callbacks cb 0x7efde4009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1b12d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf5800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a17201b0 0x55b3a1b12830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1b132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1b12d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48578/0 (socket says 172.21.15.27:48578) 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf4e00640 1 -- 172.21.15.27:0/656739546 learned_addr learned my addr 172.21.15.27:0/656739546 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf5800640 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a17201b0 0x55b3a1b12830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38956/0 (socket says 172.21.15.27:38956) 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf6200640 1 -- 172.21.15.27:0/656739546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a17201b0 msgr2=0x55b3a1b12830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf6200640 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a17201b0 0x55b3a1b12830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf6200640 1 -- 172.21.15.27:0/656739546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 msgr2=0x55b3a1b12d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf6200640 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1b12d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf6200640 1 -- 172.21.15.27:0/656739546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a1b7af80 con 0x55b3a1749070 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.359+0000 7efdf4e00640 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1b12d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdf6200640 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1b132b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efde800fad0 tx=0x7efde8004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d3dbcc10b94e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdd7400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdd7400640 1 -- 172.21.15.27:0/656739546 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8035030 con 0x55b3a1749070 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde802d070 con 0x55b3a1749070 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdd7400640 1 -- 172.21.15.27:0/656739546 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7efde8028b50 con 0x55b3a1749070 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdd7400640 1 -- 172.21.15.27:0/656739546 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efde800f750 con 0x55b3a1749070 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdfefe9740 10 client.27242 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.360+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3a1b7c200 con 0x55b3a1749070 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdd7400640 1 -- 172.21.15.27:0/656739546 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde802d070 con 0x55b3a1749070 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdd7400640 1 -- 172.21.15.27:0/656739546 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efde8005040 con 0x55b3a1749070 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdfefe9740 10 client.27242 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdfefe9740 10 client.27242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdfefe9740 20 client.27242 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdfefe9740 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a16dcda0 0x55b3a1b7e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdfefe9740 4 client.27242 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3a1b00250 con 0x55b3a16dcda0 2024-09-21T07:19:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.361+0000 7efdf5800640 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a16dcda0 0x55b3a1b7e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdf5800640 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a16dcda0 0x55b3a1b7e8f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72955b398cdc88e6 server_cookie=ffbc450d43c5b6dc in_seq=0 out_seq=0 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdd7400640 10 client.27242 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdd7400640 1 -- 172.21.15.27:0/656739546 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3a1b00250 con 0x55b3a16dcda0 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdd7400640 10 client.27242 handle_command_reply: tid=0 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 client.27242 shutdown 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 msgr2=0x55b3a1b132b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1b132b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efde800fad0 tx=0x7efde8004e20 comp rx=0 tx=0).stop 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 shutdown_connections 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3a16dcda0 0x55b3a1b7e8f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3a1749070 0x55b3a1b132b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3a1728390 0x55b3a1b12d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdd7400640 0 client.27242 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 --2- 172.21.15.27:0/656739546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3a17201b0 0x55b3a1b12830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.362+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 >> 172.21.15.27:0/656739546 conn(0x55b3a173d460 msgr2=0x55b3a1af0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:04.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.363+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 shutdown_connections 2024-09-21T07:19:04.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.363+0000 7efdfefe9740 1 -- 172.21.15.27:0/656739546 wait complete. 2024-09-21T07:19:04.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.363+0000 7efdfefe9740 20 client.27242 trim_cache size 0 max 16384 2024-09-21T07:19:04.517 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:19:04.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdd80b6700 msgr2=0x7efdd80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdd80b6700 0x7efdd80b8c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efde80049d0 tx=0x7efde8009500 comp rx=0 tx=0).stop 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf01094a0 msgr2=0x7efdf019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf01094a0 0x7efdf019aa10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdf006c9f0 tx=0x7efddc00ee50 comp rx=0 tx=0).stop 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 shutdown_connections 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdd80b6700 0x7efdd80b8c10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf0109df0 0x7efdf019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf01094a0 0x7efdf019aa10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 --2- 172.21.15.27:0/3757834288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf0103b60 0x7efdf019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 >> 172.21.15.27:0/3757834288 conn(0x7efdf007b850 msgr2=0x7efdf01074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 shutdown_connections 2024-09-21T07:19:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:04.364+0000 7efdf8200640 1 -- 172.21.15.27:0/3757834288 wait complete. 2024-09-21T07:19:05.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: pgmap v559: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 50 KiB/s wr, 24 op/s 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: Unable to set osd_memory_target on smithi153 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2432502671' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:19:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:19:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: pgmap v559: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 50 KiB/s wr, 24 op/s 2024-09-21T07:19:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T07:19:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: Unable to set osd_memory_target on smithi153 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2432502671' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:19:05.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:19:05.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:19:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: pgmap v559: 129 pgs: 129 active+clean; 2.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 50 KiB/s wr, 24 op/s 2024-09-21T07:19:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: Unable to set osd_memory_target on smithi153 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2432502671' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:19:05.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:19:05.232 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T07:19:05.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T07:19:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:05.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:05 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T07:19:05.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:05 smithi027 ceph-mon[30779]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T07:19:06.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:05 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T07:19:06.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:05 smithi119 ceph-mon[34519]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T07:19:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:05 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T07:19:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:05 smithi153 ceph-mon[34747]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T07:19:06.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 -- 172.21.15.27:0/2064915827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334109e00 msgr2=0x7fa334110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 --2- 172.21.15.27:0/2064915827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334109e00 0x7fa334110630 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009ca0 tx=0x7fa32c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 -- 172.21.15.27:0/2064915827 shutdown_connections 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 --2- 172.21.15.27:0/2064915827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334109e00 0x7fa334110630 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 --2- 172.21.15.27:0/2064915827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3341094b0 0x7fa3341098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 --2- 172.21.15.27:0/2064915827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa334103b70 0x7fa334103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 -- 172.21.15.27:0/2064915827 >> 172.21.15.27:0/2064915827 conn(0x7fa3340ff5a0 msgr2=0x7fa3341019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 -- 172.21.15.27:0/2064915827 shutdown_connections 2024-09-21T07:19:06.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.454+0000 7fa338a00640 1 -- 172.21.15.27:0/2064915827 wait complete. 2024-09-21T07:19:06.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.455+0000 7fa338a00640 1 Processor -- start 2024-09-21T07:19:06.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa338a00640 1 -- start start 2024-09-21T07:19:06.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa338a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334103b70 0x7fa334075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa338a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3341094b0 0x7fa334075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa338a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa334109e00 0x7fa334076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa338a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113ac0 con 0x7fa334103b70 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa338a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113940 con 0x7fa3341094b0 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa338a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113c40 con 0x7fa334109e00 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa331e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334103b70 0x7fa334075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa331e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334103b70 0x7fa334075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48602/0 (socket says 172.21.15.27:48602) 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa331e00640 1 -- 172.21.15.27:0/2646401139 learned_addr learned my addr 172.21.15.27:0/2646401139 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.456+0000 7fa329400640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3341094b0 0x7fa334075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa332800640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa334109e00 0x7fa334076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa331e00640 1 -- 172.21.15.27:0/2646401139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa334109e00 msgr2=0x7fa334076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa331e00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa334109e00 0x7fa334076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa331e00640 1 -- 172.21.15.27:0/2646401139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3341094b0 msgr2=0x7fa334075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa331e00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3341094b0 0x7fa334075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa331e00640 1 -- 172.21.15.27:0/2646401139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3341a8730 con 0x7fa334103b70 2024-09-21T07:19:06.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa329400640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3341094b0 0x7fa334075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa331e00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334103b70 0x7fa334075710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9f0 tx=0x7fa32400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.457+0000 7fa32b400640 1 -- 172.21.15.27:0/2646401139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32400cd80 con 0x7fa334103b70 2024-09-21T07:19:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.458+0000 7fa32b400640 1 -- 172.21.15.27:0/2646401139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3240108e0 con 0x7fa334103b70 2024-09-21T07:19:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.458+0000 7fa32b400640 1 -- 172.21.15.27:0/2646401139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324018980 con 0x7fa334103b70 2024-09-21T07:19:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.458+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3341a8a20 con 0x7fa334103b70 2024-09-21T07:19:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.458+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a8f10 con 0x7fa334103b70 2024-09-21T07:19:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.461+0000 7fa33f81a740 1 Processor -- start 2024-09-21T07:19:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.461+0000 7fa33f81a740 1 -- start start 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.461+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e81269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.461+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e8126ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.461+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8127430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa33f81a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a2e809ca90 con 0x55a2e7d7f850 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa33f81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a2e808bec0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa33f81a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a2e808fcd0 con 0x55a2e7d56990 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa32b400640 1 -- 172.21.15.27:0/2646401139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa324010430 con 0x7fa334103b70 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e81269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e81269b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48608/0 (socket says 172.21.15.27:48608) 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 -- 172.21.15.27:0/99119002 learned_addr learned my addr 172.21.15.27:0/99119002 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:06.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa329400640 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8127430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa331e00640 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e8126ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 -- 172.21.15.27:0/99119002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 msgr2=0x55a2e8126ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e8126ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 -- 172.21.15.27:0/99119002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 msgr2=0x55a2e8127430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8127430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa332800640 1 -- 172.21.15.27:0/99119002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e80319d0 con 0x55a2e7d7f850 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa331e00640 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e8126ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.462+0000 7fa329400640 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8127430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.463+0000 7fa332800640 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e81269b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02f410 tx=0x7fa32c02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b192a90c60e90673 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.463+0000 7fa323000640 1 -- 172.21.15.27:0/99119002 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c031cf0 con 0x55a2e7d7f850 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.463+0000 7fa323000640 1 -- 172.21.15.27:0/99119002 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32c036030 con 0x55a2e7d7f850 2024-09-21T07:19:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.463+0000 7fa323000640 1 -- 172.21.15.27:0/99119002 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0042a0 con 0x55a2e7d7f850 2024-09-21T07:19:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.464+0000 7fa32b400640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3180b6680 0x7fa3180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.464+0000 7fa32b400640 1 -- 172.21.15.27:0/2646401139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7fa3240e7eb0 con 0x7fa334103b70 2024-09-21T07:19:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.464+0000 7fa331e00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3180b6680 0x7fa3180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 -- 172.21.15.27:0/99119002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 msgr2=0x55a2e81269b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e81269b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02f410 tx=0x7fa32c02f440 comp rx=0 tx=0).stop 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 -- 172.21.15.27:0/99119002 shutdown_connections 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e8126ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8127430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 --2- 172.21.15.27:0/99119002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e81269b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 -- 172.21.15.27:0/99119002 >> 172.21.15.27:0/99119002 conn(0x55a2e7d73c40 msgr2=0x55a2e81248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 -- 172.21.15.27:0/99119002 shutdown_connections 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa33f81a740 1 -- 172.21.15.27:0/99119002 wait complete. 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.465+0000 7fa331e00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3180b6680 0x7fa3180b8b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a2e8125630 tx=0x7fa324000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:06.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 -- start start 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 Processor -- start 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e81324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8132a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e813e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a2e7eed7b0 con 0x55a2e7d7f850 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a2e8030650 con 0x55a2e7d5eb70 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.466+0000 7fa33f81a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a2e8134fb0 con 0x55a2e7d56990 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa33f81a740 10 client.0 _ll_register_callbacks cb 0x55a2e7d970a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e813e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa329400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8132a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e813e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48610/0 (socket says 172.21.15.27:48610) 2024-09-21T07:19:06.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa332800640 1 -- 172.21.15.27:0/3621204662 learned_addr learned my addr 172.21.15.27:0/3621204662 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa332800640 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e81324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa329400640 1 -- 172.21.15.27:0/3621204662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 msgr2=0x55a2e81324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa329400640 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e81324f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa329400640 1 -- 172.21.15.27:0/3621204662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 msgr2=0x55a2e813e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa329400640 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e813e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa329400640 1 -- 172.21.15.27:0/3621204662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e7df9ba0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.467+0000 7fa332800640 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e81324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa329400640 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8132a30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa33406c6e0 tx=0x7fa31c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e1d0a72f0d814e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa317400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa317400640 1 -- 172.21.15.27:0/3621204662 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0120d0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32c036030 con 0x55a2e7d5eb70 2024-09-21T07:19:06.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa317400640 1 -- 172.21.15.27:0/3621204662 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7fa31c004030 con 0x55a2e7d5eb70 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa317400640 1 -- 172.21.15.27:0/3621204662 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa31c025050 con 0x55a2e7d5eb70 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa33f81a740 10 client.27254 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.468+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2e8140fa0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa317400640 1 -- 172.21.15.27:0/3621204662 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32c036030 con 0x55a2e7d5eb70 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa317400640 1 -- 172.21.15.27:0/3621204662 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa31c01c460 con 0x55a2e7d5eb70 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa33f81a740 10 client.27254 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa33f81a740 10 client.27254 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa33f81a740 20 client.27254 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa33f81a740 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d3e1d0 0x55a2e81444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa33f81a740 4 client.27254 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2e809e9e0 con 0x55a2e7d3e1d0 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.469+0000 7fa332800640 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d3e1d0 0x55a2e81444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.470+0000 7fa332800640 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d3e1d0 0x55a2e81444f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d0b93ea522e079 server_cookie=bfde0208913a039a in_seq=0 out_seq=0 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.470+0000 7fa317400640 10 client.27254 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa317400640 1 -- 172.21.15.27:0/3621204662 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2e809e9e0 con 0x55a2e7d3e1d0 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa317400640 10 client.27254 handle_command_reply: tid=0 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 client.27254 shutdown 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 msgr2=0x55a2e8132a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8132a30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa33406c6e0 tx=0x7fa31c00f790 comp rx=0 tx=0).stop 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 shutdown_connections 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d3e1d0 0x55a2e81444f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d7f850 0x55a2e813e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d5eb70 0x55a2e8132a30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 --2- 172.21.15.27:0/3621204662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d56990 0x55a2e81324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa317400640 0 client.27254 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 >> 172.21.15.27:0/3621204662 conn(0x55a2e7d73c40 msgr2=0x55a2e8129b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 shutdown_connections 2024-09-21T07:19:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.471+0000 7fa33f81a740 1 -- 172.21.15.27:0/3621204662 wait complete. 2024-09-21T07:19:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.472+0000 7fa33f81a740 20 client.27254 trim_cache size 0 max 16384 2024-09-21T07:19:06.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 Processor -- start 2024-09-21T07:19:06.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 -- start start 2024-09-21T07:19:06.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d56990 0x55a2e812cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d5eb70 0x55a2e8123570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d7f850 0x55a2e81341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341b5a80 con 0x55a2e7d56990 2024-09-21T07:19:06.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341b3350 con 0x55a2e7d7f850 2024-09-21T07:19:06.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.490+0000 7fa33f81a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341b11c0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa332800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d56990 0x55a2e812cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d5eb70 0x55a2e8123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d5eb70 0x55a2e8123570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39004/0 (socket says 172.21.15.27:39004) 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 -- 172.21.15.27:0/1736483582 learned_addr learned my addr 172.21.15.27:0/1736483582 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa332800640 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d7f850 0x55a2e81341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 -- 172.21.15.27:0/1736483582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d7f850 msgr2=0x55a2e81341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d7f850 0x55a2e81341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 -- 172.21.15.27:0/1736483582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d56990 msgr2=0x55a2e812cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d56990 0x55a2e812cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.491+0000 7fa329400640 1 -- 172.21.15.27:0/1736483582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e8125210 con 0x55a2e7d5eb70 2024-09-21T07:19:06.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.492+0000 7fa329400640 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d5eb70 0x55a2e8123570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa31c012870 tx=0x7fa31c01c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2cbcc91989aeedc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.492+0000 7fa322600640 1 -- 172.21.15.27:0/1736483582 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0120d0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.492+0000 7fa322600640 1 -- 172.21.15.27:0/1736483582 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31c012270 con 0x55a2e7d5eb70 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.492+0000 7fa322600640 1 -- 172.21.15.27:0/1736483582 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c003900 con 0x55a2e7d5eb70 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 -- 172.21.15.27:0/1736483582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d5eb70 msgr2=0x55a2e8123570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d5eb70 0x55a2e8123570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa31c012870 tx=0x7fa31c01c5f0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 -- 172.21.15.27:0/1736483582 shutdown_connections 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d5eb70 0x55a2e8123570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d7f850 0x55a2e81341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 --2- 172.21.15.27:0/1736483582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d56990 0x55a2e812cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 -- 172.21.15.27:0/1736483582 >> 172.21.15.27:0/1736483582 conn(0x55a2e7d73c40 msgr2=0x55a2e81279e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:06.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 -- 172.21.15.27:0/1736483582 shutdown_connections 2024-09-21T07:19:06.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 -- 172.21.15.27:0/1736483582 wait complete. 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.493+0000 7fa33f81a740 1 -- start start 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 1 Processor -- start 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d56990 0x55a2e8136c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d5eb70 0x55a2e8137250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d7f850 0x55a2e81a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341b1810 con 0x55a2e7d5eb70 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341b2c80 con 0x55a2e7d56990 2024-09-21T07:19:06.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341b1ee0 con 0x55a2e7d7f850 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.494+0000 7fa33f81a740 10 client.0 _ll_register_callbacks cb 0x7fa31c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa332800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d56990 0x55a2e8136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa332800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d56990 0x55a2e8136c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:32858/0 (socket says 172.21.15.27:32858) 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa332800640 1 -- 172.21.15.27:0/277917825 learned_addr learned my addr 172.21.15.27:0/277917825 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa332800640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d7f850 0x55a2e81a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa329400640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d5eb70 0x55a2e8137250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa329400640 1 -- 172.21.15.27:0/277917825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d7f850 msgr2=0x55a2e81a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa329400640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d7f850 0x55a2e81a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa329400640 1 -- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d56990 msgr2=0x55a2e8136c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa329400640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d56990 0x55a2e8136c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa329400640 1 -- 172.21.15.27:0/277917825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e81a85b0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.495+0000 7fa332800640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d56990 0x55a2e8136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa329400640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d5eb70 0x55a2e8137250 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00a010 tx=0x7fa31c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79e01d14e9d3289 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa317400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa317400640 1 -- 172.21.15.27:0/277917825 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0120d0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa317400640 1 -- 172.21.15.27:0/277917825 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7fa31c018070 con 0x55a2e7d5eb70 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa31c012270 con 0x55a2e7d5eb70 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa317400640 1 -- 172.21.15.27:0/277917825 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa31c0052e0 con 0x55a2e7d5eb70 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa33f81a740 10 client.17229 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.496+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2e81af980 con 0x55a2e7d5eb70 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa317400640 1 -- 172.21.15.27:0/277917825 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31c012270 con 0x55a2e7d5eb70 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa317400640 1 -- 172.21.15.27:0/277917825 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa31c02a440 con 0x55a2e7d5eb70 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa33f81a740 10 client.17229 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa33f81a740 10 client.17229 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:19:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa33f81a740 20 client.17229 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa33f81a740 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d13580 0x55a2e81b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa33f81a740 4 client.17229 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T07:19:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x55a2e809a490 con 0x55a2e7d13580 2024-09-21T07:19:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.497+0000 7fa332800640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d13580 0x55a2e81b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.498+0000 7fa332800640 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d13580 0x55a2e81b2080 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eda85cbfda45eb server_cookie=34d54d0cec16a640 in_seq=0 out_seq=0 2024-09-21T07:19:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.498+0000 7fa317400640 10 client.17229 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:06.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.507+0000 7fa317400640 1 -- 172.21.15.27:0/277917825 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a2e809a490 con 0x55a2e7d13580 2024-09-21T07:19:06.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.507+0000 7fa317400640 10 client.17229 handle_command_reply: tid=0 2024-09-21T07:19:06.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.507+0000 7fa33f81a740 1 client.17229 shutdown 2024-09-21T07:19:06.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.507+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d5eb70 msgr2=0x55a2e8137250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.507+0000 7fa33f81a740 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d5eb70 0x55a2e8137250 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00a010 tx=0x7fa31c01c930 comp rx=0 tx=0).stop 2024-09-21T07:19:06.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.507+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 shutdown_connections 2024-09-21T07:19:06.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.507+0000 7fa33f81a740 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a2e7d13580 0x55a2e81b2080 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa33f81a740 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2e7d7f850 0x55a2e81a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa33f81a740 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a2e7d5eb70 0x55a2e8137250 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa33f81a740 1 --2- 172.21.15.27:0/277917825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2e7d56990 0x55a2e8136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 >> 172.21.15.27:0/277917825 conn(0x55a2e7d73c40 msgr2=0x55a2e8133f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:06.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa317400640 0 client.17229 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:06.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 shutdown_connections 2024-09-21T07:19:06.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa33f81a740 1 -- 172.21.15.27:0/277917825 wait complete. 2024-09-21T07:19:06.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.508+0000 7fa33f81a740 20 client.17229 trim_cache size 0 max 16384 2024-09-21T07:19:06.511 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 26, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "db39a904": { 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 26, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T07:19:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:06.513 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3180b6680 msgr2=0x7fa3180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3180b6680 0x7fa3180b8b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a2e8125630 tx=0x7fa324000bc0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334103b70 msgr2=0x7fa334075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334103b70 0x7fa334075710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9f0 tx=0x7fa32400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 shutdown_connections 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa334109e00 0x7fa334076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa3180b6680 0x7fa3180b8b90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3341094b0 0x7fa334075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 --2- 172.21.15.27:0/2646401139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa334103b70 0x7fa334075710 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:06.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 >> 172.21.15.27:0/2646401139 conn(0x7fa3340ff5a0 msgr2=0x7fa334108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:06.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 shutdown_connections 2024-09-21T07:19:06.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:06.509+0000 7fa338a00640 1 -- 172.21.15.27:0/2646401139 wait complete. 2024-09-21T07:19:06.687 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:19:06.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:19:06.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:06 smithi027 ceph-mon[30779]: pgmap v561: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 60 KiB/s wr, 29 op/s 2024-09-21T07:19:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:06 smithi119 ceph-mon[34519]: pgmap v561: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 60 KiB/s wr, 29 op/s 2024-09-21T07:19:07.164 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'db39a904', quiesce took 4.6 seconds. Will release after: 14.65 2024-09-21T07:19:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:06 smithi153 ceph-mon[34747]: pgmap v561: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 60 KiB/s wr, 29 op/s 2024-09-21T07:19:07.257 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:07.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 -- 172.21.15.27:0/2707178394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc10c010 msgr2=0x7fc7bc112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/2707178394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc10c010 0x7fc7bc112840 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009ca0 tx=0x7fc7b402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 -- 172.21.15.27:0/2707178394 shutdown_connections 2024-09-21T07:19:07.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/2707178394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7bc10b6c0 0x7fc7bc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/2707178394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc10c010 0x7fc7bc112840 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/2707178394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc105d80 0x7fc7bc106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 -- 172.21.15.27:0/2707178394 >> 172.21.15.27:0/2707178394 conn(0x7fc7bc101810 msgr2=0x7fc7bc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:07.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 -- 172.21.15.27:0/2707178394 shutdown_connections 2024-09-21T07:19:07.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.879+0000 7fc7c0c00640 1 -- 172.21.15.27:0/2707178394 wait complete. 2024-09-21T07:19:07.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.880+0000 7fc7c0c00640 1 Processor -- start 2024-09-21T07:19:07.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.880+0000 7fc7c0c00640 1 -- start start 2024-09-21T07:19:07.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7c0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc105d80 0x7fc7bc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7c0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc10b6c0 0x7fc7bc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7c0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7bc10c010 0x7fc7bc1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7c0c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc115cd0 con 0x7fc7bc105d80 2024-09-21T07:19:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7c0c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc115b50 con 0x7fc7bc10b6c0 2024-09-21T07:19:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7c0c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc115e50 con 0x7fc7bc10c010 2024-09-21T07:19:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc105d80 0x7fc7bc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc105d80 0x7fc7bc19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48636/0 (socket says 172.21.15.27:48636) 2024-09-21T07:19:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3420855623 learned_addr learned my addr 172.21.15.27:0/3420855623 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7ba800640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7bc10c010 0x7fc7bc1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.881+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc10b6c0 0x7fc7bc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7ba800640 1 -- 172.21.15.27:0/3420855623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc10b6c0 msgr2=0x7fc7bc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7ba800640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc10b6c0 0x7fc7bc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7ba800640 1 -- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc105d80 msgr2=0x7fc7bc19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7ba800640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc105d80 0x7fc7bc19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7ba800640 1 -- 172.21.15.27:0/3420855623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc1a2690 con 0x7fc7bc10c010 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc10b6c0 0x7fc7bc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc105d80 0x7fc7bc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7ba800640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7bc10c010 0x7fc7bc1a1f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7b402ade0 tx=0x7fc7b4031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:07.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7b2a00640 1 -- 172.21.15.27:0/3420855623 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4003b70 con 0x7fc7bc10c010 2024-09-21T07:19:07.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc1a2920 con 0x7fc7bc10c010 2024-09-21T07:19:07.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7b2a00640 1 -- 172.21.15.27:0/3420855623 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b4031910 con 0x7fc7bc10c010 2024-09-21T07:19:07.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.882+0000 7fc7b2a00640 1 -- 172.21.15.27:0/3420855623 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b402f950 con 0x7fc7bc10c010 2024-09-21T07:19:07.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.883+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc1a90e0 con 0x7fc7bc10c010 2024-09-21T07:19:07.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.886+0000 7fc7c78ab740 1 Processor -- start 2024-09-21T07:19:07.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7b2a00640 1 -- 172.21.15.27:0/3420855623 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc7b402faf0 con 0x7fc7bc10c010 2024-09-21T07:19:07.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7c78ab740 1 -- start start 2024-09-21T07:19:07.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d1490 0x55b2df9a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9a2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b2df917e90 con 0x55b2df5fa350 2024-09-21T07:19:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b2df9072c0 con 0x55b2df5d1490 2024-09-21T07:19:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.888+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2df90b0d0 con 0x55b2df5d9670 2024-09-21T07:19:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d1490 0x55b2df9a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d1490 0x55b2df9a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:32878/0 (socket says 172.21.15.27:32878) 2024-09-21T07:19:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 -- 172.21.15.27:0/216825055 learned_addr learned my addr 172.21.15.27:0/216825055 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 -- 172.21.15.27:0/216825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 msgr2=0x55b2df9a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 -- 172.21.15.27:0/216825055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 msgr2=0x55b2df9a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 -- 172.21.15.27:0/216825055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df665930 con 0x55b2df5d1490 2024-09-21T07:19:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b2a00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7a00b6680 0x7fc7a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d1490 0x55b2df9a2250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac011650 tx=0x7fc7ac009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1560e584c3a322d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:07.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.889+0000 7fc7b2a00640 1 -- 172.21.15.27:0/3420855623 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7fc7b4108b40 con 0x7fc7bc10c010 2024-09-21T07:19:07.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.890+0000 7fc7ab000640 1 -- 172.21.15.27:0/216825055 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac01e070 con 0x55b2df5d1490 2024-09-21T07:19:07.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.890+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7a00b6680 0x7fc7a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.890+0000 7fc7ab000640 1 -- 172.21.15.27:0/216825055 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ac00fb70 con 0x55b2df5d1490 2024-09-21T07:19:07.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.890+0000 7fc7ab000640 1 -- 172.21.15.27:0/216825055 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac019c00 con 0x55b2df5d1490 2024-09-21T07:19:07.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 -- 172.21.15.27:0/216825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d1490 msgr2=0x55b2df9a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7a00b6680 0x7fc7a00b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005f60 tx=0x7fc7a4015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d1490 0x55b2df9a2250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac011650 tx=0x7fc7ac009f90 comp rx=0 tx=0).stop 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 -- 172.21.15.27:0/216825055 shutdown_connections 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d1490 0x55b2df9a2250 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/216825055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 -- 172.21.15.27:0/216825055 >> 172.21.15.27:0/216825055 conn(0x55b2df5ee740 msgr2=0x55b2df99fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 -- 172.21.15.27:0/216825055 shutdown_connections 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.891+0000 7fc7c78ab740 1 -- 172.21.15.27:0/216825055 wait complete. 2024-09-21T07:19:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 -- start start 2024-09-21T07:19:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 Processor -- start 2024-09-21T07:19:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b2df768890 con 0x55b2df5d1490 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b2df8a2160 con 0x55b2df5fa350 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2df9b04f0 con 0x55b2df5d9670 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.892+0000 7fc7c78ab740 10 client.0 _ll_register_callbacks cb 0x55b2df8bea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48662/0 (socket says 172.21.15.27:48662) 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7ba800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3838896493 learned_addr learned my addr 172.21.15.27:0/3838896493 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3838896493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 msgr2=0x55b2df9ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3838896493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 msgr2=0x55b2df9b97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9b97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.893+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3838896493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df9b9d70 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9ad8e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac01c040 tx=0x7fc7ac009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e754f8b759e13c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc79f400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc79f400640 1 -- 172.21.15.27:0/3838896493 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac01e040 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7ac00fb70 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc79f400640 1 -- 172.21.15.27:0/3838896493 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7fc7ac00f6c0 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc79f400640 1 -- 172.21.15.27:0/3838896493 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7ac030050 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc7c78ab740 10 client.17241 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.894+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2df9bc680 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc79f400640 1 -- 172.21.15.27:0/3838896493 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ac00fb70 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc79f400640 1 -- 172.21.15.27:0/3838896493 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc7ac021070 con 0x55b2df5d1490 2024-09-21T07:19:07.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc7c78ab740 10 client.17241 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc7c78ab740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc7c78ab740 20 client.17241 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df5b8cd0 0x55b2df9bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc7c78ab740 4 client.17241 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2df919de0 con 0x55b2df5b8cd0 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.895+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df5b8cd0 0x55b2df9bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.896+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df5b8cd0 0x55b2df9bfa30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de2ebeda255183c server_cookie=958182f07d734feb in_seq=0 out_seq=0 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.896+0000 7fc79f400640 10 client.17241 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc79f400640 1 -- 172.21.15.27:0/3838896493 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b2df919de0 con 0x55b2df5b8cd0 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc79f400640 10 client.17241 handle_command_reply: tid=0 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 client.17241 shutdown 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 msgr2=0x55b2df9ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9ad8e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac01c040 tx=0x7fc7ac009ef0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 shutdown_connections 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df5b8cd0 0x55b2df9bfa30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3838896493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9ad8e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 >> 172.21.15.27:0/3838896493 conn(0x55b2df5ee740 msgr2=0x55b2df99e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:07.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.897+0000 7fc79f400640 0 client.17241 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:07.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.898+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 shutdown_connections 2024-09-21T07:19:07.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.898+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3838896493 wait complete. 2024-09-21T07:19:07.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.898+0000 7fc7c78ab740 20 client.17241 trim_cache size 0 max 16384 2024-09-21T07:19:07.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.916+0000 7fc7c78ab740 1 Processor -- start 2024-09-21T07:19:07.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.916+0000 7fc7c78ab740 1 -- start start 2024-09-21T07:19:07.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9acf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1b5b60 con 0x55b2df5d1490 2024-09-21T07:19:07.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1b3430 con 0x55b2df5fa350 2024-09-21T07:19:07.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1b12a0 con 0x55b2df5d9670 2024-09-21T07:19:07.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9a7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48674/0 (socket says 172.21.15.27:48674) 2024-09-21T07:19:07.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3005154799 learned_addr learned my addr 172.21.15.27:0/3005154799 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:07.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.917+0000 7fc7ba800640 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3005154799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 msgr2=0x55b2df9acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3005154799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 msgr2=0x55b2df9aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7b9e00640 1 -- 172.21.15.27:0/3005154799 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df9a45a0 con 0x55b2df5d1490 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9a7530 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009c90 tx=0x7fc7ac0114d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76daf10f64a65f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7aa600640 1 -- 172.21.15.27:0/3005154799 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac01e040 con 0x55b2df5d1490 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7aa600640 1 -- 172.21.15.27:0/3005154799 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ac027070 con 0x55b2df5d1490 2024-09-21T07:19:07.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.918+0000 7fc7aa600640 1 -- 172.21.15.27:0/3005154799 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00f920 con 0x55b2df5d1490 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3005154799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 msgr2=0x55b2df9a7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9a7530 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009c90 tx=0x7fc7ac0114d0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3005154799 shutdown_connections 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d9670 0x55b2df9acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5fa350 0x55b2df9aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/3005154799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5d1490 0x55b2df9a7530 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3005154799 >> 172.21.15.27:0/3005154799 conn(0x55b2df5ee740 msgr2=0x55b2df9a6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3005154799 shutdown_connections 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.919+0000 7fc7c78ab740 1 -- 172.21.15.27:0/3005154799 wait complete. 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.920+0000 7fc7c78ab740 1 -- start start 2024-09-21T07:19:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.920+0000 7fc7c78ab740 1 Processor -- start 2024-09-21T07:19:07.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.920+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d1490 0x55b2df9b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.920+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d9670 0x55b2df9b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7c78ab740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1b18f0 con 0x55b2df5fa350 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1b2d60 con 0x55b2df5d9670 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7c78ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1b1fc0 con 0x55b2df5d1490 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7c78ab740 10 client.0 _ll_register_callbacks cb 0x7fc7ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d1490 0x55b2df9b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d1490 0x55b2df9b60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39056/0 (socket says 172.21.15.27:39056) 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 -- 172.21.15.27:0/1419166637 learned_addr learned my addr 172.21.15.27:0/1419166637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d9670 0x55b2df9b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 -- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d9670 msgr2=0x55b2df9b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d9670 0x55b2df9b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 -- 172.21.15.27:0/1419166637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 msgr2=0x55b2df9b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7b9e00640 1 -- 172.21.15.27:0/1419166637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2dfa29290 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.921+0000 7fc7ba800640 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc7b9e00640 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d1490 0x55b2df9b60d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac006010 tx=0x7fc7ac019410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda6835f675f9e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc79f400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc79f400640 1 -- 172.21.15.27:0/1419166637 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac01e040 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7ac027070 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc79f400640 1 -- 172.21.15.27:0/1419166637 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7fc7ac00f8c0 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc79f400640 1 -- 172.21.15.27:0/1419166637 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7ac00fac0 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc7c78ab740 10 client.27256 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.922+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2dfa2a3c0 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc79f400640 1 -- 172.21.15.27:0/1419166637 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ac027070 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc79f400640 1 -- 172.21.15.27:0/1419166637 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc7ac021050 con 0x55b2df5d1490 2024-09-21T07:19:07.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc7c78ab740 10 client.27256 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:07.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc7c78ab740 10 client.27256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:07.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc7c78ab740 20 client.27256 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:07.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df58e080 0x55b2dfa2ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:07.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc7c78ab740 4 client.27256 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:19:07.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.923+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b2df915890 con 0x55b2df58e080 2024-09-21T07:19:07.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.924+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df58e080 0x55b2dfa2ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:07.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.924+0000 7fc7b3e00640 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df58e080 0x55b2dfa2ca90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f3198a1c5ab090 server_cookie=8c279c144853b516 in_seq=0 out_seq=0 2024-09-21T07:19:07.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.924+0000 7fc79f400640 10 client.27256 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:07.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc79f400640 1 -- 172.21.15.27:0/1419166637 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b2df915890 con 0x55b2df58e080 2024-09-21T07:19:07.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc79f400640 10 client.27256 handle_command_reply: tid=0 2024-09-21T07:19:07.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 client.27256 shutdown 2024-09-21T07:19:07.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d1490 msgr2=0x55b2df9b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d1490 0x55b2df9b60d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac006010 tx=0x7fc7ac019410 comp rx=0 tx=0).stop 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 shutdown_connections 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2df58e080 0x55b2dfa2ca90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2df5fa350 0x55b2df9b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2df5d9670 0x55b2df9b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 --2- 172.21.15.27:0/1419166637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2df5d1490 0x55b2df9b60d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 >> 172.21.15.27:0/1419166637 conn(0x55b2df5ee740 msgr2=0x55b2df9a2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc79f400640 0 client.27256 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 shutdown_connections 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.926+0000 7fc7c78ab740 1 -- 172.21.15.27:0/1419166637 wait complete. 2024-09-21T07:19:07.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.927+0000 7fc7c78ab740 20 client.27256 trim_cache size 0 max 16384 2024-09-21T07:19:07.931 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:19:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "88a53041-1567-48da-a7e8-c17f689afb84", 2024-09-21T07:19:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:19:07.931 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.927+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7a00b6680 msgr2=0x7fc7a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.927+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7a00b6680 0x7fc7a00b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005f60 tx=0x7fc7a4015040 comp rx=0 tx=0).stop 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.927+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7bc10c010 msgr2=0x7fc7bc1a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.927+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7bc10c010 0x7fc7bc1a1f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7b402ade0 tx=0x7fc7b4031660 comp rx=0 tx=0).stop 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 shutdown_connections 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7bc10c010 0x7fc7bc1a1f60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc7a00b6680 0x7fc7a00b8b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc10b6c0 0x7fc7bc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 --2- 172.21.15.27:0/3420855623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc105d80 0x7fc7bc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:07.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 >> 172.21.15.27:0/3420855623 conn(0x7fc7bc101810 msgr2=0x7fc7bc10ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:07.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 shutdown_connections 2024-09-21T07:19:07.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:07.928+0000 7fc7c0c00640 1 -- 172.21.15.27:0/3420855623 wait complete. 2024-09-21T07:19:08.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:19:08.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:08 smithi153 ceph-mon[34747]: pgmap v562: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 31 op/s 2024-09-21T07:19:08.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:08 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:19:08.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:08.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:08 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:19:08.980 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:09.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:08 smithi119 ceph-mon[34519]: pgmap v562: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 31 op/s 2024-09-21T07:19:09.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:08 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:19:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:08 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:19:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:08 smithi027 ceph-mon[30779]: pgmap v562: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 31 op/s 2024-09-21T07:19:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:08 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:19:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:08 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:19:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.640+0000 7f7a3be00640 1 -- 172.21.15.27:0/1918708690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c075dd0 msgr2=0x7f7a3c0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.640+0000 7f7a3be00640 1 --2- 172.21.15.27:0/1918708690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c0761e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009900 tx=0x7f7a2402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.641+0000 7f7a3be00640 1 -- 172.21.15.27:0/1918708690 shutdown_connections 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.641+0000 7f7a3be00640 1 --2- 172.21.15.27:0/1918708690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c1141d0 0x7f7a3c1166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.641+0000 7f7a3be00640 1 --2- 172.21.15.27:0/1918708690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c0761e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.641+0000 7f7a3be00640 1 --2- 172.21.15.27:0/1918708690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c077390 0x7f7a3c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.641+0000 7f7a3be00640 1 -- 172.21.15.27:0/1918708690 >> 172.21.15.27:0/1918708690 conn(0x7f7a3c0ff330 msgr2=0x7f7a3c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.641+0000 7f7a3be00640 1 -- 172.21.15.27:0/1918708690 shutdown_connections 2024-09-21T07:19:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.641+0000 7f7a3be00640 1 -- 172.21.15.27:0/1918708690 wait complete. 2024-09-21T07:19:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.642+0000 7f7a3be00640 1 Processor -- start 2024-09-21T07:19:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.642+0000 7f7a3be00640 1 -- start start 2024-09-21T07:19:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.642+0000 7f7a3be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c077390 0x7f7a3c1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c1141d0 0x7f7a3c1a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c119e10 con 0x7f7a3c077390 2024-09-21T07:19:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c119c90 con 0x7f7a3c1141d0 2024-09-21T07:19:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c119f90 con 0x7f7a3c075dd0 2024-09-21T07:19:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c077390 0x7f7a3c1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c1a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:48584/0 (socket says 172.21.15.27:48584) 2024-09-21T07:19:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3aa00640 1 -- 172.21.15.27:0/224385676 learned_addr learned my addr 172.21.15.27:0/224385676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3b400640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c1141d0 0x7f7a3c1a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3aa00640 1 -- 172.21.15.27:0/224385676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c1141d0 msgr2=0x7f7a3c1a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c1141d0 0x7f7a3c1a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3aa00640 1 -- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c077390 msgr2=0x7f7a3c1a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.643+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c077390 0x7f7a3c1a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a3aa00640 1 -- 172.21.15.27:0/224385676 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c1a67c0 con 0x7f7a3c075dd0 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a3a000640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c077390 0x7f7a3c1a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c1a0890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00d950 tx=0x7f7a2c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a33400640 1 -- 172.21.15.27:0/224385676 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c017030 con 0x7f7a3c075dd0 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c1a6ab0 con 0x7f7a3c075dd0 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c1ad240 con 0x7f7a3c075dd0 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a33400640 1 -- 172.21.15.27:0/224385676 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c004020 con 0x7f7a3c075dd0 2024-09-21T07:19:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.644+0000 7f7a33400640 1 -- 172.21.15.27:0/224385676 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c005650 con 0x7f7a3c075dd0 2024-09-21T07:19:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.647+0000 7f7a487df740 1 Processor -- start 2024-09-21T07:19:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a487df740 1 -- start start 2024-09-21T07:19:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a487df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a52041b0 0x55c8a55a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a487df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51db2f0 0x55c8a55a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a487df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a51e34d0 0x55c8a55a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a487df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c8a551e6b0 con 0x55c8a52041b0 2024-09-21T07:19:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a487df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c8a550dae0 con 0x55c8a51e34d0 2024-09-21T07:19:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a487df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c8a55118f0 con 0x55c8a51db2f0 2024-09-21T07:19:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a52041b0 0x55c8a55a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a52041b0 0x55c8a55a8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54924/0 (socket says 172.21.15.27:54924) 2024-09-21T07:19:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a3b400640 1 -- 172.21.15.27:0/806301361 learned_addr learned my addr 172.21.15.27:0/806301361 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.648+0000 7f7a3a000640 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a51e34d0 0x55c8a55a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.649+0000 7f7a3b400640 1 -- 172.21.15.27:0/806301361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51db2f0 msgr2=0x55c8a55a9350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.649+0000 7f7a3b400640 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51db2f0 0x55c8a55a9350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.649+0000 7f7a3b400640 1 -- 172.21.15.27:0/806301361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a51e34d0 msgr2=0x55c8a55a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.649+0000 7f7a3b400640 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a51e34d0 0x55c8a55a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.649+0000 7f7a3b400640 1 -- 172.21.15.27:0/806301361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a54da780 con 0x55c8a52041b0 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.649+0000 7f7a3b400640 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a52041b0 0x55c8a55a8e10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009f50 tx=0x7f7a340118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16b9ab9394a173e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.649+0000 7f7a2b400640 1 -- 172.21.15.27:0/806301361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34004b20 con 0x55c8a52041b0 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.650+0000 7f7a2b400640 1 -- 172.21.15.27:0/806301361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a34004cc0 con 0x55c8a52041b0 2024-09-21T07:19:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.650+0000 7f7a2b400640 1 -- 172.21.15.27:0/806301361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a340053a0 con 0x55c8a52041b0 2024-09-21T07:19:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.650+0000 7f7a487df740 1 -- 172.21.15.27:0/806301361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a52041b0 msgr2=0x55c8a55a8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.650+0000 7f7a487df740 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a52041b0 0x55c8a55a8e10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009f50 tx=0x7f7a340118d0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.650+0000 7f7a33400640 1 -- 172.21.15.27:0/224385676 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7a2c009a30 con 0x7f7a3c075dd0 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 -- 172.21.15.27:0/806301361 shutdown_connections 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51db2f0 0x55c8a55a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a51e34d0 0x55c8a55a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 --2- 172.21.15.27:0/806301361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a52041b0 0x55c8a55a8e10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 -- 172.21.15.27:0/806301361 >> 172.21.15.27:0/806301361 conn(0x55c8a51f8300 msgr2=0x55c8a55a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 -- 172.21.15.27:0/806301361 shutdown_connections 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 -- 172.21.15.27:0/806301361 wait complete. 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.651+0000 7f7a487df740 1 -- start start 2024-09-21T07:19:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.652+0000 7f7a487df740 1 Processor -- start 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a487df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a487df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a487df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a487df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c8a5445ab0 con 0x55c8a51db2f0 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a487df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c8a54b4570 con 0x55c8a52041b0 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a487df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c8a55b7590 con 0x55c8a51e34d0 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a487df740 10 client.0 _ll_register_callbacks cb 0x55c8a54c3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a3b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54932/0 (socket says 172.21.15.27:54932) 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a3b400640 1 -- 172.21.15.27:0/281463960 learned_addr learned my addr 172.21.15.27:0/281463960 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a33400640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a200b6590 0x7f7a200b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a3a000640 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.653+0000 7f7a33400640 1 -- 172.21.15.27:0/224385676 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7f7a2c0e70c0 con 0x7f7a3c075dd0 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a3b400640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a200b6590 0x7f7a200b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a3a000640 1 -- 172.21.15.27:0/281463960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 msgr2=0x55c8a55b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a3a000640 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a3a000640 1 -- 172.21.15.27:0/281463960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 msgr2=0x55c8a55b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a3a000640 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a3a000640 1 -- 172.21.15.27:0/281463960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a526f790 con 0x55c8a52041b0 2024-09-21T07:19:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a3a000640 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b4e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009230 tx=0x7f7a24031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd3a5ca5906f76b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.654+0000 7f7a1fe00640 1 -- 172.21.15.27:0/281463960 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24035040 con 0x55c8a52041b0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a34004cc0 con 0x55c8a52041b0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a1fe00640 1 -- 172.21.15.27:0/281463960 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7f7a24004240 con 0x55c8a52041b0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a1fe00640 1 -- 172.21.15.27:0/281463960 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a2404f050 con 0x55c8a52041b0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a487df740 10 client.27290 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a3b400640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a200b6590 0x7f7a200b8aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005080 tx=0x7f7a34009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a52e8620 con 0x55c8a52041b0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a1fe00640 1 -- 172.21.15.27:0/281463960 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a34004cc0 con 0x55c8a52041b0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a1fe00640 1 -- 172.21.15.27:0/281463960 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7a240049e0 con 0x55c8a52041b0 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a487df740 10 client.27290 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.655+0000 7f7a487df740 10 client.27290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.656+0000 7f7a487df740 20 client.27290 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.656+0000 7f7a487df740 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51c2d80 0x55c8a55c6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.656+0000 7f7a487df740 4 client.27290 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.656+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8a5520600 con 0x55c8a51c2d80 2024-09-21T07:19:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.656+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51c2d80 0x55c8a55c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.656+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51c2d80 0x55c8a55c6520 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f74a67f28ec75cf server_cookie=f9266cda2ed3185d in_seq=0 out_seq=0 2024-09-21T07:19:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.656+0000 7f7a1fe00640 10 client.27290 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.657+0000 7f7a1fe00640 1 -- 172.21.15.27:0/281463960 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c8a5520600 con 0x55c8a51c2d80 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.657+0000 7f7a1fe00640 10 client.27290 handle_command_reply: tid=0 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.657+0000 7f7a487df740 1 client.27290 shutdown 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.657+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 msgr2=0x55c8a55b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.657+0000 7f7a487df740 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b4e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009230 tx=0x7f7a24031810 comp rx=0 tx=0).stop 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 shutdown_connections 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51c2d80 0x55c8a55c6520 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b4e30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a1fe00640 0 client.27290 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 --2- 172.21.15.27:0/281463960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 >> 172.21.15.27:0/281463960 conn(0x55c8a51f8300 msgr2=0x55c8a55a9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 shutdown_connections 2024-09-21T07:19:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 1 -- 172.21.15.27:0/281463960 wait complete. 2024-09-21T07:19:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.658+0000 7f7a487df740 20 client.27290 trim_cache size 0 max 16384 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.673+0000 7f7a487df740 1 Processor -- start 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.673+0000 7f7a487df740 1 -- start start 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a487df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a487df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a487df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a487df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18012940 con 0x55c8a51db2f0 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a487df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18010210 con 0x55c8a52041b0 2024-09-21T07:19:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a487df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1800e080 con 0x55c8a51e34d0 2024-09-21T07:19:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a3a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a3aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a3a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:48600/0 (socket says 172.21.15.27:48600) 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a3a000640 1 -- 172.21.15.27:0/3491887287 learned_addr learned my addr 172.21.15.27:0/3491887287 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.674+0000 7f7a3b400640 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a3a000640 1 -- 172.21.15.27:0/3491887287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 msgr2=0x55c8a55b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a3a000640 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a3a000640 1 -- 172.21.15.27:0/3491887287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 msgr2=0x55c8a55a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a3a000640 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a3a000640 1 -- 172.21.15.27:0/3491887287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a55abc20 con 0x55c8a51e34d0 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a3a000640 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b5fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a24006010 tx=0x7f7a2402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8de877fcfac75d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a2aa00640 1 -- 172.21.15.27:0/3491887287 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24035030 con 0x55c8a51e34d0 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.675+0000 7f7a2aa00640 1 -- 172.21.15.27:0/3491887287 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a240035f0 con 0x55c8a51e34d0 2024-09-21T07:19:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a2aa00640 1 -- 172.21.15.27:0/3491887287 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24000e60 con 0x55c8a51e34d0 2024-09-21T07:19:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a487df740 1 -- 172.21.15.27:0/3491887287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 msgr2=0x55c8a55b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a487df740 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b5fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a24006010 tx=0x7f7a2402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a487df740 1 -- 172.21.15.27:0/3491887287 shutdown_connections 2024-09-21T07:19:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a487df740 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55b5fe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a487df740 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a487df740 1 --2- 172.21.15.27:0/3491887287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.676+0000 7f7a487df740 1 -- 172.21.15.27:0/3491887287 >> 172.21.15.27:0/3491887287 conn(0x55c8a51f8300 msgr2=0x55c8a55b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.677+0000 7f7a487df740 1 -- 172.21.15.27:0/3491887287 shutdown_connections 2024-09-21T07:19:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.677+0000 7f7a487df740 1 -- 172.21.15.27:0/3491887287 wait complete. 2024-09-21T07:19:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.677+0000 7f7a487df740 1 -- start start 2024-09-21T07:19:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.677+0000 7f7a487df740 1 Processor -- start 2024-09-21T07:19:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a487df740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a487df740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55ba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a487df740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a487df740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1800e6d0 con 0x55c8a51db2f0 2024-09-21T07:19:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a487df740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1800fb40 con 0x55c8a52041b0 2024-09-21T07:19:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a487df740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1800eda0 con 0x55c8a51e34d0 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a487df740 10 client.0 _ll_register_callbacks cb 0x7f7a24009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a3a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55ba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a3aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a3a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55ba620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:48616/0 (socket says 172.21.15.27:48616) 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a3a000640 1 -- 172.21.15.27:0/1015094715 learned_addr learned my addr 172.21.15.27:0/1015094715 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.678+0000 7f7a3b400640 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a3a000640 1 -- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 msgr2=0x55c8a55bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a3a000640 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a3a000640 1 -- 172.21.15.27:0/1015094715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 msgr2=0x55c8a55bd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a3a000640 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55bd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a3a000640 1 -- 172.21.15.27:0/1015094715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a55bb400 con 0x55c8a51e34d0 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a3a000640 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55ba620 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009850 tx=0x7f7a2402ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d2db2c171c9cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a1fe00640 1 -- 172.21.15.27:0/1015094715 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24035030 con 0x55c8a51e34d0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a240035f0 con 0x55c8a51e34d0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.679+0000 7f7a1fe00640 1 -- 172.21.15.27:0/1015094715 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7f7a24004f30 con 0x55c8a51e34d0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a1fe00640 1 -- 172.21.15.27:0/1015094715 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a2406c050 con 0x55c8a51e34d0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 10 client.27271 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a5630fa0 con 0x55c8a51e34d0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a1fe00640 1 -- 172.21.15.27:0/1015094715 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a240035f0 con 0x55c8a51e34d0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a1fe00640 1 -- 172.21.15.27:0/1015094715 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7a24046860 con 0x55c8a51e34d0 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 10 client.27271 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 10 client.27271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 20 client.27271 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51981c0 0x55c8a56336b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 4 client.27271 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:19:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.680+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8a551c0b0 con 0x55c8a51981c0 2024-09-21T07:19:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.681+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51981c0 0x55c8a56336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.681+0000 7f7a3aa00640 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51981c0 0x55c8a56336b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce8597660f1a59d0 server_cookie=931b20e667afd891 in_seq=0 out_seq=0 2024-09-21T07:19:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.681+0000 7f7a1fe00640 10 client.27271 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.691+0000 7f7a1fe00640 1 -- 172.21.15.27:0/1015094715 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8a551c0b0 con 0x55c8a51981c0 2024-09-21T07:19:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.691+0000 7f7a1fe00640 10 client.27271 handle_command_reply: tid=0 2024-09-21T07:19:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.691+0000 7f7a487df740 1 client.27271 shutdown 2024-09-21T07:19:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 msgr2=0x55c8a55ba620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55ba620 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009850 tx=0x7f7a2402ab70 comp rx=0 tx=0).stop 2024-09-21T07:19:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 shutdown_connections 2024-09-21T07:19:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8a51981c0 0x55c8a56336b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8a51e34d0 0x55c8a55ba620 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8a52041b0 0x55c8a55bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 --2- 172.21.15.27:0/1015094715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8a51db2f0 0x55c8a55bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a1fe00640 0 client.27271 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 >> 172.21.15.27:0/1015094715 conn(0x55c8a51f8300 msgr2=0x55c8a55a5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 shutdown_connections 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 1 -- 172.21.15.27:0/1015094715 wait complete. 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.692+0000 7f7a487df740 20 client.27271 trim_cache size 0 max 16384 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:19:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:19:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:19:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a200b6590 msgr2=0x7f7a200b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a200b6590 0x7f7a200b8aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005080 tx=0x7f7a34009500 comp rx=0 tx=0).stop 2024-09-21T07:19:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c075dd0 msgr2=0x7f7a3c1a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c1a0890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00d950 tx=0x7f7a2c00de10 comp rx=0 tx=0).stop 2024-09-21T07:19:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 shutdown_connections 2024-09-21T07:19:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a200b6590 0x7f7a200b8aa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c1141d0 0x7f7a3c1a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c077390 0x7f7a3c1a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 --2- 172.21.15.27:0/224385676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a3c075dd0 0x7f7a3c1a0890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 >> 172.21.15.27:0/224385676 conn(0x7f7a3c0ff330 msgr2=0x7f7a3c1100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 shutdown_connections 2024-09-21T07:19:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:09.693+0000 7f7a3be00640 1 -- 172.21.15.27:0/224385676 wait complete. 2024-09-21T07:19:09.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 -- 172.21.15.27:0/4006176105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e6410c020 msgr2=0x7f0e64112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 --2- 172.21.15.27:0/4006176105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e6410c020 0x7f0e64112850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009ca0 tx=0x7f0e5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 -- 172.21.15.27:0/4006176105 shutdown_connections 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 --2- 172.21.15.27:0/4006176105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e6410c020 0x7f0e64112850 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 --2- 172.21.15.27:0/4006176105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410b6d0 0x7f0e6410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 --2- 172.21.15.27:0/4006176105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e64105d90 0x7f0e641061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 -- 172.21.15.27:0/4006176105 >> 172.21.15.27:0/4006176105 conn(0x7f0e64101860 msgr2=0x7f0e64103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 -- 172.21.15.27:0/4006176105 shutdown_connections 2024-09-21T07:19:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.718+0000 7f0e6a400640 1 -- 172.21.15.27:0/4006176105 wait complete. 2024-09-21T07:19:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.719+0000 7f0e6a400640 1 Processor -- start 2024-09-21T07:19:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.719+0000 7f0e6a400640 1 -- start start 2024-09-21T07:19:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e6a400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e64105d90 0x7f0e6419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e6a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410b6d0 0x7f0e6419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e6a400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6410c020 0x7f0e641a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e6a400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64115ce0 con 0x7f0e6410b6d0 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e6a400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64115b60 con 0x7f0e6410c020 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e6a400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64115e60 con 0x7f0e64105d90 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e63400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e64105d90 0x7f0e6419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410b6d0 0x7f0e6419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e63400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e64105d90 0x7f0e6419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:48634/0 (socket says 172.21.15.27:48634) 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e63400640 1 -- 172.21.15.27:0/546509748 learned_addr learned my addr 172.21.15.27:0/546509748 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.720+0000 7f0e63e00640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6410c020 0x7f0e641a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e63400640 1 -- 172.21.15.27:0/546509748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6410c020 msgr2=0x7f0e641a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e63400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6410c020 0x7f0e641a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e63400640 1 -- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410b6d0 msgr2=0x7f0e6419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e63400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410b6d0 0x7f0e6419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e63400640 1 -- 172.21.15.27:0/546509748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e641a26e0 con 0x7f0e64105d90 2024-09-21T07:19:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e62a00640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410b6d0 0x7f0e6419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e63400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e64105d90 0x7f0e6419c770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e9f0 tx=0x7f0e5400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e5be00640 1 -- 172.21.15.27:0/546509748 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400cd80 con 0x7f0e64105d90 2024-09-21T07:19:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e5be00640 1 -- 172.21.15.27:0/546509748 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e540054d0 con 0x7f0e64105d90 2024-09-21T07:19:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.721+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e641a29d0 con 0x7f0e64105d90 2024-09-21T07:19:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.722+0000 7f0e5be00640 1 -- 172.21.15.27:0/546509748 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54010690 con 0x7f0e64105d90 2024-09-21T07:19:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.722+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e641a9130 con 0x7f0e64105d90 2024-09-21T07:19:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 Processor -- start 2024-09-21T07:19:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 -- start start 2024-09-21T07:19:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c87b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83f2390 0x55b2c87ba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b2c8730f70 con 0x55b2c83f2390 2024-09-21T07:19:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b2c87203a0 con 0x55b2c8413070 2024-09-21T07:19:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e710d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2c87241b0 con 0x55b2c83ea1b0 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.725+0000 7f0e5be00640 1 -- 172.21.15.27:0/546509748 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0e54004e90 con 0x7f0e64105d90 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83f2390 0x55b2c87ba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c87b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83f2390 0x55b2c87ba7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54964/0 (socket says 172.21.15.27:54964) 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e62a00640 1 -- 172.21.15.27:0/1095619626 learned_addr learned my addr 172.21.15.27:0/1095619626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63400640 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63e00640 1 -- 172.21.15.27:0/1095619626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 msgr2=0x55b2c87ba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87ba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63e00640 1 -- 172.21.15.27:0/1095619626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83f2390 msgr2=0x55b2c87ba7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83f2390 0x55b2c87ba7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63e00640 1 -- 172.21.15.27:0/1095619626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c86eac90 con 0x55b2c8413070 2024-09-21T07:19:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e62a00640 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83f2390 0x55b2c87ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.726+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c87b9d60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005e30 tx=0x7f0e5c02f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22709b1c13ecd0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.727+0000 7f0e58c00640 1 -- 172.21.15.27:0/1095619626 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c031cf0 con 0x55b2c8413070 2024-09-21T07:19:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.727+0000 7f0e58c00640 1 -- 172.21.15.27:0/1095619626 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c036030 con 0x55b2c8413070 2024-09-21T07:19:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.727+0000 7f0e58c00640 1 -- 172.21.15.27:0/1095619626 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c004040 con 0x55b2c8413070 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.727+0000 7f0e5be00640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6700 0x7f0e480b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e5be00640 1 -- 172.21.15.27:0/546509748 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7f0e540e7ef0 con 0x7f0e64105d90 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 -- 172.21.15.27:0/1095619626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 msgr2=0x55b2c87b9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c87b9d60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005e30 tx=0x7f0e5c02f410 comp rx=0 tx=0).stop 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e63400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6700 0x7f0e480b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 -- 172.21.15.27:0/1095619626 shutdown_connections 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87ba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83f2390 0x55b2c87ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1095619626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c87b9d60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 -- 172.21.15.27:0/1095619626 >> 172.21.15.27:0/1095619626 conn(0x55b2c8407460 msgr2=0x55b2c87b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 -- 172.21.15.27:0/1095619626 shutdown_connections 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.728+0000 7f0e710d3740 1 -- 172.21.15.27:0/1095619626 wait complete. 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.729+0000 7f0e63400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6700 0x7f0e480b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55b2c87b89e0 tx=0x7f0e54004a90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.729+0000 7f0e710d3740 1 -- start start 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.729+0000 7f0e710d3740 1 Processor -- start 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.729+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83f2390 0x55b2c87c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e710d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b2c8581700 con 0x55b2c8413070 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e710d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b2c86bb540 con 0x55b2c83f2390 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e710d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2c87c8360 con 0x55b2c83ea1b0 2024-09-21T07:19:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e710d3740 10 client.0 _ll_register_callbacks cb 0x55b2c86cb430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:48652/0 (socket says 172.21.15.27:48652) 2024-09-21T07:19:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 -- 172.21.15.27:0/1993846868 learned_addr learned my addr 172.21.15.27:0/1993846868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e62a00640 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83f2390 0x55b2c87c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 -- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83f2390 msgr2=0x55b2c87c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83f2390 0x55b2c87c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 -- 172.21.15.27:0/1993846868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 msgr2=0x55b2c87d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.730+0000 7f0e63e00640 1 -- 172.21.15.27:0/1993846868 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c847e650 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.731+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c54e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c003c50 tx=0x7f0e5c02b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2485ab46f580e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.731+0000 7f0e4d600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.731+0000 7f0e4d600640 1 -- 172.21.15.27:0/1993846868 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c057060 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.731+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2c86bdb50 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.731+0000 7f0e4d600640 1 -- 172.21.15.27:0/1993846868 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7f0e5c031450 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.731+0000 7f0e4d600640 1 -- 172.21.15.27:0/1993846868 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e5c05e050 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.731+0000 7f0e710d3740 10 client.27283 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2c86d8910 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e4d600640 1 -- 172.21.15.27:0/1993846868 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b2c86bdb50 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e4d600640 1 -- 172.21.15.27:0/1993846868 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0e5c043070 con 0x55b2c83ea1b0 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e710d3740 10 client.27283 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e710d3740 10 client.27283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e710d3740 20 client.27283 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83d19f0 0x55b2c87d72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e710d3740 4 client.27283 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.732+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2c8732ec0 con 0x55b2c83d19f0 2024-09-21T07:19:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.733+0000 7f0e62a00640 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83d19f0 0x55b2c87d72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.733+0000 7f0e62a00640 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83d19f0 0x55b2c87d72d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4d6a0d77a75619 server_cookie=62898e765fab3d6d in_seq=0 out_seq=0 2024-09-21T07:19:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.733+0000 7f0e4d600640 10 client.27283 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.734+0000 7f0e4d600640 1 -- 172.21.15.27:0/1993846868 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b2c8732ec0 con 0x55b2c83d19f0 2024-09-21T07:19:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.734+0000 7f0e4d600640 10 client.27283 handle_command_reply: tid=0 2024-09-21T07:19:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.734+0000 7f0e710d3740 1 client.27283 shutdown 2024-09-21T07:19:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.734+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 msgr2=0x55b2c87c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c54e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c003c50 tx=0x7f0e5c02b040 comp rx=0 tx=0).stop 2024-09-21T07:19:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 shutdown_connections 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83d19f0 0x55b2c87d72d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83f2390 0x55b2c87c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1993846868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c54e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e4d600640 0 client.27283 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 >> 172.21.15.27:0/1993846868 conn(0x55b2c8407460 msgr2=0x55b2c87bcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 shutdown_connections 2024-09-21T07:19:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 1 -- 172.21.15.27:0/1993846868 wait complete. 2024-09-21T07:19:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.735+0000 7f0e710d3740 20 client.27283 trim_cache size 0 max 16384 2024-09-21T07:19:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.753+0000 7f0e710d3740 1 Processor -- start 2024-09-21T07:19:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.753+0000 7f0e710d3740 1 -- start start 2024-09-21T07:19:09.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e710d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b5970 con 0x55b2c8413070 2024-09-21T07:19:09.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e710d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b3240 con 0x55b2c83ea1b0 2024-09-21T07:19:09.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e710d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b10b0 con 0x55b2c83f2390 2024-09-21T07:19:09.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87c26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54972/0 (socket says 172.21.15.27:54972) 2024-09-21T07:19:09.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e63e00640 1 -- 172.21.15.27:0/1650594700 learned_addr learned my addr 172.21.15.27:0/1650594700 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e62a00640 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.754+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e63e00640 1 -- 172.21.15.27:0/1650594700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 msgr2=0x55b2c87d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e63e00640 1 -- 172.21.15.27:0/1650594700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83ea1b0 msgr2=0x55b2c87c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e63e00640 1 -- 172.21.15.27:0/1650594700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c87bc050 con 0x55b2c8413070 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e63e00640 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87c26d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c02f6c0 tx=0x7f0e5c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565e9cede1a885fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e4fe00640 1 -- 172.21.15.27:0/1650594700 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c04a040 con 0x55b2c8413070 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e4fe00640 1 -- 172.21.15.27:0/1650594700 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c036070 con 0x55b2c8413070 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.755+0000 7f0e4fe00640 1 -- 172.21.15.27:0/1650594700 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c02f930 con 0x55b2c8413070 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 -- 172.21.15.27:0/1650594700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 msgr2=0x55b2c87c26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87c26d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c02f6c0 tx=0x7f0e5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 -- 172.21.15.27:0/1650594700 shutdown_connections 2024-09-21T07:19:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c8413070 0x55b2c87c26d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 --2- 172.21.15.27:0/1650594700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c83ea1b0 0x55b2c87c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 -- 172.21.15.27:0/1650594700 >> 172.21.15.27:0/1650594700 conn(0x55b2c8407460 msgr2=0x55b2c87c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 -- 172.21.15.27:0/1650594700 shutdown_connections 2024-09-21T07:19:09.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.756+0000 7f0e710d3740 1 -- 172.21.15.27:0/1650594700 wait complete. 2024-09-21T07:19:09.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.757+0000 7f0e710d3740 1 -- start start 2024-09-21T07:19:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.757+0000 7f0e710d3740 1 Processor -- start 2024-09-21T07:19:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.757+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83ea1b0 0x55b2c87d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.757+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.757+0000 7f0e710d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c8840a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e710d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b1700 con 0x55b2c83ea1b0 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e710d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b2b70 con 0x55b2c8413070 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e710d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641b1dd0 con 0x55b2c83f2390 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e710d3740 10 client.0 _ll_register_callbacks cb 0x7f0e5c031e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83ea1b0 0x55b2c87d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83ea1b0 0x55b2c87d89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54986/0 (socket says 172.21.15.27:54986) 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 -- 172.21.15.27:0/3426747327 learned_addr learned my addr 172.21.15.27:0/3426747327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e62a00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c8840a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 -- 172.21.15.27:0/3426747327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 msgr2=0x55b2c87d8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 -- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 msgr2=0x55b2c8840a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c8840a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e63e00640 1 -- 172.21.15.27:0/3426747327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c8840f90 con 0x55b2c83ea1b0 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.758+0000 7f0e62a00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e63e00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83ea1b0 0x55b2c87d89e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55b2c87db250 tx=0x7f0e5c003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b643920d44893b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e4d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e4d600640 1 -- 172.21.15.27:0/3426747327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c02fb30 con 0x55b2c83ea1b0 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e4d600640 1 -- 172.21.15.27:0/3426747327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9549+0+0 (secure 0 0 0) 0x7f0e5c031680 con 0x55b2c83ea1b0 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e5c036070 con 0x55b2c83ea1b0 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e4d600640 1 -- 172.21.15.27:0/3426747327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e5c075050 con 0x55b2c83ea1b0 2024-09-21T07:19:09.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e710d3740 10 client.17283 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.759+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2c88421d0 con 0x55b2c83ea1b0 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e4d600640 1 -- 172.21.15.27:0/3426747327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c036070 con 0x55b2c83ea1b0 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e4d600640 1 -- 172.21.15.27:0/3426747327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0e5c043070 con 0x55b2c83ea1b0 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e710d3740 10 client.17283 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e710d3740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e710d3740 20 client.17283 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e710d3740 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83a6da0 0x55b2c8844850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e710d3740 4 client.17283 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2c872e970 con 0x55b2c83a6da0 2024-09-21T07:19:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.760+0000 7f0e62a00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83a6da0 0x55b2c8844850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:09.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.761+0000 7f0e62a00640 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83a6da0 0x55b2c8844850 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9900118e4303672d server_cookie=84eacab4919b498f in_seq=0 out_seq=0 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.761+0000 7f0e4d600640 10 client.17283 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.761+0000 7f0e4d600640 1 -- 172.21.15.27:0/3426747327 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x55b2c872e970 con 0x55b2c83a6da0 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.761+0000 7f0e4d600640 10 client.17283 handle_command_reply: tid=0 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.761+0000 7f0e710d3740 1 client.17283 shutdown 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83ea1b0 msgr2=0x55b2c87d89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83ea1b0 0x55b2c87d89e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55b2c87db250 tx=0x7f0e5c003d90 comp rx=0 tx=0).stop 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 shutdown_connections 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b2c83a6da0 0x55b2c8844850 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2c83f2390 0x55b2c87d8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2c8413070 0x55b2c8840a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 --2- 172.21.15.27:0/3426747327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b2c83ea1b0 0x55b2c87d89e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 >> 172.21.15.27:0/3426747327 conn(0x55b2c8407460 msgr2=0x55b2c87bd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e4d600640 0 client.17283 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 shutdown_connections 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 1 -- 172.21.15.27:0/3426747327 wait complete. 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.762+0000 7f0e710d3740 20 client.17283 trim_cache size 0 max 16384 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "88a53041-1567-48da-a7e8-c17f689afb84", 2024-09-21T07:19:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:19:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:19:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:19:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6700 msgr2=0x7f0e480b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6700 0x7f0e480b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55b2c87b89e0 tx=0x7f0e54004a90 comp rx=0 tx=0).stop 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e64105d90 msgr2=0x7f0e6419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e64105d90 0x7f0e6419c770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e9f0 tx=0x7f0e5400edf0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 shutdown_connections 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0e480b6700 0x7f0e480b8c10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e6410c020 0x7f0e641a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e6410b6d0 0x7f0e6419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 --2- 172.21.15.27:0/546509748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e64105d90 0x7f0e6419c770 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:09.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 >> 172.21.15.27:0/546509748 conn(0x7f0e64101860 msgr2=0x7f0e64108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:09.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 shutdown_connections 2024-09-21T07:19:09.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:09.763+0000 7f0e6a400640 1 -- 172.21.15.27:0/546509748 wait complete. 2024-09-21T07:19:10.656 INFO:tasks.cephfs.filesystem:scrub status for tag:88a53041-1567-48da-a7e8-c17f689afb84 - {'path': '/', 'tag': '88a53041-1567-48da-a7e8-c17f689afb84', 'options': 'recursive,force'} 2024-09-21T07:19:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:10 smithi119 ceph-mon[34519]: pgmap v563: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-09-21T07:19:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:10 smithi027 ceph-mon[30779]: pgmap v563: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-09-21T07:19:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:10 smithi153 ceph-mon[34747]: pgmap v563: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-09-21T07:19:12.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:12 smithi153 ceph-mon[34747]: pgmap v564: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-09-21T07:19:12.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:12 smithi153 ceph-mon[34747]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T07:19:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:12 smithi119 ceph-mon[34519]: pgmap v564: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-09-21T07:19:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:12 smithi119 ceph-mon[34519]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T07:19:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:12 smithi027 ceph-mon[30779]: pgmap v564: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-09-21T07:19:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:12 smithi027 ceph-mon[30779]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T07:19:14.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:14 smithi153 ceph-mon[34747]: pgmap v566: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:19:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:14 smithi119 ceph-mon[34519]: pgmap v566: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:19:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:14 smithi027 ceph-mon[30779]: pgmap v566: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:19:16.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:16 smithi153 ceph-mon[34747]: pgmap v567: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:19:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:16 smithi119 ceph-mon[34519]: pgmap v567: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:19:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:16 smithi027 ceph-mon[30779]: pgmap v567: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:19:18.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:18 smithi153 ceph-mon[34747]: pgmap v568: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:19:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:18 smithi119 ceph-mon[34519]: pgmap v568: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:19:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:18 smithi027 ceph-mon[30779]: pgmap v568: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:19:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:20 smithi119 ceph-mon[34519]: pgmap v569: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:19:21.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:20 smithi027 ceph-mon[30779]: pgmap v569: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:19:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:20 smithi153 ceph-mon[34747]: pgmap v569: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-21T07:19:21.815 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:19:22.211 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.889+0000 7f07b1800640 1 -- 172.21.15.27:0/3695006976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac103ae0 msgr2=0x7f07ac103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.889+0000 7f07b1800640 1 --2- 172.21.15.27:0/3695006976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac103ae0 0x7f07ac103ef0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f079c009900 tx=0x7f079c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.889+0000 7f07b1800640 1 -- 172.21.15.27:0/3695006976 shutdown_connections 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.889+0000 7f07b1800640 1 --2- 172.21.15.27:0/3695006976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac109d60 0x7f07ac110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.889+0000 7f07b1800640 1 --2- 172.21.15.27:0/3695006976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 0x7f07ac109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.889+0000 7f07b1800640 1 --2- 172.21.15.27:0/3695006976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac103ae0 0x7f07ac103ef0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.889+0000 7f07b1800640 1 -- 172.21.15.27:0/3695006976 >> 172.21.15.27:0/3695006976 conn(0x7f07ac0ff520 msgr2=0x7f07ac101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.890+0000 7f07b1800640 1 -- 172.21.15.27:0/3695006976 shutdown_connections 2024-09-21T07:19:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.890+0000 7f07b1800640 1 -- 172.21.15.27:0/3695006976 wait complete. 2024-09-21T07:19:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 Processor -- start 2024-09-21T07:19:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 -- start start 2024-09-21T07:19:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac103ae0 0x7f07ac19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 0x7f07ac19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac109d60 0x7f07ac19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac113a20 con 0x7f07ac109410 2024-09-21T07:19:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1138a0 con 0x7f07ac109d60 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.891+0000 7f07b1800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac113ba0 con 0x7f07ac103ae0 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 0x7f07ac19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac103ae0 0x7f07ac19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac103ae0 0x7f07ac19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:37584/0 (socket says 172.21.15.27:37584) 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 0x7f07ac19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44222/0 (socket says 172.21.15.27:44222) 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 -- 172.21.15.27:0/3416547963 learned_addr learned my addr 172.21.15.27:0/3416547963 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07ab400640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac109d60 0x7f07ac19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 -- 172.21.15.27:0/3416547963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac109d60 msgr2=0x7f07ac19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac109d60 0x7f07ac19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 -- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 msgr2=0x7f07ac19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 0x7f07ac19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07aaa00640 1 -- 172.21.15.27:0/3416547963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac1a0390 con 0x7f07ac103ae0 2024-09-21T07:19:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.892+0000 7f07a3e00640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 0x7f07ac19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.893+0000 7f07aaa00640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac103ae0 0x7f07ac19a460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f079c005940 tx=0x7f079c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.893+0000 7f07a3400640 1 -- 172.21.15.27:0/3416547963 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c03f030 con 0x7f07ac103ae0 2024-09-21T07:19:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.893+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac1a0620 con 0x7f07ac103ae0 2024-09-21T07:19:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.893+0000 7f07a3400640 1 -- 172.21.15.27:0/3416547963 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f079c0044c0 con 0x7f07ac103ae0 2024-09-21T07:19:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.893+0000 7f07a3400640 1 -- 172.21.15.27:0/3416547963 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c035070 con 0x7f07ac103ae0 2024-09-21T07:19:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.893+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac1a6e10 con 0x7f07ac103ae0 2024-09-21T07:19:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.897+0000 7f07a3400640 1 -- 172.21.15.27:0/3416547963 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f079c042030 con 0x7f07ac103ae0 2024-09-21T07:19:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 Processor -- start 2024-09-21T07:19:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 -- start start 2024-09-21T07:19:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476f1f1b0 0x5564772c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564772396b0 con 0x556476f1f1b0 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556477228ae0 con 0x556476ef62f0 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.898+0000 7f07b8878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55647722c8f0 con 0x556476efe4d0 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476f1f1b0 0x5564772c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476f1f1b0 0x5564772c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44224/0 (socket says 172.21.15.27:44224) 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07aaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07ab400640 1 -- 172.21.15.27:0/741134406 learned_addr learned my addr 172.21.15.27:0/741134406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3400640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07900b6700 0x7f07900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3e00640 1 -- 172.21.15.27:0/741134406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 msgr2=0x5564772c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3e00640 1 --2- 172.21.15.27:0/741134406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3400640 1 -- 172.21.15.27:0/3416547963 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f079c107e30 con 0x7f07ac103ae0 2024-09-21T07:19:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3e00640 1 -- 172.21.15.27:0/741134406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476f1f1b0 msgr2=0x5564772c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3e00640 1 --2- 172.21.15.27:0/741134406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476f1f1b0 0x5564772c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07ab400640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07900b6700 0x7f07900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.899+0000 7f07a3e00640 1 -- 172.21.15.27:0/741134406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564771f5780 con 0x556476efe4d0 2024-09-21T07:19:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.900+0000 7f07a3e00640 1 --2- 172.21.15.27:0/741134406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772c46b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f07ac06c640 tx=0x7f07940118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e45a8f8659c6f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.900+0000 7f07ab400640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07900b6700 0x7f07900b8c10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009fd0 tx=0x7f07a4004ab0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.900+0000 7f079be00640 1 -- 172.21.15.27:0/741134406 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794004020 con 0x556476efe4d0 2024-09-21T07:19:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.900+0000 7f079be00640 1 -- 172.21.15.27:0/741134406 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940054d0 con 0x556476efe4d0 2024-09-21T07:19:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.900+0000 7f079be00640 1 -- 172.21.15.27:0/741134406 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794004bd0 con 0x556476efe4d0 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 -- 172.21.15.27:0/741134406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 msgr2=0x5564772c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 --2- 172.21.15.27:0/741134406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772c46b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f07ac06c640 tx=0x7f07940118d0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 -- 172.21.15.27:0/741134406 shutdown_connections 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 --2- 172.21.15.27:0/741134406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772c46b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 --2- 172.21.15.27:0/741134406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 --2- 172.21.15.27:0/741134406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476f1f1b0 0x5564772c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 -- 172.21.15.27:0/741134406 >> 172.21.15.27:0/741134406 conn(0x556476f13300 msgr2=0x5564772c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 -- 172.21.15.27:0/741134406 shutdown_connections 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.901+0000 7f07b8878740 1 -- 172.21.15.27:0/741134406 wait complete. 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 -- start start 2024-09-21T07:19:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 Processor -- start 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476ef62f0 0x5564772d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556477160ab0 con 0x556476ef62f0 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5564771cf570 con 0x556476f1f1b0 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.902+0000 7f07b8878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5564772d6790 con 0x556476efe4d0 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07b8878740 10 client.0 _ll_register_callbacks cb 0x5564771de100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476ef62f0 0x5564772d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476ef62f0 0x5564772d3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44236/0 (socket says 172.21.15.27:44236) 2024-09-21T07:19:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 -- 172.21.15.27:0/695346673 learned_addr learned my addr 172.21.15.27:0/695346673 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07ab400640 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 -- 172.21.15.27:0/695346673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 msgr2=0x5564772d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 -- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 msgr2=0x5564772d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07aaa00640 1 -- 172.21.15.27:0/695346673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556476f8a790 con 0x556476ef62f0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.903+0000 7f07a3e00640 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f07aaa00640 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476ef62f0 0x5564772d3550 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f079c0cd860 tx=0x7f079c03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a806f5aea1be097c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f0798c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f0798c00640 1 -- 172.21.15.27:0/695346673 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c049070 con 0x556476ef62f0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f0798c00640 1 -- 172.21.15.27:0/695346673 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f079c0b4d20 con 0x556476ef62f0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07940054d0 con 0x556476ef62f0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f0798c00640 1 -- 172.21.15.27:0/695346673 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f079c105600 con 0x556476ef62f0 2024-09-21T07:19:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f07b8878740 10 client.17295 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556477003620 con 0x556476ef62f0 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.904+0000 7f0798c00640 1 -- 172.21.15.27:0/695346673 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940054d0 con 0x556476ef62f0 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f0798c00640 1 -- 172.21.15.27:0/695346673 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f079c0d6e00 con 0x556476ef62f0 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f07b8878740 10 client.17295 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f07b8878740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f07b8878740 20 client.17295 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f07b8878740 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eddd80 0x5564772e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f07b8878740 4 client.17295 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55647723b600 con 0x556476eddd80 2024-09-21T07:19:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.905+0000 7f07a3e00640 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eddd80 0x5564772e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.906+0000 7f0798c00640 10 client.17295 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.906+0000 7f07a3e00640 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eddd80 0x5564772e5750 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e253a05237b147 server_cookie=b9304927bd6768b6 in_seq=0 out_seq=0 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.907+0000 7f0798c00640 1 -- 172.21.15.27:0/695346673 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55647723b600 con 0x556476eddd80 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.907+0000 7f0798c00640 10 client.17295 handle_command_reply: tid=0 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.907+0000 7f07b8878740 1 client.17295 shutdown 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.907+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476ef62f0 msgr2=0x5564772d3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.907+0000 7f07b8878740 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476ef62f0 0x5564772d3550 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f079c0cd860 tx=0x7f079c03b460 comp rx=0 tx=0).stop 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 shutdown_connections 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eddd80 0x5564772e5750 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476efe4d0 0x5564772d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 --2- 172.21.15.27:0/695346673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476ef62f0 0x5564772d3550 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f0798c00640 0 client.17295 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 >> 172.21.15.27:0/695346673 conn(0x556476f13300 msgr2=0x5564772c00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 shutdown_connections 2024-09-21T07:19:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 1 -- 172.21.15.27:0/695346673 wait complete. 2024-09-21T07:19:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.908+0000 7f07b8878740 20 client.17295 trim_cache size 0 max 16384 2024-09-21T07:19:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 Processor -- start 2024-09-21T07:19:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 -- start start 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476ef62f0 0x5564772ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1b1500 con 0x556476efe4d0 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1aedd0 con 0x556476f1f1b0 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.939+0000 7f07b8878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1acc40 con 0x556476ef62f0 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772c13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44244/0 (socket says 172.21.15.27:44244) 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 -- 172.21.15.27:0/191043401 learned_addr learned my addr 172.21.15.27:0/191043401 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07ab400640 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07aaa00640 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476ef62f0 0x5564772ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 -- 172.21.15.27:0/191043401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476ef62f0 msgr2=0x5564772ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476ef62f0 0x5564772ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 -- 172.21.15.27:0/191043401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 msgr2=0x5564772d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07a3e00640 1 -- 172.21.15.27:0/191043401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564772c4a20 con 0x556476efe4d0 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.940+0000 7f07ab400640 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.941+0000 7f07a3e00640 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772c13d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0794009ef0 tx=0x7f0794009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d78a62aa21ceae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.941+0000 7f079b400640 1 -- 172.21.15.27:0/191043401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079401d040 con 0x556476efe4d0 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.941+0000 7f079b400640 1 -- 172.21.15.27:0/191043401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f079400fbe0 con 0x556476efe4d0 2024-09-21T07:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.941+0000 7f079b400640 1 -- 172.21.15.27:0/191043401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940038b0 con 0x556476efe4d0 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.942+0000 7f07b8878740 1 -- 172.21.15.27:0/191043401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 msgr2=0x5564772c13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.942+0000 7f07b8878740 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772c13d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0794009ef0 tx=0x7f0794009310 comp rx=0 tx=0).stop 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.942+0000 7f07b8878740 1 -- 172.21.15.27:0/191043401 shutdown_connections 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.942+0000 7f07b8878740 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476f1f1b0 0x5564772d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.942+0000 7f07b8878740 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772c13d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.942+0000 7f07b8878740 1 --2- 172.21.15.27:0/191043401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476ef62f0 0x5564772ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.942+0000 7f07b8878740 1 -- 172.21.15.27:0/191043401 >> 172.21.15.27:0/191043401 conn(0x556476f13300 msgr2=0x5564772c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.943+0000 7f07b8878740 1 -- 172.21.15.27:0/191043401 shutdown_connections 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.943+0000 7f07b8878740 1 -- 172.21.15.27:0/191043401 wait complete. 2024-09-21T07:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.943+0000 7f07b8878740 1 -- start start 2024-09-21T07:19:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.943+0000 7f07b8878740 1 Processor -- start 2024-09-21T07:19:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.944+0000 7f07b8878740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.944+0000 7f07b8878740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.944+0000 7f07b8878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476f1f1b0 0x5564772d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.944+0000 7f07b8878740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1ad290 con 0x556476efe4d0 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.944+0000 7f07b8878740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1ae700 con 0x556476ef62f0 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.944+0000 7f07b8878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1ad960 con 0x556476f1f1b0 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.944+0000 7f07b8878740 10 client.0 _ll_register_callbacks cb 0x7f079c005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07ab400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476f1f1b0 0x5564772d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772d7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44250/0 (socket says 172.21.15.27:44250) 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 -- 172.21.15.27:0/2537660197 learned_addr learned my addr 172.21.15.27:0/2537660197 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07aaa00640 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 -- 172.21.15.27:0/2537660197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476f1f1b0 msgr2=0x5564772d7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476f1f1b0 0x5564772d7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 -- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 msgr2=0x5564772dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 -- 172.21.15.27:0/2537660197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564772d80d0 con 0x556476efe4d0 2024-09-21T07:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f07a3e00640 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772d7580 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0794000c00 tx=0x7f0794005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb14f913af88bd68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.945+0000 7f0798c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f0798c00640 1 -- 172.21.15.27:0/2537660197 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079401d030 con 0x556476efe4d0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f0798c00640 1 -- 172.21.15.27:0/2537660197 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f079400fd90 con 0x556476efe4d0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f079400fbe0 con 0x556476efe4d0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f0798c00640 1 -- 172.21.15.27:0/2537660197 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0794031440 con 0x556476efe4d0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f07b8878740 10 client.17301 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55647734ff30 con 0x556476efe4d0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f0798c00640 1 -- 172.21.15.27:0/2537660197 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f079400fbe0 con 0x556476efe4d0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.946+0000 7f0798c00640 1 -- 172.21.15.27:0/2537660197 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f079402f020 con 0x556476efe4d0 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07b8878740 10 client.17301 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07b8878740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07b8878740 20 client.17301 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07b8878740 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eb31c0 0x556477352380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07b8878740 4 client.17301 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:19:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564772370b0 con 0x556476eb31c0 2024-09-21T07:19:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07aaa00640 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eb31c0 0x556477352380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f07aaa00640 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eb31c0 0x556477352380 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2817b863ba70135 server_cookie=5798a4946e59b5e in_seq=0 out_seq=0 2024-09-21T07:19:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.947+0000 7f0798c00640 10 client.17301 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f0798c00640 1 -- 172.21.15.27:0/2537660197 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564772370b0 con 0x556476eb31c0 2024-09-21T07:19:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f0798c00640 10 client.17301 handle_command_reply: tid=0 2024-09-21T07:19:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 client.17301 shutdown 2024-09-21T07:19:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 msgr2=0x5564772d7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772d7580 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0794000c00 tx=0x7f0794005fc0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 shutdown_connections 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556476eb31c0 0x556477352380 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556476f1f1b0 0x5564772d7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556476efe4d0 0x5564772d7580 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f0798c00640 0 client.17301 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 --2- 172.21.15.27:0/2537660197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556476ef62f0 0x5564772dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 >> 172.21.15.27:0/2537660197 conn(0x556476f13300 msgr2=0x5564772ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 shutdown_connections 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.956+0000 7f07b8878740 1 -- 172.21.15.27:0/2537660197 wait complete. 2024-09-21T07:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b8878740 20 client.17301 trim_cache size 0 max 16384 2024-09-21T07:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07900b6700 msgr2=0x7f07900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07900b6700 0x7f07900b8c10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009fd0 tx=0x7f07a4004ab0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac103ae0 msgr2=0x7f07ac19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac103ae0 0x7f07ac19a460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f079c005940 tx=0x7f079c0040c0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 shutdown_connections 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f07900b6700 0x7f07900b8c10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07ac109d60 0x7f07ac19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07ac109410 0x7f07ac19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 --2- 172.21.15.27:0/3416547963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07ac103ae0 0x7f07ac19a460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.957+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 >> 172.21.15.27:0/3416547963 conn(0x7f07ac0ff520 msgr2=0x7f07ac101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.958+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 shutdown_connections 2024-09-21T07:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:22.958+0000 7f07b1800640 1 -- 172.21.15.27:0/3416547963 wait complete. 2024-09-21T07:19:22.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 -- 172.21.15.27:0/1708180285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368114750 msgr2=0x7f4368116b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 --2- 172.21.15.27:0/1708180285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368114750 0x7f4368116b70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4360009ca0 tx=0x7f436002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 -- 172.21.15.27:0/1708180285 shutdown_connections 2024-09-21T07:19:22.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 --2- 172.21.15.27:0/1708180285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368114750 0x7f4368116b70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 --2- 172.21.15.27:0/1708180285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 0x7f4368114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 --2- 172.21.15.27:0/1708180285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368077940 0x7f4368077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 -- 172.21.15.27:0/1708180285 >> 172.21.15.27:0/1708180285 conn(0x7f43681017c0 msgr2=0x7f4368103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:22.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 -- 172.21.15.27:0/1708180285 shutdown_connections 2024-09-21T07:19:22.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.991+0000 7f436d000640 1 -- 172.21.15.27:0/1708180285 wait complete. 2024-09-21T07:19:22.994 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:22 smithi027 ceph-mon[30779]: pgmap v570: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-21T07:19:22.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.993+0000 7f436d000640 1 Processor -- start 2024-09-21T07:19:22.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.993+0000 7f436d000640 1 -- start start 2024-09-21T07:19:22.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f436d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368077940 0x7f43681a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f436d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 0x7f43681a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f436d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368114750 0x7f43681a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:22.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f436d000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f436811a0c0 con 0x7f4368077940 2024-09-21T07:19:22.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f436d000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4368119f40 con 0x7f4368078290 2024-09-21T07:19:22.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f436d000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f436811a240 con 0x7f4368114750 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f4365e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368077940 0x7f43681a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f435fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 0x7f43681a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f435fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 0x7f43681a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56534/0 (socket says 172.21.15.27:56534) 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f4365e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368077940 0x7f43681a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44266/0 (socket says 172.21.15.27:44266) 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f435fe00640 1 -- 172.21.15.27:0/746922124 learned_addr learned my addr 172.21.15.27:0/746922124 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.994+0000 7f4366800640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368114750 0x7f43681a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f4365e00640 1 -- 172.21.15.27:0/746922124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368114750 msgr2=0x7f43681a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f4365e00640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368114750 0x7f43681a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f4365e00640 1 -- 172.21.15.27:0/746922124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 msgr2=0x7f43681a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f4365e00640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 0x7f43681a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f4365e00640 1 -- 172.21.15.27:0/746922124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43681a69b0 con 0x7f4368077940 2024-09-21T07:19:22.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f4366800640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368114750 0x7f43681a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:22.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f435fe00640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 0x7f43681a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:22.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f4365e00640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368077940 0x7f43681a0a40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f435800e9f0 tx=0x7f435800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:22.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f435ea00640 1 -- 172.21.15.27:0/746922124 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435800ccc0 con 0x7f4368077940 2024-09-21T07:19:22.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f435ea00640 1 -- 172.21.15.27:0/746922124 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43580108e0 con 0x7f4368077940 2024-09-21T07:19:22.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43681a6ca0 con 0x7f4368077940 2024-09-21T07:19:22.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.995+0000 7f435ea00640 1 -- 172.21.15.27:0/746922124 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358018880 con 0x7f4368077940 2024-09-21T07:19:22.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.996+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43681ad400 con 0x7f4368077940 2024-09-21T07:19:22.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:22.997+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4368077d50 con 0x7f4368077940 2024-09-21T07:19:23.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.001+0000 7f435ea00640 1 -- 172.21.15.27:0/746922124 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4358010430 con 0x7f4368077940 2024-09-21T07:19:23.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.003+0000 7f435ea00640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f434c0b6700 0x7f434c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:23.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.003+0000 7f435ea00640 1 -- 172.21.15.27:0/746922124 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f43580e82c0 con 0x7f4368077940 2024-09-21T07:19:23.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.003+0000 7f435fe00640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f434c0b6700 0x7f434c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:23.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.004+0000 7f435fe00640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f434c0b6700 0x7f434c0b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f436807ade0 tx=0x7f4350005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:23.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.005+0000 7f435ea00640 1 -- 172.21.15.27:0/746922124 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f43580b27e0 con 0x7f4368077940 2024-09-21T07:19:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:22 smithi119 ceph-mon[34519]: pgmap v570: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-21T07:19:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:22 smithi153 ceph-mon[34747]: pgmap v570: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-21T07:19:23.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.240+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f436810f940 con 0x7f4368077940 2024-09-21T07:19:23.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.241+0000 7f435ea00640 1 -- 172.21.15.27:0/746922124 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f43580b7720 con 0x7f4368077940 2024-09-21T07:19:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:19:23.244 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:19:23.244 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:19:23.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f434c0b6700 msgr2=0x7f434c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f434c0b6700 0x7f434c0b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f436807ade0 tx=0x7f4350005f90 comp rx=0 tx=0).stop 2024-09-21T07:19:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368077940 msgr2=0x7f43681a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368077940 0x7f43681a0a40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f435800e9f0 tx=0x7f435800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:19:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 shutdown_connections 2024-09-21T07:19:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368114750 0x7f43681a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f434c0b6700 0x7f434c0b8c10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4368078290 0x7f43681a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 --2- 172.21.15.27:0/746922124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4368077940 0x7f43681a0a40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 >> 172.21.15.27:0/746922124 conn(0x7f43681017c0 msgr2=0x7f4368116710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 shutdown_connections 2024-09-21T07:19:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:23.244+0000 7f436d000640 1 -- 172.21.15.27:0/746922124 wait complete. 2024-09-21T07:19:24.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:23 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/746922124' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:24.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:23 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/746922124' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:24.064 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id db39a904 --release --await' 2024-09-21T07:19:24.064 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id db39a904 --release --await 2024-09-21T07:19:24.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:23 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/746922124' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:24.888 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:24 smithi027 ceph-mon[30779]: pgmap v571: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-21T07:19:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:24 smithi119 ceph-mon[34519]: pgmap v571: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-21T07:19:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 -- 172.21.15.27:0/509801649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf8103990 msgr2=0x7fccf8103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 --2- 172.21.15.27:0/509801649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf8103990 0x7fccf8103da0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009900 tx=0x7fcce802a410 comp rx=0 tx=0).stop 2024-09-21T07:19:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 -- 172.21.15.27:0/509801649 shutdown_connections 2024-09-21T07:19:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 --2- 172.21.15.27:0/509801649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccf8109c20 0x7fccf8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 --2- 172.21.15.27:0/509801649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf81092d0 0x7fccf81096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 --2- 172.21.15.27:0/509801649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf8103990 0x7fccf8103da0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 -- 172.21.15.27:0/509801649 >> 172.21.15.27:0/509801649 conn(0x7fccf80ff440 msgr2=0x7fccf8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 -- 172.21.15.27:0/509801649 shutdown_connections 2024-09-21T07:19:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.071+0000 7fccfd000640 1 -- 172.21.15.27:0/509801649 wait complete. 2024-09-21T07:19:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.072+0000 7fccfd000640 1 Processor -- start 2024-09-21T07:19:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.072+0000 7fccfd000640 1 -- start start 2024-09-21T07:19:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccfd000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccf8103990 0x7fccf8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccfd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81092d0 0x7fccf8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccfd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8109c20 0x7fccf8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccfd000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81138e0 con 0x7fccf81092d0 2024-09-21T07:19:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccfd000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8113760 con 0x7fccf8109c20 2024-09-21T07:19:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccfd000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8113a60 con 0x7fccf8103990 2024-09-21T07:19:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fcceea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81092d0 0x7fccf8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fcceea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81092d0 0x7fccf8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44292/0 (socket says 172.21.15.27:44292) 2024-09-21T07:19:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fcceea00640 1 -- 172.21.15.27:0/4254998916 learned_addr learned my addr 172.21.15.27:0/4254998916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccf7400640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccf8103990 0x7fccf8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fccf7e00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8109c20 0x7fccf8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.073+0000 7fcceea00640 1 -- 172.21.15.27:0/4254998916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccf8103990 msgr2=0x7fccf8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fcceea00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccf8103990 0x7fccf8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fcceea00640 1 -- 172.21.15.27:0/4254998916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8109c20 msgr2=0x7fccf8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fcceea00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8109c20 0x7fccf8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fcceea00640 1 -- 172.21.15.27:0/4254998916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf81a8690 con 0x7fccf81092d0 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fccf7e00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8109c20 0x7fccf8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fcceea00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81092d0 0x7fccf8075c50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcce000e970 tx=0x7fcce000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fccf4c00640 1 -- 172.21.15.27:0/4254998916 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce000cd80 con 0x7fccf81092d0 2024-09-21T07:19:25.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fccf4c00640 1 -- 172.21.15.27:0/4254998916 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce00035f0 con 0x7fccf81092d0 2024-09-21T07:19:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf81a8980 con 0x7fccf81092d0 2024-09-21T07:19:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fccf4c00640 1 -- 172.21.15.27:0/4254998916 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0010690 con 0x7fccf81092d0 2024-09-21T07:19:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.074+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81a8ec0 con 0x7fccf81092d0 2024-09-21T07:19:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 Processor -- start 2024-09-21T07:19:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 -- start start 2024-09-21T07:19:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8951340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c8951880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8589840 0x55d3c8951dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d3c88c8290 con 0x55d3c8589840 2024-09-21T07:19:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fccf4c00640 1 -- 172.21.15.27:0/4254998916 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcce0003aa0 con 0x7fccf81092d0 2024-09-21T07:19:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d3c88b76c0 con 0x55d3c8581660 2024-09-21T07:19:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.078+0000 7fcd051ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d3c88bb4d0 con 0x55d3c85aa520 2024-09-21T07:19:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fcceea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8589840 0x55d3c8951dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c8951880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fcceea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8589840 0x55d3c8951dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44296/0 (socket says 172.21.15.27:44296) 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fcceea00640 1 -- 172.21.15.27:0/1262551329 learned_addr learned my addr 172.21.15.27:0/1262551329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c8951880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56568/0 (socket says 172.21.15.27:56568) 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7e00640 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8951340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7400640 1 -- 172.21.15.27:0/1262551329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 msgr2=0x55d3c8951340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7400640 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8951340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7400640 1 -- 172.21.15.27:0/1262551329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8589840 msgr2=0x55d3c8951dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7400640 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8589840 0x55d3c8951dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.079+0000 7fccf7400640 1 -- 172.21.15.27:0/1262551329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c8844460 con 0x55d3c8581660 2024-09-21T07:19:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.080+0000 7fccf7400640 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c8951880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fccf8077160 tx=0x7fcce8032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660c7f5564c6fe7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.080+0000 7fccecc00640 1 -- 172.21.15.27:0/1262551329 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8040070 con 0x55d3c8581660 2024-09-21T07:19:25.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.080+0000 7fccecc00640 1 -- 172.21.15.27:0/1262551329 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8004030 con 0x55d3c8581660 2024-09-21T07:19:25.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.080+0000 7fccecc00640 1 -- 172.21.15.27:0/1262551329 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8035040 con 0x55d3c8581660 2024-09-21T07:19:25.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.080+0000 7fccf4c00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fccdc0b6680 0x7fccdc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fccf4c00640 1 -- 172.21.15.27:0/4254998916 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fcce00e7e60 con 0x7fccf81092d0 2024-09-21T07:19:25.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fccf7e00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fccdc0b6680 0x7fccdc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 -- 172.21.15.27:0/1262551329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 msgr2=0x55d3c8951880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c8951880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fccf8077160 tx=0x7fcce8032b20 comp rx=0 tx=0).stop 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 -- 172.21.15.27:0/1262551329 shutdown_connections 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8589840 0x55d3c8951dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c8951880 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1262551329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8951340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 -- 172.21.15.27:0/1262551329 >> 172.21.15.27:0/1262551329 conn(0x55d3c859e910 msgr2=0x55d3c894f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.081+0000 7fcd051ca740 1 -- 172.21.15.27:0/1262551329 shutdown_connections 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.082+0000 7fcd051ca740 1 -- 172.21.15.27:0/1262551329 wait complete. 2024-09-21T07:19:25.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.082+0000 7fccf7e00640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fccdc0b6680 0x7fccdc0b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fccf0005fd0 tx=0x7fccf0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:25.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.082+0000 7fcd051ca740 1 -- start start 2024-09-21T07:19:25.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.082+0000 7fcd051ca740 1 Processor -- start 2024-09-21T07:19:25.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c895cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8589840 0x55d3c895d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c8968db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcd051ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d3c8718bb0 con 0x55d3c85aa520 2024-09-21T07:19:25.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcd051ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d3c88526c0 con 0x55d3c8581660 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcd051ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d3c895fac0 con 0x55d3c8589840 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcd051ca740 10 client.0 _ll_register_callbacks cb 0x55d3c886d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c8968db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fccf7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c895cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fcceea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8589840 0x55d3c895d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c8968db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44300/0 (socket says 172.21.15.27:44300) 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.083+0000 7fccf7e00640 1 -- 172.21.15.27:0/2541872132 learned_addr learned my addr 172.21.15.27:0/2541872132 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fccf7400640 1 -- 172.21.15.27:0/2541872132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8589840 msgr2=0x55d3c895d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fccf7400640 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8589840 0x55d3c895d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fccf7400640 1 -- 172.21.15.27:0/2541872132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 msgr2=0x55d3c8968db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fccf7400640 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c8968db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fccf7400640 1 -- 172.21.15.27:0/2541872132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c8615b00 con 0x55d3c8581660 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fcceea00640 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8589840 0x55d3c895d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:25.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fccf7e00640 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c8968db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fccf7400640 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c895cf30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55d3c894ffc0 tx=0x7fcce802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42993237a4fc3574 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.084+0000 7fcce5600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.085+0000 7fcce5600640 1 -- 172.21.15.27:0/2541872132 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce804c070 con 0x55d3c8581660 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.085+0000 7fcce5600640 1 -- 172.21.15.27:0/2541872132 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fcce80314a0 con 0x55d3c8581660 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.085+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8004030 con 0x55d3c8581660 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.085+0000 7fcce5600640 1 -- 172.21.15.27:0/2541872132 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcce8058050 con 0x55d3c8581660 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.085+0000 7fcd051ca740 10 client.27350 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:25.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.085+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3c868c960 con 0x55d3c8581660 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcce5600640 1 -- 172.21.15.27:0/2541872132 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8004030 con 0x55d3c8581660 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcce5600640 1 -- 172.21.15.27:0/2541872132 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcce8004a20 con 0x55d3c8581660 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcd051ca740 10 client.27350 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcd051ca740 10 client.27350 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcd051ca740 20 client.27350 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcd051ca740 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c8568ea0 0x55d3c896ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcd051ca740 4 client.27350 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:25.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3c88ca1e0 con 0x55d3c8568ea0 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.086+0000 7fcceea00640 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c8568ea0 0x55d3c896ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.087+0000 7fcceea00640 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c8568ea0 0x55d3c896ecb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f7b4f3113674a7 server_cookie=bb9d1d272158718e in_seq=0 out_seq=0 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.087+0000 7fcce5600640 10 client.27350 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcce5600640 1 -- 172.21.15.27:0/2541872132 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3c88ca1e0 con 0x55d3c8568ea0 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcce5600640 10 client.27350 handle_command_reply: tid=0 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 client.27350 shutdown 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 msgr2=0x55d3c895cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c895cf30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55d3c894ffc0 tx=0x7fcce802a410 comp rx=0 tx=0).stop 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 shutdown_connections 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c8568ea0 0x55d3c896ecb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8589840 0x55d3c895d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c8968db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcce5600640 0 client.27350 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:25.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 --2- 172.21.15.27:0/2541872132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8581660 0x55d3c895cf30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 >> 172.21.15.27:0/2541872132 conn(0x55d3c859e910 msgr2=0x55d3c8954520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:25.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.088+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 shutdown_connections 2024-09-21T07:19:25.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.089+0000 7fcd051ca740 1 -- 172.21.15.27:0/2541872132 wait complete. 2024-09-21T07:19:25.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.089+0000 7fcd051ca740 20 client.27350 trim_cache size 0 max 16384 2024-09-21T07:19:25.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.107+0000 7fcd051ca740 1 Processor -- start 2024-09-21T07:19:25.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.107+0000 7fcd051ca740 1 -- start start 2024-09-21T07:19:25.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.107+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8581660 0x55d3c8952d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.107+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c895bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.107+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8968d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.107+0000 7fcd051ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b58e0 con 0x55d3c8581660 2024-09-21T07:19:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fcd051ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b31b0 con 0x55d3c8589840 2024-09-21T07:19:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fcd051ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b1020 con 0x55d3c85aa520 2024-09-21T07:19:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8581660 0x55d3c8952d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8581660 0x55d3c8952d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44314/0 (socket says 172.21.15.27:44314) 2024-09-21T07:19:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 -- 172.21.15.27:0/1913944973 learned_addr learned my addr 172.21.15.27:0/1913944973 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fcceea00640 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c895bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7e00640 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8968d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 -- 172.21.15.27:0/1913944973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 msgr2=0x55d3c8968d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8968d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 -- 172.21.15.27:0/1913944973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 msgr2=0x55d3c895bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c895bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7400640 1 -- 172.21.15.27:0/1913944973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c894fb20 con 0x55d3c8581660 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fccf7e00640 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8968d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.108+0000 7fcceea00640 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c895bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.109+0000 7fccf7400640 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8581660 0x55d3c8952d30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000c00 tx=0x7fcce8032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826247e77b3690b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.109+0000 7fcce7e00640 1 -- 172.21.15.27:0/1913944973 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8055070 con 0x55d3c8581660 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.109+0000 7fcce7e00640 1 -- 172.21.15.27:0/1913944973 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8005150 con 0x55d3c8581660 2024-09-21T07:19:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.109+0000 7fcce7e00640 1 -- 172.21.15.27:0/1913944973 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce804c040 con 0x55d3c8581660 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 -- 172.21.15.27:0/1913944973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8581660 msgr2=0x55d3c8952d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8581660 0x55d3c8952d30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000c00 tx=0x7fcce8032fc0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 -- 172.21.15.27:0/1913944973 shutdown_connections 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c85aa520 0x55d3c8968d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c895bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 --2- 172.21.15.27:0/1913944973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c8581660 0x55d3c8952d30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 -- 172.21.15.27:0/1913944973 >> 172.21.15.27:0/1913944973 conn(0x55d3c859e910 msgr2=0x55d3c895b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 -- 172.21.15.27:0/1913944973 shutdown_connections 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 -- 172.21.15.27:0/1913944973 wait complete. 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.110+0000 7fcd051ca740 1 -- start start 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 1 Processor -- start 2024-09-21T07:19:25.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8581660 0x55d3c89700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c8970620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c89d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b1670 con 0x55d3c85aa520 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b2ae0 con 0x55d3c8589840 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b1d40 con 0x55d3c8581660 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fcd051ca740 10 client.0 _ll_register_callbacks cb 0x7fcce00eb7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fccf7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8581660 0x55d3c89700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.111+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c89d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c89d8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44330/0 (socket says 172.21.15.27:44330) 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7e00640 1 -- 172.21.15.27:0/3121576311 learned_addr learned my addr 172.21.15.27:0/3121576311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fcceea00640 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c8970620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7400640 1 -- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 msgr2=0x55d3c8970620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7400640 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c8970620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7400640 1 -- 172.21.15.27:0/3121576311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 msgr2=0x55d3c89d8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:25.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7400640 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c89d8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7400640 1 -- 172.21.15.27:0/3121576311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c89d8970 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7e00640 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c89d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fccf7400640 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8581660 0x55d3c89700c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009900 tx=0x7fcce8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce14c8c3b4c176e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.112+0000 7fcce5600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.113+0000 7fcce5600640 1 -- 172.21.15.27:0/3121576311 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8055030 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.113+0000 7fcce5600640 1 -- 172.21.15.27:0/3121576311 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fcce80048d0 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.113+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8005150 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.113+0000 7fcce5600640 1 -- 172.21.15.27:0/3121576311 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcce806d050 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.113+0000 7fcd051ca740 10 client.27328 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.113+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3c89d9a50 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcce5600640 1 -- 172.21.15.27:0/3121576311 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8005150 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcce5600640 1 -- 172.21.15.27:0/3121576311 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcce805d7a0 con 0x55d3c8581660 2024-09-21T07:19:25.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcd051ca740 10 client.27328 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcd051ca740 10 client.27328 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcd051ca740 20 client.27328 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcd051ca740 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c853e250 0x55d3c89dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcd051ca740 4 client.27328 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "db39a904", "release": true, "await": true}] 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "db39a904", "release": true, "await": true}) -- 0x55d3c88c5c90 con 0x55d3c853e250 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.114+0000 7fcceea00640 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c853e250 0x55d3c89dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.115+0000 7fcceea00640 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c853e250 0x55d3c89dc150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3174d06b1295c45 server_cookie=cfa281b3ad5bbddb in_seq=0 out_seq=0 2024-09-21T07:19:25.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.115+0000 7fcce5600640 10 client.27328 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcce5600640 1 -- 172.21.15.27:0/3121576311 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d3c88c5c90 con 0x55d3c853e250 2024-09-21T07:19:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcce5600640 10 client.27328 handle_command_reply: tid=0 2024-09-21T07:19:25.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 client.27328 shutdown 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8581660 msgr2=0x55d3c89700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8581660 0x55d3c89700c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009900 tx=0x7fcce8004600 comp rx=0 tx=0).stop 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 shutdown_connections 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d3c853e250 0x55d3c89dc150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3c85aa520 0x55d3c89d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3c8589840 0x55d3c8970620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 --2- 172.21.15.27:0/3121576311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3c8581660 0x55d3c89700c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcce5600640 0 client.27328 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 >> 172.21.15.27:0/3121576311 conn(0x55d3c859e910 msgr2=0x55d3c895bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 shutdown_connections 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 1 -- 172.21.15.27:0/3121576311 wait complete. 2024-09-21T07:19:25.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.119+0000 7fcd051ca740 20 client.27328 trim_cache size 0 max 16384 2024-09-21T07:19:25.123 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:25.123 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:19:25.123 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:19:25.123 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 28, 2024-09-21T07:19:25.123 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:19:25.123 INFO:teuthology.orchestra.run.smithi027.stdout: "db39a904": { 2024-09-21T07:19:25.123 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 28, 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 139.2, 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:19:25.124 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fccdc0b6680 msgr2=0x7fccdc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fccdc0b6680 0x7fccdc0b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fccf0005fd0 tx=0x7fccf0009500 comp rx=0 tx=0).stop 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81092d0 msgr2=0x7fccf8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81092d0 0x7fccf8075c50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcce000e970 tx=0x7fcce000ee30 comp rx=0 tx=0).stop 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 shutdown_connections 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fccdc0b6680 0x7fccdc0b8b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8109c20 0x7fccf8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81092d0 0x7fccf8075c50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.120+0000 7fccfd000640 1 --2- 172.21.15.27:0/4254998916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccf8103990 0x7fccf8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.121+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 >> 172.21.15.27:0/4254998916 conn(0x7fccf80ff440 msgr2=0x7fccf8108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.121+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 shutdown_connections 2024-09-21T07:19:25.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:25.121+0000 7fccfd000640 1 -- 172.21.15.27:0/4254998916 wait complete. 2024-09-21T07:19:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:24 smithi153 ceph-mon[34747]: pgmap v571: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-21T07:19:26.035 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'db39a904', total seconds elapsed: 23.5 2024-09-21T07:19:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:26 smithi119 ceph-mon[34519]: pgmap v572: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T07:19:27.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:26 smithi027 ceph-mon[30779]: pgmap v572: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T07:19:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:26 smithi153 ceph-mon[34747]: pgmap v572: 129 pgs: 129 active+clean; 3.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T07:19:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:28 smithi119 ceph-mon[34519]: pgmap v573: 129 pgs: 129 active+clean; 3.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-21T07:19:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:28 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:19:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:28 smithi027 ceph-mon[30779]: pgmap v573: 129 pgs: 129 active+clean; 3.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-21T07:19:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:28 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:19:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:28 smithi153 ceph-mon[34747]: pgmap v573: 129 pgs: 129 active+clean; 3.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-21T07:19:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:28 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:19:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:30 smithi119 ceph-mon[34519]: pgmap v574: 129 pgs: 129 active+clean; 3.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-21T07:19:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:30 smithi027 ceph-mon[30779]: pgmap v574: 129 pgs: 129 active+clean; 3.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-21T07:19:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:30 smithi153 ceph-mon[34747]: pgmap v574: 129 pgs: 129 active+clean; 3.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-21T07:19:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:32 smithi119 ceph-mon[34519]: pgmap v575: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-21T07:19:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:32 smithi027 ceph-mon[30779]: pgmap v575: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-21T07:19:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:32 smithi153 ceph-mon[34747]: pgmap v575: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-21T07:19:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:34 smithi119 ceph-mon[34519]: pgmap v576: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-09-21T07:19:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:34 smithi027 ceph-mon[30779]: pgmap v576: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-09-21T07:19:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:34 smithi153 ceph-mon[34747]: pgmap v576: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-09-21T07:19:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:127/600: open fd = 3 2024-09-21T07:19:35.215 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:128/600: open fd = 3 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:129/600: open fd = 3 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:130/600: open fd = 3 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.216 INFO:tasks.workunit.client.0.smithi027.stdout:131/600: open fd = 3 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:132/600: open fd = 3 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:133/600: open fd = 3 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:134/600: open fd = 3 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.217 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:135/600: open fd = 3 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:136/600: open fd = 3 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:137/600: open fd = 3 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:138/600: open fd = 3 2024-09-21T07:19:35.218 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:139/600: open fd = 3 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:140/600: open fd = 3 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:141/600: open fd = 3 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.219 INFO:tasks.workunit.client.0.smithi027.stdout:142/600: open fd = 3 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:143/600: open fd = 3 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:144/600: open fd = 3 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:145/600: open fd = 3 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.220 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:146/600: open fd = 3 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:147/600: open fd = 3 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:148/600: open fd = 3 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:149/600: open fd = 3 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.221 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:150/600: open fd = 3 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:151/600: open fd = 3 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:152/600: open fd = 3 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.222 INFO:tasks.workunit.client.0.smithi027.stdout:153/600: open fd = 3 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:154/600: open fd = 3 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:155/600: open fd = 3 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:156/600: open fd = 3 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.223 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:157/600: open fd = 3 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:158/600: open fd = 3 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:159/600: open fd = 3 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:160/600: open fd = 3 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:161/600: open fd = 3 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:162/600: open fd = 3 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:163/600: open fd = 3 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:164/600: open fd = 3 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:165/600: open fd = 3 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:166/600: open fd = 3 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:167/600: open fd = 3 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:168/600: open fd = 3 2024-09-21T07:19:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:169/600: open fd = 3 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:170/600: open fd = 3 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:171/600: open fd = 3 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:172/600: open fd = 3 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:173/600: open fd = 3 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:174/600: open fd = 3 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:175/600: open fd = 3 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:176/600: open fd = 3 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:177/600: open fd = 3 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:178/600: open fd = 3 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:179/600: open fd = 3 2024-09-21T07:19:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:180/600: open fd = 3 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:181/600: open fd = 3 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:182/600: open fd = 3 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.231 INFO:tasks.workunit.client.0.smithi027.stdout:183/600: open fd = 3 2024-09-21T07:19:35.231 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.231 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:184/600: open fd = 3 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:185/600: open fd = 3 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:186/600: open fd = 3 2024-09-21T07:19:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:187/600: open fd = 3 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:188/600: open fd = 3 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:19:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 -- 172.21.15.27:0/4053274054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010bf70 msgr2=0x7f18901127a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 --2- 172.21.15.27:0/4053274054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010bf70 0x7f18901127a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1888009ca0 tx=0x7f188802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 -- 172.21.15.27:0/4053274054 shutdown_connections 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 --2- 172.21.15.27:0/4053274054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f189010b620 0x7f189010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 --2- 172.21.15.27:0/4053274054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010bf70 0x7f18901127a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 --2- 172.21.15.27:0/4053274054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1890105cf0 0x7f1890106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 -- 172.21.15.27:0/4053274054 >> 172.21.15.27:0/4053274054 conn(0x7f18901016f0 msgr2=0x7f1890103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 -- 172.21.15.27:0/4053274054 shutdown_connections 2024-09-21T07:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.162+0000 7f1898200640 1 -- 172.21.15.27:0/4053274054 wait complete. 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.163+0000 7f1898200640 1 Processor -- start 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1898200640 1 -- start start 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1898200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1890105cf0 0x7f189019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1898200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010b620 0x7f189019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1898200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f189010bf70 0x7f18901a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1898200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1890115c30 con 0x7f189010b620 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1898200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1890115ab0 con 0x7f189010bf70 2024-09-21T07:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1898200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1890115db0 con 0x7f1890105cf0 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1894e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010b620 0x7f189019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1895800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1890105cf0 0x7f189019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1895800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1890105cf0 0x7f189019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57412/0 (socket says 172.21.15.27:57412) 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1895800640 1 -- 172.21.15.27:0/2646762967 learned_addr learned my addr 172.21.15.27:0/2646762967 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.164+0000 7f1896200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f189010bf70 0x7f18901a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f1895800640 1 -- 172.21.15.27:0/2646762967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f189010bf70 msgr2=0x7f18901a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f1895800640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f189010bf70 0x7f18901a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f1895800640 1 -- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010b620 msgr2=0x7f189019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f1895800640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010b620 0x7f189019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f1895800640 1 -- 172.21.15.27:0/2646762967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18901a2540 con 0x7f1890105cf0 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f1894e00640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010b620 0x7f189019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f1895800640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1890105cf0 0x7f189019c610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f188400ea60 tx=0x7f188400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.165+0000 7f188e000640 1 -- 172.21.15.27:0/2646762967 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188400cde0 con 0x7f1890105cf0 2024-09-21T07:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.166+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18901a2830 con 0x7f1890105cf0 2024-09-21T07:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.166+0000 7f188e000640 1 -- 172.21.15.27:0/2646762967 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18840035f0 con 0x7f1890105cf0 2024-09-21T07:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.166+0000 7f188e000640 1 -- 172.21.15.27:0/2646762967 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884010690 con 0x7f1890105cf0 2024-09-21T07:19:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.166+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18901a8fc0 con 0x7f1890105cf0 2024-09-21T07:19:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 Processor -- start 2024-09-21T07:19:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 -- start start 2024-09-21T07:19:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc49b1b0 0x559dfc83fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc4722f0 0x559dfc840170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc47a4d0 0x559dfc8406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559dfc7b56b0 con 0x559dfc49b1b0 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559dfc7a4ae0 con 0x559dfc47a4d0 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f189f272740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559dfc7a88f0 con 0x559dfc4722f0 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f1896200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc49b1b0 0x559dfc83fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f1896200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc49b1b0 0x559dfc83fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51682/0 (socket says 172.21.15.27:51682) 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f1896200640 1 -- 172.21.15.27:0/1509936797 learned_addr learned my addr 172.21.15.27:0/1509936797 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.169+0000 7f1894e00640 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc47a4d0 0x559dfc8406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f1896200640 1 -- 172.21.15.27:0/1509936797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc4722f0 msgr2=0x559dfc840170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f1896200640 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc4722f0 0x559dfc840170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f1896200640 1 -- 172.21.15.27:0/1509936797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc47a4d0 msgr2=0x559dfc8406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f1896200640 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc47a4d0 0x559dfc8406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f1896200640 1 -- 172.21.15.27:0/1509936797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfc771780 con 0x559dfc49b1b0 2024-09-21T07:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f188e000640 1 -- 172.21.15.27:0/2646762967 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f18840037a0 con 0x7f1890105cf0 2024-09-21T07:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f1896200640 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc49b1b0 0x559dfc83fc30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f188802f410 tx=0x7f188802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f74a84249d4133c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.170+0000 7f1882a00640 1 -- 172.21.15.27:0/1509936797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888031cf0 con 0x559dfc49b1b0 2024-09-21T07:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.171+0000 7f1882a00640 1 -- 172.21.15.27:0/1509936797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1888036030 con 0x559dfc49b1b0 2024-09-21T07:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.171+0000 7f1882a00640 1 -- 172.21.15.27:0/1509936797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18880042a0 con 0x559dfc49b1b0 2024-09-21T07:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 -- 172.21.15.27:0/1509936797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc49b1b0 msgr2=0x559dfc83fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc49b1b0 0x559dfc83fc30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f188802f410 tx=0x7f188802f440 comp rx=0 tx=0).stop 2024-09-21T07:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 -- 172.21.15.27:0/1509936797 shutdown_connections 2024-09-21T07:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc4722f0 0x559dfc840170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc47a4d0 0x559dfc8406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 --2- 172.21.15.27:0/1509936797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc49b1b0 0x559dfc83fc30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 -- 172.21.15.27:0/1509936797 >> 172.21.15.27:0/1509936797 conn(0x559dfc48f300 msgr2=0x559dfc83db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 -- 172.21.15.27:0/1509936797 shutdown_connections 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 -- 172.21.15.27:0/1509936797 wait complete. 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f188e000640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f18780b6590 0x7f18780b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f189f272740 1 -- start start 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.172+0000 7f188e000640 1 -- 172.21.15.27:0/2646762967 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f18840e7dc0 con 0x7f1890105cf0 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f1896200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f18780b6590 0x7f18780b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f189f272740 1 Processor -- start 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f189f272740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc4722f0 0x559dfc84b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f189f272740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc47a4d0 0x559dfc84b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f189f272740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc49b1b0 0x559dfc84bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f1896200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f18780b6590 0x7f18780b8aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1888005e30 tx=0x7f188802a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f189f272740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559dfc6dcab0 con 0x559dfc4722f0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f189f272740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559dfc74b570 con 0x559dfc49b1b0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.173+0000 7f189f272740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559dfc84e1d0 con 0x559dfc47a4d0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f189f272740 10 client.0 _ll_register_callbacks cb 0x559dfc75a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc4722f0 0x559dfc84b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc4722f0 0x559dfc84b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51696/0 (socket says 172.21.15.27:51696) 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 -- 172.21.15.27:0/3130364479 learned_addr learned my addr 172.21.15.27:0/3130364479 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1896200640 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc49b1b0 0x559dfc84bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1894e00640 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc47a4d0 0x559dfc84b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 -- 172.21.15.27:0/3130364479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc47a4d0 msgr2=0x559dfc84b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc47a4d0 0x559dfc84b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 -- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc49b1b0 msgr2=0x559dfc84bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc49b1b0 0x559dfc84bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 -- 172.21.15.27:0/3130364479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfc506790 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1895800640 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc4722f0 0x559dfc84b260 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f188400d010 tx=0x7f18840ad630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1a8baba126c6be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.174+0000 7f1877400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f1877400640 1 -- 172.21.15.27:0/3130364479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840acfd0 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f1877400640 1 -- 172.21.15.27:0/3130364479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f18840aed90 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1888036030 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f1877400640 1 -- 172.21.15.27:0/3130364479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18840ad370 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f189f272740 10 client.17337 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559dfc57f620 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f1877400640 1 -- 172.21.15.27:0/3130364479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1888036030 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f1877400640 1 -- 172.21.15.27:0/3130364479 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18840a91b0 con 0x559dfc4722f0 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f189f272740 10 client.17337 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.175+0000 7f189f272740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.176+0000 7f189f272740 20 client.17337 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.176+0000 7f189f272740 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc459d80 0x559dfc85d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.176+0000 7f189f272740 4 client.17337 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.176+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dfc7b7600 con 0x559dfc459d80 2024-09-21T07:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.176+0000 7f1894e00640 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc459d80 0x559dfc85d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.176+0000 7f1894e00640 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc459d80 0x559dfc85d200 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a08480b5b42cc6ce server_cookie=b7af3f62461a8394 in_seq=0 out_seq=0 2024-09-21T07:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.176+0000 7f1877400640 10 client.17337 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.177+0000 7f1877400640 1 -- 172.21.15.27:0/3130364479 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559dfc7b7600 con 0x559dfc459d80 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.177+0000 7f1877400640 10 client.17337 handle_command_reply: tid=0 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.177+0000 7f189f272740 1 client.17337 shutdown 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc4722f0 msgr2=0x559dfc84b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc4722f0 0x559dfc84b260 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f188400d010 tx=0x7f18840ad630 comp rx=0 tx=0).stop 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 shutdown_connections 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc459d80 0x559dfc85d200 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc47a4d0 0x559dfc84b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc49b1b0 0x559dfc84bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 --2- 172.21.15.27:0/3130364479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc4722f0 0x559dfc84b260 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 >> 172.21.15.27:0/3130364479 conn(0x559dfc48f300 msgr2=0x559dfc840c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f1877400640 0 client.17337 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 shutdown_connections 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 1 -- 172.21.15.27:0/3130364479 wait complete. 2024-09-21T07:19:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.178+0000 7f189f272740 20 client.17337 trim_cache size 0 max 16384 2024-09-21T07:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 Processor -- start 2024-09-21T07:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 -- start start 2024-09-21T07:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc83caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc84ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc862970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18901b5820 con 0x559dfc47a4d0 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18901b30f0 con 0x559dfc4722f0 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f189f272740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18901b0f60 con 0x559dfc49b1b0 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f1894e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc84ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f1895800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc83caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f1895800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc83caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35670/0 (socket says 172.21.15.27:35670) 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f1895800640 1 -- 172.21.15.27:0/2500117249 learned_addr learned my addr 172.21.15.27:0/2500117249 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.194+0000 7f1896200640 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc862970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1895800640 1 -- 172.21.15.27:0/2500117249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 msgr2=0x559dfc862970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1895800640 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc862970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1895800640 1 -- 172.21.15.27:0/2500117249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 msgr2=0x559dfc84ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1895800640 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc84ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1895800640 1 -- 172.21.15.27:0/2500117249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfc84baf0 con 0x559dfc4722f0 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1895800640 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc83caa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f18840e5210 tx=0x7f1884003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17dedb99f0f56f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1882000640 1 -- 172.21.15.27:0/2500117249 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840acfd0 con 0x559dfc4722f0 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1882000640 1 -- 172.21.15.27:0/2500117249 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18840adf60 con 0x559dfc4722f0 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.195+0000 7f1882000640 1 -- 172.21.15.27:0/2500117249 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840aedf0 con 0x559dfc4722f0 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 -- 172.21.15.27:0/2500117249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 msgr2=0x559dfc83caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc83caa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f18840e5210 tx=0x7f1884003e00 comp rx=0 tx=0).stop 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 -- 172.21.15.27:0/2500117249 shutdown_connections 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc862970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc84ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 --2- 172.21.15.27:0/2500117249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc83caa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 -- 172.21.15.27:0/2500117249 >> 172.21.15.27:0/2500117249 conn(0x559dfc48f300 msgr2=0x559dfc84c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 -- 172.21.15.27:0/2500117249 shutdown_connections 2024-09-21T07:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.196+0000 7f189f272740 1 -- 172.21.15.27:0/2500117249 wait complete. 2024-09-21T07:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 -- start start 2024-09-21T07:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 Processor -- start 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc853c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc8510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc851610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18901b15b0 con 0x559dfc47a4d0 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18901b2a20 con 0x559dfc4722f0 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18901b1c80 con 0x559dfc49b1b0 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.197+0000 7f189f272740 10 client.0 _ll_register_callbacks cb 0x7f18840a9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1896200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc851610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1894e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc8510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1895800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc853c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1896200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc851610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57440/0 (socket says 172.21.15.27:57440) 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1896200640 1 -- 172.21.15.27:0/2505614907 learned_addr learned my addr 172.21.15.27:0/2505614907 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1895800640 1 -- 172.21.15.27:0/2505614907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 msgr2=0x559dfc851610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1895800640 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc851610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1895800640 1 -- 172.21.15.27:0/2505614907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 msgr2=0x559dfc8510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1895800640 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc8510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1895800640 1 -- 172.21.15.27:0/2505614907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfc851da0 con 0x559dfc4722f0 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1894e00640 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc8510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1895800640 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc853c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18840e5210 tx=0x7f18840ad630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856f79c81d8ce03a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.198+0000 7f1877400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f1877400640 1 -- 172.21.15.27:0/2505614907 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840acfd0 con 0x559dfc4722f0 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f1877400640 1 -- 172.21.15.27:0/2505614907 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f18840aed90 con 0x559dfc4722f0 2024-09-21T07:19:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18840adf60 con 0x559dfc4722f0 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f1877400640 1 -- 172.21.15.27:0/2505614907 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18840b2340 con 0x559dfc4722f0 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f189f272740 10 client.27374 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559dfc8c7ae0 con 0x559dfc4722f0 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f1877400640 1 -- 172.21.15.27:0/2505614907 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18840adf60 con 0x559dfc4722f0 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f1877400640 1 -- 172.21.15.27:0/2505614907 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18840b3660 con 0x559dfc4722f0 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f189f272740 10 client.27374 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f189f272740 10 client.27374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f189f272740 20 client.27374 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.199+0000 7f189f272740 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc42f1c0 0x559dfc8c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.200+0000 7f189f272740 4 client.27374 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.200+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dfc7b30b0 con 0x559dfc42f1c0 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.200+0000 7f1894e00640 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc42f1c0 0x559dfc8c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.200+0000 7f1894e00640 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc42f1c0 0x559dfc8c9f80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1aa1890c5e25ce server_cookie=bf3620027809598c in_seq=0 out_seq=0 2024-09-21T07:19:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.200+0000 7f1877400640 10 client.27374 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.211+0000 7f1877400640 1 -- 172.21.15.27:0/2505614907 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559dfc7b30b0 con 0x559dfc42f1c0 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.211+0000 7f1877400640 10 client.27374 handle_command_reply: tid=0 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.211+0000 7f189f272740 1 client.27374 shutdown 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.211+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 msgr2=0x559dfc853c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.211+0000 7f189f272740 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc853c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18840e5210 tx=0x7f18840ad630 comp rx=0 tx=0).stop 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.211+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 shutdown_connections 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559dfc42f1c0 0x559dfc8c9f80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559dfc49b1b0 0x559dfc851610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559dfc47a4d0 0x559dfc8510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 1 --2- 172.21.15.27:0/2505614907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559dfc4722f0 0x559dfc853c10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 >> 172.21.15.27:0/2505614907 conn(0x559dfc48f300 msgr2=0x559dfc83c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f1877400640 0 client.27374 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 shutdown_connections 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 1 -- 172.21.15.27:0/2505614907 wait complete. 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f189f272740 20 client.27374 trim_cache size 0 max 16384 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f18780b6590 msgr2=0x7f18780b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f1898200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f18780b6590 0x7f18780b8aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1888005e30 tx=0x7f188802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1890105cf0 msgr2=0x7f189019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.212+0000 7f1898200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1890105cf0 0x7f189019c610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f188400ea60 tx=0x7f188400ef20 comp rx=0 tx=0).stop 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 shutdown_connections 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f18780b6590 0x7f18780b8aa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f189010bf70 0x7f18901a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189010b620 0x7f189019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 --2- 172.21.15.27:0/2646762967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1890105cf0 0x7f189019c610 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 >> 172.21.15.27:0/2646762967 conn(0x7f18901016f0 msgr2=0x7f189010ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 shutdown_connections 2024-09-21T07:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:36.213+0000 7f1898200640 1 -- 172.21.15.27:0/2646762967 wait complete. 2024-09-21T07:19:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:36 smithi119 ceph-mon[34519]: pgmap v577: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-21T07:19:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:36 smithi027 ceph-mon[30779]: pgmap v577: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-21T07:19:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:36 smithi153 ceph-mon[34747]: pgmap v577: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-21T07:19:38.979 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:38 smithi153 ceph-mon[34747]: pgmap v578: 129 pgs: 129 active+clean; 3.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-21T07:19:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:38 smithi119 ceph-mon[34519]: pgmap v578: 129 pgs: 129 active+clean; 3.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-21T07:19:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:38 smithi027 ceph-mon[30779]: pgmap v578: 129 pgs: 129 active+clean; 3.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-21T07:19:40.657 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:19:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:40 smithi119 ceph-mon[34519]: pgmap v579: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:19:41.058 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:41.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:40 smithi027 ceph-mon[30779]: pgmap v579: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:19:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:40 smithi153 ceph-mon[34747]: pgmap v579: 129 pgs: 129 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:19:41.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 -- 172.21.15.27:0/3726658795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810c010 msgr2=0x7f2438112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 --2- 172.21.15.27:0/3726658795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810c010 0x7f2438112840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2430009ca0 tx=0x7f243002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 -- 172.21.15.27:0/3726658795 shutdown_connections 2024-09-21T07:19:41.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 --2- 172.21.15.27:0/3726658795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f243810b6c0 0x7f243810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 --2- 172.21.15.27:0/3726658795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810c010 0x7f2438112840 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 --2- 172.21.15.27:0/3726658795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2438105d80 0x7f2438106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 -- 172.21.15.27:0/3726658795 >> 172.21.15.27:0/3726658795 conn(0x7f2438101810 msgr2=0x7f2438103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:41.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 -- 172.21.15.27:0/3726658795 shutdown_connections 2024-09-21T07:19:41.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.691+0000 7f243f200640 1 -- 172.21.15.27:0/3726658795 wait complete. 2024-09-21T07:19:41.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 Processor -- start 2024-09-21T07:19:41.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 -- start start 2024-09-21T07:19:41.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2438105d80 0x7f243819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810b6c0 0x7f243819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243810c010 0x7f24381a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2438115d20 con 0x7f243810b6c0 2024-09-21T07:19:41.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2438115ba0 con 0x7f243810c010 2024-09-21T07:19:41.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.692+0000 7f243f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2438115ea0 con 0x7f2438105d80 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810b6c0 0x7f243819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2438105d80 0x7f243819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810b6c0 0x7f243819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56262/0 (socket says 172.21.15.27:56262) 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 -- 172.21.15.27:0/3043508587 learned_addr learned my addr 172.21.15.27:0/3043508587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f243d200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243810c010 0x7f24381a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 -- 172.21.15.27:0/3043508587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2438105d80 msgr2=0x7f243819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2438105d80 0x7f243819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 -- 172.21.15.27:0/3043508587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243810c010 msgr2=0x7f24381a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243810c010 0x7f24381a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f2437400640 1 -- 172.21.15.27:0/3043508587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24381a2680 con 0x7f243810b6c0 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.693+0000 7f243d200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243810c010 0x7f24381a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.694+0000 7f2437400640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810b6c0 0x7f243819ccb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f242400d740 tx=0x7f242400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.694+0000 7f2434c00640 1 -- 172.21.15.27:0/3043508587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424017070 con 0x7f243810b6c0 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.694+0000 7f2434c00640 1 -- 172.21.15.27:0/3043508587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242400de50 con 0x7f243810b6c0 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.694+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24381a2970 con 0x7f243810b6c0 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.694+0000 7f2434c00640 1 -- 172.21.15.27:0/3043508587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242400f620 con 0x7f243810b6c0 2024-09-21T07:19:41.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.694+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381a90d0 con 0x7f243810b6c0 2024-09-21T07:19:41.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 Processor -- start 2024-09-21T07:19:41.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 -- start start 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d9287fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d928800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d928805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2434c00640 1 -- 172.21.15.27:0/3043508587 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f242401a070 con 0x7f243810b6c0 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558d927f6f70 con 0x558d924d9070 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558d927e63a0 con 0x558d924b01b0 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.698+0000 7f2445e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d927ea1b0 con 0x558d924b8390 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d9287fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d9287fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56270/0 (socket says 172.21.15.27:56270) 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 -- 172.21.15.27:0/626939984 learned_addr learned my addr 172.21.15.27:0/626939984 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f2437e00640 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d928800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f2437400640 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d928805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 -- 172.21.15.27:0/626939984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 msgr2=0x558d928805e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d928805e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 -- 172.21.15.27:0/626939984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 msgr2=0x558d928800a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d928800a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f243d200640 1 -- 172.21.15.27:0/626939984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d927b0c90 con 0x558d924d9070 2024-09-21T07:19:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.699+0000 7f2437e00640 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d928800a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:41.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.700+0000 7f243d200640 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d9287fb60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2430009870 tx=0x7f24300098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf5c3730c121e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.700+0000 7f242d600640 1 -- 172.21.15.27:0/626939984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430031d70 con 0x558d924d9070 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.700+0000 7f242d600640 1 -- 172.21.15.27:0/626939984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2430036040 con 0x558d924d9070 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.700+0000 7f242d600640 1 -- 172.21.15.27:0/626939984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430004330 con 0x558d924d9070 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2434c00640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f241c0b6680 0x7f241c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 -- 172.21.15.27:0/626939984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 msgr2=0x558d9287fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d9287fb60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2430009870 tx=0x7f24300098a0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2434c00640 1 -- 172.21.15.27:0/3043508587 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f24240e74b0 con 0x7f243810b6c0 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 -- 172.21.15.27:0/626939984 shutdown_connections 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d928805e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d928800a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 --2- 172.21.15.27:0/626939984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d9287fb60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 -- 172.21.15.27:0/626939984 >> 172.21.15.27:0/626939984 conn(0x558d924cd460 msgr2=0x558d9287da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:41.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2437e00640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f241c0b6680 0x7f241c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 -- 172.21.15.27:0/626939984 shutdown_connections 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.701+0000 7f2445e60740 1 -- 172.21.15.27:0/626939984 wait complete. 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.702+0000 7f2445e60740 1 -- start start 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.702+0000 7f2437e00640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f241c0b6680 0x7f241c0b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x558d9287e7e0 tx=0x7f2428005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.702+0000 7f2445e60740 1 Processor -- start 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2445e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2445e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d9288bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2445e60740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d92897600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2445e60740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558d92647700 con 0x558d924d9070 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2445e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558d92781540 con 0x558d924b01b0 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2445e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d9288e2e0 con 0x558d924b8390 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2445e60740 10 client.0 _ll_register_callbacks cb 0x558d92791430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2437400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d9288bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2437400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d9288bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35440/0 (socket says 172.21.15.27:35440) 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2437400640 1 -- 172.21.15.27:0/2209319301 learned_addr learned my addr 172.21.15.27:0/2209319301 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f243d200640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d92897600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.703+0000 7f2437e00640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2437400640 1 -- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 msgr2=0x558d9288b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2437400640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2437400640 1 -- 172.21.15.27:0/2209319301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 msgr2=0x558d92897600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2437400640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d92897600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2437400640 1 -- 172.21.15.27:0/2209319301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d92544650 con 0x558d924b8390 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2437e00640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2437400640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d9288bcb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2424009610 tx=0x7f24240eaa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4934648efafc5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2422000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2422000640 1 -- 172.21.15.27:0/2209319301 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424021070 con 0x558d924b8390 2024-09-21T07:19:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2422000640 1 -- 172.21.15.27:0/2209319301 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f24240942d0 con 0x558d924b8390 2024-09-21T07:19:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d92783b50 con 0x558d924b8390 2024-09-21T07:19:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.704+0000 7f2422000640 1 -- 172.21.15.27:0/2209319301 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f24240a9540 con 0x558d924b8390 2024-09-21T07:19:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 10 client.27355 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d9279e910 con 0x558d924b8390 2024-09-21T07:19:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2422000640 1 -- 172.21.15.27:0/2209319301 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558d92783b50 con 0x558d924b8390 2024-09-21T07:19:41.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2422000640 1 -- 172.21.15.27:0/2209319301 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f24240b1020 con 0x558d924b8390 2024-09-21T07:19:41.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 10 client.27355 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 10 client.27355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 20 client.27355 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d924979f0 0x558d9289d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 4 client.27355 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.705+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d927f8ec0 con 0x558d924979f0 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.706+0000 7f2437e00640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d924979f0 0x558d9289d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.706+0000 7f2437e00640 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d924979f0 0x558d9289d600 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6f287c05da8b34 server_cookie=1785fd1e5d422019 in_seq=0 out_seq=0 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.706+0000 7f2422000640 10 client.27355 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.707+0000 7f2422000640 1 -- 172.21.15.27:0/2209319301 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d927f8ec0 con 0x558d924979f0 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.707+0000 7f2422000640 10 client.27355 handle_command_reply: tid=0 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.707+0000 7f2445e60740 1 client.27355 shutdown 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 msgr2=0x558d9288bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d9288bcb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2424009610 tx=0x7f24240eaa90 comp rx=0 tx=0).stop 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 shutdown_connections 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d924979f0 0x558d9289d600 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d9288bcb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d92897600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 --2- 172.21.15.27:0/2209319301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2422000640 0 client.27355 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 >> 172.21.15.27:0/2209319301 conn(0x558d924cd460 msgr2=0x558d92882d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 shutdown_connections 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 1 -- 172.21.15.27:0/2209319301 wait complete. 2024-09-21T07:19:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.708+0000 7f2445e60740 20 client.27355 trim_cache size 0 max 16384 2024-09-21T07:19:41.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.726+0000 7f2445e60740 1 Processor -- start 2024-09-21T07:19:41.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.726+0000 7f2445e60740 1 -- start start 2024-09-21T07:19:41.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2445e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2445e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d92884e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2445e60740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d928853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2445e60740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24381b5a60 con 0x558d924d9070 2024-09-21T07:19:41.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2445e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f24381b3330 con 0x558d924b01b0 2024-09-21T07:19:41.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2445e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24381b11a0 con 0x558d924b8390 2024-09-21T07:19:41.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f243d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d928853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2437e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f243d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d928853b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56288/0 (socket says 172.21.15.27:56288) 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f243d200640 1 -- 172.21.15.27:0/3667381044 learned_addr learned my addr 172.21.15.27:0/3667381044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.727+0000 7f2437400640 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d92884e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f2437e00640 1 -- 172.21.15.27:0/3667381044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 msgr2=0x558d92884e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f2437e00640 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d92884e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f2437e00640 1 -- 172.21.15.27:0/3667381044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 msgr2=0x558d928853b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f2437e00640 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d928853b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f2437e00640 1 -- 172.21.15.27:0/3667381044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9288a4d0 con 0x558d924b01b0 2024-09-21T07:19:41.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f2437e00640 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288d1a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x558d9289b560 tx=0x7f2428017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e141713799e06a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:41.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f242cc00640 1 -- 172.21.15.27:0/3667381044 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428003850 con 0x558d924b01b0 2024-09-21T07:19:41.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f242cc00640 1 -- 172.21.15.27:0/3667381044 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242802b430 con 0x558d924b01b0 2024-09-21T07:19:41.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.728+0000 7f242cc00640 1 -- 172.21.15.27:0/3667381044 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428003ea0 con 0x558d924b01b0 2024-09-21T07:19:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.729+0000 7f2445e60740 1 -- 172.21.15.27:0/3667381044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 msgr2=0x558d9288d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.729+0000 7f2445e60740 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288d1a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x558d9289b560 tx=0x7f2428017000 comp rx=0 tx=0).stop 2024-09-21T07:19:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.729+0000 7f2445e60740 1 -- 172.21.15.27:0/3667381044 shutdown_connections 2024-09-21T07:19:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.729+0000 7f2445e60740 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924b8390 0x558d92884e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.729+0000 7f2445e60740 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924d9070 0x558d928853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.729+0000 7f2445e60740 1 --2- 172.21.15.27:0/3667381044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d9288d1a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.729+0000 7f2445e60740 1 -- 172.21.15.27:0/3667381044 >> 172.21.15.27:0/3667381044 conn(0x558d924cd460 msgr2=0x558d9288ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.730+0000 7f2445e60740 1 -- 172.21.15.27:0/3667381044 shutdown_connections 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.730+0000 7f2445e60740 1 -- 172.21.15.27:0/3667381044 wait complete. 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.730+0000 7f2445e60740 1 -- start start 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.730+0000 7f2445e60740 1 Processor -- start 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2445e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d92893b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2445e60740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924b8390 0x558d9288e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2445e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924d9070 0x558d9288ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2445e60740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24381b17f0 con 0x558d924b8390 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2445e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f24381b2c60 con 0x558d924b01b0 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2445e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24381b1ec0 con 0x558d924d9070 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2445e60740 10 client.0 _ll_register_callbacks cb 0x7f2424009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2437400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924b8390 0x558d9288e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2437e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d92893b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2437e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d92893b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45922/0 (socket says 172.21.15.27:45922) 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f2437e00640 1 -- 172.21.15.27:0/2783811964 learned_addr learned my addr 172.21.15.27:0/2783811964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.731+0000 7f243d200640 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924d9070 0x558d9288ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2437e00640 1 -- 172.21.15.27:0/2783811964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924d9070 msgr2=0x558d9288ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2437e00640 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924d9070 0x558d9288ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2437e00640 1 -- 172.21.15.27:0/2783811964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924b8390 msgr2=0x558d9288e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2437e00640 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924b8390 0x558d9288e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2437e00640 1 -- 172.21.15.27:0/2783811964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d9288eff0 con 0x558d924b01b0 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2437400640 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924b8390 0x558d9288e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2437e00640 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d92893b70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f242802f040 tx=0x7f24280051a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6710fff3815c91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:41.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.732+0000 7f2422000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.734+0000 7f2422000640 1 -- 172.21.15.27:0/2783811964 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428005650 con 0x558d924b01b0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.734+0000 7f2422000640 1 -- 172.21.15.27:0/2783811964 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f242802b5e0 con 0x558d924b01b0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.734+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f242802b430 con 0x558d924b01b0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2422000640 1 -- 172.21.15.27:0/2783811964 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f242802b7e0 con 0x558d924b01b0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 10 client.27392 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d92908d50 con 0x558d924b01b0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2422000640 1 -- 172.21.15.27:0/2783811964 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242802b430 con 0x558d924b01b0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2422000640 1 -- 172.21.15.27:0/2783811964 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f242803b070 con 0x558d924b01b0 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 10 client.27392 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:41.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 20 client.27392 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:41.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d9246cda0 0x558d9290b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:41.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 4 client.27392 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:19:41.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d927f4970 con 0x558d9246cda0 2024-09-21T07:19:41.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.735+0000 7f2437400640 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d9246cda0 0x558d9290b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:41.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.736+0000 7f2437400640 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d9246cda0 0x558d9290b490 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b02051ef740919 server_cookie=1993a27170cd6532 in_seq=0 out_seq=0 2024-09-21T07:19:41.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.736+0000 7f2422000640 10 client.27392 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:41.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.736+0000 7f2422000640 1 -- 172.21.15.27:0/2783811964 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x558d927f4970 con 0x558d9246cda0 2024-09-21T07:19:41.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.736+0000 7f2422000640 10 client.27392 handle_command_reply: tid=0 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.736+0000 7f2445e60740 1 client.27392 shutdown 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 msgr2=0x558d92893b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d92893b70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f242802f040 tx=0x7f24280051a0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 shutdown_connections 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558d9246cda0 0x558d9290b490 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d924d9070 0x558d9288ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558d924b8390 0x558d9288e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 --2- 172.21.15.27:0/2783811964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d924b01b0 0x558d92893b70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2422000640 0 client.27392 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 >> 172.21.15.27:0/2783811964 conn(0x558d924cd460 msgr2=0x558d9288d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 shutdown_connections 2024-09-21T07:19:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 1 -- 172.21.15.27:0/2783811964 wait complete. 2024-09-21T07:19:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.737+0000 7f2445e60740 20 client.27392 trim_cache size 0 max 16384 2024-09-21T07:19:41.743 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:41.743 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:19:41.743 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:19:41.743 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:19:41.743 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:19:41.743 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:19:41.743 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:19:41.744 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.745 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.750 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:19:41.750 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:19:41.750 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:41.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:41.752 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:41.752 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f241c0b6680 msgr2=0x7f241c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f241c0b6680 0x7f241c0b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x558d9287e7e0 tx=0x7f2428005f90 comp rx=0 tx=0).stop 2024-09-21T07:19:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810b6c0 msgr2=0x7f243819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810b6c0 0x7f243819ccb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f242400d740 tx=0x7f242400dc00 comp rx=0 tx=0).stop 2024-09-21T07:19:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 shutdown_connections 2024-09-21T07:19:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f241c0b6680 0x7f241c0b8b90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243810c010 0x7f24381a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f243810b6c0 0x7f243819ccb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 --2- 172.21.15.27:0/3043508587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2438105d80 0x7f243819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 >> 172.21.15.27:0/3043508587 conn(0x7f2438101810 msgr2=0x7f2438108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 shutdown_connections 2024-09-21T07:19:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:41.738+0000 7f243f200640 1 -- 172.21.15.27:0/3043508587 wait complete. 2024-09-21T07:19:42.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:19:42.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:19:42.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:42 smithi027 ceph-mon[30779]: pgmap v580: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-09-21T07:19:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:42 smithi119 ceph-mon[34519]: pgmap v580: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-09-21T07:19:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:42 smithi153 ceph-mon[34747]: pgmap v580: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-09-21T07:19:43.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 -- 172.21.15.27:0/3358661866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 msgr2=0x7fa5e0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:43.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/3358661866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 0x7fa5e0112840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009ca0 tx=0x7fa5d802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 -- 172.21.15.27:0/3358661866 shutdown_connections 2024-09-21T07:19:43.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/3358661866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 0x7fa5e0112840 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/3358661866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e010b6c0 0x7fa5e010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/3358661866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0105d80 0x7fa5e0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 -- 172.21.15.27:0/3358661866 >> 172.21.15.27:0/3358661866 conn(0x7fa5e01017d0 msgr2=0x7fa5e0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:43.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 -- 172.21.15.27:0/3358661866 shutdown_connections 2024-09-21T07:19:43.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.582+0000 7fa5e6c00640 1 -- 172.21.15.27:0/3358661866 wait complete. 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.583+0000 7fa5e6c00640 1 Processor -- start 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.583+0000 7fa5e6c00640 1 -- start start 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e6c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0105d80 0x7fa5e019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e6c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e010b6c0 0x7fa5e019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e6c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 0x7fa5e01a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e6c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0115cd0 con 0x7fa5e010c010 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e6c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0115b50 con 0x7fa5e010b6c0 2024-09-21T07:19:43.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e6c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0115e50 con 0x7fa5e0105d80 2024-09-21T07:19:43.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 0x7fa5e01a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:43.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 0x7fa5e01a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56324/0 (socket says 172.21.15.27:56324) 2024-09-21T07:19:43.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5e4c00640 1 -- 172.21.15.27:0/817504565 learned_addr learned my addr 172.21.15.27:0/817504565 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:43.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5dfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0105d80 0x7fa5e019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:43.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5d7400640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e010b6c0 0x7fa5e019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5dfe00640 1 -- 172.21.15.27:0/817504565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e010b6c0 msgr2=0x7fa5e019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5dfe00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e010b6c0 0x7fa5e019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5dfe00640 1 -- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 msgr2=0x7fa5e01a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.584+0000 7fa5dfe00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 0x7fa5e01a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.585+0000 7fa5dfe00640 1 -- 172.21.15.27:0/817504565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e01a25b0 con 0x7fa5e0105d80 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.585+0000 7fa5dfe00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0105d80 0x7fa5e019c640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9f0 tx=0x7fa5d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.585+0000 7fa5dd600640 1 -- 172.21.15.27:0/817504565 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000ccc0 con 0x7fa5e0105d80 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.585+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e01a28a0 con 0x7fa5e0105d80 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.585+0000 7fa5dd600640 1 -- 172.21.15.27:0/817504565 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5d00108e0 con 0x7fa5e0105d80 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.585+0000 7fa5dd600640 1 -- 172.21.15.27:0/817504565 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0018880 con 0x7fa5e0105d80 2024-09-21T07:19:43.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.585+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01a9000 con 0x7fa5e0105d80 2024-09-21T07:19:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.586+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e0106190 con 0x7fa5e0105d80 2024-09-21T07:19:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.589+0000 7fa5dd600640 1 -- 172.21.15.27:0/817504565 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa5d0010430 con 0x7fa5e0105d80 2024-09-21T07:19:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.591+0000 7fa5dd600640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5c40b6680 0x7fa5c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.591+0000 7fa5dd600640 1 -- 172.21.15.27:0/817504565 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fa5d00e82f0 con 0x7fa5e0105d80 2024-09-21T07:19:43.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.591+0000 7fa5d7400640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5c40b6680 0x7fa5c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:43.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.592+0000 7fa5d7400640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5c40b6680 0x7fa5c40b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5e010b180 tx=0x7fa5c800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:43.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.593+0000 7fa5dd600640 1 -- 172.21.15.27:0/817504565 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa5d00b2810 con 0x7fa5e0105d80 2024-09-21T07:19:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.832+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e019d770 con 0x7fa5e0105d80 2024-09-21T07:19:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.833+0000 7fa5dd600640 1 -- 172.21.15.27:0/817504565 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fa5d00b7750 con 0x7fa5e0105d80 2024-09-21T07:19:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:19:43.835 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:19:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.836+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5c40b6680 msgr2=0x7fa5c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5c40b6680 0x7fa5c40b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5e010b180 tx=0x7fa5c800b390 comp rx=0 tx=0).stop 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0105d80 msgr2=0x7fa5e019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0105d80 0x7fa5e019c640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9f0 tx=0x7fa5d000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 shutdown_connections 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5c40b6680 0x7fa5c40b8b90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5e010c010 0x7fa5e01a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e010b6c0 0x7fa5e019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 --2- 172.21.15.27:0/817504565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5e0105d80 0x7fa5e019c640 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 >> 172.21.15.27:0/817504565 conn(0x7fa5e01017d0 msgr2=0x7fa5e0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 shutdown_connections 2024-09-21T07:19:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:43.837+0000 7fa5e6c00640 1 -- 172.21.15.27:0/817504565 wait complete. 2024-09-21T07:19:44.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:19:44.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:44 smithi027 ceph-mon[30779]: pgmap v581: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-09-21T07:19:44.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:44 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/817504565' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:44.982 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:44 smithi119 ceph-mon[34519]: pgmap v581: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-09-21T07:19:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:44 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/817504565' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:44 smithi153 ceph-mon[34747]: pgmap v581: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-09-21T07:19:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:44 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/817504565' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:45.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 -- 172.21.15.27:0/292692976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10a1b0 msgr2=0x7fca1c10a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 --2- 172.21.15.27:0/292692976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10a1b0 0x7fca1c10a5c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca10009900 tx=0x7fca1002a410 comp rx=0 tx=0).stop 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 -- 172.21.15.27:0/292692976 shutdown_connections 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 --2- 172.21.15.27:0/292692976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca1c110440 0x7fca1c116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 --2- 172.21.15.27:0/292692976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca1c10faf0 0x7fca1c10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 --2- 172.21.15.27:0/292692976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10a1b0 0x7fca1c10a5c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 -- 172.21.15.27:0/292692976 >> 172.21.15.27:0/292692976 conn(0x7fca1c0780e0 msgr2=0x7fca1c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 -- 172.21.15.27:0/292692976 shutdown_connections 2024-09-21T07:19:45.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.611+0000 7fca23800640 1 -- 172.21.15.27:0/292692976 wait complete. 2024-09-21T07:19:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.613+0000 7fca23800640 1 Processor -- start 2024-09-21T07:19:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.613+0000 7fca23800640 1 -- start start 2024-09-21T07:19:45.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca23800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca1c10a1b0 0x7fca1c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca23800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10faf0 0x7fca1c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca23800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca1c110440 0x7fca1c1a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca23800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c11a110 con 0x7fca1c10faf0 2024-09-21T07:19:45.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca23800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c119f90 con 0x7fca1c10a1b0 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca23800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c11a290 con 0x7fca1c110440 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10faf0 0x7fca1c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10faf0 0x7fca1c1a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56348/0 (socket says 172.21.15.27:56348) 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 -- 172.21.15.27:0/4082519314 learned_addr learned my addr 172.21.15.27:0/4082519314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca21800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca1c110440 0x7fca1c1a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca20e00640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca1c10a1b0 0x7fca1c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 -- 172.21.15.27:0/4082519314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca1c110440 msgr2=0x7fca1c1a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca1c110440 0x7fca1c1a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 -- 172.21.15.27:0/4082519314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca1c10a1b0 msgr2=0x7fca1c1a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca1c10a1b0 0x7fca1c1a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.614+0000 7fca1be00640 1 -- 172.21.15.27:0/4082519314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c1a6a10 con 0x7fca1c10faf0 2024-09-21T07:19:45.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.615+0000 7fca1be00640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10faf0 0x7fca1c1a1040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fca0800e9f0 tx=0x7fca0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:45.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.615+0000 7fca19600640 1 -- 172.21.15.27:0/4082519314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0800ccc0 con 0x7fca1c10faf0 2024-09-21T07:19:45.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.615+0000 7fca19600640 1 -- 172.21.15.27:0/4082519314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca080108e0 con 0x7fca1c10faf0 2024-09-21T07:19:45.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.615+0000 7fca19600640 1 -- 172.21.15.27:0/4082519314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08018860 con 0x7fca1c10faf0 2024-09-21T07:19:45.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.615+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c1a6d00 con 0x7fca1c10faf0 2024-09-21T07:19:45.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.615+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1ad460 con 0x7fca1c10faf0 2024-09-21T07:19:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 Processor -- start 2024-09-21T07:19:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 -- start start 2024-09-21T07:19:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca19600640 1 -- 172.21.15.27:0/4082519314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fca08010430 con 0x7fca1c10faf0 2024-09-21T07:19:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436432e070 0x5643646d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646d5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56436464bfb0 con 0x56436432e070 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56436463b3e0 con 0x5643643051b0 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56436463f1f0 con 0x56436430d390 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca20e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca21800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436432e070 0x5643646d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca20e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45974/0 (socket says 172.21.15.27:45974) 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca1be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646d5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.619+0000 7fca20e00640 1 -- 172.21.15.27:0/354549982 learned_addr learned my addr 172.21.15.27:0/354549982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.620+0000 7fca20e00640 1 -- 172.21.15.27:0/354549982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 msgr2=0x5643646d5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.620+0000 7fca20e00640 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646d5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.620+0000 7fca20e00640 1 -- 172.21.15.27:0/354549982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436432e070 msgr2=0x5643646d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.620+0000 7fca20e00640 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436432e070 0x5643646d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.620+0000 7fca20e00640 1 -- 172.21.15.27:0/354549982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564364605cd0 con 0x5643643051b0 2024-09-21T07:19:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.620+0000 7fca1be00640 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646d5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.620+0000 7fca20e00640 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d52e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca10009900 tx=0x7fca10032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab2f84a2ccaf69c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.621+0000 7fca0e000640 1 -- 172.21.15.27:0/354549982 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10040070 con 0x5643643051b0 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.621+0000 7fca0e000640 1 -- 172.21.15.27:0/354549982 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca10004030 con 0x5643643051b0 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.621+0000 7fca0e000640 1 -- 172.21.15.27:0/354549982 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10035040 con 0x5643643051b0 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.621+0000 7fca19600640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca040b6680 0x7fca040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 -- 172.21.15.27:0/354549982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 msgr2=0x5643646d52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d52e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca10009900 tx=0x7fca10032b90 comp rx=0 tx=0).stop 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca19600640 1 -- 172.21.15.27:0/4082519314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fca080e7db0 con 0x7fca1c10faf0 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca21800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca040b6680 0x7fca040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 -- 172.21.15.27:0/354549982 shutdown_connections 2024-09-21T07:19:45.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646d5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d52e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/354549982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436432e070 0x5643646d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 -- 172.21.15.27:0/354549982 >> 172.21.15.27:0/354549982 conn(0x564364322460 msgr2=0x5643646d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 -- 172.21.15.27:0/354549982 shutdown_connections 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.622+0000 7fca2a4d1740 1 -- 172.21.15.27:0/354549982 wait complete. 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca21800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca040b6680 0x7fca040b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca14005fd0 tx=0x7fca14009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 -- start start 2024-09-21T07:19:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 Processor -- start 2024-09-21T07:19:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x5643646ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56436449c700 con 0x5643643051b0 2024-09-21T07:19:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5643645d6580 con 0x56436432e070 2024-09-21T07:19:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643646e3520 con 0x56436430d390 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.623+0000 7fca2a4d1740 10 client.0 _ll_register_callbacks cb 0x5643645e6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca20e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca1be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca21800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x5643646ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca1be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35502/0 (socket says 172.21.15.27:35502) 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca1be00640 1 -- 172.21.15.27:0/250261871 learned_addr learned my addr 172.21.15.27:0/250261871 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca21800640 1 -- 172.21.15.27:0/250261871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 msgr2=0x5643646e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca21800640 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca21800640 1 -- 172.21.15.27:0/250261871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 msgr2=0x5643646e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca21800640 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca21800640 1 -- 172.21.15.27:0/250261871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564364399650 con 0x56436432e070 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.624+0000 7fca1be00640 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca21800640 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x5643646ec8a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fca14011e90 tx=0x7fca14004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e021ca9c182e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca02a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca02a00640 1 -- 172.21.15.27:0/250261871 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14004b20 con 0x56436432e070 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5643645d8b90 con 0x56436432e070 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca02a00640 1 -- 172.21.15.27:0/250261871 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fca14004cc0 con 0x56436432e070 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca02a00640 1 -- 172.21.15.27:0/250261871 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca1400f870 con 0x56436432e070 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca2a4d1740 10 client.27410 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.625+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643645f3950 con 0x56436432e070 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca02a00640 1 -- 172.21.15.27:0/250261871 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5643645d8b90 con 0x56436432e070 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca02a00640 1 -- 172.21.15.27:0/250261871 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fca14026070 con 0x56436432e070 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca2a4d1740 10 client.27410 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca2a4d1740 10 client.27410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca2a4d1740 20 client.27410 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642ec9f0 0x5643646f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca2a4d1740 4 client.27410 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.626+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56436464df00 con 0x5643642ec9f0 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.627+0000 7fca20e00640 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642ec9f0 0x5643646f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.627+0000 7fca20e00640 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642ec9f0 0x5643646f2780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fefef038aaa2732d server_cookie=8be06ea847824f95 in_seq=0 out_seq=0 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.627+0000 7fca02a00640 10 client.27410 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca02a00640 1 -- 172.21.15.27:0/250261871 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56436464df00 con 0x5643642ec9f0 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca02a00640 10 client.27410 handle_command_reply: tid=0 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 client.27410 shutdown 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 msgr2=0x5643646ec8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x5643646ec8a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fca14011e90 tx=0x7fca14004ab0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 shutdown_connections 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642ec9f0 0x5643646f2780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x5643646ec8a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/250261871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 >> 172.21.15.27:0/250261871 conn(0x564364322460 msgr2=0x5643646d7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:45.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca02a00640 0 client.27410 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:45.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.628+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 shutdown_connections 2024-09-21T07:19:45.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.629+0000 7fca2a4d1740 1 -- 172.21.15.27:0/250261871 wait complete. 2024-09-21T07:19:45.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.629+0000 7fca2a4d1740 20 client.27410 trim_cache size 0 max 16384 2024-09-21T07:19:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 Processor -- start 2024-09-21T07:19:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 -- start start 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436430d390 0x5643646ec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436432e070 0x5643646d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b9d50 con 0x56436430d390 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b7620 con 0x5643643051b0 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.643+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b5490 con 0x56436432e070 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45994/0 (socket says 172.21.15.27:45994) 2024-09-21T07:19:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 -- 172.21.15.27:0/4111317764 learned_addr learned my addr 172.21.15.27:0/4111317764 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca1be00640 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436430d390 0x5643646ec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca21800640 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436432e070 0x5643646d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 -- 172.21.15.27:0/4111317764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436432e070 msgr2=0x5643646d25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436432e070 0x5643646d25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 -- 172.21.15.27:0/4111317764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436430d390 msgr2=0x5643646ec7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436430d390 0x5643646ec7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 -- 172.21.15.27:0/4111317764 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643646ef1d0 con 0x5643643051b0 2024-09-21T07:19:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.644+0000 7fca20e00640 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d85b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5643646d3a20 tx=0x7fca10005180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0d562a0cf4725b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.645+0000 7fca0d600640 1 -- 172.21.15.27:0/4111317764 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10035040 con 0x5643643051b0 2024-09-21T07:19:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.645+0000 7fca0d600640 1 -- 172.21.15.27:0/4111317764 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca10031d10 con 0x5643643051b0 2024-09-21T07:19:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.645+0000 7fca0d600640 1 -- 172.21.15.27:0/4111317764 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10040040 con 0x5643643051b0 2024-09-21T07:19:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 -- 172.21.15.27:0/4111317764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 msgr2=0x5643646d85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d85b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5643646d3a20 tx=0x7fca10005180 comp rx=0 tx=0).stop 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 -- 172.21.15.27:0/4111317764 shutdown_connections 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436432e070 0x5643646d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56436430d390 0x5643646ec7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/4111317764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643643051b0 0x5643646d85b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 -- 172.21.15.27:0/4111317764 >> 172.21.15.27:0/4111317764 conn(0x564364322460 msgr2=0x5643646d7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 -- 172.21.15.27:0/4111317764 shutdown_connections 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 -- 172.21.15.27:0/4111317764 wait complete. 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 -- start start 2024-09-21T07:19:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.646+0000 7fca2a4d1740 1 Processor -- start 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca2a4d1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x56436475c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b5ae0 con 0x5643643051b0 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b6f50 con 0x56436432e070 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca2a4d1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b61b0 con 0x56436430d390 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca2a4d1740 10 client.0 _ll_register_callbacks cb 0x7fca14003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca20e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca21800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x56436475c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca20e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56386/0 (socket says 172.21.15.27:56386) 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca21800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x56436475c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46006/0 (socket says 172.21.15.27:46006) 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca20e00640 1 -- 172.21.15.27:0/3265218922 learned_addr learned my addr 172.21.15.27:0/3265218922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.647+0000 7fca1be00640 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca21800640 1 -- 172.21.15.27:0/3265218922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 msgr2=0x5643646e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca21800640 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca21800640 1 -- 172.21.15.27:0/3265218922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 msgr2=0x5643646e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca21800640 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca21800640 1 -- 172.21.15.27:0/3265218922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643646e58c0 con 0x56436432e070 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca21800640 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x56436475c140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca14011f30 tx=0x7fca14005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bb85d2a9afbc3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca02a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca02a00640 1 -- 172.21.15.27:0/3265218922 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1400fc20 con 0x56436432e070 2024-09-21T07:19:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca10031d10 con 0x56436432e070 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.648+0000 7fca02a00640 1 -- 172.21.15.27:0/3265218922 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fca1400fdc0 con 0x56436432e070 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca02a00640 1 -- 172.21.15.27:0/3265218922 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca14005000 con 0x56436432e070 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 10 client.27416 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56436475d690 con 0x56436432e070 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca02a00640 1 -- 172.21.15.27:0/3265218922 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca10031d10 con 0x56436432e070 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca02a00640 1 -- 172.21.15.27:0/3265218922 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fca14026040 con 0x56436432e070 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 10 client.27416 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 10 client.27416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 20 client.27416 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642c1da0 0x56436475fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 4 client.27416 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.649+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643646499b0 con 0x5643642c1da0 2024-09-21T07:19:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.650+0000 7fca20e00640 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642c1da0 0x56436475fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.650+0000 7fca20e00640 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642c1da0 0x56436475fe70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2aad5dca0d1c2977 server_cookie=71e2941e5d4c2961 in_seq=0 out_seq=0 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.650+0000 7fca02a00640 10 client.27416 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca02a00640 1 -- 172.21.15.27:0/3265218922 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643646499b0 con 0x5643642c1da0 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca02a00640 10 client.27416 handle_command_reply: tid=0 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 client.27416 shutdown 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 msgr2=0x56436475c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x56436475c140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca14011f30 tx=0x7fca14005e30 comp rx=0 tx=0).stop 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 shutdown_connections 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5643642c1da0 0x56436475fe70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56436430d390 0x5643646e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56436432e070 0x56436475c140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca02a00640 0 client.27416 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 --2- 172.21.15.27:0/3265218922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643643051b0 0x5643646e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 >> 172.21.15.27:0/3265218922 conn(0x564364322460 msgr2=0x5643646d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 shutdown_connections 2024-09-21T07:19:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 1 -- 172.21.15.27:0/3265218922 wait complete. 2024-09-21T07:19:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.651+0000 7fca2a4d1740 20 client.27416 trim_cache size 0 max 16384 2024-09-21T07:19:45.657 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:19:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca040b6680 msgr2=0x7fca040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca040b6680 0x7fca040b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca14005fd0 tx=0x7fca14009500 comp rx=0 tx=0).stop 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10faf0 msgr2=0x7fca1c1a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10faf0 0x7fca1c1a1040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fca0800e9f0 tx=0x7fca0800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 shutdown_connections 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca1c110440 0x7fca1c1a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca040b6680 0x7fca040b8b90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca1c10faf0 0x7fca1c1a1040 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 --2- 172.21.15.27:0/4082519314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca1c10a1b0 0x7fca1c1a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 >> 172.21.15.27:0/4082519314 conn(0x7fca1c0780e0 msgr2=0x7fca1c0789c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 shutdown_connections 2024-09-21T07:19:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:45.652+0000 7fca23800640 1 -- 172.21.15.27:0/4082519314 wait complete. 2024-09-21T07:19:46.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:19:46.914 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:46 smithi119 ceph-mon[34519]: pgmap v582: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:19:47.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:46 smithi027 ceph-mon[30779]: pgmap v582: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:19:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:46 smithi153 ceph-mon[34747]: pgmap v582: 129 pgs: 129 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:19:47.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 -- 172.21.15.27:0/1077270008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2010faf0 msgr2=0x7fef2010ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 --2- 172.21.15.27:0/1077270008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2010faf0 0x7fef2010ff00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009900 tx=0x7fef0c02a410 comp rx=0 tx=0).stop 2024-09-21T07:19:47.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 -- 172.21.15.27:0/1077270008 shutdown_connections 2024-09-21T07:19:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 --2- 172.21.15.27:0/1077270008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef20110440 0x7fef20116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 --2- 172.21.15.27:0/1077270008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2010faf0 0x7fef2010ff00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 --2- 172.21.15.27:0/1077270008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef2010a1b0 0x7fef2010a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 -- 172.21.15.27:0/1077270008 >> 172.21.15.27:0/1077270008 conn(0x7fef200780e0 msgr2=0x7fef201081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 -- 172.21.15.27:0/1077270008 shutdown_connections 2024-09-21T07:19:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.500+0000 7fef28400640 1 -- 172.21.15.27:0/1077270008 wait complete. 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.501+0000 7fef28400640 1 Processor -- start 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.501+0000 7fef28400640 1 -- start start 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef28400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2010a1b0 0x7fef201a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef28400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef2010faf0 0x7fef201a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef28400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef20110440 0x7fef201a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef28400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef2011a120 con 0x7fef20110440 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef28400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef20119fa0 con 0x7fef2010a1b0 2024-09-21T07:19:47.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef28400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef2011a2a0 con 0x7fef2010faf0 2024-09-21T07:19:47.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef26400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef20110440 0x7fef201a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef26400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef20110440 0x7fef201a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56416/0 (socket says 172.21.15.27:56416) 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef25a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2010a1b0 0x7fef201a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef26400640 1 -- 172.21.15.27:0/4240583094 learned_addr learned my addr 172.21.15.27:0/4240583094 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.502+0000 7fef25000640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef2010faf0 0x7fef201a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.503+0000 7fef26400640 1 -- 172.21.15.27:0/4240583094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef2010faf0 msgr2=0x7fef201a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.503+0000 7fef26400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef2010faf0 0x7fef201a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.503+0000 7fef26400640 1 -- 172.21.15.27:0/4240583094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2010a1b0 msgr2=0x7fef201a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.503+0000 7fef26400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2010a1b0 0x7fef201a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.503+0000 7fef26400640 1 -- 172.21.15.27:0/4240583094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef201a6a60 con 0x7fef20110440 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.503+0000 7fef26400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef20110440 0x7fef201a6330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef18009fd0 tx=0x7fef1800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.503+0000 7fef1e000640 1 -- 172.21.15.27:0/4240583094 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18017070 con 0x7fef20110440 2024-09-21T07:19:47.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.504+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef201a6d50 con 0x7fef20110440 2024-09-21T07:19:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.504+0000 7fef1e000640 1 -- 172.21.15.27:0/4240583094 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef18003e20 con 0x7fef20110440 2024-09-21T07:19:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.504+0000 7fef1e000640 1 -- 172.21.15.27:0/4240583094 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18010460 con 0x7fef20110440 2024-09-21T07:19:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.504+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef201ad4b0 con 0x7fef20110440 2024-09-21T07:19:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.507+0000 7fef2f08c740 1 Processor -- start 2024-09-21T07:19:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.507+0000 7fef1e000640 1 -- 172.21.15.27:0/4240583094 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fef18003970 con 0x7fef20110440 2024-09-21T07:19:47.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef2f08c740 1 -- start start 2024-09-21T07:19:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b41820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef2f08c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f8b8ab7fb0 con 0x55f8b8779390 2024-09-21T07:19:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef2f08c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f8b8aa73e0 con 0x55f8b87711b0 2024-09-21T07:19:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef2f08c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f8b8aab1f0 con 0x55f8b879a070 2024-09-21T07:19:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef25000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b41820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef25000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b41820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56426/0 (socket says 172.21.15.27:56426) 2024-09-21T07:19:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef25000640 1 -- 172.21.15.27:0/3031917362 learned_addr learned my addr 172.21.15.27:0/3031917362 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.508+0000 7fef25a00640 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.509+0000 7fef25000640 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.509+0000 7fef25a00640 1 -- 172.21.15.27:0/3031917362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 msgr2=0x55f8b8b40da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.509+0000 7fef25a00640 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b40da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.509+0000 7fef25a00640 1 -- 172.21.15.27:0/3031917362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 msgr2=0x55f8b8b41820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.509+0000 7fef25a00640 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b41820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.509+0000 7fef25a00640 1 -- 172.21.15.27:0/3031917362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8b8a71cd0 con 0x55f8b87711b0 2024-09-21T07:19:47.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.509+0000 7fef25a00640 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b412e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fef14009c00 tx=0x7fef14011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2f6199f8a67251 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:47.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.510+0000 7fef12a00640 1 -- 172.21.15.27:0/3031917362 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14004230 con 0x55f8b87711b0 2024-09-21T07:19:47.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.510+0000 7fef12a00640 1 -- 172.21.15.27:0/3031917362 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef140043d0 con 0x55f8b87711b0 2024-09-21T07:19:47.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.510+0000 7fef12a00640 1 -- 172.21.15.27:0/3031917362 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14004c60 con 0x55f8b87711b0 2024-09-21T07:19:47.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.510+0000 7fef1e000640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fef080b6680 0x7fef080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.510+0000 7fef1e000640 1 -- 172.21.15.27:0/4240583094 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fef180e77f0 con 0x7fef20110440 2024-09-21T07:19:47.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.510+0000 7fef25000640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fef080b6680 0x7fef080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 -- 172.21.15.27:0/3031917362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 msgr2=0x55f8b8b412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b412e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fef14009c00 tx=0x7fef14011810 comp rx=0 tx=0).stop 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef25000640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fef080b6680 0x7fef080b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005ab0 tx=0x7fef0c005780 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 -- 172.21.15.27:0/3031917362 shutdown_connections 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b41820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b412e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 --2- 172.21.15.27:0/3031917362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b40da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 -- 172.21.15.27:0/3031917362 >> 172.21.15.27:0/3031917362 conn(0x55f8b878e460 msgr2=0x55f8b8b3ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:47.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 -- 172.21.15.27:0/3031917362 shutdown_connections 2024-09-21T07:19:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.511+0000 7fef2f08c740 1 -- 172.21.15.27:0/3031917362 wait complete. 2024-09-21T07:19:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.512+0000 7fef2f08c740 1 -- start start 2024-09-21T07:19:47.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.512+0000 7fef2f08c740 1 Processor -- start 2024-09-21T07:19:47.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b4c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b4ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b879a070 0x55f8b8b58900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef2f08c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f8b8908700 con 0x55f8b8779390 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef2f08c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f8b8a42580 con 0x55f8b879a070 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef2f08c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f8b8b4f580 con 0x55f8b87711b0 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef2f08c740 10 client.0 _ll_register_callbacks cb 0x55f8b8a52470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b4c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b4c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35550/0 (socket says 172.21.15.27:35550) 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 -- 172.21.15.27:0/1966900098 learned_addr learned my addr 172.21.15.27:0/1966900098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25000640 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b4ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef26400640 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b879a070 0x55f8b8b58900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 -- 172.21.15.27:0/1966900098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b879a070 msgr2=0x55f8b8b58900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b879a070 0x55f8b8b58900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 -- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 msgr2=0x55f8b8b4ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b4ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.513+0000 7fef25a00640 1 -- 172.21.15.27:0/1966900098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8b8805650 con 0x55f8b87711b0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef25a00640 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b4c510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef14004940 tx=0x7fef14003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147837fe6f67c1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef07400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef07400640 1 -- 172.21.15.27:0/1966900098 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14027020 con 0x55f8b87711b0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8b8a44b90 con 0x55f8b87711b0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef07400640 1 -- 172.21.15.27:0/1966900098 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fef14003b90 con 0x55f8b87711b0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef07400640 1 -- 172.21.15.27:0/1966900098 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef14032050 con 0x55f8b87711b0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef2f08c740 10 client.27397 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.514+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8b8a5f950 con 0x55f8b87711b0 2024-09-21T07:19:47.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef07400640 1 -- 172.21.15.27:0/1966900098 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f8b8a44b90 con 0x55f8b87711b0 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef07400640 1 -- 172.21.15.27:0/1966900098 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fef14018070 con 0x55f8b87711b0 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef2f08c740 10 client.27397 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef2f08c740 10 client.27397 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef2f08c740 20 client.27397 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b87589f0 0x55f8b8b5e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef2f08c740 4 client.27397 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8b8ab9f00 con 0x55f8b87589f0 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.515+0000 7fef25000640 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b87589f0 0x55f8b8b5e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.516+0000 7fef25000640 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b87589f0 0x55f8b8b5e570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eda954e6d96885b1 server_cookie=2e981be44e3eb0a5 in_seq=0 out_seq=0 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.516+0000 7fef07400640 10 client.27397 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef07400640 1 -- 172.21.15.27:0/1966900098 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8b8ab9f00 con 0x55f8b87589f0 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef07400640 10 client.27397 handle_command_reply: tid=0 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 client.27397 shutdown 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 msgr2=0x55f8b8b4c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b4c510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef14004940 tx=0x7fef14003820 comp rx=0 tx=0).stop 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 shutdown_connections 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b87589f0 0x55f8b8b5e570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b879a070 0x55f8b8b58900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef07400640 0 client.27397 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b4ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1966900098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b4c510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.517+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 >> 172.21.15.27:0/1966900098 conn(0x55f8b878e460 msgr2=0x55f8b8b43f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:47.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.518+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 shutdown_connections 2024-09-21T07:19:47.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.518+0000 7fef2f08c740 1 -- 172.21.15.27:0/1966900098 wait complete. 2024-09-21T07:19:47.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.518+0000 7fef2f08c740 20 client.27397 trim_cache size 0 max 16384 2024-09-21T07:19:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 Processor -- start 2024-09-21T07:19:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 -- start start 2024-09-21T07:19:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b43e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b8779390 0x55f8b8b443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b879a070 0x55f8b8b49700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef201b9d40 con 0x55f8b879a070 2024-09-21T07:19:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef201b7610 con 0x55f8b8779390 2024-09-21T07:19:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef2f08c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef201b5480 con 0x55f8b87711b0 2024-09-21T07:19:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef26400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b879a070 0x55f8b8b49700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef25000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b8779390 0x55f8b8b443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef25a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b43e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef25000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b8779390 0x55f8b8b443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46046/0 (socket says 172.21.15.27:46046) 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.532+0000 7fef25000640 1 -- 172.21.15.27:0/2020036082 learned_addr learned my addr 172.21.15.27:0/2020036082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef25a00640 1 -- 172.21.15.27:0/2020036082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b8779390 msgr2=0x55f8b8b443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef25a00640 1 --2- 172.21.15.27:0/2020036082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b8779390 0x55f8b8b443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef25a00640 1 -- 172.21.15.27:0/2020036082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b879a070 msgr2=0x55f8b8b49700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef25a00640 1 --2- 172.21.15.27:0/2020036082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b879a070 0x55f8b8b49700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef25a00640 1 -- 172.21.15.27:0/2020036082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8b8b4d7d0 con 0x55f8b87711b0 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef25a00640 1 --2- 172.21.15.27:0/2020036082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b43e60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef14009c00 tx=0x7fef14011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56104a14369f464e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef12000640 1 -- 172.21.15.27:0/2020036082 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14027040 con 0x55f8b87711b0 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef12000640 1 -- 172.21.15.27:0/2020036082 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef14023b30 con 0x55f8b87711b0 2024-09-21T07:19:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.533+0000 7fef12000640 1 -- 172.21.15.27:0/2020036082 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1401e040 con 0x55f8b87711b0 2024-09-21T07:19:47.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.534+0000 7fef2f08c740 1 -- 172.21.15.27:0/2020036082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 msgr2=0x55f8b8b43e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.534+0000 7fef2f08c740 1 --2- 172.21.15.27:0/2020036082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b43e60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef14009c00 tx=0x7fef14011d10 comp rx=0 tx=0).stop 2024-09-21T07:19:47.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.534+0000 7fef2f08c740 1 -- 172.21.15.27:0/2020036082 shutdown_connections 2024-09-21T07:19:47.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.534+0000 7fef2f08c740 1 --2- 172.21.15.27:0/2020036082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b879a070 0x55f8b8b49700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.534+0000 7fef2f08c740 1 --2- 172.21.15.27:0/2020036082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b8779390 0x55f8b8b443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.534+0000 7fef2f08c740 1 --2- 172.21.15.27:0/2020036082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b87711b0 0x55f8b8b43e60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.534+0000 7fef2f08c740 1 -- 172.21.15.27:0/2020036082 >> 172.21.15.27:0/2020036082 conn(0x55f8b878e460 msgr2=0x55f8b8b4ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:47.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.535+0000 7fef2f08c740 1 -- 172.21.15.27:0/2020036082 shutdown_connections 2024-09-21T07:19:47.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.535+0000 7fef2f08c740 1 -- 172.21.15.27:0/2020036082 wait complete. 2024-09-21T07:19:47.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.535+0000 7fef2f08c740 1 -- start start 2024-09-21T07:19:47.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.535+0000 7fef2f08c740 1 Processor -- start 2024-09-21T07:19:47.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b50e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef2f08c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b51360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef2f08c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef201b5ad0 con 0x55f8b8779390 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef2f08c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef201b6f40 con 0x55f8b87711b0 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef2f08c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef201b61a0 con 0x55f8b879a070 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef2f08c740 10 client.0 _ll_register_callbacks cb 0x7fef14009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b50e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46054/0 (socket says 172.21.15.27:46054) 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25a00640 1 -- 172.21.15.27:0/1637098459 learned_addr learned my addr 172.21.15.27:0/1637098459 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef26400640 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b51360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25a00640 1 -- 172.21.15.27:0/1637098459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 msgr2=0x55f8b8b51360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25a00640 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b51360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25a00640 1 -- 172.21.15.27:0/1637098459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 msgr2=0x55f8b8b50e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.536+0000 7fef25a00640 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b50e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.537+0000 7fef25a00640 1 -- 172.21.15.27:0/1637098459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8b8bc7e50 con 0x55f8b87711b0 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.537+0000 7fef25a00640 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b508e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fef14009b20 tx=0x7fef14023d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e130369fc3e721ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.537+0000 7fef07400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.537+0000 7fef07400640 1 -- 172.21.15.27:0/1637098459 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14003b40 con 0x55f8b87711b0 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.537+0000 7fef07400640 1 -- 172.21.15.27:0/1637098459 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fef14003ce0 con 0x55f8b87711b0 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.537+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef14023b30 con 0x55f8b87711b0 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.537+0000 7fef07400640 1 -- 172.21.15.27:0/1637098459 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef14048050 con 0x55f8b87711b0 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 10 client.27434 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:47.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8b8bc90d0 con 0x55f8b87711b0 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef07400640 1 -- 172.21.15.27:0/1637098459 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef14023b30 con 0x55f8b87711b0 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef07400640 1 -- 172.21.15.27:0/1637098459 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fef14018040 con 0x55f8b87711b0 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 10 client.27434 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 10 client.27434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 20 client.27434 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b872dda0 0x55f8b8bcb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 4 client.27434 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.538+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8b8ab59b0 con 0x55f8b872dda0 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.539+0000 7fef25000640 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b872dda0 0x55f8b8bcb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.539+0000 7fef25000640 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b872dda0 0x55f8b8bcb7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea74e908990e7896 server_cookie=15f148d85006303e in_seq=0 out_seq=0 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.539+0000 7fef07400640 10 client.27434 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:47.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef07400640 1 -- 172.21.15.27:0/1637098459 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8b8ab59b0 con 0x55f8b872dda0 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef07400640 10 client.27434 handle_command_reply: tid=0 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 client.27434 shutdown 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 msgr2=0x55f8b8b508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b508e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fef14009b20 tx=0x7fef14023d90 comp rx=0 tx=0).stop 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 shutdown_connections 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55f8b872dda0 0x55f8b8bcb7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b879a070 0x55f8b8b51360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f8b8779390 0x55f8b8b50e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 --2- 172.21.15.27:0/1637098459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b87711b0 0x55f8b8b508e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef07400640 0 client.27434 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 >> 172.21.15.27:0/1637098459 conn(0x55f8b878e460 msgr2=0x55f8b8b4a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 shutdown_connections 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 1 -- 172.21.15.27:0/1637098459 wait complete. 2024-09-21T07:19:47.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.540+0000 7fef2f08c740 20 client.27434 trim_cache size 0 max 16384 2024-09-21T07:19:47.545 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:19:47.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fef080b6680 msgr2=0x7fef080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fef080b6680 0x7fef080b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005ab0 tx=0x7fef0c005780 comp rx=0 tx=0).stop 2024-09-21T07:19:47.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef20110440 msgr2=0x7fef201a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef20110440 0x7fef201a6330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef18009fd0 tx=0x7fef1800b260 comp rx=0 tx=0).stop 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 shutdown_connections 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef2010faf0 0x7fef201a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fef080b6680 0x7fef080b8b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef20110440 0x7fef201a6330 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 --2- 172.21.15.27:0/4240583094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2010a1b0 0x7fef201a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 >> 172.21.15.27:0/4240583094 conn(0x7fef200780e0 msgr2=0x7fef20078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 shutdown_connections 2024-09-21T07:19:47.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:47.541+0000 7fef28400640 1 -- 172.21.15.27:0/4240583094 wait complete. 2024-09-21T07:19:48.300 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:19:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:49.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:48 smithi027 ceph-mon[30779]: pgmap v583: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-21T07:19:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:48 smithi119 ceph-mon[34519]: pgmap v583: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-21T07:19:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:48 smithi153 ceph-mon[34747]: pgmap v583: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 -- 172.21.15.27:0/4039534986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440077380 msgr2=0x7f54400757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 --2- 172.21.15.27:0/4039534986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440077380 0x7f54400757b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5430009900 tx=0x7f543002a410 comp rx=0 tx=0).stop 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 -- 172.21.15.27:0/4039534986 shutdown_connections 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 --2- 172.21.15.27:0/4039534986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5440114440 0x7f5440116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 --2- 172.21.15.27:0/4039534986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 0x7f54400761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 --2- 172.21.15.27:0/4039534986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440077380 0x7f54400757b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 -- 172.21.15.27:0/4039534986 >> 172.21.15.27:0/4039534986 conn(0x7f54400ff5e0 msgr2=0x7f5440101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 -- 172.21.15.27:0/4039534986 shutdown_connections 2024-09-21T07:19:49.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.317+0000 7f5445200640 1 -- 172.21.15.27:0/4039534986 wait complete. 2024-09-21T07:19:49.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.319+0000 7f5445200640 1 Processor -- start 2024-09-21T07:19:49.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.319+0000 7f5445200640 1 -- start start 2024-09-21T07:19:49.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f5445200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 0x7f54401a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f5445200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5440077380 0x7f54401a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f5445200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440114440 0x7f54401a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f5445200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f544011a090 con 0x7f5440114440 2024-09-21T07:19:49.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f5445200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5440119f10 con 0x7f5440075dd0 2024-09-21T07:19:49.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f5445200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f544011a210 con 0x7f5440077380 2024-09-21T07:19:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f543de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 0x7f54401a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f543de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 0x7f54401a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50944/0 (socket says 172.21.15.27:50944) 2024-09-21T07:19:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f543de00640 1 -- 172.21.15.27:0/3371112227 learned_addr learned my addr 172.21.15.27:0/3371112227 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f543d400640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5440077380 0x7f54401a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.320+0000 7f543e800640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440114440 0x7f54401a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f543de00640 1 -- 172.21.15.27:0/3371112227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5440077380 msgr2=0x7f54401a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f543de00640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5440077380 0x7f54401a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f543de00640 1 -- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440114440 msgr2=0x7f54401a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f543de00640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440114440 0x7f54401a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f543de00640 1 -- 172.21.15.27:0/3371112227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401a6a90 con 0x7f5440075dd0 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f543d400640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5440077380 0x7f54401a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f543de00640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 0x7f54401a0b20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5430005940 tx=0x7f54300040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f5436a00640 1 -- 172.21.15.27:0/3371112227 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543003f030 con 0x7f5440075dd0 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54401a6d20 con 0x7f5440075dd0 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f5436a00640 1 -- 172.21.15.27:0/3371112227 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54300044c0 con 0x7f5440075dd0 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.321+0000 7f5436a00640 1 -- 172.21.15.27:0/3371112227 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430035070 con 0x7f5440075dd0 2024-09-21T07:19:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.322+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401ad4e0 con 0x7f5440075dd0 2024-09-21T07:19:49.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.325+0000 7f544bed2740 1 Processor -- start 2024-09-21T07:19:49.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.325+0000 7f544bed2740 1 -- start start 2024-09-21T07:19:49.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f5436a00640 1 -- 172.21.15.27:0/3371112227 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5430042030 con 0x7f5440075dd0 2024-09-21T07:19:49.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f544bed2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f544bed2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f544bed2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fdd390 0x5635f23a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f544bed2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5635f231bfb0 con 0x5635f1fd51b0 2024-09-21T07:19:49.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f544bed2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5635f230b3e0 con 0x5635f1fdd390 2024-09-21T07:19:49.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f544bed2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635f230f1f0 con 0x5635f1ffe070 2024-09-21T07:19:49.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f543de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.326+0000 7f543d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fdd390 0x5635f23a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37616/0 (socket says 172.21.15.27:37616) 2024-09-21T07:19:49.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543de00640 1 -- 172.21.15.27:0/3628386131 learned_addr learned my addr 172.21.15.27:0/3628386131 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543e800640 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543e800640 1 -- 172.21.15.27:0/3628386131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fdd390 msgr2=0x5635f23a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543e800640 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fdd390 0x5635f23a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543e800640 1 -- 172.21.15.27:0/3628386131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 msgr2=0x5635f23a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543e800640 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.327+0000 7f543e800640 1 -- 172.21.15.27:0/3628386131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f22d5cd0 con 0x5635f1ffe070 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.328+0000 7f543e800640 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23a4ba0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5438009f00 tx=0x7f5438011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abefb0a15ab5e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.328+0000 7f542b400640 1 -- 172.21.15.27:0/3628386131 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438004d20 con 0x5635f1ffe070 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.328+0000 7f542b400640 1 -- 172.21.15.27:0/3628386131 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438004050 con 0x5635f1ffe070 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.328+0000 7f542b400640 1 -- 172.21.15.27:0/3628386131 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438005260 con 0x5635f1ffe070 2024-09-21T07:19:49.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.328+0000 7f5436a00640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f54240b6680 0x7f54240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.328+0000 7f5436a00640 1 -- 172.21.15.27:0/3371112227 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f5430107b50 con 0x7f5440075dd0 2024-09-21T07:19:49.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.328+0000 7f543de00640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f54240b6680 0x7f54240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 -- 172.21.15.27:0/3628386131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 msgr2=0x5635f23a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23a4ba0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5438009f00 tx=0x7f5438011810 comp rx=0 tx=0).stop 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 -- 172.21.15.27:0/3628386131 shutdown_connections 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fdd390 0x5635f23a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 --2- 172.21.15.27:0/3628386131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23a4ba0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 -- 172.21.15.27:0/3628386131 >> 172.21.15.27:0/3628386131 conn(0x5635f1ff2460 msgr2=0x5635f23a2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f543de00640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f54240b6680 0x7f54240b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5430006010 tx=0x7f543010b600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 -- 172.21.15.27:0/3628386131 shutdown_connections 2024-09-21T07:19:49.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.329+0000 7f544bed2740 1 -- 172.21.15.27:0/3628386131 wait complete. 2024-09-21T07:19:49.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.330+0000 7f544bed2740 1 -- start start 2024-09-21T07:19:49.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.330+0000 7f544bed2740 1 Processor -- start 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f544bed2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fd51b0 0x5635f23b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f544bed2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f544bed2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f544bed2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5635f216c700 con 0x5635f1fdd390 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f544bed2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5635f22a6580 con 0x5635f1ffe070 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f544bed2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635f23b3380 con 0x5635f1fd51b0 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f544bed2740 10 client.0 _ll_register_callbacks cb 0x5635f22b6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f543e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fd51b0 0x5635f23b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f543d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f543e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fd51b0 0x5635f23b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42098/0 (socket says 172.21.15.27:42098) 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f543e800640 1 -- 172.21.15.27:0/3606049004 learned_addr learned my addr 172.21.15.27:0/3606049004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.331+0000 7f543e800640 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f543e800640 1 -- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 msgr2=0x5635f23bc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f543e800640 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23bc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f543e800640 1 -- 172.21.15.27:0/3606049004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 msgr2=0x5635f23b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f543e800640 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f543e800640 1 -- 172.21.15.27:0/3606049004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f2069650 con 0x5635f1fd51b0 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f543d400640 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f543e800640 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fd51b0 0x5635f23b0720 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5438011d10 tx=0x7f5438004810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8d3fe4f4870415 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.332+0000 7f541fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.333+0000 7f541fe00640 1 -- 172.21.15.27:0/3606049004 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543802c020 con 0x5635f1fd51b0 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.333+0000 7f541fe00640 1 -- 172.21.15.27:0/3606049004 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f5438004460 con 0x5635f1fd51b0 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.333+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5635f22a8b90 con 0x5635f1fd51b0 2024-09-21T07:19:49.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.333+0000 7f541fe00640 1 -- 172.21.15.27:0/3606049004 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5438037050 con 0x5635f1fd51b0 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.333+0000 7f544bed2740 10 client.27415 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.333+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635f22c3950 con 0x5635f1fd51b0 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.333+0000 7f541fe00640 1 -- 172.21.15.27:0/3606049004 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5635f22a8b90 con 0x5635f1fd51b0 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f541fe00640 1 -- 172.21.15.27:0/3606049004 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5438028570 con 0x5635f1fd51b0 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f544bed2740 10 client.27415 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f544bed2740 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f544bed2740 20 client.27415 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f544bed2740 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1fbc9f0 0x5635f23c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f544bed2740 4 client.27415 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:49.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635f231df00 con 0x5635f1fbc9f0 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.334+0000 7f543d400640 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1fbc9f0 0x5635f23c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.335+0000 7f541fe00640 10 client.27415 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.335+0000 7f543d400640 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1fbc9f0 0x5635f23c25f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b477dff3851060 server_cookie=738a595cb1b9e8ff in_seq=0 out_seq=0 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f541fe00640 1 -- 172.21.15.27:0/3606049004 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635f231df00 con 0x5635f1fbc9f0 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f541fe00640 10 client.27415 handle_command_reply: tid=0 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 client.27415 shutdown 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fd51b0 msgr2=0x5635f23b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fd51b0 0x5635f23b0720 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5438011d10 tx=0x7f5438004810 comp rx=0 tx=0).stop 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 shutdown_connections 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1fbc9f0 0x5635f23c25f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23bc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f541fe00640 0 client.27415 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 --2- 172.21.15.27:0/3606049004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fd51b0 0x5635f23b0720 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.336+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 >> 172.21.15.27:0/3606049004 conn(0x5635f1ff2460 msgr2=0x5635f23a7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.337+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 shutdown_connections 2024-09-21T07:19:49.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.337+0000 7f544bed2740 1 -- 172.21.15.27:0/3606049004 wait complete. 2024-09-21T07:19:49.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.337+0000 7f544bed2740 20 client.27415 trim_cache size 0 max 16384 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.351+0000 7f544bed2740 1 Processor -- start 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.351+0000 7f544bed2740 1 -- start start 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f544bed2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f544bed2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fdd390 0x5635f23a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f544bed2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f544bed2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54401b9d80 con 0x5635f1fd51b0 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f544bed2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54401b7650 con 0x5635f1ffe070 2024-09-21T07:19:49.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f544bed2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54401b54c0 con 0x5635f1fdd390 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f543e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f543d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fdd390 0x5635f23a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f543e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37634/0 (socket says 172.21.15.27:37634) 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f543e800640 1 -- 172.21.15.27:0/3164652355 learned_addr learned my addr 172.21.15.27:0/3164652355 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.352+0000 7f543e800640 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.353+0000 7f543e800640 1 -- 172.21.15.27:0/3164652355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fdd390 msgr2=0x5635f23a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.353+0000 7f543e800640 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fdd390 0x5635f23a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.353+0000 7f543e800640 1 -- 172.21.15.27:0/3164652355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 msgr2=0x5635f23ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.353+0000 7f543e800640 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.353+0000 7f543e800640 1 -- 172.21.15.27:0/3164652355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f23b17e0 con 0x5635f1fd51b0 2024-09-21T07:19:49.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.353+0000 7f543d400640 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fdd390 0x5635f23a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:49.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.353+0000 7f543e800640 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a2040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5438009560 tx=0x7f54380058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d735f6a152d15c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.354+0000 7f542aa00640 1 -- 172.21.15.27:0/3164652355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543802c040 con 0x5635f1fd51b0 2024-09-21T07:19:49.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.354+0000 7f542aa00640 1 -- 172.21.15.27:0/3164652355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438028bb0 con 0x5635f1fd51b0 2024-09-21T07:19:49.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.354+0000 7f542aa00640 1 -- 172.21.15.27:0/3164652355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438019690 con 0x5635f1fd51b0 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 -- 172.21.15.27:0/3164652355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 msgr2=0x5635f23a2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a2040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5438009560 tx=0x7f54380058d0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 -- 172.21.15.27:0/3164652355 shutdown_connections 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1fdd390 0x5635f23a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1ffe070 0x5635f23ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 --2- 172.21.15.27:0/3164652355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fd51b0 0x5635f23a2040 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 -- 172.21.15.27:0/3164652355 >> 172.21.15.27:0/3164652355 conn(0x5635f1ff2460 msgr2=0x5635f23aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 -- 172.21.15.27:0/3164652355 shutdown_connections 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.355+0000 7f544bed2740 1 -- 172.21.15.27:0/3164652355 wait complete. 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.356+0000 7f544bed2740 1 -- start start 2024-09-21T07:19:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.356+0000 7f544bed2740 1 Processor -- start 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f544bed2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fd51b0 0x5635f23b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f544bed2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f544bed2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f544bed2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54401b5b10 con 0x5635f1fdd390 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f544bed2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54401b6f80 con 0x5635f1fd51b0 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f544bed2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54401b61e0 con 0x5635f1ffe070 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f544bed2740 10 client.0 _ll_register_callbacks cb 0x7f5438005820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f543d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f543d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37650/0 (socket says 172.21.15.27:37650) 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f543d400640 1 -- 172.21.15.27:0/50223488 learned_addr learned my addr 172.21.15.27:0/50223488 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f543e800640 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fd51b0 0x5635f23b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.357+0000 7f543e800640 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f543d400640 1 -- 172.21.15.27:0/50223488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 msgr2=0x5635f23b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f543d400640 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f543d400640 1 -- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fd51b0 msgr2=0x5635f23b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f543d400640 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fd51b0 0x5635f23b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f543d400640 1 -- 172.21.15.27:0/50223488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f242bec0 con 0x5635f1fdd390 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f543d400640 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b4b10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f54401163a0 tx=0x7f542c012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87481a330079248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f541fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f541fe00640 1 -- 172.21.15.27:0/50223488 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c02c060 con 0x5635f1fdd390 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f541fe00640 1 -- 172.21.15.27:0/50223488 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f542c027670 con 0x5635f1fdd390 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.358+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5438028bb0 con 0x5635f1fdd390 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f541fe00640 1 -- 172.21.15.27:0/50223488 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f542c033050 con 0x5635f1fdd390 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f544bed2740 10 client.17421 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635f242d170 con 0x5635f1fdd390 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f541fe00640 1 -- 172.21.15.27:0/50223488 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438028bb0 con 0x5635f1fdd390 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f541fe00640 1 -- 172.21.15.27:0/50223488 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f542c00f3c0 con 0x5635f1fdd390 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f544bed2740 10 client.17421 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f544bed2740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.359+0000 7f544bed2740 20 client.17421 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.360+0000 7f544bed2740 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1f91da0 0x5635f242f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.360+0000 7f544bed2740 4 client.17421 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.360+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635f23199b0 con 0x5635f1f91da0 2024-09-21T07:19:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.360+0000 7f543e800640 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1f91da0 0x5635f242f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f543e800640 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1f91da0 0x5635f242f960 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d4f51d08fd6be6 server_cookie=7e88245cbdf7e3eb in_seq=0 out_seq=0 2024-09-21T07:19:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f541fe00640 10 client.17421 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f541fe00640 1 -- 172.21.15.27:0/50223488 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635f23199b0 con 0x5635f1f91da0 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f541fe00640 10 client.17421 handle_command_reply: tid=0 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 client.17421 shutdown 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 msgr2=0x5635f23b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b4b10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f54401163a0 tx=0x7f542c012f90 comp rx=0 tx=0).stop 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 shutdown_connections 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635f1f91da0 0x5635f242f960 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635f1ffe070 0x5635f23b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635f1fdd390 0x5635f23b4b10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 --2- 172.21.15.27:0/50223488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635f1fd51b0 0x5635f23b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 >> 172.21.15.27:0/50223488 conn(0x5635f1ff2460 msgr2=0x5635f23b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.361+0000 7f541fe00640 0 client.17421 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.362+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 shutdown_connections 2024-09-21T07:19:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.362+0000 7f544bed2740 1 -- 172.21.15.27:0/50223488 wait complete. 2024-09-21T07:19:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.362+0000 7f544bed2740 20 client.17421 trim_cache size 0 max 16384 2024-09-21T07:19:49.366 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:19:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f54240b6680 msgr2=0x7f54240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f54240b6680 0x7f54240b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5430006010 tx=0x7f543010b600 comp rx=0 tx=0).stop 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 msgr2=0x7f54401a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 0x7f54401a0b20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5430005940 tx=0x7f54300040c0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 shutdown_connections 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5440077380 0x7f54401a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f54240b6680 0x7f54240b8b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5440114440 0x7f54401a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 --2- 172.21.15.27:0/3371112227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5440075dd0 0x7f54401a0b20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 >> 172.21.15.27:0/3371112227 conn(0x7f54400ff5e0 msgr2=0x7f5440110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 shutdown_connections 2024-09-21T07:19:49.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:49.363+0000 7f5445200640 1 -- 172.21.15.27:0/3371112227 wait complete. 2024-09-21T07:19:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 -- 172.21.15.27:0/1293351477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c1050b0 msgr2=0x7f3c5c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 --2- 172.21.15.27:0/1293351477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c1050b0 0x7f3c5c10d5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009860 tx=0x7f3c4802a340 comp rx=0 tx=0).stop 2024-09-21T07:19:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 -- 172.21.15.27:0/1293351477 shutdown_connections 2024-09-21T07:19:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 --2- 172.21.15.27:0/1293351477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c1050b0 0x7f3c5c10d5f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 --2- 172.21.15.27:0/1293351477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c5c10dbc0 0x7f3c5c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 --2- 172.21.15.27:0/1293351477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c104760 0x7f3c5c104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 -- 172.21.15.27:0/1293351477 >> 172.21.15.27:0/1293351477 conn(0x7f3c5c0fe230 msgr2=0x7f3c5c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 -- 172.21.15.27:0/1293351477 shutdown_connections 2024-09-21T07:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.421+0000 7f3c5b200640 1 -- 172.21.15.27:0/1293351477 wait complete. 2024-09-21T07:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.422+0000 7f3c5b200640 1 Processor -- start 2024-09-21T07:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.422+0000 7f3c5b200640 1 -- start start 2024-09-21T07:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c104760 0x7f3c5c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c5c1050b0 0x7f3c5c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c10dbc0 0x7f3c5c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1137e0 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c113660 con 0x7f3c5c1050b0 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c113960 con 0x7f3c5c104760 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c10dbc0 0x7f3c5c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c10dbc0 0x7f3c5c1a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37666/0 (socket says 172.21.15.27:37666) 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c5a800640 1 -- 172.21.15.27:0/4104985005 learned_addr learned my addr 172.21.15.27:0/4104985005 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c59e00640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c104760 0x7f3c5c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.423+0000 7f3c59400640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c5c1050b0 0x7f3c5c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c5a800640 1 -- 172.21.15.27:0/4104985005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c104760 msgr2=0x7f3c5c19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c5a800640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c104760 0x7f3c5c19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c5a800640 1 -- 172.21.15.27:0/4104985005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c5c1050b0 msgr2=0x7f3c5c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c5a800640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c5c1050b0 0x7f3c5c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c5a800640 1 -- 172.21.15.27:0/4104985005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c1a23e0 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c5a800640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c10dbc0 0x7f3c5c1a1ce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400efd0 tx=0x7f3c5400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c52a00640 1 -- 172.21.15.27:0/4104985005 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54003b70 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c52a00640 1 -- 172.21.15.27:0/4104985005 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c54003d10 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5c1a26d0 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.424+0000 7f3c52a00640 1 -- 172.21.15.27:0/4104985005 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c540108a0 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.425+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5c1a8e60 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.428+0000 7f3c67e30740 1 Processor -- start 2024-09-21T07:19:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.429+0000 7f3c52a00640 1 -- 172.21.15.27:0/4104985005 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3c54004020 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c67e30740 1 -- start start 2024-09-21T07:19:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b951b0 0x5617e0f39e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b744d0 0x5617e0f3a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c67e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5617e0eaf6b0 con 0x5617e0b6c2f0 2024-09-21T07:19:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c67e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5617e0e9eae0 con 0x5617e0b744d0 2024-09-21T07:19:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c67e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5617e0ea28f0 con 0x5617e0b951b0 2024-09-21T07:19:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f3a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37676/0 (socket says 172.21.15.27:37676) 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c59e00640 1 -- 172.21.15.27:0/2214555492 learned_addr learned my addr 172.21.15.27:0/2214555492 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.430+0000 7f3c59400640 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b744d0 0x5617e0f3a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59400640 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b951b0 0x5617e0f39e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59e00640 1 -- 172.21.15.27:0/2214555492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b951b0 msgr2=0x5617e0f39e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59e00640 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b951b0 0x5617e0f39e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59e00640 1 -- 172.21.15.27:0/2214555492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b744d0 msgr2=0x5617e0f3a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59e00640 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b744d0 0x5617e0f3a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59e00640 1 -- 172.21.15.27:0/2214555492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617e0e6b780 con 0x5617e0b6c2f0 2024-09-21T07:19:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59400640 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b951b0 0x5617e0f39e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c52a00640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c400b6680 0x7f3c400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c59e00640 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f3a350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009cc0 tx=0x7f3c4c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d21578dfb7365c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.431+0000 7f3c52a00640 1 -- 172.21.15.27:0/4104985005 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f3c540e7ee0 con 0x7f3c5c10dbc0 2024-09-21T07:19:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.432+0000 7f3c59400640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c400b6680 0x7f3c400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.432+0000 7f3c47000640 1 -- 172.21.15.27:0/2214555492 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c005220 con 0x5617e0b6c2f0 2024-09-21T07:19:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.432+0000 7f3c47000640 1 -- 172.21.15.27:0/2214555492 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c004c50 con 0x5617e0b6c2f0 2024-09-21T07:19:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.432+0000 7f3c47000640 1 -- 172.21.15.27:0/2214555492 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0197f0 con 0x5617e0b6c2f0 2024-09-21T07:19:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 -- 172.21.15.27:0/2214555492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 msgr2=0x5617e0f3a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c59400640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c400b6680 0x7f3c400b8b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3c48035040 tx=0x7f3c48005780 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f3a350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009cc0 tx=0x7f3c4c011810 comp rx=0 tx=0).stop 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 -- 172.21.15.27:0/2214555492 shutdown_connections 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b744d0 0x5617e0f3a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f3a350 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2214555492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b951b0 0x5617e0f39e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 -- 172.21.15.27:0/2214555492 >> 172.21.15.27:0/2214555492 conn(0x5617e0b89300 msgr2=0x5617e0f37d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 -- 172.21.15.27:0/2214555492 shutdown_connections 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 -- 172.21.15.27:0/2214555492 wait complete. 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.433+0000 7f3c67e30740 1 -- start start 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 1 Processor -- start 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f45900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f45e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5617e0dd6ab0 con 0x5617e0b6c2f0 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5617e0e45570 con 0x5617e0b951b0 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5617e0f48590 con 0x5617e0b744d0 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c67e30740 10 client.0 _ll_register_callbacks cb 0x5617e0e54100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c5a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f45e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c5a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f45e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51020/0 (socket says 172.21.15.27:51020) 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c5a800640 1 -- 172.21.15.27:0/1239696479 learned_addr learned my addr 172.21.15.27:0/1239696479 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.434+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37680/0 (socket says 172.21.15.27:37680) 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c59400640 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f45900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c5a800640 1 -- 172.21.15.27:0/1239696479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 msgr2=0x5617e0f45900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c5a800640 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f45900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c5a800640 1 -- 172.21.15.27:0/1239696479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 msgr2=0x5617e0f453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c5a800640 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c5a800640 1 -- 172.21.15.27:0/1239696479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617e0c00790 con 0x5617e0b951b0 2024-09-21T07:19:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c59e00640 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c5a800640 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f45e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c540ad680 tx=0x7f3c540ad840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e95d3fa82cae41f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c3b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.435+0000 7f3c3b400640 1 -- 172.21.15.27:0/1239696479 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c540b2020 con 0x5617e0b951b0 2024-09-21T07:19:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.436+0000 7f3c3b400640 1 -- 172.21.15.27:0/1239696479 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f3c540acbe0 con 0x5617e0b951b0 2024-09-21T07:19:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.436+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4c004c50 con 0x5617e0b951b0 2024-09-21T07:19:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.436+0000 7f3c3b400640 1 -- 172.21.15.27:0/1239696479 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c540ae080 con 0x5617e0b951b0 2024-09-21T07:19:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.437+0000 7f3c67e30740 10 client.27464 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.437+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5617e0c79620 con 0x5617e0b951b0 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.437+0000 7f3c3b400640 1 -- 172.21.15.27:0/1239696479 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c004c50 con 0x5617e0b951b0 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.437+0000 7f3c3b400640 1 -- 172.21.15.27:0/1239696479 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c5401f020 con 0x5617e0b951b0 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.438+0000 7f3c67e30740 10 client.27464 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.438+0000 7f3c67e30740 10 client.27464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.438+0000 7f3c67e30740 20 client.27464 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.438+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b53d80 0x5617e0f57550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.438+0000 7f3c67e30740 4 client.27464 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.438+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617e0eb1600 con 0x5617e0b53d80 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.438+0000 7f3c59e00640 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b53d80 0x5617e0f57550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.439+0000 7f3c59e00640 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b53d80 0x5617e0f57550 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806f05f27024dd3c server_cookie=e9b72bb64a54867a in_seq=0 out_seq=0 2024-09-21T07:19:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.439+0000 7f3c3b400640 10 client.27464 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c3b400640 1 -- 172.21.15.27:0/1239696479 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617e0eb1600 con 0x5617e0b53d80 2024-09-21T07:19:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c3b400640 10 client.27464 handle_command_reply: tid=0 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 client.27464 shutdown 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 msgr2=0x5617e0f45e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f45e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c540ad680 tx=0x7f3c540ad840 comp rx=0 tx=0).stop 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 shutdown_connections 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b53d80 0x5617e0f57550 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f45900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f45e40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1239696479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 >> 172.21.15.27:0/1239696479 conn(0x5617e0b89300 msgr2=0x5617e0f3ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c3b400640 0 client.27464 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 shutdown_connections 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.440+0000 7f3c67e30740 1 -- 172.21.15.27:0/1239696479 wait complete. 2024-09-21T07:19:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.441+0000 7f3c67e30740 20 client.27464 trim_cache size 0 max 16384 2024-09-21T07:19:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.456+0000 7f3c67e30740 1 Processor -- start 2024-09-21T07:19:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.456+0000 7f3c67e30740 1 -- start start 2024-09-21T07:19:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f36a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f5d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c67e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1b5720 con 0x5617e0b6c2f0 2024-09-21T07:19:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c67e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1b2ff0 con 0x5617e0b951b0 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c67e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1b0e60 con 0x5617e0b744d0 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f36a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f36a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37692/0 (socket says 172.21.15.27:37692) 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 -- 172.21.15.27:0/1895777149 learned_addr learned my addr 172.21.15.27:0/1895777149 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59400640 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f5cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c5a800640 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f5d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 -- 172.21.15.27:0/1895777149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 msgr2=0x5617e0f5cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f5cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 -- 172.21.15.27:0/1895777149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 msgr2=0x5617e0f5d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f5d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.457+0000 7f3c59e00640 1 -- 172.21.15.27:0/1895777149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617e0f3abf0 con 0x5617e0b6c2f0 2024-09-21T07:19:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.458+0000 7f3c59e00640 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f36a70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009c90 tx=0x7f3c4c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2973ba8635a8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.458+0000 7f3c46600640 1 -- 172.21.15.27:0/1895777149 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c032030 con 0x5617e0b6c2f0 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.458+0000 7f3c46600640 1 -- 172.21.15.27:0/1895777149 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c009e70 con 0x5617e0b6c2f0 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.458+0000 7f3c46600640 1 -- 172.21.15.27:0/1895777149 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c02e710 con 0x5617e0b6c2f0 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 -- 172.21.15.27:0/1895777149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 msgr2=0x5617e0f36a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f36a70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009c90 tx=0x7f3c4c009260 comp rx=0 tx=0).stop 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 -- 172.21.15.27:0/1895777149 shutdown_connections 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f5cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0f5d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 --2- 172.21.15.27:0/1895777149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f36a70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 -- 172.21.15.27:0/1895777149 >> 172.21.15.27:0/1895777149 conn(0x5617e0b89300 msgr2=0x5617e0f36360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 -- 172.21.15.27:0/1895777149 shutdown_connections 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 -- 172.21.15.27:0/1895777149 wait complete. 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 -- start start 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.459+0000 7f3c67e30740 1 Processor -- start 2024-09-21T07:19:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c67e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0fb9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c67e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1b14b0 con 0x5617e0b6c2f0 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c67e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1b2920 con 0x5617e0b951b0 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c67e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1b1b80 con 0x5617e0b744d0 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c67e30740 10 client.0 _ll_register_callbacks cb 0x7f3c54004830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f4b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37704/0 (socket says 172.21.15.27:37704) 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c59e00640 1 -- 172.21.15.27:0/2461116515 learned_addr learned my addr 172.21.15.27:0/2461116515 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c5a800640 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0fb9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.460+0000 7f3c59400640 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f4b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c59e00640 1 -- 172.21.15.27:0/2461116515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 msgr2=0x5617e0f4b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c59e00640 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f4b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c59e00640 1 -- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 msgr2=0x5617e0fb9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c59e00640 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0fb9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c59e00640 1 -- 172.21.15.27:0/2461116515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617e0fba5c0 con 0x5617e0b6c2f0 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c59e00640 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f4b170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c000c00 tx=0x7f3c4c02e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1ef9b6a6bd92d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c3b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c3b400640 1 -- 172.21.15.27:0/2461116515 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c032040 con 0x5617e0b6c2f0 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c3b400640 1 -- 172.21.15.27:0/2461116515 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f3c4c009870 con 0x5617e0b6c2f0 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.461+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4c009e70 con 0x5617e0b6c2f0 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c3b400640 1 -- 172.21.15.27:0/2461116515 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c004020 con 0x5617e0b6c2f0 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 10 client.17439 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5617e0fc1900 con 0x5617e0b6c2f0 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c3b400640 1 -- 172.21.15.27:0/2461116515 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c009e70 con 0x5617e0b6c2f0 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c3b400640 1 -- 172.21.15.27:0/2461116515 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c4c02e6b0 con 0x5617e0b6c2f0 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 10 client.17439 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 20 client.17439 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b291c0 0x5617e0fc3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 4 client.17439 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.462+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617e0ead0b0 con 0x5617e0b291c0 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.463+0000 7f3c59400640 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b291c0 0x5617e0fc3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.463+0000 7f3c59400640 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b291c0 0x5617e0fc3ec0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29de4736d8b26925 server_cookie=f1474c313a9ce2ca in_seq=0 out_seq=0 2024-09-21T07:19:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.463+0000 7f3c3b400640 10 client.17439 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c3b400640 1 -- 172.21.15.27:0/2461116515 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617e0ead0b0 con 0x5617e0b291c0 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c3b400640 10 client.17439 handle_command_reply: tid=0 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 client.17439 shutdown 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 msgr2=0x5617e0f4b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f4b170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c000c00 tx=0x7f3c4c02e4a0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 shutdown_connections 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5617e0b291c0 0x5617e0fc3ec0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617e0b744d0 0x5617e0f4b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617e0b951b0 0x5617e0fb9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 --2- 172.21.15.27:0/2461116515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5617e0b6c2f0 0x5617e0f4b170 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c3b400640 0 client.17439 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.474+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 >> 172.21.15.27:0/2461116515 conn(0x5617e0b89300 msgr2=0x5617e0f36360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.475+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 shutdown_connections 2024-09-21T07:19:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.475+0000 7f3c67e30740 1 -- 172.21.15.27:0/2461116515 wait complete. 2024-09-21T07:19:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.475+0000 7f3c67e30740 20 client.17439 trim_cache size 0 max 16384 2024-09-21T07:19:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:19:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:19:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:19:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.475+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c400b6680 msgr2=0x7f3c400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.475+0000 7f3c5b200640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c400b6680 0x7f3c400b8b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3c48035040 tx=0x7f3c48005780 comp rx=0 tx=0).stop 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.475+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c10dbc0 msgr2=0x7f3c5c1a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.475+0000 7f3c5b200640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c10dbc0 0x7f3c5c1a1ce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400efd0 tx=0x7f3c5400c6a0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 shutdown_connections 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c400b6680 0x7f3c400b8b90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c5c10dbc0 0x7f3c5c1a1ce0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c5c1050b0 0x7f3c5c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 --2- 172.21.15.27:0/4104985005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c5c104760 0x7f3c5c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 >> 172.21.15.27:0/4104985005 conn(0x7f3c5c0fe230 msgr2=0x7f3c5c10cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 shutdown_connections 2024-09-21T07:19:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:19:49.476+0000 7f3c5b200640 1 -- 172.21.15.27:0/4104985005 wait complete. 2024-09-21T07:19:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:50 smithi119 ceph-mon[34519]: pgmap v584: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-21T07:19:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:50 smithi027 ceph-mon[30779]: pgmap v584: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-21T07:19:51.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:19:51.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:19:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:50 smithi153 ceph-mon[34747]: pgmap v584: 129 pgs: 129 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-21T07:19:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 -- 172.21.15.27:0/3388791523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 msgr2=0x7f93d0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 --2- 172.21.15.27:0/3388791523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d0105fd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009900 tx=0x7f93c402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 -- 172.21.15.27:0/3388791523 shutdown_connections 2024-09-21T07:19:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 --2- 172.21.15.27:0/3388791523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93d010b500 0x7f93d010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 --2- 172.21.15.27:0/3388791523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93d010be50 0x7f93d0112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 --2- 172.21.15.27:0/3388791523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d0105fd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 -- 172.21.15.27:0/3388791523 >> 172.21.15.27:0/3388791523 conn(0x7f93d0101670 msgr2=0x7f93d0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 -- 172.21.15.27:0/3388791523 shutdown_connections 2024-09-21T07:19:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.129+0000 7f93d6a00640 1 -- 172.21.15.27:0/3388791523 wait complete. 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.130+0000 7f93d6a00640 1 Processor -- start 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.130+0000 7f93d6a00640 1 -- start start 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93d010b500 0x7f93d019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93d010be50 0x7f93d01a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d6a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0115b60 con 0x7f93d0105bc0 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d6a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01159e0 con 0x7f93d010be50 2024-09-21T07:19:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d6a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0115ce0 con 0x7f93d010b500 2024-09-21T07:19:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37714/0 (socket says 172.21.15.27:37714) 2024-09-21T07:19:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d5600640 1 -- 172.21.15.27:0/242554325 learned_addr learned my addr 172.21.15.27:0/242554325 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d4c00640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93d010b500 0x7f93d019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.131+0000 7f93d6000640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93d010be50 0x7f93d01a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6000640 1 -- 172.21.15.27:0/242554325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93d010b500 msgr2=0x7f93d019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6000640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93d010b500 0x7f93d019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6000640 1 -- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 msgr2=0x7f93d019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6000640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6000640 1 -- 172.21.15.27:0/242554325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d01a2480 con 0x7f93d010be50 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d5600640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6000640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93d010be50 0x7f93d01a1d50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009fd0 tx=0x7f93c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93ce000640 1 -- 172.21.15.27:0/242554325 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c800ce20 con 0x7f93d010be50 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d01a2770 con 0x7f93d010be50 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93ce000640 1 -- 172.21.15.27:0/242554325 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c8005020 con 0x7f93d010be50 2024-09-21T07:19:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93ce000640 1 -- 172.21.15.27:0/242554325 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8010690 con 0x7f93d010be50 2024-09-21T07:19:52.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.132+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d01a8ed0 con 0x7f93d010be50 2024-09-21T07:19:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.136+0000 7f93dec6c740 1 Processor -- start 2024-09-21T07:19:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.136+0000 7f93ce000640 1 -- 172.21.15.27:0/242554325 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f93c80054d0 con 0x7f93d010be50 2024-09-21T07:19:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.136+0000 7f93dec6c740 1 -- start start 2024-09-21T07:19:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606869a2350 0x560686d49f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d4a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d4a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93dec6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560686cbfe90 con 0x560686981670 2024-09-21T07:19:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93dec6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560686caf2c0 con 0x5606869a2350 2024-09-21T07:19:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93dec6c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560686cb30d0 con 0x560686979490 2024-09-21T07:19:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93d4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d4a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93d4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606869a2350 0x560686d49f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93d5600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d4a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.137+0000 7f93d4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606869a2350 0x560686d49f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51058/0 (socket says 172.21.15.27:51058) 2024-09-21T07:19:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93d4c00640 1 -- 172.21.15.27:0/3179634834 learned_addr learned my addr 172.21.15.27:0/3179634834 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93d4c00640 1 -- 172.21.15.27:0/3179634834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 msgr2=0x560686d4a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3179634834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d4a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93d4c00640 1 -- 172.21.15.27:0/3179634834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 msgr2=0x560686d4a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3179634834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d4a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93d4c00640 1 -- 172.21.15.27:0/3179634834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560686a0d930 con 0x5606869a2350 2024-09-21T07:19:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3179634834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606869a2350 0x560686d49f00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93bc010900 tx=0x7f93bc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970bc8d73130bb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93ce000640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93b80b6680 0x7f93b80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93ce000640 1 -- 172.21.15.27:0/242554325 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f93c80e81b0 con 0x7f93d010be50 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.138+0000 7f93c2600640 1 -- 172.21.15.27:0/3179634834 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc016070 con 0x5606869a2350 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.139+0000 7f93c2600640 1 -- 172.21.15.27:0/3179634834 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc00e3c0 con 0x5606869a2350 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.139+0000 7f93c2600640 1 -- 172.21.15.27:0/3179634834 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc019040 con 0x5606869a2350 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.139+0000 7f93d5600640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93b80b6680 0x7f93b80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.139+0000 7f93dec6c740 1 -- 172.21.15.27:0/3179634834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606869a2350 msgr2=0x560686d49f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.139+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3179634834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606869a2350 0x560686d49f00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93bc010900 tx=0x7f93bc005f90 comp rx=0 tx=0).stop 2024-09-21T07:19:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.139+0000 7f93d5600640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93b80b6680 0x7f93b80b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x560686d48b80 tx=0x7f93c4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 -- 172.21.15.27:0/3179634834 shutdown_connections 2024-09-21T07:19:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3179634834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3179634834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d4a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3179634834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606869a2350 0x560686d49f00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 -- 172.21.15.27:0/3179634834 >> 172.21.15.27:0/3179634834 conn(0x560686996740 msgr2=0x560686d47df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 -- 172.21.15.27:0/3179634834 shutdown_connections 2024-09-21T07:19:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 -- 172.21.15.27:0/3179634834 wait complete. 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.140+0000 7f93dec6c740 1 -- start start 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 1 Processor -- start 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686979490 0x560686d55a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686981670 0x560686d55fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606869a2350 0x560686d61a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560686b10890 con 0x560686979490 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560686c4a160 con 0x560686981670 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560686d586e0 con 0x5606869a2350 2024-09-21T07:19:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.141+0000 7f93dec6c740 10 client.0 _ll_register_callbacks cb 0x560686c66a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686979490 0x560686d55a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686979490 0x560686d55a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37732/0 (socket says 172.21.15.27:37732) 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 -- 172.21.15.27:0/2019289665 learned_addr learned my addr 172.21.15.27:0/2019289665 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d4c00640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686981670 0x560686d55fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d6000640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606869a2350 0x560686d61a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 -- 172.21.15.27:0/2019289665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606869a2350 msgr2=0x560686d61a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606869a2350 0x560686d61a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 -- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686981670 msgr2=0x560686d55fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686981670 0x560686d55fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d5600640 1 -- 172.21.15.27:0/2019289665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560686d61fa0 con 0x560686979490 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.142+0000 7f93d4c00640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686981670 0x560686d55fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93d5600640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686979490 0x560686d55a60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93c4006010 tx=0x7f93c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727ffbcb16d9675f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:52.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93b6a00640 1 -- 172.21.15.27:0/2019289665 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4035040 con 0x560686979490 2024-09-21T07:19:52.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93b6a00640 1 -- 172.21.15.27:0/2019289665 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f93c4031b30 con 0x560686979490 2024-09-21T07:19:52.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93bc00e3c0 con 0x560686979490 2024-09-21T07:19:52.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93b6a00640 1 -- 172.21.15.27:0/2019289665 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93c4031d30 con 0x560686979490 2024-09-21T07:19:52.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93dec6c740 10 client.17451 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.143+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560686d648d0 con 0x560686979490 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93b6a00640 1 -- 172.21.15.27:0/2019289665 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc00e3c0 con 0x560686979490 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93b6a00640 1 -- 172.21.15.27:0/2019289665 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f93c4044030 con 0x560686979490 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93dec6c740 10 client.17451 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93dec6c740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93dec6c740 20 client.17451 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93dec6c740 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560686960cd0 0x560686d67e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93dec6c740 4 client.17451 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560686cc1de0 con 0x560686960cd0 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.144+0000 7f93d4c00640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560686960cd0 0x560686d67e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.145+0000 7f93d4c00640 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560686960cd0 0x560686d67e30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547413becfa9cf70 server_cookie=de96b1bf7489a04 in_seq=0 out_seq=0 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.145+0000 7f93b6a00640 10 client.17451 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93b6a00640 1 -- 172.21.15.27:0/2019289665 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560686cc1de0 con 0x560686960cd0 2024-09-21T07:19:52.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93b6a00640 10 client.17451 handle_command_reply: tid=0 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 client.17451 shutdown 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686979490 msgr2=0x560686d55a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686979490 0x560686d55a60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93c4006010 tx=0x7f93c4009260 comp rx=0 tx=0).stop 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 shutdown_connections 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560686960cd0 0x560686d67e30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606869a2350 0x560686d61a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686981670 0x560686d55fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 --2- 172.21.15.27:0/2019289665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686979490 0x560686d55a60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 >> 172.21.15.27:0/2019289665 conn(0x560686996740 msgr2=0x560686d4d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.146+0000 7f93b6a00640 0 client.17451 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.147+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 shutdown_connections 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.147+0000 7f93dec6c740 1 -- 172.21.15.27:0/2019289665 wait complete. 2024-09-21T07:19:52.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.147+0000 7f93dec6c740 20 client.17451 trim_cache size 0 max 16384 2024-09-21T07:19:52.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.165+0000 7f93dec6c740 1 Processor -- start 2024-09-21T07:19:52.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.165+0000 7f93dec6c740 1 -- start start 2024-09-21T07:19:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686996740 0x560686d4f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686979490 0x560686d4f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686981670 0x560686d473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93dec6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b5630 con 0x560686996740 2024-09-21T07:19:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93dec6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b2f00 con 0x560686979490 2024-09-21T07:19:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93dec6c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b0d70 con 0x560686981670 2024-09-21T07:19:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93d5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686996740 0x560686d4f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93d4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686979490 0x560686d4f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93d5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686996740 0x560686d4f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37740/0 (socket says 172.21.15.27:37740) 2024-09-21T07:19:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93d4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686979490 0x560686d4f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51074/0 (socket says 172.21.15.27:51074) 2024-09-21T07:19:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93d5600640 1 -- 172.21.15.27:0/3041298698 learned_addr learned my addr 172.21.15.27:0/3041298698 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.166+0000 7f93d6000640 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686981670 0x560686d473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93d4c00640 1 -- 172.21.15.27:0/3041298698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686981670 msgr2=0x560686d473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686981670 0x560686d473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93d4c00640 1 -- 172.21.15.27:0/3041298698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686996740 msgr2=0x560686d4f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686996740 0x560686d4f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93d4c00640 1 -- 172.21.15.27:0/3041298698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560686d58210 con 0x560686979490 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93d5600640 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686996740 0x560686d4f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686979490 0x560686d4f550 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00ee40 tx=0x7f93bc00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7885b2e14130dd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93c1c00640 1 -- 172.21.15.27:0/3041298698 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc02e070 con 0x560686979490 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93c1c00640 1 -- 172.21.15.27:0/3041298698 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc00e3c0 con 0x560686979490 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.167+0000 7f93c1c00640 1 -- 172.21.15.27:0/3041298698 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc016050 con 0x560686979490 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 -- 172.21.15.27:0/3041298698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686979490 msgr2=0x560686d4f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686979490 0x560686d4f550 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00ee40 tx=0x7f93bc00eee0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 -- 172.21.15.27:0/3041298698 shutdown_connections 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686981670 0x560686d473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686979490 0x560686d4f550 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3041298698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686996740 0x560686d4f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 -- 172.21.15.27:0/3041298698 >> 172.21.15.27:0/3041298698 conn(0x560686d56120 msgr2=0x560686d540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 -- 172.21.15.27:0/3041298698 shutdown_connections 2024-09-21T07:19:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.168+0000 7f93dec6c740 1 -- 172.21.15.27:0/3041298698 wait complete. 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.169+0000 7f93dec6c740 1 -- start start 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.169+0000 7f93dec6c740 1 Processor -- start 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d59f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93dec6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686996740 0x560686d5a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93dec6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b13c0 con 0x560686981670 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93dec6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b2830 con 0x560686996740 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93dec6c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b1a90 con 0x560686979490 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93dec6c740 10 client.0 _ll_register_callbacks cb 0x7f93c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686996740 0x560686d5a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d59f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d5600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d59f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37748/0 (socket says 172.21.15.27:37748) 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 -- 172.21.15.27:0/3374134399 learned_addr learned my addr 172.21.15.27:0/3374134399 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 -- 172.21.15.27:0/3374134399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 msgr2=0x560686d599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 -- 172.21.15.27:0/3374134399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686996740 msgr2=0x560686d5a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686996740 0x560686d5a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d4c00640 1 -- 172.21.15.27:0/3374134399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560686dd18b0 con 0x560686981670 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.170+0000 7f93d5600640 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93d4c00640 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d59f10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005fd0 tx=0x7f93bc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e859d82f23b5ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93b6a00640 1 -- 172.21.15.27:0/3374134399 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc037020 con 0x560686981670 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93b6a00640 1 -- 172.21.15.27:0/3374134399 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f93bc00e880 con 0x560686981670 2024-09-21T07:19:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93bc00e3c0 con 0x560686981670 2024-09-21T07:19:52.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93b6a00640 1 -- 172.21.15.27:0/3374134399 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93bc042050 con 0x560686981670 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93dec6c740 10 client.17457 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.171+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560686dd2b50 con 0x560686981670 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93b6a00640 1 -- 172.21.15.27:0/3374134399 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc00e3c0 con 0x560686981670 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93b6a00640 1 -- 172.21.15.27:0/3374134399 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f93bc00fb30 con 0x560686981670 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93dec6c740 10 client.17457 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93dec6c740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93dec6c740 20 client.17457 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606869a2350 0x560686dd50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93dec6c740 4 client.17457 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.172+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560686cbd890 con 0x5606869a2350 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.173+0000 7f93d5600640 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606869a2350 0x560686dd50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.173+0000 7f93d5600640 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606869a2350 0x560686dd50e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d191a7af9cecaeaf server_cookie=6ab577fb34dbd541 in_seq=0 out_seq=0 2024-09-21T07:19:52.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.173+0000 7f93b6a00640 10 client.17457 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:52.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93b6a00640 1 -- 172.21.15.27:0/3374134399 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560686cbd890 con 0x5606869a2350 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93b6a00640 10 client.17457 handle_command_reply: tid=0 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 client.17457 shutdown 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 msgr2=0x560686d59f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d59f10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005fd0 tx=0x7f93bc005db0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 shutdown_connections 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606869a2350 0x560686dd50e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560686996740 0x560686d5a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560686981670 0x560686d59f10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 --2- 172.21.15.27:0/3374134399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560686979490 0x560686d599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 >> 172.21.15.27:0/3374134399 conn(0x560686d56120 msgr2=0x560686d4af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93b6a00640 0 client.17457 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 shutdown_connections 2024-09-21T07:19:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.175+0000 7f93dec6c740 1 -- 172.21.15.27:0/3374134399 wait complete. 2024-09-21T07:19:52.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93dec6c740 20 client.17457 trim_cache size 0 max 16384 2024-09-21T07:19:52.179 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:52.179 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:19:52.179 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "7aef99b8-6e45-4b02-ae58-adedbae905f2", 2024-09-21T07:19:52.179 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:19:52.179 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93b80b6680 msgr2=0x7f93b80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93b80b6680 0x7f93b80b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x560686d48b80 tx=0x7f93c4005a20 comp rx=0 tx=0).stop 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93d010be50 msgr2=0x7f93d01a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93d010be50 0x7f93d01a1d50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009fd0 tx=0x7f93c800ef10 comp rx=0 tx=0).stop 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 shutdown_connections 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93d010b500 0x7f93d019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93b80b6680 0x7f93b80b8b90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93d010be50 0x7f93d01a1d50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.176+0000 7f93d6a00640 1 --2- 172.21.15.27:0/242554325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d0105bc0 0x7f93d019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.177+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 >> 172.21.15.27:0/242554325 conn(0x7f93d0101670 msgr2=0x7f93d010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.177+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 shutdown_connections 2024-09-21T07:19:52.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:52.177+0000 7f93d6a00640 1 -- 172.21.15.27:0/242554325 wait complete. 2024-09-21T07:19:53.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:52 smithi027 ceph-mon[30779]: pgmap v585: 129 pgs: 129 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-21T07:19:53.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:52 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:19:53.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:52 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:53.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:52 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:19:53.008 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:52 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:52 smithi119 ceph-mon[34519]: pgmap v585: 129 pgs: 129 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-21T07:19:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:52 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:19:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:52 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:52 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:19:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:52 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:53.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:19:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:52 smithi153 ceph-mon[34747]: pgmap v585: 129 pgs: 129 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-21T07:19:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:52 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:19:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:52 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:52 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:19:53.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:52 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:19:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:53.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:53 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:19:54.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:53 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 -- 172.21.15.27:0/3677586946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b0105d90 msgr2=0x7fc8b01061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 --2- 172.21.15.27:0/3677586946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b0105d90 0x7fc8b01061a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009900 tx=0x7fc8a402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 -- 172.21.15.27:0/3677586946 shutdown_connections 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 --2- 172.21.15.27:0/3677586946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b010b6d0 0x7fc8b010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 --2- 172.21.15.27:0/3677586946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 0x7fc8b0112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 --2- 172.21.15.27:0/3677586946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b0105d90 0x7fc8b01061a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 -- 172.21.15.27:0/3677586946 >> 172.21.15.27:0/3677586946 conn(0x7fc8b0101860 msgr2=0x7fc8b0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 -- 172.21.15.27:0/3677586946 shutdown_connections 2024-09-21T07:19:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.077+0000 7fc8b8400640 1 -- 172.21.15.27:0/3677586946 wait complete. 2024-09-21T07:19:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 Processor -- start 2024-09-21T07:19:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 -- start start 2024-09-21T07:19:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b0105d90 0x7fc8b019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b010b6d0 0x7fc8b019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 0x7fc8b01a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b0115ce0 con 0x7fc8b010b6d0 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b0115b60 con 0x7fc8b010c020 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b8400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b0115e60 con 0x7fc8b0105d90 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.079+0000 7fc8b5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b010b6d0 0x7fc8b019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 0x7fc8b01a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 0x7fc8b01a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51090/0 (socket says 172.21.15.27:51090) 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b5a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b0105d90 0x7fc8b019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b010b6d0 0x7fc8b019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37766/0 (socket says 172.21.15.27:37766) 2024-09-21T07:19:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 -- 172.21.15.27:0/2550642970 learned_addr learned my addr 172.21.15.27:0/2550642970 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 -- 172.21.15.27:0/2550642970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b0105d90 msgr2=0x7fc8b019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b0105d90 0x7fc8b019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 -- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b010b6d0 msgr2=0x7fc8b019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b010b6d0 0x7fc8b019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b6400640 1 -- 172.21.15.27:0/2550642970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b01a2680 con 0x7fc8b010c020 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b0105d90 0x7fc8b019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.080+0000 7fc8b5000640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b010b6d0 0x7fc8b019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.081+0000 7fc8b6400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 0x7fc8b01a1f50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009fd0 tx=0x7fc8a800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.081+0000 7fc8ae000640 1 -- 172.21.15.27:0/2550642970 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8017070 con 0x7fc8b010c020 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.081+0000 7fc8ae000640 1 -- 172.21.15.27:0/2550642970 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a8003e20 con 0x7fc8b010c020 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.081+0000 7fc8ae000640 1 -- 172.21.15.27:0/2550642970 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8010460 con 0x7fc8b010c020 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.081+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b01a2970 con 0x7fc8b010c020 2024-09-21T07:19:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.081+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b01a90d0 con 0x7fc8b010c020 2024-09-21T07:19:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.084+0000 7fc8bf0d9740 1 Processor -- start 2024-09-21T07:19:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.085+0000 7fc8ae000640 1 -- 172.21.15.27:0/2550642970 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc8a8003970 con 0x7fc8b010c020 2024-09-21T07:19:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.085+0000 7fc8bf0d9740 1 -- start start 2024-09-21T07:19:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eaf2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eaf30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55852ea69f70 con 0x55852e7231b0 2024-09-21T07:19:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55852ea593a0 con 0x55852e72b390 2024-09-21T07:19:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55852ea5d1b0 con 0x55852e74c070 2024-09-21T07:19:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8b5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eaf30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8b5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eaf30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37770/0 (socket says 172.21.15.27:37770) 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8b5a00640 1 -- 172.21.15.27:0/2203405959 learned_addr learned my addr 172.21.15.27:0/2203405959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8b5000640 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.086+0000 7fc8b5000640 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eaf2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5a00640 1 -- 172.21.15.27:0/2203405959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 msgr2=0x55852eaf2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eaf2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5a00640 1 -- 172.21.15.27:0/2203405959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 msgr2=0x55852eaf35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5a00640 1 -- 172.21.15.27:0/2203405959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852ea23c90 con 0x55852e7231b0 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5000640 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8ae000640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8980b6700 0x7fc8980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eaf30a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009900 tx=0x7fc8a4031550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b38fbf0676a533 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:54.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8ae000640 1 -- 172.21.15.27:0/2550642970 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fc8a80e7780 con 0x7fc8b010c020 2024-09-21T07:19:54.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.087+0000 7fc8b5000640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8980b6700 0x7fc8980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.088+0000 7fc8a2a00640 1 -- 172.21.15.27:0/2203405959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4040070 con 0x55852e7231b0 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.088+0000 7fc8a2a00640 1 -- 172.21.15.27:0/2203405959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a40318c0 con 0x55852e7231b0 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.088+0000 7fc8a2a00640 1 -- 172.21.15.27:0/2203405959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4035040 con 0x55852e7231b0 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.088+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/2203405959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 msgr2=0x55852eaf30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.088+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eaf30a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009900 tx=0x7fc8a4031550 comp rx=0 tx=0).stop 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.088+0000 7fc8b5000640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8980b6700 0x7fc8980b8c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00bfd0 tx=0x7fc89c005c90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/2203405959 shutdown_connections 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eaf30a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/2203405959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eaf2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/2203405959 >> 172.21.15.27:0/2203405959 conn(0x55852e740460 msgr2=0x55852eaf0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/2203405959 shutdown_connections 2024-09-21T07:19:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/2203405959 wait complete. 2024-09-21T07:19:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.089+0000 7fc8bf0d9740 1 -- start start 2024-09-21T07:19:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 1 Processor -- start 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eafe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eafed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb0a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55852e8ba700 con 0x55852e7231b0 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55852e9f4540 con 0x55852e72b390 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55852eb012e0 con 0x55852e74c070 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8bf0d9740 10 client.0 _ll_register_callbacks cb 0x55852ea04430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.090+0000 7fc8b5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eafe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eafed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb0a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eafe7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37778/0 (socket says 172.21.15.27:37778) 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb0a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42258/0 (socket says 172.21.15.27:42258) 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eafed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51116/0 (socket says 172.21.15.27:51116) 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5a00640 1 -- 172.21.15.27:0/92383119 learned_addr learned my addr 172.21.15.27:0/92383119 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 -- 172.21.15.27:0/92383119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 msgr2=0x55852eb0a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb0a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 -- 172.21.15.27:0/92383119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 msgr2=0x55852eafe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eafe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 -- 172.21.15.27:0/92383119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852e7b7650 con 0x55852e72b390 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc8b5000640 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eafed40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc8b006b1e0 tx=0x7fc89c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1127038dec5026c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.091+0000 7fc897400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.092+0000 7fc897400640 1 -- 172.21.15.27:0/92383119 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c010a40 con 0x55852e72b390 2024-09-21T07:19:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.092+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55852e9f6b50 con 0x55852e72b390 2024-09-21T07:19:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.092+0000 7fc897400640 1 -- 172.21.15.27:0/92383119 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fc89c010be0 con 0x55852e72b390 2024-09-21T07:19:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.092+0000 7fc897400640 1 -- 172.21.15.27:0/92383119 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc89c034050 con 0x55852e72b390 2024-09-21T07:19:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.092+0000 7fc8bf0d9740 10 client.27500 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.092+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55852ea11910 con 0x55852e72b390 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.092+0000 7fc897400640 1 -- 172.21.15.27:0/92383119 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55852e9f6b50 con 0x55852e72b390 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc897400640 1 -- 172.21.15.27:0/92383119 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc89c021420 con 0x55852e72b390 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc8bf0d9740 10 client.27500 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc8bf0d9740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc8bf0d9740 20 client.27500 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e70a9f0 0x55852eb10730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc8bf0d9740 4 client.27500 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55852ea6bec0 con 0x55852e70a9f0 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.093+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e70a9f0 0x55852eb10730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.094+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e70a9f0 0x55852eb10730 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a96faaed1ed150c server_cookie=748c16a56016fc3 in_seq=0 out_seq=0 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.094+0000 7fc897400640 10 client.27500 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc897400640 1 -- 172.21.15.27:0/92383119 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55852ea6bec0 con 0x55852e70a9f0 2024-09-21T07:19:54.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc897400640 10 client.27500 handle_command_reply: tid=0 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 client.27500 shutdown 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 msgr2=0x55852eafed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eafed40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc8b006b1e0 tx=0x7fc89c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 shutdown_connections 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e70a9f0 0x55852eb10730 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb0a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eafed40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/92383119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e7231b0 0x55852eafe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 >> 172.21.15.27:0/92383119 conn(0x55852e740460 msgr2=0x55852eaf5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.095+0000 7fc897400640 0 client.27500 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.096+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 shutdown_connections 2024-09-21T07:19:54.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.096+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/92383119 wait complete. 2024-09-21T07:19:54.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.096+0000 7fc8bf0d9740 20 client.27500 trim_cache size 0 max 16384 2024-09-21T07:19:54.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 Processor -- start 2024-09-21T07:19:54.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 -- start start 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e7231b0 0x55852eb0a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e74c070 0x55852eaf81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b01b5a00 con 0x55852e74c070 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b01b32d0 con 0x55852e72b390 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.114+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b01b1140 con 0x55852e7231b0 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e74c070 0x55852eaf81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e74c070 0x55852eaf81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37780/0 (socket says 172.21.15.27:37780) 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b6400640 1 -- 172.21.15.27:0/1330045043 learned_addr learned my addr 172.21.15.27:0/1330045043 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b5000640 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e7231b0 0x55852eb0a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b5000640 1 -- 172.21.15.27:0/1330045043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e7231b0 msgr2=0x55852eb0a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b5000640 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e7231b0 0x55852eb0a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b5000640 1 -- 172.21.15.27:0/1330045043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e74c070 msgr2=0x55852eaf81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b5000640 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e74c070 0x55852eaf81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b5000640 1 -- 172.21.15.27:0/1330045043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852e70ac10 con 0x55852e72b390 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.115+0000 7fc8b6400640 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e74c070 0x55852eaf81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.116+0000 7fc8b5000640 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf7c70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc89c005d00 tx=0x7fc89c00e830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa20a3a90148ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.116+0000 7fc8a2000640 1 -- 172.21.15.27:0/1330045043 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c012070 con 0x55852e72b390 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.116+0000 7fc8a2000640 1 -- 172.21.15.27:0/1330045043 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c012210 con 0x55852e72b390 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.116+0000 7fc8a2000640 1 -- 172.21.15.27:0/1330045043 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c021e20 con 0x55852e72b390 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1330045043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 msgr2=0x55852eaf7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf7c70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc89c005d00 tx=0x7fc89c00e830 comp rx=0 tx=0).stop 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1330045043 shutdown_connections 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e74c070 0x55852eaf81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e72b390 0x55852eaf7c70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1330045043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e7231b0 0x55852eb0a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1330045043 >> 172.21.15.27:0/1330045043 conn(0x55852e740460 msgr2=0x55852eafce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1330045043 shutdown_connections 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1330045043 wait complete. 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 -- start start 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.117+0000 7fc8bf0d9740 1 Processor -- start 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e7231b0 0x55852eb02dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e72b390 0x55852eb03350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8bf0d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb7a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b01b1790 con 0x55852e72b390 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b01b2c00 con 0x55852e7231b0 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8bf0d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b01b1e60 con 0x55852e74c070 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8bf0d9740 10 client.0 _ll_register_callbacks cb 0x7fc89c00e9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8b5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e72b390 0x55852eb03350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8b5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e7231b0 0x55852eb02dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8b5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e72b390 0x55852eb03350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37792/0 (socket says 172.21.15.27:37792) 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8b5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e7231b0 0x55852eb02dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51136/0 (socket says 172.21.15.27:51136) 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.118+0000 7fc8b5000640 1 -- 172.21.15.27:0/1013387253 learned_addr learned my addr 172.21.15.27:0/1013387253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc8b6400640 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb7a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc8b5a00640 1 -- 172.21.15.27:0/1013387253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 msgr2=0x55852eb7a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb7a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc8b5a00640 1 -- 172.21.15.27:0/1013387253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e72b390 msgr2=0x55852eb03350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:54.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e72b390 0x55852eb03350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc8b5a00640 1 -- 172.21.15.27:0/1013387253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852eb7a700 con 0x55852e7231b0 2024-09-21T07:19:54.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc8b5a00640 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e7231b0 0x55852eb02dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55852eb10e60 tx=0x7fc8a402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70beaa47a8418b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:54.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.119+0000 7fc897400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc897400640 1 -- 172.21.15.27:0/1013387253 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4035040 con 0x55852e7231b0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c012210 con 0x55852e7231b0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc897400640 1 -- 172.21.15.27:0/1013387253 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fc8a4004240 con 0x55852e7231b0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc897400640 1 -- 172.21.15.27:0/1013387253 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8a405e050 con 0x55852e7231b0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc8bf0d9740 10 client.27506 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55852eb7b9f0 con 0x55852e7231b0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc897400640 1 -- 172.21.15.27:0/1013387253 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c012210 con 0x55852e7231b0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc897400640 1 -- 172.21.15.27:0/1013387253 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8a404f420 con 0x55852e7231b0 2024-09-21T07:19:54.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc8bf0d9740 10 client.27506 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc8bf0d9740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.120+0000 7fc8bf0d9740 20 client.27506 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.121+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e6dfda0 0x55852eb7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.121+0000 7fc8bf0d9740 4 client.27506 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.121+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55852eaff0d0 con 0x55852e6dfda0 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.121+0000 7fc8b5000640 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e6dfda0 0x55852eb7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.121+0000 7fc8b5000640 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e6dfda0 0x55852eb7dd40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ae82f229746636 server_cookie=c742062ae240e4a9 in_seq=0 out_seq=0 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.121+0000 7fc897400640 10 client.27506 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc897400640 1 -- 172.21.15.27:0/1013387253 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55852eaff0d0 con 0x55852e6dfda0 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc897400640 10 client.27506 handle_command_reply: tid=0 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 client.27506 shutdown 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e7231b0 msgr2=0x55852eb02dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e7231b0 0x55852eb02dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55852eb10e60 tx=0x7fc8a402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 shutdown_connections 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55852e6dfda0 0x55852eb7dd40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55852e74c070 0x55852eb7a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55852e72b390 0x55852eb03350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 --2- 172.21.15.27:0/1013387253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55852e7231b0 0x55852eb02dd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 >> 172.21.15.27:0/1013387253 conn(0x55852e740460 msgr2=0x55852eaf7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc897400640 0 client.27506 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 shutdown_connections 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.122+0000 7fc8bf0d9740 1 -- 172.21.15.27:0/1013387253 wait complete. 2024-09-21T07:19:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8bf0d9740 20 client.27506 trim_cache size 0 max 16384 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:19:54.131 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:19:54.132 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:19:54.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8980b6700 msgr2=0x7fc8980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8980b6700 0x7fc8980b8c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00bfd0 tx=0x7fc89c005c90 comp rx=0 tx=0).stop 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 msgr2=0x7fc8b01a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 0x7fc8b01a1f50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009fd0 tx=0x7fc8a800b260 comp rx=0 tx=0).stop 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 shutdown_connections 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8980b6700 0x7fc8980b8c10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8b010c020 0x7fc8b01a1f50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b010b6d0 0x7fc8b019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.123+0000 7fc8b8400640 1 --2- 172.21.15.27:0/2550642970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8b0105d90 0x7fc8b019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:54.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.124+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 >> 172.21.15.27:0/2550642970 conn(0x7fc8b0101860 msgr2=0x7fc8b0103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.124+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 shutdown_connections 2024-09-21T07:19:54.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:54.124+0000 7fc8b8400640 1 -- 172.21.15.27:0/2550642970 wait complete. 2024-09-21T07:19:54.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:53 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:19:54.784 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:54 smithi027 ceph-mon[30779]: pgmap v586: 129 pgs: 129 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:19:54.824 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:19:54.824 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:19:55.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:54 smithi119 ceph-mon[34519]: pgmap v586: 129 pgs: 129 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:19:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:54 smithi153 ceph-mon[34747]: pgmap v586: 129 pgs: 129 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:19:55.209 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:55.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 -- 172.21.15.27:0/2870389256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410be40 msgr2=0x7fd564112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:55.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 --2- 172.21.15.27:0/2870389256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410be40 0x7fd564112670 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009ca0 tx=0x7fd55c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 -- 172.21.15.27:0/2870389256 shutdown_connections 2024-09-21T07:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 --2- 172.21.15.27:0/2870389256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd56410b4f0 0x7fd56410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 --2- 172.21.15.27:0/2870389256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410be40 0x7fd564112670 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 --2- 172.21.15.27:0/2870389256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564105bb0 0x7fd564105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 -- 172.21.15.27:0/2870389256 >> 172.21.15.27:0/2870389256 conn(0x7fd564101680 msgr2=0x7fd564103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 -- 172.21.15.27:0/2870389256 shutdown_connections 2024-09-21T07:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.811+0000 7fd563e00640 1 -- 172.21.15.27:0/2870389256 wait complete. 2024-09-21T07:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.812+0000 7fd563e00640 1 Processor -- start 2024-09-21T07:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.812+0000 7fd563e00640 1 -- start start 2024-09-21T07:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd563e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564105bb0 0x7fd56419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd563e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410b4f0 0x7fd56419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd563e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd56410be40 0x7fd5641a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd563e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd564115b00 con 0x7fd56410b4f0 2024-09-21T07:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd563e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd564115980 con 0x7fd564105bb0 2024-09-21T07:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd563e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd564115c80 con 0x7fd56410be40 2024-09-21T07:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd55a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410b4f0 0x7fd56419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd55a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410b4f0 0x7fd56419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37802/0 (socket says 172.21.15.27:37802) 2024-09-21T07:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd55a000640 1 -- 172.21.15.27:0/3398651481 learned_addr learned my addr 172.21.15.27:0/3398651481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd562a00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564105bb0 0x7fd56419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.813+0000 7fd563400640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd56410be40 0x7fd5641a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55a000640 1 -- 172.21.15.27:0/3398651481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd56410be40 msgr2=0x7fd5641a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55a000640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd56410be40 0x7fd5641a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55a000640 1 -- 172.21.15.27:0/3398651481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564105bb0 msgr2=0x7fd56419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55a000640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564105bb0 0x7fd56419c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55a000640 1 -- 172.21.15.27:0/3398651481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5641a2420 con 0x7fd56410b4f0 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd562a00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564105bb0 0x7fd56419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55a000640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410b4f0 0x7fd56419ca50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9c0 tx=0x7fd54c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55be00640 1 -- 172.21.15.27:0/3398651481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00ccc0 con 0x7fd56410b4f0 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5641a2710 con 0x7fd56410b4f0 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55be00640 1 -- 172.21.15.27:0/3398651481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd54c0054d0 con 0x7fd56410b4f0 2024-09-21T07:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd55be00640 1 -- 172.21.15.27:0/3398651481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c010430 con 0x7fd56410b4f0 2024-09-21T07:19:55.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.814+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641a8e70 con 0x7fd56410b4f0 2024-09-21T07:19:55.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.816+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd564105fc0 con 0x7fd56410b4f0 2024-09-21T07:19:55.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.818+0000 7fd55be00640 1 -- 172.21.15.27:0/3398651481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd54c005000 con 0x7fd56410b4f0 2024-09-21T07:19:55.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.820+0000 7fd55be00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd5480b6700 0x7fd5480b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:55.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.820+0000 7fd55be00640 1 -- 172.21.15.27:0/3398651481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fd54c0e7fe0 con 0x7fd56410b4f0 2024-09-21T07:19:55.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.820+0000 7fd562a00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd5480b6700 0x7fd5480b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:55.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.821+0000 7fd562a00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd5480b6700 0x7fd5480b8c10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd554009fc0 tx=0x7fd554005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:55.822+0000 7fd55be00640 1 -- 172.21.15.27:0/3398651481 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd54c056710 con 0x7fd56410b4f0 2024-09-21T07:19:56.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.060+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd56419d5e0 con 0x7fd56410b4f0 2024-09-21T07:19:56.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.061+0000 7fd55be00640 1 -- 172.21.15.27:0/3398651481 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fd56419d5e0 con 0x7fd56410b4f0 2024-09-21T07:19:56.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:19:56.063 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:19:56.063 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:19:56.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd5480b6700 msgr2=0x7fd5480b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:56.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd5480b6700 0x7fd5480b8c10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd554009fc0 tx=0x7fd554005f90 comp rx=0 tx=0).stop 2024-09-21T07:19:56.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410b4f0 msgr2=0x7fd56419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:56.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410b4f0 0x7fd56419ca50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9c0 tx=0x7fd54c00ee80 comp rx=0 tx=0).stop 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 shutdown_connections 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd56410be40 0x7fd5641a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd5480b6700 0x7fd5480b8c10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd56410b4f0 0x7fd56419ca50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 --2- 172.21.15.27:0/3398651481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564105bb0 0x7fd56419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.063+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 >> 172.21.15.27:0/3398651481 conn(0x7fd564101680 msgr2=0x7fd564109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.064+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 shutdown_connections 2024-09-21T07:19:56.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:56.064+0000 7fd563e00640 1 -- 172.21.15.27:0/3398651481 wait complete. 2024-09-21T07:19:56.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:19:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:56 smithi119 ceph-mon[34519]: pgmap v587: 129 pgs: 129 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:19:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:56 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3398651481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:56 smithi027 ceph-mon[30779]: pgmap v587: 129 pgs: 129 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:19:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:56 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3398651481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:56 smithi153 ceph-mon[34747]: pgmap v587: 129 pgs: 129 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:19:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:56 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3398651481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:19:57.185 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:57.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 -- 172.21.15.27:0/2154517931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c075750 msgr2=0x7ff71c075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 --2- 172.21.15.27:0/2154517931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c075750 0x7ff71c075b60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff710009900 tx=0x7ff71002a410 comp rx=0 tx=0).stop 2024-09-21T07:19:57.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 -- 172.21.15.27:0/2154517931 shutdown_connections 2024-09-21T07:19:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 --2- 172.21.15.27:0/2154517931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff71c0760a0 0x7ff71c10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 --2- 172.21.15.27:0/2154517931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71c10ff50 0x7ff71c112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 --2- 172.21.15.27:0/2154517931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c075750 0x7ff71c075b60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 -- 172.21.15.27:0/2154517931 >> 172.21.15.27:0/2154517931 conn(0x7ff71c0ff440 msgr2=0x7ff71c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 -- 172.21.15.27:0/2154517931 shutdown_connections 2024-09-21T07:19:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.774+0000 7ff722a00640 1 -- 172.21.15.27:0/2154517931 wait complete. 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.775+0000 7ff722a00640 1 Processor -- start 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.775+0000 7ff722a00640 1 -- start start 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff722a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71c075750 0x7ff71c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff722a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c0760a0 0x7ff71c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff722a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff71c10ff50 0x7ff71c1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff722a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c115ac0 con 0x7ff71c0760a0 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff722a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c115940 con 0x7ff71c075750 2024-09-21T07:19:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff722a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c115c40 con 0x7ff71c10ff50 2024-09-21T07:19:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff720c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c0760a0 0x7ff71c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff721600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71c075750 0x7ff71c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff720c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c0760a0 0x7ff71c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37830/0 (socket says 172.21.15.27:37830) 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff720c00640 1 -- 172.21.15.27:0/3425475438 learned_addr learned my addr 172.21.15.27:0/3425475438 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.776+0000 7ff722000640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff71c10ff50 0x7ff71c1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff721600640 1 -- 172.21.15.27:0/3425475438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff71c10ff50 msgr2=0x7ff71c1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff721600640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff71c10ff50 0x7ff71c1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff721600640 1 -- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c0760a0 msgr2=0x7ff71c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff721600640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c0760a0 0x7ff71c19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff721600640 1 -- 172.21.15.27:0/3425475438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c1a2440 con 0x7ff71c075750 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff721600640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71c075750 0x7ff71c19c4d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff710006010 tx=0x7ff710004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff71a000640 1 -- 172.21.15.27:0/3425475438 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71003f070 con 0x7ff71c075750 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff71a000640 1 -- 172.21.15.27:0/3425475438 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710003ca0 con 0x7ff71c075750 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c1a26d0 con 0x7ff71c075750 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.777+0000 7ff71a000640 1 -- 172.21.15.27:0/3425475438 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710035070 con 0x7ff71c075750 2024-09-21T07:19:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.778+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1a8e90 con 0x7ff71c075750 2024-09-21T07:19:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 Processor -- start 2024-09-21T07:19:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 -- start start 2024-09-21T07:19:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff71a000640 1 -- 172.21.15.27:0/3425475438 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff71003b460 con 0x7ff71c075750 2024-09-21T07:19:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf88f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ab0befffb0 con 0x55ab0bbe2070 2024-09-21T07:19:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ab0beef3e0 con 0x55ab0bbc1390 2024-09-21T07:19:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.781+0000 7ff72ac66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab0bef31f0 con 0x55ab0bbb91b0 2024-09-21T07:19:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf88f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf88f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37842/0 (socket says 172.21.15.27:37842) 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 -- 172.21.15.27:0/2178636740 learned_addr learned my addr 172.21.15.27:0/2178636740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff721600640 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff720c00640 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 -- 172.21.15.27:0/2178636740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 msgr2=0x55ab0bf894b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf894b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 -- 172.21.15.27:0/2178636740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 msgr2=0x55ab0bf899f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf899f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.782+0000 7ff722000640 1 -- 172.21.15.27:0/2178636740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0beb9cd0 con 0x55ab0bbe2070 2024-09-21T07:19:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.783+0000 7ff722000640 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf88f70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff714009fd0 tx=0x7ff7140118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3718408256062909 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.783+0000 7ff70e600640 1 -- 172.21.15.27:0/2178636740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714004d20 con 0x55ab0bbe2070 2024-09-21T07:19:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.783+0000 7ff70e600640 1 -- 172.21.15.27:0/2178636740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714004050 con 0x55ab0bbe2070 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.783+0000 7ff70e600640 1 -- 172.21.15.27:0/2178636740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714005280 con 0x55ab0bbe2070 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.783+0000 7ff71a000640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff7040b6680 0x7ff7040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff71a000640 1 -- 172.21.15.27:0/3425475438 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7ff710107f70 con 0x7ff71c075750 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff721600640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff7040b6680 0x7ff7040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 -- 172.21.15.27:0/2178636740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 msgr2=0x55ab0bf88f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf88f70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff714009fd0 tx=0x7ff7140118d0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 -- 172.21.15.27:0/2178636740 shutdown_connections 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf894b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf899f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2178636740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf88f70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 -- 172.21.15.27:0/2178636740 >> 172.21.15.27:0/2178636740 conn(0x55ab0bbd6460 msgr2=0x55ab0bf86e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 -- 172.21.15.27:0/2178636740 shutdown_connections 2024-09-21T07:19:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff72ac66740 1 -- 172.21.15.27:0/2178636740 wait complete. 2024-09-21T07:19:57.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.784+0000 7ff721600640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff7040b6680 0x7ff7040b8b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55ab0bf87bf0 tx=0x7ff71010ba20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.785+0000 7ff72ac66740 1 -- start start 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.785+0000 7ff72ac66740 1 Processor -- start 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbc1390 0x55ab0bf94c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0bfa0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff72ac66740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ab0bd50700 con 0x55ab0bbb91b0 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff72ac66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ab0be8a580 con 0x55ab0bbe2070 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff72ac66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab0bf97750 con 0x55ab0bbc1390 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff72ac66740 10 client.0 _ll_register_callbacks cb 0x55ab0be9a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff722000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff722000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf946f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37850/0 (socket says 172.21.15.27:37850) 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff722000640 1 -- 172.21.15.27:0/3402472201 learned_addr learned my addr 172.21.15.27:0/3402472201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff720c00640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbc1390 0x55ab0bf94c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.786+0000 7ff722000640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0bfa0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff720c00640 1 -- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 msgr2=0x55ab0bfa0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff720c00640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0bfa0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff720c00640 1 -- 172.21.15.27:0/3402472201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbb91b0 msgr2=0x55ab0bf946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff720c00640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf946f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff720c00640 1 -- 172.21.15.27:0/3402472201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0bc4d650 con 0x55ab0bbc1390 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff722000640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff720c00640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbc1390 0x55ab0bf94c30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff708005d00 tx=0x7ff708012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856aff2b518317d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff702a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:19:57.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff702a00640 1 -- 172.21.15.27:0/3402472201 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708010910 con 0x55ab0bbc1390 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.787+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab0be8cb90 con 0x55ab0bbc1390 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff702a00640 1 -- 172.21.15.27:0/3402472201 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7ff708010ab0 con 0x55ab0bbc1390 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff702a00640 1 -- 172.21.15.27:0/3402472201 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff708027050 con 0x55ab0bbc1390 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff72ac66740 10 client.27493 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab0bea7950 con 0x55ab0bbc1390 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff702a00640 1 -- 172.21.15.27:0/3402472201 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ab0be8cb90 con 0x55ab0bbc1390 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff702a00640 1 -- 172.21.15.27:0/3402472201 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff70800f3c0 con 0x55ab0bbc1390 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff72ac66740 10 client.27493 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.788+0000 7ff72ac66740 10 client.27493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.789+0000 7ff72ac66740 20 client.27493 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.789+0000 7ff72ac66740 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bba09f0 0x55ab0bfa6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.789+0000 7ff72ac66740 4 client.27493 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.789+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0bf01f00 con 0x55ab0bba09f0 2024-09-21T07:19:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.789+0000 7ff722000640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bba09f0 0x55ab0bfa6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.790+0000 7ff722000640 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bba09f0 0x55ab0bfa6940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8a2649ad86ede53 server_cookie=7557dab76052e027 in_seq=0 out_seq=0 2024-09-21T07:19:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.790+0000 7ff702a00640 10 client.27493 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.790+0000 7ff702a00640 1 -- 172.21.15.27:0/3402472201 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab0bf01f00 con 0x55ab0bba09f0 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.790+0000 7ff702a00640 10 client.27493 handle_command_reply: tid=0 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.790+0000 7ff72ac66740 1 client.27493 shutdown 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbc1390 msgr2=0x55ab0bf94c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbc1390 0x55ab0bf94c30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff708005d00 tx=0x7ff708012820 comp rx=0 tx=0).stop 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 shutdown_connections 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bba09f0 0x55ab0bfa6940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbc1390 0x55ab0bf94c30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0bfa0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff702a00640 0 client.27493 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 --2- 172.21.15.27:0/3402472201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 >> 172.21.15.27:0/3402472201 conn(0x55ab0bbd6460 msgr2=0x55ab0bf8c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 shutdown_connections 2024-09-21T07:19:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 1 -- 172.21.15.27:0/3402472201 wait complete. 2024-09-21T07:19:57.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.791+0000 7ff72ac66740 20 client.27493 trim_cache size 0 max 16384 2024-09-21T07:19:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 Processor -- start 2024-09-21T07:19:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 -- start start 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf96320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf86610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf86b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1b5760 con 0x55ab0bbe2070 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1b3030 con 0x55ab0bbc1390 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.805+0000 7ff72ac66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1b0ea0 con 0x55ab0bbb91b0 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf96320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf96320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:42326/0 (socket says 172.21.15.27:42326) 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 -- 172.21.15.27:0/2925235707 learned_addr learned my addr 172.21.15.27:0/2925235707 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf86b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff720c00640 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf86610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 -- 172.21.15.27:0/2925235707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 msgr2=0x55ab0bf86610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf86610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 -- 172.21.15.27:0/2925235707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 msgr2=0x55ab0bf86b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf86b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 -- 172.21.15.27:0/2925235707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0bf87090 con 0x55ab0bbb91b0 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.806+0000 7ff722000640 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf96320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff714011910 tx=0x7ff714009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4c99359e8c854 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.807+0000 7ff70dc00640 1 -- 172.21.15.27:0/2925235707 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714004ac0 con 0x55ab0bbb91b0 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.807+0000 7ff70dc00640 1 -- 172.21.15.27:0/2925235707 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714004760 con 0x55ab0bbb91b0 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.807+0000 7ff70dc00640 1 -- 172.21.15.27:0/2925235707 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714005490 con 0x55ab0bbb91b0 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.807+0000 7ff72ac66740 1 -- 172.21.15.27:0/2925235707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 msgr2=0x55ab0bf96320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.807+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf96320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff714011910 tx=0x7ff714009500 comp rx=0 tx=0).stop 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 -- 172.21.15.27:0/2925235707 shutdown_connections 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbe2070 0x55ab0bf86b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbc1390 0x55ab0bf86610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2925235707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf96320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 -- 172.21.15.27:0/2925235707 >> 172.21.15.27:0/2925235707 conn(0x55ab0bbd6460 msgr2=0x55ab0bf95c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:57.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 -- 172.21.15.27:0/2925235707 shutdown_connections 2024-09-21T07:19:57.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 -- 172.21.15.27:0/2925235707 wait complete. 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 -- start start 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.808+0000 7ff72ac66740 1 Processor -- start 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf99110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbc1390 0x55ab0bf99650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff72ac66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0c010300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff72ac66740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1b14f0 con 0x55ab0bbc1390 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff72ac66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1b2960 con 0x55ab0bbe2070 2024-09-21T07:19:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff72ac66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1b1bc0 con 0x55ab0bbb91b0 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff72ac66740 10 client.0 _ll_register_callbacks cb 0x7ff70800bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff720c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbc1390 0x55ab0bf99650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff722000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf99110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff720c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbc1390 0x55ab0bf99650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37868/0 (socket says 172.21.15.27:37868) 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff720c00640 1 -- 172.21.15.27:0/2579600871 learned_addr learned my addr 172.21.15.27:0/2579600871 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.809+0000 7ff722000640 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0c010300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.810+0000 7ff720c00640 1 -- 172.21.15.27:0/2579600871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 msgr2=0x55ab0bf99110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.810+0000 7ff720c00640 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf99110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.810+0000 7ff720c00640 1 -- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 msgr2=0x55ab0c010300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.810+0000 7ff720c00640 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0c010300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.810+0000 7ff720c00640 1 -- 172.21.15.27:0/2579600871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0c010870 con 0x55ab0bbc1390 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.810+0000 7ff720c00640 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbc1390 0x55ab0bf99650 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff70800f9d0 tx=0x7ff70800f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b4a2b95f7e0a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.810+0000 7ff702a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff702a00640 1 -- 172.21.15.27:0/2579600871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70800fe60 con 0x55ab0bbc1390 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff714004760 con 0x55ab0bbc1390 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff702a00640 1 -- 172.21.15.27:0/2579600871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7ff7080050b0 con 0x55ab0bbc1390 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff702a00640 1 -- 172.21.15.27:0/2579600871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff70803d050 con 0x55ab0bbc1390 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff72ac66740 10 client.17499 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab0c011b20 con 0x55ab0bbc1390 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff702a00640 1 -- 172.21.15.27:0/2579600871 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714004760 con 0x55ab0bbc1390 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.811+0000 7ff702a00640 1 -- 172.21.15.27:0/2579600871 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff7080035f0 con 0x55ab0bbc1390 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.812+0000 7ff72ac66740 10 client.17499 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.812+0000 7ff72ac66740 10 client.17499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.812+0000 7ff72ac66740 20 client.17499 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.812+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bb75da0 0x55ab0c014300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.812+0000 7ff72ac66740 4 client.17499 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:19:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.812+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab0befd9b0 con 0x55ab0bb75da0 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.812+0000 7ff722000640 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bb75da0 0x55ab0c014300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff722000640 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bb75da0 0x55ab0c014300 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b32ad189339cb338 server_cookie=b20da5c073c62ef9 in_seq=0 out_seq=0 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff702a00640 10 client.17499 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff702a00640 1 -- 172.21.15.27:0/2579600871 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab0befd9b0 con 0x55ab0bb75da0 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff702a00640 10 client.17499 handle_command_reply: tid=0 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 client.17499 shutdown 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbc1390 msgr2=0x55ab0bf99650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbc1390 0x55ab0bf99650 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff70800f9d0 tx=0x7ff70800f5f0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 shutdown_connections 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55ab0bb75da0 0x55ab0c014300 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ab0bbe2070 0x55ab0c010300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab0bbc1390 0x55ab0bf99650 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 --2- 172.21.15.27:0/2579600871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab0bbb91b0 0x55ab0bf99110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 >> 172.21.15.27:0/2579600871 conn(0x55ab0bbd6460 msgr2=0x55ab0bf95ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff702a00640 0 client.17499 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:19:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 shutdown_connections 2024-09-21T07:19:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.813+0000 7ff72ac66740 1 -- 172.21.15.27:0/2579600871 wait complete. 2024-09-21T07:19:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff72ac66740 20 client.17499 trim_cache size 0 max 16384 2024-09-21T07:19:57.818 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff7040b6680 msgr2=0x7ff7040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff7040b6680 0x7ff7040b8b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55ab0bf87bf0 tx=0x7ff71010ba20 comp rx=0 tx=0).stop 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71c075750 msgr2=0x7ff71c19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71c075750 0x7ff71c19c4d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff710006010 tx=0x7ff710004060 comp rx=0 tx=0).stop 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 shutdown_connections 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff71c10ff50 0x7ff71c1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff7040b6680 0x7ff7040b8b90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff71c0760a0 0x7ff71c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 --2- 172.21.15.27:0/3425475438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff71c075750 0x7ff71c19c4d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 >> 172.21.15.27:0/3425475438 conn(0x7ff71c0ff440 msgr2=0x7ff71c109ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 shutdown_connections 2024-09-21T07:19:57.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:57.814+0000 7ff722a00640 1 -- 172.21.15.27:0/3425475438 wait complete. 2024-09-21T07:19:58.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:19:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:19:58 smithi119 ceph-mon[34519]: pgmap v588: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-09-21T07:19:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:19:58 smithi027 ceph-mon[30779]: pgmap v588: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-09-21T07:19:59.091 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:19:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:19:58 smithi153 ceph-mon[34747]: pgmap v588: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-09-21T07:19:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 -- 172.21.15.27:0/308342457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64109c20 msgr2=0x7efe64110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 --2- 172.21.15.27:0/308342457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64109c20 0x7efe64110450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009ca0 tx=0x7efe5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 -- 172.21.15.27:0/308342457 shutdown_connections 2024-09-21T07:19:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 --2- 172.21.15.27:0/308342457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe641092d0 0x7efe641096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 --2- 172.21.15.27:0/308342457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64109c20 0x7efe64110450 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 --2- 172.21.15.27:0/308342457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64103990 0x7efe64103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 -- 172.21.15.27:0/308342457 >> 172.21.15.27:0/308342457 conn(0x7efe640ff3e0 msgr2=0x7efe64101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 -- 172.21.15.27:0/308342457 shutdown_connections 2024-09-21T07:19:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.684+0000 7efe63200640 1 -- 172.21.15.27:0/308342457 wait complete. 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.685+0000 7efe63200640 1 Processor -- start 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.685+0000 7efe63200640 1 -- start start 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe63200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64103990 0x7efe6419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe63200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe641092d0 0x7efe6419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe63200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64109c20 0x7efe6419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe63200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe641138e0 con 0x7efe64103990 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe63200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe64113760 con 0x7efe64109c20 2024-09-21T07:19:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe63200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe64113a60 con 0x7efe641092d0 2024-09-21T07:19:59.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe61e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64103990 0x7efe6419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe61e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64103990 0x7efe6419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59492/0 (socket says 172.21.15.27:59492) 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe61e00640 1 -- 172.21.15.27:0/2269208478 learned_addr learned my addr 172.21.15.27:0/2269208478 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe61400640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe641092d0 0x7efe6419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe62800640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64109c20 0x7efe6419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.686+0000 7efe61e00640 1 -- 172.21.15.27:0/2269208478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe641092d0 msgr2=0x7efe6419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe61e00640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe641092d0 0x7efe6419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe61e00640 1 -- 172.21.15.27:0/2269208478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64109c20 msgr2=0x7efe6419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe61e00640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64109c20 0x7efe6419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe61e00640 1 -- 172.21.15.27:0/2269208478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe641a0220 con 0x7efe64103990 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe61e00640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64103990 0x7efe6419a2b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7efe5400e9f0 tx=0x7efe5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe5aa00640 1 -- 172.21.15.27:0/2269208478 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5400cd80 con 0x7efe64103990 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe5aa00640 1 -- 172.21.15.27:0/2269208478 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe540108e0 con 0x7efe64103990 2024-09-21T07:19:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe641a0510 con 0x7efe64103990 2024-09-21T07:19:59.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.687+0000 7efe5aa00640 1 -- 172.21.15.27:0/2269208478 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54018980 con 0x7efe64103990 2024-09-21T07:19:59.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.688+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641a6c70 con 0x7efe64103990 2024-09-21T07:19:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.692+0000 7efe6fa4b740 1 Processor -- start 2024-09-21T07:19:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.692+0000 7efe6fa4b740 1 -- start start 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.692+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecaa9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaaa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6e2390 0x5587ecaaa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5587eca20fb0 con 0x5587ec6da1b0 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5587eca103e0 con 0x5587ec703070 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5587eca141f0 con 0x5587ec6e2390 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe5aa00640 1 -- 172.21.15.27:0/2269208478 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efe54010430 con 0x7efe64103990 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecaa9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecaa9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39322/0 (socket says 172.21.15.27:39322) 2024-09-21T07:19:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 -- 172.21.15.27:0/1488785265 learned_addr learned my addr 172.21.15.27:0/1488785265 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe61400640 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6e2390 0x5587ecaaa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe61e00640 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaaa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 -- 172.21.15.27:0/1488785265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6e2390 msgr2=0x5587ecaaa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6e2390 0x5587ecaaa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 -- 172.21.15.27:0/1488785265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 msgr2=0x5587ecaaa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaaa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.693+0000 7efe62800640 1 -- 172.21.15.27:0/1488785265 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ec9dacd0 con 0x5587ec703070 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.694+0000 7efe61400640 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6e2390 0x5587ecaaa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.694+0000 7efe62800640 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecaa9f70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efe5c02f410 tx=0x7efe5c02f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf2e62b76022bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.694+0000 7efe4f400640 1 -- 172.21.15.27:0/1488785265 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c031cf0 con 0x5587ec703070 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.694+0000 7efe4f400640 1 -- 172.21.15.27:0/1488785265 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5c036030 con 0x5587ec703070 2024-09-21T07:19:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.694+0000 7efe4f400640 1 -- 172.21.15.27:0/1488785265 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0042a0 con 0x5587ec703070 2024-09-21T07:19:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.695+0000 7efe5aa00640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efe480b6680 0x7efe480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.695+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1488785265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 msgr2=0x5587ecaa9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.695+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecaa9f70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efe5c02f410 tx=0x7efe5c02f440 comp rx=0 tx=0).stop 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe61e00640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efe480b6680 0x7efe480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe5aa00640 1 -- 172.21.15.27:0/2269208478 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7efe540e7e60 con 0x7efe64103990 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1488785265 shutdown_connections 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6e2390 0x5587ecaaa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaaa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1488785265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecaa9f70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1488785265 >> 172.21.15.27:0/1488785265 conn(0x5587ec6f7460 msgr2=0x5587ecaa7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1488785265 shutdown_connections 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1488785265 wait complete. 2024-09-21T07:19:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.696+0000 7efe61e00640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efe480b6680 0x7efe480b8b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5587ecaa8bf0 tx=0x7efe54000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 -- start start 2024-09-21T07:19:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 Processor -- start 2024-09-21T07:19:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6da1b0 0x5587ecab9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecab9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5587ec871700 con 0x5587ec6e2390 2024-09-21T07:19:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5587ec9ab580 con 0x5587ec6da1b0 2024-09-21T07:19:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5587ecabc870 con 0x5587ec703070 2024-09-21T07:19:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.697+0000 7efe6fa4b740 10 client.0 _ll_register_callbacks cb 0x5587ec9bb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecab9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe62800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6da1b0 0x5587ecab9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecab9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59514/0 (socket says 172.21.15.27:59514) 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 -- 172.21.15.27:0/2438881194 learned_addr learned my addr 172.21.15.27:0/2438881194 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe62800640 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 -- 172.21.15.27:0/2438881194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 msgr2=0x5587ecac5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 -- 172.21.15.27:0/2438881194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6da1b0 msgr2=0x5587ecab9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6da1b0 0x5587ecab9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.698+0000 7efe61400640 1 -- 172.21.15.27:0/2438881194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ec76e650 con 0x5587ec6e2390 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe61400640 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecab9fd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efe50012700 tx=0x7efe50012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7d9e1db83c93cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe43e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe43e00640 1 -- 172.21.15.27:0/2438881194 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50004050 con 0x5587ec6e2390 2024-09-21T07:19:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587ec9adb90 con 0x5587ec6e2390 2024-09-21T07:19:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe43e00640 1 -- 172.21.15.27:0/2438881194 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7efe500041f0 con 0x5587ec6e2390 2024-09-21T07:19:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe43e00640 1 -- 172.21.15.27:0/2438881194 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe50025050 con 0x5587ec6e2390 2024-09-21T07:19:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe6fa4b740 10 client.17511 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587ec9c8950 con 0x5587ec6e2390 2024-09-21T07:19:59.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.699+0000 7efe43e00640 1 -- 172.21.15.27:0/2438881194 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5587ec9adb90 con 0x5587ec6e2390 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe43e00640 1 -- 172.21.15.27:0/2438881194 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe500039c0 con 0x5587ec6e2390 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe6fa4b740 10 client.17511 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe6fa4b740 10 client.17511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe6fa4b740 20 client.17511 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec6c19f0 0x5587ecacb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe6fa4b740 4 client.17511 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587eca22f00 con 0x5587ec6c19f0 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.700+0000 7efe62800640 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec6c19f0 0x5587ecacb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.701+0000 7efe62800640 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec6c19f0 0x5587ecacb9f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=455c238232cf1752 server_cookie=3a2bff29d44aff1c in_seq=0 out_seq=0 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.701+0000 7efe43e00640 10 client.17511 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.701+0000 7efe43e00640 1 -- 172.21.15.27:0/2438881194 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5587eca22f00 con 0x5587ec6c19f0 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.701+0000 7efe43e00640 10 client.17511 handle_command_reply: tid=0 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 client.17511 shutdown 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 msgr2=0x5587ecab9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecab9fd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efe50012700 tx=0x7efe50012bc0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 shutdown_connections 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec6c19f0 0x5587ecacb9f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecab9fd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/2438881194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6da1b0 0x5587ecab9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 >> 172.21.15.27:0/2438881194 conn(0x5587ec6f7460 msgr2=0x5587ecaa6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe43e00640 0 client.17511 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 shutdown_connections 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 1 -- 172.21.15.27:0/2438881194 wait complete. 2024-09-21T07:19:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.702+0000 7efe6fa4b740 20 client.17511 trim_cache size 0 max 16384 2024-09-21T07:19:59.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.716+0000 7efe6fa4b740 1 Processor -- start 2024-09-21T07:19:59.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.716+0000 7efe6fa4b740 1 -- start start 2024-09-21T07:19:59.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6e2390 0x5587ecabb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe641b1340 con 0x5587ec6da1b0 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe641aec10 con 0x5587ec6e2390 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe641aca80 con 0x5587ec703070 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe62800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe62800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59522/0 (socket says 172.21.15.27:59522) 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe62800640 1 -- 172.21.15.27:0/784596155 learned_addr learned my addr 172.21.15.27:0/784596155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe61400640 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6e2390 0x5587ecabb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.717+0000 7efe62800640 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe61400640 1 -- 172.21.15.27:0/784596155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 msgr2=0x5587ecac5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe61400640 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe61400640 1 -- 172.21.15.27:0/784596155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 msgr2=0x5587ecaab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe61400640 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe61400640 1 -- 172.21.15.27:0/784596155 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ecabab30 con 0x5587ec6e2390 2024-09-21T07:19:59.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe62800640 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe61400640 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6e2390 0x5587ecabb6c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe50003f80 tx=0x7efe50005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4684750e2a58499 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe4ea00640 1 -- 172.21.15.27:0/784596155 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50005d30 con 0x5587ec6e2390 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe4ea00640 1 -- 172.21.15.27:0/784596155 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5000f790 con 0x5587ec6e2390 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.718+0000 7efe4ea00640 1 -- 172.21.15.27:0/784596155 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5000fe10 con 0x5587ec6e2390 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 -- 172.21.15.27:0/784596155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6e2390 msgr2=0x5587ecabb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6e2390 0x5587ecabb6c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe50003f80 tx=0x7efe50005770 comp rx=0 tx=0).stop 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 -- 172.21.15.27:0/784596155 shutdown_connections 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec703070 0x5587ecac5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec6e2390 0x5587ecabb6c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/784596155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6da1b0 0x5587ecaab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 -- 172.21.15.27:0/784596155 >> 172.21.15.27:0/784596155 conn(0x5587ec6f7460 msgr2=0x5587ecabafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 -- 172.21.15.27:0/784596155 shutdown_connections 2024-09-21T07:19:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.719+0000 7efe6fa4b740 1 -- 172.21.15.27:0/784596155 wait complete. 2024-09-21T07:19:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 -- start start 2024-09-21T07:19:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 Processor -- start 2024-09-21T07:19:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6da1b0 0x5587ecac1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecac2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecabcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe641ad0d0 con 0x5587ec6e2390 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe641ae540 con 0x5587ec703070 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.720+0000 7efe6fa4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe641ad7a0 con 0x5587ec6da1b0 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe6fa4b740 10 client.0 _ll_register_callbacks cb 0x7efe50009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecac2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe62800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6da1b0 0x5587ecac1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecac2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59532/0 (socket says 172.21.15.27:59532) 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 -- 172.21.15.27:0/1961264002 learned_addr learned my addr 172.21.15.27:0/1961264002 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe62800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6da1b0 0x5587ecac1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:32914/0 (socket says 172.21.15.27:32914) 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe62800640 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecabcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 -- 172.21.15.27:0/1961264002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6da1b0 msgr2=0x5587ecac1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6da1b0 0x5587ecac1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 -- 172.21.15.27:0/1961264002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 msgr2=0x5587ecabcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecabcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe61400640 1 -- 172.21.15.27:0/1961264002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ecabd030 con 0x5587ec6e2390 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.721+0000 7efe62800640 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecabcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe61400640 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecac2100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efe5000ffc0 tx=0x7efe50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72aad456900b625e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe43e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe43e00640 1 -- 172.21.15.27:0/1961264002 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50038060 con 0x5587ec6e2390 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe43e00640 1 -- 172.21.15.27:0/1961264002 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7efe50018040 con 0x5587ec6e2390 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe5000f790 con 0x5587ec6e2390 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe43e00640 1 -- 172.21.15.27:0/1961264002 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe5003f050 con 0x5587ec6e2390 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe6fa4b740 10 client.17517 fetch_fsmap learned FSMap version 27 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587ecb36b80 con 0x5587ec6e2390 2024-09-21T07:19:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.722+0000 7efe43e00640 1 -- 172.21.15.27:0/1961264002 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5000f790 con 0x5587ec6e2390 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe43e00640 1 -- 172.21.15.27:0/1961264002 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe500036f0 con 0x5587ec6e2390 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe6fa4b740 10 client.17517 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe6fa4b740 10 client.17517 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe6fa4b740 20 client.17517 populate_metadata read hostname 'smithi027' 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec696da0 0x5587ecb391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe6fa4b740 4 client.17517 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587eca1e9b0 con 0x5587ec696da0 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.723+0000 7efe62800640 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec696da0 0x5587ecb391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe43e00640 10 client.17517 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe62800640 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec696da0 0x5587ecb391f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f0c35ec1921b87e server_cookie=126da84770cf8fb9 in_seq=0 out_seq=0 2024-09-21T07:19:59.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe43e00640 1 -- 172.21.15.27:0/1961264002 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587eca1e9b0 con 0x5587ec696da0 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe43e00640 10 client.17517 handle_command_reply: tid=0 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 client.17517 shutdown 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 msgr2=0x5587ecac2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecac2100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efe5000ffc0 tx=0x7efe50000bc0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 shutdown_connections 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5587ec696da0 0x5587ecb391f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587ec703070 0x5587ecabcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5587ec6e2390 0x5587ecac2100 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 --2- 172.21.15.27:0/1961264002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587ec6da1b0 0x5587ecac1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 >> 172.21.15.27:0/1961264002 conn(0x5587ec6f7460 msgr2=0x5587ecaab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.724+0000 7efe43e00640 0 client.17517 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.725+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 shutdown_connections 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.725+0000 7efe6fa4b740 1 -- 172.21.15.27:0/1961264002 wait complete. 2024-09-21T07:19:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.725+0000 7efe6fa4b740 20 client.17517 trim_cache size 0 max 16384 2024-09-21T07:19:59.729 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:19:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.725+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efe480b6680 msgr2=0x7efe480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.725+0000 7efe63200640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efe480b6680 0x7efe480b8b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5587ecaa8bf0 tx=0x7efe54000bc0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.725+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64103990 msgr2=0x7efe6419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.725+0000 7efe63200640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64103990 0x7efe6419a2b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7efe5400e9f0 tx=0x7efe5400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 shutdown_connections 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe641092d0 0x7efe6419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efe480b6680 0x7efe480b8b90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64109c20 0x7efe6419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 --2- 172.21.15.27:0/2269208478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe64103990 0x7efe6419a2b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 >> 172.21.15.27:0/2269208478 conn(0x7efe640ff3e0 msgr2=0x7efe64101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 shutdown_connections 2024-09-21T07:19:59.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:19:59.726+0000 7efe63200640 1 -- 172.21.15.27:0/2269208478 wait complete. 2024-09-21T07:20:00.609 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:20:00.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:00 smithi027 ceph-mon[30779]: pgmap v589: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:20:00.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:00 smithi027 ceph-mon[30779]: overall HEALTH_OK 2024-09-21T07:20:01.011 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:00 smithi119 ceph-mon[34519]: pgmap v589: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:20:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:00 smithi119 ceph-mon[34519]: overall HEALTH_OK 2024-09-21T07:20:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:00 smithi153 ceph-mon[34747]: pgmap v589: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:20:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:00 smithi153 ceph-mon[34747]: overall HEALTH_OK 2024-09-21T07:20:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 -- 172.21.15.27:0/3586242804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41039b0 msgr2=0x7fdcd4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/3586242804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41039b0 0x7fdcd4103dc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009860 tx=0x7fdcc402a340 comp rx=0 tx=0).stop 2024-09-21T07:20:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 -- 172.21.15.27:0/3586242804 shutdown_connections 2024-09-21T07:20:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/3586242804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd41092f0 0x7fdcd4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/3586242804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 0x7fdcd4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/3586242804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41039b0 0x7fdcd4103dc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 -- 172.21.15.27:0/3586242804 >> 172.21.15.27:0/3586242804 conn(0x7fdcd40ff400 msgr2=0x7fdcd4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:01.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 -- 172.21.15.27:0/3586242804 shutdown_connections 2024-09-21T07:20:01.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.621+0000 7fdcd3e00640 1 -- 172.21.15.27:0/3586242804 wait complete. 2024-09-21T07:20:01.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.622+0000 7fdcd3e00640 1 Processor -- start 2024-09-21T07:20:01.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.622+0000 7fdcd3e00640 1 -- start start 2024-09-21T07:20:01.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.622+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd41039b0 0x7fdcd4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41092f0 0x7fdcd4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 0x7fdcd4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4113900 con 0x7fdcd4109c40 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4113780 con 0x7fdcd41092f0 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4113a80 con 0x7fdcd41039b0 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 0x7fdcd4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd41039b0 0x7fdcd4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 0x7fdcd4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59558/0 (socket says 172.21.15.27:59558) 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 -- 172.21.15.27:0/111582312 learned_addr learned my addr 172.21.15.27:0/111582312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd41039b0 0x7fdcd4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:32930/0 (socket says 172.21.15.27:32930) 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd2000640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41092f0 0x7fdcd4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 -- 172.21.15.27:0/111582312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd41039b0 msgr2=0x7fdcd4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd41039b0 0x7fdcd4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 -- 172.21.15.27:0/111582312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41092f0 msgr2=0x7fdcd4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41092f0 0x7fdcd4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.623+0000 7fdcd3400640 1 -- 172.21.15.27:0/111582312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd41a8570 con 0x7fdcd4109c40 2024-09-21T07:20:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.624+0000 7fdcd3400640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 0x7fdcd4076190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009fd0 tx=0x7fdccc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.624+0000 7fdccb400640 1 -- 172.21.15.27:0/111582312 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004b20 con 0x7fdcd4109c40 2024-09-21T07:20:01.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.624+0000 7fdccb400640 1 -- 172.21.15.27:0/111582312 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc003e20 con 0x7fdcd4109c40 2024-09-21T07:20:01.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.624+0000 7fdccb400640 1 -- 172.21.15.27:0/111582312 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004820 con 0x7fdcd4109c40 2024-09-21T07:20:01.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.624+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd41a8860 con 0x7fdcd4109c40 2024-09-21T07:20:01.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.624+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd41a8d50 con 0x7fdcd4109c40 2024-09-21T07:20:01.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.628+0000 7fdce01f9740 1 Processor -- start 2024-09-21T07:20:01.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.629+0000 7fdccb400640 1 -- 172.21.15.27:0/111582312 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdccc01d020 con 0x7fdcd4109c40 2024-09-21T07:20:01.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.629+0000 7fdce01f9740 1 -- start start 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afd0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afd14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afd19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdce01f9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56097af47fb0 con 0x56097ac011b0 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdce01f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56097af373e0 con 0x56097ac09390 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdce01f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56097af3b1f0 con 0x56097ac2a070 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afd14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdcd2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afd0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afd14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59570/0 (socket says 172.21.15.27:59570) 2024-09-21T07:20:01.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdcd2a00640 1 -- 172.21.15.27:0/1097826641 learned_addr learned my addr 172.21.15.27:0/1097826641 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:01.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.630+0000 7fdcd2000640 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afd19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcd2a00640 1 -- 172.21.15.27:0/1097826641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 msgr2=0x56097afd0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afd0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcd2a00640 1 -- 172.21.15.27:0/1097826641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 msgr2=0x56097afd19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afd19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcd2a00640 1 -- 172.21.15.27:0/1097826641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097af01cd0 con 0x56097ac011b0 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afd14b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009830 tx=0x7fdcc4005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff35ac863cd08a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcbf400640 1 -- 172.21.15.27:0/1097826641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4041020 con 0x56097ac011b0 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcbf400640 1 -- 172.21.15.27:0/1097826641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc4030450 con 0x56097ac011b0 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.631+0000 7fdcbf400640 1 -- 172.21.15.27:0/1097826641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4037040 con 0x56097ac011b0 2024-09-21T07:20:01.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.632+0000 7fdccb400640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdcb80b6680 0x7fdcb80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.632+0000 7fdccb400640 1 -- 172.21.15.27:0/111582312 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fdccc0e7ab0 con 0x7fdcd4109c40 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.632+0000 7fdcd2000640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdcb80b6680 0x7fdcb80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.632+0000 7fdce01f9740 1 -- 172.21.15.27:0/1097826641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 msgr2=0x56097afd14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.632+0000 7fdce01f9740 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afd14b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009830 tx=0x7fdcc4005d10 comp rx=0 tx=0).stop 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 -- 172.21.15.27:0/1097826641 shutdown_connections 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afd19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afd14b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 --2- 172.21.15.27:0/1097826641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afd0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 -- 172.21.15.27:0/1097826641 >> 172.21.15.27:0/1097826641 conn(0x56097ac1e460 msgr2=0x56097afcee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 -- 172.21.15.27:0/1097826641 shutdown_connections 2024-09-21T07:20:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdcd2000640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdcb80b6680 0x7fdcb80b8b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009fd0 tx=0x7fdcc0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 -- 172.21.15.27:0/1097826641 wait complete. 2024-09-21T07:20:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 -- start start 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.633+0000 7fdce01f9740 1 Processor -- start 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afdc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afdcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afe8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdce01f9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56097ad98700 con 0x56097ac011b0 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdce01f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56097aed2580 con 0x56097ac09390 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdce01f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56097afdf6f0 con 0x56097ac2a070 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdce01f9740 10 client.0 _ll_register_callbacks cb 0x56097aee2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afdc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afdc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59574/0 (socket says 172.21.15.27:59574) 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdcd2a00640 1 -- 172.21.15.27:0/448011411 learned_addr learned my addr 172.21.15.27:0/448011411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdcd3400640 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afe8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.634+0000 7fdcd2000640 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afdcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.635+0000 7fdcd2a00640 1 -- 172.21.15.27:0/448011411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 msgr2=0x56097afe8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.635+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afe8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.635+0000 7fdcd2a00640 1 -- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 msgr2=0x56097afdcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.635+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afdcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.635+0000 7fdcd2a00640 1 -- 172.21.15.27:0/448011411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097ac95650 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.635+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afdc6b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdcc402a860 tx=0x7fdcc4004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ced8c724ba3246 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.635+0000 7fdcb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.636+0000 7fdcb3e00640 1 -- 172.21.15.27:0/448011411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc404c070 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.636+0000 7fdcb3e00640 1 -- 172.21.15.27:0/448011411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fdcc4030650 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.636+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56097aed4b90 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.636+0000 7fdcb3e00640 1 -- 172.21.15.27:0/448011411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcc4058050 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.636+0000 7fdce01f9740 10 client.17529 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.636+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56097aeef950 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.636+0000 7fdcb3e00640 1 -- 172.21.15.27:0/448011411 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56097aed4b90 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdcb3e00640 1 -- 172.21.15.27:0/448011411 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdcc403d480 con 0x56097ac011b0 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdce01f9740 10 client.17529 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdce01f9740 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdce01f9740 20 client.17529 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:01.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdce01f9740 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abe89f0 0x56097afee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdce01f9740 4 client.17529 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:01.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56097af49f00 con 0x56097abe89f0 2024-09-21T07:20:01.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.637+0000 7fdcd2000640 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abe89f0 0x56097afee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.638+0000 7fdcd2000640 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abe89f0 0x56097afee5f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de56b016a12e2034 server_cookie=15a31a5eb7a4944f in_seq=0 out_seq=0 2024-09-21T07:20:01.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.638+0000 7fdcb3e00640 10 client.17529 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdcb3e00640 1 -- 172.21.15.27:0/448011411 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56097af49f00 con 0x56097abe89f0 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdcb3e00640 10 client.17529 handle_command_reply: tid=0 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 client.17529 shutdown 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 msgr2=0x56097afdc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afdc6b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdcc402a860 tx=0x7fdcc4004420 comp rx=0 tx=0).stop 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 shutdown_connections 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abe89f0 0x56097afee5f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac2a070 0x56097afe8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac09390 0x56097afdcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 --2- 172.21.15.27:0/448011411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac011b0 0x56097afdc6b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 >> 172.21.15.27:0/448011411 conn(0x56097ac1e460 msgr2=0x56097afd4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdcb3e00640 0 client.17529 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 shutdown_connections 2024-09-21T07:20:01.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 1 -- 172.21.15.27:0/448011411 wait complete. 2024-09-21T07:20:01.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.639+0000 7fdce01f9740 20 client.17529 trim_cache size 0 max 16384 2024-09-21T07:20:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.653+0000 7fdce01f9740 1 Processor -- start 2024-09-21T07:20:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.653+0000 7fdce01f9740 1 -- start start 2024-09-21T07:20:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afe8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afd98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afd9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdce01f9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b57d0 con 0x56097ac2a070 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdce01f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b30a0 con 0x56097ac011b0 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdce01f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b0f10 con 0x56097ac09390 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afd9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afe8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afd9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59586/0 (socket says 172.21.15.27:59586) 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afe8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39386/0 (socket says 172.21.15.27:39386) 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdcd3400640 1 -- 172.21.15.27:0/2810748154 learned_addr learned my addr 172.21.15.27:0/2810748154 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.654+0000 7fdcd2000640 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afd98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd3400640 1 -- 172.21.15.27:0/2810748154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 msgr2=0x56097afd98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd3400640 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afd98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd3400640 1 -- 172.21.15.27:0/2810748154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 msgr2=0x56097afe8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd3400640 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afe8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd3400640 1 -- 172.21.15.27:0/2810748154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097afdd970 con 0x56097ac2a070 2024-09-21T07:20:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd2000640 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afd98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afe8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcd3400640 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afd9de0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004d90 tx=0x7fdccc0acea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771a5009e492034e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcbea00640 1 -- 172.21.15.27:0/2810748154 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc01a040 con 0x56097ac2a070 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcbea00640 1 -- 172.21.15.27:0/2810748154 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc0adc50 con 0x56097ac2a070 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.655+0000 7fdcbea00640 1 -- 172.21.15.27:0/2810748154 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0aca10 con 0x56097ac2a070 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 -- 172.21.15.27:0/2810748154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 msgr2=0x56097afd9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afd9de0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004d90 tx=0x7fdccc0acea0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 -- 172.21.15.27:0/2810748154 shutdown_connections 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afd98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afd9de0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2810748154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afe8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 -- 172.21.15.27:0/2810748154 >> 172.21.15.27:0/2810748154 conn(0x56097ac1e460 msgr2=0x56097afcdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 -- 172.21.15.27:0/2810748154 shutdown_connections 2024-09-21T07:20:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.656+0000 7fdce01f9740 1 -- 172.21.15.27:0/2810748154 wait complete. 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.657+0000 7fdce01f9740 1 -- start start 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.657+0000 7fdce01f9740 1 Processor -- start 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afeed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdce01f9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdce01f9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b1560 con 0x56097ac2a070 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdce01f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b29d0 con 0x56097ac011b0 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdce01f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b1c30 con 0x56097ac09390 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdce01f9740 10 client.0 _ll_register_callbacks cb 0x7fdcc0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59588/0 (socket says 172.21.15.27:59588) 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdcd3400640 1 -- 172.21.15.27:0/2530133111 learned_addr learned my addr 172.21.15.27:0/2530133111 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdcd2000640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afeed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.658+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd2000640 1 -- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 msgr2=0x56097afee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd2000640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd2000640 1 -- 172.21.15.27:0/2530133111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 msgr2=0x56097afef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd2000640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd2000640 1 -- 172.21.15.27:0/2530133111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097b057d60 con 0x56097ac09390 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd3400640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcd2000640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afeed20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0004010 tx=0x7fdcc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb59ffbfb6091cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcb3e00640 1 -- 172.21.15.27:0/2530133111 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0011e40 con 0x56097ac09390 2024-09-21T07:20:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdccc0adc50 con 0x56097ac09390 2024-09-21T07:20:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.659+0000 7fdcb3e00640 1 -- 172.21.15.27:0/2530133111 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7fdcc001a050 con 0x56097ac09390 2024-09-21T07:20:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.660+0000 7fdcb3e00640 1 -- 172.21.15.27:0/2530133111 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcc003f050 con 0x56097ac09390 2024-09-21T07:20:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.660+0000 7fdce01f9740 10 client.27535 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:01.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.660+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56097b059010 con 0x56097ac09390 2024-09-21T07:20:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.660+0000 7fdcb3e00640 1 -- 172.21.15.27:0/2530133111 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc0adc50 con 0x56097ac09390 2024-09-21T07:20:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdcb3e00640 1 -- 172.21.15.27:0/2530133111 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdcc0004c80 con 0x56097ac09390 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdce01f9740 10 client.27535 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdce01f9740 10 client.27535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdce01f9740 20 client.27535 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abbdda0 0x56097b05b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdce01f9740 4 client.27535 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56097af459b0 con 0x56097abbdda0 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.661+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abbdda0 0x56097b05b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdcd2a00640 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abbdda0 0x56097b05b7a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e766543fc0b49859 server_cookie=6797651f6a840c1c in_seq=0 out_seq=0 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdcb3e00640 10 client.27535 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdcb3e00640 1 -- 172.21.15.27:0/2530133111 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56097af459b0 con 0x56097abbdda0 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdcb3e00640 10 client.27535 handle_command_reply: tid=0 2024-09-21T07:20:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 client.27535 shutdown 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 msgr2=0x56097afeed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afeed20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0004010 tx=0x7fdcc00046c0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 shutdown_connections 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56097abbdda0 0x56097b05b7a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56097ac09390 0x56097afeed20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56097ac2a070 0x56097afef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdcb3e00640 0 client.27535 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 --2- 172.21.15.27:0/2530133111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56097ac011b0 0x56097afee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.662+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 >> 172.21.15.27:0/2530133111 conn(0x56097ac1e460 msgr2=0x56097afd2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.663+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 shutdown_connections 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.663+0000 7fdce01f9740 1 -- 172.21.15.27:0/2530133111 wait complete. 2024-09-21T07:20:01.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.663+0000 7fdce01f9740 20 client.27535 trim_cache size 0 max 16384 2024-09-21T07:20:01.668 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:20:01.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdcb80b6680 msgr2=0x7fdcb80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdcb80b6680 0x7fdcb80b8b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009fd0 tx=0x7fdcc0005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:01.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 msgr2=0x7fdcd4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 0x7fdcd4076190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009fd0 tx=0x7fdccc00c800 comp rx=0 tx=0).stop 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 shutdown_connections 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdcb80b6680 0x7fdcb80b8b90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd4109c40 0x7fdcd4076190 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd41092f0 0x7fdcd4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 --2- 172.21.15.27:0/111582312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd41039b0 0x7fdcd4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 >> 172.21.15.27:0/111582312 conn(0x7fdcd40ff400 msgr2=0x7fdcd4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 shutdown_connections 2024-09-21T07:20:01.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:01.664+0000 7fdcd3e00640 1 -- 172.21.15.27:0/111582312 wait complete. 2024-09-21T07:20:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 -- 172.21.15.27:0/421882062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c103910 msgr2=0x7f871c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 --2- 172.21.15.27:0/421882062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c103910 0x7f871c103d20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8710009860 tx=0x7f871002a340 comp rx=0 tx=0).stop 2024-09-21T07:20:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 -- 172.21.15.27:0/421882062 shutdown_connections 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 --2- 172.21.15.27:0/421882062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f871c109b90 0x7f871c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 --2- 172.21.15.27:0/421882062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c109240 0x7f871c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 --2- 172.21.15.27:0/421882062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c103910 0x7f871c103d20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 -- 172.21.15.27:0/421882062 >> 172.21.15.27:0/421882062 conn(0x7f871c0ff330 msgr2=0x7f871c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 -- 172.21.15.27:0/421882062 shutdown_connections 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.672+0000 7f8722c00640 1 -- 172.21.15.27:0/421882062 wait complete. 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.673+0000 7f8722c00640 1 Processor -- start 2024-09-21T07:20:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722c00640 1 -- start start 2024-09-21T07:20:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c103910 0x7f871c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c109240 0x7f871c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f871c109b90 0x7f871c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f871c113850 con 0x7f871c103910 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1136d0 con 0x7f871c109b90 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1139d0 con 0x7f871c109240 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8721800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c103910 0x7f871c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8721800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c103910 0x7f871c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59596/0 (socket says 172.21.15.27:59596) 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8721800640 1 -- 172.21.15.27:0/3356400541 learned_addr learned my addr 172.21.15.27:0/3356400541 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8720e00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c109240 0x7f871c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.674+0000 7f8722200640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f871c109b90 0x7f871c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f8721800640 1 -- 172.21.15.27:0/3356400541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c109240 msgr2=0x7f871c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f8721800640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c109240 0x7f871c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f8721800640 1 -- 172.21.15.27:0/3356400541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f871c109b90 msgr2=0x7f871c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f8721800640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f871c109b90 0x7f871c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f8721800640 1 -- 172.21.15.27:0/3356400541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c1a23c0 con 0x7f871c103910 2024-09-21T07:20:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f8720e00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c109240 0x7f871c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f8721800640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c103910 0x7f871c19c490 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f87100303f0 tx=0x7f8710032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.675+0000 7f871a000640 1 -- 172.21.15.27:0/3356400541 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710040020 con 0x7f871c103910 2024-09-21T07:20:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.676+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c1a2650 con 0x7f871c103910 2024-09-21T07:20:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.676+0000 7f871a000640 1 -- 172.21.15.27:0/3356400541 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8710030dd0 con 0x7f871c103910 2024-09-21T07:20:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.676+0000 7f871a000640 1 -- 172.21.15.27:0/3356400541 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710037040 con 0x7f871c103910 2024-09-21T07:20:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.676+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c1a8e40 con 0x7f871c103910 2024-09-21T07:20:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.679+0000 7f872b3ef740 1 Processor -- start 2024-09-21T07:20:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.679+0000 7f872b3ef740 1 -- start start 2024-09-21T07:20:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.679+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93b81b0 0x55fdc975ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.679+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc975d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.679+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93974d0 0x55fdc975d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.679+0000 7f871a000640 1 -- 172.21.15.27:0/3356400541 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8710043020 con 0x7f871c103910 2024-09-21T07:20:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f872b3ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fdc96d26b0 con 0x55fdc93b81b0 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f872b3ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fdc96c1ae0 con 0x55fdc938f2f0 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f872b3ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fdc96c58f0 con 0x55fdc93974d0 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93b81b0 0x55fdc975ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93b81b0 0x55fdc975ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59608/0 (socket says 172.21.15.27:59608) 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 -- 172.21.15.27:0/3277894973 learned_addr learned my addr 172.21.15.27:0/3277894973 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8721800640 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc975d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8720e00640 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93974d0 0x55fdc975d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 -- 172.21.15.27:0/3277894973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93974d0 msgr2=0x55fdc975d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93974d0 0x55fdc975d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 -- 172.21.15.27:0/3277894973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 msgr2=0x55fdc975d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc975d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8722200640 1 -- 172.21.15.27:0/3277894973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc968e780 con 0x55fdc93b81b0 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.680+0000 7f8720e00640 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93974d0 0x55fdc975d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.681+0000 7f8722200640 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93b81b0 0x55fdc975ce10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8714011440 tx=0x7f8714011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5e31617a809214 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.681+0000 7f870ea00640 1 -- 172.21.15.27:0/3277894973 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714003c30 con 0x55fdc93b81b0 2024-09-21T07:20:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.681+0000 7f870ea00640 1 -- 172.21.15.27:0/3277894973 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8714003dd0 con 0x55fdc93b81b0 2024-09-21T07:20:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.681+0000 7f870ea00640 1 -- 172.21.15.27:0/3277894973 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871400fa20 con 0x55fdc93b81b0 2024-09-21T07:20:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.681+0000 7f871a000640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87040b6680 0x7f87040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f871a000640 1 -- 172.21.15.27:0/3356400541 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f8710107650 con 0x7f871c103910 2024-09-21T07:20:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f8721800640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87040b6680 0x7f87040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 -- 172.21.15.27:0/3277894973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93b81b0 msgr2=0x55fdc975ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93b81b0 0x55fdc975ce10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8714011440 tx=0x7f8714011900 comp rx=0 tx=0).stop 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 -- 172.21.15.27:0/3277894973 shutdown_connections 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93974d0 0x55fdc975d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc975d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3277894973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93b81b0 0x55fdc975ce10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 -- 172.21.15.27:0/3277894973 >> 172.21.15.27:0/3277894973 conn(0x55fdc93ac300 msgr2=0x55fdc975ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 -- 172.21.15.27:0/3277894973 shutdown_connections 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f872b3ef740 1 -- 172.21.15.27:0/3277894973 wait complete. 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.682+0000 7f8721800640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87040b6680 0x7f87040b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55fdc975ba90 tx=0x7f871010d530 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 -- start start 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 Processor -- start 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc9768450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc9768990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc97748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fdc95f9ab0 con 0x55fdc93974d0 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fdc9668570 con 0x55fdc93b81b0 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.683+0000 7f872b3ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fdc976b590 con 0x55fdc938f2f0 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f872b3ef740 10 client.0 _ll_register_callbacks cb 0x55fdc9677100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc9768990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8722200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc9768450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc9768990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59620/0 (socket says 172.21.15.27:59620) 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 -- 172.21.15.27:0/3519922244 learned_addr learned my addr 172.21.15.27:0/3519922244 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8722200640 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc97748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 -- 172.21.15.27:0/3519922244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 msgr2=0x55fdc9768450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc9768450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 -- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 msgr2=0x55fdc97748b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc97748b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8720e00640 1 -- 172.21.15.27:0/3519922244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc9423790 con 0x55fdc93974d0 2024-09-21T07:20:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.684+0000 7f8722200640 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc97748b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f8720e00640 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc9768990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f87080096c0 tx=0x7f8708011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53f63ba145e6cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f8703400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f8703400640 1 -- 172.21.15.27:0/3519922244 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870800fbb0 con 0x55fdc93974d0 2024-09-21T07:20:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f8703400640 1 -- 172.21.15.27:0/3519922244 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f870800fd50 con 0x55fdc93974d0 2024-09-21T07:20:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8714003dd0 con 0x55fdc93974d0 2024-09-21T07:20:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f8703400640 1 -- 172.21.15.27:0/3519922244 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8708027050 con 0x55fdc93974d0 2024-09-21T07:20:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f872b3ef740 10 client.17547 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdc949c620 con 0x55fdc93974d0 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.685+0000 7f8703400640 1 -- 172.21.15.27:0/3519922244 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8714003dd0 con 0x55fdc93974d0 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f8703400640 1 -- 172.21.15.27:0/3519922244 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8708022030 con 0x55fdc93974d0 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f872b3ef740 10 client.17547 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f872b3ef740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f872b3ef740 20 client.17547 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc9376d80 0x55fdc977a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f872b3ef740 4 client.17547 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc96d4600 con 0x55fdc9376d80 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.686+0000 7f8722200640 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc9376d80 0x55fdc977a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.687+0000 7f8722200640 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc9376d80 0x55fdc977a690 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc432543e77df3ab server_cookie=20dbc400f4d38ea5 in_seq=0 out_seq=0 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.687+0000 7f8703400640 10 client.17547 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.687+0000 7f8703400640 1 -- 172.21.15.27:0/3519922244 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fdc96d4600 con 0x55fdc9376d80 2024-09-21T07:20:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.687+0000 7f8703400640 10 client.17547 handle_command_reply: tid=0 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.687+0000 7f872b3ef740 1 client.17547 shutdown 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 msgr2=0x55fdc9768990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc9768990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f87080096c0 tx=0x7f8708011db0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 shutdown_connections 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc9376d80 0x55fdc977a690 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc97748b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc9768990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 --2- 172.21.15.27:0/3519922244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc9768450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 >> 172.21.15.27:0/3519922244 conn(0x55fdc93ac300 msgr2=0x55fdc975de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f8703400640 0 client.17547 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 shutdown_connections 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 1 -- 172.21.15.27:0/3519922244 wait complete. 2024-09-21T07:20:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.688+0000 7f872b3ef740 20 client.17547 trim_cache size 0 max 16384 2024-09-21T07:20:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.703+0000 7f872b3ef740 1 Processor -- start 2024-09-21T07:20:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.703+0000 7f872b3ef740 1 -- start start 2024-09-21T07:20:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc975e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc97747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc977fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f872b3ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b5770 con 0x55fdc93974d0 2024-09-21T07:20:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f872b3ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b3040 con 0x55fdc93b81b0 2024-09-21T07:20:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f872b3ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b0eb0 con 0x55fdc938f2f0 2024-09-21T07:20:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc97747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc97747b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59630/0 (socket says 172.21.15.27:59630) 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 -- 172.21.15.27:0/2077531140 learned_addr learned my addr 172.21.15.27:0/2077531140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8722200640 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc975e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8722200640 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc977fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 -- 172.21.15.27:0/2077531140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 msgr2=0x55fdc975e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc975e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 -- 172.21.15.27:0/2077531140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 msgr2=0x55fdc977fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc977fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8720e00640 1 -- 172.21.15.27:0/2077531140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc9769710 con 0x55fdc93974d0 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.704+0000 7f8722200640 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc977fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.705+0000 7f8720e00640 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc97747b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8708006010 tx=0x7f87080093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda1dc7a249f3a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.705+0000 7f870e000640 1 -- 172.21.15.27:0/2077531140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870802f070 con 0x55fdc93974d0 2024-09-21T07:20:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.705+0000 7f870e000640 1 -- 172.21.15.27:0/2077531140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87080198b0 con 0x55fdc93974d0 2024-09-21T07:20:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.705+0000 7f870e000640 1 -- 172.21.15.27:0/2077531140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708000e60 con 0x55fdc93974d0 2024-09-21T07:20:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 -- 172.21.15.27:0/2077531140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 msgr2=0x55fdc97747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc97747b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8708006010 tx=0x7f87080093f0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 -- 172.21.15.27:0/2077531140 shutdown_connections 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc93b81b0 0x55fdc977fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc97747b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 --2- 172.21.15.27:0/2077531140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc938f2f0 0x55fdc975e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 -- 172.21.15.27:0/2077531140 >> 172.21.15.27:0/2077531140 conn(0x55fdc93ac300 msgr2=0x55fdc975de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 -- 172.21.15.27:0/2077531140 shutdown_connections 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 -- 172.21.15.27:0/2077531140 wait complete. 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.706+0000 7f872b3ef740 1 -- start start 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 1 Processor -- start 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc976dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc976e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93b81b0 0x55fdc97e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b1500 con 0x55fdc93974d0 2024-09-21T07:20:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b2970 con 0x55fdc938f2f0 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b1bd0 con 0x55fdc93b81b0 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f872b3ef740 10 client.0 _ll_register_callbacks cb 0x7f8708011560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f8720e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc976e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f8720e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc976e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59634/0 (socket says 172.21.15.27:59634) 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f8720e00640 1 -- 172.21.15.27:0/162854109 learned_addr learned my addr 172.21.15.27:0/162854109 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f8722200640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc976dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.707+0000 7f8722200640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93b81b0 0x55fdc97e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8720e00640 1 -- 172.21.15.27:0/162854109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93b81b0 msgr2=0x55fdc97e3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8720e00640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93b81b0 0x55fdc97e3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8720e00640 1 -- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 msgr2=0x55fdc976dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8720e00640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc976dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8720e00640 1 -- 172.21.15.27:0/162854109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc97e3540 con 0x55fdc93974d0 2024-09-21T07:20:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8722200640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc976dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8720e00640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc976e290 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f87080059d0 tx=0x7f8708000e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543dca887881c1f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8703400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8703400640 1 -- 172.21.15.27:0/162854109 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870802f040 con 0x55fdc93974d0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8703400640 1 -- 172.21.15.27:0/162854109 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9461+0+0 (secure 0 0 0) 0x7f870800fbb0 con 0x55fdc93974d0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87080198b0 con 0x55fdc93974d0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f8703400640 1 -- 172.21.15.27:0/162854109 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f870800fdd0 con 0x55fdc93974d0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f872b3ef740 10 client.17553 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.708+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdc97e4700 con 0x55fdc93974d0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f8703400640 1 -- 172.21.15.27:0/162854109 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87080198b0 con 0x55fdc93974d0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f8703400640 1 -- 172.21.15.27:0/162854109 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8708022070 con 0x55fdc93974d0 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f872b3ef740 10 client.17553 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f872b3ef740 10 client.17553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f872b3ef740 20 client.17553 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f872b3ef740 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc934c1c0 0x55fdc97e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f872b3ef740 4 client.17553 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:20:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.709+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdc96d00b0 con 0x55fdc934c1c0 2024-09-21T07:20:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.710+0000 7f8722200640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc934c1c0 0x55fdc97e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.710+0000 7f8722200640 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc934c1c0 0x55fdc97e6df0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa4eb70f82a4204 server_cookie=c7e0a0d1b93d0e92 in_seq=0 out_seq=0 2024-09-21T07:20:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.710+0000 7f8703400640 10 client.17553 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.722+0000 7f8703400640 1 -- 172.21.15.27:0/162854109 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fdc96d00b0 con 0x55fdc934c1c0 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.722+0000 7f8703400640 10 client.17553 handle_command_reply: tid=0 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.722+0000 7f872b3ef740 1 client.17553 shutdown 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 msgr2=0x55fdc976e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc976e290 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f87080059d0 tx=0x7f8708000e40 comp rx=0 tx=0).stop 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 shutdown_connections 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fdc934c1c0 0x55fdc97e6df0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fdc93b81b0 0x55fdc97e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdc93974d0 0x55fdc976e290 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 --2- 172.21.15.27:0/162854109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdc938f2f0 0x55fdc976dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f8703400640 0 client.17553 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 >> 172.21.15.27:0/162854109 conn(0x55fdc93ac300 msgr2=0x55fdc9759360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 shutdown_connections 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 1 -- 172.21.15.27:0/162854109 wait complete. 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.723+0000 7f872b3ef740 20 client.17553 trim_cache size 0 max 16384 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:20:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87040b6680 msgr2=0x7f87040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87040b6680 0x7f87040b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55fdc975ba90 tx=0x7f871010d530 comp rx=0 tx=0).stop 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c103910 msgr2=0x7f871c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c103910 0x7f871c19c490 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f87100303f0 tx=0x7f8710032da0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 shutdown_connections 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f871c109240 0x7f871c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87040b6680 0x7f87040b8b90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f871c109b90 0x7f871c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 --2- 172.21.15.27:0/3356400541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c103910 0x7f871c19c490 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 >> 172.21.15.27:0/3356400541 conn(0x7f871c0ff330 msgr2=0x7f871c10e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 shutdown_connections 2024-09-21T07:20:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:02.724+0000 7f8722c00640 1 -- 172.21.15.27:0/3356400541 wait complete. 2024-09-21T07:20:03.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:02 smithi119 ceph-mon[34519]: pgmap v590: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 79 KiB/s wr, 19 op/s 2024-09-21T07:20:03.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:02 smithi027 ceph-mon[30779]: pgmap v590: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 79 KiB/s wr, 19 op/s 2024-09-21T07:20:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:02 smithi153 ceph-mon[34747]: pgmap v590: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 79 KiB/s wr, 19 op/s 2024-09-21T07:20:03.356 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:20:03.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:20:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:04.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 -- 172.21.15.27:0/1576132094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654077380 msgr2=0x7f56540757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 --2- 172.21.15.27:0/1576132094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654077380 0x7f56540757b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5644009900 tx=0x7f564402a410 comp rx=0 tx=0).stop 2024-09-21T07:20:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 -- 172.21.15.27:0/1576132094 shutdown_connections 2024-09-21T07:20:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 --2- 172.21.15.27:0/1576132094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5654114450 0x7f5654116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 --2- 172.21.15.27:0/1576132094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 0x7f56540761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 --2- 172.21.15.27:0/1576132094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654077380 0x7f56540757b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 -- 172.21.15.27:0/1576132094 >> 172.21.15.27:0/1576132094 conn(0x7f56540ff590 msgr2=0x7f56541019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 -- 172.21.15.27:0/1576132094 shutdown_connections 2024-09-21T07:20:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.413+0000 7f565a000640 1 -- 172.21.15.27:0/1576132094 wait complete. 2024-09-21T07:20:04.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.415+0000 7f565a000640 1 Processor -- start 2024-09-21T07:20:04.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.415+0000 7f565a000640 1 -- start start 2024-09-21T07:20:04.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f565a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 0x7f56541a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f565a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5654077380 0x7f56541a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f565a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654114450 0x7f56541a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f565a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f565411a0a0 con 0x7f5654077380 2024-09-21T07:20:04.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f565a000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5654119f20 con 0x7f5654075dd0 2024-09-21T07:20:04.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f565a000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f565411a220 con 0x7f5654114450 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f5653400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 0x7f56541a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f5653400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 0x7f56541a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39478/0 (socket says 172.21.15.27:39478) 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f5653400640 1 -- 172.21.15.27:0/2952194240 learned_addr learned my addr 172.21.15.27:0/2952194240 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f5653e00640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654114450 0x7f56541a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.416+0000 7f564be00640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5654077380 0x7f56541a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5653400640 1 -- 172.21.15.27:0/2952194240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654114450 msgr2=0x7f56541a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5653400640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654114450 0x7f56541a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5653400640 1 -- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5654077380 msgr2=0x7f56541a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5653400640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5654077380 0x7f56541a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5653400640 1 -- 172.21.15.27:0/2952194240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56541a6a60 con 0x7f5654075dd0 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5653e00640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654114450 0x7f56541a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:04.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5653400640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 0x7f56541a0af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5644005940 tx=0x7f56440040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:04.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5650c00640 1 -- 172.21.15.27:0/2952194240 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564403f030 con 0x7f5654075dd0 2024-09-21T07:20:04.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5650c00640 1 -- 172.21.15.27:0/2952194240 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56440044c0 con 0x7f5654075dd0 2024-09-21T07:20:04.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f5650c00640 1 -- 172.21.15.27:0/2952194240 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644035070 con 0x7f5654075dd0 2024-09-21T07:20:04.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.417+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56541a6cf0 con 0x7f5654075dd0 2024-09-21T07:20:04.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.418+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56541ad4b0 con 0x7f5654075dd0 2024-09-21T07:20:04.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.421+0000 7f5660c12740 1 Processor -- start 2024-09-21T07:20:04.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.421+0000 7f5660c12740 1 -- start start 2024-09-21T07:20:04.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.421+0000 7f5660c12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ef350 0x55b3e3696d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.421+0000 7f5660c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32c6490 0x55b3e3697250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.421+0000 7f5660c12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e3697790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.421+0000 7f5660c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b3e360ce90 con 0x55b3e32c6490 2024-09-21T07:20:04.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5660c12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3e35fc2c0 con 0x55b3e32ce670 2024-09-21T07:20:04.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5660c12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3e36000d0 con 0x55b3e32ef350 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ef350 0x55b3e3696d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f564be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e3697790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ef350 0x55b3e3696d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33046/0 (socket says 172.21.15.27:33046) 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 -- 172.21.15.27:0/3688212576 learned_addr learned my addr 172.21.15.27:0/3688212576 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 -- 172.21.15.27:0/3688212576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 msgr2=0x55b3e3697790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e3697790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 -- 172.21.15.27:0/3688212576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32c6490 msgr2=0x55b3e3697250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32c6490 0x55b3e3697250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f5653e00640 1 -- 172.21.15.27:0/3688212576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e335a930 con 0x55b3e32ef350 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.422+0000 7f564be00640 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e3697790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:04.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.423+0000 7f5653e00640 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ef350 0x55b3e3696d10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f564c0117e0 tx=0x7f564c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287096ba4dad6c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:04.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.423+0000 7f5643e00640 1 -- 172.21.15.27:0/3688212576 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c011310 con 0x55b3e32ef350 2024-09-21T07:20:04.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.423+0000 7f5643e00640 1 -- 172.21.15.27:0/3688212576 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f564c003e20 con 0x55b3e32ef350 2024-09-21T07:20:04.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.423+0000 7f5643e00640 1 -- 172.21.15.27:0/3688212576 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c004480 con 0x55b3e32ef350 2024-09-21T07:20:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.423+0000 7f5650c00640 1 -- 172.21.15.27:0/2952194240 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5644042030 con 0x7f5654075dd0 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 -- 172.21.15.27:0/3688212576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ef350 msgr2=0x55b3e3696d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ef350 0x55b3e3696d10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f564c0117e0 tx=0x7f564c011ca0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 -- 172.21.15.27:0/3688212576 shutdown_connections 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e3697790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32c6490 0x55b3e3697250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 --2- 172.21.15.27:0/3688212576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ef350 0x55b3e3696d10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 -- 172.21.15.27:0/3688212576 >> 172.21.15.27:0/3688212576 conn(0x55b3e32e3740 msgr2=0x55b3e3694c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 -- 172.21.15.27:0/3688212576 shutdown_connections 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.424+0000 7f5660c12740 1 -- 172.21.15.27:0/3688212576 wait complete. 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.425+0000 7f5660c12740 1 -- start start 2024-09-21T07:20:04.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.425+0000 7f5660c12740 1 Processor -- start 2024-09-21T07:20:04.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5650c00640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f56380b6610 0x7f56380b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5660c12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e36a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5660c12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e36a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5660c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5660c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b3e345d890 con 0x55b3e32ef350 2024-09-21T07:20:04.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5660c12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3e3597160 con 0x55b3e32ce670 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5660c12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3e36a54f0 con 0x55b3e32c6490 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5650c00640 1 -- 172.21.15.27:0/2952194240 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9549+0+0 (secure 0 0 0) 0x7f5644107e70 con 0x7f5654075dd0 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5660c12740 10 client.0 _ll_register_callbacks cb 0x55b3e35b3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5653e00640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f56380b6610 0x7f56380b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5653400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e36a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5653400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e36a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33058/0 (socket says 172.21.15.27:33058) 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5653400640 1 -- 172.21.15.27:0/1696151702 learned_addr learned my addr 172.21.15.27:0/1696151702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f564be00640 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e36a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.426+0000 7f5653e00640 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f564be00640 1 -- 172.21.15.27:0/1696151702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 msgr2=0x55b3e36a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f564be00640 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e36a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f564be00640 1 -- 172.21.15.27:0/1696151702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 msgr2=0x55b3e36ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:04.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f564be00640 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f564be00640 1 -- 172.21.15.27:0/1696151702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e36aedb0 con 0x55b3e32ce670 2024-09-21T07:20:04.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f564be00640 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e36a2e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f565406aa90 tx=0x7f563c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df901163588e6712 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:04.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f5640c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:04.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f5640c00640 1 -- 172.21.15.27:0/1696151702 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c004030 con 0x55b3e32ce670 2024-09-21T07:20:04.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f5640c00640 1 -- 172.21.15.27:0/1696151702 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9549+0+0 (secure 0 0 0) 0x7f563c0041d0 con 0x55b3e32ce670 2024-09-21T07:20:04.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.427+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c003e20 con 0x55b3e32ce670 2024-09-21T07:20:04.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.428+0000 7f5653e00640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f56380b6610 0x7f56380b8b20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f564c003df0 tx=0x7f564c005bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.428+0000 7f5640c00640 1 -- 172.21.15.27:0/1696151702 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f563c025050 con 0x55b3e32ce670 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.428+0000 7f5660c12740 10 client.27596 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.428+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3e36b1740 con 0x55b3e32ce670 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.428+0000 7f5640c00640 1 -- 172.21.15.27:0/1696151702 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f564c003e20 con 0x55b3e32ce670 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.428+0000 7f5640c00640 1 -- 172.21.15.27:0/1696151702 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f563c003900 con 0x55b3e32ce670 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.429+0000 7f5660c12740 10 client.27596 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.429+0000 7f5660c12740 10 client.27596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.429+0000 7f5660c12740 20 client.27596 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.429+0000 7f5660c12740 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e32adcd0 0x55b3e36b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.429+0000 7f5660c12740 4 client.27596 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.429+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3e360ede0 con 0x55b3e32adcd0 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.429+0000 7f5653400640 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e32adcd0 0x55b3e36b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.430+0000 7f5640c00640 10 client.27596 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.430+0000 7f5653400640 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e32adcd0 0x55b3e36b4c10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe69557b716a998 server_cookie=19d53d82961deb5f in_seq=0 out_seq=0 2024-09-21T07:20:04.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.430+0000 7f5640c00640 1 -- 172.21.15.27:0/1696151702 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3e360ede0 con 0x55b3e32adcd0 2024-09-21T07:20:04.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.430+0000 7f5640c00640 10 client.27596 handle_command_reply: tid=0 2024-09-21T07:20:04.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.430+0000 7f5660c12740 1 client.27596 shutdown 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 msgr2=0x55b3e36a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e36a2e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f565406aa90 tx=0x7f563c012b60 comp rx=0 tx=0).stop 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 shutdown_connections 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e32adcd0 0x55b3e36b4c10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5640c00640 0 client.27596 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ce670 0x55b3e36a2e00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 --2- 172.21.15.27:0/1696151702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e36a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 >> 172.21.15.27:0/1696151702 conn(0x55b3e32e3740 msgr2=0x55b3e3699ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 shutdown_connections 2024-09-21T07:20:04.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 1 -- 172.21.15.27:0/1696151702 wait complete. 2024-09-21T07:20:04.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.431+0000 7f5660c12740 20 client.27596 trim_cache size 0 max 16384 2024-09-21T07:20:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 Processor -- start 2024-09-21T07:20:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 -- start start 2024-09-21T07:20:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e369a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ce670 0x55b3e36ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ef350 0x55b3e369bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f56541b9ea0 con 0x55b3e32ce670 2024-09-21T07:20:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56541b7770 con 0x55b3e32ef350 2024-09-21T07:20:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.450+0000 7f5660c12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56541b55e0 con 0x55b3e32c6490 2024-09-21T07:20:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ce670 0x55b3e36ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ce670 0x55b3e36ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59656/0 (socket says 172.21.15.27:59656) 2024-09-21T07:20:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 -- 172.21.15.27:0/3801375353 learned_addr learned my addr 172.21.15.27:0/3801375353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f5653400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e369a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f5653e00640 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ef350 0x55b3e369bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 -- 172.21.15.27:0/3801375353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 msgr2=0x55b3e369a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e369a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 -- 172.21.15.27:0/3801375353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ef350 msgr2=0x55b3e369bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ef350 0x55b3e369bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f564be00640 1 -- 172.21.15.27:0/3801375353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e3699b70 con 0x55b3e32ce670 2024-09-21T07:20:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.451+0000 7f5653e00640 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ef350 0x55b3e369bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.452+0000 7f564be00640 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ce670 0x55b3e36ae740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f563c012420 tx=0x7f563c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec7134fc46eecf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.452+0000 7f5643400640 1 -- 172.21.15.27:0/3801375353 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c02e070 con 0x55b3e32ce670 2024-09-21T07:20:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.452+0000 7f5643400640 1 -- 172.21.15.27:0/3801375353 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563c003aa0 con 0x55b3e32ce670 2024-09-21T07:20:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.452+0000 7f5643400640 1 -- 172.21.15.27:0/3801375353 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c022040 con 0x55b3e32ce670 2024-09-21T07:20:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 -- 172.21.15.27:0/3801375353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ce670 msgr2=0x55b3e36ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ce670 0x55b3e36ae740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f563c012420 tx=0x7f563c010430 comp rx=0 tx=0).stop 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 -- 172.21.15.27:0/3801375353 shutdown_connections 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32ef350 0x55b3e369bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ce670 0x55b3e36ae740 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 --2- 172.21.15.27:0/3801375353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32c6490 0x55b3e369a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 -- 172.21.15.27:0/3801375353 >> 172.21.15.27:0/3801375353 conn(0x55b3e32e3740 msgr2=0x55b3e3699e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 -- 172.21.15.27:0/3801375353 shutdown_connections 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.453+0000 7f5660c12740 1 -- 172.21.15.27:0/3801375353 wait complete. 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 -- start start 2024-09-21T07:20:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 Processor -- start 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32c6490 0x55b3e36aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ce670 0x55b3e36ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f56541b5c30 con 0x55b3e32ef350 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56541b70a0 con 0x55b3e32c6490 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.454+0000 7f5660c12740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56541b6300 con 0x55b3e32ce670 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5660c12740 10 client.0 _ll_register_callbacks cb 0x7f563c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5653e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5653400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32c6490 0x55b3e36aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f564be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ce670 0x55b3e36ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5653400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32c6490 0x55b3e36aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39512/0 (socket says 172.21.15.27:39512) 2024-09-21T07:20:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5653e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59668/0 (socket says 172.21.15.27:59668) 2024-09-21T07:20:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5653400640 1 -- 172.21.15.27:0/2407344651 learned_addr learned my addr 172.21.15.27:0/2407344651 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f564be00640 1 -- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32c6490 msgr2=0x55b3e36aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f564be00640 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32c6490 0x55b3e36aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f564be00640 1 -- 172.21.15.27:0/2407344651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 msgr2=0x55b3e36a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f564be00640 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f564be00640 1 -- 172.21.15.27:0/2407344651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e36a5fe0 con 0x55b3e32ce670 2024-09-21T07:20:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5653400640 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32c6490 0x55b3e36aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.455+0000 7f5653e00640 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f564be00640 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ce670 0x55b3e36ab100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f563c012ba0 tx=0x7f563c0057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92a78b99a0e048d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5640c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5640c00640 1 -- 172.21.15.27:0/2407344651 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c02e070 con 0x55b3e32ce670 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563c003aa0 con 0x55b3e32ce670 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5640c00640 1 -- 172.21.15.27:0/2407344651 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9549+0+0 (secure 0 0 0) 0x7f563c0035f0 con 0x55b3e32ce670 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5640c00640 1 -- 172.21.15.27:0/2407344651 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f563c003850 con 0x55b3e32ce670 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5660c12740 10 client.27571 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3e3720150 con 0x55b3e32ce670 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.456+0000 7f5640c00640 1 -- 172.21.15.27:0/2407344651 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563c003aa0 con 0x55b3e32ce670 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5640c00640 1 -- 172.21.15.27:0/2407344651 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f563c010890 con 0x55b3e32ce670 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5660c12740 10 client.27571 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5660c12740 10 client.27571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5660c12740 20 client.27571 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5660c12740 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e3283080 0x55b3e37227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5660c12740 4 client.27571 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:20:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b3e360a890 con 0x55b3e3283080 2024-09-21T07:20:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.457+0000 7f5653400640 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e3283080 0x55b3e37227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.458+0000 7f5653400640 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e3283080 0x55b3e37227c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d273b2ed0a0467de server_cookie=173f27ed09b02b01 in_seq=0 out_seq=0 2024-09-21T07:20:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.458+0000 7f5640c00640 10 client.27571 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5640c00640 1 -- 172.21.15.27:0/2407344651 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b3e360a890 con 0x55b3e3283080 2024-09-21T07:20:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5640c00640 10 client.27571 handle_command_reply: tid=0 2024-09-21T07:20:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 client.27571 shutdown 2024-09-21T07:20:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ce670 msgr2=0x55b3e36ab100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ce670 0x55b3e36ab100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f563c012ba0 tx=0x7f563c0057d0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 shutdown_connections 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b3e3283080 0x55b3e37227c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3e32ce670 0x55b3e36ab100 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b3e32ef350 0x55b3e36a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 --2- 172.21.15.27:0/2407344651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b3e32c6490 0x55b3e36aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 >> 172.21.15.27:0/2407344651 conn(0x55b3e32e3740 msgr2=0x55b3e36944a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.460+0000 7f5640c00640 0 client.27571 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.461+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 shutdown_connections 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.461+0000 7f5660c12740 1 -- 172.21.15.27:0/2407344651 wait complete. 2024-09-21T07:20:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.461+0000 7f5660c12740 20 client.27571 trim_cache size 0 max 16384 2024-09-21T07:20:04.464 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:20:04.464 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:20:04.464 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "31290691-93d9-4f7e-b443-15fac385f103", 2024-09-21T07:20:04.464 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:20:04.464 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.461+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f56380b6610 msgr2=0x7f56380b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.461+0000 7f565a000640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f56380b6610 0x7f56380b8b20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f564c003df0 tx=0x7f564c005bd0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.461+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 msgr2=0x7f56541a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.461+0000 7f565a000640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 0x7f56541a0af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5644005940 tx=0x7f56440040c0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 shutdown_connections 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5654114450 0x7f56541a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f56380b6610 0x7f56380b8b20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5654077380 0x7f56541a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 --2- 172.21.15.27:0/2952194240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654075dd0 0x7f56541a0af0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 >> 172.21.15.27:0/2952194240 conn(0x7f56540ff590 msgr2=0x7f5654110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 shutdown_connections 2024-09-21T07:20:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:04.462+0000 7f565a000640 1 -- 172.21.15.27:0/2952194240 wait complete. 2024-09-21T07:20:05.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:04 smithi119 ceph-mon[34519]: pgmap v591: 129 pgs: 129 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 82 KiB/s wr, 30 op/s 2024-09-21T07:20:05.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:04 smithi119 ceph-mon[34519]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T07:20:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:04 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:20:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:04 smithi027 ceph-mon[30779]: pgmap v591: 129 pgs: 129 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 82 KiB/s wr, 30 op/s 2024-09-21T07:20:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:04 smithi027 ceph-mon[30779]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T07:20:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:04 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:20:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:04 smithi153 ceph-mon[34747]: pgmap v591: 129 pgs: 129 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 82 KiB/s wr, 30 op/s 2024-09-21T07:20:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:04 smithi153 ceph-mon[34747]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T07:20:05.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:04 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:20:05.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:20:05.629 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:06.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:05 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:20:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:05 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:05 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:20:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:05 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:20:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:20:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:20:06.074 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:05 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:20:06.074 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:05 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:06.074 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:05 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:20:06.074 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:05 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:06.075 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:20:06.075 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:20:06.075 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:20:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:05 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:20:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:05 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:05 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:20:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:05 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:20:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:20:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:20:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.256+0000 7f832d200640 1 -- 172.21.15.27:0/3846899736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8328105d90 msgr2=0x7f83281061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.256+0000 7f832d200640 1 --2- 172.21.15.27:0/3846899736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8328105d90 0x7f83281061a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8318009ca0 tx=0x7f831802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.256+0000 7f832d200640 1 -- 172.21.15.27:0/3846899736 shutdown_connections 2024-09-21T07:20:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.256+0000 7f832d200640 1 --2- 172.21.15.27:0/3846899736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832810b6d0 0x7f832810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.256+0000 7f832d200640 1 --2- 172.21.15.27:0/3846899736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f832810c020 0x7f8328112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.256+0000 7f832d200640 1 --2- 172.21.15.27:0/3846899736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8328105d90 0x7f83281061a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.256+0000 7f832d200640 1 -- 172.21.15.27:0/3846899736 >> 172.21.15.27:0/3846899736 conn(0x7f8328101860 msgr2=0x7f8328103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.257+0000 7f832d200640 1 -- 172.21.15.27:0/3846899736 shutdown_connections 2024-09-21T07:20:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.257+0000 7f832d200640 1 -- 172.21.15.27:0/3846899736 wait complete. 2024-09-21T07:20:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 Processor -- start 2024-09-21T07:20:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 -- start start 2024-09-21T07:20:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8328105d90 0x7f832819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f832810b6d0 0x7f832819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832810c020 0x7f83281a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8328115d30 con 0x7f832810b6d0 2024-09-21T07:20:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8328115bb0 con 0x7f8328105d90 2024-09-21T07:20:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.258+0000 7f832d200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8328115eb0 con 0x7f832810c020 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f831d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f832810b6d0 0x7f832819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8328105d90 0x7f832819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8328105d90 0x7f832819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39532/0 (socket says 172.21.15.27:39532) 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8326800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832810c020 0x7f83281a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 -- 172.21.15.27:0/2903443831 learned_addr learned my addr 172.21.15.27:0/2903443831 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f831d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f832810b6d0 0x7f832819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59688/0 (socket says 172.21.15.27:59688) 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 -- 172.21.15.27:0/2903443831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832810c020 msgr2=0x7f83281a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832810c020 0x7f83281a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 -- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f832810b6d0 msgr2=0x7f832819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f832810b6d0 0x7f832819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8325e00640 1 -- 172.21.15.27:0/2903443831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83281a2690 con 0x7f8328105d90 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.259+0000 7f8326800640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832810c020 0x7f83281a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.260+0000 7f8325e00640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8328105d90 0x7f832819c720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8318005e30 tx=0x7f8318031840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.260+0000 7f831f400640 1 -- 172.21.15.27:0/2903443831 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318003b70 con 0x7f8328105d90 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.260+0000 7f831f400640 1 -- 172.21.15.27:0/2903443831 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8318003d10 con 0x7f8328105d90 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.260+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83281a2920 con 0x7f8328105d90 2024-09-21T07:20:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.260+0000 7f831f400640 1 -- 172.21.15.27:0/2903443831 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831802f900 con 0x7f8328105d90 2024-09-21T07:20:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.260+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83281a90e0 con 0x7f8328105d90 2024-09-21T07:20:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 Processor -- start 2024-09-21T07:20:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 -- start start 2024-09-21T07:20:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311ca070 0x560631570d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a11b0 0x5606315712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x5606315717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5606314e7f70 con 0x5606311ca070 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5606314d73a0 con 0x5606311a9390 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.265+0000 7f8333dfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606314db1b0 con 0x5606311a11b0 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f8326800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311ca070 0x560631570d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f8326800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311ca070 0x560631570d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59698/0 (socket says 172.21.15.27:59698) 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f8326800640 1 -- 172.21.15.27:0/3657341997 learned_addr learned my addr 172.21.15.27:0/3657341997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f831d400640 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x5606315717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f831f400640 1 -- 172.21.15.27:0/2903443831 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f831802faa0 con 0x7f8328105d90 2024-09-21T07:20:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f831d400640 1 -- 172.21.15.27:0/3657341997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a11b0 msgr2=0x5606315712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f831d400640 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a11b0 0x5606315712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f831d400640 1 -- 172.21.15.27:0/3657341997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311ca070 msgr2=0x560631570d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f831d400640 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311ca070 0x560631570d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f831d400640 1 -- 172.21.15.27:0/3657341997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606314a1c90 con 0x5606311a9390 2024-09-21T07:20:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.266+0000 7f8326800640 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311ca070 0x560631570d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.267+0000 7f831d400640 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x5606315717e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f832806c7f0 tx=0x7f8310011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2720cdfc98e46bec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.267+0000 7f8317400640 1 -- 172.21.15.27:0/3657341997 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310004250 con 0x5606311a9390 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.267+0000 7f8317400640 1 -- 172.21.15.27:0/3657341997 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8310009e10 con 0x5606311a9390 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.267+0000 7f8317400640 1 -- 172.21.15.27:0/3657341997 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310004cd0 con 0x5606311a9390 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f831f400640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f830c0b6700 0x7f830c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8333dfa740 1 -- 172.21.15.27:0/3657341997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 msgr2=0x5606315717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x5606315717e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f832806c7f0 tx=0x7f8310011810 comp rx=0 tx=0).stop 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8333dfa740 1 -- 172.21.15.27:0/3657341997 shutdown_connections 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a11b0 0x5606315712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x5606315717e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3657341997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311ca070 0x560631570d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8333dfa740 1 -- 172.21.15.27:0/3657341997 >> 172.21.15.27:0/3657341997 conn(0x5606311be460 msgr2=0x56063156ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f831f400640 1 -- 172.21.15.27:0/2903443831 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9549+0+0 (secure 0 0 0) 0x7f8318108e30 con 0x7f8328105d90 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.268+0000 7f8326800640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f830c0b6700 0x7f830c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.269+0000 7f8333dfa740 1 -- 172.21.15.27:0/3657341997 shutdown_connections 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.269+0000 7f8333dfa740 1 -- 172.21.15.27:0/3657341997 wait complete. 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.269+0000 7f8326800640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f830c0b6700 0x7f830c0b8c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8320012700 tx=0x7f8320005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.269+0000 7f8333dfa740 1 -- start start 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 1 Processor -- start 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x56063157c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x56063157cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311ca070 0x560631588860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560631338700 con 0x5606311a11b0 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560631472540 con 0x5606311a9390 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56063157f540 con 0x5606311ca070 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8333dfa740 10 client.0 _ll_register_callbacks cb 0x560631482430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f8325e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x56063157c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f831d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x56063157cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f831d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x56063157cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39554/0 (socket says 172.21.15.27:39554) 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.270+0000 7f831d400640 1 -- 172.21.15.27:0/3621798635 learned_addr learned my addr 172.21.15.27:0/3621798635 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f8326800640 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311ca070 0x560631588860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f8325e00640 1 -- 172.21.15.27:0/3621798635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311ca070 msgr2=0x560631588860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f8325e00640 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311ca070 0x560631588860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f8325e00640 1 -- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 msgr2=0x56063157cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f8325e00640 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x56063157cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f8325e00640 1 -- 172.21.15.27:0/3621798635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560631235650 con 0x5606311a11b0 2024-09-21T07:20:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f831d400640 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x56063157cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f8325e00640 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x56063157c5b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8318106120 tx=0x7f83180cafa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba51c071b125f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.271+0000 7f830be00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f830be00640 1 -- 172.21.15.27:0/3621798635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180cdf90 con 0x5606311a11b0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f830be00640 1 -- 172.21.15.27:0/3621798635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9549+0+0 (secure 0 0 0) 0x7f83180cfd50 con 0x5606311a11b0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560631474b50 con 0x5606311a11b0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f830be00640 1 -- 172.21.15.27:0/3621798635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83181065b0 con 0x5606311a11b0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f8333dfa740 10 client.17580 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56063148f910 con 0x5606311a11b0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f830be00640 1 -- 172.21.15.27:0/3621798635 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560631474b50 con 0x5606311a11b0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f830be00640 1 -- 172.21.15.27:0/3621798635 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f83180cda80 con 0x5606311a11b0 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f8333dfa740 10 client.17580 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.272+0000 7f8333dfa740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.273+0000 7f8333dfa740 20 client.17580 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.273+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606311889f0 0x56063158e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.273+0000 7f8333dfa740 4 client.17580 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.273+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606314e9ec0 con 0x5606311889f0 2024-09-21T07:20:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.273+0000 7f831d400640 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606311889f0 0x56063158e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.273+0000 7f831d400640 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606311889f0 0x56063158e560 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78d127f7f4204f2 server_cookie=cfb3a3ec76d2cd67 in_seq=0 out_seq=0 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.273+0000 7f830be00640 10 client.17580 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.274+0000 7f830be00640 1 -- 172.21.15.27:0/3621798635 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606314e9ec0 con 0x5606311889f0 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.274+0000 7f830be00640 10 client.17580 handle_command_reply: tid=0 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.274+0000 7f8333dfa740 1 client.17580 shutdown 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 msgr2=0x56063157c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x56063157c5b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8318106120 tx=0x7f83180cafa0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 shutdown_connections 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606311889f0 0x56063158e560 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311ca070 0x560631588860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311a9390 0x56063157cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3621798635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x56063157c5b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 >> 172.21.15.27:0/3621798635 conn(0x5606311be460 msgr2=0x560631573f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f830be00640 0 client.17580 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 shutdown_connections 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 1 -- 172.21.15.27:0/3621798635 wait complete. 2024-09-21T07:20:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.275+0000 7f8333dfa740 20 client.17580 trim_cache size 0 max 16384 2024-09-21T07:20:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.293+0000 7f8333dfa740 1 Processor -- start 2024-09-21T07:20:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.293+0000 7f8333dfa740 1 -- start start 2024-09-21T07:20:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x560631577d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631574570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8333dfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b5950 con 0x5606311a11b0 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8333dfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b3220 con 0x5606311ca070 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8333dfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b1090 con 0x5606311a9390 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8325e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x560631577d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8325e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x560631577d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59710/0 (socket says 172.21.15.27:59710) 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8325e00640 1 -- 172.21.15.27:0/3226133038 learned_addr learned my addr 172.21.15.27:0/3226133038 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f831d400640 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631574570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8326800640 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8325e00640 1 -- 172.21.15.27:0/3226133038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 msgr2=0x560631574570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8325e00640 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631574570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.294+0000 7f8325e00640 1 -- 172.21.15.27:0/3226133038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 msgr2=0x5606315797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.295+0000 7f8325e00640 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315797a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.295+0000 7f8325e00640 1 -- 172.21.15.27:0/3226133038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560631573050 con 0x5606311a11b0 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.295+0000 7f8325e00640 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x560631577d60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8318004af0 tx=0x7f831802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc971f8a58d6d2c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.295+0000 7f8316a00640 1 -- 172.21.15.27:0/3226133038 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180cdf90 con 0x5606311a11b0 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.295+0000 7f8316a00640 1 -- 172.21.15.27:0/3226133038 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8318036030 con 0x5606311a11b0 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.295+0000 7f8316a00640 1 -- 172.21.15.27:0/3226133038 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180ce5f0 con 0x5606311a11b0 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 -- 172.21.15.27:0/3226133038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 msgr2=0x560631577d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x560631577d60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8318004af0 tx=0x7f831802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 -- 172.21.15.27:0/3226133038 shutdown_connections 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631574570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 --2- 172.21.15.27:0/3226133038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x560631577d60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 -- 172.21.15.27:0/3226133038 >> 172.21.15.27:0/3226133038 conn(0x5606311be460 msgr2=0x560631573e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 -- 172.21.15.27:0/3226133038 shutdown_connections 2024-09-21T07:20:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.296+0000 7f8333dfa740 1 -- 172.21.15.27:0/3226133038 wait complete. 2024-09-21T07:20:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 -- start start 2024-09-21T07:20:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 Processor -- start 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x5606315809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631580f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b16e0 con 0x5606311a11b0 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b2b50 con 0x5606311ca070 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.297+0000 7f8333dfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b1db0 con 0x5606311a9390 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8333dfa740 10 client.0 _ll_register_callbacks cb 0x7f831802ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8325e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x5606315809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8325e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x5606315809f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59718/0 (socket says 172.21.15.27:59718) 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8325e00640 1 -- 172.21.15.27:0/618371226 learned_addr learned my addr 172.21.15.27:0/618371226 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8326800640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f831d400640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631580f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8326800640 1 -- 172.21.15.27:0/618371226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 msgr2=0x560631580f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8326800640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631580f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8326800640 1 -- 172.21.15.27:0/618371226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 msgr2=0x5606315809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8326800640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x5606315809f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8326800640 1 -- 172.21.15.27:0/618371226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560631581720 con 0x5606311ca070 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.298+0000 7f8325e00640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x5606315809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f8326800640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315f7ea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8320009fd0 tx=0x7f83200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b18f85b3c4f0acf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f830be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f830be00640 1 -- 172.21.15.27:0/618371226 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320011910 con 0x5606311ca070 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8318036030 con 0x5606311ca070 2024-09-21T07:20:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f830be00640 1 -- 172.21.15.27:0/618371226 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9549+0+0 (secure 0 0 0) 0x7f8320011ab0 con 0x5606311ca070 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f830be00640 1 -- 172.21.15.27:0/618371226 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8320035050 con 0x5606311ca070 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f8333dfa740 10 client.27620 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.299+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5606315f9430 con 0x5606311ca070 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f830be00640 1 -- 172.21.15.27:0/618371226 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8318036030 con 0x5606311ca070 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f830be00640 1 -- 172.21.15.27:0/618371226 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f83200047a0 con 0x5606311ca070 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f8333dfa740 10 client.27620 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f8333dfa740 10 client.27620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f8333dfa740 20 client.27620 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f8333dfa740 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56063115dda0 0x5606315fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f8333dfa740 4 client.27620 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606314e5970 con 0x56063115dda0 2024-09-21T07:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.300+0000 7f8325e00640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56063115dda0 0x5606315fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8325e00640 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56063115dda0 0x5606315fbca0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9fde340aeda4ad server_cookie=caff393b48131cc4 in_seq=0 out_seq=0 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f830be00640 10 client.27620 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f830be00640 1 -- 172.21.15.27:0/618371226 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x5606314e5970 con 0x56063115dda0 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f830be00640 10 client.27620 handle_command_reply: tid=0 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 client.27620 shutdown 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 msgr2=0x5606315f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315f7ea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8320009fd0 tx=0x7f83200046c0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 shutdown_connections 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56063115dda0 0x5606315fbca0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606311a9390 0x560631580f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606311ca070 0x5606315f7ea0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 --2- 172.21.15.27:0/618371226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606311a11b0 0x5606315809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f830be00640 0 client.27620 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.301+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 >> 172.21.15.27:0/618371226 conn(0x5606311be460 msgr2=0x5606315745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 shutdown_connections 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f8333dfa740 1 -- 172.21.15.27:0/618371226 wait complete. 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f8333dfa740 20 client.27620 trim_cache size 0 max 16384 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "31290691-93d9-4f7e-b443-15fac385f103", 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:20:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:20:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:20:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f830c0b6700 msgr2=0x7f830c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f830c0b6700 0x7f830c0b8c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8320012700 tx=0x7f8320005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8328105d90 msgr2=0x7f832819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8328105d90 0x7f832819c720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8318005e30 tx=0x7f8318031840 comp rx=0 tx=0).stop 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 shutdown_connections 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832810c020 0x7f83281a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f830c0b6700 0x7f830c0b8c10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f832810b6d0 0x7f832819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.302+0000 7f832d200640 1 --2- 172.21.15.27:0/2903443831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8328105d90 0x7f832819c720 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.303+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 >> 172.21.15.27:0/2903443831 conn(0x7f8328101860 msgr2=0x7f8328103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.303+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 shutdown_connections 2024-09-21T07:20:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:06.303+0000 7f832d200640 1 -- 172.21.15.27:0/2903443831 wait complete. 2024-09-21T07:20:07.173 INFO:tasks.cephfs.filesystem:scrub status for tag:31290691-93d9-4f7e-b443-15fac385f103 - {'path': '/', 'tag': '31290691-93d9-4f7e-b443-15fac385f103', 'options': 'recursive,force'} 2024-09-21T07:20:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:07 smithi153 ceph-mon[34747]: pgmap v593: 129 pgs: 129 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 97 KiB/s wr, 34 op/s 2024-09-21T07:20:07.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:07 smithi119 ceph-mon[34519]: pgmap v593: 129 pgs: 129 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 97 KiB/s wr, 34 op/s 2024-09-21T07:20:07.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:07 smithi027 ceph-mon[30779]: pgmap v593: 129 pgs: 129 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 97 KiB/s wr, 34 op/s 2024-09-21T07:20:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:09 smithi153 ceph-mon[34747]: pgmap v594: 129 pgs: 129 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-21T07:20:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:09 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:20:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:09 smithi119 ceph-mon[34519]: pgmap v594: 129 pgs: 129 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-21T07:20:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:09 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:20:09.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:09 smithi027 ceph-mon[30779]: pgmap v594: 129 pgs: 129 active+clean; 4.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-21T07:20:09.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:09 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:20:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:11 smithi153 ceph-mon[34747]: pgmap v595: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-21T07:20:11.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:11 smithi119 ceph-mon[34519]: pgmap v595: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-21T07:20:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:11 smithi027 ceph-mon[30779]: pgmap v595: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-21T07:20:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:12 smithi119 ceph-mon[34519]: pgmap v596: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-21T07:20:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:12 smithi119 ceph-mon[34519]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T07:20:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:12 smithi027 ceph-mon[30779]: pgmap v596: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-21T07:20:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:12 smithi027 ceph-mon[30779]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T07:20:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:12 smithi153 ceph-mon[34747]: pgmap v596: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-21T07:20:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:12 smithi153 ceph-mon[34747]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T07:20:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:14 smithi119 ceph-mon[34519]: pgmap v598: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:20:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:14 smithi027 ceph-mon[30779]: pgmap v598: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:20:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:14 smithi153 ceph-mon[34747]: pgmap v598: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:20:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 -- 172.21.15.27:0/3501782050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc105280 msgr2=0x7f21bc10d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 --2- 172.21.15.27:0/3501782050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc105280 0x7f21bc10d7c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009900 tx=0x7f21a802a400 comp rx=0 tx=0).stop 2024-09-21T07:20:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 -- 172.21.15.27:0/3501782050 shutdown_connections 2024-09-21T07:20:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 --2- 172.21.15.27:0/3501782050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21bc10dd90 0x7f21bc110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 --2- 172.21.15.27:0/3501782050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc105280 0x7f21bc10d7c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 --2- 172.21.15.27:0/3501782050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21bc104930 0x7f21bc104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 -- 172.21.15.27:0/3501782050 >> 172.21.15.27:0/3501782050 conn(0x7f21bc0fe3e0 msgr2=0x7f21bc100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 -- 172.21.15.27:0/3501782050 shutdown_connections 2024-09-21T07:20:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.929+0000 7f21c4000640 1 -- 172.21.15.27:0/3501782050 wait complete. 2024-09-21T07:20:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.930+0000 7f21c4000640 1 Processor -- start 2024-09-21T07:20:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.930+0000 7f21c4000640 1 -- start start 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c4000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc104930 0x7f21bc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c4000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21bc105280 0x7f21bc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c4000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21bc10dd90 0x7f21bc1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c4000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc113a40 con 0x7f21bc104930 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c4000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1138c0 con 0x7f21bc10dd90 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c4000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc113bc0 con 0x7f21bc105280 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc104930 0x7f21bc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc104930 0x7f21bc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44208/0 (socket says 172.21.15.27:44208) 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c1600640 1 -- 172.21.15.27:0/3157553140 learned_addr learned my addr 172.21.15.27:0/3157553140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c0c00640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21bc105280 0x7f21bc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.931+0000 7f21c2000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21bc10dd90 0x7f21bc1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c1600640 1 -- 172.21.15.27:0/3157553140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21bc105280 msgr2=0x7f21bc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c1600640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21bc105280 0x7f21bc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c1600640 1 -- 172.21.15.27:0/3157553140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21bc10dd90 msgr2=0x7f21bc1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c1600640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21bc10dd90 0x7f21bc1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c1600640 1 -- 172.21.15.27:0/3157553140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc1a2570 con 0x7f21bc104930 2024-09-21T07:20:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c1600640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc104930 0x7f21bc19c640 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f21b000e9f0 tx=0x7f21b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21ba000640 1 -- 172.21.15.27:0/3157553140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b000ccc0 con 0x7f21bc104930 2024-09-21T07:20:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21ba000640 1 -- 172.21.15.27:0/3157553140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b00108e0 con 0x7f21bc104930 2024-09-21T07:20:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc1a2860 con 0x7f21bc104930 2024-09-21T07:20:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21ba000640 1 -- 172.21.15.27:0/3157553140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0018880 con 0x7f21bc104930 2024-09-21T07:20:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.932+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1a8ff0 con 0x7f21bc104930 2024-09-21T07:20:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.937+0000 7f21cb02c740 1 Processor -- start 2024-09-21T07:20:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.937+0000 7f21cb02c740 1 -- start start 2024-09-21T07:20:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.937+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd08ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.937+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd08b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.937+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd08b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.937+0000 7f21cb02c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561edd0006b0 con 0x561edcce61b0 2024-09-21T07:20:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.937+0000 7f21cb02c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561edcfefae0 con 0x561edccbd2f0 2024-09-21T07:20:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21cb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561edcff38f0 con 0x561edccc54d0 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21ba000640 1 -- 172.21.15.27:0/3157553140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f21b0010430 con 0x7f21bc104930 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd08ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd08ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44214/0 (socket says 172.21.15.27:44214) 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 -- 172.21.15.27:0/2795764551 learned_addr learned my addr 172.21.15.27:0/2795764551 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c0c00640 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd08b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 -- 172.21.15.27:0/2795764551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 msgr2=0x561edd08b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd08b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 -- 172.21.15.27:0/2795764551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 msgr2=0x561edd08b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c1600640 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd08b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd08b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c2000640 1 -- 172.21.15.27:0/2795764551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561edcfbc780 con 0x561edcce61b0 2024-09-21T07:20:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.938+0000 7f21c1600640 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd08b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.939+0000 7f21c2000640 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd08ac30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f21b4011440 tx=0x7f21b4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899593a04fc9e4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.939+0000 7f21ae600640 1 -- 172.21.15.27:0/2795764551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4003c30 con 0x561edcce61b0 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.939+0000 7f21ae600640 1 -- 172.21.15.27:0/2795764551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b4003dd0 con 0x561edcce61b0 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.939+0000 7f21ae600640 1 -- 172.21.15.27:0/2795764551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b400fa10 con 0x561edcce61b0 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.939+0000 7f21ba000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21a40b6700 0x7f21a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21ba000640 1 -- 172.21.15.27:0/3157553140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f21b00e7ea0 con 0x7f21bc104930 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21c1600640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21a40b6700 0x7f21a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 -- 172.21.15.27:0/2795764551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 msgr2=0x561edd08ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd08ac30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f21b4011440 tx=0x7f21b4011900 comp rx=0 tx=0).stop 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 -- 172.21.15.27:0/2795764551 shutdown_connections 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd08b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd08b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 --2- 172.21.15.27:0/2795764551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd08ac30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 -- 172.21.15.27:0/2795764551 >> 172.21.15.27:0/2795764551 conn(0x561edccda300 msgr2=0x561edd088b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 -- 172.21.15.27:0/2795764551 shutdown_connections 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.940+0000 7f21cb02c740 1 -- 172.21.15.27:0/2795764551 wait complete. 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.941+0000 7f21c1600640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21a40b6700 0x7f21a40b8c10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x561edd0898b0 tx=0x7f21b000ce60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.941+0000 7f21cb02c740 1 -- start start 2024-09-21T07:20:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.941+0000 7f21cb02c740 1 Processor -- start 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd096260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccc54d0 0x561edd0967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edcce61b0 0x561edd096d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21cb02c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561edcf27ab0 con 0x561edccc54d0 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21cb02c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561edcf96570 con 0x561edccbd2f0 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21cb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561edd0993b0 con 0x561edcce61b0 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21cb02c740 10 client.0 _ll_register_callbacks cb 0x561edcfa5100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccc54d0 0x561edd0967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd096260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccc54d0 0x561edd0967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44216/0 (socket says 172.21.15.27:44216) 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd096260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35906/0 (socket says 172.21.15.27:35906) 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c0c00640 1 -- 172.21.15.27:0/12299881 learned_addr learned my addr 172.21.15.27:0/12299881 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edcce61b0 0x561edd096d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 -- 172.21.15.27:0/12299881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edcce61b0 msgr2=0x561edd096d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edcce61b0 0x561edd096d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 -- 172.21.15.27:0/12299881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccc54d0 msgr2=0x561edd0967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccc54d0 0x561edd0967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c2000640 1 -- 172.21.15.27:0/12299881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561edcd51790 con 0x561edccbd2f0 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.942+0000 7f21c0c00640 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccc54d0 0x561edd0967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21c2000640 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd096260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f21b400f750 tx=0x7f21b400f650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e90c3b00a27274 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21a2a00640 1 -- 172.21.15.27:0/12299881 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b402c020 con 0x561edccbd2f0 2024-09-21T07:20:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b4003dd0 con 0x561edccbd2f0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21a2a00640 1 -- 172.21.15.27:0/12299881 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f21b4004460 con 0x561edccbd2f0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21a2a00640 1 -- 172.21.15.27:0/12299881 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21b4003c10 con 0x561edccbd2f0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21cb02c740 10 client.27629 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.943+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561edcdca620 con 0x561edccbd2f0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21a2a00640 1 -- 172.21.15.27:0/12299881 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b4003dd0 con 0x561edccbd2f0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21a2a00640 1 -- 172.21.15.27:0/12299881 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f21b4025040 con 0x561edccbd2f0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21cb02c740 10 client.27629 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21cb02c740 10 client.27629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21cb02c740 20 client.27629 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21cb02c740 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcca4d80 0x561edd0a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21cb02c740 4 client.27629 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561edd002600 con 0x561edcca4d80 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.944+0000 7f21c0c00640 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcca4d80 0x561edd0a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.945+0000 7f21c0c00640 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcca4d80 0x561edd0a8210 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1561682b06767503 server_cookie=92f13d2f6c0d6080 in_seq=0 out_seq=0 2024-09-21T07:20:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.945+0000 7f21a2a00640 10 client.27629 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21a2a00640 1 -- 172.21.15.27:0/12299881 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561edd002600 con 0x561edcca4d80 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21a2a00640 10 client.27629 handle_command_reply: tid=0 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 client.27629 shutdown 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 msgr2=0x561edd096260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd096260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f21b400f750 tx=0x7f21b400f650 comp rx=0 tx=0).stop 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 shutdown_connections 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcca4d80 0x561edd0a8210 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edcce61b0 0x561edd096d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccc54d0 0x561edd0967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 --2- 172.21.15.27:0/12299881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd096260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21a2a00640 0 client.27629 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 >> 172.21.15.27:0/12299881 conn(0x561edccda300 msgr2=0x561edd08bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 shutdown_connections 2024-09-21T07:20:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 1 -- 172.21.15.27:0/12299881 wait complete. 2024-09-21T07:20:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.946+0000 7f21cb02c740 20 client.27629 trim_cache size 0 max 16384 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 Processor -- start 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 -- start start 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccbd2f0 0x561edd08c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd093430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edcce61b0 0x561edd093970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1b5960 con 0x561edccbd2f0 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1b3230 con 0x561edcce61b0 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21cb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1b10a0 con 0x561edccc54d0 2024-09-21T07:20:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21c2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccbd2f0 0x561edd08c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.962+0000 7f21c2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccbd2f0 0x561edd08c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44230/0 (socket says 172.21.15.27:44230) 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 -- 172.21.15.27:0/4225154881 learned_addr learned my addr 172.21.15.27:0/4225154881 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c0c00640 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd093430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edcce61b0 0x561edd093970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 -- 172.21.15.27:0/4225154881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 msgr2=0x561edd093430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd093430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 -- 172.21.15.27:0/4225154881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edcce61b0 msgr2=0x561edd093970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edcce61b0 0x561edd093970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 -- 172.21.15.27:0/4225154881 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561edd099290 con 0x561edccbd2f0 2024-09-21T07:20:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21c2000640 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccbd2f0 0x561edd08c370 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009d00 tx=0x7f21b4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85221afef05dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21adc00640 1 -- 172.21.15.27:0/4225154881 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b402c070 con 0x561edccbd2f0 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21adc00640 1 -- 172.21.15.27:0/4225154881 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b4039070 con 0x561edccbd2f0 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.963+0000 7f21adc00640 1 -- 172.21.15.27:0/4225154881 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4004800 con 0x561edccbd2f0 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 -- 172.21.15.27:0/4225154881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccbd2f0 msgr2=0x561edd08c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccbd2f0 0x561edd08c370 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009d00 tx=0x7f21b4009900 comp rx=0 tx=0).stop 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 -- 172.21.15.27:0/4225154881 shutdown_connections 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd093430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edcce61b0 0x561edd093970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 --2- 172.21.15.27:0/4225154881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edccbd2f0 0x561edd08c370 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 -- 172.21.15.27:0/4225154881 >> 172.21.15.27:0/4225154881 conn(0x561edccda300 msgr2=0x561edd08bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 -- 172.21.15.27:0/4225154881 shutdown_connections 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.964+0000 7f21cb02c740 1 -- 172.21.15.27:0/4225154881 wait complete. 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 -- start start 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 Processor -- start 2024-09-21T07:20:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd0ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd10b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1b16f0 con 0x561edcce61b0 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1b2b60 con 0x561edccbd2f0 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.965+0000 7f21cb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1b1dc0 con 0x561edccc54d0 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21cb02c740 10 client.0 _ll_register_callbacks cb 0x7f21b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd10b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd10b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44246/0 (socket says 172.21.15.27:44246) 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 -- 172.21.15.27:0/1868877137 learned_addr learned my addr 172.21.15.27:0/1868877137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c0c00640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd0ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 -- 172.21.15.27:0/1868877137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 msgr2=0x561edd0ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd0ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 -- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 msgr2=0x561edd0ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd0ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c2000640 1 -- 172.21.15.27:0/1868877137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561edd10b6c0 con 0x561edcce61b0 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.966+0000 7f21c0c00640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd0ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21c2000640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd10b050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f21b4004df0 tx=0x7f21b4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beed1968048ca2be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21a2a00640 1 -- 172.21.15.27:0/1868877137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b402c070 con 0x561edcce61b0 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b4039070 con 0x561edcce61b0 2024-09-21T07:20:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21a2a00640 1 -- 172.21.15.27:0/1868877137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f21b40046a0 con 0x561edcce61b0 2024-09-21T07:20:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21a2a00640 1 -- 172.21.15.27:0/1868877137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21b40048a0 con 0x561edcce61b0 2024-09-21T07:20:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21cb02c740 10 client.17604 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561edd112ac0 con 0x561edcce61b0 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.967+0000 7f21a2a00640 1 -- 172.21.15.27:0/1868877137 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b4039070 con 0x561edcce61b0 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21a2a00640 1 -- 172.21.15.27:0/1868877137 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f21b4043440 con 0x561edcce61b0 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21cb02c740 10 client.17604 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21cb02c740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21cb02c740 20 client.17604 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21cb02c740 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcc7a1c0 0x561edd1151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21cb02c740 4 client.17604 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561edcffe0b0 con 0x561edcc7a1c0 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.968+0000 7f21c0c00640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcc7a1c0 0x561edd1151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.969+0000 7f21c0c00640 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcc7a1c0 0x561edd1151c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b729bce74dbdda7d server_cookie=f217ed6d396d749e in_seq=0 out_seq=0 2024-09-21T07:20:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.969+0000 7f21a2a00640 10 client.17604 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21a2a00640 1 -- 172.21.15.27:0/1868877137 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561edcffe0b0 con 0x561edcc7a1c0 2024-09-21T07:20:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21a2a00640 10 client.17604 handle_command_reply: tid=0 2024-09-21T07:20:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 client.17604 shutdown 2024-09-21T07:20:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 msgr2=0x561edd10b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd10b050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f21b4004df0 tx=0x7f21b4013040 comp rx=0 tx=0).stop 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 shutdown_connections 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561edcc7a1c0 0x561edd1151c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561edccc54d0 0x561edd0ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21a2a00640 0 client.17604 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561edcce61b0 0x561edd10b050 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 --2- 172.21.15.27:0/1868877137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561edccbd2f0 0x561edd0ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 >> 172.21.15.27:0/1868877137 conn(0x561edccda300 msgr2=0x561edd088ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 shutdown_connections 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.983+0000 7f21cb02c740 1 -- 172.21.15.27:0/1868877137 wait complete. 2024-09-21T07:20:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21cb02c740 20 client.17604 trim_cache size 0 max 16384 2024-09-21T07:20:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:20:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:20:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:20:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:20:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21a40b6700 msgr2=0x7f21a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21a40b6700 0x7f21a40b8c10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x561edd0898b0 tx=0x7f21b000ce60 comp rx=0 tx=0).stop 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc104930 msgr2=0x7f21bc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc104930 0x7f21bc19c640 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f21b000e9f0 tx=0x7f21b000edf0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 shutdown_connections 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21bc105280 0x7f21bc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21a40b6700 0x7f21a40b8c10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21bc10dd90 0x7f21bc1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 --2- 172.21.15.27:0/3157553140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21bc104930 0x7f21bc19c640 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 >> 172.21.15.27:0/3157553140 conn(0x7f21bc0fe3e0 msgr2=0x7f21bc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.984+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 shutdown_connections 2024-09-21T07:20:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:15.985+0000 7f21c4000640 1 -- 172.21.15.27:0/3157553140 wait complete. 2024-09-21T07:20:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:16 smithi119 ceph-mon[34519]: pgmap v599: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T07:20:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:16 smithi027 ceph-mon[30779]: pgmap v599: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T07:20:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:16 smithi153 ceph-mon[34747]: pgmap v599: 129 pgs: 129 active+clean; 5.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T07:20:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:18 smithi119 ceph-mon[34519]: pgmap v600: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-21T07:20:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:18 smithi027 ceph-mon[30779]: pgmap v600: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-21T07:20:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:18 smithi153 ceph-mon[34747]: pgmap v600: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-21T07:20:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:20 smithi119 ceph-mon[34519]: pgmap v601: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:21.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:20 smithi027 ceph-mon[30779]: pgmap v601: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:20 smithi153 ceph-mon[34747]: pgmap v601: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:22 smithi119 ceph-mon[34519]: pgmap v602: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:20:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:22 smithi027 ceph-mon[30779]: pgmap v602: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:20:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:22 smithi153 ceph-mon[34747]: pgmap v602: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:20:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:24 smithi119 ceph-mon[34519]: pgmap v603: 129 pgs: 129 active+clean; 5.4 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:20:25.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:24 smithi027 ceph-mon[30779]: pgmap v603: 129 pgs: 129 active+clean; 5.4 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:20:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:24 smithi153 ceph-mon[34747]: pgmap v603: 129 pgs: 129 active+clean; 5.4 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:20:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:26 smithi119 ceph-mon[34519]: pgmap v604: 129 pgs: 129 active+clean; 5.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-09-21T07:20:27.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:26 smithi027 ceph-mon[30779]: pgmap v604: 129 pgs: 129 active+clean; 5.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-09-21T07:20:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:26 smithi153 ceph-mon[34747]: pgmap v604: 129 pgs: 129 active+clean; 5.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-09-21T07:20:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:28 smithi119 ceph-mon[34519]: pgmap v605: 129 pgs: 129 active+clean; 5.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-09-21T07:20:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:28 smithi027 ceph-mon[30779]: pgmap v605: 129 pgs: 129 active+clean; 5.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-09-21T07:20:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:28 smithi153 ceph-mon[34747]: pgmap v605: 129 pgs: 129 active+clean; 5.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-09-21T07:20:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 -- 172.21.15.27:0/2372043266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010b440 msgr2=0x7f225010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 --2- 172.21.15.27:0/2372043266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010b440 0x7f225010b850 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f223c009900 tx=0x7f223c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 -- 172.21.15.27:0/2372043266 shutdown_connections 2024-09-21T07:20:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 --2- 172.21.15.27:0/2372043266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f225010bd90 0x7f22501125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 --2- 172.21.15.27:0/2372043266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010b440 0x7f225010b850 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 --2- 172.21.15.27:0/2372043266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2250105b10 0x7f2250105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 -- 172.21.15.27:0/2372043266 >> 172.21.15.27:0/2372043266 conn(0x7f2250101550 msgr2=0x7f22501039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 -- 172.21.15.27:0/2372043266 shutdown_connections 2024-09-21T07:20:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.192+0000 7f2256400640 1 -- 172.21.15.27:0/2372043266 wait complete. 2024-09-21T07:20:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.193+0000 7f2256400640 1 Processor -- start 2024-09-21T07:20:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.193+0000 7f2256400640 1 -- start start 2024-09-21T07:20:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2256400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2250105b10 0x7f225019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2256400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f225010b440 0x7f225019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2256400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010bd90 0x7f22501a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2256400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2250115aa0 con 0x7f225010bd90 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2256400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2250115920 con 0x7f225010b440 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2256400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2250115c20 con 0x7f2250105b10 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2255a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010bd90 0x7f22501a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2255a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010bd90 0x7f22501a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60208/0 (socket says 172.21.15.27:60208) 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2255a00640 1 -- 172.21.15.27:0/1525649026 learned_addr learned my addr 172.21.15.27:0/1525649026 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f2255000640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2250105b10 0x7f225019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.194+0000 7f224fe00640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f225010b440 0x7f225019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f2255a00640 1 -- 172.21.15.27:0/1525649026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2250105b10 msgr2=0x7f225019c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f2255a00640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2250105b10 0x7f225019c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f2255a00640 1 -- 172.21.15.27:0/1525649026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f225010b440 msgr2=0x7f225019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f2255a00640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f225010b440 0x7f225019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f2255a00640 1 -- 172.21.15.27:0/1525649026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22501a23d0 con 0x7f225010bd90 2024-09-21T07:20:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f2255a00640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010bd90 0x7f22501a1cd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f224800eb10 tx=0x7f224800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f224d600640 1 -- 172.21.15.27:0/1525649026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248004b20 con 0x7f225010bd90 2024-09-21T07:20:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f224d600640 1 -- 172.21.15.27:0/1525649026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2248004cc0 con 0x7f225010bd90 2024-09-21T07:20:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22501a26c0 con 0x7f225010bd90 2024-09-21T07:20:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.195+0000 7f224d600640 1 -- 172.21.15.27:0/1525649026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248010670 con 0x7f225010bd90 2024-09-21T07:20:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.196+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22501a8e50 con 0x7f225010bd90 2024-09-21T07:20:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.200+0000 7f225ec03740 1 Processor -- start 2024-09-21T07:20:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.200+0000 7f224d600640 1 -- 172.21.15.27:0/1525649026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2248005020 con 0x7f225010bd90 2024-09-21T07:20:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f225ec03740 1 -- start start 2024-09-21T07:20:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f225ec03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69451b0 0x55bda6ce9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f225ec03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda691c2f0 0x55bda6cea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f225ec03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda69244d0 0x55bda6cea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f225ec03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bda6c5f6b0 con 0x55bda69451b0 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f225ec03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bda6c4eae0 con 0x55bda69244d0 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f225ec03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bda6c528f0 con 0x55bda691c2f0 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f224fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69451b0 0x55bda6ce9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f224fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69451b0 0x55bda6ce9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60224/0 (socket says 172.21.15.27:60224) 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f224fe00640 1 -- 172.21.15.27:0/3255152605 learned_addr learned my addr 172.21.15.27:0/3255152605 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f2255000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda691c2f0 0x55bda6cea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.201+0000 7f224fe00640 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda69244d0 0x55bda6cea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224fe00640 1 -- 172.21.15.27:0/3255152605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda691c2f0 msgr2=0x55bda6cea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224fe00640 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda691c2f0 0x55bda6cea350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224fe00640 1 -- 172.21.15.27:0/3255152605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda69244d0 msgr2=0x55bda6cea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224fe00640 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda69244d0 0x55bda6cea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224fe00640 1 -- 172.21.15.27:0/3255152605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda6c1b780 con 0x55bda69451b0 2024-09-21T07:20:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224fe00640 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69451b0 0x55bda6ce9e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f223c005ab0 tx=0x7f223c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f4e232148c625b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224d600640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22380b6680 0x7f22380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f2242000640 1 -- 172.21.15.27:0/3255152605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c044020 con 0x55bda69451b0 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f2242000640 1 -- 172.21.15.27:0/3255152605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c004750 con 0x55bda69451b0 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f224d600640 1 -- 172.21.15.27:0/1525649026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f22480e8090 con 0x7f225010bd90 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f2242000640 1 -- 172.21.15.27:0/3255152605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c035040 con 0x55bda69451b0 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.202+0000 7f2255000640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22380b6680 0x7f22380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 -- 172.21.15.27:0/3255152605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69451b0 msgr2=0x55bda6ce9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f2255000640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22380b6680 0x7f22380b8b90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55bda6ce8a90 tx=0x7f2244005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69451b0 0x55bda6ce9e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f223c005ab0 tx=0x7f223c005d50 comp rx=0 tx=0).stop 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 -- 172.21.15.27:0/3255152605 shutdown_connections 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda691c2f0 0x55bda6cea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda69244d0 0x55bda6cea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 --2- 172.21.15.27:0/3255152605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69451b0 0x55bda6ce9e10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 -- 172.21.15.27:0/3255152605 >> 172.21.15.27:0/3255152605 conn(0x55bda6939300 msgr2=0x55bda6ce7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 -- 172.21.15.27:0/3255152605 shutdown_connections 2024-09-21T07:20:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.203+0000 7f225ec03740 1 -- 172.21.15.27:0/3255152605 wait complete. 2024-09-21T07:20:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.204+0000 7f225ec03740 1 -- start start 2024-09-21T07:20:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.204+0000 7f225ec03740 1 Processor -- start 2024-09-21T07:20:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f225ec03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6cf5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f225ec03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6cf5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f225ec03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d01820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f225ec03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bda6b86ab0 con 0x55bda69244d0 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f225ec03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bda6bf5570 con 0x55bda691c2f0 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f225ec03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bda6cf8530 con 0x55bda69451b0 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f225ec03740 10 client.0 _ll_register_callbacks cb 0x55bda6c04100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f224fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6cf5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f2255a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d01820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f224fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6cf5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60240/0 (socket says 172.21.15.27:60240) 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f2255a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d01820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39484/0 (socket says 172.21.15.27:39484) 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f224fe00640 1 -- 172.21.15.27:0/679790723 learned_addr learned my addr 172.21.15.27:0/679790723 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.205+0000 7f2255000640 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6cf5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f224fe00640 1 -- 172.21.15.27:0/679790723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 msgr2=0x55bda6d01820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f224fe00640 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d01820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f224fe00640 1 -- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 msgr2=0x55bda6cf5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f224fe00640 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6cf5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f224fe00640 1 -- 172.21.15.27:0/679790723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda69b0790 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f224fe00640 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6cf5990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f223c02a910 tx=0x7f223c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa67c3657b02d5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f2236a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f2236a00640 1 -- 172.21.15.27:0/679790723 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c044030 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f2236a00640 1 -- 172.21.15.27:0/679790723 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f223c003e20 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f223c004750 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f2236a00640 1 -- 172.21.15.27:0/679790723 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f223c057050 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f225ec03740 10 client.17616 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.206+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bda6a29620 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f2236a00640 1 -- 172.21.15.27:0/679790723 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c004750 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f2236a00640 1 -- 172.21.15.27:0/679790723 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f223c0494e0 con 0x55bda69244d0 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f225ec03740 10 client.17616 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f225ec03740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f225ec03740 20 client.17616 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f225ec03740 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda6903d80 0x55bda6d07370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f225ec03740 4 client.17616 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bda6c61600 con 0x55bda6903d80 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.207+0000 7f2255000640 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda6903d80 0x55bda6d07370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.208+0000 7f2255000640 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda6903d80 0x55bda6d07370 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd262ff537974cad server_cookie=9df3e1adf4fdb3dc in_seq=0 out_seq=0 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.208+0000 7f2236a00640 10 client.17616 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f2236a00640 1 -- 172.21.15.27:0/679790723 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bda6c61600 con 0x55bda6903d80 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f2236a00640 10 client.17616 handle_command_reply: tid=0 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 client.17616 shutdown 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 msgr2=0x55bda6cf5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6cf5990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f223c02a910 tx=0x7f223c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 shutdown_connections 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda6903d80 0x55bda6d07370 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d01820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6cf5990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f2236a00640 0 client.17616 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 --2- 172.21.15.27:0/679790723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6cf5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 >> 172.21.15.27:0/679790723 conn(0x55bda6939300 msgr2=0x55bda6ceae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 shutdown_connections 2024-09-21T07:20:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 1 -- 172.21.15.27:0/679790723 wait complete. 2024-09-21T07:20:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.209+0000 7f225ec03740 20 client.17616 trim_cache size 0 max 16384 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 Processor -- start 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 -- start start 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6ce6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6ceb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6cf7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b56c0 con 0x55bda69244d0 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b2f90 con 0x55bda691c2f0 2024-09-21T07:20:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f225ec03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b0e00 con 0x55bda69451b0 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.225+0000 7f224fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6ceb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6ceb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60256/0 (socket says 172.21.15.27:60256) 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 -- 172.21.15.27:0/112506333 learned_addr learned my addr 172.21.15.27:0/112506333 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f2255000640 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6ce6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f2255a00640 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6cf7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 -- 172.21.15.27:0/112506333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 msgr2=0x55bda6cf7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6cf7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 -- 172.21.15.27:0/112506333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 msgr2=0x55bda6ce6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6ce6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 -- 172.21.15.27:0/112506333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda6cec950 con 0x55bda69244d0 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f224fe00640 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6ceb800 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f223c055040 tx=0x7f223c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340ffe093b9f03f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f2241600640 1 -- 172.21.15.27:0/112506333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c044040 con 0x55bda69244d0 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.226+0000 7f2241600640 1 -- 172.21.15.27:0/112506333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c005d50 con 0x55bda69244d0 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f2241600640 1 -- 172.21.15.27:0/112506333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c035070 con 0x55bda69244d0 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 -- 172.21.15.27:0/112506333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 msgr2=0x55bda6ceb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6ceb800 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f223c055040 tx=0x7f223c005a20 comp rx=0 tx=0).stop 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 -- 172.21.15.27:0/112506333 shutdown_connections 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6cf7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6ceb800 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 --2- 172.21.15.27:0/112506333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6ce6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 -- 172.21.15.27:0/112506333 >> 172.21.15.27:0/112506333 conn(0x55bda6939300 msgr2=0x55bda6cf6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 -- 172.21.15.27:0/112506333 shutdown_connections 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.227+0000 7f225ec03740 1 -- 172.21.15.27:0/112506333 wait complete. 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.228+0000 7f225ec03740 1 -- start start 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.228+0000 7f225ec03740 1 Processor -- start 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.228+0000 7f225ec03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6d0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.228+0000 7f225ec03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6d0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.228+0000 7f225ec03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.228+0000 7f225ec03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b1450 con 0x55bda69244d0 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f225ec03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b28c0 con 0x55bda691c2f0 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f225ec03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b1b20 con 0x55bda69451b0 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f225ec03740 10 client.0 _ll_register_callbacks cb 0x7f2244009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f224fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6d0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6d0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6d0a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:42010/0 (socket says 172.21.15.27:42010) 2024-09-21T07:20:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 -- 172.21.15.27:0/2566240387 learned_addr learned my addr 172.21.15.27:0/2566240387 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255a00640 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 -- 172.21.15.27:0/2566240387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 msgr2=0x55bda6d66be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d66be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 -- 172.21.15.27:0/2566240387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 msgr2=0x55bda6d0a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6d0a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 -- 172.21.15.27:0/2566240387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda6d67280 con 0x55bda691c2f0 2024-09-21T07:20:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255a00640 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:20:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2255000640 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6d0a0b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55bda6cf8590 tx=0x7f22440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a3a28b537cd33f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.229+0000 7f2236a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f2236a00640 1 -- 172.21.15.27:0/2566240387 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244035060 con 0x55bda691c2f0 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f223c005d50 con 0x55bda691c2f0 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f2236a00640 1 -- 172.21.15.27:0/2566240387 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f2244030430 con 0x55bda691c2f0 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f2236a00640 1 -- 172.21.15.27:0/2566240387 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f224403c050 con 0x55bda691c2f0 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f225ec03740 10 client.27653 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bda6d71920 con 0x55bda691c2f0 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f2236a00640 1 -- 172.21.15.27:0/2566240387 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c005d50 con 0x55bda691c2f0 2024-09-21T07:20:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.230+0000 7f2236a00640 1 -- 172.21.15.27:0/2566240387 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f224402f070 con 0x55bda691c2f0 2024-09-21T07:20:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f225ec03740 10 client.27653 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f225ec03740 10 client.27653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f225ec03740 20 client.27653 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f225ec03740 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda68d91c0 0x55bda6d740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f225ec03740 4 client.27653 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:20:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bda6c5d0b0 con 0x55bda68d91c0 2024-09-21T07:20:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f224fe00640 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda68d91c0 0x55bda6d740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f224fe00640 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda68d91c0 0x55bda6d740c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b078b27d463c92fc server_cookie=1c99a2b2e2aaffbb in_seq=0 out_seq=0 2024-09-21T07:20:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.231+0000 7f2236a00640 10 client.27653 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f2236a00640 1 -- 172.21.15.27:0/2566240387 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bda6c5d0b0 con 0x55bda68d91c0 2024-09-21T07:20:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f2236a00640 10 client.27653 handle_command_reply: tid=0 2024-09-21T07:20:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 client.27653 shutdown 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 msgr2=0x55bda6d0a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6d0a0b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55bda6cf8590 tx=0x7f22440046c0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 shutdown_connections 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bda68d91c0 0x55bda6d740c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bda69451b0 0x55bda6d66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bda69244d0 0x55bda6d0a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 --2- 172.21.15.27:0/2566240387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bda691c2f0 0x55bda6d0a0b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f2236a00640 0 client.27653 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.245+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 >> 172.21.15.27:0/2566240387 conn(0x55bda6939300 msgr2=0x55bda6ceb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.246+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 shutdown_connections 2024-09-21T07:20:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.246+0000 7f225ec03740 1 -- 172.21.15.27:0/2566240387 wait complete. 2024-09-21T07:20:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.246+0000 7f225ec03740 20 client.27653 trim_cache size 0 max 16384 2024-09-21T07:20:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:20:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:20:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:20:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22380b6680 msgr2=0x7f22380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22380b6680 0x7f22380b8b90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55bda6ce8a90 tx=0x7f2244005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010bd90 msgr2=0x7f22501a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010bd90 0x7f22501a1cd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f224800eb10 tx=0x7f224800c2d0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 shutdown_connections 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22380b6680 0x7f22380b8b90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225010bd90 0x7f22501a1cd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f225010b440 0x7f225019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 --2- 172.21.15.27:0/1525649026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2250105b10 0x7f225019c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 >> 172.21.15.27:0/1525649026 conn(0x7f2250101550 msgr2=0x7f2250103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 shutdown_connections 2024-09-21T07:20:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:29.247+0000 7f2256400640 1 -- 172.21.15.27:0/1525649026 wait complete. 2024-09-21T07:20:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:30 smithi119 ceph-mon[34519]: pgmap v606: 129 pgs: 129 active+clean; 5.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-21T07:20:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:30 smithi027 ceph-mon[30779]: pgmap v606: 129 pgs: 129 active+clean; 5.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-21T07:20:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:30 smithi153 ceph-mon[34747]: pgmap v606: 129 pgs: 129 active+clean; 5.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-21T07:20:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:32 smithi119 ceph-mon[34519]: pgmap v607: 129 pgs: 129 active+clean; 1.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:32 smithi027 ceph-mon[30779]: pgmap v607: 129 pgs: 129 active+clean; 1.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:32 smithi153 ceph-mon[34747]: pgmap v607: 129 pgs: 129 active+clean; 1.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:34 smithi119 ceph-mon[34519]: pgmap v608: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-09-21T07:20:35.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:34 smithi027 ceph-mon[30779]: pgmap v608: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-09-21T07:20:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:34 smithi153 ceph-mon[34747]: pgmap v608: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-09-21T07:20:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:36 smithi119 ceph-mon[34519]: pgmap v609: 129 pgs: 129 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:20:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:36 smithi027 ceph-mon[30779]: pgmap v609: 129 pgs: 129 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:20:37.174 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:20:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:36 smithi153 ceph-mon[34747]: pgmap v609: 129 pgs: 129 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:20:37.536 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:20:37.536 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T07:20:37.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:20:37.579 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:189/600: open fd = 3 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:190/600: open fd = 3 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.758 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:191/600: open fd = 3 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:192/600: open fd = 3 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:193/600: open fd = 3 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.759 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:194/600: open fd = 3 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:195/600: open fd = 3 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:196/600: open fd = 3 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:197/600: open fd = 3 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.760 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:198/600: open fd = 3 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:199/600: open fd = 3 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:200/600: open fd = 3 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:201/600: open fd = 3 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.761 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:202/600: open fd = 3 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:203/600: open fd = 3 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:204/600: open fd = 3 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.762 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:205/600: open fd = 3 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:206/600: open fd = 3 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:207/600: open fd = 3 2024-09-21T07:20:37.763 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:208/600: open fd = 3 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:209/600: open fd = 3 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:210/600: open fd = 3 2024-09-21T07:20:37.764 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:211/600: open fd = 3 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:212/600: open fd = 3 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:213/600: open fd = 3 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.765 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:214/600: open fd = 3 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:215/600: open fd = 3 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:216/600: open fd = 3 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:217/600: open fd = 3 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.766 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:218/600: open fd = 3 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:219/600: open fd = 3 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:220/600: open fd = 3 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.767 INFO:tasks.workunit.client.0.smithi027.stdout:221/600: open fd = 3 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:222/600: open fd = 3 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:223/600: open fd = 3 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:224/600: open fd = 3 2024-09-21T07:20:37.768 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:225/600: open fd = 3 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:226/600: open fd = 3 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:227/600: open fd = 3 2024-09-21T07:20:37.769 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:228/600: open fd = 3 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:229/600: open fd = 3 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.770 INFO:tasks.workunit.client.0.smithi027.stdout:230/600: open fd = 3 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:231/600: open fd = 3 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:232/600: open fd = 3 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:233/600: open fd = 3 2024-09-21T07:20:37.771 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.773 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.773 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.773 INFO:tasks.workunit.client.0.smithi027.stdout:234/600: open fd = 3 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:235/600: open fd = 3 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:236/600: open fd = 3 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:237/600: open fd = 3 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.774 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:238/600: open fd = 3 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:239/600: open fd = 3 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:240/600: open fd = 3 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:241/600: open fd = 3 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.775 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:242/600: open fd = 3 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:243/600: open fd = 3 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:244/600: open fd = 3 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:245/600: open fd = 3 2024-09-21T07:20:37.776 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:246/600: open fd = 3 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:247/600: open fd = 3 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:248/600: open fd = 3 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.777 INFO:tasks.workunit.client.0.smithi027.stdout:249/600: open fd = 3 2024-09-21T07:20:37.778 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.778 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.778 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:20:37.778 INFO:tasks.workunit.client.0.smithi027.stdout:250/600: open fd = 3 2024-09-21T07:20:37.778 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:37.778 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:20:38.175 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:38.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 -- 172.21.15.27:0/1448292306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b60 msgr2=0x7fe744103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 --2- 172.21.15.27:0/1448292306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b60 0x7fe744103f70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe734009900 tx=0x7fe73402a410 comp rx=0 tx=0).stop 2024-09-21T07:20:38.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 -- 172.21.15.27:0/1448292306 shutdown_connections 2024-09-21T07:20:38.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 --2- 172.21.15.27:0/1448292306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 0x7fe7441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 --2- 172.21.15.27:0/1448292306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744109df0 0x7fe744110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 --2- 172.21.15.27:0/1448292306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744103b60 0x7fe744103f70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 -- 172.21.15.27:0/1448292306 >> 172.21.15.27:0/1448292306 conn(0x7fe74407b850 msgr2=0x7fe74407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 -- 172.21.15.27:0/1448292306 shutdown_connections 2024-09-21T07:20:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.197+0000 7fe74ac00640 1 -- 172.21.15.27:0/1448292306 wait complete. 2024-09-21T07:20:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.198+0000 7fe74ac00640 1 Processor -- start 2024-09-21T07:20:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe74ac00640 1 -- start start 2024-09-21T07:20:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe74ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744103b60 0x7fe744075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe74ac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 0x7fe744075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe74ac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744109df0 0x7fe744076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe74ac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe744113b00 con 0x7fe744109df0 2024-09-21T07:20:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe74ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe744113980 con 0x7fe744103b60 2024-09-21T07:20:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe74ac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe744113c80 con 0x7fe7441094a0 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe73be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 0x7fe744075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe73be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 0x7fe744075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49846/0 (socket says 172.21.15.27:49846) 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.199+0000 7fe73be00640 1 -- 172.21.15.27:0/4287466166 learned_addr learned my addr 172.21.15.27:0/4287466166 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe743e00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744103b60 0x7fe744075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe748c00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744109df0 0x7fe744076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe73be00640 1 -- 172.21.15.27:0/4287466166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744103b60 msgr2=0x7fe744075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe73be00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744103b60 0x7fe744075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe73be00640 1 -- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744109df0 msgr2=0x7fe744076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe73be00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744109df0 0x7fe744076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe73be00640 1 -- 172.21.15.27:0/4287466166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7441a8770 con 0x7fe7441094a0 2024-09-21T07:20:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe748c00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744109df0 0x7fe744076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.200+0000 7fe73be00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 0x7fe744075c50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00e9f0 tx=0x7fe72c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.201+0000 7fe741600640 1 -- 172.21.15.27:0/4287466166 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00ccc0 con 0x7fe7441094a0 2024-09-21T07:20:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.201+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7441a8a60 con 0x7fe7441094a0 2024-09-21T07:20:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.201+0000 7fe741600640 1 -- 172.21.15.27:0/4287466166 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72c0108e0 con 0x7fe7441094a0 2024-09-21T07:20:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.201+0000 7fe741600640 1 -- 172.21.15.27:0/4287466166 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c018860 con 0x7fe7441094a0 2024-09-21T07:20:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.201+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7441a8fc0 con 0x7fe7441094a0 2024-09-21T07:20:38.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.204+0000 7fe7517cd740 1 Processor -- start 2024-09-21T07:20:38.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.204+0000 7fe7517cd740 1 -- start start 2024-09-21T07:20:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f519b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f51a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe7517cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f44f490f70 con 0x55f44f173070 2024-09-21T07:20:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe7517cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f44f4803a0 con 0x55f44f14a1b0 2024-09-21T07:20:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe7517cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f44f4841b0 con 0x55f44f152390 2024-09-21T07:20:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f51a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f51a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40070/0 (socket says 172.21.15.27:40070) 2024-09-21T07:20:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe743e00640 1 -- 172.21.15.27:0/2181799299 learned_addr learned my addr 172.21.15.27:0/2181799299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.205+0000 7fe748c00640 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f519b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe743e00640 1 -- 172.21.15.27:0/2181799299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 msgr2=0x55f44f51a5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe743e00640 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51a5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe743e00640 1 -- 172.21.15.27:0/2181799299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 msgr2=0x55f44f519b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe743e00640 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f519b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe743e00640 1 -- 172.21.15.27:0/2181799299 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44f44ac90 con 0x55f44f14a1b0 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe748c00640 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f519b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe743e00640 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f51a0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe7340040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3146238c2277a04f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.206+0000 7fe741600640 1 -- 172.21.15.27:0/4287466166 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe72c010430 con 0x7fe7441094a0 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.207+0000 7fe739200640 1 -- 172.21.15.27:0/2181799299 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734035040 con 0x55f44f14a1b0 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.207+0000 7fe739200640 1 -- 172.21.15.27:0/2181799299 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7340054f0 con 0x55f44f14a1b0 2024-09-21T07:20:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.207+0000 7fe739200640 1 -- 172.21.15.27:0/2181799299 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340035f0 con 0x55f44f14a1b0 2024-09-21T07:20:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 -- 172.21.15.27:0/2181799299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 msgr2=0x55f44f51a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f51a0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe7340040c0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 -- 172.21.15.27:0/2181799299 shutdown_connections 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f51a0a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 --2- 172.21.15.27:0/2181799299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f519b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 -- 172.21.15.27:0/2181799299 >> 172.21.15.27:0/2181799299 conn(0x55f44f167460 msgr2=0x55f44f517a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe741600640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7280b6680 0x7fe7280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe7517cd740 1 -- 172.21.15.27:0/2181799299 shutdown_connections 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe741600640 1 -- 172.21.15.27:0/4287466166 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe72c0e7e80 con 0x7fe7441094a0 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.208+0000 7fe748c00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7280b6680 0x7fe7280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.209+0000 7fe7517cd740 1 -- 172.21.15.27:0/2181799299 wait complete. 2024-09-21T07:20:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.209+0000 7fe748c00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7280b6680 0x7fe7280b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0099a0 tx=0x7fe73c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.210+0000 7fe7517cd740 1 -- start start 2024-09-21T07:20:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.210+0000 7fe7517cd740 1 Processor -- start 2024-09-21T07:20:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.211+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f5257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.211+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f525d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.211+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f5315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.211+0000 7fe7517cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f44f2e1700 con 0x55f44f173070 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.211+0000 7fe7517cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f44f41b540 con 0x55f44f14a1b0 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.211+0000 7fe7517cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f44f5282e0 con 0x55f44f152390 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.211+0000 7fe7517cd740 10 client.0 _ll_register_callbacks cb 0x55f44f42b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f5257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f5257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40078/0 (socket says 172.21.15.27:40078) 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 -- 172.21.15.27:0/3762736472 learned_addr learned my addr 172.21.15.27:0/3762736472 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe73be00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f525d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe748c00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f5315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 -- 172.21.15.27:0/3762736472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 msgr2=0x55f44f525d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f525d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 -- 172.21.15.27:0/3762736472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 msgr2=0x55f44f5315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f5315d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe743e00640 1 -- 172.21.15.27:0/3762736472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44f1de650 con 0x55f44f14a1b0 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.212+0000 7fe748c00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f5315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.213+0000 7fe743e00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f5257c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe734005ab0 tx=0x7fe73402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529d370e549f6f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.213+0000 7fe731600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.213+0000 7fe731600640 1 -- 172.21.15.27:0/3762736472 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734053060 con 0x55f44f14a1b0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.213+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f44f41db50 con 0x55f44f14a1b0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.213+0000 7fe731600640 1 -- 172.21.15.27:0/3762736472 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe734004980 con 0x55f44f14a1b0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe731600640 1 -- 172.21.15.27:0/3762736472 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe734035050 con 0x55f44f14a1b0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 10 client.27665 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f44f438910 con 0x55f44f14a1b0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe731600640 1 -- 172.21.15.27:0/3762736472 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f44f41db50 con 0x55f44f14a1b0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe731600640 1 -- 172.21.15.27:0/3762736472 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7340050c0 con 0x55f44f14a1b0 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 10 client.27665 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 10 client.27665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 20 client.27665 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f1319f0 0x55f44f5374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 4 client.27665 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:38.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.214+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f44f492ec0 con 0x55f44f1319f0 2024-09-21T07:20:38.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.215+0000 7fe73be00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f1319f0 0x55f44f5374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.215+0000 7fe73be00640 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f1319f0 0x55f44f5374d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c2c0c9c4f31304b server_cookie=baeb29e1de6b8e40 in_seq=0 out_seq=0 2024-09-21T07:20:38.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.215+0000 7fe731600640 10 client.27665 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:38.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.216+0000 7fe731600640 1 -- 172.21.15.27:0/3762736472 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f44f492ec0 con 0x55f44f1319f0 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.216+0000 7fe731600640 10 client.27665 handle_command_reply: tid=0 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.216+0000 7fe7517cd740 1 client.27665 shutdown 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 msgr2=0x55f44f5257c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f5257c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe734005ab0 tx=0x7fe73402a410 comp rx=0 tx=0).stop 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 shutdown_connections 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f1319f0 0x55f44f5374d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f525d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f5315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe731600640 0 client.27665 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:38.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 --2- 172.21.15.27:0/3762736472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f5257c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 >> 172.21.15.27:0/3762736472 conn(0x55f44f167460 msgr2=0x55f44f51cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:38.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 shutdown_connections 2024-09-21T07:20:38.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 1 -- 172.21.15.27:0/3762736472 wait complete. 2024-09-21T07:20:38.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.217+0000 7fe7517cd740 20 client.27665 trim_cache size 0 max 16384 2024-09-21T07:20:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 Processor -- start 2024-09-21T07:20:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 -- start start 2024-09-21T07:20:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f520d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f517000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b5930 con 0x55f44f173070 2024-09-21T07:20:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b3200 con 0x55f44f14a1b0 2024-09-21T07:20:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.237+0000 7fe7517cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b1070 con 0x55f44f152390 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f520d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49872/0 (socket says 172.21.15.27:49872) 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe743e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f520d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40080/0 (socket says 172.21.15.27:40080) 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 -- 172.21.15.27:0/4137090299 learned_addr learned my addr 172.21.15.27:0/4137090299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe748c00640 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f517000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 -- 172.21.15.27:0/4137090299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 msgr2=0x55f44f520d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f520d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 -- 172.21.15.27:0/4137090299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 msgr2=0x55f44f517000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f517000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 -- 172.21.15.27:0/4137090299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44f51be50 con 0x55f44f152390 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe743e00640 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f520d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe748c00640 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f517000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.238+0000 7fe73be00640 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51b2a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0ad850 tx=0x7fe72c00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8f4283e4769d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.239+0000 7fe733e00640 1 -- 172.21.15.27:0/4137090299 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0adfc0 con 0x55f44f152390 2024-09-21T07:20:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.239+0000 7fe733e00640 1 -- 172.21.15.27:0/4137090299 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72c0b23a0 con 0x55f44f152390 2024-09-21T07:20:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.239+0000 7fe733e00640 1 -- 172.21.15.27:0/4137090299 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0ac590 con 0x55f44f152390 2024-09-21T07:20:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 -- 172.21.15.27:0/4137090299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 msgr2=0x55f44f51b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51b2a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0ad850 tx=0x7fe72c00ce60 comp rx=0 tx=0).stop 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 -- 172.21.15.27:0/4137090299 shutdown_connections 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f152390 0x55f44f51b2a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f173070 0x55f44f517000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 --2- 172.21.15.27:0/4137090299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f14a1b0 0x55f44f520d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 -- 172.21.15.27:0/4137090299 >> 172.21.15.27:0/4137090299 conn(0x55f44f167460 msgr2=0x55f44f51ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 -- 172.21.15.27:0/4137090299 shutdown_connections 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 -- 172.21.15.27:0/4137090299 wait complete. 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.240+0000 7fe7517cd740 1 -- start start 2024-09-21T07:20:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 1 Processor -- start 2024-09-21T07:20:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f14a1b0 0x55f44f5386c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f152390 0x55f44f538c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f173070 0x55f44f539140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b16c0 con 0x55f44f14a1b0 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b2b30 con 0x55f44f152390 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7441b1d90 con 0x55f44f173070 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.241+0000 7fe7517cd740 10 client.0 _ll_register_callbacks cb 0x7fe734005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f152390 0x55f44f538c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f152390 0x55f44f538c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40088/0 (socket says 172.21.15.27:40088) 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 -- 172.21.15.27:0/1141186955 learned_addr learned my addr 172.21.15.27:0/1141186955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe743e00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f14a1b0 0x55f44f5386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe748c00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f173070 0x55f44f539140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 -- 172.21.15.27:0/1141186955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f173070 msgr2=0x55f44f539140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f173070 0x55f44f539140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 -- 172.21.15.27:0/1141186955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f14a1b0 msgr2=0x55f44f5386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f14a1b0 0x55f44f5386c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 -- 172.21.15.27:0/1141186955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44f5a0dd0 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe743e00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f14a1b0 0x55f44f5386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe73be00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f152390 0x55f44f538c00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c6a0 tx=0x7fe72c0d10b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4a1acb6d23ab1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.242+0000 7fe731600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe731600640 1 -- 172.21.15.27:0/1141186955 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0e5640 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe731600640 1 -- 172.21.15.27:0/1141186955 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe72c0a9210 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c0b23a0 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe731600640 1 -- 172.21.15.27:0/1141186955 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe72c010720 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe7517cd740 10 client.27671 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f44f5a2060 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe731600640 1 -- 172.21.15.27:0/1141186955 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72c0b23a0 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe731600640 1 -- 172.21.15.27:0/1141186955 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe72c0b3730 con 0x55f44f152390 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe7517cd740 10 client.27671 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe7517cd740 10 client.27671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.243+0000 7fe7517cd740 20 client.27671 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.244+0000 7fe7517cd740 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f106da0 0x55f44f5a4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.244+0000 7fe7517cd740 4 client.27671 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:20:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.244+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f44f48e970 con 0x55f44f106da0 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.244+0000 7fe743e00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f106da0 0x55f44f5a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.244+0000 7fe743e00640 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f106da0 0x55f44f5a4730 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3882d7504a012c09 server_cookie=6fa390e8f873280 in_seq=0 out_seq=0 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.244+0000 7fe731600640 10 client.27671 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe731600640 1 -- 172.21.15.27:0/1141186955 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x55f44f48e970 con 0x55f44f106da0 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe731600640 10 client.27671 handle_command_reply: tid=0 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 client.27671 shutdown 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f152390 msgr2=0x55f44f538c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f152390 0x55f44f538c00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c6a0 tx=0x7fe72c0d10b0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 shutdown_connections 2024-09-21T07:20:38.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f44f106da0 0x55f44f5a4730 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f44f173070 0x55f44f539140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f44f152390 0x55f44f538c00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 --2- 172.21.15.27:0/1141186955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f44f14a1b0 0x55f44f5386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 >> 172.21.15.27:0/1141186955 conn(0x55f44f167460 msgr2=0x55f44f526a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe731600640 0 client.27671 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 shutdown_connections 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 1 -- 172.21.15.27:0/1141186955 wait complete. 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.245+0000 7fe7517cd740 20 client.27671 trim_cache size 0 max 16384 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:20:38.250 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.251 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:20:38.252 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.253 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.255 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:38.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7280b6680 msgr2=0x7fe7280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7280b6680 0x7fe7280b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0099a0 tx=0x7fe73c009500 comp rx=0 tx=0).stop 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 msgr2=0x7fe744075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 0x7fe744075c50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00e9f0 tx=0x7fe72c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 shutdown_connections 2024-09-21T07:20:38.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7441094a0 0x7fe744075c50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe7280b6680 0x7fe7280b8b90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe744109df0 0x7fe744076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 --2- 172.21.15.27:0/4287466166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe744103b60 0x7fe744075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:38.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 >> 172.21.15.27:0/4287466166 conn(0x7fe74407b850 msgr2=0x7fe744107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:38.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 shutdown_connections 2024-09-21T07:20:38.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:38.246+0000 7fe74ac00640 1 -- 172.21.15.27:0/4287466166 wait complete. 2024-09-21T07:20:39.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 -- 172.21.15.27:0/1153824757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0114740 msgr2=0x7f47e0116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 --2- 172.21.15.27:0/1153824757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0114740 0x7f47e0116b60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009ca0 tx=0x7f47d802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 -- 172.21.15.27:0/1153824757 shutdown_connections 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 --2- 172.21.15.27:0/1153824757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0078290 0x7f47e0114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 --2- 172.21.15.27:0/1153824757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0114740 0x7f47e0116b60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 --2- 172.21.15.27:0/1153824757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e0077940 0x7f47e0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 -- 172.21.15.27:0/1153824757 >> 172.21.15.27:0/1153824757 conn(0x7f47e0101810 msgr2=0x7f47e0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 -- 172.21.15.27:0/1153824757 shutdown_connections 2024-09-21T07:20:39.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.009+0000 7f47e5c00640 1 -- 172.21.15.27:0/1153824757 wait complete. 2024-09-21T07:20:39.011 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:38 smithi027 ceph-mon[30779]: pgmap v610: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:20:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.010+0000 7f47e5c00640 1 Processor -- start 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.010+0000 7f47e5c00640 1 -- start start 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47e5c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0077940 0x7f47e01a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47e5c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0078290 0x7f47e01a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47e5c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e0114740 0x7f47e01a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47e5c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47e011a0b0 con 0x7f47e0078290 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47e5c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0119f30 con 0x7f47e0114740 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47e5c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47e011a230 con 0x7f47e0077940 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47dea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0077940 0x7f47e01a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47dea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0077940 0x7f47e01a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49890/0 (socket says 172.21.15.27:49890) 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47dea00640 1 -- 172.21.15.27:0/3727943543 learned_addr learned my addr 172.21.15.27:0/3727943543 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47d7e00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0078290 0x7f47e01a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.011+0000 7f47df400640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e0114740 0x7f47e01a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47dea00640 1 -- 172.21.15.27:0/3727943543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e0114740 msgr2=0x7f47e01a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47dea00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e0114740 0x7f47e01a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47dea00640 1 -- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0078290 msgr2=0x7f47e01a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47dea00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0078290 0x7f47e01a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47dea00640 1 -- 172.21.15.27:0/3727943543 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e01a69f0 con 0x7f47e0077940 2024-09-21T07:20:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47dea00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0077940 0x7f47e01a0a80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f47d000e9f0 tx=0x7f47d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47d7400640 1 -- 172.21.15.27:0/3727943543 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d000cd80 con 0x7f47e0077940 2024-09-21T07:20:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.012+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e01a6ce0 con 0x7f47e0077940 2024-09-21T07:20:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.013+0000 7f47d7400640 1 -- 172.21.15.27:0/3727943543 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d00108e0 con 0x7f47e0077940 2024-09-21T07:20:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.013+0000 7f47d7400640 1 -- 172.21.15.27:0/3727943543 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0018980 con 0x7f47e0077940 2024-09-21T07:20:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.013+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e01ad440 con 0x7f47e0077940 2024-09-21T07:20:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.014+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e0077d50 con 0x7f47e0077940 2024-09-21T07:20:39.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.017+0000 7f47d7400640 1 -- 172.21.15.27:0/3727943543 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f47d0010430 con 0x7f47e0077940 2024-09-21T07:20:39.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.018+0000 7f47d7400640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47c40b6680 0x7f47c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:39.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.018+0000 7f47d7400640 1 -- 172.21.15.27:0/3727943543 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f47d00e8390 con 0x7f47e0077940 2024-09-21T07:20:39.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.018+0000 7f47d7e00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47c40b6680 0x7f47c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:39.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.019+0000 7f47d7e00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47c40b6680 0x7f47c40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f47e007a1d0 tx=0x7f47c8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:39.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.020+0000 7f47d7400640 1 -- 172.21.15.27:0/3727943543 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f47d00ae4d0 con 0x7f47e0077940 2024-09-21T07:20:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:38 smithi119 ceph-mon[34519]: pgmap v610: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:20:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:38 smithi153 ceph-mon[34747]: pgmap v610: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:20:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.260+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47e01a1bb0 con 0x7f47e0077940 2024-09-21T07:20:39.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.262+0000 7f47d7400640 1 -- 172.21.15.27:0/3727943543 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f47d00b28b0 con 0x7f47e0077940 2024-09-21T07:20:39.263 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:20:39.263 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:20:39.263 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:20:39.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47c40b6680 msgr2=0x7f47c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:39.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47c40b6680 0x7f47c40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f47e007a1d0 tx=0x7f47c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:39.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0077940 msgr2=0x7f47e01a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:39.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0077940 0x7f47e01a0a80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f47d000e9f0 tx=0x7f47d000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 shutdown_connections 2024-09-21T07:20:39.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f47c40b6680 0x7f47c40b8b90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e0114740 0x7f47e01a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e0078290 0x7f47e01a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 --2- 172.21.15.27:0/3727943543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47e0077940 0x7f47e01a0a80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:39.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 >> 172.21.15.27:0/3727943543 conn(0x7f47e0101810 msgr2=0x7f47e0116790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:39.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 shutdown_connections 2024-09-21T07:20:39.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:39.264+0000 7f47e5c00640 1 -- 172.21.15.27:0/3727943543 wait complete. 2024-09-21T07:20:39.542 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:20:39.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:20:39.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:39 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3727943543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:20:40.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:39 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3727943543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:20:40.078 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:39 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3727943543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:20:40.275 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T07:20:40.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T07:20:40.389 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:40.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:40 smithi027 ceph-mon[30779]: pgmap v611: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:20:41.031 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:40 smithi119 ceph-mon[34519]: pgmap v611: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.038+0000 7fc9dd400640 1 -- 172.21.15.27:0/2393225499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d810de30 msgr2=0x7fc9d8110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.038+0000 7fc9dd400640 1 --2- 172.21.15.27:0/2393225499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d810de30 0x7fc9d8110320 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009ca0 tx=0x7fc9d002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.038+0000 7fc9dd400640 1 -- 172.21.15.27:0/2393225499 shutdown_connections 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.038+0000 7fc9dd400640 1 --2- 172.21.15.27:0/2393225499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d810de30 0x7fc9d8110320 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.038+0000 7fc9dd400640 1 --2- 172.21.15.27:0/2393225499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d8105320 0x7fc9d810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.038+0000 7fc9dd400640 1 --2- 172.21.15.27:0/2393225499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d81049d0 0x7fc9d8104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.039+0000 7fc9dd400640 1 -- 172.21.15.27:0/2393225499 >> 172.21.15.27:0/2393225499 conn(0x7fc9d8079740 msgr2=0x7fc9d8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.039+0000 7fc9dd400640 1 -- 172.21.15.27:0/2393225499 shutdown_connections 2024-09-21T07:20:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.039+0000 7fc9dd400640 1 -- 172.21.15.27:0/2393225499 wait complete. 2024-09-21T07:20:41.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.040+0000 7fc9dd400640 1 Processor -- start 2024-09-21T07:20:41.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.040+0000 7fc9dd400640 1 -- start start 2024-09-21T07:20:41.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.040+0000 7fc9dd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d81049d0 0x7fc9d8102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.040+0000 7fc9dd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d8105320 0x7fc9d8102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.040+0000 7fc9dd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d810de30 0x7fc9d8103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9dd400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8113ab0 con 0x7fc9d8105320 2024-09-21T07:20:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9dd400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8113930 con 0x7fc9d810de30 2024-09-21T07:20:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9dd400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8113c30 con 0x7fc9d81049d0 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d8105320 0x7fc9d8102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d8105320 0x7fc9d8102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35002/0 (socket says 172.21.15.27:35002) 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d5e00640 1 -- 172.21.15.27:0/4178156282 learned_addr learned my addr 172.21.15.27:0/4178156282 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d6800640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d81049d0 0x7fc9d8102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d7200640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d810de30 0x7fc9d8103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d5e00640 1 -- 172.21.15.27:0/4178156282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d81049d0 msgr2=0x7fc9d8102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d5e00640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d81049d0 0x7fc9d8102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d5e00640 1 -- 172.21.15.27:0/4178156282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d810de30 msgr2=0x7fc9d8103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.041+0000 7fc9d5e00640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d810de30 0x7fc9d8103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.042+0000 7fc9d5e00640 1 -- 172.21.15.27:0/4178156282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d81a87a0 con 0x7fc9d8105320 2024-09-21T07:20:41.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.042+0000 7fc9d6800640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d81049d0 0x7fc9d8102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:41.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.042+0000 7fc9d5e00640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d8105320 0x7fc9d8102c50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400e9c0 tx=0x7fc9c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.042+0000 7fc9cf400640 1 -- 172.21.15.27:0/4178156282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400ccc0 con 0x7fc9d8105320 2024-09-21T07:20:41.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.042+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d81a8a90 con 0x7fc9d8105320 2024-09-21T07:20:41.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.042+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d81a8fd0 con 0x7fc9d8105320 2024-09-21T07:20:41.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.042+0000 7fc9cf400640 1 -- 172.21.15.27:0/4178156282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c40054d0 con 0x7fc9d8105320 2024-09-21T07:20:41.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.043+0000 7fc9cf400640 1 -- 172.21.15.27:0/4178156282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4010430 con 0x7fc9d8105320 2024-09-21T07:20:41.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.044+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b8005290 con 0x7fc9d8105320 2024-09-21T07:20:41.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.046+0000 7fc9cf400640 1 -- 172.21.15.27:0/4178156282 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc9c4005020 con 0x7fc9d8105320 2024-09-21T07:20:41.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.048+0000 7fc9cf400640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9bc0b6700 0x7fc9bc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.048+0000 7fc9cf400640 1 -- 172.21.15.27:0/4178156282 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fc9c40e7b60 con 0x7fc9d8105320 2024-09-21T07:20:41.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.048+0000 7fc9d6800640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9bc0b6700 0x7fc9bc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.049+0000 7fc9d6800640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9bc0b6700 0x7fc9bc0b8c10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8104160 tx=0x7fc9c8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.056+0000 7fc9cf400640 1 -- 172.21.15.27:0/4178156282 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9c40adca0 con 0x7fc9d8105320 2024-09-21T07:20:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:40 smithi153 ceph-mon[34747]: pgmap v611: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:20:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.302+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9b8005850 con 0x7fc9d8105320 2024-09-21T07:20:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.303+0000 7fc9cf400640 1 -- 172.21.15.27:0/4178156282 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fc9c40b2080 con 0x7fc9d8105320 2024-09-21T07:20:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:20:41.306 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:20:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9bc0b6700 msgr2=0x7fc9bc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9bc0b6700 0x7fc9bc0b8c10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8104160 tx=0x7fc9c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d8105320 msgr2=0x7fc9d8102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d8105320 0x7fc9d8102c50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400e9c0 tx=0x7fc9c400ee80 comp rx=0 tx=0).stop 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 shutdown_connections 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc9bc0b6700 0x7fc9bc0b8c10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d810de30 0x7fc9d8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d8105320 0x7fc9d8102c50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 --2- 172.21.15.27:0/4178156282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9d81049d0 0x7fc9d8102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.305+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 >> 172.21.15.27:0/4178156282 conn(0x7fc9d8079740 msgr2=0x7fc9d80ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.306+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 shutdown_connections 2024-09-21T07:20:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.306+0000 7fc9dd400640 1 -- 172.21.15.27:0/4178156282 wait complete. 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.834+0000 7ff28e800640 1 -- 172.21.15.27:0/2522903835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff288076100 msgr2=0x7ff28810fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.834+0000 7ff28e800640 1 --2- 172.21.15.27:0/2522903835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff288076100 0x7ff28810fc10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff274009900 tx=0x7ff27402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.834+0000 7ff28e800640 1 -- 172.21.15.27:0/2522903835 shutdown_connections 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.834+0000 7ff28e800640 1 --2- 172.21.15.27:0/2522903835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff288110150 0x7ff288112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.834+0000 7ff28e800640 1 --2- 172.21.15.27:0/2522903835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff288076100 0x7ff28810fc10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.834+0000 7ff28e800640 1 --2- 172.21.15.27:0/2522903835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2880757b0 0x7ff288075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.835+0000 7ff28e800640 1 -- 172.21.15.27:0/2522903835 >> 172.21.15.27:0/2522903835 conn(0x7ff28807b8e0 msgr2=0x7ff28807bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.835+0000 7ff28e800640 1 -- 172.21.15.27:0/2522903835 shutdown_connections 2024-09-21T07:20:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.835+0000 7ff28e800640 1 -- 172.21.15.27:0/2522903835 wait complete. 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 Processor -- start 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 -- start start 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2880757b0 0x7ff28819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff288076100 0x7ff28819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff288110150 0x7ff2881a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff288115d50 con 0x7ff2880757b0 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff288115bd0 con 0x7ff288076100 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.836+0000 7ff28e800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff288115ed0 con 0x7ff288110150 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2880757b0 0x7ff28819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2880757b0 0x7ff28819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35012/0 (socket says 172.21.15.27:35012) 2024-09-21T07:20:41.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 -- 172.21.15.27:0/1671129078 learned_addr learned my addr 172.21.15.27:0/1671129078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:41.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff286a00640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff288076100 0x7ff28819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287e00640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff288110150 0x7ff2881a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 -- 172.21.15.27:0/1671129078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff288110150 msgr2=0x7ff2881a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff288110150 0x7ff2881a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 -- 172.21.15.27:0/1671129078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff288076100 msgr2=0x7ff28819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff288076100 0x7ff28819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff287400640 1 -- 172.21.15.27:0/1671129078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2881a2630 con 0x7ff2880757b0 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.837+0000 7ff286a00640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff288076100 0x7ff28819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.838+0000 7ff287400640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2880757b0 0x7ff28819c6c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff27800ea60 tx=0x7ff27800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.838+0000 7ff27fe00640 1 -- 172.21.15.27:0/1671129078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27800cde0 con 0x7ff2880757b0 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.838+0000 7ff27fe00640 1 -- 172.21.15.27:0/1671129078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2780035f0 con 0x7ff2880757b0 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.838+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881a2920 con 0x7ff2880757b0 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.838+0000 7ff27fe00640 1 -- 172.21.15.27:0/1671129078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278010690 con 0x7ff2880757b0 2024-09-21T07:20:41.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.838+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2881a9080 con 0x7ff2880757b0 2024-09-21T07:20:41.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.841+0000 7ff2955f2740 1 Processor -- start 2024-09-21T07:20:41.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff2955f2740 1 -- start start 2024-09-21T07:20:41.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab8479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab847ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff27fe00640 1 -- 172.21.15.27:0/1671129078 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff2780037a0 con 0x7ff2880757b0 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab848430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff2955f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608ab7bda90 con 0x5608ab477990 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff2955f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5608ab7acec0 con 0x5608ab4a0850 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff2955f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5608ab7b0cd0 con 0x5608ab47fb70 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff287400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab847ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff287e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab8479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.842+0000 7ff287400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab847ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35028/0 (socket says 172.21.15.27:35028) 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff287400640 1 -- 172.21.15.27:0/237479151 learned_addr learned my addr 172.21.15.27:0/237479151 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:41.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff286a00640 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab848430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff287e00640 1 -- 172.21.15.27:0/237479151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 msgr2=0x5608ab848430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff287e00640 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab848430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff287e00640 1 -- 172.21.15.27:0/237479151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 msgr2=0x5608ab847ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff287e00640 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab847ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff287e00640 1 -- 172.21.15.27:0/237479151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ab7529d0 con 0x5608ab4a0850 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.843+0000 7ff287e00640 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab8479b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff280009f00 tx=0x7ff280011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3412558e6f4bfe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.844+0000 7ff27cc00640 1 -- 172.21.15.27:0/237479151 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280003c30 con 0x5608ab4a0850 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.844+0000 7ff27cc00640 1 -- 172.21.15.27:0/237479151 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280003dd0 con 0x5608ab4a0850 2024-09-21T07:20:41.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.844+0000 7ff27cc00640 1 -- 172.21.15.27:0/237479151 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28000f960 con 0x5608ab4a0850 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.844+0000 7ff27fe00640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff26c0b6700 0x7ff26c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 -- 172.21.15.27:0/237479151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 msgr2=0x5608ab8479b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab8479b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff280009f00 tx=0x7ff280011810 comp rx=0 tx=0).stop 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff27fe00640 1 -- 172.21.15.27:0/1671129078 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7ff2780e7d80 con 0x7ff2880757b0 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff287400640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff26c0b6700 0x7ff26c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 -- 172.21.15.27:0/237479151 shutdown_connections 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab848430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab847ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 --2- 172.21.15.27:0/237479151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab8479b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 -- 172.21.15.27:0/237479151 >> 172.21.15.27:0/237479151 conn(0x5608ab494c40 msgr2=0x5608ab8458a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 -- 172.21.15.27:0/237479151 shutdown_connections 2024-09-21T07:20:41.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.845+0000 7ff2955f2740 1 -- 172.21.15.27:0/237479151 wait complete. 2024-09-21T07:20:41.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.846+0000 7ff287400640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff26c0b6700 0x7ff26c0b8c10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ad4c0 tx=0x7ff2780eb830 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.846+0000 7ff2955f2740 1 -- start start 2024-09-21T07:20:41.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.846+0000 7ff2955f2740 1 Processor -- start 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab853540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab853a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab85f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff2955f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608ab60e7b0 con 0x5608ab477990 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff2955f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5608ab751650 con 0x5608ab4a0850 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff2955f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5608ab856190 con 0x5608ab47fb70 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff2955f2740 10 client.0 _ll_register_callbacks cb 0x5608ab4b80a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff287e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab853540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff287e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab853540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35042/0 (socket says 172.21.15.27:35042) 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff287e00640 1 -- 172.21.15.27:0/1825156501 learned_addr learned my addr 172.21.15.27:0/1825156501 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff286a00640 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab853a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.847+0000 7ff287e00640 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab85f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff287e00640 1 -- 172.21.15.27:0/1825156501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 msgr2=0x5608ab853a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff287e00640 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab853a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff287e00640 1 -- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 msgr2=0x5608ab85f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff287e00640 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab85f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff287e00640 1 -- 172.21.15.27:0/1825156501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608ab51aba0 con 0x5608ab477990 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff287e00640 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab853540 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff280005c10 tx=0x7ff280009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2ccdb8c83c93c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff271600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff271600640 1 -- 172.21.15.27:0/1825156501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28002e060 con 0x5608ab477990 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff271600640 1 -- 172.21.15.27:0/1825156501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7ff28000fa90 con 0x5608ab477990 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff280003dd0 con 0x5608ab477990 2024-09-21T07:20:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff271600640 1 -- 172.21.15.27:0/1825156501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff28001f040 con 0x5608ab477990 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff2955f2740 10 client.17664 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.848+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608ab862090 con 0x5608ab477990 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff271600640 1 -- 172.21.15.27:0/1825156501 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280003dd0 con 0x5608ab477990 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff271600640 1 -- 172.21.15.27:0/1825156501 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff280018070 con 0x5608ab477990 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff2955f2740 10 client.17664 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff2955f2740 10 client.17664 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff2955f2740 20 client.17664 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab45f1d0 0x5608ab8654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff2955f2740 4 client.17664 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.849+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608ab7bf9e0 con 0x5608ab45f1d0 2024-09-21T07:20:41.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.850+0000 7ff286a00640 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab45f1d0 0x5608ab8654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.850+0000 7ff286a00640 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab45f1d0 0x5608ab8654a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4458b220a4cb6b99 server_cookie=b76f1fa6635422fa in_seq=0 out_seq=0 2024-09-21T07:20:41.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.850+0000 7ff271600640 10 client.17664 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:41.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff271600640 1 -- 172.21.15.27:0/1825156501 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608ab7bf9e0 con 0x5608ab45f1d0 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff271600640 10 client.17664 handle_command_reply: tid=0 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 client.17664 shutdown 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 msgr2=0x5608ab853540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab853540 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff280005c10 tx=0x7ff280009cc0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 shutdown_connections 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab45f1d0 0x5608ab8654a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab853a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab85f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1825156501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab853540 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 >> 172.21.15.27:0/1825156501 conn(0x5608ab494c40 msgr2=0x5608ab84ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.851+0000 7ff271600640 0 client.17664 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.852+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 shutdown_connections 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.852+0000 7ff2955f2740 1 -- 172.21.15.27:0/1825156501 wait complete. 2024-09-21T07:20:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.852+0000 7ff2955f2740 20 client.17664 trim_cache size 0 max 16384 2024-09-21T07:20:41.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 Processor -- start 2024-09-21T07:20:41.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 -- start start 2024-09-21T07:20:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab8456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab8493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab85f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b58d0 con 0x5608ab477990 2024-09-21T07:20:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b31a0 con 0x5608ab4a0850 2024-09-21T07:20:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.870+0000 7ff2955f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b1010 con 0x5608ab47fb70 2024-09-21T07:20:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab8456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab8456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35052/0 (socket says 172.21.15.27:35052) 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 -- 172.21.15.27:0/1037779728 learned_addr learned my addr 172.21.15.27:0/1037779728 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff286a00640 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab8493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 -- 172.21.15.27:0/1037779728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 msgr2=0x5608ab8493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab8493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 -- 172.21.15.27:0/1037779728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 msgr2=0x5608ab85f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab85f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 -- 172.21.15.27:0/1037779728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ab854600 con 0x5608ab477990 2024-09-21T07:20:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff286a00640 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab8493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.871+0000 7ff287e00640 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab8456f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff280009bb0 tx=0x7ff280005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c455a3289343856 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.872+0000 7ff273e00640 1 -- 172.21.15.27:0/1037779728 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28003e070 con 0x5608ab477990 2024-09-21T07:20:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.872+0000 7ff273e00640 1 -- 172.21.15.27:0/1037779728 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28000f480 con 0x5608ab477990 2024-09-21T07:20:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.872+0000 7ff273e00640 1 -- 172.21.15.27:0/1037779728 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28002e070 con 0x5608ab477990 2024-09-21T07:20:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 -- 172.21.15.27:0/1037779728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 msgr2=0x5608ab8456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab8456f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff280009bb0 tx=0x7ff280005f10 comp rx=0 tx=0).stop 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 -- 172.21.15.27:0/1037779728 shutdown_connections 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab47fb70 0x5608ab8493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab4a0850 0x5608ab85f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 --2- 172.21.15.27:0/1037779728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab477990 0x5608ab8456f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 -- 172.21.15.27:0/1037779728 >> 172.21.15.27:0/1037779728 conn(0x5608ab494c40 msgr2=0x5608ab844570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 -- 172.21.15.27:0/1037779728 shutdown_connections 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 -- 172.21.15.27:0/1037779728 wait complete. 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.873+0000 7ff2955f2740 1 -- start start 2024-09-21T07:20:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 1 Processor -- start 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab477990 0x5608ab8573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab47fb70 0x5608ab8578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab4a0850 0x5608ab857e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b1660 con 0x5608ab47fb70 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b2ad0 con 0x5608ab477990 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b1d30 con 0x5608ab4a0850 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff2955f2740 10 client.0 _ll_register_callbacks cb 0x7ff2780eb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff286a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab47fb70 0x5608ab8578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.874+0000 7ff286a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab47fb70 0x5608ab8578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35056/0 (socket says 172.21.15.27:35056) 2024-09-21T07:20:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff286a00640 1 -- 172.21.15.27:0/2023930427 learned_addr learned my addr 172.21.15.27:0/2023930427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff287e00640 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab477990 0x5608ab8573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff287e00640 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab4a0850 0x5608ab857e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff286a00640 1 -- 172.21.15.27:0/2023930427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab4a0850 msgr2=0x5608ab857e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff286a00640 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab4a0850 0x5608ab857e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff286a00640 1 -- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab477990 msgr2=0x5608ab8573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff286a00640 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab477990 0x5608ab8573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff286a00640 1 -- 172.21.15.27:0/2023930427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608ab8ceca0 con 0x5608ab47fb70 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff286a00640 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab47fb70 0x5608ab8578e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff274005940 tx=0x7ff274005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817e5d51184b886 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.875+0000 7ff271600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.876+0000 7ff271600640 1 -- 172.21.15.27:0/2023930427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274035040 con 0x5608ab47fb70 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.876+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28000f480 con 0x5608ab47fb70 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.876+0000 7ff271600640 1 -- 172.21.15.27:0/2023930427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7ff274004970 con 0x5608ab47fb70 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.876+0000 7ff271600640 1 -- 172.21.15.27:0/2023930427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff274058050 con 0x5608ab47fb70 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.876+0000 7ff2955f2740 10 client.17670 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.876+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608ab8cff40 con 0x5608ab47fb70 2024-09-21T07:20:41.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.876+0000 7ff271600640 1 -- 172.21.15.27:0/2023930427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28000f480 con 0x5608ab47fb70 2024-09-21T07:20:41.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff271600640 1 -- 172.21.15.27:0/2023930427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff2740314a0 con 0x5608ab47fb70 2024-09-21T07:20:41.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff2955f2740 10 client.17670 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff2955f2740 10 client.17670 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff2955f2740 20 client.17670 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff2955f2740 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab434580 0x5608ab8d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff2955f2740 4 client.17670 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x5608ab7bb490 con 0x5608ab434580 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.877+0000 7ff287e00640 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab434580 0x5608ab8d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.878+0000 7ff287e00640 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab434580 0x5608ab8d2730 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed28d586732ef6a server_cookie=43f475d8bcf92662 in_seq=0 out_seq=0 2024-09-21T07:20:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.878+0000 7ff271600640 10 client.17670 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:41.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.889+0000 7ff271600640 1 -- 172.21.15.27:0/2023930427 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5608ab7bb490 con 0x5608ab434580 2024-09-21T07:20:41.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.889+0000 7ff271600640 10 client.17670 handle_command_reply: tid=0 2024-09-21T07:20:41.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.889+0000 7ff2955f2740 1 client.17670 shutdown 2024-09-21T07:20:41.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4178156282' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:20:41.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab47fb70 msgr2=0x5608ab8578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab47fb70 0x5608ab8578e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff274005940 tx=0x7ff274005340 comp rx=0 tx=0).stop 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 shutdown_connections 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5608ab434580 0x5608ab8d2730 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608ab4a0850 0x5608ab857e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608ab47fb70 0x5608ab8578e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 --2- 172.21.15.27:0/2023930427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608ab477990 0x5608ab8573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 >> 172.21.15.27:0/2023930427 conn(0x5608ab494c40 msgr2=0x5608ab8488b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff271600640 0 client.17670 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 shutdown_connections 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 1 -- 172.21.15.27:0/2023930427 wait complete. 2024-09-21T07:20:41.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.890+0000 7ff2955f2740 20 client.17670 trim_cache size 0 max 16384 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 30, 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "b9c6a190": { 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 30, 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.1, 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:41.895 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff26c0b6700 msgr2=0x7ff26c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff26c0b6700 0x7ff26c0b8c10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ad4c0 tx=0x7ff2780eb830 comp rx=0 tx=0).stop 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2880757b0 msgr2=0x7ff28819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2880757b0 0x7ff28819c6c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff27800ea60 tx=0x7ff27800ef20 comp rx=0 tx=0).stop 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 shutdown_connections 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff288110150 0x7ff2881a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff26c0b6700 0x7ff26c0b8c10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff288076100 0x7ff28819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 --2- 172.21.15.27:0/1671129078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2880757b0 0x7ff28819c6c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:41.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 >> 172.21.15.27:0/1671129078 conn(0x7ff28807b8e0 msgr2=0x7ff288107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:41.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 shutdown_connections 2024-09-21T07:20:41.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:41.891+0000 7ff28e800640 1 -- 172.21.15.27:0/1671129078 wait complete. 2024-09-21T07:20:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4178156282' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:20:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4178156282' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:20:42.264 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:20:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.446+0000 7fe87a400640 1 -- 172.21.15.27:0/3178253530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109430 msgr2=0x7fe874109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.446+0000 7fe87a400640 1 --2- 172.21.15.27:0/3178253530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109430 0x7fe874109840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe860009860 tx=0x7fe86002a340 comp rx=0 tx=0).stop 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.446+0000 7fe87a400640 1 -- 172.21.15.27:0/3178253530 shutdown_connections 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.446+0000 7fe87a400640 1 --2- 172.21.15.27:0/3178253530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe874109d80 0x7fe8741105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.446+0000 7fe87a400640 1 --2- 172.21.15.27:0/3178253530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109430 0x7fe874109840 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.446+0000 7fe87a400640 1 --2- 172.21.15.27:0/3178253530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 0x7fe874103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.446+0000 7fe87a400640 1 -- 172.21.15.27:0/3178253530 >> 172.21.15.27:0/3178253530 conn(0x7fe8740ff500 msgr2=0x7fe874101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.447+0000 7fe87a400640 1 -- 172.21.15.27:0/3178253530 shutdown_connections 2024-09-21T07:20:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.447+0000 7fe87a400640 1 -- 172.21.15.27:0/3178253530 wait complete. 2024-09-21T07:20:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.448+0000 7fe87a400640 1 Processor -- start 2024-09-21T07:20:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.448+0000 7fe87a400640 1 -- start start 2024-09-21T07:20:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.449+0000 7fe87a400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 0x7fe874075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.449+0000 7fe87a400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe874109430 0x7fe874075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.449+0000 7fe87a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109d80 0x7fe874076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.449+0000 7fe87a400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe874113a40 con 0x7fe874109d80 2024-09-21T07:20:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.449+0000 7fe87a400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741138c0 con 0x7fe874103b00 2024-09-21T07:20:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.449+0000 7fe87a400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe874113bc0 con 0x7fe874109430 2024-09-21T07:20:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.449+0000 7fe873400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 0x7fe874075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 0x7fe874075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40218/0 (socket says 172.21.15.27:40218) 2024-09-21T07:20:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 -- 172.21.15.27:0/2762384960 learned_addr learned my addr 172.21.15.27:0/2762384960 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109d80 0x7fe874076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe872a00640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe874109430 0x7fe874075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 -- 172.21.15.27:0/2762384960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe874109430 msgr2=0x7fe874075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe874109430 0x7fe874075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 -- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109d80 msgr2=0x7fe874076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109d80 0x7fe874076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 -- 172.21.15.27:0/2762384960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8741a8770 con 0x7fe874103b00 2024-09-21T07:20:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.450+0000 7fe873400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 0x7fe874075710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe86400e9f0 tx=0x7fe86400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.451+0000 7fe86be00640 1 -- 172.21.15.27:0/2762384960 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400cd80 con 0x7fe874103b00 2024-09-21T07:20:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.451+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741a8a60 con 0x7fe874103b00 2024-09-21T07:20:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.451+0000 7fe86be00640 1 -- 172.21.15.27:0/2762384960 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8640108e0 con 0x7fe874103b00 2024-09-21T07:20:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.451+0000 7fe86be00640 1 -- 172.21.15.27:0/2762384960 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864018980 con 0x7fe874103b00 2024-09-21T07:20:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.451+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a8fd0 con 0x7fe874103b00 2024-09-21T07:20:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 Processor -- start 2024-09-21T07:20:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 -- start start 2024-09-21T07:20:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 0x55de282cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f074d0 0x55de282cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55de282426b0 con 0x55de27f281b0 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55de28231ae0 con 0x55de27f074d0 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.454+0000 7fe8814a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de282358f0 con 0x55de27eff2f0 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35072/0 (socket says 172.21.15.27:35072) 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 -- 172.21.15.27:0/3513602901 learned_addr learned my addr 172.21.15.27:0/3513602901 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe872a00640 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f074d0 0x55de282cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 -- 172.21.15.27:0/3513602901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 msgr2=0x55de282cd170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 0x55de282cd170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 -- 172.21.15.27:0/3513602901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f074d0 msgr2=0x55de282cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f074d0 0x55de282cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 -- 172.21.15.27:0/3513602901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de281fe780 con 0x55de27f281b0 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe86be00640 1 -- 172.21.15.27:0/2762384960 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe864010430 con 0x7fe874103b00 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.455+0000 7fe873e00640 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282ccc30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00e450 tx=0x7fe86c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fcb0c909844a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.456+0000 7fe868c00640 1 -- 172.21.15.27:0/3513602901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00d910 con 0x55de27f281b0 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.456+0000 7fe868c00640 1 -- 172.21.15.27:0/3513602901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c01c070 con 0x55de27f281b0 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.456+0000 7fe868c00640 1 -- 172.21.15.27:0/3513602901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c017de0 con 0x55de27f281b0 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 -- 172.21.15.27:0/3513602901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 msgr2=0x55de282ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282ccc30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00e450 tx=0x7fe86c0095c0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 -- 172.21.15.27:0/3513602901 shutdown_connections 2024-09-21T07:20:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 0x55de282cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f074d0 0x55de282cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3513602901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282ccc30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 -- 172.21.15.27:0/3513602901 >> 172.21.15.27:0/3513602901 conn(0x55de27f1c300 msgr2=0x55de282cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 -- 172.21.15.27:0/3513602901 shutdown_connections 2024-09-21T07:20:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 -- 172.21.15.27:0/3513602901 wait complete. 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.457+0000 7fe8814a3740 1 -- start start 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 1 Processor -- start 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe86be00640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8580b65c0 0x7fe8580b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282d8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282d87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55de28169ab0 con 0x55de27f281b0 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55de281d8570 con 0x55de27eff2f0 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de282db3b0 con 0x55de27f074d0 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe86be00640 1 -- 172.21.15.27:0/2762384960 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe8640e7e40 con 0x7fe874103b00 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.458+0000 7fe8814a3740 10 client.0 _ll_register_callbacks cb 0x55de281e7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe873e00640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8580b65c0 0x7fe8580b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282d87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282d87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49992/0 (socket says 172.21.15.27:49992) 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 -- 172.21.15.27:0/2165188729 learned_addr learned my addr 172.21.15.27:0/2165188729 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe873e00640 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282d8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe873400640 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 -- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 msgr2=0x55de282d8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282d8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 -- 172.21.15.27:0/2165188729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 msgr2=0x55de282d8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282d8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.459+0000 7fe872a00640 1 -- 172.21.15.27:0/2165188729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de27f93790 con 0x55de27f074d0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe873e00640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8580b65c0 0x7fe8580b8ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009e20 tx=0x7fe86c017a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe872a00640 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282d87a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe87406c9b0 tx=0x7fe860005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ab0ef1c418110c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe85d600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe85d600640 1 -- 172.21.15.27:0/2165188729 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860037040 con 0x55de27f074d0 2024-09-21T07:20:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86c01c070 con 0x55de27f074d0 2024-09-21T07:20:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe85d600640 1 -- 172.21.15.27:0/2165188729 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe860030450 con 0x55de27f074d0 2024-09-21T07:20:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe85d600640 1 -- 172.21.15.27:0/2165188729 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe86004e050 con 0x55de27f074d0 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe8814a3740 10 client.27673 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de2800c620 con 0x55de27f074d0 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.460+0000 7fe85d600640 1 -- 172.21.15.27:0/2165188729 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c01c070 con 0x55de27f074d0 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe85d600640 1 -- 172.21.15.27:0/2165188729 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe86003d450 con 0x55de27f074d0 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe8814a3740 10 client.27673 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe8814a3740 10 client.27673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe8814a3740 20 client.27673 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ee6d80 0x55de282ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe8814a3740 4 client.27673 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de28244600 con 0x55de27ee6d80 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.461+0000 7fe873400640 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ee6d80 0x55de282ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.462+0000 7fe873400640 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ee6d80 0x55de282ea3f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3651eb7f9fdf2c server_cookie=7bf10fdb2ecdaf54 in_seq=0 out_seq=0 2024-09-21T07:20:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.462+0000 7fe85d600640 10 client.27673 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe85d600640 1 -- 172.21.15.27:0/2165188729 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de28244600 con 0x55de27ee6d80 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe85d600640 10 client.27673 handle_command_reply: tid=0 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 client.27673 shutdown 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 msgr2=0x55de282d87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282d87a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe87406c9b0 tx=0x7fe860005d30 comp rx=0 tx=0).stop 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 shutdown_connections 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ee6d80 0x55de282ea3f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282d87a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282d8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe85d600640 0 client.27673 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2165188729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282d8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 >> 172.21.15.27:0/2165188729 conn(0x55de27f1c300 msgr2=0x55de282cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 shutdown_connections 2024-09-21T07:20:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 1 -- 172.21.15.27:0/2165188729 wait complete. 2024-09-21T07:20:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.463+0000 7fe8814a3740 20 client.27673 trim_cache size 0 max 16384 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.479+0000 7fe8814a3740 1 Processor -- start 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.479+0000 7fe8814a3740 1 -- start start 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.479+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 0x55de282ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.479+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f074d0 0x55de282d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f281b0 0x55de282d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe8814a3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741b5a00 con 0x55de27f074d0 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe8814a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741b32d0 con 0x55de27f281b0 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe8814a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741b1140 con 0x55de27eff2f0 2024-09-21T07:20:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f074d0 0x55de282d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f074d0 0x55de282d5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35094/0 (socket says 172.21.15.27:35094) 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 -- 172.21.15.27:0/3555089503 learned_addr learned my addr 172.21.15.27:0/3555089503 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe873400640 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 0x55de282ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe873e00640 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f281b0 0x55de282d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 -- 172.21.15.27:0/3555089503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 msgr2=0x55de282ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 0x55de282ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 -- 172.21.15.27:0/3555089503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f281b0 msgr2=0x55de282d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f281b0 0x55de282d5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe872a00640 1 -- 172.21.15.27:0/3555089503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de282d6740 con 0x55de27f074d0 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.480+0000 7fe873e00640 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f281b0 0x55de282d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.481+0000 7fe872a00640 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f074d0 0x55de282d5430 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe860000c00 tx=0x7fe8600096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525fa2ddbffe0944 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.481+0000 7fe85fe00640 1 -- 172.21.15.27:0/3555089503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860037050 con 0x55de27f074d0 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.481+0000 7fe85fe00640 1 -- 172.21.15.27:0/3555089503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe860030900 con 0x55de27f074d0 2024-09-21T07:20:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.481+0000 7fe85fe00640 1 -- 172.21.15.27:0/3555089503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8600052a0 con 0x55de27f074d0 2024-09-21T07:20:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 -- 172.21.15.27:0/3555089503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f074d0 msgr2=0x55de282d5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f074d0 0x55de282d5430 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe860000c00 tx=0x7fe8600096f0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 -- 172.21.15.27:0/3555089503 shutdown_connections 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27f281b0 0x55de282d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f074d0 0x55de282d5430 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 --2- 172.21.15.27:0/3555089503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27eff2f0 0x55de282ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 -- 172.21.15.27:0/3555089503 >> 172.21.15.27:0/3555089503 conn(0x55de27f1c300 msgr2=0x55de282cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 -- 172.21.15.27:0/3555089503 shutdown_connections 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.482+0000 7fe8814a3740 1 -- 172.21.15.27:0/3555089503 wait complete. 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.483+0000 7fe8814a3740 1 -- start start 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.483+0000 7fe8814a3740 1 Processor -- start 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.483+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.483+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe8814a3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe8814a3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741b1790 con 0x55de27f281b0 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe8814a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741b2c00 con 0x55de27eff2f0 2024-09-21T07:20:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe8814a3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741b1e60 con 0x55de27f074d0 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe8814a3740 10 client.0 _ll_register_callbacks cb 0x7fe86c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282de6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40260/0 (socket says 172.21.15.27:40260) 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 -- 172.21.15.27:0/2884301208 learned_addr learned my addr 172.21.15.27:0/2884301208 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe872a00640 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 -- 172.21.15.27:0/2884301208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 msgr2=0x55de282dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 -- 172.21.15.27:0/2884301208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 msgr2=0x55de282df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282df150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.484+0000 7fe873400640 1 -- 172.21.15.27:0/2884301208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de28353ea0 con 0x55de27eff2f0 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe873400640 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282de6d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe8640ad870 tx=0x7fe864010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=986c0e215a79a213 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe85d600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe85d600640 1 -- 172.21.15.27:0/2884301208 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8640a90a0 con 0x55de27eff2f0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe85d600640 1 -- 172.21.15.27:0/2884301208 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe8640aebb0 con 0x55de27eff2f0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe860030900 con 0x55de27eff2f0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe85d600640 1 -- 172.21.15.27:0/2884301208 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8640e5600 con 0x55de27eff2f0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe8814a3740 10 client.27716 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de28355190 con 0x55de27eff2f0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.485+0000 7fe85d600640 1 -- 172.21.15.27:0/2884301208 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe860030900 con 0x55de27eff2f0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe85d600640 1 -- 172.21.15.27:0/2884301208 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe8640d4080 con 0x55de27eff2f0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe8814a3740 10 client.27716 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe8814a3740 10 client.27716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe8814a3740 20 client.27716 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ebc1c0 0x55de28357950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe8814a3740 4 client.27716 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de282400b0 con 0x55de27ebc1c0 2024-09-21T07:20:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.486+0000 7fe872a00640 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ebc1c0 0x55de28357950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.487+0000 7fe872a00640 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ebc1c0 0x55de28357950 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb353e743c628f9 server_cookie=8647f51ea33dcb99 in_seq=0 out_seq=0 2024-09-21T07:20:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.487+0000 7fe85d600640 10 client.27716 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.500+0000 7fe85d600640 1 -- 172.21.15.27:0/2884301208 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de282400b0 con 0x55de27ebc1c0 2024-09-21T07:20:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe85d600640 10 client.27716 handle_command_reply: tid=0 2024-09-21T07:20:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 client.27716 shutdown 2024-09-21T07:20:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 msgr2=0x55de282de6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282de6d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe8640ad870 tx=0x7fe864010ca0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 shutdown_connections 2024-09-21T07:20:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de27ebc1c0 0x55de28357950 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de27f074d0 0x55de282dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de27f281b0 0x55de282df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 --2- 172.21.15.27:0/2884301208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de27eff2f0 0x55de282de6d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe85d600640 0 client.27716 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 >> 172.21.15.27:0/2884301208 conn(0x55de27f1c300 msgr2=0x55de282caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 shutdown_connections 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 1 -- 172.21.15.27:0/2884301208 wait complete. 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.501+0000 7fe8814a3740 20 client.27716 trim_cache size 0 max 16384 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:20:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:20:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:20:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:20:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8580b65c0 msgr2=0x7fe8580b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8580b65c0 0x7fe8580b8ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009e20 tx=0x7fe86c017a20 comp rx=0 tx=0).stop 2024-09-21T07:20:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 msgr2=0x7fe874075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 0x7fe874075710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe86400e9f0 tx=0x7fe86400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 shutdown_connections 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe874109430 0x7fe874075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8580b65c0 0x7fe8580b8ad0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874109d80 0x7fe874076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 --2- 172.21.15.27:0/2762384960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe874103b00 0x7fe874075710 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 >> 172.21.15.27:0/2762384960 conn(0x7fe8740ff500 msgr2=0x7fe874106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 shutdown_connections 2024-09-21T07:20:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:42.502+0000 7fe87a400640 1 -- 172.21.15.27:0/2762384960 wait complete. 2024-09-21T07:20:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:42 smithi119 ceph-mon[34519]: pgmap v612: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:20:43.039 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:42 smithi027 ceph-mon[30779]: pgmap v612: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:20:43.083 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b9c6a190', quiesce took 5.5 seconds. Will release after: 15.55 2024-09-21T07:20:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:42 smithi153 ceph-mon[34747]: pgmap v612: 129 pgs: 129 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:20:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 -- 172.21.15.27:0/33680936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3800076100 msgr2=0x7f380010f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 --2- 172.21.15.27:0/33680936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3800076100 0x7f380010f9b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009900 tx=0x7f37ec02a410 comp rx=0 tx=0).stop 2024-09-21T07:20:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 -- 172.21.15.27:0/33680936 shutdown_connections 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 --2- 172.21.15.27:0/33680936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f380010fef0 0x7f3800112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 --2- 172.21.15.27:0/33680936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3800076100 0x7f380010f9b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 --2- 172.21.15.27:0/33680936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38000757b0 0x7f3800075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 -- 172.21.15.27:0/33680936 >> 172.21.15.27:0/33680936 conn(0x7f38000ff440 msgr2=0x7f3800101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 -- 172.21.15.27:0/33680936 shutdown_connections 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.750+0000 7f3807000640 1 -- 172.21.15.27:0/33680936 wait complete. 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.751+0000 7f3807000640 1 Processor -- start 2024-09-21T07:20:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.751+0000 7f3807000640 1 -- start start 2024-09-21T07:20:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3807000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38000757b0 0x7f380019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3807000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3800076100 0x7f380019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3807000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f380010fef0 0x7f38001a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3807000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3800115aa0 con 0x7f38000757b0 2024-09-21T07:20:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3807000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3800115920 con 0x7f380010fef0 2024-09-21T07:20:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3807000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3800115c20 con 0x7f3800076100 2024-09-21T07:20:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3805c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38000757b0 0x7f380019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3805200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3800076100 0x7f380019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3805c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38000757b0 0x7f380019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35106/0 (socket says 172.21.15.27:35106) 2024-09-21T07:20:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.752+0000 7f3805c00640 1 -- 172.21.15.27:0/4170535810 learned_addr learned my addr 172.21.15.27:0/4170535810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.753+0000 7f3806600640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f380010fef0 0x7f38001a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.753+0000 7f3806600640 1 -- 172.21.15.27:0/4170535810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3800076100 msgr2=0x7f380019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.753+0000 7f3806600640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3800076100 0x7f380019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.753+0000 7f3806600640 1 -- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38000757b0 msgr2=0x7f380019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.753+0000 7f3806600640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38000757b0 0x7f380019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.753+0000 7f3806600640 1 -- 172.21.15.27:0/4170535810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38001a2400 con 0x7f380010fef0 2024-09-21T07:20:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.753+0000 7f3806600640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f380010fef0 0x7f38001a1cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009fd0 tx=0x7f37f800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.754+0000 7f37fe000640 1 -- 172.21.15.27:0/4170535810 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f800ce20 con 0x7f380010fef0 2024-09-21T07:20:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.754+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38001a26f0 con 0x7f380010fef0 2024-09-21T07:20:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.754+0000 7f37fe000640 1 -- 172.21.15.27:0/4170535810 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f8005020 con 0x7f380010fef0 2024-09-21T07:20:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.754+0000 7f37fe000640 1 -- 172.21.15.27:0/4170535810 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8010690 con 0x7f380010fef0 2024-09-21T07:20:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.754+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38001a8e50 con 0x7f380010fef0 2024-09-21T07:20:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.758+0000 7f380f28e740 1 Processor -- start 2024-09-21T07:20:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.758+0000 7f37fe000640 1 -- 172.21.15.27:0/4170535810 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f37f80054d0 con 0x7f380010fef0 2024-09-21T07:20:43.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.758+0000 7f380f28e740 1 -- start start 2024-09-21T07:20:43.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f380f28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0f10070 0x55eed12b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f380f28e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0ee71b0 0x55eed12b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f380f28e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0eef390 0x55eed12b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f380f28e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eed122dfb0 con 0x55eed0ee71b0 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f380f28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eed121d3e0 con 0x55eed0f10070 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f380f28e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eed12211f0 con 0x55eed0eef390 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f3805200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0f10070 0x55eed12b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f3805200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0f10070 0x55eed12b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40278/0 (socket says 172.21.15.27:40278) 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f3805c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0ee71b0 0x55eed12b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.759+0000 7f3805200640 1 -- 172.21.15.27:0/618016484 learned_addr learned my addr 172.21.15.27:0/618016484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f3805200640 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0eef390 0x55eed12b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f3805200640 1 -- 172.21.15.27:0/618016484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0eef390 msgr2=0x55eed12b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f3805200640 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0eef390 0x55eed12b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f3805200640 1 -- 172.21.15.27:0/618016484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0ee71b0 msgr2=0x55eed12b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f3805200640 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0ee71b0 0x55eed12b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f3805200640 1 -- 172.21.15.27:0/618016484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed11e7cd0 con 0x55eed0f10070 2024-09-21T07:20:43.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f3805200640 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0f10070 0x55eed12b6f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009900 tx=0x7f37ec005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0efcc4b3b1501c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:43.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.760+0000 7f37fe000640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37e80b6700 0x7f37e80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.761+0000 7f37f2a00640 1 -- 172.21.15.27:0/618016484 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec035030 con 0x55eed0f10070 2024-09-21T07:20:43.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.761+0000 7f37f2a00640 1 -- 172.21.15.27:0/618016484 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37ec0035f0 con 0x55eed0f10070 2024-09-21T07:20:43.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.761+0000 7f37f2a00640 1 -- 172.21.15.27:0/618016484 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0044e0 con 0x55eed0f10070 2024-09-21T07:20:43.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.761+0000 7f37fe000640 1 -- 172.21.15.27:0/4170535810 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f37f80e8180 con 0x7f380010fef0 2024-09-21T07:20:43.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.761+0000 7f3805c00640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37e80b6700 0x7f37e80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 -- 172.21.15.27:0/618016484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0f10070 msgr2=0x55eed12b6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0f10070 0x55eed12b6f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009900 tx=0x7f37ec005780 comp rx=0 tx=0).stop 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f3805c00640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37e80b6700 0x7f37e80b8c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55eed12b5bf0 tx=0x7f37f4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 -- 172.21.15.27:0/618016484 shutdown_connections 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0eef390 0x55eed12b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0ee71b0 0x55eed12b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 --2- 172.21.15.27:0/618016484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0f10070 0x55eed12b6f70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 -- 172.21.15.27:0/618016484 >> 172.21.15.27:0/618016484 conn(0x55eed0f04460 msgr2=0x55eed12b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 -- 172.21.15.27:0/618016484 shutdown_connections 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 -- 172.21.15.27:0/618016484 wait complete. 2024-09-21T07:20:43.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.762+0000 7f380f28e740 1 -- start start 2024-09-21T07:20:43.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 1 Processor -- start 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eed107e700 con 0x55eed0eef390 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eed11b8580 con 0x55eed0ee71b0 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eed12c56f0 con 0x55eed0f10070 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.763+0000 7f380f28e740 10 client.0 _ll_register_callbacks cb 0x55eed11c8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35118/0 (socket says 172.21.15.27:35118) 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40292/0 (socket says 172.21.15.27:40292) 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 -- 172.21.15.27:0/4206079472 learned_addr learned my addr 172.21.15.27:0/4206079472 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3806600640 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 -- 172.21.15.27:0/4206079472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 msgr2=0x55eed12cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 -- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 msgr2=0x55eed12c27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805200640 1 -- 172.21.15.27:0/4206079472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed0f7b650 con 0x55eed0eef390 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.764+0000 7f3805c00640 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f3805200640 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12c2ce0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f37ec031440 tx=0x7f37ec003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327c2c19a27682f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:43.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f37e7400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:43.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f37e7400640 1 -- 172.21.15.27:0/4206079472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec035050 con 0x55eed0eef390 2024-09-21T07:20:43.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eed11bab90 con 0x55eed0eef390 2024-09-21T07:20:43.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f37e7400640 1 -- 172.21.15.27:0/4206079472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f37ec0041a0 con 0x55eed0eef390 2024-09-21T07:20:43.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f37e7400640 1 -- 172.21.15.27:0/4206079472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37ec059050 con 0x55eed0eef390 2024-09-21T07:20:43.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f380f28e740 10 client.17700 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:43.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.765+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eed11d5950 con 0x55eed0eef390 2024-09-21T07:20:43.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f37e7400640 1 -- 172.21.15.27:0/4206079472 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eed11bab90 con 0x55eed0eef390 2024-09-21T07:20:43.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f37e7400640 1 -- 172.21.15.27:0/4206079472 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f37ec03d450 con 0x55eed0eef390 2024-09-21T07:20:43.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f380f28e740 10 client.17700 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f380f28e740 10 client.17700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f380f28e740 20 client.17700 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f380f28e740 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ece9f0 0x55eed12d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f380f28e740 4 client.17700 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eed122ff00 con 0x55eed0ece9f0 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.766+0000 7f3805c00640 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ece9f0 0x55eed12d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.767+0000 7f3805c00640 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ece9f0 0x55eed12d46e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7669c80f5bf584b0 server_cookie=cef2c5d18d6ac7d3 in_seq=0 out_seq=0 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.767+0000 7f37e7400640 10 client.17700 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f37e7400640 1 -- 172.21.15.27:0/4206079472 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eed122ff00 con 0x55eed0ece9f0 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f37e7400640 10 client.17700 handle_command_reply: tid=0 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 client.17700 shutdown 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 msgr2=0x55eed12c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12c2ce0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f37ec031440 tx=0x7f37ec003e60 comp rx=0 tx=0).stop 2024-09-21T07:20:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 shutdown_connections 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ece9f0 0x55eed12d46e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12c2ce0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 --2- 172.21.15.27:0/4206079472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f37e7400640 0 client.17700 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 >> 172.21.15.27:0/4206079472 conn(0x55eed0f04460 msgr2=0x55eed12ba100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.768+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 shutdown_connections 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.769+0000 7f380f28e740 1 -- 172.21.15.27:0/4206079472 wait complete. 2024-09-21T07:20:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.769+0000 7f380f28e740 20 client.17700 trim_cache size 0 max 16384 2024-09-21T07:20:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 Processor -- start 2024-09-21T07:20:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 -- start start 2024-09-21T07:20:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38001b57b0 con 0x55eed0eef390 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f38001b3080 con 0x55eed0ee71b0 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f380f28e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f38001b0ef0 con 0x55eed0f10070 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f3805200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.783+0000 7f3805200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35120/0 (socket says 172.21.15.27:35120) 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805200640 1 -- 172.21.15.27:0/2758218133 learned_addr learned my addr 172.21.15.27:0/2758218133 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805c00640 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3806600640 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805c00640 1 -- 172.21.15.27:0/2758218133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 msgr2=0x55eed12ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805c00640 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805c00640 1 -- 172.21.15.27:0/2758218133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 msgr2=0x55eed12b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805c00640 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805c00640 1 -- 172.21.15.27:0/2758218133 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed12b9260 con 0x55eed0ee71b0 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805200640 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.784+0000 7f3805c00640 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c4680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55eed121d470 tx=0x7f37f401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=595d1d2bfad954fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f37f2000640 1 -- 172.21.15.27:0/2758218133 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4003d50 con 0x55eed0ee71b0 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f37f2000640 1 -- 172.21.15.27:0/2758218133 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f401fdb0 con 0x55eed0ee71b0 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f37f2000640 1 -- 172.21.15.27:0/2758218133 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4004a00 con 0x55eed0ee71b0 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f380f28e740 1 -- 172.21.15.27:0/2758218133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 msgr2=0x55eed12c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f380f28e740 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c4680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55eed121d470 tx=0x7f37f401b000 comp rx=0 tx=0).stop 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f380f28e740 1 -- 172.21.15.27:0/2758218133 shutdown_connections 2024-09-21T07:20:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f380f28e740 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed12ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f380f28e740 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f380f28e740 1 --2- 172.21.15.27:0/2758218133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12c4680 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.785+0000 7f380f28e740 1 -- 172.21.15.27:0/2758218133 >> 172.21.15.27:0/2758218133 conn(0x55eed0f04460 msgr2=0x55eed12c1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.786+0000 7f380f28e740 1 -- 172.21.15.27:0/2758218133 shutdown_connections 2024-09-21T07:20:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.786+0000 7f380f28e740 1 -- 172.21.15.27:0/2758218133 wait complete. 2024-09-21T07:20:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.786+0000 7f380f28e740 1 -- start start 2024-09-21T07:20:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.786+0000 7f380f28e740 1 Processor -- start 2024-09-21T07:20:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f380f28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12d6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f380f28e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f380f28e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed1337ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f380f28e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38001b1540 con 0x55eed0eef390 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f380f28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f38001b29b0 con 0x55eed0ee71b0 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f380f28e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f38001b1c10 con 0x55eed0f10070 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f380f28e740 10 client.0 _ll_register_callbacks cb 0x7f37ec0316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f3805c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f3805200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f3805200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12d6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35132/0 (socket says 172.21.15.27:35132) 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f3805200640 1 -- 172.21.15.27:0/1507442719 learned_addr learned my addr 172.21.15.27:0/1507442719 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.787+0000 7f3806600640 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed1337ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f3805200640 1 -- 172.21.15.27:0/1507442719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 msgr2=0x55eed1337ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f3805200640 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed1337ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f3805200640 1 -- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 msgr2=0x55eed12d6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f3805200640 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12d6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f3805200640 1 -- 172.21.15.27:0/1507442719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed1338510 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f3805c00640 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12d6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f3805200640 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12d6610 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f37ec03da10 tx=0x7f37ec005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9605852634c165 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f37e7400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f37e7400640 1 -- 172.21.15.27:0/1507442719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec035070 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f37e7400640 1 -- 172.21.15.27:0/1507442719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f37ec031980 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.788+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f401fdb0 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f37e7400640 1 -- 172.21.15.27:0/1507442719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37ec03dc90 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 10 client.17706 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eed133f830 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f37e7400640 1 -- 172.21.15.27:0/1507442719 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f401fdb0 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f37e7400640 1 -- 172.21.15.27:0/1507442719 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f37ec003f00 con 0x55eed0eef390 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 10 client.17706 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 10 client.17706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:20:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 20 client.17706 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ea3da0 0x55eed1341df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 4 client.17706 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:20:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.789+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eed122b9b0 con 0x55eed0ea3da0 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.790+0000 7f3805c00640 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ea3da0 0x55eed1341df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.790+0000 7f3805c00640 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ea3da0 0x55eed1341df0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e63d1f66d4c0456e server_cookie=e904a721bce23556 in_seq=0 out_seq=0 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.790+0000 7f37e7400640 10 client.17706 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.790+0000 7f37e7400640 1 -- 172.21.15.27:0/1507442719 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eed122b9b0 con 0x55eed0ea3da0 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.790+0000 7f37e7400640 10 client.17706 handle_command_reply: tid=0 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.790+0000 7f380f28e740 1 client.17706 shutdown 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 msgr2=0x55eed12d6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12d6610 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f37ec03da10 tx=0x7f37ec005e40 comp rx=0 tx=0).stop 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 shutdown_connections 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55eed0ea3da0 0x55eed1341df0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed0f10070 0x55eed1337ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eed0eef390 0x55eed12d6610 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 --2- 172.21.15.27:0/1507442719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eed0ee71b0 0x55eed12d6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 >> 172.21.15.27:0/1507442719 conn(0x55eed0f04460 msgr2=0x55eed12c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f37e7400640 0 client.17706 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 shutdown_connections 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 1 -- 172.21.15.27:0/1507442719 wait complete. 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.791+0000 7f380f28e740 20 client.17706 trim_cache size 0 max 16384 2024-09-21T07:20:43.796 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:20:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37e80b6700 msgr2=0x7f37e80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37e80b6700 0x7f37e80b8c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55eed12b5bf0 tx=0x7f37f4005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f380010fef0 msgr2=0x7f38001a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f380010fef0 0x7f38001a1cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009fd0 tx=0x7f37f800ef10 comp rx=0 tx=0).stop 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 shutdown_connections 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3800076100 0x7f380019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37e80b6700 0x7f37e80b8c10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f380010fef0 0x7f38001a1cd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 --2- 172.21.15.27:0/4170535810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38000757b0 0x7f380019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 >> 172.21.15.27:0/4170535810 conn(0x7f38000ff440 msgr2=0x7f3800109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 shutdown_connections 2024-09-21T07:20:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:43.792+0000 7f3807000640 1 -- 172.21.15.27:0/4170535810 wait complete. 2024-09-21T07:20:44.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:20:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:44 smithi119 ceph-mon[34519]: pgmap v613: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:20:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:44 smithi027 ceph-mon[30779]: pgmap v613: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:20:45.173 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:44 smithi153 ceph-mon[34747]: pgmap v613: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:20:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.778+0000 7fe69fe00640 1 -- 172.21.15.27:0/623579023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0077940 msgr2=0x7fe6a0077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.778+0000 7fe69fe00640 1 --2- 172.21.15.27:0/623579023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0077940 0x7fe6a0077d50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe690009900 tx=0x7fe69002a410 comp rx=0 tx=0).stop 2024-09-21T07:20:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.779+0000 7fe69fe00640 1 -- 172.21.15.27:0/623579023 shutdown_connections 2024-09-21T07:20:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.779+0000 7fe69fe00640 1 --2- 172.21.15.27:0/623579023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0114570 0x7fe6a0116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.779+0000 7fe69fe00640 1 --2- 172.21.15.27:0/623579023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a0078290 0x7fe6a0114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.779+0000 7fe69fe00640 1 --2- 172.21.15.27:0/623579023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0077940 0x7fe6a0077d50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.779+0000 7fe69fe00640 1 -- 172.21.15.27:0/623579023 >> 172.21.15.27:0/623579023 conn(0x7fe6a0101680 msgr2=0x7fe6a0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:45.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.779+0000 7fe69fe00640 1 -- 172.21.15.27:0/623579023 shutdown_connections 2024-09-21T07:20:45.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.779+0000 7fe69fe00640 1 -- 172.21.15.27:0/623579023 wait complete. 2024-09-21T07:20:45.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.780+0000 7fe69fe00640 1 Processor -- start 2024-09-21T07:20:45.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.780+0000 7fe69fe00640 1 -- start start 2024-09-21T07:20:45.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.780+0000 7fe69fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0077940 0x7fe6a01a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.780+0000 7fe69fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0078290 0x7fe6a01a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.780+0000 7fe69fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a0114570 0x7fe6a01a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.780+0000 7fe69fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0119f40 con 0x7fe6a0078290 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.780+0000 7fe69fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0119dc0 con 0x7fe6a0114570 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a011a0c0 con 0x7fe6a0077940 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0078290 0x7fe6a01a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0078290 0x7fe6a01a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35152/0 (socket says 172.21.15.27:35152) 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 -- 172.21.15.27:0/1643060887 learned_addr learned my addr 172.21.15.27:0/1643060887 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69ea00640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0077940 0x7fe6a01a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69f400640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a0114570 0x7fe6a01a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 -- 172.21.15.27:0/1643060887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0077940 msgr2=0x7fe6a01a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0077940 0x7fe6a01a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 -- 172.21.15.27:0/1643060887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a0114570 msgr2=0x7fe6a01a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a0114570 0x7fe6a01a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.781+0000 7fe69e000640 1 -- 172.21.15.27:0/1643060887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a01a6860 con 0x7fe6a0078290 2024-09-21T07:20:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.782+0000 7fe69e000640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0078290 0x7fe6a01a0e90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00e9f0 tx=0x7fe68c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.782+0000 7fe697400640 1 -- 172.21.15.27:0/1643060887 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c00ccc0 con 0x7fe6a0078290 2024-09-21T07:20:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.782+0000 7fe697400640 1 -- 172.21.15.27:0/1643060887 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68c0108e0 con 0x7fe6a0078290 2024-09-21T07:20:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.782+0000 7fe697400640 1 -- 172.21.15.27:0/1643060887 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c018860 con 0x7fe6a0078290 2024-09-21T07:20:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.782+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a01a6b50 con 0x7fe6a0078290 2024-09-21T07:20:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.782+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01ad2b0 con 0x7fe6a0078290 2024-09-21T07:20:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.785+0000 7fe6ac5e3740 1 Processor -- start 2024-09-21T07:20:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe6ac5e3740 1 -- start start 2024-09-21T07:20:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe697400640 1 -- 172.21.15.27:0/1643060887 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe68c010430 con 0x7fe6a0078290 2024-09-21T07:20:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7bf2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bf39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557ec7b69fb0 con 0x557ec784c070 2024-09-21T07:20:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557ec7b593e0 con 0x557ec78231b0 2024-09-21T07:20:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ec7b5d1f0 con 0x557ec782b390 2024-09-21T07:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe69f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7bf2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe69ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe69ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40328/0 (socket says 172.21.15.27:40328) 2024-09-21T07:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe69ea00640 1 -- 172.21.15.27:0/108531158 learned_addr learned my addr 172.21.15.27:0/108531158 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe69e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bf39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.786+0000 7fe69f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7bf2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35158/0 (socket says 172.21.15.27:35158) 2024-09-21T07:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.787+0000 7fe69ea00640 1 -- 172.21.15.27:0/108531158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 msgr2=0x557ec7bf39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.787+0000 7fe69ea00640 1 --2- 172.21.15.27:0/108531158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bf39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.787+0000 7fe69ea00640 1 -- 172.21.15.27:0/108531158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 msgr2=0x557ec7bf2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.787+0000 7fe69ea00640 1 --2- 172.21.15.27:0/108531158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7bf2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.787+0000 7fe69ea00640 1 -- 172.21.15.27:0/108531158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec7b23cd0 con 0x557ec78231b0 2024-09-21T07:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.787+0000 7fe69ea00640 1 --2- 172.21.15.27:0/108531158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf34b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe690009230 tx=0x7fe690004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=325b03f02153aa53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.788+0000 7fe68be00640 1 -- 172.21.15.27:0/108531158 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690035040 con 0x557ec78231b0 2024-09-21T07:20:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.788+0000 7fe68be00640 1 -- 172.21.15.27:0/108531158 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6900054f0 con 0x557ec78231b0 2024-09-21T07:20:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.788+0000 7fe68be00640 1 -- 172.21.15.27:0/108531158 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6900035f0 con 0x557ec78231b0 2024-09-21T07:20:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.788+0000 7fe697400640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6840b6680 0x7fe6840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe697400640 1 -- 172.21.15.27:0/1643060887 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe68c0e7db0 con 0x7fe6a0078290 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe69f400640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6840b6680 0x7fe6840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/108531158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 msgr2=0x557ec7bf34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/108531158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf34b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe690009230 tx=0x7fe690004040 comp rx=0 tx=0).stop 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/108531158 shutdown_connections 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/108531158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bf39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/108531158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf34b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/108531158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7bf2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/108531158 >> 172.21.15.27:0/108531158 conn(0x557ec7840460 msgr2=0x557ec7bf0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/108531158 shutdown_connections 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe69f400640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6840b6680 0x7fe6840b8b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe698005fd0 tx=0x7fe698009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.789+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/108531158 wait complete. 2024-09-21T07:20:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.790+0000 7fe6ac5e3740 1 -- start start 2024-09-21T07:20:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.790+0000 7fe6ac5e3740 1 Processor -- start 2024-09-21T07:20:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec78231b0 0x557ec7bfe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bfec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec784c070 0x557ec7c0a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557ec79ba700 con 0x557ec78231b0 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557ec7af4580 con 0x557ec784c070 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ec7c016f0 con 0x557ec782b390 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe6ac5e3740 10 client.0 _ll_register_callbacks cb 0x557ec7b04470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe69ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec78231b0 0x557ec7bfe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe69f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec784c070 0x557ec7c0a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe69f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec784c070 0x557ec7c0a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40344/0 (socket says 172.21.15.27:40344) 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe69e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bfec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe69ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec78231b0 0x557ec7bfe720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35164/0 (socket says 172.21.15.27:35164) 2024-09-21T07:20:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.791+0000 7fe69f400640 1 -- 172.21.15.27:0/2260735154 learned_addr learned my addr 172.21.15.27:0/2260735154 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe69e000640 1 -- 172.21.15.27:0/2260735154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec784c070 msgr2=0x557ec7c0a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe69e000640 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec784c070 0x557ec7c0a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe69e000640 1 -- 172.21.15.27:0/2260735154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec78231b0 msgr2=0x557ec7bfe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe69e000640 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec78231b0 0x557ec7bfe720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe69e000640 1 -- 172.21.15.27:0/2260735154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec78b7650 con 0x557ec782b390 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe69e000640 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bfec60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0e50f0 tx=0x7fe68c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929f92a68a2df992 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe688c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe688c00640 1 -- 172.21.15.27:0/2260735154 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0ad1f0 con 0x557ec782b390 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557ec7af6b90 con 0x557ec782b390 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe688c00640 1 -- 172.21.15.27:0/2260735154 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe68c0ac570 con 0x557ec782b390 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe688c00640 1 -- 172.21.15.27:0/2260735154 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe68c0ad010 con 0x557ec782b390 2024-09-21T07:20:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe6ac5e3740 10 client.27709 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.792+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557ec7b11950 con 0x557ec782b390 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe688c00640 1 -- 172.21.15.27:0/2260735154 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557ec7af6b90 con 0x557ec782b390 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe688c00640 1 -- 172.21.15.27:0/2260735154 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe68c0ac770 con 0x557ec782b390 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe6ac5e3740 10 client.27709 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe6ac5e3740 10 client.27709 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe6ac5e3740 20 client.27709 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec780a9f0 0x557ec7c107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe6ac5e3740 4 client.27709 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ec7b6bf00 con 0x557ec780a9f0 2024-09-21T07:20:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.793+0000 7fe69ea00640 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec780a9f0 0x557ec7c107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.794+0000 7fe69ea00640 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec780a9f0 0x557ec7c107a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1722e97a0d9144a6 server_cookie=a568b4e264d6a62f in_seq=0 out_seq=0 2024-09-21T07:20:45.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.794+0000 7fe688c00640 10 client.27709 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:20:45.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe688c00640 1 -- 172.21.15.27:0/2260735154 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ec7b6bf00 con 0x557ec780a9f0 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe688c00640 10 client.27709 handle_command_reply: tid=0 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 client.27709 shutdown 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 msgr2=0x557ec7bfec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bfec60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0e50f0 tx=0x7fe68c009740 comp rx=0 tx=0).stop 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 shutdown_connections 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec780a9f0 0x557ec7c107a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7bfec60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec784c070 0x557ec7c0a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe688c00640 0 client.27709 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/2260735154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec78231b0 0x557ec7bfe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 >> 172.21.15.27:0/2260735154 conn(0x557ec7840460 msgr2=0x557ec7bf6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.795+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 shutdown_connections 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.796+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/2260735154 wait complete. 2024-09-21T07:20:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.796+0000 7fe6ac5e3740 20 client.27709 trim_cache size 0 max 16384 2024-09-21T07:20:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 Processor -- start 2024-09-21T07:20:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 -- start start 2024-09-21T07:20:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b9c70 con 0x557ec784c070 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b7540 con 0x557ec78231b0 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b53b0 con 0x557ec782b390 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.810+0000 7fe69f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c0a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35168/0 (socket says 172.21.15.27:35168) 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 -- 172.21.15.27:0/125442308 learned_addr learned my addr 172.21.15.27:0/125442308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 -- 172.21.15.27:0/125442308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 msgr2=0x557ec7c00350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c00350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 -- 172.21.15.27:0/125442308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 msgr2=0x557ec7bf4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 -- 172.21.15.27:0/125442308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec7bff9e0 con 0x557ec784c070 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69ea00640 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.811+0000 7fe69f400640 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c0a8e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe698003f20 tx=0x7fe69801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382829f5c915c003 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.812+0000 7fe68b400640 1 -- 172.21.15.27:0/125442308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69802d070 con 0x557ec784c070 2024-09-21T07:20:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.812+0000 7fe68b400640 1 -- 172.21.15.27:0/125442308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe69800fa50 con 0x557ec784c070 2024-09-21T07:20:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.812+0000 7fe68b400640 1 -- 172.21.15.27:0/125442308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69801dc30 con 0x557ec784c070 2024-09-21T07:20:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/125442308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 msgr2=0x557ec7c0a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c0a8e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe698003f20 tx=0x7fe69801a000 comp rx=0 tx=0).stop 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/125442308 shutdown_connections 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c0a8e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/125442308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7bf4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/125442308 >> 172.21.15.27:0/125442308 conn(0x557ec7840460 msgr2=0x557ec7bffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/125442308 shutdown_connections 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/125442308 wait complete. 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 -- start start 2024-09-21T07:20:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.813+0000 7fe6ac5e3740 1 Processor -- start 2024-09-21T07:20:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7c06950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c06f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe6ac5e3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c01900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b5a00 con 0x557ec784c070 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b6e70 con 0x557ec78231b0 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe6ac5e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b60d0 con 0x557ec782b390 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe6ac5e3740 10 client.0 _ll_register_callbacks cb 0x7fe68c0a9740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe69f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c01900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe69f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c01900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35182/0 (socket says 172.21.15.27:35182) 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe69f400640 1 -- 172.21.15.27:0/1568434890 learned_addr learned my addr 172.21.15.27:0/1568434890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe69e000640 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c06f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.814+0000 7fe69ea00640 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7c06950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe69f400640 1 -- 172.21.15.27:0/1568434890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 msgr2=0x557ec7c06f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe69f400640 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c06f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe69f400640 1 -- 172.21.15.27:0/1568434890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 msgr2=0x557ec7c06950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe69f400640 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7c06950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe69f400640 1 -- 172.21.15.27:0/1568434890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec7c01e40 con 0x557ec784c070 2024-09-21T07:20:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe69f400640 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c01900 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe698003ef0 tx=0x7fe698005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65ca80e20cd9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe688c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.815+0000 7fe688c00640 1 -- 172.21.15.27:0/1568434890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698036060 con 0x557ec784c070 2024-09-21T07:20:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe688c00640 1 -- 172.21.15.27:0/1568434890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fe69801d570 con 0x557ec784c070 2024-09-21T07:20:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe69800fa50 con 0x557ec784c070 2024-09-21T07:20:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe688c00640 1 -- 172.21.15.27:0/1568434890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe69803d050 con 0x557ec784c070 2024-09-21T07:20:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe6ac5e3740 10 client.17724 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557ec7c7b8b0 con 0x557ec784c070 2024-09-21T07:20:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe688c00640 1 -- 172.21.15.27:0/1568434890 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe69800fa50 con 0x557ec784c070 2024-09-21T07:20:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe688c00640 1 -- 172.21.15.27:0/1568434890 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe69801d910 con 0x557ec784c070 2024-09-21T07:20:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.816+0000 7fe6ac5e3740 10 client.17724 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe6ac5e3740 10 client.17724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe6ac5e3740 20 client.17724 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec77dfda0 0x557ec7c7ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe6ac5e3740 4 client.17724 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ec7b679b0 con 0x557ec77dfda0 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe69ea00640 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec77dfda0 0x557ec7c7ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe69ea00640 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec77dfda0 0x557ec7c7ded0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8341cfad9e4e22d5 server_cookie=9cb1c1ae4180b437 in_seq=0 out_seq=0 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.817+0000 7fe688c00640 10 client.17724 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe688c00640 1 -- 172.21.15.27:0/1568434890 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ec7b679b0 con 0x557ec77dfda0 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe688c00640 10 client.17724 handle_command_reply: tid=0 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 client.17724 shutdown 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 msgr2=0x557ec7c01900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c01900 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe698003ef0 tx=0x7fe698005f00 comp rx=0 tx=0).stop 2024-09-21T07:20:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 shutdown_connections 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x557ec77dfda0 0x557ec7c7ded0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ec782b390 0x557ec7c06f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557ec784c070 0x557ec7c01900 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 --2- 172.21.15.27:0/1568434890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ec78231b0 0x557ec7c06950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 >> 172.21.15.27:0/1568434890 conn(0x557ec7840460 msgr2=0x557ec7bf4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe688c00640 0 client.17724 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 shutdown_connections 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.818+0000 7fe6ac5e3740 1 -- 172.21.15.27:0/1568434890 wait complete. 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe6ac5e3740 20 client.17724 trim_cache size 0 max 16384 2024-09-21T07:20:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:20:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6840b6680 msgr2=0x7fe6840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6840b6680 0x7fe6840b8b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe698005fd0 tx=0x7fe698009500 comp rx=0 tx=0).stop 2024-09-21T07:20:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0078290 msgr2=0x7fe6a01a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0078290 0x7fe6a01a0e90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00e9f0 tx=0x7fe68c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 shutdown_connections 2024-09-21T07:20:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe6840b6680 0x7fe6840b8b90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a0114570 0x7fe6a01a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6a0078290 0x7fe6a01a0e90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 --2- 172.21.15.27:0/1643060887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0077940 0x7fe6a01a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.819+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 >> 172.21.15.27:0/1643060887 conn(0x7fe6a0101680 msgr2=0x7fe6a01165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.820+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 shutdown_connections 2024-09-21T07:20:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:45.820+0000 7fe69fe00640 1 -- 172.21.15.27:0/1643060887 wait complete. 2024-09-21T07:20:46.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:20:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:46 smithi119 ceph-mon[34519]: pgmap v614: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:20:47.084 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:46 smithi027 ceph-mon[30779]: pgmap v614: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:20:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:46 smithi153 ceph-mon[34747]: pgmap v614: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.681+0000 7f00be800640 1 -- 172.21.15.27:0/541588716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 msgr2=0x7f00b8116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.681+0000 7f00be800640 1 --2- 172.21.15.27:0/541588716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 0x7f00b8116990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009ca0 tx=0x7f00b002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.681+0000 7f00be800640 1 -- 172.21.15.27:0/541588716 shutdown_connections 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.681+0000 7f00be800640 1 --2- 172.21.15.27:0/541588716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b8078290 0x7f00b8114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.681+0000 7f00be800640 1 --2- 172.21.15.27:0/541588716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 0x7f00b8116990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.681+0000 7f00be800640 1 --2- 172.21.15.27:0/541588716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00b8077940 0x7f00b8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.681+0000 7f00be800640 1 -- 172.21.15.27:0/541588716 >> 172.21.15.27:0/541588716 conn(0x7f00b8101680 msgr2=0x7f00b8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.682+0000 7f00be800640 1 -- 172.21.15.27:0/541588716 shutdown_connections 2024-09-21T07:20:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.682+0000 7f00be800640 1 -- 172.21.15.27:0/541588716 wait complete. 2024-09-21T07:20:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 Processor -- start 2024-09-21T07:20:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 -- start start 2024-09-21T07:20:47.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b8077940 0x7f00b81a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00b8078290 0x7f00b81a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 0x7f00b81a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8119ed0 con 0x7f00b8078290 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8119d50 con 0x7f00b8114570 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00be800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00b811a050 con 0x7f00b8077940 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00bca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00b8078290 0x7f00b81a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 0x7f00b81a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00bca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00b8078290 0x7f00b81a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35196/0 (socket says 172.21.15.27:35196) 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00bca00640 1 -- 172.21.15.27:0/999419256 learned_addr learned my addr 172.21.15.27:0/999419256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:47.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.683+0000 7f00bd400640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b8077940 0x7f00b81a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.684+0000 7f00bde00640 1 -- 172.21.15.27:0/999419256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b8077940 msgr2=0x7f00b81a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.684+0000 7f00bde00640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b8077940 0x7f00b81a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.684+0000 7f00bde00640 1 -- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00b8078290 msgr2=0x7f00b81a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.684+0000 7f00bde00640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00b8078290 0x7f00b81a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.684+0000 7f00bde00640 1 -- 172.21.15.27:0/999419256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b81a67f0 con 0x7f00b8114570 2024-09-21T07:20:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.684+0000 7f00bde00640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 0x7f00b81a60c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f00b002add0 tx=0x7f00b0030d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.684+0000 7f00b6000640 1 -- 172.21.15.27:0/999419256 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0003e40 con 0x7f00b8114570 2024-09-21T07:20:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.685+0000 7f00b6000640 1 -- 172.21.15.27:0/999419256 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00b0040070 con 0x7f00b8114570 2024-09-21T07:20:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.685+0000 7f00b6000640 1 -- 172.21.15.27:0/999419256 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00459a0 con 0x7f00b8114570 2024-09-21T07:20:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.685+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b81a6a80 con 0x7f00b8114570 2024-09-21T07:20:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.685+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81ad240 con 0x7f00b8114570 2024-09-21T07:20:47.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.688+0000 7f00c6c00740 1 Processor -- start 2024-09-21T07:20:47.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00b6000640 1 -- 172.21.15.27:0/999419256 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f00b0004680 con 0x7f00b8114570 2024-09-21T07:20:47.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00c6c00740 1 -- start start 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe57ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b8390 0x564dfe5809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00c6c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564dfe4f6fb0 con 0x564dfe1b01b0 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00c6c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564dfe4e63e0 con 0x564dfe1b8390 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.689+0000 7f00c6c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564dfe4ea1f0 con 0x564dfe1d9070 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe57ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe57ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:50154/0 (socket says 172.21.15.27:50154) 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 -- 172.21.15.27:0/1224075768 learned_addr learned my addr 172.21.15.27:0/1224075768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b8390 0x564dfe5809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 -- 172.21.15.27:0/1224075768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b8390 msgr2=0x564dfe5809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b8390 0x564dfe5809f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 -- 172.21.15.27:0/1224075768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 msgr2=0x564dfe5804b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5804b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 -- 172.21.15.27:0/1224075768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dfe4b0cd0 con 0x564dfe1d9070 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bd400640 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.690+0000 7f00bca00640 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe57ff70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00a40095f0 tx=0x7f00a4009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d1ca36c1226315 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.691+0000 7f00aa600640 1 -- 172.21.15.27:0/1224075768 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a400f7c0 con 0x564dfe1d9070 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.691+0000 7f00b6000640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00a00b6680 0x7f00a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.691+0000 7f00aa600640 1 -- 172.21.15.27:0/1224075768 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a400fe10 con 0x564dfe1d9070 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.691+0000 7f00aa600640 1 -- 172.21.15.27:0/1224075768 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a401fc90 con 0x564dfe1d9070 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.691+0000 7f00b6000640 1 -- 172.21.15.27:0/999419256 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f00b01089b0 con 0x7f00b8114570 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.691+0000 7f00bd400640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00a00b6680 0x7f00a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00bd400640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00a00b6680 0x7f00a00b8b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x564dfe57ebf0 tx=0x7f00ac005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 -- 172.21.15.27:0/1224075768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 msgr2=0x564dfe57ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe57ff70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00a40095f0 tx=0x7f00a4009500 comp rx=0 tx=0).stop 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 -- 172.21.15.27:0/1224075768 shutdown_connections 2024-09-21T07:20:47.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b8390 0x564dfe5809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 --2- 172.21.15.27:0/1224075768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe57ff70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 -- 172.21.15.27:0/1224075768 >> 172.21.15.27:0/1224075768 conn(0x564dfe1cd460 msgr2=0x564dfe57de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:47.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 -- 172.21.15.27:0/1224075768 shutdown_connections 2024-09-21T07:20:47.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.692+0000 7f00c6c00740 1 -- 172.21.15.27:0/1224075768 wait complete. 2024-09-21T07:20:47.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.693+0000 7f00c6c00740 1 -- start start 2024-09-21T07:20:47.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.693+0000 7f00c6c00740 1 Processor -- start 2024-09-21T07:20:47.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b8390 0x564dfe58bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe597a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00c6c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564dfe347700 con 0x564dfe1b8390 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00c6c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564dfe481580 con 0x564dfe1b01b0 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00c6c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564dfe58e750 con 0x564dfe1d9070 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00c6c00740 10 client.0 _ll_register_callbacks cb 0x564dfe491470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00bca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b8390 0x564dfe58bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00bd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00bca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b8390 0x564dfe58bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35224/0 (socket says 172.21.15.27:35224) 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.694+0000 7f00bca00640 1 -- 172.21.15.27:0/3349134111 learned_addr learned my addr 172.21.15.27:0/3349134111 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bde00640 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe597a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bde00640 1 -- 172.21.15.27:0/3349134111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 msgr2=0x564dfe58b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bde00640 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bde00640 1 -- 172.21.15.27:0/3349134111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b8390 msgr2=0x564dfe58bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bde00640 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b8390 0x564dfe58bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bde00640 1 -- 172.21.15.27:0/3349134111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dfe244650 con 0x564dfe1d9070 2024-09-21T07:20:47.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bca00640 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b8390 0x564dfe58bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f00bde00640 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe597a70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f00b0105cf0 tx=0x7f00b00d7d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b60e37efe6c1d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.695+0000 7f009ea00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.696+0000 7f009ea00640 1 -- 172.21.15.27:0/3349134111 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00cdbf0 con 0x564dfe1d9070 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.696+0000 7f009ea00640 1 -- 172.21.15.27:0/3349134111 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f00b00c8d80 con 0x564dfe1d9070 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.696+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564dfe483b90 con 0x564dfe1d9070 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.696+0000 7f009ea00640 1 -- 172.21.15.27:0/3349134111 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00b00cdf90 con 0x564dfe1d9070 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.696+0000 7f00c6c00740 10 client.27727 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.696+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564dfe49e950 con 0x564dfe1d9070 2024-09-21T07:20:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.696+0000 7f009ea00640 1 -- 172.21.15.27:0/3349134111 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564dfe483b90 con 0x564dfe1d9070 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f009ea00640 1 -- 172.21.15.27:0/3349134111 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f00b004d030 con 0x564dfe1d9070 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f00c6c00740 10 client.27727 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f00c6c00740 10 client.27727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f00c6c00740 20 client.27727 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f00c6c00740 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe1979f0 0x564dfe59d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f00c6c00740 4 client.27727 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dfe4f8f00 con 0x564dfe1979f0 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.697+0000 7f00bd400640 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe1979f0 0x564dfe59d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.698+0000 7f00bd400640 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe1979f0 0x564dfe59d920 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590d98a2b30aea10 server_cookie=4c7649dea1ff8ad8 in_seq=0 out_seq=0 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.698+0000 7f009ea00640 10 client.27727 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f009ea00640 1 -- 172.21.15.27:0/3349134111 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564dfe4f8f00 con 0x564dfe1979f0 2024-09-21T07:20:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f009ea00640 10 client.27727 handle_command_reply: tid=0 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 client.27727 shutdown 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 msgr2=0x564dfe597a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe597a70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f00b0105cf0 tx=0x7f00b00d7d40 comp rx=0 tx=0).stop 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 shutdown_connections 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe1979f0 0x564dfe59d920 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1d9070 0x564dfe597a70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b8390 0x564dfe58bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 --2- 172.21.15.27:0/3349134111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f009ea00640 0 client.27727 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 >> 172.21.15.27:0/3349134111 conn(0x564dfe1cd460 msgr2=0x564dfe583100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.699+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 shutdown_connections 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.700+0000 7f00c6c00740 1 -- 172.21.15.27:0/3349134111 wait complete. 2024-09-21T07:20:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.700+0000 7f00c6c00740 20 client.27727 trim_cache size 0 max 16384 2024-09-21T07:20:47.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 Processor -- start 2024-09-21T07:20:47.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 -- start start 2024-09-21T07:20:47.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe597970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1d9070 0x564dfe597eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b9b40 con 0x564dfe1d9070 2024-09-21T07:20:47.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b7410 con 0x564dfe1b01b0 2024-09-21T07:20:47.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00c6c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b5280 con 0x564dfe1b8390 2024-09-21T07:20:47.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00bde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1d9070 0x564dfe597eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00bde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1d9070 0x564dfe597eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35238/0 (socket says 172.21.15.27:35238) 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.714+0000 7f00bde00640 1 -- 172.21.15.27:0/2815266550 learned_addr learned my addr 172.21.15.27:0/2815266550 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bca00640 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe597970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bd400640 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bde00640 1 -- 172.21.15.27:0/2815266550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 msgr2=0x564dfe597970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bde00640 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe597970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bde00640 1 -- 172.21.15.27:0/2815266550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 msgr2=0x564dfe58d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bde00640 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bde00640 1 -- 172.21.15.27:0/2815266550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dfe579280 con 0x564dfe1d9070 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00bde00640 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1d9070 0x564dfe597eb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006010 tx=0x7f00b002a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b267f7b0c5cff05b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00a9c00640 1 -- 172.21.15.27:0/2815266550 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00cdbf0 con 0x564dfe1d9070 2024-09-21T07:20:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.715+0000 7f00a9c00640 1 -- 172.21.15.27:0/2815266550 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00b0036070 con 0x564dfe1d9070 2024-09-21T07:20:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.716+0000 7f00a9c00640 1 -- 172.21.15.27:0/2815266550 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00c8de0 con 0x564dfe1d9070 2024-09-21T07:20:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.716+0000 7f00c6c00740 1 -- 172.21.15.27:0/2815266550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1d9070 msgr2=0x564dfe597eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.716+0000 7f00c6c00740 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1d9070 0x564dfe597eb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006010 tx=0x7f00b002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 -- 172.21.15.27:0/2815266550 shutdown_connections 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe597970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1d9070 0x564dfe597eb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 --2- 172.21.15.27:0/2815266550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1b01b0 0x564dfe58d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 -- 172.21.15.27:0/2815266550 >> 172.21.15.27:0/2815266550 conn(0x564dfe1cd460 msgr2=0x564dfe58cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 -- 172.21.15.27:0/2815266550 shutdown_connections 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 -- 172.21.15.27:0/2815266550 wait complete. 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.717+0000 7f00c6c00740 1 -- start start 2024-09-21T07:20:47.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 1 Processor -- start 2024-09-21T07:20:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe593f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1d9070 0x564dfe58e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b58d0 con 0x564dfe1b01b0 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b6d40 con 0x564dfe1d9070 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b5fa0 con 0x564dfe1b8390 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00c6c00740 10 client.0 _ll_register_callbacks cb 0x7f00b00ca900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00bd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00bd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5939e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35242/0 (socket says 172.21.15.27:35242) 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00bd400640 1 -- 172.21.15.27:0/571211502 learned_addr learned my addr 172.21.15.27:0/571211502 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00bca00640 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe593f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.718+0000 7f00bde00640 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1d9070 0x564dfe58e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f00bd400640 1 -- 172.21.15.27:0/571211502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 msgr2=0x564dfe593f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f00bd400640 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe593f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f00bd400640 1 -- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1d9070 msgr2=0x564dfe58e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f00bd400640 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1d9070 0x564dfe58e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f00bd400640 1 -- 172.21.15.27:0/571211502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dfe594690 con 0x564dfe1b01b0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f00bd400640 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5939e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009460 tx=0x7f00ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb2916eac0a362d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f009ea00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.719+0000 7f009ea00640 1 -- 172.21.15.27:0/571211502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac02b430 con 0x564dfe1b01b0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b0036070 con 0x564dfe1b01b0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f009ea00640 1 -- 172.21.15.27:0/571211502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f00ac02b5d0 con 0x564dfe1b01b0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f009ea00640 1 -- 172.21.15.27:0/571211502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00ac03e050 con 0x564dfe1b01b0 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f00c6c00740 10 client.17742 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564dfe6088b0 con 0x564dfe1b01b0 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f009ea00640 1 -- 172.21.15.27:0/571211502 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00b0036070 con 0x564dfe1b01b0 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f009ea00640 1 -- 172.21.15.27:0/571211502 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f00ac030970 con 0x564dfe1b01b0 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f00c6c00740 10 client.17742 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f00c6c00740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.720+0000 7f00c6c00740 20 client.17742 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.721+0000 7f00c6c00740 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe16cda0 0x564dfe60b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.721+0000 7f00c6c00740 4 client.17742 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.721+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564dfe4f49b0 con 0x564dfe16cda0 2024-09-21T07:20:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.721+0000 7f00bca00640 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe16cda0 0x564dfe60b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.721+0000 7f00bca00640 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe16cda0 0x564dfe60b050 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1881a1c76d2971 server_cookie=8329650f18b0356b in_seq=0 out_seq=0 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.721+0000 7f009ea00640 10 client.17742 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f009ea00640 1 -- 172.21.15.27:0/571211502 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564dfe4f49b0 con 0x564dfe16cda0 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f009ea00640 10 client.17742 handle_command_reply: tid=0 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 client.17742 shutdown 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 msgr2=0x564dfe5939e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5939e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009460 tx=0x7f00ac017000 comp rx=0 tx=0).stop 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 shutdown_connections 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564dfe16cda0 0x564dfe60b050 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564dfe1b8390 0x564dfe593f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564dfe1d9070 0x564dfe58e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 --2- 172.21.15.27:0/571211502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfe1b01b0 0x564dfe5939e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f009ea00640 0 client.17742 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 >> 172.21.15.27:0/571211502 conn(0x564dfe1cd460 msgr2=0x564dfe580fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 shutdown_connections 2024-09-21T07:20:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.722+0000 7f00c6c00740 1 -- 172.21.15.27:0/571211502 wait complete. 2024-09-21T07:20:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00c6c00740 20 client.17742 trim_cache size 0 max 16384 2024-09-21T07:20:47.727 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00a00b6680 msgr2=0x7f00a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00a00b6680 0x7f00a00b8b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x564dfe57ebf0 tx=0x7f00ac005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 msgr2=0x7f00b81a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 0x7f00b81a60c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f00b002add0 tx=0x7f00b0030d80 comp rx=0 tx=0).stop 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 shutdown_connections 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f00a00b6680 0x7f00a00b8b90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b8114570 0x7f00b81a60c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00b8078290 0x7f00b81a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 --2- 172.21.15.27:0/999419256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00b8077940 0x7f00b81a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.723+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 >> 172.21.15.27:0/999419256 conn(0x7f00b8101680 msgr2=0x7f00b81165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:47.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.724+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 shutdown_connections 2024-09-21T07:20:47.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:47.724+0000 7f00be800640 1 -- 172.21.15.27:0/999419256 wait complete. 2024-09-21T07:20:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:48 smithi119 ceph-mon[34519]: pgmap v615: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:48 smithi027 ceph-mon[30779]: pgmap v615: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:49.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:48 smithi153 ceph-mon[34747]: pgmap v615: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:20:49.684 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:20:49.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:20:50.080 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:50.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 -- 172.21.15.27:0/661305818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc10b6c0 msgr2=0x7f22dc10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 --2- 172.21.15.27:0/661305818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc10b6c0 0x7f22dc10bad0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009900 tx=0x7f22c802a410 comp rx=0 tx=0).stop 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 -- 172.21.15.27:0/661305818 shutdown_connections 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 --2- 172.21.15.27:0/661305818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22dc10c010 0x7f22dc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 --2- 172.21.15.27:0/661305818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc10b6c0 0x7f22dc10bad0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 --2- 172.21.15.27:0/661305818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22dc105d80 0x7f22dc106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 -- 172.21.15.27:0/661305818 >> 172.21.15.27:0/661305818 conn(0x7f22dc101810 msgr2=0x7f22dc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 -- 172.21.15.27:0/661305818 shutdown_connections 2024-09-21T07:20:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.702+0000 7f22e4000640 1 -- 172.21.15.27:0/661305818 wait complete. 2024-09-21T07:20:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.703+0000 7f22e4000640 1 Processor -- start 2024-09-21T07:20:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.703+0000 7f22e4000640 1 -- start start 2024-09-21T07:20:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e4000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc105d80 0x7f22dc19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e4000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22dc10b6c0 0x7f22dc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e4000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22dc10c010 0x7f22dc1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e4000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc115cd0 con 0x7f22dc105d80 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e4000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc115b50 con 0x7f22dc10c010 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e4000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc115e50 con 0x7f22dc10b6c0 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc105d80 0x7f22dc19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc105d80 0x7f22dc19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44144/0 (socket says 172.21.15.27:44144) 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e1600640 1 -- 172.21.15.27:0/4037301820 learned_addr learned my addr 172.21.15.27:0/4037301820 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e0c00640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22dc10b6c0 0x7f22dc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.704+0000 7f22e2000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22dc10c010 0x7f22dc1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22e1600640 1 -- 172.21.15.27:0/4037301820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22dc10b6c0 msgr2=0x7f22dc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22e1600640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22dc10b6c0 0x7f22dc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22e1600640 1 -- 172.21.15.27:0/4037301820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22dc10c010 msgr2=0x7f22dc1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22e1600640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22dc10c010 0x7f22dc1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22e1600640 1 -- 172.21.15.27:0/4037301820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc1a2610 con 0x7f22dc105d80 2024-09-21T07:20:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22e2000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22dc10c010 0x7f22dc1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22e1600640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc105d80 0x7f22dc19c6a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f22d000e9f0 tx=0x7f22d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.705+0000 7f22da000640 1 -- 172.21.15.27:0/4037301820 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d000cd80 con 0x7f22dc105d80 2024-09-21T07:20:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.706+0000 7f22da000640 1 -- 172.21.15.27:0/4037301820 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d00108e0 con 0x7f22dc105d80 2024-09-21T07:20:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.706+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1a2900 con 0x7f22dc105d80 2024-09-21T07:20:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.706+0000 7f22da000640 1 -- 172.21.15.27:0/4037301820 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0018980 con 0x7f22dc105d80 2024-09-21T07:20:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.706+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1a9060 con 0x7f22dc105d80 2024-09-21T07:20:50.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.709+0000 7f22eadfa740 1 Processor -- start 2024-09-21T07:20:50.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.709+0000 7f22eadfa740 1 -- start start 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.709+0000 7f22da000640 1 -- 172.21.15.27:0/4037301820 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f22d0010430 con 0x7f22dc105d80 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.709+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8ad019350 0x55d8ad3c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.709+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8acff0490 0x55d8ad3c1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff8670 0x55d8ad3c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22eadfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad336e90 con 0x55d8acff0490 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22eadfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad3262c0 con 0x55d8acff8670 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22eadfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad32a0d0 con 0x55d8ad019350 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8acff0490 0x55d8ad3c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8ad019350 0x55d8ad3c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8ad019350 0x55d8ad3c0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51506/0 (socket says 172.21.15.27:51506) 2024-09-21T07:20:50.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 -- 172.21.15.27:0/4089187645 learned_addr learned my addr 172.21.15.27:0/4089187645 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:50.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8acff0490 0x55d8ad3c1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44146/0 (socket says 172.21.15.27:44146) 2024-09-21T07:20:50.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e0c00640 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff8670 0x55d8ad3c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 -- 172.21.15.27:0/4089187645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff8670 msgr2=0x55d8ad3c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff8670 0x55d8ad3c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 -- 172.21.15.27:0/4089187645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8acff0490 msgr2=0x55d8ad3c1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8acff0490 0x55d8ad3c1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.710+0000 7f22e2000640 1 -- 172.21.15.27:0/4089187645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ad084930 con 0x55d8ad019350 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.711+0000 7f22e2000640 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8ad019350 0x55d8ad3c0d10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009fd0 tx=0x7f22d40108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3015c7170cd6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.711+0000 7f22ce600640 1 -- 172.21.15.27:0/4089187645 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4011760 con 0x55d8ad019350 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.711+0000 7f22ce600640 1 -- 172.21.15.27:0/4089187645 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d4011db0 con 0x55d8ad019350 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.711+0000 7f22ce600640 1 -- 172.21.15.27:0/4089187645 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d400e3c0 con 0x55d8ad019350 2024-09-21T07:20:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.711+0000 7f22da000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22c40b6700 0x7f22c40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22da000640 1 -- 172.21.15.27:0/4037301820 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f22d00e7e80 con 0x7f22dc105d80 2024-09-21T07:20:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22e1600640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22c40b6700 0x7f22c40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22eadfa740 1 -- 172.21.15.27:0/4089187645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8ad019350 msgr2=0x55d8ad3c0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8ad019350 0x55d8ad3c0d10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009fd0 tx=0x7f22d40108e0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22eadfa740 1 -- 172.21.15.27:0/4089187645 shutdown_connections 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff8670 0x55d8ad3c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8acff0490 0x55d8ad3c1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4089187645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8ad019350 0x55d8ad3c0d10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22eadfa740 1 -- 172.21.15.27:0/4089187645 >> 172.21.15.27:0/4089187645 conn(0x55d8ad00d740 msgr2=0x55d8ad3bec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.712+0000 7f22e1600640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22c40b6700 0x7f22c40b8c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55d8ad3bf990 tx=0x7f22d00eb930 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.713+0000 7f22eadfa740 1 -- 172.21.15.27:0/4089187645 shutdown_connections 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.713+0000 7f22eadfa740 1 -- 172.21.15.27:0/4089187645 wait complete. 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.713+0000 7f22eadfa740 1 -- start start 2024-09-21T07:20:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.713+0000 7f22eadfa740 1 Processor -- start 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3d8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22eadfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad187890 con 0x55d8ad019350 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22eadfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad2c1160 con 0x55d8acff0490 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22eadfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad3cf4f0 con 0x55d8acff8670 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22eadfa740 10 client.0 _ll_register_callbacks cb 0x55d8ad2dda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22e2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22e2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50218/0 (socket says 172.21.15.27:50218) 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22e0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22e2000640 1 -- 172.21.15.27:0/1561085110 learned_addr learned my addr 172.21.15.27:0/1561085110 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.714+0000 7f22e2000640 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3d8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.715+0000 7f22e0c00640 1 -- 172.21.15.27:0/1561085110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 msgr2=0x55d8ad3cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.715+0000 7f22e0c00640 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.715+0000 7f22e0c00640 1 -- 172.21.15.27:0/1561085110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 msgr2=0x55d8ad3d8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.715+0000 7f22e0c00640 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3d8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.715+0000 7f22e0c00640 1 -- 172.21.15.27:0/1561085110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ad3d8dd0 con 0x55d8acff8670 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.715+0000 7f22e0c00640 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cce60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f22c802a930 tx=0x7f22c8032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa47ff6990858cda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.715+0000 7f22c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.716+0000 7f22c2a00640 1 -- 172.21.15.27:0/1561085110 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8035040 con 0x55d8acff8670 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.716+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d4011db0 con 0x55d8acff8670 2024-09-21T07:20:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.716+0000 7f22c2a00640 1 -- 172.21.15.27:0/1561085110 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f22c8004050 con 0x55d8acff8670 2024-09-21T07:20:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.716+0000 7f22c2a00640 1 -- 172.21.15.27:0/1561085110 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22c804e050 con 0x55d8acff8670 2024-09-21T07:20:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.716+0000 7f22eadfa740 10 client.27745 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.716+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8ad3db750 con 0x55d8acff8670 2024-09-21T07:20:50.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.716+0000 7f22c2a00640 1 -- 172.21.15.27:0/1561085110 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d4011db0 con 0x55d8acff8670 2024-09-21T07:20:50.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22c2a00640 1 -- 172.21.15.27:0/1561085110 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22c8004750 con 0x55d8acff8670 2024-09-21T07:20:50.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22eadfa740 10 client.27745 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:50.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22eadfa740 10 client.27745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22eadfa740 20 client.27745 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22eadfa740 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfd7cd0 0x55d8ad3dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22eadfa740 4 client.27745 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8ad338de0 con 0x55d8acfd7cd0 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.717+0000 7f22e2000640 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfd7cd0 0x55d8ad3dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.718+0000 7f22e2000640 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfd7cd0 0x55d8ad3dec60 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68e806fb0ef461 server_cookie=5153dcc6ec39008b in_seq=0 out_seq=0 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.718+0000 7f22c2a00640 10 client.27745 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22c2a00640 1 -- 172.21.15.27:0/1561085110 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8ad338de0 con 0x55d8acfd7cd0 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22c2a00640 10 client.27745 handle_command_reply: tid=0 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 client.27745 shutdown 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 msgr2=0x55d8ad3cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cce60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f22c802a930 tx=0x7f22c8032e90 comp rx=0 tx=0).stop 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 shutdown_connections 2024-09-21T07:20:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfd7cd0 0x55d8ad3dec60 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cce60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22c2a00640 0 client.27745 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 --2- 172.21.15.27:0/1561085110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 >> 172.21.15.27:0/1561085110 conn(0x55d8ad00d740 msgr2=0x55d8ad3bd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 shutdown_connections 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.719+0000 7f22eadfa740 1 -- 172.21.15.27:0/1561085110 wait complete. 2024-09-21T07:20:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.720+0000 7f22eadfa740 20 client.27745 trim_cache size 0 max 16384 2024-09-21T07:20:50.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.738+0000 7f22eadfa740 1 Processor -- start 2024-09-21T07:20:50.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.738+0000 7f22eadfa740 1 -- start start 2024-09-21T07:20:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.738+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.738+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.738+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.738+0000 7f22eadfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1b59a0 con 0x55d8ad019350 2024-09-21T07:20:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22eadfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1b3270 con 0x55d8acff0490 2024-09-21T07:20:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22eadfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1b10e0 con 0x55d8acff8670 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44158/0 (socket says 172.21.15.27:44158) 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 -- 172.21.15.27:0/4207705740 learned_addr learned my addr 172.21.15.27:0/4207705740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e0c00640 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 -- 172.21.15.27:0/4207705740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 msgr2=0x55d8ad3cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 -- 172.21.15.27:0/4207705740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 msgr2=0x55d8ad3bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e2000640 1 -- 172.21.15.27:0/4207705740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ad3ba400 con 0x55d8ad019350 2024-09-21T07:20:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.739+0000 7f22e0c00640 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.740+0000 7f22e2000640 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3c5f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f22d401ba30 tx=0x7f22d4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66691b476358e8d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.740+0000 7f22cdc00640 1 -- 172.21.15.27:0/4207705740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d401bad0 con 0x55d8ad019350 2024-09-21T07:20:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.740+0000 7f22cdc00640 1 -- 172.21.15.27:0/4207705740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d401bc70 con 0x55d8ad019350 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.740+0000 7f22cdc00640 1 -- 172.21.15.27:0/4207705740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4005280 con 0x55d8ad019350 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 -- 172.21.15.27:0/4207705740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 msgr2=0x55d8ad3c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3c5f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f22d401ba30 tx=0x7f22d4009500 comp rx=0 tx=0).stop 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 -- 172.21.15.27:0/4207705740 shutdown_connections 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3c5f60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 --2- 172.21.15.27:0/4207705740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 -- 172.21.15.27:0/4207705740 >> 172.21.15.27:0/4207705740 conn(0x55d8ad00d740 msgr2=0x55d8ad3be520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 -- 172.21.15.27:0/4207705740 shutdown_connections 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.741+0000 7f22eadfa740 1 -- 172.21.15.27:0/4207705740 wait complete. 2024-09-21T07:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.742+0000 7f22eadfa740 1 -- start start 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.742+0000 7f22eadfa740 1 Processor -- start 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.742+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.742+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.742+0000 7f22eadfa740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.742+0000 7f22eadfa740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1b1730 con 0x55d8ad019350 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22eadfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1b2ba0 con 0x55d8acff0490 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22eadfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1b1e00 con 0x55d8acff8670 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22eadfa740 10 client.0 _ll_register_callbacks cb 0x7f22c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44170/0 (socket says 172.21.15.27:44170) 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e2000640 1 -- 172.21.15.27:0/3046960135 learned_addr learned my addr 172.21.15.27:0/3046960135 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e0c00640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e2000640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e0c00640 1 -- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 msgr2=0x55d8ad3d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e0c00640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e0c00640 1 -- 172.21.15.27:0/3046960135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 msgr2=0x55d8ad3cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e0c00640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e0c00640 1 -- 172.21.15.27:0/3046960135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ad3cfe20 con 0x55d8acff8670 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.743+0000 7f22e2000640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22e0c00640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3d4f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009850 tx=0x7f22c80319a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceef4767c2b7ff2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22c2a00640 1 -- 172.21.15.27:0/3046960135 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8035070 con 0x55d8acff8670 2024-09-21T07:20:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22c2a00640 1 -- 172.21.15.27:0/3046960135 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f22c80048d0 con 0x55d8acff8670 2024-09-21T07:20:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d401bc70 con 0x55d8acff8670 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22c2a00640 1 -- 172.21.15.27:0/3046960135 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22c805f050 con 0x55d8acff8670 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22eadfa740 10 client.27751 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.744+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8ad449f80 con 0x55d8acff8670 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22c2a00640 1 -- 172.21.15.27:0/3046960135 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d401bc70 con 0x55d8acff8670 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22c2a00640 1 -- 172.21.15.27:0/3046960135 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22c805a030 con 0x55d8acff8670 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22eadfa740 10 client.27751 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22eadfa740 10 client.27751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22eadfa740 20 client.27751 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22eadfa740 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfad080 0x55d8ad44c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22eadfa740 4 client.27751 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:20:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8ad334890 con 0x55d8acfad080 2024-09-21T07:20:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.745+0000 7f22e2000640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfad080 0x55d8ad44c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.746+0000 7f22e2000640 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfad080 0x55d8ad44c730 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6da526513494f8a server_cookie=5136f10d973cf04e in_seq=0 out_seq=0 2024-09-21T07:20:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.746+0000 7f22c2a00640 10 client.27751 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22c2a00640 1 -- 172.21.15.27:0/3046960135 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8ad334890 con 0x55d8acfad080 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22c2a00640 10 client.27751 handle_command_reply: tid=0 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 client.27751 shutdown 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 msgr2=0x55d8ad3d4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3d4f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009850 tx=0x7f22c80319a0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 shutdown_connections 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d8acfad080 0x55d8ad44c730 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d8acff8670 0x55d8ad3d4f00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d8ad019350 0x55d8ad3cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 --2- 172.21.15.27:0/3046960135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d8acff0490 0x55d8ad3d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 >> 172.21.15.27:0/3046960135 conn(0x55d8ad00d740 msgr2=0x55d8ad3c1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22c2a00640 0 client.27751 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 shutdown_connections 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 1 -- 172.21.15.27:0/3046960135 wait complete. 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.748+0000 7f22eadfa740 20 client.27751 trim_cache size 0 max 16384 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "4426db32-e542-4900-a1fd-f4a7dd06dc29", 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:20:50.752 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22c40b6700 msgr2=0x7f22c40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22c40b6700 0x7f22c40b8c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55d8ad3bf990 tx=0x7f22d00eb930 comp rx=0 tx=0).stop 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc105d80 msgr2=0x7f22dc19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc105d80 0x7f22dc19c6a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f22d000e9f0 tx=0x7f22d000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 shutdown_connections 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22dc10b6c0 0x7f22dc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f22c40b6700 0x7f22c40b8c10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22dc10c010 0x7f22dc1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 --2- 172.21.15.27:0/4037301820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22dc105d80 0x7f22dc19c6a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.749+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 >> 172.21.15.27:0/4037301820 conn(0x7f22dc101810 msgr2=0x7f22dc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:50.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.750+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 shutdown_connections 2024-09-21T07:20:50.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:50.750+0000 7f22e4000640 1 -- 172.21.15.27:0/4037301820 wait complete. 2024-09-21T07:20:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:50 smithi119 ceph-mon[34519]: pgmap v616: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:20:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:50 smithi027 ceph-mon[30779]: pgmap v616: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:20:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:50 smithi153 ceph-mon[34747]: pgmap v616: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:20:51.646 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:20:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:52.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:51 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:20:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:51 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:51 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:20:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:51 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:52.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:51 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:20:52.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:51 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:52.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:51 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:20:52.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:51 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:51 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:20:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:51 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:51 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:20:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:51 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:20:52.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 -- 172.21.15.27:0/717137582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8440105d80 msgr2=0x7f8440106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 --2- 172.21.15.27:0/717137582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8440105d80 0x7f8440106190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f843002a410 comp rx=0 tx=0).stop 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 -- 172.21.15.27:0/717137582 shutdown_connections 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 --2- 172.21.15.27:0/717137582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844010c010 0x7f8440112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 --2- 172.21.15.27:0/717137582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f844010b6c0 0x7f844010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 --2- 172.21.15.27:0/717137582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8440105d80 0x7f8440106190 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 -- 172.21.15.27:0/717137582 >> 172.21.15.27:0/717137582 conn(0x7f8440101810 msgr2=0x7f8440103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 -- 172.21.15.27:0/717137582 shutdown_connections 2024-09-21T07:20:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.636+0000 7f8445000640 1 -- 172.21.15.27:0/717137582 wait complete. 2024-09-21T07:20:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 Processor -- start 2024-09-21T07:20:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 -- start start 2024-09-21T07:20:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8440105d80 0x7f844019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f844010b6c0 0x7f844019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844010c010 0x7f84401a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8440115d20 con 0x7f844010b6c0 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8440115ba0 con 0x7f8440105d80 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.638+0000 7f8445000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8440115ea0 con 0x7f844010c010 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f843de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8440105d80 0x7f844019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f8435400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f844010b6c0 0x7f844019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f843de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8440105d80 0x7f844019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50264/0 (socket says 172.21.15.27:50264) 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f843de00640 1 -- 172.21.15.27:0/2039404544 learned_addr learned my addr 172.21.15.27:0/2039404544 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f843e800640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844010c010 0x7f84401a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f8435400640 1 -- 172.21.15.27:0/2039404544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844010c010 msgr2=0x7f84401a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f8435400640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844010c010 0x7f84401a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f8435400640 1 -- 172.21.15.27:0/2039404544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8440105d80 msgr2=0x7f844019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f8435400640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8440105d80 0x7f844019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.639+0000 7f8435400640 1 -- 172.21.15.27:0/2039404544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84401a2640 con 0x7f844010b6c0 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.640+0000 7f8435400640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f844010b6c0 0x7f844019cc70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f844006caa0 tx=0x7f842800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.640+0000 7f8437400640 1 -- 172.21.15.27:0/2039404544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800cc40 con 0x7f844010b6c0 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.640+0000 7f8437400640 1 -- 172.21.15.27:0/2039404544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842800cde0 con 0x7f844010b6c0 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.640+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84401a2930 con 0x7f844010b6c0 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.640+0000 7f8437400640 1 -- 172.21.15.27:0/2039404544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428018780 con 0x7f844010b6c0 2024-09-21T07:20:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.640+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84401a9090 con 0x7f844010b6c0 2024-09-21T07:20:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.643+0000 7f844bc7d740 1 Processor -- start 2024-09-21T07:20:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.643+0000 7f844bc7d740 1 -- start start 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f8437400640 1 -- 172.21.15.27:0/2039404544 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8428010a90 con 0x7f844010b6c0 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298ad92b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298ad930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298ad935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f844bc7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56298ad09f70 con 0x56298a9c31b0 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f844bc7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56298acf93a0 con 0x56298a9ec070 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f844bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56298acfd1b0 con 0x56298a9cb390 2024-09-21T07:20:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f843de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298ad930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f843de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298ad930a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44204/0 (socket says 172.21.15.27:44204) 2024-09-21T07:20:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f843de00640 1 -- 172.21.15.27:0/2029622925 learned_addr learned my addr 172.21.15.27:0/2029622925 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f843e800640 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298ad92b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.644+0000 7f8435400640 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298ad935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.645+0000 7f843de00640 1 -- 172.21.15.27:0/2029622925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 msgr2=0x56298ad935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.645+0000 7f843de00640 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298ad935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.645+0000 7f843de00640 1 -- 172.21.15.27:0/2029622925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 msgr2=0x56298ad92b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.645+0000 7f843de00640 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298ad92b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.645+0000 7f843de00640 1 -- 172.21.15.27:0/2029622925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56298acc3c90 con 0x56298a9c31b0 2024-09-21T07:20:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.645+0000 7f843e800640 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298ad92b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.645+0000 7f843de00640 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298ad930a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8430006010 tx=0x7f8430004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adb56b660cbe21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.646+0000 7f842f400640 1 -- 172.21.15.27:0/2029622925 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430040070 con 0x56298a9c31b0 2024-09-21T07:20:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.646+0000 7f8437400640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84240b6700 0x7f84240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.646+0000 7f842f400640 1 -- 172.21.15.27:0/2029622925 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84300054d0 con 0x56298a9c31b0 2024-09-21T07:20:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.646+0000 7f842f400640 1 -- 172.21.15.27:0/2029622925 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035040 con 0x56298a9c31b0 2024-09-21T07:20:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.646+0000 7f8437400640 1 -- 172.21.15.27:0/2039404544 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f84280e7cf0 con 0x7f844010b6c0 2024-09-21T07:20:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.646+0000 7f843e800640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84240b6700 0x7f84240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 -- 172.21.15.27:0/2029622925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 msgr2=0x56298ad930a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f843e800640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84240b6700 0x7f84240b8c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8438009fd0 tx=0x7f8438009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298ad930a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8430006010 tx=0x7f8430004100 comp rx=0 tx=0).stop 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 -- 172.21.15.27:0/2029622925 shutdown_connections 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298ad935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298ad930a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2029622925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298ad92b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 -- 172.21.15.27:0/2029622925 >> 172.21.15.27:0/2029622925 conn(0x56298a9e0460 msgr2=0x56298ad90a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 -- 172.21.15.27:0/2029622925 shutdown_connections 2024-09-21T07:20:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.647+0000 7f844bc7d740 1 -- 172.21.15.27:0/2029622925 wait complete. 2024-09-21T07:20:52.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.648+0000 7f844bc7d740 1 -- start start 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.648+0000 7f844bc7d740 1 Processor -- start 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.648+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad9e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.648+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9cb390 0x56298ad9ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adaa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f844bc7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56298ab5a700 con 0x56298a9cb390 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f844bc7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56298ac94540 con 0x56298a9ec070 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f844bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56298ada12e0 con 0x56298a9c31b0 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f844bc7d740 10 client.0 _ll_register_callbacks cb 0x56298aca4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f8435400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9cb390 0x56298ad9ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f8435400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9cb390 0x56298ad9ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44216/0 (socket says 172.21.15.27:44216) 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f8435400640 1 -- 172.21.15.27:0/3962044897 learned_addr learned my addr 172.21.15.27:0/3962044897 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f843de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad9e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f843e800640 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adaa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f843de00640 1 -- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 msgr2=0x56298adaa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f843de00640 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adaa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f843de00640 1 -- 172.21.15.27:0/3962044897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9cb390 msgr2=0x56298ad9ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f843de00640 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9cb390 0x56298ad9ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f843de00640 1 -- 172.21.15.27:0/3962044897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56298aa57650 con 0x56298a9c31b0 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.649+0000 7f8435400640 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9cb390 0x56298ad9ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.650+0000 7f843de00640 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad9e720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8430005ab0 tx=0x7f8430009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a22b08e6e6540f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:52.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.650+0000 7f8423e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:20:52.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.650+0000 7f8423e00640 1 -- 172.21.15.27:0/3962044897 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035070 con 0x56298a9c31b0 2024-09-21T07:20:52.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.650+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56298ac96b50 con 0x56298a9c31b0 2024-09-21T07:20:52.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.650+0000 7f8423e00640 1 -- 172.21.15.27:0/3962044897 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f8430005000 con 0x56298a9c31b0 2024-09-21T07:20:52.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.650+0000 7f8423e00640 1 -- 172.21.15.27:0/3962044897 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8430040040 con 0x56298a9c31b0 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 10 client.27763 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56298acb1910 con 0x56298a9c31b0 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f8423e00640 1 -- 172.21.15.27:0/3962044897 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56298ac96b50 con 0x56298a9c31b0 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f8423e00640 1 -- 172.21.15.27:0/3962044897 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8430003b60 con 0x56298a9c31b0 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 10 client.27763 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 10 client.27763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 20 client.27763 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a9aa9f0 0x56298adb0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 4 client.27763 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56298ad0bec0 con 0x56298a9aa9f0 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.651+0000 7f8435400640 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a9aa9f0 0x56298adb0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.652+0000 7f8435400640 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a9aa9f0 0x56298adb0430 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a11e70ed26aeabc server_cookie=e0faff20f91f19cd in_seq=0 out_seq=0 2024-09-21T07:20:52.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.652+0000 7f8423e00640 10 client.27763 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:52.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f8423e00640 1 -- 172.21.15.27:0/3962044897 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56298ad0bec0 con 0x56298a9aa9f0 2024-09-21T07:20:52.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f8423e00640 10 client.27763 handle_command_reply: tid=0 2024-09-21T07:20:52.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 client.27763 shutdown 2024-09-21T07:20:52.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 msgr2=0x56298ad9e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad9e720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8430005ab0 tx=0x7f8430009260 comp rx=0 tx=0).stop 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 shutdown_connections 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a9aa9f0 0x56298adb0430 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adaa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9cb390 0x56298ad9ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 --2- 172.21.15.27:0/3962044897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad9e720 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f8423e00640 0 client.27763 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.653+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 >> 172.21.15.27:0/3962044897 conn(0x56298a9e0460 msgr2=0x56298ad95d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.654+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 shutdown_connections 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.654+0000 7f844bc7d740 1 -- 172.21.15.27:0/3962044897 wait complete. 2024-09-21T07:20:52.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.654+0000 7f844bc7d740 20 client.27763 trim_cache size 0 max 16384 2024-09-21T07:20:52.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.672+0000 7f844bc7d740 1 Processor -- start 2024-09-21T07:20:52.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.672+0000 7f844bc7d740 1 -- start start 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.672+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad96620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.672+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9cb390 0x56298ad942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9ec070 0x56298ad90000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f844bc7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b5a00 con 0x56298a9ec070 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f844bc7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b32d0 con 0x56298a9cb390 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f844bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b1140 con 0x56298a9c31b0 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9ec070 0x56298ad90000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9ec070 0x56298ad90000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44232/0 (socket says 172.21.15.27:44232) 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 -- 172.21.15.27:0/1989538274 learned_addr learned my addr 172.21.15.27:0/1989538274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843de00640 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad96620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f8435400640 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9cb390 0x56298ad942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 -- 172.21.15.27:0/1989538274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 msgr2=0x56298ad96620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad96620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 -- 172.21.15.27:0/1989538274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9cb390 msgr2=0x56298ad942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9cb390 0x56298ad942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843e800640 1 -- 172.21.15.27:0/1989538274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56298ad9f7e0 con 0x56298a9ec070 2024-09-21T07:20:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.673+0000 7f843de00640 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad96620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.674+0000 7f843e800640 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9ec070 0x56298ad90000 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8438011370 tx=0x7f8438019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2377f1b5a15c66a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.674+0000 7f842ea00640 1 -- 172.21.15.27:0/1989538274 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380113a0 con 0x56298a9ec070 2024-09-21T07:20:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.674+0000 7f842ea00640 1 -- 172.21.15.27:0/1989538274 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843800fda0 con 0x56298a9ec070 2024-09-21T07:20:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.674+0000 7f842ea00640 1 -- 172.21.15.27:0/1989538274 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843801ecb0 con 0x56298a9ec070 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 -- 172.21.15.27:0/1989538274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9ec070 msgr2=0x56298ad90000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9ec070 0x56298ad90000 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8438011370 tx=0x7f8438019000 comp rx=0 tx=0).stop 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 -- 172.21.15.27:0/1989538274 shutdown_connections 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9ec070 0x56298ad90000 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9cb390 0x56298ad942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 --2- 172.21.15.27:0/1989538274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9c31b0 0x56298ad96620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 -- 172.21.15.27:0/1989538274 >> 172.21.15.27:0/1989538274 conn(0x56298a9e0460 msgr2=0x56298ad93b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 -- 172.21.15.27:0/1989538274 shutdown_connections 2024-09-21T07:20:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.675+0000 7f844bc7d740 1 -- 172.21.15.27:0/1989538274 wait complete. 2024-09-21T07:20:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.676+0000 7f844bc7d740 1 -- start start 2024-09-21T07:20:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.676+0000 7f844bc7d740 1 Processor -- start 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298adb3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298adb10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f844bc7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adb1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f844bc7d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b1790 con 0x56298a9c31b0 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f844bc7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b2c00 con 0x56298a9ec070 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f844bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b1e60 con 0x56298a9cb390 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f844bc7d740 10 client.0 _ll_register_callbacks cb 0x7f8430009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298adb3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298adb3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44242/0 (socket says 172.21.15.27:44242) 2024-09-21T07:20:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843de00640 1 -- 172.21.15.27:0/2075856023 learned_addr learned my addr 172.21.15.27:0/2075856023 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f8435400640 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298adb10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843e800640 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adb1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843de00640 1 -- 172.21.15.27:0/2075856023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 msgr2=0x56298adb10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843de00640 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298adb10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843de00640 1 -- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 msgr2=0x56298adb1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.677+0000 7f843de00640 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adb1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f843de00640 1 -- 172.21.15.27:0/2075856023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56298adb1d90 con 0x56298a9c31b0 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f843de00640 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298adb3c10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8430003f30 tx=0x7f8430004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b371e507275d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f8423e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f8423e00640 1 -- 172.21.15.27:0/2075856023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843005c070 con 0x56298a9c31b0 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f8423e00640 1 -- 172.21.15.27:0/2075856023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f8430031b50 con 0x56298a9c31b0 2024-09-21T07:20:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843800fda0 con 0x56298a9c31b0 2024-09-21T07:20:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f8423e00640 1 -- 172.21.15.27:0/2075856023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8430035040 con 0x56298a9c31b0 2024-09-21T07:20:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f844bc7d740 10 client.17778 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.678+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56298ae1aac0 con 0x56298a9c31b0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f8423e00640 1 -- 172.21.15.27:0/2075856023 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843800fda0 con 0x56298a9c31b0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f8423e00640 1 -- 172.21.15.27:0/2075856023 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8430004cd0 con 0x56298a9c31b0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f844bc7d740 10 client.17778 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f844bc7d740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f844bc7d740 20 client.17778 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a97fda0 0x56298ae1d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f844bc7d740 4 client.17778 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.679+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56298ad07970 con 0x56298a97fda0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.680+0000 7f8435400640 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a97fda0 0x56298ae1d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.680+0000 7f8435400640 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a97fda0 0x56298ae1d030 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f4c24d682de1380 server_cookie=e658e53412ff9b72 in_seq=0 out_seq=0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.680+0000 7f8423e00640 10 client.17778 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.680+0000 7f8423e00640 1 -- 172.21.15.27:0/2075856023 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x56298ad07970 con 0x56298a97fda0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.680+0000 7f8423e00640 10 client.17778 handle_command_reply: tid=0 2024-09-21T07:20:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.680+0000 7f844bc7d740 1 client.17778 shutdown 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 msgr2=0x56298adb3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298adb3c10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8430003f30 tx=0x7f8430004be0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 shutdown_connections 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56298a97fda0 0x56298ae1d030 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56298a9cb390 0x56298adb10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f8423e00640 0 client.17778 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298a9ec070 0x56298adb1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 --2- 172.21.15.27:0/2075856023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56298a9c31b0 0x56298adb3c10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 >> 172.21.15.27:0/2075856023 conn(0x56298a9e0460 msgr2=0x56298ad94550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 shutdown_connections 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 1 -- 172.21.15.27:0/2075856023 wait complete. 2024-09-21T07:20:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.681+0000 7f844bc7d740 20 client.17778 trim_cache size 0 max 16384 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "4426db32-e542-4900-a1fd-f4a7dd06dc29", 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:52.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:20:52.689 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.690 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84240b6700 msgr2=0x7f84240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84240b6700 0x7f84240b8c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8438009fd0 tx=0x7f8438009500 comp rx=0 tx=0).stop 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f844010b6c0 msgr2=0x7f844019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f844010b6c0 0x7f844019cc70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f844006caa0 tx=0x7f842800ee50 comp rx=0 tx=0).stop 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 shutdown_connections 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f844010c010 0x7f84401a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f84240b6700 0x7f84240b8c10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f844010b6c0 0x7f844019cc70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 --2- 172.21.15.27:0/2039404544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8440105d80 0x7f844019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 >> 172.21.15.27:0/2039404544 conn(0x7f8440101810 msgr2=0x7f844010ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 shutdown_connections 2024-09-21T07:20:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:52.682+0000 7f8445000640 1 -- 172.21.15.27:0/2039404544 wait complete. 2024-09-21T07:20:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:52 smithi119 ceph-mon[34519]: pgmap v617: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:20:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:52 smithi027 ceph-mon[30779]: pgmap v617: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:20:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:52 smithi153 ceph-mon[34747]: pgmap v617: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:20:53.570 INFO:tasks.cephfs.filesystem:scrub status for tag:4426db32-e542-4900-a1fd-f4a7dd06dc29 - {'path': '/', 'tag': '4426db32-e542-4900-a1fd-f4a7dd06dc29', 'options': 'recursive,force'} 2024-09-21T07:20:55.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:54 smithi119 ceph-mon[34519]: pgmap v618: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-21T07:20:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:54 smithi027 ceph-mon[30779]: pgmap v618: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-21T07:20:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:54 smithi153 ceph-mon[34747]: pgmap v618: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.709+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1374500737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410bf50 msgr2=0x7f0b84112780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.709+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1374500737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410bf50 0x7f0b84112780 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ca0 tx=0x7f0b7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.709+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1374500737 shutdown_connections 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.709+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1374500737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b8410b600 0x7f0b8410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.710+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1374500737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410bf50 0x7f0b84112780 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.710+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1374500737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b84105cd0 0x7f0b841060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.710+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1374500737 >> 172.21.15.27:0/1374500737 conn(0x7f0b841016f0 msgr2=0x7f0b84103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.710+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1374500737 shutdown_connections 2024-09-21T07:20:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.710+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1374500737 wait complete. 2024-09-21T07:20:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 Processor -- start 2024-09-21T07:20:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 -- start start 2024-09-21T07:20:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b84105cd0 0x7f0b8419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410b600 0x7f0b8419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b8410bf50 0x7f0b841a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84115c10 con 0x7f0b8410b600 2024-09-21T07:20:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84115a90 con 0x7f0b84105cd0 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b8bc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84115d90 con 0x7f0b8410bf50 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.711+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410b600 0x7f0b8419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410b600 0x7f0b8419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44258/0 (socket says 172.21.15.27:44258) 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 -- 172.21.15.27:0/1513039021 learned_addr learned my addr 172.21.15.27:0/1513039021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b89200640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b84105cd0 0x7f0b8419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b89c00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b8410bf50 0x7f0b841a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 -- 172.21.15.27:0/1513039021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b8410bf50 msgr2=0x7f0b841a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b8410bf50 0x7f0b841a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 -- 172.21.15.27:0/1513039021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b84105cd0 msgr2=0x7f0b8419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b84105cd0 0x7f0b8419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 -- 172.21.15.27:0/1513039021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b841a2500 con 0x7f0b8410b600 2024-09-21T07:20:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.712+0000 7f0b7be00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410b600 0x7f0b8419cb70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00d950 tx=0x7f0b6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.713+0000 7f0b82000640 1 -- 172.21.15.27:0/1513039021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c017070 con 0x7f0b8410b600 2024-09-21T07:20:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.713+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b841a27f0 con 0x7f0b8410b600 2024-09-21T07:20:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.713+0000 7f0b82000640 1 -- 172.21.15.27:0/1513039021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b6c009c30 con 0x7f0b8410b600 2024-09-21T07:20:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.713+0000 7f0b82000640 1 -- 172.21.15.27:0/1513039021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0047a0 con 0x7f0b8410b600 2024-09-21T07:20:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.713+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b841a8f80 con 0x7f0b8410b600 2024-09-21T07:20:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 Processor -- start 2024-09-21T07:20:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 -- start start 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e940378c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940379170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e9403796b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e9402ee6b0 con 0x55e93ffab2f0 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e9402ddae0 con 0x55e93ffd41b0 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9402e18f0 con 0x55e93ffb34d0 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.716+0000 7f0b82000640 1 -- 172.21.15.27:0/1513039021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0b6c004020 con 0x7f0b8410b600 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940379170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940379170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44260/0 (socket says 172.21.15.27:44260) 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 -- 172.21.15.27:0/2819667698 learned_addr learned my addr 172.21.15.27:0/2819667698 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89c00640 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e940378c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b7be00640 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e9403796b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 -- 172.21.15.27:0/2819667698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 msgr2=0x55e9403796b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e9403796b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 -- 172.21.15.27:0/2819667698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 msgr2=0x55e940378c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e940378c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 -- 172.21.15.27:0/2819667698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9402aa780 con 0x55e93ffab2f0 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89c00640 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e940378c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.717+0000 7f0b89200640 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940379170 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400bd50 tx=0x7f0b740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f431e5481edbc4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.718+0000 7f0b7a000640 1 -- 172.21.15.27:0/2819667698 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74005220 con 0x55e93ffab2f0 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.718+0000 7f0b7a000640 1 -- 172.21.15.27:0/2819667698 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7400be40 con 0x55e93ffab2f0 2024-09-21T07:20:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.718+0000 7f0b7a000640 1 -- 172.21.15.27:0/2819667698 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74011710 con 0x55e93ffab2f0 2024-09-21T07:20:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.718+0000 7f0b82000640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b680b6680 0x7f0b680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.718+0000 7f0b82000640 1 -- 172.21.15.27:0/1513039021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f0b6c0e7630 con 0x7f0b8410b600 2024-09-21T07:20:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b89c00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b680b6680 0x7f0b680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2819667698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 msgr2=0x55e940379170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940379170 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400bd50 tx=0x7f0b740107a0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2819667698 shutdown_connections 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e9403796b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940379170 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2819667698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e940378c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2819667698 >> 172.21.15.27:0/2819667698 conn(0x55e93ffc8300 msgr2=0x55e940376b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2819667698 shutdown_connections 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2819667698 wait complete. 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.719+0000 7f0b89c00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b680b6680 0x7f0b680b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c02f500 tx=0x7f0b7c009820 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.720+0000 7f0b92cbb740 1 -- start start 2024-09-21T07:20:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.720+0000 7f0b92cbb740 1 Processor -- start 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.720+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e940384220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.720+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e940384760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e940384ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e940215ab0 con 0x55e93ffd41b0 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e940284570 con 0x55e93ffb34d0 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9403873b0 con 0x55e93ffab2f0 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b92cbb740 10 client.0 _ll_register_callbacks cb 0x55e940293100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e940384ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e940384ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44264/0 (socket says 172.21.15.27:44264) 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 -- 172.21.15.27:0/2498386090 learned_addr learned my addr 172.21.15.27:0/2498386090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89200640 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e940384220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b7be00640 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e940384760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 -- 172.21.15.27:0/2498386090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 msgr2=0x55e940384220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e940384220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 -- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 msgr2=0x55e940384760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e940384760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.721+0000 7f0b89c00640 1 -- 172.21.15.27:0/2498386090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94003f790 con 0x55e93ffd41b0 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b89c00640 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e940384ca0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c035000 tx=0x7f0b7c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c871b1d69ee18999 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b72a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b72a00640 1 -- 172.21.15.27:0/2498386090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c003d00 con 0x55e93ffd41b0 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b72a00640 1 -- 172.21.15.27:0/2498386090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f0b7c004040 con 0x55e93ffd41b0 2024-09-21T07:20:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e9402f0600 con 0x55e93ffd41b0 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b72a00640 1 -- 172.21.15.27:0/2498386090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c004240 con 0x55e93ffd41b0 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b92cbb740 10 client.17790 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9400b8620 con 0x55e93ffd41b0 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.722+0000 7f0b72a00640 1 -- 172.21.15.27:0/2498386090 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e9402f0600 con 0x55e93ffd41b0 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b72a00640 1 -- 172.21.15.27:0/2498386090 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0b7c051070 con 0x55e93ffd41b0 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b92cbb740 10 client.17790 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b92cbb740 10 client.17790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b92cbb740 20 client.17790 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff92d80 0x55e9403963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b92cbb740 4 client.17790 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9402ec0b0 con 0x55e93ff92d80 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.723+0000 7f0b89200640 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff92d80 0x55e9403963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.724+0000 7f0b89200640 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff92d80 0x55e9403963e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ac6de76a17f393 server_cookie=5584a5cd19313293 in_seq=0 out_seq=0 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.724+0000 7f0b72a00640 10 client.17790 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b72a00640 1 -- 172.21.15.27:0/2498386090 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9402ec0b0 con 0x55e93ff92d80 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b72a00640 10 client.17790 handle_command_reply: tid=0 2024-09-21T07:20:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 client.17790 shutdown 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 msgr2=0x55e940384ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e940384ca0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c035000 tx=0x7f0b7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 shutdown_connections 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff92d80 0x55e9403963e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e940384ca0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e940384760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b72a00640 0 client.17790 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/2498386090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e940384220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 >> 172.21.15.27:0/2498386090 conn(0x55e93ffc8300 msgr2=0x55e940379c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 shutdown_connections 2024-09-21T07:20:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 1 -- 172.21.15.27:0/2498386090 wait complete. 2024-09-21T07:20:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.725+0000 7f0b92cbb740 20 client.17790 trim_cache size 0 max 16384 2024-09-21T07:20:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 Processor -- start 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 -- start start 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940375aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e940385e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e9403813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841b5920 con 0x55e93ffab2f0 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841b31f0 con 0x55e93ffd41b0 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841b1060 con 0x55e93ffb34d0 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b89200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940375aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e940385e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b89200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940375aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44266/0 (socket says 172.21.15.27:44266) 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e940385e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51658/0 (socket says 172.21.15.27:51658) 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b89200640 1 -- 172.21.15.27:0/456988877 learned_addr learned my addr 172.21.15.27:0/456988877 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.741+0000 7f0b89c00640 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e9403813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b7be00640 1 -- 172.21.15.27:0/456988877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 msgr2=0x55e9403813f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b7be00640 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e9403813f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b7be00640 1 -- 172.21.15.27:0/456988877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 msgr2=0x55e940375aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b7be00640 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940375aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b7be00640 1 -- 172.21.15.27:0/456988877 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94037b160 con 0x55e93ffb34d0 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b7be00640 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e940385e50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0e4970 tx=0x7f0b6c005190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b99004a6560e50e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b79600640 1 -- 172.21.15.27:0/456988877 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0ac9c0 con 0x55e93ffb34d0 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b79600640 1 -- 172.21.15.27:0/456988877 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b6c0acb80 con 0x55e93ffb34d0 2024-09-21T07:20:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.742+0000 7f0b79600640 1 -- 172.21.15.27:0/456988877 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0ac7e0 con 0x55e93ffb34d0 2024-09-21T07:20:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 -- 172.21.15.27:0/456988877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 msgr2=0x55e940385e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e940385e50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0e4970 tx=0x7f0b6c005190 comp rx=0 tx=0).stop 2024-09-21T07:20:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 -- 172.21.15.27:0/456988877 shutdown_connections 2024-09-21T07:20:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffb34d0 0x55e940385e50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffd41b0 0x55e9403813f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/456988877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffab2f0 0x55e940375aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 -- 172.21.15.27:0/456988877 >> 172.21.15.27:0/456988877 conn(0x55e93ffc8300 msgr2=0x55e940385740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 -- 172.21.15.27:0/456988877 shutdown_connections 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.743+0000 7f0b92cbb740 1 -- 172.21.15.27:0/456988877 wait complete. 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 -- start start 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 Processor -- start 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e94038a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e94038a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e9403f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841b16b0 con 0x55e93ffd41b0 2024-09-21T07:20:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841b2b20 con 0x55e93ffb34d0 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841b1d80 con 0x55e93ffab2f0 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.744+0000 7f0b92cbb740 10 client.0 _ll_register_callbacks cb 0x7f0b7c0099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b89c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e9403f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e94038a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e94038a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50344/0 (socket says 172.21.15.27:50344) 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 -- 172.21.15.27:0/1862093378 learned_addr learned my addr 172.21.15.27:0/1862093378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b89c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e9403f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44270/0 (socket says 172.21.15.27:44270) 2024-09-21T07:20:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b89200640 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e94038a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 -- 172.21.15.27:0/1862093378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 msgr2=0x55e94038a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e94038a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 -- 172.21.15.27:0/1862093378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 msgr2=0x55e9403f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e9403f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 -- 172.21.15.27:0/1862093378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9403f8440 con 0x55e93ffb34d0 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b7be00640 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e94038a6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0a9640 tx=0x7f0b6c0a9510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f341b680f51b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b72a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:20:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.745+0000 7f0b72a00640 1 -- 172.21.15.27:0/1862093378 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c098070 con 0x55e93ffb34d0 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b72a00640 1 -- 172.21.15.27:0/1862093378 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f0b6c0e71c0 con 0x55e93ffb34d0 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c0acb80 con 0x55e93ffb34d0 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b72a00640 1 -- 172.21.15.27:0/1862093378 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c0ac9c0 con 0x55e93ffb34d0 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 10 client.27824 fetch_fsmap learned FSMap version 27 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e940400990 con 0x55e93ffb34d0 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b72a00640 1 -- 172.21.15.27:0/1862093378 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b6c0acb80 con 0x55e93ffb34d0 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b72a00640 1 -- 172.21.15.27:0/1862093378 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0b6c0abd40 con 0x55e93ffb34d0 2024-09-21T07:20:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 10 client.27824 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 10 client.27824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 20 client.27824 populate_metadata read hostname 'smithi027' 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff681c0 0x55e940402ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 4 client.27824 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.746+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e940386b90 con 0x55e93ff681c0 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.747+0000 7f0b89200640 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff681c0 0x55e940402ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.747+0000 7f0b89200640 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff681c0 0x55e940402ff0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1203dd75312da0 server_cookie=846ed3ee7e7ea1d4 in_seq=0 out_seq=0 2024-09-21T07:20:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.747+0000 7f0b72a00640 10 client.27824 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b72a00640 1 -- 172.21.15.27:0/1862093378 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e940386b90 con 0x55e93ff681c0 2024-09-21T07:20:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b72a00640 10 client.27824 handle_command_reply: tid=0 2024-09-21T07:20:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 client.27824 shutdown 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 msgr2=0x55e94038a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e94038a6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0a9640 tx=0x7f0b6c0a9510 comp rx=0 tx=0).stop 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 shutdown_connections 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e93ff681c0 0x55e940402ff0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e93ffd41b0 0x55e9403f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e93ffb34d0 0x55e94038a6f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 --2- 172.21.15.27:0/1862093378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e93ffab2f0 0x55e94038a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b72a00640 0 client.27824 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 >> 172.21.15.27:0/1862093378 conn(0x55e93ffc8300 msgr2=0x55e940375870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 shutdown_connections 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.758+0000 7f0b92cbb740 1 -- 172.21.15.27:0/1862093378 wait complete. 2024-09-21T07:20:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b92cbb740 20 client.27824 trim_cache size 0 max 16384 2024-09-21T07:20:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:20:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:20:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:20:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:20:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b680b6680 msgr2=0x7f0b680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b680b6680 0x7f0b680b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c02f500 tx=0x7f0b7c009820 comp rx=0 tx=0).stop 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410b600 msgr2=0x7f0b8419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410b600 0x7f0b8419cb70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00d950 tx=0x7f0b6c00de10 comp rx=0 tx=0).stop 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 shutdown_connections 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b8410bf50 0x7f0b841a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b680b6680 0x7f0b680b8b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b8410b600 0x7f0b8419cb70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 --2- 172.21.15.27:0/1513039021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b84105cd0 0x7f0b8419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 >> 172.21.15.27:0/1513039021 conn(0x7f0b841016f0 msgr2=0x7f0b84103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 shutdown_connections 2024-09-21T07:20:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:20:55.759+0000 7f0b8bc00640 1 -- 172.21.15.27:0/1513039021 wait complete. 2024-09-21T07:20:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:56 smithi119 ceph-mon[34519]: pgmap v619: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-21T07:20:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:56 smithi027 ceph-mon[30779]: pgmap v619: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-21T07:20:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:56 smithi153 ceph-mon[34747]: pgmap v619: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-09-21T07:20:58.635 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:20:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:20:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:20:58 smithi027 ceph-mon[30779]: pgmap v620: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-21T07:20:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:20:58 smithi153 ceph-mon[34747]: pgmap v620: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-21T07:20:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:20:58 smithi119 ceph-mon[34519]: pgmap v620: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-21T07:20:59.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 -- 172.21.15.27:0/4160718139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 msgr2=0x7fd84c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:59.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 --2- 172.21.15.27:0/4160718139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 0x7fd84c112840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd844009ca0 tx=0x7fd84402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 -- 172.21.15.27:0/4160718139 shutdown_connections 2024-09-21T07:20:59.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 --2- 172.21.15.27:0/4160718139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 0x7fd84c112840 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 --2- 172.21.15.27:0/4160718139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c10b6c0 0x7fd84c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 --2- 172.21.15.27:0/4160718139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd84c105d80 0x7fd84c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 -- 172.21.15.27:0/4160718139 >> 172.21.15.27:0/4160718139 conn(0x7fd84c1017d0 msgr2=0x7fd84c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:59.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 -- 172.21.15.27:0/4160718139 shutdown_connections 2024-09-21T07:20:59.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.640+0000 7fd851000640 1 -- 172.21.15.27:0/4160718139 wait complete. 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.641+0000 7fd851000640 1 Processor -- start 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.641+0000 7fd851000640 1 -- start start 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd851000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c105d80 0x7fd84c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd851000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd84c10b6c0 0x7fd84c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd851000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 0x7fd84c1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd851000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c115cd0 con 0x7fd84c10c010 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd851000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c115b50 con 0x7fd84c105d80 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd851000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c115e50 con 0x7fd84c10b6c0 2024-09-21T07:20:59.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd849e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c105d80 0x7fd84c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:59.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd849e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c105d80 0x7fd84c19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46288/0 (socket says 172.21.15.27:46288) 2024-09-21T07:20:59.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd849e00640 1 -- 172.21.15.27:0/1487297330 learned_addr learned my addr 172.21.15.27:0/1487297330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:20:59.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd84a800640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 0x7fd84c1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:59.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.642+0000 7fd843e00640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd84c10b6c0 0x7fd84c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:59.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd849e00640 1 -- 172.21.15.27:0/1487297330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd84c10b6c0 msgr2=0x7fd84c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd849e00640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd84c10b6c0 0x7fd84c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd849e00640 1 -- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 msgr2=0x7fd84c1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd849e00640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 0x7fd84c1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd849e00640 1 -- 172.21.15.27:0/1487297330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c1a25f0 con 0x7fd84c105d80 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd84a800640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 0x7fd84c1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd849e00640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c105d80 0x7fd84c19c680 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e9f0 tx=0x7fd83c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd842a00640 1 -- 172.21.15.27:0/1487297330 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00cd80 con 0x7fd84c105d80 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd842a00640 1 -- 172.21.15.27:0/1487297330 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c0054d0 con 0x7fd84c105d80 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd842a00640 1 -- 172.21.15.27:0/1487297330 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c010690 con 0x7fd84c105d80 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c1a28e0 con 0x7fd84c105d80 2024-09-21T07:20:59.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.643+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c1a9040 con 0x7fd84c105d80 2024-09-21T07:20:59.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.644+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84c106190 con 0x7fd84c105d80 2024-09-21T07:20:59.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.647+0000 7fd842a00640 1 -- 172.21.15.27:0/1487297330 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd83c004e30 con 0x7fd84c105d80 2024-09-21T07:20:59.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.649+0000 7fd842a00640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8300b6680 0x7fd8300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:20:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.649+0000 7fd842a00640 1 -- 172.21.15.27:0/1487297330 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7fd83c0e8190 con 0x7fd84c105d80 2024-09-21T07:20:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.649+0000 7fd843e00640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8300b6680 0x7fd8300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:20:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.650+0000 7fd843e00640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8300b6680 0x7fd8300b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd84c10b180 tx=0x7fd834005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:20:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.650+0000 7fd842a00640 1 -- 172.21.15.27:0/1487297330 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd83c056710 con 0x7fd84c105d80 2024-09-21T07:20:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.892+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd84c19d7b0 con 0x7fd84c105d80 2024-09-21T07:20:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.893+0000 7fd842a00640 1 -- 172.21.15.27:0/1487297330 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fd83c0ae2d0 con 0x7fd84c105d80 2024-09-21T07:20:59.895 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:20:59.895 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:20:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:20:59.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8300b6680 msgr2=0x7fd8300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:59.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8300b6680 0x7fd8300b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd84c10b180 tx=0x7fd834005f90 comp rx=0 tx=0).stop 2024-09-21T07:20:59.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c105d80 msgr2=0x7fd84c19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:20:59.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c105d80 0x7fd84c19c680 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e9f0 tx=0x7fd83c00edf0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 shutdown_connections 2024-09-21T07:20:59.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd84c10b6c0 0x7fd84c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8300b6680 0x7fd8300b8b90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd84c10c010 0x7fd84c1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 --2- 172.21.15.27:0/1487297330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd84c105d80 0x7fd84c19c680 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:20:59.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 >> 172.21.15.27:0/1487297330 conn(0x7fd84c1017d0 msgr2=0x7fd84c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:20:59.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 shutdown_connections 2024-09-21T07:20:59.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:20:59.896+0000 7fd851000640 1 -- 172.21.15.27:0/1487297330 wait complete. 2024-09-21T07:21:00.760 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id b9c6a190 --release --await' 2024-09-21T07:21:00.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id b9c6a190 --release --await 2024-09-21T07:21:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:00 smithi027 ceph-mon[30779]: pgmap v621: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-09-21T07:21:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:00 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1487297330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:01.147 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:00 smithi153 ceph-mon[34747]: pgmap v621: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-09-21T07:21:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:00 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1487297330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:00 smithi119 ceph-mon[34519]: pgmap v621: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-09-21T07:21:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:00 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1487297330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 -- 172.21.15.27:0/1619222562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da80757b0 msgr2=0x7f7da8075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 --2- 172.21.15.27:0/1619222562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da80757b0 0x7f7da8075bc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009900 tx=0x7f7d9802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 -- 172.21.15.27:0/1619222562 shutdown_connections 2024-09-21T07:21:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 --2- 172.21.15.27:0/1619222562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da8076100 0x7f7da810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 --2- 172.21.15.27:0/1619222562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da810ff10 0x7f7da8112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 --2- 172.21.15.27:0/1619222562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da80757b0 0x7f7da8075bc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 -- 172.21.15.27:0/1619222562 >> 172.21.15.27:0/1619222562 conn(0x7f7da80ff440 msgr2=0x7f7da8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 -- 172.21.15.27:0/1619222562 shutdown_connections 2024-09-21T07:21:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.794+0000 7f7daca00640 1 -- 172.21.15.27:0/1619222562 wait complete. 2024-09-21T07:21:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.796+0000 7f7daca00640 1 Processor -- start 2024-09-21T07:21:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.796+0000 7f7daca00640 1 -- start start 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7daca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da80757b0 0x7f7da819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7daca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da8076100 0x7f7da819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7daca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da810ff10 0x7f7da81a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7daca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8115b10 con 0x7f7da8076100 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7daca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8115990 con 0x7f7da810ff10 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7daca00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8115c90 con 0x7f7da80757b0 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7d9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da8076100 0x7f7da819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7d9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da8076100 0x7f7da819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44874/0 (socket says 172.21.15.27:44874) 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7d9fe00640 1 -- 172.21.15.27:0/2478768481 learned_addr learned my addr 172.21.15.27:0/2478768481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:01.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7da7e00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da810ff10 0x7f7da81a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7d9fe00640 1 -- 172.21.15.27:0/2478768481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da80757b0 msgr2=0x7f7da819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da80757b0 0x7f7da819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7d9fe00640 1 -- 172.21.15.27:0/2478768481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da810ff10 msgr2=0x7f7da81a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.797+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da810ff10 0x7f7da81a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.798+0000 7f7d9fe00640 1 -- 172.21.15.27:0/2478768481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da81a24d0 con 0x7f7da8076100 2024-09-21T07:21:01.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.798+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da8076100 0x7f7da819cb00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000e9c0 tx=0x7f7d9000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:01.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.798+0000 7f7da4c00640 1 -- 172.21.15.27:0/2478768481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000ccc0 con 0x7f7da8076100 2024-09-21T07:21:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.798+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da81a27c0 con 0x7f7da8076100 2024-09-21T07:21:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.798+0000 7f7da4c00640 1 -- 172.21.15.27:0/2478768481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d900035f0 con 0x7f7da8076100 2024-09-21T07:21:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.798+0000 7f7da4c00640 1 -- 172.21.15.27:0/2478768481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000ce60 con 0x7f7da8076100 2024-09-21T07:21:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.798+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da81a8f20 con 0x7f7da8076100 2024-09-21T07:21:01.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.801+0000 7f7db4df1740 1 Processor -- start 2024-09-21T07:21:01.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7db4df1740 1 -- start start 2024-09-21T07:21:01.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7da4c00640 1 -- 172.21.15.27:0/2478768481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7d900037a0 con 0x7f7da8076100 2024-09-21T07:21:01.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db6744340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6744880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6744dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7db4df1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557db66bb290 con 0x557db6374660 2024-09-21T07:21:01.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7db4df1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557db66aa6c0 con 0x557db639d520 2024-09-21T07:21:01.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7db4df1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557db66ae4d0 con 0x557db637c840 2024-09-21T07:21:01.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7da7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6744880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.802+0000 7f7da7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db6744340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7d9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6744dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db6744340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46304/0 (socket says 172.21.15.27:46304) 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6744880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44884/0 (socket says 172.21.15.27:44884) 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 -- 172.21.15.27:0/551365237 learned_addr learned my addr 172.21.15.27:0/551365237 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 -- 172.21.15.27:0/551365237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 msgr2=0x557db6744dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 --2- 172.21.15.27:0/551365237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6744dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 -- 172.21.15.27:0/551365237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 msgr2=0x557db6744880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 --2- 172.21.15.27:0/551365237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6744880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 -- 172.21.15.27:0/551365237 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db6637460 con 0x557db639d520 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.803+0000 7f7da7e00640 1 --2- 172.21.15.27:0/551365237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db6744340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7da0011440 tx=0x7f7da0011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d540fadde52b8968 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.804+0000 7f7d97e00640 1 -- 172.21.15.27:0/551365237 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0003c30 con 0x557db639d520 2024-09-21T07:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.804+0000 7f7d97e00640 1 -- 172.21.15.27:0/551365237 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da0003dd0 con 0x557db639d520 2024-09-21T07:21:01.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.804+0000 7f7d97e00640 1 -- 172.21.15.27:0/551365237 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da000fae0 con 0x557db639d520 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.804+0000 7f7da4c00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d8c0b6680 0x7f7d8c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.804+0000 7f7da4c00640 1 -- 172.21.15.27:0/2478768481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f7d900e7fa0 con 0x7f7da8076100 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.804+0000 7f7da7400640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d8c0b6680 0x7f7d8c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 -- 172.21.15.27:0/551365237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 msgr2=0x557db6744340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 --2- 172.21.15.27:0/551365237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db6744340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7da0011440 tx=0x7f7da0011900 comp rx=0 tx=0).stop 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 -- 172.21.15.27:0/551365237 shutdown_connections 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 --2- 172.21.15.27:0/551365237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6744dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 --2- 172.21.15.27:0/551365237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6744880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 --2- 172.21.15.27:0/551365237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db6744340 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 -- 172.21.15.27:0/551365237 >> 172.21.15.27:0/551365237 conn(0x557db6391910 msgr2=0x557db6742230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 -- 172.21.15.27:0/551365237 shutdown_connections 2024-09-21T07:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7db4df1740 1 -- 172.21.15.27:0/551365237 wait complete. 2024-09-21T07:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.805+0000 7f7da7400640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d8c0b6680 0x7f7d8c0b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x557db6742fc0 tx=0x7f7d98031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.806+0000 7f7db4df1740 1 -- start start 2024-09-21T07:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.806+0000 7f7db4df1740 1 Processor -- start 2024-09-21T07:21:01.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db674fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6750430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db675be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7db4df1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557db650bbb0 con 0x557db6374660 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7db4df1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557db66456c0 con 0x557db639d520 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7db4df1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557db6752b20 con 0x557db637c840 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7db4df1740 10 client.0 _ll_register_callbacks cb 0x557db6660490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7d9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6750430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7d9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6750430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49008/0 (socket says 172.21.15.27:49008) 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1986406326 learned_addr learned my addr 172.21.15.27:0/1986406326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7da7400640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db674fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7da7e00640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db675be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 msgr2=0x557db675be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db675be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1986406326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 msgr2=0x557db674fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.807+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db674fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.808+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1986406326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db6408b00 con 0x557db637c840 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.808+0000 7f7da7400640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db674fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.808+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6750430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d900ad6e0 tx=0x7f7d900e5880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc23569f978e0a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.808+0000 7f7d94c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.808+0000 7f7d94c00640 1 -- 172.21.15.27:0/1986406326 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900ad330 con 0x557db637c840 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.808+0000 7f7d94c00640 1 -- 172.21.15.27:0/1986406326 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f7d900eb020 con 0x557db637c840 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.808+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da0003dd0 con 0x557db637c840 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7d94c00640 1 -- 172.21.15.27:0/1986406326 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d900ad150 con 0x557db637c840 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 10 client.27802 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:01.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557db647f960 con 0x557db637c840 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7d94c00640 1 -- 172.21.15.27:0/1986406326 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da0003dd0 con 0x557db637c840 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7d94c00640 1 -- 172.21.15.27:0/1986406326 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7d900ac6b0 con 0x557db637c840 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 10 client.27802 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 10 client.27802 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 20 client.27802 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db635bea0 0x557db6761d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 4 client.27802 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.809+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557db66bd1e0 con 0x557db635bea0 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.810+0000 7f7da7400640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db635bea0 0x557db6761d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.810+0000 7f7da7400640 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db635bea0 0x557db6761d70 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72f68f2166beb3a server_cookie=9e67685bb44dac9f in_seq=0 out_seq=0 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.810+0000 7f7d94c00640 10 client.27802 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.811+0000 7f7d94c00640 1 -- 172.21.15.27:0/1986406326 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557db66bd1e0 con 0x557db635bea0 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.811+0000 7f7d94c00640 10 client.27802 handle_command_reply: tid=0 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.811+0000 7f7db4df1740 1 client.27802 shutdown 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 msgr2=0x557db6750430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6750430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d900ad6e0 tx=0x7f7d900e5880 comp rx=0 tx=0).stop 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 shutdown_connections 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db635bea0 0x557db6761d70 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6750430 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db639d520 0x557db675be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1986406326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db674fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7d94c00640 0 client.27802 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 >> 172.21.15.27:0/1986406326 conn(0x557db6391910 msgr2=0x557db67474d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 shutdown_connections 2024-09-21T07:21:01.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 1 -- 172.21.15.27:0/1986406326 wait complete. 2024-09-21T07:21:01.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.812+0000 7f7db4df1740 20 client.27802 trim_cache size 0 max 16384 2024-09-21T07:21:01.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.830+0000 7f7db4df1740 1 Processor -- start 2024-09-21T07:21:01.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.830+0000 7f7db4df1740 1 -- start start 2024-09-21T07:21:01.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6751920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db637c840 0x557db67674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db639d520 0x557db6767a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7db4df1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81b5990 con 0x557db6374660 2024-09-21T07:21:01.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7db4df1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81b3260 con 0x557db637c840 2024-09-21T07:21:01.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7db4df1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81b10d0 con 0x557db639d520 2024-09-21T07:21:01.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6751920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6751920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44902/0 (socket says 172.21.15.27:44902) 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7400640 1 -- 172.21.15.27:0/931686377 learned_addr learned my addr 172.21.15.27:0/931686377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7e00640 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db639d520 0x557db6767a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db637c840 0x557db67674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7400640 1 -- 172.21.15.27:0/931686377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db639d520 msgr2=0x557db6767a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7400640 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db639d520 0x557db6767a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7400640 1 -- 172.21.15.27:0/931686377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db637c840 msgr2=0x557db67674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.831+0000 7f7da7400640 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db637c840 0x557db67674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.832+0000 7f7da7400640 1 -- 172.21.15.27:0/931686377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db6742750 con 0x557db6374660 2024-09-21T07:21:01.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.832+0000 7f7da7400640 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6751920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009230 tx=0x7f7d9803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3311e074e9077881 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:01.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.832+0000 7f7d97400640 1 -- 172.21.15.27:0/931686377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98057070 con 0x557db6374660 2024-09-21T07:21:01.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.832+0000 7f7d97400640 1 -- 172.21.15.27:0/931686377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d98003bc0 con 0x557db6374660 2024-09-21T07:21:01.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.832+0000 7f7d97400640 1 -- 172.21.15.27:0/931686377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98035040 con 0x557db6374660 2024-09-21T07:21:01.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 -- 172.21.15.27:0/931686377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 msgr2=0x557db6751920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6751920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009230 tx=0x7f7d9803d000 comp rx=0 tx=0).stop 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 -- 172.21.15.27:0/931686377 shutdown_connections 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db639d520 0x557db6767a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db637c840 0x557db67674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 --2- 172.21.15.27:0/931686377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db6374660 0x557db6751920 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 -- 172.21.15.27:0/931686377 >> 172.21.15.27:0/931686377 conn(0x557db6391910 msgr2=0x557db6751210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 -- 172.21.15.27:0/931686377 shutdown_connections 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.833+0000 7f7db4df1740 1 -- 172.21.15.27:0/931686377 wait complete. 2024-09-21T07:21:01.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 -- start start 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 Processor -- start 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db6374660 0x557db6754f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6755460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db639d520 0x557db67559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81b1720 con 0x557db639d520 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81b2b90 con 0x557db6374660 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81b1df0 con 0x557db637c840 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.834+0000 7f7db4df1740 10 client.0 _ll_register_callbacks cb 0x7f7d900eb740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:01.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6755460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7da7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db6374660 0x557db6754f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7da7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db639d520 0x557db67559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6755460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49026/0 (socket says 172.21.15.27:49026) 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1623416765 learned_addr learned my addr 172.21.15.27:0/1623416765 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1623416765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db6374660 msgr2=0x557db6754f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db6374660 0x557db6754f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1623416765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db639d520 msgr2=0x557db67559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db639d520 0x557db67559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 -- 172.21.15.27:0/1623416765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db67cc8a0 con 0x557db637c840 2024-09-21T07:21:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6755460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d900aac50 tx=0x7f7d900e8290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed59ae3f801aa274 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.835+0000 7f7d94c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.836+0000 7f7d94c00640 1 -- 172.21.15.27:0/1623416765 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900ad330 con 0x557db637c840 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.836+0000 7f7d94c00640 1 -- 172.21.15.27:0/1623416765 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f7d900eb2d0 con 0x557db637c840 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.836+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d98003bc0 con 0x557db637c840 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.836+0000 7f7d94c00640 1 -- 172.21.15.27:0/1623416765 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d900ad4f0 con 0x557db637c840 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.836+0000 7f7db4df1740 10 client.27808 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.836+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557db67cdb40 con 0x557db637c840 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.836+0000 7f7d94c00640 1 -- 172.21.15.27:0/1623416765 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d98003bc0 con 0x557db637c840 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7d94c00640 1 -- 172.21.15.27:0/1623416765 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7d900c3b90 con 0x557db637c840 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7db4df1740 10 client.27808 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7db4df1740 10 client.27808 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7db4df1740 20 client.27808 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db6331250 0x557db67d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7db4df1740 4 client.27808 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b9c6a190", "release": true, "await": true}] 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b9c6a190", "release": true, "await": true}) -- 0x557db66b8c90 con 0x557db6331250 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.837+0000 7f7da7400640 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db6331250 0x557db67d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.838+0000 7f7da7400640 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db6331250 0x557db67d0330 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950e7d66005dd5ec server_cookie=a3bcfef58b1c089c in_seq=0 out_seq=0 2024-09-21T07:21:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.838+0000 7f7d94c00640 10 client.27808 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7d94c00640 1 -- 172.21.15.27:0/1623416765 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557db66b8c90 con 0x557db6331250 2024-09-21T07:21:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7d94c00640 10 client.27808 handle_command_reply: tid=0 2024-09-21T07:21:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 client.27808 shutdown 2024-09-21T07:21:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 msgr2=0x557db6755460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6755460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d900aac50 tx=0x7f7d900e8290 comp rx=0 tx=0).stop 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 shutdown_connections 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557db6331250 0x557db67d0330 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557db637c840 0x557db6755460 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557db639d520 0x557db67559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7d94c00640 0 client.27808 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 --2- 172.21.15.27:0/1623416765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557db6374660 0x557db6754f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 >> 172.21.15.27:0/1623416765 conn(0x557db6391910 msgr2=0x557db674e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 shutdown_connections 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 1 -- 172.21.15.27:0/1623416765 wait complete. 2024-09-21T07:21:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.842+0000 7f7db4df1740 20 client.27808 trim_cache size 0 max 16384 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 32, 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "b9c6a190": { 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 32, 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:21:01.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.1, 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:01.847 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d8c0b6680 msgr2=0x7f7d8c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d8c0b6680 0x7f7d8c0b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x557db6742fc0 tx=0x7f7d98031400 comp rx=0 tx=0).stop 2024-09-21T07:21:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da8076100 msgr2=0x7f7da819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da8076100 0x7f7da819cb00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000e9c0 tx=0x7f7d9000edc0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 shutdown_connections 2024-09-21T07:21:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d8c0b6680 0x7f7d8c0b8b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7da810ff10 0x7f7da81a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da8076100 0x7f7da819cb00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 --2- 172.21.15.27:0/2478768481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7da80757b0 0x7f7da819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.843+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 >> 172.21.15.27:0/2478768481 conn(0x7f7da80ff440 msgr2=0x7f7da8109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.844+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 shutdown_connections 2024-09-21T07:21:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:01.844+0000 7f7daca00640 1 -- 172.21.15.27:0/2478768481 wait complete. 2024-09-21T07:21:02.622 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b9c6a190', total seconds elapsed: 25.1 2024-09-21T07:21:02.623 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T07:21:02.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:21:02.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:02 smithi027 ceph-mon[30779]: pgmap v622: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-21T07:21:03.011 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:02 smithi153 ceph-mon[34747]: pgmap v622: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-21T07:21:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:02 smithi119 ceph-mon[34519]: pgmap v622: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-21T07:21:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 -- 172.21.15.27:0/290367890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 msgr2=0x7f3428109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 --2- 172.21.15.27:0/290367890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 0x7f3428109700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3414009900 tx=0x7f341402a410 comp rx=0 tx=0).stop 2024-09-21T07:21:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 -- 172.21.15.27:0/290367890 shutdown_connections 2024-09-21T07:21:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 --2- 172.21.15.27:0/290367890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3428109c40 0x7f3428110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 --2- 172.21.15.27:0/290367890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 0x7f3428109700 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 --2- 172.21.15.27:0/290367890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34281039b0 0x7f3428103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 -- 172.21.15.27:0/290367890 >> 172.21.15.27:0/290367890 conn(0x7f34280ff400 msgr2=0x7f3428101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 -- 172.21.15.27:0/290367890 shutdown_connections 2024-09-21T07:21:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.651+0000 7f3427e00640 1 -- 172.21.15.27:0/290367890 wait complete. 2024-09-21T07:21:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.652+0000 7f3427e00640 1 Processor -- start 2024-09-21T07:21:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.652+0000 7f3427e00640 1 -- start start 2024-09-21T07:21:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3427e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281039b0 0x7f3428075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3427e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 0x7f3428075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3427e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428109c40 0x7f3428076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3427e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3428113950 con 0x7f34281092f0 2024-09-21T07:21:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3427e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f34281137d0 con 0x7f3428109c40 2024-09-21T07:21:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3427e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3428113ad0 con 0x7f34281039b0 2024-09-21T07:21:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3426000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 0x7f3428075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3426a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281039b0 0x7f3428075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3426a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281039b0 0x7f3428075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49046/0 (socket says 172.21.15.27:49046) 2024-09-21T07:21:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3426a00640 1 -- 172.21.15.27:0/1073886686 learned_addr learned my addr 172.21.15.27:0/1073886686 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3427400640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428109c40 0x7f3428076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3426a00640 1 -- 172.21.15.27:0/1073886686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428109c40 msgr2=0x7f3428076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.653+0000 7f3426a00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428109c40 0x7f3428076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f3426a00640 1 -- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 msgr2=0x7f3428075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f3426a00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 0x7f3428075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f3426a00640 1 -- 172.21.15.27:0/1073886686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34281a85a0 con 0x7f34281039b0 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f3426000640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 0x7f3428075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f3426a00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281039b0 0x7f3428075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f341800d950 tx=0x7f341800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f341f400640 1 -- 172.21.15.27:0/1073886686 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418017030 con 0x7f34281039b0 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34281a8890 con 0x7f34281039b0 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281a8df0 con 0x7f34281039b0 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.654+0000 7f341f400640 1 -- 172.21.15.27:0/1073886686 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3418004020 con 0x7f34281039b0 2024-09-21T07:21:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.655+0000 7f341f400640 1 -- 172.21.15.27:0/1073886686 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418005650 con 0x7f34281039b0 2024-09-21T07:21:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.656+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3428103dc0 con 0x7f34281039b0 2024-09-21T07:21:03.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.658+0000 7f341f400640 1 -- 172.21.15.27:0/1073886686 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3418009a30 con 0x7f34281039b0 2024-09-21T07:21:03.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.660+0000 7f341f400640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f340c0b6700 0x7f340c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:03.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.660+0000 7f341f400640 1 -- 172.21.15.27:0/1073886686 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9461+0+0 (secure 0 0 0) 0x7f34180e7740 con 0x7f34281039b0 2024-09-21T07:21:03.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.660+0000 7f3426000640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f340c0b6700 0x7f340c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:03.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.661+0000 7f3426000640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f340c0b6700 0x7f340c0b8c10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f341402aa00 tx=0x7f3414005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:03.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.662+0000 7f341f400640 1 -- 172.21.15.27:0/1073886686 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34180ad900 con 0x7f34281039b0 2024-09-21T07:21:03.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.921+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3428076a10 con 0x7f34281039b0 2024-09-21T07:21:03.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.922+0000 7f341f400640 1 -- 172.21.15.27:0/1073886686 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f3428076a10 con 0x7f34281039b0 2024-09-21T07:21:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:21:03.925 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:21:03.925 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:21:03.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f340c0b6700 msgr2=0x7f340c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:03.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f340c0b6700 0x7f340c0b8c10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f341402aa00 tx=0x7f3414005a20 comp rx=0 tx=0).stop 2024-09-21T07:21:03.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281039b0 msgr2=0x7f3428075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:03.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281039b0 0x7f3428075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f341800d950 tx=0x7f341800de10 comp rx=0 tx=0).stop 2024-09-21T07:21:03.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 shutdown_connections 2024-09-21T07:21:03.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f340c0b6700 0x7f340c0b8c10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428109c40 0x7f3428076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34281092f0 0x7f3428075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 --2- 172.21.15.27:0/1073886686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281039b0 0x7f3428075710 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:03.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 >> 172.21.15.27:0/1073886686 conn(0x7f34280ff400 msgr2=0x7f3428101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:03.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 shutdown_connections 2024-09-21T07:21:03.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:03.925+0000 7f3427e00640 1 -- 172.21.15.27:0/1073886686 wait complete. 2024-09-21T07:21:04.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:03 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:21:04.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:03 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:21:04.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:03 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:21:04.672 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T07:21:04.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T07:21:05.057 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:04 smithi027 ceph-mon[30779]: pgmap v623: 129 pgs: 129 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T07:21:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:04 smithi027 ceph-mon[30779]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T07:21:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:04 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1073886686' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:04 smithi153 ceph-mon[34747]: pgmap v623: 129 pgs: 129 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T07:21:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:04 smithi153 ceph-mon[34747]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T07:21:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:04 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1073886686' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:04 smithi119 ceph-mon[34519]: pgmap v623: 129 pgs: 129 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-21T07:21:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:04 smithi119 ceph-mon[34519]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T07:21:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:04 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1073886686' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 -- 172.21.15.27:0/1998994161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 msgr2=0x7f66b0103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 --2- 172.21.15.27:0/1998994161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b0103f90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009900 tx=0x7f66a002a410 comp rx=0 tx=0).stop 2024-09-21T07:21:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 -- 172.21.15.27:0/1998994161 shutdown_connections 2024-09-21T07:21:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 --2- 172.21.15.27:0/1998994161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b01094c0 0x7f66b01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 --2- 172.21.15.27:0/1998994161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b0109e10 0x7f66b0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 --2- 172.21.15.27:0/1998994161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b0103f90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 -- 172.21.15.27:0/1998994161 >> 172.21.15.27:0/1998994161 conn(0x7f66b00ff5d0 msgr2=0x7f66b0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:05.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 -- 172.21.15.27:0/1998994161 shutdown_connections 2024-09-21T07:21:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.741+0000 7f66b6600640 1 -- 172.21.15.27:0/1998994161 wait complete. 2024-09-21T07:21:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.742+0000 7f66b6600640 1 Processor -- start 2024-09-21T07:21:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.742+0000 7f66b6600640 1 -- start start 2024-09-21T07:21:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66b6600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66b6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b01094c0 0x7f66b019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66b6600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0109e10 0x7f66b019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66b6600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0113ad0 con 0x7f66b0103b80 2024-09-21T07:21:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66b6600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0113950 con 0x7f66b01094c0 2024-09-21T07:21:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66b6600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0113c50 con 0x7f66b0109e10 2024-09-21T07:21:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66af400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66af400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44942/0 (socket says 172.21.15.27:44942) 2024-09-21T07:21:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66af400640 1 -- 172.21.15.27:0/1619879768 learned_addr learned my addr 172.21.15.27:0/1619879768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66afe00640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0109e10 0x7f66b019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.743+0000 7f66a6a00640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b01094c0 0x7f66b019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66af400640 1 -- 172.21.15.27:0/1619879768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0109e10 msgr2=0x7f66b019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66af400640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0109e10 0x7f66b019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66af400640 1 -- 172.21.15.27:0/1619879768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b01094c0 msgr2=0x7f66b019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66af400640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b01094c0 0x7f66b019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66af400640 1 -- 172.21.15.27:0/1619879768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b01a03d0 con 0x7f66b0103b80 2024-09-21T07:21:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66afe00640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0109e10 0x7f66b019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66a6a00640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b01094c0 0x7f66b019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66af400640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b019a460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f66a0005940 tx=0x7f66a00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.744+0000 7f66acc00640 1 -- 172.21.15.27:0/1619879768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a003f030 con 0x7f66b0103b80 2024-09-21T07:21:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.745+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b01a0660 con 0x7f66b0103b80 2024-09-21T07:21:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.745+0000 7f66acc00640 1 -- 172.21.15.27:0/1619879768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00044c0 con 0x7f66b0103b80 2024-09-21T07:21:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.745+0000 7f66acc00640 1 -- 172.21.15.27:0/1619879768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0035070 con 0x7f66b0103b80 2024-09-21T07:21:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.745+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b01a6e20 con 0x7f66b0103b80 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.748+0000 7f66bd2cb740 1 Processor -- start 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.748+0000 7f66bd2cb740 1 -- start start 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.748+0000 7f66acc00640 1 -- 172.21.15.27:0/1619879768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f66a0042030 con 0x7f66b0103b80 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68cfc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b6892c990 0x555b68cfcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68934b70 0x555b68cfd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555b68c72a90 con 0x555b6892c990 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555b68c61ec0 con 0x555b68955850 2024-09-21T07:21:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b68c65cd0 con 0x555b68934b70 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66af400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b6892c990 0x555b68cfcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68cfc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66af400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b6892c990 0x555b68cfcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44958/0 (socket says 172.21.15.27:44958) 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66af400640 1 -- 172.21.15.27:0/694481127 learned_addr learned my addr 172.21.15.27:0/694481127 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.749+0000 7f66a6a00640 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68934b70 0x555b68cfd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66a6a00640 1 -- 172.21.15.27:0/694481127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 msgr2=0x555b68cfc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66a6a00640 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68cfc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66a6a00640 1 -- 172.21.15.27:0/694481127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b6892c990 msgr2=0x555b68cfcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66a6a00640 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b6892c990 0x555b68cfcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66a6a00640 1 -- 172.21.15.27:0/694481127 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b68c079d0 con 0x555b68934b70 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66afe00640 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68cfc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66a6a00640 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68934b70 0x555b68cfd430 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f66b006b810 tx=0x7f66980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc5d595d77decbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.750+0000 7f66a4c00640 1 -- 172.21.15.27:0/694481127 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669800f540 con 0x555b68934b70 2024-09-21T07:21:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.751+0000 7f66a4c00640 1 -- 172.21.15.27:0/694481127 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669800fb90 con 0x555b68934b70 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.751+0000 7f66a4c00640 1 -- 172.21.15.27:0/694481127 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698019cb0 con 0x555b68934b70 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.751+0000 7f66acc00640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66940b6700 0x7f66940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.751+0000 7f66acc00640 1 -- 172.21.15.27:0/1619879768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9549+0+0 (secure 0 0 0) 0x7f66a0107e40 con 0x7f66b0103b80 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.751+0000 7f66afe00640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66940b6700 0x7f66940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.751+0000 7f66bd2cb740 1 -- 172.21.15.27:0/694481127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68934b70 msgr2=0x555b68cfd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68934b70 0x555b68cfd430 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f66b006b810 tx=0x7f66980118d0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 -- 172.21.15.27:0/694481127 shutdown_connections 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68934b70 0x555b68cfd430 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b6892c990 0x555b68cfcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/694481127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68cfc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 -- 172.21.15.27:0/694481127 >> 172.21.15.27:0/694481127 conn(0x555b68949c40 msgr2=0x555b68cfa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66afe00640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66940b6700 0x7f66940b8c10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005fd0 tx=0x7f66a8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 -- 172.21.15.27:0/694481127 shutdown_connections 2024-09-21T07:21:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.752+0000 7f66bd2cb740 1 -- 172.21.15.27:0/694481127 wait complete. 2024-09-21T07:21:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.753+0000 7f66bd2cb740 1 -- start start 2024-09-21T07:21:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.753+0000 7f66bd2cb740 1 Processor -- start 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555b68ac37b0 con 0x555b68934b70 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555b68c06650 con 0x555b6892c990 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b68d0b190 con 0x555b68955850 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66bd2cb740 10 client.0 _ll_register_callbacks cb 0x555b6896d0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66a6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66a6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44972/0 (socket says 172.21.15.27:44972) 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66a6a00640 1 -- 172.21.15.27:0/4181533300 learned_addr learned my addr 172.21.15.27:0/4181533300 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.754+0000 7f66afe00640 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f66a6a00640 1 -- 172.21.15.27:0/4181533300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 msgr2=0x555b68d188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f66a6a00640 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f66a6a00640 1 -- 172.21.15.27:0/4181533300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 msgr2=0x555b68d084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f66a6a00640 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f66a6a00640 1 -- 172.21.15.27:0/4181533300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b689cfba0 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f66a6a00640 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d089f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6698000c00 tx=0x7f669800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289850962d44ba4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f669d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f669d600640 1 -- 172.21.15.27:0/4181533300 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698028020 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f669d600640 1 -- 172.21.15.27:0/4181533300 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9549+0+0 (secure 0 0 0) 0x7f6698016040 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.755+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669800fb90 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.756+0000 7f669d600640 1 -- 172.21.15.27:0/4181533300 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6698033050 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.756+0000 7f66bd2cb740 10 client.17838 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.756+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b68d1b4c0 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.756+0000 7f669d600640 1 -- 172.21.15.27:0/4181533300 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669800fb90 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f669d600640 1 -- 172.21.15.27:0/4181533300 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66980035f0 con 0x555b68934b70 2024-09-21T07:21:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f66bd2cb740 10 client.17838 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f66bd2cb740 10 client.17838 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f66bd2cb740 20 client.17838 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b689141d0 0x555b68d1e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f66bd2cb740 4 client.17838 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b68c749e0 con 0x555b689141d0 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.757+0000 7f66af400640 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b689141d0 0x555b68d1e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.758+0000 7f66af400640 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b689141d0 0x555b68d1e930 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a04edd64b2348e9 server_cookie=1696f808c890a66d in_seq=0 out_seq=0 2024-09-21T07:21:05.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.758+0000 7f669d600640 10 client.17838 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.758+0000 7f669d600640 1 -- 172.21.15.27:0/4181533300 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b68c749e0 con 0x555b689141d0 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.758+0000 7f669d600640 10 client.17838 handle_command_reply: tid=0 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.758+0000 7f66bd2cb740 1 client.17838 shutdown 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 msgr2=0x555b68d089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d089f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6698000c00 tx=0x7f669800ff70 comp rx=0 tx=0).stop 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 shutdown_connections 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b689141d0 0x555b68d1e930 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d089f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/4181533300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f669d600640 0 client.17838 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:05.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 >> 172.21.15.27:0/4181533300 conn(0x555b68949c40 msgr2=0x555b68cffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:05.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 shutdown_connections 2024-09-21T07:21:05.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 1 -- 172.21.15.27:0/4181533300 wait complete. 2024-09-21T07:21:05.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.759+0000 7f66bd2cb740 20 client.17838 trim_cache size 0 max 16384 2024-09-21T07:21:05.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 Processor -- start 2024-09-21T07:21:05.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 -- start start 2024-09-21T07:21:05.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6892c990 0x555b68cf98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d18840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68d18d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01b16c0 con 0x555b68934b70 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01aef90 con 0x555b68955850 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.778+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01ace00 con 0x555b6892c990 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6892c990 0x555b68cf98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66a6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d18840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6892c990 0x555b68cf98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49082/0 (socket says 172.21.15.27:49082) 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68d18d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 -- 172.21.15.27:0/2992173258 learned_addr learned my addr 172.21.15.27:0/2992173258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:05.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 -- 172.21.15.27:0/2992173258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 msgr2=0x555b68d18d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 --2- 172.21.15.27:0/2992173258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68d18d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 -- 172.21.15.27:0/2992173258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 msgr2=0x555b68d18840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 --2- 172.21.15.27:0/2992173258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d18840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 -- 172.21.15.27:0/2992173258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b68cff810 con 0x555b6892c990 2024-09-21T07:21:05.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.779+0000 7f66af400640 1 --2- 172.21.15.27:0/2992173258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6892c990 0x555b68cf98c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006010 tx=0x7f66a010b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0323852a15a47dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:05.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.780+0000 7f669fe00640 1 -- 172.21.15.27:0/2992173258 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0049040 con 0x555b6892c990 2024-09-21T07:21:05.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.780+0000 7f669fe00640 1 -- 172.21.15.27:0/2992173258 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00cd3f0 con 0x555b6892c990 2024-09-21T07:21:05.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.780+0000 7f669fe00640 1 -- 172.21.15.27:0/2992173258 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00d2310 con 0x555b6892c990 2024-09-21T07:21:05.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.780+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2992173258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6892c990 msgr2=0x555b68cf98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.780+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2992173258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6892c990 0x555b68cf98c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006010 tx=0x7f66a010b8f0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2992173258 shutdown_connections 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2992173258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b68955850 0x555b68d18d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2992173258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d18840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2992173258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b6892c990 0x555b68cf98c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2992173258 >> 172.21.15.27:0/2992173258 conn(0x555b68949c40 msgr2=0x555b68cfa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2992173258 shutdown_connections 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2992173258 wait complete. 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.781+0000 7f66bd2cb740 1 -- start start 2024-09-21T07:21:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 1 Processor -- start 2024-09-21T07:21:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d14f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d0f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d0fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01ad450 con 0x555b68934b70 2024-09-21T07:21:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01ae8c0 con 0x555b6892c990 2024-09-21T07:21:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01adb20 con 0x555b68955850 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.782+0000 7f66bd2cb740 10 client.0 _ll_register_callbacks cb 0x7f66a010b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d14f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66a6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d0f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d14f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:46412/0 (socket says 172.21.15.27:46412) 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 -- 172.21.15.27:0/2285971746 learned_addr learned my addr 172.21.15.27:0/2285971746 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66afe00640 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d0fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 -- 172.21.15.27:0/2285971746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 msgr2=0x555b68d0fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d0fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 -- 172.21.15.27:0/2285971746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 msgr2=0x555b68d0f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d0f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 -- 172.21.15.27:0/2285971746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b68d103e0 con 0x555b6892c990 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66a6a00640 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d0f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f66af400640 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d14f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66a00cab50 tx=0x7f66a01080c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28decb44d26b7c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.783+0000 7f669d600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.784+0000 7f669d600640 1 -- 172.21.15.27:0/2285971746 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0049070 con 0x555b6892c990 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.784+0000 7f669d600640 1 -- 172.21.15.27:0/2285971746 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9549+0+0 (secure 0 0 0) 0x7f66a00d6fc0 con 0x555b6892c990 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.784+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a00cd3f0 con 0x555b6892c990 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.784+0000 7f669d600640 1 -- 172.21.15.27:0/2285971746 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66a00cd6b0 con 0x555b6892c990 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.784+0000 7f66bd2cb740 10 client.27872 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.784+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b68d89fc0 con 0x555b6892c990 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.784+0000 7f669d600640 1 -- 172.21.15.27:0/2285971746 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00cd3f0 con 0x555b6892c990 2024-09-21T07:21:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f669d600640 1 -- 172.21.15.27:0/2285971746 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66a00c9230 con 0x555b6892c990 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f66bd2cb740 10 client.27872 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f66bd2cb740 10 client.27872 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f66bd2cb740 20 client.27872 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b688e9580 0x555b68d8c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f66bd2cb740 4 client.27872 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x555b68c70490 con 0x555b688e9580 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.785+0000 7f66a6a00640 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b688e9580 0x555b68d8c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.786+0000 7f66a6a00640 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b688e9580 0x555b68d8c600 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9146cd6f3b567c server_cookie=c47ac6661f06e583 in_seq=0 out_seq=0 2024-09-21T07:21:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.786+0000 7f669d600640 10 client.27872 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:05.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.800+0000 7f669d600640 1 -- 172.21.15.27:0/2285971746 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555b68c70490 con 0x555b688e9580 2024-09-21T07:21:05.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.800+0000 7f669d600640 10 client.27872 handle_command_reply: tid=0 2024-09-21T07:21:05.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.800+0000 7f66bd2cb740 1 client.27872 shutdown 2024-09-21T07:21:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 msgr2=0x555b68d14f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d14f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66a00cab50 tx=0x7f66a01080c0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 shutdown_connections 2024-09-21T07:21:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b688e9580 0x555b68d8c600 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b68955850 0x555b68d0fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b68934b70 0x555b68d0f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 --2- 172.21.15.27:0/2285971746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b6892c990 0x555b68d14f20 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66a00cab50 tx=0x7f66a01080c0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f669d600640 0 client.27872 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 >> 172.21.15.27:0/2285971746 conn(0x555b68949c40 msgr2=0x555b68cfd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 shutdown_connections 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.801+0000 7f66bd2cb740 1 -- 172.21.15.27:0/2285971746 wait complete. 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.802+0000 7f66bd2cb740 20 client.27872 trim_cache size 0 max 16384 2024-09-21T07:21:05.804 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 34, 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "9623a0a1": { 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 34, 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:21:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.1, 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:05.806 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.802+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66940b6700 msgr2=0x7f66940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.802+0000 7f66b6600640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66940b6700 0x7f66940b8c10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005fd0 tx=0x7f66a8009500 comp rx=0 tx=0).stop 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.802+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 msgr2=0x7f66b019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.802+0000 7f66b6600640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b019a460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f66a0005940 tx=0x7f66a00040c0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 shutdown_connections 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0109e10 0x7f66b019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f66940b6700 0x7f66940b8c10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b01094c0 0x7f66b019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 --2- 172.21.15.27:0/1619879768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0103b80 0x7f66b019a460 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 >> 172.21.15.27:0/1619879768 conn(0x7f66b00ff5d0 msgr2=0x7f66b01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 shutdown_connections 2024-09-21T07:21:05.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:05.803+0000 7f66b6600640 1 -- 172.21.15.27:0/1619879768 wait complete. 2024-09-21T07:21:06.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:05 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:21:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:05 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:21:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:05 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:21:06.708 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9623a0a1', quiesce took 4.1 seconds. Will release after: 16.950000000000003 2024-09-21T07:21:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:06 smithi027 ceph-mon[30779]: pgmap v625: 129 pgs: 129 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 51 KiB/s wr, 28 op/s 2024-09-21T07:21:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:21:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:21:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:21:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:06 smithi153 ceph-mon[34747]: pgmap v625: 129 pgs: 129 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 51 KiB/s wr, 28 op/s 2024-09-21T07:21:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:21:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:21:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:21:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:06 smithi119 ceph-mon[34519]: pgmap v625: 129 pgs: 129 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 51 KiB/s wr, 28 op/s 2024-09-21T07:21:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:21:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:21:07.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:21:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 -- 172.21.15.27:0/1938106196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102e80 msgr2=0x7efdd81073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 --2- 172.21.15.27:0/1938106196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102e80 0x7efdd81073a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009900 tx=0x7efdc402a410 comp rx=0 tx=0).stop 2024-09-21T07:21:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 -- 172.21.15.27:0/1938106196 shutdown_connections 2024-09-21T07:21:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 --2- 172.21.15.27:0/1938106196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd81079f0 0x7efdd8107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 --2- 172.21.15.27:0/1938106196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102e80 0x7efdd81073a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 --2- 172.21.15.27:0/1938106196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd8102530 0x7efdd8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 -- 172.21.15.27:0/1938106196 >> 172.21.15.27:0/1938106196 conn(0x7efdd80fe230 msgr2=0x7efdd8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 -- 172.21.15.27:0/1938106196 shutdown_connections 2024-09-21T07:21:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.968+0000 7efddea00640 1 -- 172.21.15.27:0/1938106196 wait complete. 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 Processor -- start 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 -- start start 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102530 0x7efdd81a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd8102e80 0x7efdd81a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd81079f0 0x7efdd81a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8119e30 con 0x7efdd8102530 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8119cb0 con 0x7efdd8102e80 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.969+0000 7efddea00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8119fb0 con 0x7efdd81079f0 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102530 0x7efdd81a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102530 0x7efdd81a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35736/0 (socket says 172.21.15.27:35736) 2024-09-21T07:21:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 -- 172.21.15.27:0/1930037204 learned_addr learned my addr 172.21.15.27:0/1930037204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddcc00640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd8102e80 0x7efdd81a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efdde000640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd81079f0 0x7efdd81a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 -- 172.21.15.27:0/1930037204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd81079f0 msgr2=0x7efdd81a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd81079f0 0x7efdd81a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 -- 172.21.15.27:0/1930037204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd8102e80 msgr2=0x7efdd81a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd8102e80 0x7efdd81a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 -- 172.21.15.27:0/1930037204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd81a67e0 con 0x7efdd8102530 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efdde000640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd81079f0 0x7efdd81a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.970+0000 7efddd600640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102530 0x7efdd81a08b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00eab0 tx=0x7efdcc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.971+0000 7efdd6000640 1 -- 172.21.15.27:0/1930037204 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc00ce40 con 0x7efdd8102530 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.971+0000 7efdd6000640 1 -- 172.21.15.27:0/1930037204 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdcc011920 con 0x7efdd8102530 2024-09-21T07:21:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.971+0000 7efdd6000640 1 -- 172.21.15.27:0/1930037204 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc010a30 con 0x7efdd8102530 2024-09-21T07:21:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.971+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd81a6ad0 con 0x7efdd8102530 2024-09-21T07:21:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.971+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd81ad260 con 0x7efdd8102530 2024-09-21T07:21:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 Processor -- start 2024-09-21T07:21:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 -- start start 2024-09-21T07:21:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31f21b0 0x5571c3596e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3597350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c3597890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5571c350c6b0 con 0x5571c31c92f0 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5571c34fbae0 con 0x5571c31f21b0 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efde708e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5571c34ff8f0 con 0x5571c31d14d0 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.974+0000 7efdd6000640 1 -- 172.21.15.27:0/1930037204 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efdcc010bd0 con 0x7efdd8102530 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efdde000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31f21b0 0x5571c3596e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddcc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c3597890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3597350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efdde000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31f21b0 0x5571c3596e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:34580/0 (socket says 172.21.15.27:34580) 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddcc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c3597890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:37114/0 (socket says 172.21.15.27:37114) 2024-09-21T07:21:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efdde000640 1 -- 172.21.15.27:0/3592453939 learned_addr learned my addr 172.21.15.27:0/3592453939 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddcc00640 1 -- 172.21.15.27:0/3592453939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31f21b0 msgr2=0x5571c3596e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddcc00640 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31f21b0 0x5571c3596e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddcc00640 1 -- 172.21.15.27:0/3592453939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 msgr2=0x5571c3597350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddcc00640 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3597350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efddcc00640 1 -- 172.21.15.27:0/3592453939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c34c8780 con 0x5571c31d14d0 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.975+0000 7efdde000640 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31f21b0 0x5571c3596e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.976+0000 7efddcc00640 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c3597890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efdd806c500 tx=0x7efdc4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d68c211bbaafb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.976+0000 7efdca600640 1 -- 172.21.15.27:0/3592453939 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4035040 con 0x5571c31d14d0 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.976+0000 7efdca600640 1 -- 172.21.15.27:0/3592453939 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc4004030 con 0x5571c31d14d0 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.976+0000 7efdca600640 1 -- 172.21.15.27:0/3592453939 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4005550 con 0x5571c31d14d0 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.976+0000 7efdd6000640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdc00b6680 0x7efdc00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.976+0000 7efdd6000640 1 -- 172.21.15.27:0/1930037204 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9549+0+0 (secure 0 0 0) 0x7efdcc0e8190 con 0x7efdd8102530 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efdde000640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdc00b6680 0x7efdc00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 -- 172.21.15.27:0/3592453939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 msgr2=0x5571c3597890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c3597890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efdd806c500 tx=0x7efdc4000bc0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 -- 172.21.15.27:0/3592453939 shutdown_connections 2024-09-21T07:21:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c3597890 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3597350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 --2- 172.21.15.27:0/3592453939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31f21b0 0x5571c3596e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 -- 172.21.15.27:0/3592453939 >> 172.21.15.27:0/3592453939 conn(0x5571c31e6300 msgr2=0x5571c3594d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 -- 172.21.15.27:0/3592453939 shutdown_connections 2024-09-21T07:21:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efde708e740 1 -- 172.21.15.27:0/3592453939 wait complete. 2024-09-21T07:21:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.977+0000 7efdde000640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdc00b6680 0x7efdc00b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efdd0005fd0 tx=0x7efdd000b5c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 -- start start 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 Processor -- start 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c35a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c35a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5571c3433ab0 con 0x5571c31c92f0 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5571c34a2570 con 0x5571c31d14d0 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5571c35a5530 con 0x5571c31f21b0 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.978+0000 7efde708e740 10 client.0 _ll_register_callbacks cb 0x5571c34b1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c35a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c35a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c35a2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35744/0 (socket says 172.21.15.27:35744) 2024-09-21T07:21:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddd600640 1 -- 172.21.15.27:0/1631387468 learned_addr learned my addr 172.21.15.27:0/1631387468 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efdde000640 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddcc00640 1 -- 172.21.15.27:0/1631387468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 msgr2=0x5571c35a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddcc00640 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddcc00640 1 -- 172.21.15.27:0/1631387468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 msgr2=0x5571c35a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddcc00640 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c35a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddcc00640 1 -- 172.21.15.27:0/1631387468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c325d790 con 0x5571c31d14d0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efddcc00640 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c35a2980 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006010 tx=0x7efdc40058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5667cc49978c08f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.979+0000 7efdbea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efdbea00640 1 -- 172.21.15.27:0/1631387468 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4035070 con 0x5571c31d14d0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc4004030 con 0x5571c31d14d0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efdbea00640 1 -- 172.21.15.27:0/1631387468 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9549+0+0 (secure 0 0 0) 0x7efdc40043f0 con 0x5571c31d14d0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efdbea00640 1 -- 172.21.15.27:0/1631387468 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdc4005290 con 0x5571c31d14d0 2024-09-21T07:21:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efde708e740 10 client.27884 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571c32d6620 con 0x5571c31d14d0 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efdbea00640 1 -- 172.21.15.27:0/1631387468 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc4004030 con 0x5571c31d14d0 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.980+0000 7efdbea00640 1 -- 172.21.15.27:0/1631387468 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efdc4004900 con 0x5571c31d14d0 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efde708e740 10 client.27884 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efde708e740 10 client.27884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efde708e740 20 client.27884 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efde708e740 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31b0d80 0x5571c35b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efde708e740 4 client.27884 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c350e600 con 0x5571c31b0d80 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efddd600640 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31b0d80 0x5571c35b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efddd600640 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31b0d80 0x5571c35b4360 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99eb92c48a3563ba server_cookie=d4551a576d17ce15 in_seq=0 out_seq=0 2024-09-21T07:21:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.981+0000 7efdbea00640 10 client.27884 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.982+0000 7efdbea00640 1 -- 172.21.15.27:0/1631387468 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571c350e600 con 0x5571c31b0d80 2024-09-21T07:21:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.982+0000 7efdbea00640 10 client.27884 handle_command_reply: tid=0 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.982+0000 7efde708e740 1 client.27884 shutdown 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 msgr2=0x5571c35a2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c35a2980 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006010 tx=0x7efdc40058b0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 shutdown_connections 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31b0d80 0x5571c35b4360 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c35a2980 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efdbea00640 0 client.27884 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 --2- 172.21.15.27:0/1631387468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c35a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 >> 172.21.15.27:0/1631387468 conn(0x5571c31e6300 msgr2=0x5571c3597e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 shutdown_connections 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 1 -- 172.21.15.27:0/1631387468 wait complete. 2024-09-21T07:21:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.983+0000 7efde708e740 20 client.27884 trim_cache size 0 max 16384 2024-09-21T07:21:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.998+0000 7efde708e740 1 Processor -- start 2024-09-21T07:21:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efde708e740 1 -- start start 2024-09-21T07:21:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efde708e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3594cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efde708e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c3593360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efde708e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efde708e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b9c80 con 0x5571c31c92f0 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efde708e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b7550 con 0x5571c31d14d0 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efde708e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b53c0 con 0x5571c31f21b0 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efddd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3594cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efddd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3594cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35754/0 (socket says 172.21.15.27:35754) 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efddd600640 1 -- 172.21.15.27:0/1872324032 learned_addr learned my addr 172.21.15.27:0/1872324032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efdde000640 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:08.999+0000 7efddcc00640 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c3593360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efddd600640 1 -- 172.21.15.27:0/1872324032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 msgr2=0x5571c35938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efddd600640 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efddd600640 1 -- 172.21.15.27:0/1872324032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 msgr2=0x5571c3593360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efddd600640 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c3593360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efddd600640 1 -- 172.21.15.27:0/1872324032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c359df50 con 0x5571c31c92f0 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efddd600640 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3594cc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0e5ae0 tx=0x7efdcc0d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6cc27808f5fb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efdc9c00640 1 -- 172.21.15.27:0/1872324032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0ad350 con 0x5571c31c92f0 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efdc9c00640 1 -- 172.21.15.27:0/1872324032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdcc0ad6f0 con 0x5571c31c92f0 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.000+0000 7efdc9c00640 1 -- 172.21.15.27:0/1872324032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0a9350 con 0x5571c31c92f0 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 -- 172.21.15.27:0/1872324032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 msgr2=0x5571c3594cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3594cc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0e5ae0 tx=0x7efdcc0d1040 comp rx=0 tx=0).stop 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 -- 172.21.15.27:0/1872324032 shutdown_connections 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31f21b0 0x5571c35938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31d14d0 0x5571c3593360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 --2- 172.21.15.27:0/1872324032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31c92f0 0x5571c3594cc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 -- 172.21.15.27:0/1872324032 >> 172.21.15.27:0/1872324032 conn(0x5571c31e6300 msgr2=0x5571c35945b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 -- 172.21.15.27:0/1872324032 shutdown_connections 2024-09-21T07:21:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.001+0000 7efde708e740 1 -- 172.21.15.27:0/1872324032 wait complete. 2024-09-21T07:21:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 -- start start 2024-09-21T07:21:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 Processor -- start 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31c92f0 0x5571c35b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c35b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31f21b0 0x5571c361cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b5a10 con 0x5571c31f21b0 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b6e80 con 0x5571c31c92f0 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.002+0000 7efde708e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b60e0 con 0x5571c31d14d0 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efde708e740 10 client.0 _ll_register_callbacks cb 0x7efdcc0ebbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31f21b0 0x5571c361cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31f21b0 0x5571c361cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35758/0 (socket says 172.21.15.27:35758) 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efddcc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c35b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 -- 172.21.15.27:0/2450719431 learned_addr learned my addr 172.21.15.27:0/2450719431 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efddd600640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31c92f0 0x5571c35b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 -- 172.21.15.27:0/2450719431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 msgr2=0x5571c35b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c35b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 -- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31c92f0 msgr2=0x5571c35b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31c92f0 0x5571c35b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efdde000640 1 -- 172.21.15.27:0/2450719431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c361d050 con 0x5571c31f21b0 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efddcc00640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c35b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.003+0000 7efddd600640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31c92f0 0x5571c35b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efdde000640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31f21b0 0x5571c361cae0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efdd00058c0 tx=0x7efdd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86451a3dac69638c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efdbea00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efdbea00640 1 -- 172.21.15.27:0/2450719431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd001bd70 con 0x5571c31f21b0 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdcc0ad6f0 con 0x5571c31f21b0 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efdbea00640 1 -- 172.21.15.27:0/2450719431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9549+0+0 (secure 0 0 0) 0x7efdd0005020 con 0x5571c31f21b0 2024-09-21T07:21:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efdbea00640 1 -- 172.21.15.27:0/2450719431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdd0032050 con 0x5571c31f21b0 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efde708e740 10 client.17862 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571c361e340 con 0x5571c31f21b0 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.004+0000 7efdbea00640 1 -- 172.21.15.27:0/2450719431 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdcc0ad6f0 con 0x5571c31f21b0 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efdbea00640 1 -- 172.21.15.27:0/2450719431 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efdd0019040 con 0x5571c31f21b0 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efde708e740 10 client.17862 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efde708e740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efde708e740 20 client.17862 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efde708e740 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31861c0 0x5571c3620bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efde708e740 4 client.17862 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571c350a0b0 con 0x5571c31861c0 2024-09-21T07:21:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.005+0000 7efddd600640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31861c0 0x5571c3620bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.006+0000 7efddd600640 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31861c0 0x5571c3620bb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a7178d615a4c969 server_cookie=52698292505210fb in_seq=0 out_seq=0 2024-09-21T07:21:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.006+0000 7efdbea00640 10 client.17862 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efdbea00640 1 -- 172.21.15.27:0/2450719431 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571c350a0b0 con 0x5571c31861c0 2024-09-21T07:21:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efdbea00640 10 client.17862 handle_command_reply: tid=0 2024-09-21T07:21:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 client.17862 shutdown 2024-09-21T07:21:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31f21b0 msgr2=0x5571c361cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31f21b0 0x5571c361cae0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efdd00058c0 tx=0x7efdd00046c0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 shutdown_connections 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5571c31861c0 0x5571c3620bb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5571c31d14d0 0x5571c35b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5571c31f21b0 0x5571c361cae0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efdbea00640 0 client.17862 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 --2- 172.21.15.27:0/2450719431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571c31c92f0 0x5571c35b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 >> 172.21.15.27:0/2450719431 conn(0x5571c31e6300 msgr2=0x5571c3593360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 shutdown_connections 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 1 -- 172.21.15.27:0/2450719431 wait complete. 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.017+0000 7efde708e740 20 client.17862 trim_cache size 0 max 16384 2024-09-21T07:21:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdc00b6680 msgr2=0x7efdc00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdc00b6680 0x7efdc00b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efdd0005fd0 tx=0x7efdd000b5c0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102530 msgr2=0x7efdd81a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102530 0x7efdd81a08b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00eab0 tx=0x7efdcc00ef70 comp rx=0 tx=0).stop 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 shutdown_connections 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd81079f0 0x7efdd81a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efdc00b6680 0x7efdc00b8b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd8102e80 0x7efdd81a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 --2- 172.21.15.27:0/1930037204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102530 0x7efdd81a08b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 >> 172.21.15.27:0/1930037204 conn(0x7efdd80fe230 msgr2=0x7efdd8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 shutdown_connections 2024-09-21T07:21:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:09.018+0000 7efddea00640 1 -- 172.21.15.27:0/1930037204 wait complete. 2024-09-21T07:21:09.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:08 smithi027 ceph-mon[30779]: pgmap v626: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-09-21T07:21:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:08 smithi153 ceph-mon[34747]: pgmap v626: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-09-21T07:21:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:08 smithi119 ceph-mon[34519]: pgmap v626: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-09-21T07:21:11.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:10 smithi027 ceph-mon[30779]: pgmap v627: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2024-09-21T07:21:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:10 smithi153 ceph-mon[34747]: pgmap v627: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2024-09-21T07:21:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:10 smithi119 ceph-mon[34519]: pgmap v627: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2024-09-21T07:21:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:12 smithi119 ceph-mon[34519]: pgmap v628: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-21T07:21:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:12 smithi119 ceph-mon[34519]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T07:21:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:12 smithi027 ceph-mon[30779]: pgmap v628: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-21T07:21:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:12 smithi027 ceph-mon[30779]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T07:21:13.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:12 smithi153 ceph-mon[34747]: pgmap v628: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-21T07:21:13.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:12 smithi153 ceph-mon[34747]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T07:21:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:14 smithi119 ceph-mon[34519]: pgmap v630: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-21T07:21:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:14 smithi027 ceph-mon[30779]: pgmap v630: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-21T07:21:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:14 smithi153 ceph-mon[34747]: pgmap v630: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-21T07:21:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:16 smithi119 ceph-mon[34519]: pgmap v631: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-21T07:21:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:16 smithi027 ceph-mon[30779]: pgmap v631: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-21T07:21:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:16 smithi153 ceph-mon[34747]: pgmap v631: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-21T07:21:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:18 smithi119 ceph-mon[34519]: pgmap v632: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-21T07:21:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:18 smithi027 ceph-mon[30779]: pgmap v632: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-21T07:21:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:18 smithi153 ceph-mon[34747]: pgmap v632: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-21T07:21:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:20 smithi119 ceph-mon[34519]: pgmap v633: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-21T07:21:21.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:20 smithi027 ceph-mon[30779]: pgmap v633: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-21T07:21:21.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:20 smithi153 ceph-mon[34747]: pgmap v633: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-21T07:21:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 -- 172.21.15.27:0/2480014326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4568105cc0 msgr2=0x7f45681060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 --2- 172.21.15.27:0/2480014326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4568105cc0 0x7f45681060d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4558009860 tx=0x7f455802a340 comp rx=0 tx=0).stop 2024-09-21T07:21:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 -- 172.21.15.27:0/2480014326 shutdown_connections 2024-09-21T07:21:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 --2- 172.21.15.27:0/2480014326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f456810bf40 0x7f4568112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 --2- 172.21.15.27:0/2480014326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456810b5f0 0x7f456810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 --2- 172.21.15.27:0/2480014326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4568105cc0 0x7f45681060d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 -- 172.21.15.27:0/2480014326 >> 172.21.15.27:0/2480014326 conn(0x7f45681016e0 msgr2=0x7f4568103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 -- 172.21.15.27:0/2480014326 shutdown_connections 2024-09-21T07:21:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.224+0000 7f456ea00640 1 -- 172.21.15.27:0/2480014326 wait complete. 2024-09-21T07:21:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.225+0000 7f456ea00640 1 Processor -- start 2024-09-21T07:21:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.225+0000 7f456ea00640 1 -- start start 2024-09-21T07:21:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f456ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4568105cc0 0x7f456819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f456ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f456810b5f0 0x7f456819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f456ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456810bf40 0x7f45681a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f456ea00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4568115c00 con 0x7f456810bf40 2024-09-21T07:21:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f456ea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4568115a80 con 0x7f4568105cc0 2024-09-21T07:21:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f456ea00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4568115d80 con 0x7f456810b5f0 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f4567e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4568105cc0 0x7f456819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f455fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f456810b5f0 0x7f456819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f4567e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4568105cc0 0x7f456819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48422/0 (socket says 172.21.15.27:48422) 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f4567e00640 1 -- 172.21.15.27:0/1719459224 learned_addr learned my addr 172.21.15.27:0/1719459224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.226+0000 7f456ca00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456810bf40 0x7f45681a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f455fe00640 1 -- 172.21.15.27:0/1719459224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4568105cc0 msgr2=0x7f456819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f455fe00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4568105cc0 0x7f456819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f455fe00640 1 -- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456810bf40 msgr2=0x7f45681a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f455fe00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456810bf40 0x7f45681a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f455fe00640 1 -- 172.21.15.27:0/1719459224 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45681a2570 con 0x7f456810b5f0 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f4567e00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4568105cc0 0x7f456819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f455fe00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f456810b5f0 0x7f456819cbe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f456806c7b0 tx=0x7f455000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f4565600640 1 -- 172.21.15.27:0/1719459224 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455000ccd0 con 0x7f456810b5f0 2024-09-21T07:21:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f4565600640 1 -- 172.21.15.27:0/1719459224 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45500035f0 con 0x7f456810b5f0 2024-09-21T07:21:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681a2860 con 0x7f456810b5f0 2024-09-21T07:21:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f4565600640 1 -- 172.21.15.27:0/1719459224 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550010690 con 0x7f456810b5f0 2024-09-21T07:21:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.227+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a8ff0 con 0x7f456810b5f0 2024-09-21T07:21:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.230+0000 7f4575a5a740 1 Processor -- start 2024-09-21T07:21:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4575a5a740 1 -- start start 2024-09-21T07:21:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b574c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1a72f0 0x55de9b575170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1af4d0 0x55de9b5756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4575a5a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55de9b4ea6b0 con 0x55de9b1a72f0 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4575a5a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55de9b4d9ae0 con 0x55de9b1d01b0 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4575a5a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de9b4dd8f0 con 0x55de9b1af4d0 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4567e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1a72f0 0x55de9b575170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4567e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1a72f0 0x55de9b575170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37632/0 (socket says 172.21.15.27:37632) 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f4567e00640 1 -- 172.21.15.27:0/2839888184 learned_addr learned my addr 172.21.15.27:0/2839888184 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.231+0000 7f456ca00640 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b574c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f4567e00640 1 -- 172.21.15.27:0/2839888184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1af4d0 msgr2=0x55de9b5756b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f4567e00640 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1af4d0 0x55de9b5756b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f4567e00640 1 -- 172.21.15.27:0/2839888184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 msgr2=0x55de9b574c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f4567e00640 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b574c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f4567e00640 1 -- 172.21.15.27:0/2839888184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9b4a6780 con 0x55de9b1a72f0 2024-09-21T07:21:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f4565600640 1 -- 172.21.15.27:0/1719459224 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4550003a80 con 0x7f456810b5f0 2024-09-21T07:21:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f4567e00640 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1a72f0 0x55de9b575170 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4558000c00 tx=0x7f4558009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbb2fc9e1929bba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f455d200640 1 -- 172.21.15.27:0/2839888184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558037040 con 0x55de9b1a72f0 2024-09-21T07:21:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f455d200640 1 -- 172.21.15.27:0/2839888184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4558030450 con 0x55de9b1a72f0 2024-09-21T07:21:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.232+0000 7f455d200640 1 -- 172.21.15.27:0/2839888184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558032a30 con 0x55de9b1a72f0 2024-09-21T07:21:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 -- 172.21.15.27:0/2839888184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1a72f0 msgr2=0x55de9b575170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1a72f0 0x55de9b575170 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4558000c00 tx=0x7f4558009510 comp rx=0 tx=0).stop 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 -- 172.21.15.27:0/2839888184 shutdown_connections 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1af4d0 0x55de9b5756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1a72f0 0x55de9b575170 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 --2- 172.21.15.27:0/2839888184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b574c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 -- 172.21.15.27:0/2839888184 >> 172.21.15.27:0/2839888184 conn(0x55de9b1c4300 msgr2=0x55de9b572b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 -- 172.21.15.27:0/2839888184 shutdown_connections 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4575a5a740 1 -- 172.21.15.27:0/2839888184 wait complete. 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.233+0000 7f4565600640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f454c0b6590 0x7f454c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.234+0000 7f4565600640 1 -- 172.21.15.27:0/1719459224 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f45500e7c90 con 0x7f456810b5f0 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.234+0000 7f4567e00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f454c0b6590 0x7f454c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.234+0000 7f4575a5a740 1 -- start start 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.234+0000 7f4575a5a740 1 Processor -- start 2024-09-21T07:21:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4567e00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f454c0b6590 0x7f454c0b8aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4558009830 tx=0x7f455802a340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b580240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b580780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b580cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4575a5a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55de9b411ab0 con 0x55de9b1af4d0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4575a5a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55de9b480570 con 0x55de9b1a72f0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4575a5a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de9b583350 con 0x55de9b1d01b0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4575a5a740 10 client.0 _ll_register_callbacks cb 0x55de9b48f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f455fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b580780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f455fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b580780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37634/0 (socket says 172.21.15.27:37634) 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f455fe00640 1 -- 172.21.15.27:0/4203762089 learned_addr learned my addr 172.21.15.27:0/4203762089 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f4567e00640 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b580240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.235+0000 7f456ca00640 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b580cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f455fe00640 1 -- 172.21.15.27:0/4203762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 msgr2=0x55de9b580cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f455fe00640 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b580cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f455fe00640 1 -- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 msgr2=0x55de9b580240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f455fe00640 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b580240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f455fe00640 1 -- 172.21.15.27:0/4203762089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9b23b790 con 0x55de9b1af4d0 2024-09-21T07:21:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f455fe00640 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b580780 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f45500a9d00 tx=0x7f45500ad500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7239fe8cb238f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f4555600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f4555600640 1 -- 172.21.15.27:0/4203762089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500acea0 con 0x55de9b1af4d0 2024-09-21T07:21:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f4555600640 1 -- 172.21.15.27:0/4203762089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f45500a9080 con 0x55de9b1af4d0 2024-09-21T07:21:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4558030450 con 0x55de9b1af4d0 2024-09-21T07:21:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f4555600640 1 -- 172.21.15.27:0/4203762089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4550009530 con 0x55de9b1af4d0 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f4575a5a740 10 client.17874 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.236+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de9b2b4620 con 0x55de9b1af4d0 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4555600640 1 -- 172.21.15.27:0/4203762089 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4558030450 con 0x55de9b1af4d0 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4555600640 1 -- 172.21.15.27:0/4203762089 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f45500b33a0 con 0x55de9b1af4d0 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4575a5a740 10 client.17874 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4575a5a740 10 client.17874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4575a5a740 20 client.17874 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4575a5a740 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b18ed80 0x55de9b592300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4575a5a740 4 client.17874 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de9b4ec600 con 0x55de9b18ed80 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.237+0000 7f4567e00640 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b18ed80 0x55de9b592300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.238+0000 7f4567e00640 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b18ed80 0x55de9b592300 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aded1a869b2837b server_cookie=624106cfad49d382 in_seq=0 out_seq=0 2024-09-21T07:21:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.238+0000 7f4555600640 10 client.17874 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4555600640 1 -- 172.21.15.27:0/4203762089 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de9b4ec600 con 0x55de9b18ed80 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4555600640 10 client.17874 handle_command_reply: tid=0 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 client.17874 shutdown 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 msgr2=0x55de9b580780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b580780 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f45500a9d00 tx=0x7f45500ad500 comp rx=0 tx=0).stop 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 shutdown_connections 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b18ed80 0x55de9b592300 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b580cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b580780 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4555600640 0 client.17874 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 --2- 172.21.15.27:0/4203762089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b580240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 >> 172.21.15.27:0/4203762089 conn(0x55de9b1c4300 msgr2=0x55de9b575c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 shutdown_connections 2024-09-21T07:21:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.239+0000 7f4575a5a740 1 -- 172.21.15.27:0/4203762089 wait complete. 2024-09-21T07:21:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.240+0000 7f4575a5a740 20 client.17874 trim_cache size 0 max 16384 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 Processor -- start 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 -- start start 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1a72f0 0x55de9b5717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b57d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b57d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b5910 con 0x55de9b1af4d0 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b31e0 con 0x55de9b1d01b0 2024-09-21T07:21:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4575a5a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b1050 con 0x55de9b1a72f0 2024-09-21T07:21:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f455fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b57d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f455fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b57d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37650/0 (socket says 172.21.15.27:37650) 2024-09-21T07:21:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f455fe00640 1 -- 172.21.15.27:0/3485037566 learned_addr learned my addr 172.21.15.27:0/3485037566 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.255+0000 7f4567e00640 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1a72f0 0x55de9b5717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f456ca00640 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b57d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f455fe00640 1 -- 172.21.15.27:0/3485037566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1a72f0 msgr2=0x55de9b5717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f455fe00640 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1a72f0 0x55de9b5717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f455fe00640 1 -- 172.21.15.27:0/3485037566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 msgr2=0x55de9b57d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f455fe00640 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b57d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f455fe00640 1 -- 172.21.15.27:0/3485037566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9b592670 con 0x55de9b1af4d0 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f456ca00640 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b57d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f455fe00640 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b57d430 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f45500e5630 tx=0x7f45500ad630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359bcbdfb85d490a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f4557e00640 1 -- 172.21.15.27:0/3485037566 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500acea0 con 0x55de9b1af4d0 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f4557e00640 1 -- 172.21.15.27:0/3485037566 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45500ad240 con 0x55de9b1af4d0 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.256+0000 7f4557e00640 1 -- 172.21.15.27:0/3485037566 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500b67b0 con 0x55de9b1af4d0 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 -- 172.21.15.27:0/3485037566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 msgr2=0x55de9b57d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b57d430 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f45500e5630 tx=0x7f45500ad630 comp rx=0 tx=0).stop 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 -- 172.21.15.27:0/3485037566 shutdown_connections 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1d01b0 0x55de9b57d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b57d430 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 --2- 172.21.15.27:0/3485037566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1a72f0 0x55de9b5717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 -- 172.21.15.27:0/3485037566 >> 172.21.15.27:0/3485037566 conn(0x55de9b1c4300 msgr2=0x55de9b5710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 -- 172.21.15.27:0/3485037566 shutdown_connections 2024-09-21T07:21:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.257+0000 7f4575a5a740 1 -- 172.21.15.27:0/3485037566 wait complete. 2024-09-21T07:21:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 -- start start 2024-09-21T07:21:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 Processor -- start 2024-09-21T07:21:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b586760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b586ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b5871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b16a0 con 0x55de9b1af4d0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b2b10 con 0x55de9b1a72f0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b1d70 con 0x55de9b1d01b0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.258+0000 7f4575a5a740 10 client.0 _ll_register_callbacks cb 0x7f4550004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f455fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b586ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f4567e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b586760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f456ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b5871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f455fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b586ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37664/0 (socket says 172.21.15.27:37664) 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f455fe00640 1 -- 172.21.15.27:0/1925869974 learned_addr learned my addr 172.21.15.27:0/1925869974 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f456ca00640 1 -- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 msgr2=0x55de9b586760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f456ca00640 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b586760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f456ca00640 1 -- 172.21.15.27:0/1925869974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 msgr2=0x55de9b586ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f456ca00640 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b586ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f456ca00640 1 -- 172.21.15.27:0/1925869974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9b5fbe30 con 0x55de9b1d01b0 2024-09-21T07:21:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f4567e00640 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b586760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f456ca00640 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b5871e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4560003d30 tx=0x7f4560004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae9f001fd73d79e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.259+0000 7f4555600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4555600640 1 -- 172.21.15.27:0/1925869974 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560020060 con 0x55de9b1d01b0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45500ad240 con 0x55de9b1d01b0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4555600640 1 -- 172.21.15.27:0/1925869974 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f45600119d0 con 0x55de9b1d01b0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4555600640 1 -- 172.21.15.27:0/1925869974 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4560027050 con 0x55de9b1d01b0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 10 client.27865 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de9b5fd150 con 0x55de9b1d01b0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4555600640 1 -- 172.21.15.27:0/1925869974 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45500ad240 con 0x55de9b1d01b0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4555600640 1 -- 172.21.15.27:0/1925869974 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4560010430 con 0x55de9b1d01b0 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 10 client.27865 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 10 client.27865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 20 client.27865 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b1641c0 0x55de9b5ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.260+0000 7f4575a5a740 4 client.27865 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:21:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.261+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de9b4e80b0 con 0x55de9b1641c0 2024-09-21T07:21:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.261+0000 7f4567e00640 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b1641c0 0x55de9b5ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.261+0000 7f4567e00640 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b1641c0 0x55de9b5ff920 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874afe49fbcfb5b6 server_cookie=a40fa2c0ecd0429e in_seq=0 out_seq=0 2024-09-21T07:21:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.261+0000 7f4555600640 10 client.27865 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.269+0000 7f4555600640 1 -- 172.21.15.27:0/1925869974 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de9b4e80b0 con 0x55de9b1641c0 2024-09-21T07:21:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.269+0000 7f4555600640 10 client.27865 handle_command_reply: tid=0 2024-09-21T07:21:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 client.27865 shutdown 2024-09-21T07:21:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 msgr2=0x55de9b5871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b5871e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4560003d30 tx=0x7f4560004410 comp rx=0 tx=0).stop 2024-09-21T07:21:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 shutdown_connections 2024-09-21T07:21:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55de9b1641c0 0x55de9b5ff920 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de9b1d01b0 0x55de9b5871e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de9b1af4d0 0x55de9b586ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4555600640 0 client.27865 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 --2- 172.21.15.27:0/1925869974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de9b1a72f0 0x55de9b586760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 >> 172.21.15.27:0/1925869974 conn(0x55de9b1c4300 msgr2=0x55de9b575c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 shutdown_connections 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 1 -- 172.21.15.27:0/1925869974 wait complete. 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.270+0000 7f4575a5a740 20 client.27865 trim_cache size 0 max 16384 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:21:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:21:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:21:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:21:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f454c0b6590 msgr2=0x7f454c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f454c0b6590 0x7f454c0b8aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4558009830 tx=0x7f455802a340 comp rx=0 tx=0).stop 2024-09-21T07:21:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f456810b5f0 msgr2=0x7f456819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f456810b5f0 0x7f456819cbe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f456806c7b0 tx=0x7f455000ee50 comp rx=0 tx=0).stop 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 shutdown_connections 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f456810b5f0 0x7f456819cbe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f454c0b6590 0x7f454c0b8aa0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456810bf40 0x7f45681a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 --2- 172.21.15.27:0/1719459224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4568105cc0 0x7f456819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 >> 172.21.15.27:0/1719459224 conn(0x7f45681016e0 msgr2=0x7f4568103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 shutdown_connections 2024-09-21T07:21:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:22.271+0000 7f456ea00640 1 -- 172.21.15.27:0/1719459224 wait complete. 2024-09-21T07:21:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:22 smithi119 ceph-mon[34519]: pgmap v634: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-21T07:21:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:22 smithi027 ceph-mon[30779]: pgmap v634: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-21T07:21:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:22 smithi153 ceph-mon[34747]: pgmap v634: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-21T07:21:23.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:21:23.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:21:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:24.539 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 -- 172.21.15.27:0/2785933637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 msgr2=0x7f0b1010a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 --2- 172.21.15.27:0/2785933637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b1010a5c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009900 tx=0x7f0b0002a410 comp rx=0 tx=0).stop 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 -- 172.21.15.27:0/2785933637 shutdown_connections 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 --2- 172.21.15.27:0/2785933637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1010faf0 0x7f0b1010ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 --2- 172.21.15.27:0/2785933637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b10110440 0x7f0b10116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 --2- 172.21.15.27:0/2785933637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b1010a5c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 -- 172.21.15.27:0/2785933637 >> 172.21.15.27:0/2785933637 conn(0x7f0b100780e0 msgr2=0x7f0b101081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.623+0000 7f0b15800640 1 -- 172.21.15.27:0/2785933637 shutdown_connections 2024-09-21T07:21:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.624+0000 7f0b15800640 1 -- 172.21.15.27:0/2785933637 wait complete. 2024-09-21T07:21:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.624+0000 7f0b15800640 1 Processor -- start 2024-09-21T07:21:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b15800640 1 -- start start 2024-09-21T07:21:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b15800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b101a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b15800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b1010faf0 0x7f0b101a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b15800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b10110440 0x7f0b101a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b15800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1011a110 con 0x7f0b1010faf0 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b15800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10119f90 con 0x7f0b1010a1b0 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b15800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1011a290 con 0x7f0b10110440 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.625+0000 7f0b06000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b1010faf0 0x7f0b101a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b101a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b101a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48482/0 (socket says 172.21.15.27:48482) 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 -- 172.21.15.27:0/1907298558 learned_addr learned my addr 172.21.15.27:0/1907298558 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0f400640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b10110440 0x7f0b101a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b06000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b1010faf0 0x7f0b101a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37686/0 (socket says 172.21.15.27:37686) 2024-09-21T07:21:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 -- 172.21.15.27:0/1907298558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b10110440 msgr2=0x7f0b101a63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b10110440 0x7f0b101a63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 -- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b1010faf0 msgr2=0x7f0b101a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b1010faf0 0x7f0b101a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 -- 172.21.15.27:0/1907298558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b101a6ad0 con 0x7f0b1010a1b0 2024-09-21T07:21:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.626+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b101a0b60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005940 tx=0x7f0b000040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.627+0000 7f0b07e00640 1 -- 172.21.15.27:0/1907298558 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0003f030 con 0x7f0b1010a1b0 2024-09-21T07:21:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.627+0000 7f0b07e00640 1 -- 172.21.15.27:0/1907298558 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000044c0 con 0x7f0b1010a1b0 2024-09-21T07:21:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.627+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b101a6d60 con 0x7f0b1010a1b0 2024-09-21T07:21:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.627+0000 7f0b07e00640 1 -- 172.21.15.27:0/1907298558 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00035070 con 0x7f0b1010a1b0 2024-09-21T07:21:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.627+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101ad520 con 0x7f0b1010a1b0 2024-09-21T07:21:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.630+0000 7f0b1c457740 1 Processor -- start 2024-09-21T07:21:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.630+0000 7f0b1c457740 1 -- start start 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.630+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb6c070 0x55566ff12d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.630+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb431b0 0x55566ff132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb4b390 0x55566ff137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b1c457740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55566fe89f70 con 0x55566fb431b0 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b1c457740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55566fe793a0 con 0x55566fb6c070 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b1c457740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55566fe7d1b0 con 0x55566fb4b390 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb6c070 0x55566ff12d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb6c070 0x55566ff12d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48484/0 (socket says 172.21.15.27:48484) 2024-09-21T07:21:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 -- 172.21.15.27:0/1683234282 learned_addr learned my addr 172.21.15.27:0/1683234282 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b06000640 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb4b390 0x55566ff137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b07e00640 1 -- 172.21.15.27:0/1907298558 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0b00042030 con 0x7f0b1010a1b0 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 -- 172.21.15.27:0/1683234282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb4b390 msgr2=0x55566ff137e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb4b390 0x55566ff137e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 -- 172.21.15.27:0/1683234282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb431b0 msgr2=0x55566ff132a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb431b0 0x55566ff132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.631+0000 7f0b0f400640 1 -- 172.21.15.27:0/1683234282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566fe43c90 con 0x55566fb6c070 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.632+0000 7f0b0f400640 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb6c070 0x55566ff12d60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009f50 tx=0x7f0b080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca62e14599df0ead server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.632+0000 7f0affe00640 1 -- 172.21.15.27:0/1683234282 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004d20 con 0x55566fb6c070 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.632+0000 7f0affe00640 1 -- 172.21.15.27:0/1683234282 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b08004050 con 0x55566fb6c070 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.632+0000 7f0affe00640 1 -- 172.21.15.27:0/1683234282 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08005280 con 0x55566fb6c070 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 -- 172.21.15.27:0/1683234282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb6c070 msgr2=0x55566ff12d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb6c070 0x55566ff12d60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009f50 tx=0x7f0b080118d0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b07e00640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0af40b6680 0x7f0af40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 -- 172.21.15.27:0/1683234282 shutdown_connections 2024-09-21T07:21:24.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb4b390 0x55566ff137e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb431b0 0x55566ff132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 --2- 172.21.15.27:0/1683234282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb6c070 0x55566ff12d60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 -- 172.21.15.27:0/1683234282 >> 172.21.15.27:0/1683234282 conn(0x55566fb60460 msgr2=0x55566ff10c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b07e00640 1 -- 172.21.15.27:0/1907298558 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0b00107e60 con 0x7f0b1010a1b0 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b1c457740 1 -- 172.21.15.27:0/1683234282 shutdown_connections 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.633+0000 7f0b0f400640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0af40b6680 0x7f0af40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.634+0000 7f0b1c457740 1 -- 172.21.15.27:0/1683234282 wait complete. 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.634+0000 7f0b0f400640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0af40b6680 0x7f0af40b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0b080048f0 tx=0x7f0b08004710 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.634+0000 7f0b1c457740 1 -- start start 2024-09-21T07:21:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 1 Processor -- start 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff1e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff1eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff2a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55566fcda700 con 0x55566fb6c070 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55566fe14540 con 0x55566fb4b390 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55566ff21540 con 0x55566fb431b0 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b1c457740 10 client.0 _ll_register_callbacks cb 0x55566fe24430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.635+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff1e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff2a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff1e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39276/0 (socket says 172.21.15.27:39276) 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 -- 172.21.15.27:0/2170942390 learned_addr learned my addr 172.21.15.27:0/2170942390 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b06000640 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff1eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 -- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 msgr2=0x55566ff1eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff1eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 -- 172.21.15.27:0/2170942390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 msgr2=0x55566ff2a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff2a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 -- 172.21.15.27:0/2170942390 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566fbd7650 con 0x55566fb431b0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff1e560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0b00006010 tx=0x7f0b000ca0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26f965a8e347978 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.636+0000 7f0afcc00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.637+0000 7f0afcc00640 1 -- 172.21.15.27:0/2170942390 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00049040 con 0x55566fb431b0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.637+0000 7f0afcc00640 1 -- 172.21.15.27:0/2170942390 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0b000d6e30 con 0x55566fb431b0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.637+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55566fe16b50 con 0x55566fb431b0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.637+0000 7f0afcc00640 1 -- 172.21.15.27:0/2170942390 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b000cd0c0 con 0x55566fb431b0 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.637+0000 7f0b1c457740 10 client.27877 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:24.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.637+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55566fe31910 con 0x55566fb431b0 2024-09-21T07:21:24.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.637+0000 7f0afcc00640 1 -- 172.21.15.27:0/2170942390 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55566fe16b50 con 0x55566fb431b0 2024-09-21T07:21:24.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0afcc00640 1 -- 172.21.15.27:0/2170942390 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0b000c92a0 con 0x55566fb431b0 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0b1c457740 10 client.27877 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0b1c457740 10 client.27877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0b1c457740 20 client.27877 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566fb2a9f0 0x55566ff30660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0b1c457740 4 client.27877 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55566fe8bec0 con 0x55566fb2a9f0 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.638+0000 7f0b06000640 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566fb2a9f0 0x55566ff30660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.639+0000 7f0b06000640 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566fb2a9f0 0x55566ff30660 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21017ceb4d5b0215 server_cookie=d90ba483b2908948 in_seq=0 out_seq=0 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.639+0000 7f0afcc00640 10 client.27877 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0afcc00640 1 -- 172.21.15.27:0/2170942390 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55566fe8bec0 con 0x55566fb2a9f0 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0afcc00640 10 client.27877 handle_command_reply: tid=0 2024-09-21T07:21:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 client.27877 shutdown 2024-09-21T07:21:24.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 msgr2=0x55566ff1e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff1e560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0b00006010 tx=0x7f0b000ca0d0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 shutdown_connections 2024-09-21T07:21:24.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566fb2a9f0 0x55566ff30660 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff2a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff1eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2170942390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff1e560 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0afcc00640 0 client.27877 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:24.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 >> 172.21.15.27:0/2170942390 conn(0x55566fb60460 msgr2=0x55566ff15ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:24.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 shutdown_connections 2024-09-21T07:21:24.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 1 -- 172.21.15.27:0/2170942390 wait complete. 2024-09-21T07:21:24.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.640+0000 7f0b1c457740 20 client.27877 trim_cache size 0 max 16384 2024-09-21T07:21:24.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.659+0000 7f0b1c457740 1 Processor -- start 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.659+0000 7f0b1c457740 1 -- start start 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb431b0 0x55566ff198d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb4b390 0x55566ff2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb6c070 0x55566ff2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b1c457740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101b9d40 con 0x55566fb4b390 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b1c457740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101b7610 con 0x55566fb431b0 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b1c457740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101b5480 con 0x55566fb6c070 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b06000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb4b390 0x55566ff2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb431b0 0x55566ff198d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb431b0 0x55566ff198d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48498/0 (socket says 172.21.15.27:48498) 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 -- 172.21.15.27:0/3187576492 learned_addr learned my addr 172.21.15.27:0/3187576492 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0f400640 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb6c070 0x55566ff2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 -- 172.21.15.27:0/3187576492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb6c070 msgr2=0x55566ff2aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb6c070 0x55566ff2aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 -- 172.21.15.27:0/3187576492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb4b390 msgr2=0x55566ff2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb4b390 0x55566ff2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0ea00640 1 -- 172.21.15.27:0/3187576492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566ff15050 con 0x55566fb431b0 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.660+0000 7f0b0f400640 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb6c070 0x55566ff2aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.661+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb431b0 0x55566ff198d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0b000cd650 tx=0x7f0b0002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cea598367afbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.661+0000 7f0aff400640 1 -- 172.21.15.27:0/3187576492 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00049070 con 0x55566fb431b0 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.661+0000 7f0aff400640 1 -- 172.21.15.27:0/3187576492 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000cd460 con 0x55566fb431b0 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.661+0000 7f0aff400640 1 -- 172.21.15.27:0/3187576492 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00105630 con 0x55566fb431b0 2024-09-21T07:21:24.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 -- 172.21.15.27:0/3187576492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb431b0 msgr2=0x55566ff198d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb431b0 0x55566ff198d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0b000cd650 tx=0x7f0b0002a410 comp rx=0 tx=0).stop 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 -- 172.21.15.27:0/3187576492 shutdown_connections 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb6c070 0x55566ff2aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb4b390 0x55566ff2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 --2- 172.21.15.27:0/3187576492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb431b0 0x55566ff198d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 -- 172.21.15.27:0/3187576492 >> 172.21.15.27:0/3187576492 conn(0x55566fb60460 msgr2=0x55566ff164d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 -- 172.21.15.27:0/3187576492 shutdown_connections 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.662+0000 7f0b1c457740 1 -- 172.21.15.27:0/3187576492 wait complete. 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 -- start start 2024-09-21T07:21:24.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 Processor -- start 2024-09-21T07:21:24.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff22a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff22fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff9a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101b5ad0 con 0x55566fb6c070 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101b6f40 con 0x55566fb4b390 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101b61a0 con 0x55566fb431b0 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.663+0000 7f0b1c457740 10 client.0 _ll_register_callbacks cb 0x7f0b00003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff22a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff22fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff22fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48512/0 (socket says 172.21.15.27:48512) 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff22a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39282/0 (socket says 172.21.15.27:39282) 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 -- 172.21.15.27:0/2607326849 learned_addr learned my addr 172.21.15.27:0/2607326849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b0f400640 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff9a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 -- 172.21.15.27:0/2607326849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 msgr2=0x55566ff22a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff22a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 -- 172.21.15.27:0/2607326849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 msgr2=0x55566ff9a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff9a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b06000640 1 -- 172.21.15.27:0/2607326849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566ff23800 con 0x55566fb4b390 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.664+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff22a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0b06000640 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff22fd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009dd0 tx=0x7f0af8010b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ffeb189b47a695e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0afcc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0afcc00640 1 -- 172.21.15.27:0/2607326849 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8028030 con 0x55566fb4b390 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b000cd460 con 0x55566fb4b390 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0afcc00640 1 -- 172.21.15.27:0/2607326849 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0af801e9d0 con 0x55566fb4b390 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0afcc00640 1 -- 172.21.15.27:0/2607326849 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0af8032050 con 0x55566fb4b390 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0b1c457740 10 client.27926 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55566ff9b630 con 0x55566fb4b390 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.665+0000 7f0afcc00640 1 -- 172.21.15.27:0/2607326849 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000cd460 con 0x55566fb4b390 2024-09-21T07:21:24.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0afcc00640 1 -- 172.21.15.27:0/2607326849 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0af8023030 con 0x55566fb4b390 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0b1c457740 10 client.27926 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0b1c457740 10 client.27926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0b1c457740 20 client.27926 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566faffda0 0x55566ff9de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0b1c457740 4 client.27926 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55566fe87970 con 0x55566faffda0 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.666+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566faffda0 0x55566ff9de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.667+0000 7f0b0ea00640 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566faffda0 0x55566ff9de20 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1785748017355f9 server_cookie=ef59b33e96aa9337 in_seq=0 out_seq=0 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.667+0000 7f0afcc00640 10 client.27926 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.667+0000 7f0afcc00640 1 -- 172.21.15.27:0/2607326849 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55566fe87970 con 0x55566faffda0 2024-09-21T07:21:24.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.667+0000 7f0afcc00640 10 client.27926 handle_command_reply: tid=0 2024-09-21T07:21:24.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.667+0000 7f0b1c457740 1 client.27926 shutdown 2024-09-21T07:21:24.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.667+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 msgr2=0x55566ff22fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.667+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff22fd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009dd0 tx=0x7f0af8010b30 comp rx=0 tx=0).stop 2024-09-21T07:21:24.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 shutdown_connections 2024-09-21T07:21:24.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55566faffda0 0x55566ff9de20 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55566fb6c070 0x55566ff9a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55566fb4b390 0x55566ff22fd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0afcc00640 0 client.27926 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:24.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 --2- 172.21.15.27:0/2607326849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55566fb431b0 0x55566ff22a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 >> 172.21.15.27:0/2607326849 conn(0x55566fb60460 msgr2=0x55566ff1fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:24.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 shutdown_connections 2024-09-21T07:21:24.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 1 -- 172.21.15.27:0/2607326849 wait complete. 2024-09-21T07:21:24.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.668+0000 7f0b1c457740 20 client.27926 trim_cache size 0 max 16384 2024-09-21T07:21:24.676 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:21:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:21:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:21:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:21:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:21:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:21:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:21:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:21:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:21:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:21:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:21:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:21:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:21:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:21:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:21:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:21:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:21:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:21:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:21:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:21:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:21:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:21:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:21:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:21:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:21:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:24.691 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0af40b6680 msgr2=0x7f0af40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0af40b6680 0x7f0af40b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0b080048f0 tx=0x7f0b08004710 comp rx=0 tx=0).stop 2024-09-21T07:21:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 msgr2=0x7f0b101a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b101a0b60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005940 tx=0x7f0b000040c0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 shutdown_connections 2024-09-21T07:21:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b10110440 0x7f0b101a63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0af40b6680 0x7f0af40b8b90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b1010faf0 0x7f0b101a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 --2- 172.21.15.27:0/1907298558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b1010a1b0 0x7f0b101a0b60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:24.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 >> 172.21.15.27:0/1907298558 conn(0x7f0b100780e0 msgr2=0x7f0b10108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:24.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 shutdown_connections 2024-09-21T07:21:24.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:24.669+0000 7f0b15800640 1 -- 172.21.15.27:0/1907298558 wait complete. 2024-09-21T07:21:25.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:24 smithi027 ceph-mon[30779]: pgmap v635: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-09-21T07:21:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:24 smithi119 ceph-mon[34519]: pgmap v635: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-09-21T07:21:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:24 smithi153 ceph-mon[34747]: pgmap v635: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.548+0000 7f6845200640 1 -- 172.21.15.27:0/4179198669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6840076080 msgr2=0x7f684010fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.548+0000 7f6845200640 1 --2- 172.21.15.27:0/4179198669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6840076080 0x7f684010fa50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f682c009900 tx=0x7f682c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.549+0000 7f6845200640 1 -- 172.21.15.27:0/4179198669 shutdown_connections 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.549+0000 7f6845200640 1 --2- 172.21.15.27:0/4179198669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f684010ff90 0x7f68401123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.549+0000 7f6845200640 1 --2- 172.21.15.27:0/4179198669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6840076080 0x7f684010fa50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.549+0000 7f6845200640 1 --2- 172.21.15.27:0/4179198669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6840075730 0x7f6840075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.549+0000 7f6845200640 1 -- 172.21.15.27:0/4179198669 >> 172.21.15.27:0/4179198669 conn(0x7f68400ff400 msgr2=0x7f6840101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.549+0000 7f6845200640 1 -- 172.21.15.27:0/4179198669 shutdown_connections 2024-09-21T07:21:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.549+0000 7f6845200640 1 -- 172.21.15.27:0/4179198669 wait complete. 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 Processor -- start 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 -- start start 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6840075730 0x7f684019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6840076080 0x7f684019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f684010ff90 0x7f68401a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6840115b20 con 0x7f684010ff90 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f68401159a0 con 0x7f6840075730 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.550+0000 7f6845200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6840115ca0 con 0x7f6840076080 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6840076080 0x7f684019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6840076080 0x7f684019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39308/0 (socket says 172.21.15.27:39308) 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683ea00640 1 -- 172.21.15.27:0/1892884217 learned_addr learned my addr 172.21.15.27:0/1892884217 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:25.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f684010ff90 0x7f68401a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683f400640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6840075730 0x7f684019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683fe00640 1 -- 172.21.15.27:0/1892884217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6840076080 msgr2=0x7f684019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683fe00640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6840076080 0x7f684019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683fe00640 1 -- 172.21.15.27:0/1892884217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6840075730 msgr2=0x7f684019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683fe00640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6840075730 0x7f684019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683fe00640 1 -- 172.21.15.27:0/1892884217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68401a24c0 con 0x7f684010ff90 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.551+0000 7f683f400640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6840075730 0x7f684019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.552+0000 7f683fe00640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f684010ff90 0x7f68401a1d90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6838009fd0 tx=0x7f683800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.552+0000 7f6837e00640 1 -- 172.21.15.27:0/1892884217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683800ce20 con 0x7f684010ff90 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.552+0000 7f6837e00640 1 -- 172.21.15.27:0/1892884217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6838005020 con 0x7f684010ff90 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.552+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68401a27b0 con 0x7f684010ff90 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.552+0000 7f6837e00640 1 -- 172.21.15.27:0/1892884217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838010690 con 0x7f684010ff90 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.552+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68401a8f10 con 0x7f684010ff90 2024-09-21T07:21:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.553+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6840075b40 con 0x7f684010ff90 2024-09-21T07:21:25.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.556+0000 7f6837e00640 1 -- 172.21.15.27:0/1892884217 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f68380054d0 con 0x7f684010ff90 2024-09-21T07:21:25.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.557+0000 7f6837e00640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68240b6700 0x7f68240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:25.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.558+0000 7f6837e00640 1 -- 172.21.15.27:0/1892884217 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f68380e8100 con 0x7f684010ff90 2024-09-21T07:21:25.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.558+0000 7f683f400640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68240b6700 0x7f68240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:25.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.559+0000 7f683f400640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68240b6700 0x7f68240b8c10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6830009fc0 tx=0x7f6830005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:25.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.560+0000 7f6837e00640 1 -- 172.21.15.27:0/1892884217 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68380ae240 con 0x7f684010ff90 2024-09-21T07:21:25.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.824+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f684019d680 con 0x7f684010ff90 2024-09-21T07:21:25.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.824+0000 7f6837e00640 1 -- 172.21.15.27:0/1892884217 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f684019d680 con 0x7f684010ff90 2024-09-21T07:21:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:21:25.827 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:21:25.827 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:21:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68240b6700 msgr2=0x7f68240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68240b6700 0x7f68240b8c10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6830009fc0 tx=0x7f6830005f90 comp rx=0 tx=0).stop 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f684010ff90 msgr2=0x7f68401a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f684010ff90 0x7f68401a1d90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6838009fd0 tx=0x7f683800ef10 comp rx=0 tx=0).stop 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 shutdown_connections 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6840076080 0x7f684019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68240b6700 0x7f68240b8c10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f684010ff90 0x7f68401a1d90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 --2- 172.21.15.27:0/1892884217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6840075730 0x7f684019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 >> 172.21.15.27:0/1892884217 conn(0x7f68400ff400 msgr2=0x7f6840109b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.827+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 shutdown_connections 2024-09-21T07:21:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:25.828+0000 7f6845200640 1 -- 172.21.15.27:0/1892884217 wait complete. 2024-09-21T07:21:26.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:21:26.158 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:21:26.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:26 smithi027 ceph-mon[30779]: pgmap v636: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-21T07:21:26.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:26 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1892884217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:27.026 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 9623a0a1 --release --await' 2024-09-21T07:21:27.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 9623a0a1 --release --await 2024-09-21T07:21:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:26 smithi119 ceph-mon[34519]: pgmap v636: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-21T07:21:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:26 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1892884217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:27.056 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:26 smithi153 ceph-mon[34747]: pgmap v636: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-21T07:21:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:26 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1892884217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:27.724 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:27.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 -- 172.21.15.27:0/101545129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df0109c40 msgr2=0x7f8df0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 --2- 172.21.15.27:0/101545129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df0109c40 0x7f8df0110470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009ca0 tx=0x7f8de802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 -- 172.21.15.27:0/101545129 shutdown_connections 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 --2- 172.21.15.27:0/101545129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df0109c40 0x7f8df0110470 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 --2- 172.21.15.27:0/101545129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df01092f0 0x7f8df0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 --2- 172.21.15.27:0/101545129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 0x7f8df0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 -- 172.21.15.27:0/101545129 >> 172.21.15.27:0/101545129 conn(0x7f8df00ff440 msgr2=0x7f8df0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 -- 172.21.15.27:0/101545129 shutdown_connections 2024-09-21T07:21:27.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.740+0000 7f8def200640 1 -- 172.21.15.27:0/101545129 wait complete. 2024-09-21T07:21:27.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 Processor -- start 2024-09-21T07:21:27.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 -- start start 2024-09-21T07:21:27.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 0x7f8df0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:27.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df01092f0 0x7f8df0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:27.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df0109c40 0x7f8df0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:27.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0113950 con 0x7f8df01039b0 2024-09-21T07:21:27.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01137d0 con 0x7f8df0109c40 2024-09-21T07:21:27.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.742+0000 7f8def200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0113ad0 con 0x7f8df01092f0 2024-09-21T07:21:27.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 0x7f8df0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8ded400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df01092f0 0x7f8df0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 0x7f8df0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37758/0 (socket says 172.21.15.27:37758) 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 -- 172.21.15.27:0/2490599761 learned_addr learned my addr 172.21.15.27:0/2490599761 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dee800640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df0109c40 0x7f8df0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 -- 172.21.15.27:0/2490599761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df01092f0 msgr2=0x7f8df0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df01092f0 0x7f8df0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 -- 172.21.15.27:0/2490599761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df0109c40 msgr2=0x7f8df0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df0109c40 0x7f8df0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dede00640 1 -- 172.21.15.27:0/2490599761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df01a8560 con 0x7f8df01039b0 2024-09-21T07:21:27.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.743+0000 7f8dee800640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df0109c40 0x7f8df0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.744+0000 7f8dede00640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 0x7f8df0075710 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e9f0 tx=0x7f8de000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.744+0000 7f8de6a00640 1 -- 172.21.15.27:0/2490599761 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de000cd80 con 0x7f8df01039b0 2024-09-21T07:21:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.744+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df01a8850 con 0x7f8df01039b0 2024-09-21T07:21:27.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.744+0000 7f8de6a00640 1 -- 172.21.15.27:0/2490599761 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de00108e0 con 0x7f8df01039b0 2024-09-21T07:21:27.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.744+0000 7f8de6a00640 1 -- 172.21.15.27:0/2490599761 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0018980 con 0x7f8df01039b0 2024-09-21T07:21:27.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.744+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df01a8de0 con 0x7f8df01039b0 2024-09-21T07:21:27.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.746+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df0103dc0 con 0x7f8df01039b0 2024-09-21T07:21:27.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.748+0000 7f8de6a00640 1 -- 172.21.15.27:0/2490599761 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8de0010430 con 0x7f8df01039b0 2024-09-21T07:21:27.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.750+0000 7f8de6a00640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8dd40b6700 0x7f8dd40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:27.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.750+0000 7f8de6a00640 1 -- 172.21.15.27:0/2490599761 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f8de00e82d0 con 0x7f8df01039b0 2024-09-21T07:21:27.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.750+0000 7f8ded400640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8dd40b6700 0x7f8dd40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:27.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.751+0000 7f8ded400640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8dd40b6700 0x7f8dd40b8c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8df0102b40 tx=0x7f8ddc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:27.753+0000 7f8de6a00640 1 -- 172.21.15.27:0/2490599761 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8de00ae470 con 0x7f8df01039b0 2024-09-21T07:21:28.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.016+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8df0076a10 con 0x7f8df01039b0 2024-09-21T07:21:28.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.016+0000 7f8de6a00640 1 -- 172.21.15.27:0/2490599761 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8de00b2850 con 0x7f8df01039b0 2024-09-21T07:21:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:21:28.019 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:21:28.019 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.019+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8dd40b6700 msgr2=0x7f8dd40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.019+0000 7f8def200640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8dd40b6700 0x7f8dd40b8c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8df0102b40 tx=0x7f8ddc005f90 comp rx=0 tx=0).stop 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.019+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 msgr2=0x7f8df0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.019+0000 7f8def200640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 0x7f8df0075710 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e9f0 tx=0x7f8de000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 shutdown_connections 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8df01092f0 0x7f8df0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8dd40b6700 0x7f8dd40b8c10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df0109c40 0x7f8df0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 --2- 172.21.15.27:0/2490599761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df01039b0 0x7f8df0075710 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 >> 172.21.15.27:0/2490599761 conn(0x7f8df00ff440 msgr2=0x7f8df0108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:28.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 shutdown_connections 2024-09-21T07:21:28.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.020+0000 7f8def200640 1 -- 172.21.15.27:0/2490599761 wait complete. 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 -- 172.21.15.27:0/219986304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 msgr2=0x7fcef4110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 --2- 172.21.15.27:0/219986304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 0x7fcef4110640 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fceec009ca0 tx=0x7fceec02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 -- 172.21.15.27:0/219986304 shutdown_connections 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 --2- 172.21.15.27:0/219986304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef41094c0 0x7fcef41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 --2- 172.21.15.27:0/219986304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 0x7fcef4110640 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 --2- 172.21.15.27:0/219986304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef4103b80 0x7fcef4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 -- 172.21.15.27:0/219986304 >> 172.21.15.27:0/219986304 conn(0x7fcef40ff5d0 msgr2=0x7fcef4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 -- 172.21.15.27:0/219986304 shutdown_connections 2024-09-21T07:21:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.556+0000 7fcefc200640 1 -- 172.21.15.27:0/219986304 wait complete. 2024-09-21T07:21:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.557+0000 7fcefc200640 1 Processor -- start 2024-09-21T07:21:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefc200640 1 -- start start 2024-09-21T07:21:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefc200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef4103b80 0x7fcef419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefc200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef41094c0 0x7fcef419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefc200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 0x7fcef41a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefc200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4113ad0 con 0x7fcef4109e10 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefc200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4113950 con 0x7fcef41094c0 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefc200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4113c50 con 0x7fcef4103b80 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcefa200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 0x7fcef41a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.558+0000 7fcef9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef4103b80 0x7fcef419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef41094c0 0x7fcef419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef4103b80 0x7fcef419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51952/0 (socket says 172.21.15.27:51952) 2024-09-21T07:21:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 -- 172.21.15.27:0/1146627462 learned_addr learned my addr 172.21.15.27:0/1146627462 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 -- 172.21.15.27:0/1146627462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef41094c0 msgr2=0x7fcef419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef41094c0 0x7fcef419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 -- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 msgr2=0x7fcef41a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 0x7fcef41a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 -- 172.21.15.27:0/1146627462 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef41a2670 con 0x7fcef4103b80 2024-09-21T07:21:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef8e00640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef41094c0 0x7fcef419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.559+0000 7fcef9800640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef4103b80 0x7fcef419c700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e9f0 tx=0x7fcee800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.560+0000 7fcef2000640 1 -- 172.21.15.27:0/1146627462 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee800cd80 con 0x7fcef4103b80 2024-09-21T07:21:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.560+0000 7fcef2000640 1 -- 172.21.15.27:0/1146627462 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee80108e0 con 0x7fcef4103b80 2024-09-21T07:21:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.560+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef41a2960 con 0x7fcef4103b80 2024-09-21T07:21:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.560+0000 7fcef2000640 1 -- 172.21.15.27:0/1146627462 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8018980 con 0x7fcef4103b80 2024-09-21T07:21:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.560+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef41a90c0 con 0x7fcef4103b80 2024-09-21T07:21:28.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 Processor -- start 2024-09-21T07:21:28.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 -- start start 2024-09-21T07:21:28.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a89a2520 0x5614a8d491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d49700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5614a8cc0290 con 0x5614a8981840 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5614a8caf6c0 con 0x5614a89a2520 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.563+0000 7fcf02e85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614a8cb34d0 con 0x5614a8979660 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcefa200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a89a2520 0x5614a8d491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcefa200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a89a2520 0x5614a8d491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53270/0 (socket says 172.21.15.27:53270) 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcefa200640 1 -- 172.21.15.27:0/1475787536 learned_addr learned my addr 172.21.15.27:0/1475787536 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcef8e00640 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcef8e00640 1 -- 172.21.15.27:0/1475787536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 msgr2=0x5614a8d49700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcef8e00640 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d49700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcef8e00640 1 -- 172.21.15.27:0/1475787536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a89a2520 msgr2=0x5614a8d491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcef8e00640 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a89a2520 0x5614a8d491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcef8e00640 1 -- 172.21.15.27:0/1475787536 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a8c3c460 con 0x5614a8981840 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.564+0000 7fcefa200640 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a89a2520 0x5614a8d491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.565+0000 7fcef8e00640 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d49c40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcef406c780 tx=0x7fcee00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f05117b0f6baedc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.565+0000 7fcee6a00640 1 -- 172.21.15.27:0/1475787536 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0004020 con 0x5614a8981840 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.565+0000 7fcef2000640 1 -- 172.21.15.27:0/1146627462 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcee8010430 con 0x7fcef4103b80 2024-09-21T07:21:28.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.565+0000 7fcee6a00640 1 -- 172.21.15.27:0/1475787536 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee00054d0 con 0x5614a8981840 2024-09-21T07:21:28.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.565+0000 7fcee6a00640 1 -- 172.21.15.27:0/1475787536 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0004bf0 con 0x5614a8981840 2024-09-21T07:21:28.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 -- 172.21.15.27:0/1475787536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 msgr2=0x5614a8d49c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d49c40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcef406c780 tx=0x7fcee00118d0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 -- 172.21.15.27:0/1475787536 shutdown_connections 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d49700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d49c40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1475787536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a89a2520 0x5614a8d491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 -- 172.21.15.27:0/1475787536 >> 172.21.15.27:0/1475787536 conn(0x5614a8996910 msgr2=0x5614a8d470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 -- 172.21.15.27:0/1475787536 shutdown_connections 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 -- 172.21.15.27:0/1475787536 wait complete. 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.566+0000 7fcf02e85740 1 -- start start 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcf02e85740 1 Processor -- start 2024-09-21T07:21:28.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcef2000640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcedc0b6590 0x7fcedc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcef2000640 1 -- 172.21.15.27:0/1146627462 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fcee80e7e60 con 0x7fcef4103b80 2024-09-21T07:21:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8979660 0x5614a8d54de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcefa200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcedc0b6590 0x7fcedc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8981840 0x5614a8d55380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a89a2520 0x5614a8d60c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcf02e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5614a8b10bb0 con 0x5614a89a2520 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcf02e85740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5614a8c4a6c0 con 0x5614a8979660 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.567+0000 7fcf02e85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614a8d579a0 con 0x5614a8981840 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcf02e85740 10 client.0 _ll_register_callbacks cb 0x5614a8c65490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8979660 0x5614a8d54de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcefa200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a89a2520 0x5614a8d60c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8979660 0x5614a8d54de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53286/0 (socket says 172.21.15.27:53286) 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 -- 172.21.15.27:0/2355923279 learned_addr learned my addr 172.21.15.27:0/2355923279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef8e00640 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8981840 0x5614a8d55380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 -- 172.21.15.27:0/2355923279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8981840 msgr2=0x5614a8d55380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8981840 0x5614a8d55380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 -- 172.21.15.27:0/2355923279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a89a2520 msgr2=0x5614a8d60c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a89a2520 0x5614a8d60c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcef9800640 1 -- 172.21.15.27:0/2355923279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a8a0db00 con 0x5614a8979660 2024-09-21T07:21:28.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.568+0000 7fcefa200640 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a89a2520 0x5614a8d60c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcefa200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcedc0b6590 0x7fcedc0b8aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fceec005e30 tx=0x7fceec031bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcef9800640 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8979660 0x5614a8d54de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcee80ad650 tx=0x7fcee80ad360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57570748c80e67f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcedb400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcedb400640 1 -- 172.21.15.27:0/2355923279 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee80aedd0 con 0x5614a8979660 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee00054d0 con 0x5614a8979660 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcedb400640 1 -- 172.21.15.27:0/2355923279 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fcee8018b20 con 0x5614a8979660 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcedb400640 1 -- 172.21.15.27:0/2355923279 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcee80b2170 con 0x5614a8979660 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcf02e85740 10 client.27950 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614a8a84960 con 0x5614a8979660 2024-09-21T07:21:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.569+0000 7fcedb400640 1 -- 172.21.15.27:0/2355923279 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee00054d0 con 0x5614a8979660 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcedb400640 1 -- 172.21.15.27:0/2355923279 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcee80ac620 con 0x5614a8979660 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcf02e85740 10 client.27950 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcf02e85740 10 client.27950 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcf02e85740 20 client.27950 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcf02e85740 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a8960ea0 0x5614a8d66d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcf02e85740 4 client.27950 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614a8cc21e0 con 0x5614a8960ea0 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.570+0000 7fcef8e00640 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a8960ea0 0x5614a8d66d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.571+0000 7fcef8e00640 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a8960ea0 0x5614a8d66d40 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7133c63c3769bdb4 server_cookie=81f16088942e11ca in_seq=0 out_seq=0 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.571+0000 7fcedb400640 10 client.27950 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcedb400640 1 -- 172.21.15.27:0/2355923279 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614a8cc21e0 con 0x5614a8960ea0 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcedb400640 10 client.27950 handle_command_reply: tid=0 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 client.27950 shutdown 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8979660 msgr2=0x5614a8d54de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8979660 0x5614a8d54de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcee80ad650 tx=0x7fcee80ad360 comp rx=0 tx=0).stop 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 shutdown_connections 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a8960ea0 0x5614a8d66d40 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8981840 0x5614a8d55380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a89a2520 0x5614a8d60c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 --2- 172.21.15.27:0/2355923279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8979660 0x5614a8d54de0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcedb400640 0 client.27950 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 >> 172.21.15.27:0/2355923279 conn(0x5614a8996910 msgr2=0x5614a8d4c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 shutdown_connections 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.572+0000 7fcf02e85740 1 -- 172.21.15.27:0/2355923279 wait complete. 2024-09-21T07:21:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.573+0000 7fcf02e85740 20 client.27950 trim_cache size 0 max 16384 2024-09-21T07:21:28.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 Processor -- start 2024-09-21T07:21:28.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 -- start start 2024-09-21T07:21:28.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8996910 0x5614a8d50390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d60b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8981840 0x5614a8d56ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b59d0 con 0x5614a8996910 2024-09-21T07:21:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b32a0 con 0x5614a8981840 2024-09-21T07:21:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.591+0000 7fcf02e85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b1110 con 0x5614a8979660 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8996910 0x5614a8d50390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d60b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d60b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51964/0 (socket says 172.21.15.27:51964) 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8996910 0x5614a8d50390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38994/0 (socket says 172.21.15.27:38994) 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 -- 172.21.15.27:0/3674872294 learned_addr learned my addr 172.21.15.27:0/3674872294 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcefa200640 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8981840 0x5614a8d56ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 -- 172.21.15.27:0/3674872294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8981840 msgr2=0x5614a8d56ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8981840 0x5614a8d56ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 -- 172.21.15.27:0/3674872294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8996910 msgr2=0x5614a8d50390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8996910 0x5614a8d50390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 -- 172.21.15.27:0/3674872294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a8d4c020 con 0x5614a8979660 2024-09-21T07:21:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.592+0000 7fcef8e00640 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d60b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009d40 tx=0x7fcee0005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4518bd2a21f4799d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:28.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.593+0000 7fcee6000640 1 -- 172.21.15.27:0/3674872294 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee001d040 con 0x5614a8979660 2024-09-21T07:21:28.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.593+0000 7fcee6000640 1 -- 172.21.15.27:0/3674872294 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee001e040 con 0x5614a8979660 2024-09-21T07:21:28.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.593+0000 7fcee6000640 1 -- 172.21.15.27:0/3674872294 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee00035f0 con 0x5614a8979660 2024-09-21T07:21:28.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.593+0000 7fcf02e85740 1 -- 172.21.15.27:0/3674872294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 msgr2=0x5614a8d60b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.593+0000 7fcf02e85740 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d60b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009d40 tx=0x7fcee0005d30 comp rx=0 tx=0).stop 2024-09-21T07:21:28.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 -- 172.21.15.27:0/3674872294 shutdown_connections 2024-09-21T07:21:28.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d60b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8981840 0x5614a8d56ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 --2- 172.21.15.27:0/3674872294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8996910 0x5614a8d50390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 -- 172.21.15.27:0/3674872294 >> 172.21.15.27:0/3674872294 conn(0x5614a8d4f5d0 msgr2=0x5614a8d4c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 -- 172.21.15.27:0/3674872294 shutdown_connections 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 -- 172.21.15.27:0/3674872294 wait complete. 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 -- start start 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.594+0000 7fcf02e85740 1 Processor -- start 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d5ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcf02e85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8996910 0x5614a8d57bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcf02e85740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b1760 con 0x5614a8981840 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcf02e85740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b2bd0 con 0x5614a8996910 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcf02e85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b1e30 con 0x5614a8979660 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcf02e85740 10 client.0 _ll_register_callbacks cb 0x7fcee8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcef9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d5ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcef9800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d5ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51966/0 (socket says 172.21.15.27:51966) 2024-09-21T07:21:28.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcef9800640 1 -- 172.21.15.27:0/1478818001 learned_addr learned my addr 172.21.15.27:0/1478818001 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:28.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcefa200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8996910 0x5614a8d57bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.595+0000 7fcef8e00640 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.596+0000 7fcefa200640 1 -- 172.21.15.27:0/1478818001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 msgr2=0x5614a8d5ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:28.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.596+0000 7fcefa200640 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d5ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.596+0000 7fcefa200640 1 -- 172.21.15.27:0/1478818001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 msgr2=0x5614a8d5d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:28.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.596+0000 7fcefa200640 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d5d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.596+0000 7fcefa200640 1 -- 172.21.15.27:0/1478818001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a8d58360 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.596+0000 7fcefa200640 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8996910 0x5614a8d57bd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fceec035000 tx=0x7fceec0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a351ffeaababae00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.596+0000 7fcedb400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.597+0000 7fcedb400640 1 -- 172.21.15.27:0/1478818001 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec05a060 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.597+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee001e040 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.597+0000 7fcedb400640 1 -- 172.21.15.27:0/1478818001 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fceec045050 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.597+0000 7fcedb400640 1 -- 172.21.15.27:0/1478818001 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fceec061050 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.597+0000 7fcf02e85740 10 client.27956 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.597+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614a8dd21e0 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.597+0000 7fcedb400640 1 -- 172.21.15.27:0/1478818001 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee001e040 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcedb400640 1 -- 172.21.15.27:0/1478818001 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fceec02fbc0 con 0x5614a8996910 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcf02e85740 10 client.27956 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcf02e85740 10 client.27956 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:21:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcf02e85740 20 client.27956 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a89a2520 0x5614a8dd49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcf02e85740 4 client.27956 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9623a0a1", "release": true, "await": true}] 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9623a0a1", "release": true, "await": true}) -- 0x5614a8cbdc90 con 0x5614a89a2520 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.598+0000 7fcef9800640 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a89a2520 0x5614a8dd49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.599+0000 7fcef9800640 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a89a2520 0x5614a8dd49b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f451177c214233 server_cookie=38bc65a4ce798e31 in_seq=0 out_seq=0 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.599+0000 7fcedb400640 10 client.27956 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcedb400640 1 -- 172.21.15.27:0/1478818001 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5614a8cbdc90 con 0x5614a89a2520 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcedb400640 10 client.27956 handle_command_reply: tid=0 2024-09-21T07:21:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 client.27956 shutdown 2024-09-21T07:21:28.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8996910 msgr2=0x5614a8d57bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8996910 0x5614a8d57bd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fceec035000 tx=0x7fceec0095c0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 shutdown_connections 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5614a89a2520 0x5614a8dd49b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614a8996910 0x5614a8d57bd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5614a8981840 0x5614a8d5d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcedb400640 0 client.27956 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 --2- 172.21.15.27:0/1478818001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614a8979660 0x5614a8d5ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 >> 172.21.15.27:0/1478818001 conn(0x5614a8d4f5d0 msgr2=0x5614a8d56820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 shutdown_connections 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.601+0000 7fcf02e85740 1 -- 172.21.15.27:0/1478818001 wait complete. 2024-09-21T07:21:28.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.602+0000 7fcf02e85740 20 client.27956 trim_cache size 0 max 16384 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 36, 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "9623a0a1": { 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 36, 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:21:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.1, 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:28.606 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.602+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcedc0b6590 msgr2=0x7fcedc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.602+0000 7fcefc200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcedc0b6590 0x7fcedc0b8aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fceec005e30 tx=0x7fceec031bd0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.602+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef4103b80 msgr2=0x7fcef419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.602+0000 7fcefc200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef4103b80 0x7fcef419c700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e9f0 tx=0x7fcee800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 shutdown_connections 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcedc0b6590 0x7fcedc0b8aa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcef4109e10 0x7fcef41a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcef41094c0 0x7fcef419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 --2- 172.21.15.27:0/1146627462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcef4103b80 0x7fcef419c700 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 >> 172.21.15.27:0/1146627462 conn(0x7fcef40ff5d0 msgr2=0x7fcef410e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 shutdown_connections 2024-09-21T07:21:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:28.603+0000 7fcefc200640 1 -- 172.21.15.27:0/1146627462 wait complete. 2024-09-21T07:21:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:28 smithi119 ceph-mon[34519]: pgmap v637: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:21:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:28 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2490599761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:28 smithi027 ceph-mon[30779]: pgmap v637: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:21:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:28 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2490599761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:28 smithi153 ceph-mon[34747]: pgmap v637: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:21:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:28 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2490599761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:29.571 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9623a0a1', total seconds elapsed: 26.9 2024-09-21T07:21:29.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:21:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:30 smithi119 ceph-mon[34519]: pgmap v638: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-21T07:21:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 -- 172.21.15.27:0/1930460975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f08103b60 msgr2=0x7f0f08103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/1930460975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f08103b60 0x7f0f08103f70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009900 tx=0x7f0ef802a410 comp rx=0 tx=0).stop 2024-09-21T07:21:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 -- 172.21.15.27:0/1930460975 shutdown_connections 2024-09-21T07:21:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/1930460975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f081094a0 0x7f0f081098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/1930460975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f08109df0 0x7f0f08110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/1930460975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f08103b60 0x7f0f08103f70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 -- 172.21.15.27:0/1930460975 >> 172.21.15.27:0/1930460975 conn(0x7f0f0807b850 msgr2=0x7f0f0807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 -- 172.21.15.27:0/1930460975 shutdown_connections 2024-09-21T07:21:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.056+0000 7f0f0ee00640 1 -- 172.21.15.27:0/1930460975 wait complete. 2024-09-21T07:21:31.059 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:30 smithi027 ceph-mon[30779]: pgmap v638: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-21T07:21:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.058+0000 7f0f0ee00640 1 Processor -- start 2024-09-21T07:21:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.058+0000 7f0f0ee00640 1 -- start start 2024-09-21T07:21:31.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f0ee00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f08103b60 0x7f0f0819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f0ee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f081094a0 0x7f0f0819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f0ee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f08109df0 0x7f0f0819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f0ee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08113ab0 con 0x7f0f081094a0 2024-09-21T07:21:31.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f0ee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08113930 con 0x7f0f08109df0 2024-09-21T07:21:31.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f0ee00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08113c30 con 0x7f0f08103b60 2024-09-21T07:21:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f08103b60 0x7f0f0819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0eff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f081094a0 0x7f0f0819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0eff400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f081094a0 0x7f0f0819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39020/0 (socket says 172.21.15.27:39020) 2024-09-21T07:21:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0eff400640 1 -- 172.21.15.27:0/904817487 learned_addr learned my addr 172.21.15.27:0/904817487 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.059+0000 7f0f0ce00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f08109df0 0x7f0f0819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f07e00640 1 -- 172.21.15.27:0/904817487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f08109df0 msgr2=0x7f0f0819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f07e00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f08109df0 0x7f0f0819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f07e00640 1 -- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f081094a0 msgr2=0x7f0f0819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f07e00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f081094a0 0x7f0f0819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f07e00640 1 -- 172.21.15.27:0/904817487 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f081a0410 con 0x7f0f08103b60 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0eff400640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f081094a0 0x7f0f0819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f07e00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f08103b60 0x7f0f0819a4a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006010 tx=0x7f0ef8004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f05600640 1 -- 172.21.15.27:0/904817487 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8035050 con 0x7f0f08103b60 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f05600640 1 -- 172.21.15.27:0/904817487 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef80044a0 con 0x7f0f08103b60 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f081a06a0 con 0x7f0f08103b60 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f05600640 1 -- 172.21.15.27:0/904817487 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8003c10 con 0x7f0f08103b60 2024-09-21T07:21:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.060+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f080755f0 con 0x7f0f08103b60 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 Processor -- start 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 -- start start 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16bd070 0x5651d1a63da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16941b0 0x5651d1a642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a64820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f05600640 1 -- 172.21.15.27:0/904817487 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0ef803a670 con 0x7f0f08103b60 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5651d19dafb0 con 0x5651d16bd070 2024-09-21T07:21:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5651d19ca3e0 con 0x5651d169c390 2024-09-21T07:21:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.064+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5651d19ce1f0 con 0x5651d16941b0 2024-09-21T07:21:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0f0ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16bd070 0x5651d1a63da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16941b0 0x5651d1a642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0eff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a64820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0f0ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16bd070 0x5651d1a63da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39028/0 (socket says 172.21.15.27:39028) 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0f0ce00640 1 -- 172.21.15.27:0/1503740176 learned_addr learned my addr 172.21.15.27:0/1503740176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0eff400640 1 -- 172.21.15.27:0/1503740176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16941b0 msgr2=0x5651d1a642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0eff400640 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16941b0 0x5651d1a642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0eff400640 1 -- 172.21.15.27:0/1503740176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16bd070 msgr2=0x5651d1a63da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0eff400640 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16bd070 0x5651d1a63da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0eff400640 1 -- 172.21.15.27:0/1503740176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d1994cd0 con 0x5651d169c390 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.065+0000 7f0f07e00640 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16941b0 0x5651d1a642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:31.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.066+0000 7f0eff400640 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a64820 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005d00 tx=0x7f0ef00108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eadbfce27623971 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:31.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.066+0000 7f0efd600640 1 -- 172.21.15.27:0/1503740176 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0011820 con 0x5651d169c390 2024-09-21T07:21:31.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.066+0000 7f0efd600640 1 -- 172.21.15.27:0/1503740176 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef001c480 con 0x5651d169c390 2024-09-21T07:21:31.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.066+0000 7f0efd600640 1 -- 172.21.15.27:0/1503740176 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef001a5e0 con 0x5651d169c390 2024-09-21T07:21:31.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.066+0000 7f0f05600640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0eec0b6680 0x7f0eec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.066+0000 7f0f05600640 1 -- 172.21.15.27:0/904817487 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0ef810b820 con 0x7f0f08103b60 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f0ce00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0eec0b6680 0x7f0eec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1503740176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 msgr2=0x5651d1a64820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a64820 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005d00 tx=0x7f0ef00108e0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1503740176 shutdown_connections 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16941b0 0x5651d1a642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a64820 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1503740176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16bd070 0x5651d1a63da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1503740176 >> 172.21.15.27:0/1503740176 conn(0x5651d16b1460 msgr2=0x5651d1a61c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f0ce00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0eec0b6680 0x7f0eec0b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0f000116b0 tx=0x7f0f00009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1503740176 shutdown_connections 2024-09-21T07:21:31.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.067+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1503740176 wait complete. 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.068+0000 7f0f15a5c740 1 -- start start 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.068+0000 7f0f15a5c740 1 Processor -- start 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5651d182b700 con 0x5651d16941b0 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5651d1965580 con 0x5651d169c390 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5651d1a72580 con 0x5651d16bd070 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f15a5c740 10 client.0 _ll_register_callbacks cb 0x5651d1975470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39040/0 (socket says 172.21.15.27:39040) 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f07e00640 1 -- 172.21.15.27:0/775728615 learned_addr learned my addr 172.21.15.27:0/775728615 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:31.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0eff400640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a6fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.069+0000 7f0f0ce00640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0f07e00640 1 -- 172.21.15.27:0/775728615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 msgr2=0x5651d1a7b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0f07e00640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a7b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0f07e00640 1 -- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 msgr2=0x5651d1a6fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0f07e00640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a6fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0f07e00640 1 -- 172.21.15.27:0/775728615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d1728650 con 0x5651d16941b0 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0eff400640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0f07e00640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6f4f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8108bc0 tx=0x7f0ef80cd8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375734fc9f53428d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:31.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0ef6000640 1 -- 172.21.15.27:0/775728615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80d0bb0 con 0x5651d16941b0 2024-09-21T07:21:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0ef6000640 1 -- 172.21.15.27:0/775728615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0ef80da7f0 con 0x5651d16941b0 2024-09-21T07:21:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.070+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651d1967b90 con 0x5651d16941b0 2024-09-21T07:21:31.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0ef6000640 1 -- 172.21.15.27:0/775728615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ef8108ff0 con 0x5651d16941b0 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 10 client.17937 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651d1982950 con 0x5651d16941b0 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0ef6000640 1 -- 172.21.15.27:0/775728615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5651d1967b90 con 0x5651d16941b0 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0ef6000640 1 -- 172.21.15.27:0/775728615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ef810b3c0 con 0x5651d16941b0 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 10 client.17937 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 10 client.17937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 20 client.17937 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d167b9f0 0x5651d1a816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 4 client.17937 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.071+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651d19dcf00 con 0x5651d167b9f0 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.072+0000 7f0eff400640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d167b9f0 0x5651d1a816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.072+0000 7f0eff400640 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d167b9f0 0x5651d1a816b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8b9e780a7370b07 server_cookie=be75bd6f4dfba149 in_seq=0 out_seq=0 2024-09-21T07:21:31.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.072+0000 7f0ef6000640 10 client.17937 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.073+0000 7f0ef6000640 1 -- 172.21.15.27:0/775728615 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651d19dcf00 con 0x5651d167b9f0 2024-09-21T07:21:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.073+0000 7f0ef6000640 10 client.17937 handle_command_reply: tid=0 2024-09-21T07:21:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.073+0000 7f0f15a5c740 1 client.17937 shutdown 2024-09-21T07:21:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 msgr2=0x5651d1a6f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6f4f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8108bc0 tx=0x7f0ef80cd8b0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 shutdown_connections 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d167b9f0 0x5651d1a816b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a7b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/775728615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6f4f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0ef6000640 0 client.17937 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 >> 172.21.15.27:0/775728615 conn(0x5651d16b1460 msgr2=0x5651d1a66f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 shutdown_connections 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 1 -- 172.21.15.27:0/775728615 wait complete. 2024-09-21T07:21:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.074+0000 7f0f15a5c740 20 client.17937 trim_cache size 0 max 16384 2024-09-21T07:21:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 Processor -- start 2024-09-21T07:21:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 -- start start 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d169c390 0x5651d1a6cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d16bd070 0x5651d1a86e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081b1590 con 0x5651d16941b0 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081aee60 con 0x5651d16bd070 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.088+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081accd0 con 0x5651d169c390 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39056/0 (socket says 172.21.15.27:39056) 2024-09-21T07:21:31.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0f0ce00640 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d16bd070 0x5651d1a86e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0f07e00640 1 -- 172.21.15.27:0/2144294195 learned_addr learned my addr 172.21.15.27:0/2144294195 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:31.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0eff400640 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d169c390 0x5651d1a6cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0eff400640 1 -- 172.21.15.27:0/2144294195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d16bd070 msgr2=0x5651d1a86e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0eff400640 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d16bd070 0x5651d1a86e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0eff400640 1 -- 172.21.15.27:0/2144294195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 msgr2=0x5651d1a6c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0eff400640 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.089+0000 7f0eff400640 1 -- 172.21.15.27:0/2144294195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d1a62580 con 0x5651d169c390 2024-09-21T07:21:31.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.090+0000 7f0eff400640 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d169c390 0x5651d1a6cc20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0ef001cdf0 tx=0x7f0ef000bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1378f9be08c4c08c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:31.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.090+0000 7f0efcc00640 1 -- 172.21.15.27:0/2144294195 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef001c820 con 0x5651d169c390 2024-09-21T07:21:31.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.090+0000 7f0efcc00640 1 -- 172.21.15.27:0/2144294195 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef001c9c0 con 0x5651d169c390 2024-09-21T07:21:31.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.090+0000 7f0efcc00640 1 -- 172.21.15.27:0/2144294195 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0011a40 con 0x5651d169c390 2024-09-21T07:21:31.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 -- 172.21.15.27:0/2144294195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d169c390 msgr2=0x5651d1a6cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d169c390 0x5651d1a6cc20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0ef001cdf0 tx=0x7f0ef000bd50 comp rx=0 tx=0).stop 2024-09-21T07:21:31.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 -- 172.21.15.27:0/2144294195 shutdown_connections 2024-09-21T07:21:31.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d169c390 0x5651d1a6cc20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d16bd070 0x5651d1a86e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/2144294195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a6c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 -- 172.21.15.27:0/2144294195 >> 172.21.15.27:0/2144294195 conn(0x5651d16b1460 msgr2=0x5651d1a6dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 -- 172.21.15.27:0/2144294195 shutdown_connections 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.091+0000 7f0f15a5c740 1 -- 172.21.15.27:0/2144294195 wait complete. 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 -- start start 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 Processor -- start 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a77ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ad320 con 0x5651d16941b0 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ae790 con 0x5651d169c390 2024-09-21T07:21:31.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.092+0000 7f0f15a5c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ad9f0 con 0x5651d16bd070 2024-09-21T07:21:31.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0f15a5c740 10 client.0 _ll_register_callbacks cb 0x7f0ef8111220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:31.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a77ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0eff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a77ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39064/0 (socket says 172.21.15.27:39064) 2024-09-21T07:21:31.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0f07e00640 1 -- 172.21.15.27:0/1106898763 learned_addr learned my addr 172.21.15.27:0/1106898763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0f0ce00640 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0eff400640 1 -- 172.21.15.27:0/1106898763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 msgr2=0x5651d1a749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0eff400640 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0eff400640 1 -- 172.21.15.27:0/1106898763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 msgr2=0x5651d1a77ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0eff400640 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a77ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0eff400640 1 -- 172.21.15.27:0/1106898763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d1a75180 con 0x5651d169c390 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.093+0000 7f0f07e00640 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0eff400640 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a744b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0010920 tx=0x7f0ef000bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe9d76682d3722b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0ef6000640 1 -- 172.21.15.27:0/1106898763 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef001c820 con 0x5651d169c390 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0ef6000640 1 -- 172.21.15.27:0/1106898763 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0ef0005110 con 0x5651d169c390 2024-09-21T07:21:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef001c9c0 con 0x5651d169c390 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0ef6000640 1 -- 172.21.15.27:0/1106898763 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ef0040050 con 0x5651d169c390 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0f15a5c740 10 client.27974 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.094+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651d1aecfd0 con 0x5651d169c390 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0ef6000640 1 -- 172.21.15.27:0/1106898763 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef001c9c0 con 0x5651d169c390 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0ef6000640 1 -- 172.21.15.27:0/1106898763 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ef00035f0 con 0x5651d169c390 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0f15a5c740 10 client.27974 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0f15a5c740 10 client.27974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0f15a5c740 20 client.27974 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d1650da0 0x5651d1aef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0f15a5c740 4 client.27974 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651d19d89b0 con 0x5651d1650da0 2024-09-21T07:21:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.095+0000 7f0f07e00640 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d1650da0 0x5651d1aef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:31.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0f07e00640 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d1650da0 0x5651d1aef500 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9c5a46a16473482 server_cookie=da914c7e882848c0 in_seq=0 out_seq=0 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0ef6000640 10 client.27974 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0ef6000640 1 -- 172.21.15.27:0/1106898763 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651d19d89b0 con 0x5651d1650da0 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0ef6000640 10 client.27974 handle_command_reply: tid=0 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0f15a5c740 1 client.27974 shutdown 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 msgr2=0x5651d1a744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a744b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0010920 tx=0x7f0ef000bf80 comp rx=0 tx=0).stop 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 shutdown_connections 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5651d1650da0 0x5651d1aef500 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.096+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651d16bd070 0x5651d1a749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651d169c390 0x5651d1a744b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0ef6000640 0 client.27974 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f15a5c740 1 --2- 172.21.15.27:0/1106898763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5651d16941b0 0x5651d1a77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 >> 172.21.15.27:0/1106898763 conn(0x5651d16b1460 msgr2=0x5651d1a64d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 shutdown_connections 2024-09-21T07:21:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f15a5c740 1 -- 172.21.15.27:0/1106898763 wait complete. 2024-09-21T07:21:31.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f15a5c740 20 client.27974 trim_cache size 0 max 16384 2024-09-21T07:21:31.102 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:21:31.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0eec0b6680 msgr2=0x7f0eec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0eec0b6680 0x7f0eec0b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0f000116b0 tx=0x7f0f00009500 comp rx=0 tx=0).stop 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f08103b60 msgr2=0x7f0f0819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.097+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f08103b60 0x7f0f0819a4a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006010 tx=0x7f0ef8004060 comp rx=0 tx=0).stop 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 shutdown_connections 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0eec0b6680 0x7f0eec0b8b90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f08109df0 0x7f0f0819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f081094a0 0x7f0f0819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 --2- 172.21.15.27:0/904817487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f08103b60 0x7f0f0819a4a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 >> 172.21.15.27:0/904817487 conn(0x7f0f0807b850 msgr2=0x7f0f08107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 shutdown_connections 2024-09-21T07:21:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:31.098+0000 7f0f0ee00640 1 -- 172.21.15.27:0/904817487 wait complete. 2024-09-21T07:21:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:30 smithi153 ceph-mon[34747]: pgmap v638: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-21T07:21:31.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:21:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:32.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 -- 172.21.15.27:0/3877337041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc10be50 msgr2=0x7fd1cc112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 --2- 172.21.15.27:0/3877337041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc10be50 0x7fd1cc112680 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009ca0 tx=0x7fd1c402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 -- 172.21.15.27:0/3877337041 shutdown_connections 2024-09-21T07:21:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 --2- 172.21.15.27:0/3877337041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc10be50 0x7fd1cc112680 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 --2- 172.21.15.27:0/3877337041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1cc10b500 0x7fd1cc10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 --2- 172.21.15.27:0/3877337041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1cc105bc0 0x7fd1cc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 -- 172.21.15.27:0/3877337041 >> 172.21.15.27:0/3877337041 conn(0x7fd1cc101670 msgr2=0x7fd1cc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 -- 172.21.15.27:0/3877337041 shutdown_connections 2024-09-21T07:21:32.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.809+0000 7fd1cb200640 1 -- 172.21.15.27:0/3877337041 wait complete. 2024-09-21T07:21:32.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 Processor -- start 2024-09-21T07:21:32.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 -- start start 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc105bc0 0x7fd1cc19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1cc10b500 0x7fd1cc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1cc10be50 0x7fd1cc1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc115b10 con 0x7fd1cc105bc0 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc115990 con 0x7fd1cc10be50 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.811+0000 7fd1cb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc115c90 con 0x7fd1cc10b500 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc105bc0 0x7fd1cc19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc105bc0 0x7fd1cc19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39086/0 (socket says 172.21.15.27:39086) 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9e00640 1 -- 172.21.15.27:0/169005205 learned_addr learned my addr 172.21.15.27:0/169005205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9400640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1cc10b500 0x7fd1cc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1ca800640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1cc10be50 0x7fd1cc1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9400640 1 -- 172.21.15.27:0/169005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1cc10be50 msgr2=0x7fd1cc1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9400640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1cc10be50 0x7fd1cc1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9400640 1 -- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc105bc0 msgr2=0x7fd1cc19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9400640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc105bc0 0x7fd1cc19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9400640 1 -- 172.21.15.27:0/169005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc1a2410 con 0x7fd1cc10b500 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.812+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc105bc0 0x7fd1cc19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.813+0000 7fd1c9400640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1cc10b500 0x7fd1cc19ca40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc06c8a0 tx=0x7fd1b800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.813+0000 7fd1c2a00640 1 -- 172.21.15.27:0/169005205 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800cd00 con 0x7fd1cc10b500 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.813+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc1a2700 con 0x7fd1cc10b500 2024-09-21T07:21:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.813+0000 7fd1c2a00640 1 -- 172.21.15.27:0/169005205 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b80035f0 con 0x7fd1cc10b500 2024-09-21T07:21:32.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.813+0000 7fd1c2a00640 1 -- 172.21.15.27:0/169005205 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8010690 con 0x7fd1cc10b500 2024-09-21T07:21:32.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.813+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a8e60 con 0x7fd1cc10b500 2024-09-21T07:21:32.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.817+0000 7fd1d7dfb740 1 Processor -- start 2024-09-21T07:21:32.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.818+0000 7fd1d7dfb740 1 -- start start 2024-09-21T07:21:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.818+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350f0070 0x562635496f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.818+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350c71b0 0x5626354974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.818+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350cf390 0x5626354979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.818+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56263540dfb0 con 0x5626350cf390 2024-09-21T07:21:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.818+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5626353fd3e0 con 0x5626350f0070 2024-09-21T07:21:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.818+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5626354011f0 con 0x5626350c71b0 2024-09-21T07:21:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350c71b0 0x5626354974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1ca800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350f0070 0x562635496f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350c71b0 0x5626354974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52062/0 (socket says 172.21.15.27:52062) 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1685397700 learned_addr learned my addr 172.21.15.27:0/1685397700 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c2a00640 1 -- 172.21.15.27:0/169005205 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd1b80037a0 con 0x7fd1cc10b500 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1685397700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350f0070 msgr2=0x562635496f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350f0070 0x562635496f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1685397700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350cf390 msgr2=0x5626354979f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9400640 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350cf390 0x5626354979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350cf390 0x5626354979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1685397700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626353c7cd0 con 0x5626350c71b0 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.819+0000 7fd1c9400640 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350cf390 0x5626354979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:21:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.820+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350c71b0 0x5626354974b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc011630 tx=0x7fd1bc011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8e1a20ee729502b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:32.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.820+0000 7fd1b7400640 1 -- 172.21.15.27:0/1685397700 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc01e070 con 0x5626350c71b0 2024-09-21T07:21:32.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.820+0000 7fd1b7400640 1 -- 172.21.15.27:0/1685397700 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc00fba0 con 0x5626350c71b0 2024-09-21T07:21:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.820+0000 7fd1b7400640 1 -- 172.21.15.27:0/1685397700 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc019c60 con 0x5626350c71b0 2024-09-21T07:21:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1685397700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350c71b0 msgr2=0x5626354974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1c2a00640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd1b00b6680 0x7fd1b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350c71b0 0x5626354974b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc011630 tx=0x7fd1bc011af0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1685397700 shutdown_connections 2024-09-21T07:21:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350c71b0 0x5626354974b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350cf390 0x5626354979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1685397700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350f0070 0x562635496f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1685397700 >> 172.21.15.27:0/1685397700 conn(0x5626350e4460 msgr2=0x562635494e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1c2a00640 1 -- 172.21.15.27:0/169005205 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fd1b80e7d20 con 0x7fd1cc10b500 2024-09-21T07:21:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1ca800640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd1b00b6680 0x7fd1b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1685397700 shutdown_connections 2024-09-21T07:21:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.821+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1685397700 wait complete. 2024-09-21T07:21:32.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.822+0000 7fd1ca800640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd1b00b6680 0x7fd1b00b8b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4005cc0 tx=0x7fd1c4005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.822+0000 7fd1d7dfb740 1 -- start start 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.822+0000 7fd1d7dfb740 1 Processor -- start 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354a2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56263525e700 con 0x5626350f0070 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562635398580 con 0x5626350c71b0 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5626354a5750 con 0x5626350cf390 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1d7dfb740 10 client.0 _ll_register_callbacks cb 0x5626353a8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1ca800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1ca800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39106/0 (socket says 172.21.15.27:39106) 2024-09-21T07:21:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1ca800640 1 -- 172.21.15.27:0/3917318937 learned_addr learned my addr 172.21.15.27:0/3917318937 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.823+0000 7fd1c9400640 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.824+0000 7fd1c9e00640 1 -- 172.21.15.27:0/3917318937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 msgr2=0x5626354a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.824+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.824+0000 7fd1c9e00640 1 -- 172.21.15.27:0/3917318937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 msgr2=0x5626354aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.824+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.824+0000 7fd1c9e00640 1 -- 172.21.15.27:0/3917318937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56263515b650 con 0x5626350c71b0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.824+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354a2700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc01c040 tx=0x7fd1bc009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d380edc21d18e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.824+0000 7fd1abe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.825+0000 7fd1abe00640 1 -- 172.21.15.27:0/3917318937 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc026020 con 0x5626350c71b0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.825+0000 7fd1abe00640 1 -- 172.21.15.27:0/3917318937 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fd1bc005590 con 0x5626350c71b0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.825+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56263539ab90 con 0x5626350c71b0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.825+0000 7fd1abe00640 1 -- 172.21.15.27:0/3917318937 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc031050 con 0x5626350c71b0 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.825+0000 7fd1d7dfb740 10 client.27986 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.825+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626353b5950 con 0x5626350c71b0 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.825+0000 7fd1abe00640 1 -- 172.21.15.27:0/3917318937 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56263539ab90 con 0x5626350c71b0 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1abe00640 1 -- 172.21.15.27:0/3917318937 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd1bc021050 con 0x5626350c71b0 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1d7dfb740 10 client.27986 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1d7dfb740 10 client.27986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1d7dfb740 20 client.27986 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5626350ae9f0 0x5626354b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1d7dfb740 4 client.27986 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56263540ff00 con 0x5626350ae9f0 2024-09-21T07:21:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.826+0000 7fd1c9400640 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5626350ae9f0 0x5626354b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.827+0000 7fd1c9400640 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5626350ae9f0 0x5626354b46e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=595517305d7ab9f server_cookie=ed74f2dc7c18cef2 in_seq=0 out_seq=0 2024-09-21T07:21:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.827+0000 7fd1abe00640 10 client.27986 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.827+0000 7fd1abe00640 1 -- 172.21.15.27:0/3917318937 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56263540ff00 con 0x5626350ae9f0 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.827+0000 7fd1abe00640 10 client.27986 handle_command_reply: tid=0 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.827+0000 7fd1d7dfb740 1 client.27986 shutdown 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 msgr2=0x5626354a2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354a2700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc01c040 tx=0x7fd1bc009f10 comp rx=0 tx=0).stop 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 shutdown_connections 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5626350ae9f0 0x5626354b46e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1abe00640 0 client.27986 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/3917318937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354a2700 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 >> 172.21.15.27:0/3917318937 conn(0x5626350e4460 msgr2=0x56263549a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 shutdown_connections 2024-09-21T07:21:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/3917318937 wait complete. 2024-09-21T07:21:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.828+0000 7fd1d7dfb740 20 client.27986 trim_cache size 0 max 16384 2024-09-21T07:21:32.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.843+0000 7fd1d7dfb740 1 Processor -- start 2024-09-21T07:21:32.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.843+0000 7fd1d7dfb740 1 -- start start 2024-09-21T07:21:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350c71b0 0x56263549dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350cf390 0x56263549f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350f0070 0x56263549fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b5810 con 0x5626350c71b0 2024-09-21T07:21:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b30e0 con 0x5626350cf390 2024-09-21T07:21:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b0f50 con 0x5626350f0070 2024-09-21T07:21:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350c71b0 0x56263549dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350c71b0 0x56263549dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39118/0 (socket says 172.21.15.27:39118) 2024-09-21T07:21:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1c9e00640 1 -- 172.21.15.27:0/2621930493 learned_addr learned my addr 172.21.15.27:0/2621930493 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1c9400640 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350cf390 0x56263549f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.844+0000 7fd1ca800640 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350f0070 0x56263549fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1c9400640 1 -- 172.21.15.27:0/2621930493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350f0070 msgr2=0x56263549fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1c9400640 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350f0070 0x56263549fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1c9400640 1 -- 172.21.15.27:0/2621930493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350c71b0 msgr2=0x56263549dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1c9400640 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350c71b0 0x56263549dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1c9400640 1 -- 172.21.15.27:0/2621930493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626354af1f0 con 0x5626350cf390 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350c71b0 0x56263549dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1c9400640 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350cf390 0x56263549f8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80e50c0 tx=0x7fd1b80ea230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff1f3936b974255 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1b6a00640 1 -- 172.21.15.27:0/2621930493 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80ad2d0 con 0x5626350cf390 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1b6a00640 1 -- 172.21.15.27:0/2621930493 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b80adec0 con 0x5626350cf390 2024-09-21T07:21:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.845+0000 7fd1b6a00640 1 -- 172.21.15.27:0/2621930493 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80a9a30 con 0x5626350cf390 2024-09-21T07:21:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/2621930493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350cf390 msgr2=0x56263549f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350cf390 0x56263549f8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80e50c0 tx=0x7fd1b80ea230 comp rx=0 tx=0).stop 2024-09-21T07:21:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/2621930493 shutdown_connections 2024-09-21T07:21:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350f0070 0x56263549fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350cf390 0x56263549f8f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/2621930493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350c71b0 0x56263549dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/2621930493 >> 172.21.15.27:0/2621930493 conn(0x5626350e4460 msgr2=0x5626354a0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/2621930493 shutdown_connections 2024-09-21T07:21:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.846+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/2621930493 wait complete. 2024-09-21T07:21:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.847+0000 7fd1d7dfb740 1 -- start start 2024-09-21T07:21:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.847+0000 7fd1d7dfb740 1 Processor -- start 2024-09-21T07:21:32.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1d7dfb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b15a0 con 0x5626350f0070 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b2a10 con 0x5626350c71b0 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1d7dfb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1b1c70 con 0x5626350cf390 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1d7dfb740 10 client.0 _ll_register_callbacks cb 0x7fd1bc0114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1ca800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1c9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53396/0 (socket says 172.21.15.27:53396) 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.848+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1341294179 learned_addr learned my addr 172.21.15.27:0/1341294179 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1c9400640 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1341294179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 msgr2=0x5626354aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1341294179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 msgr2=0x5626354a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1c9e00640 1 -- 172.21.15.27:0/1341294179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626354ab460 con 0x5626350c71b0 2024-09-21T07:21:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1c9e00640 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354aa9b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00f7d0 tx=0x7fd1bc011580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210ba0bb009ccc40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1abe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.849+0000 7fd1abe00640 1 -- 172.21.15.27:0/1341294179 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc026040 con 0x5626350c71b0 2024-09-21T07:21:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b80adec0 con 0x5626350c71b0 2024-09-21T07:21:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1abe00640 1 -- 172.21.15.27:0/1341294179 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fd1bc005410 con 0x5626350c71b0 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1abe00640 1 -- 172.21.15.27:0/1341294179 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc01e040 con 0x5626350c71b0 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 10 client.27992 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56263551f250 con 0x5626350c71b0 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1abe00640 1 -- 172.21.15.27:0/1341294179 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b80adec0 con 0x5626350c71b0 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1abe00640 1 -- 172.21.15.27:0/1341294179 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd1bc021070 con 0x5626350c71b0 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 10 client.27992 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 10 client.27992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:21:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 20 client.27992 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562635083da0 0x562635521a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 4 client.27992 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.850+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56263540b9b0 con 0x562635083da0 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.851+0000 7fd1c9400640 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562635083da0 0x562635521a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.851+0000 7fd1c9400640 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562635083da0 0x562635521a60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7dc03b1fb80955c server_cookie=6e4c2d945d59e3cf in_seq=0 out_seq=0 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.851+0000 7fd1abe00640 10 client.27992 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1abe00640 1 -- 172.21.15.27:0/1341294179 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56263540b9b0 con 0x562635083da0 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1abe00640 10 client.27992 handle_command_reply: tid=0 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 client.27992 shutdown 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 msgr2=0x5626354aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354aa9b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00f7d0 tx=0x7fd1bc011580 comp rx=0 tx=0).stop 2024-09-21T07:21:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 shutdown_connections 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562635083da0 0x562635521a60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626350cf390 0x5626354aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1abe00640 0 client.27992 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5626350f0070 0x5626354a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 --2- 172.21.15.27:0/1341294179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626350c71b0 0x5626354aa9b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 >> 172.21.15.27:0/1341294179 conn(0x5626350e4460 msgr2=0x562635493600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 shutdown_connections 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 1 -- 172.21.15.27:0/1341294179 wait complete. 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.852+0000 7fd1d7dfb740 20 client.27992 trim_cache size 0 max 16384 2024-09-21T07:21:32.857 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd1b00b6680 msgr2=0x7fd1b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd1b00b6680 0x7fd1b00b8b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4005cc0 tx=0x7fd1c4005c30 comp rx=0 tx=0).stop 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1cc10b500 msgr2=0x7fd1cc19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1cc10b500 0x7fd1cc19ca40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc06c8a0 tx=0x7fd1b800ee50 comp rx=0 tx=0).stop 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 shutdown_connections 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1cc10b500 0x7fd1cc19ca40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd1b00b6680 0x7fd1b00b8b90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1cc10be50 0x7fd1cc1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 --2- 172.21.15.27:0/169005205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1cc105bc0 0x7fd1cc19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 >> 172.21.15.27:0/169005205 conn(0x7fd1cc101670 msgr2=0x7fd1cc103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 shutdown_connections 2024-09-21T07:21:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:32.853+0000 7fd1cb200640 1 -- 172.21.15.27:0/169005205 wait complete. 2024-09-21T07:21:32.877 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:32 smithi027 ceph-mon[30779]: pgmap v639: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:21:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:32 smithi119 ceph-mon[34519]: pgmap v639: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:21:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:32 smithi153 ceph-mon[34747]: pgmap v639: 129 pgs: 129 active+clean; 2.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-21T07:21:33.634 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:21:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.633+0000 7fecd5800640 1 -- 172.21.15.27:0/1778876202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0114570 msgr2=0x7fecd0116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.633+0000 7fecd5800640 1 --2- 172.21.15.27:0/1778876202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0114570 0x7fecd0116990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009ca0 tx=0x7fecc802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.634+0000 7fecd5800640 1 -- 172.21.15.27:0/1778876202 shutdown_connections 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.634+0000 7fecd5800640 1 --2- 172.21.15.27:0/1778876202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecd0078290 0x7fecd0114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.634+0000 7fecd5800640 1 --2- 172.21.15.27:0/1778876202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0114570 0x7fecd0116990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.634+0000 7fecd5800640 1 --2- 172.21.15.27:0/1778876202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecd0077940 0x7fecd0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.634+0000 7fecd5800640 1 -- 172.21.15.27:0/1778876202 >> 172.21.15.27:0/1778876202 conn(0x7fecd0101680 msgr2=0x7fecd0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.634+0000 7fecd5800640 1 -- 172.21.15.27:0/1778876202 shutdown_connections 2024-09-21T07:21:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.634+0000 7fecd5800640 1 -- 172.21.15.27:0/1778876202 wait complete. 2024-09-21T07:21:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 Processor -- start 2024-09-21T07:21:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 -- start start 2024-09-21T07:21:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecd0077940 0x7fecd01a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0078290 0x7fecd01a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecd0114570 0x7fecd01a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0119ef0 con 0x7fecd0078290 2024-09-21T07:21:34.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0119d70 con 0x7fecd0077940 2024-09-21T07:21:34.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.635+0000 7fecd5800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecd011a070 con 0x7fecd0114570 2024-09-21T07:21:34.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecd0114570 0x7fecd01a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0078290 0x7fecd01a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecd0077940 0x7fecd01a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecd0114570 0x7fecd01a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52122/0 (socket says 172.21.15.27:52122) 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7fecd4e00640 1 -- 172.21.15.27:0/1053278559 learned_addr learned my addr 172.21.15.27:0/1053278559 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0078290 0x7fecd01a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39150/0 (socket says 172.21.15.27:39150) 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecd0077940 0x7fecd01a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53418/0 (socket says 172.21.15.27:53418) 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccf400640 1 -- 172.21.15.27:0/1053278559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecd0114570 msgr2=0x7fecd01a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccf400640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecd0114570 0x7fecd01a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccf400640 1 -- 172.21.15.27:0/1053278559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecd0077940 msgr2=0x7fecd01a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccf400640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecd0077940 0x7fecd01a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.636+0000 7feccf400640 1 -- 172.21.15.27:0/1053278559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd01a6830 con 0x7fecd0078290 2024-09-21T07:21:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.637+0000 7feccf400640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0078290 0x7fecd01a0e60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fecd006c630 tx=0x7fecbc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.637+0000 7fecccc00640 1 -- 172.21.15.27:0/1053278559 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc017070 con 0x7fecd0078290 2024-09-21T07:21:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.637+0000 7fecccc00640 1 -- 172.21.15.27:0/1053278559 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc0099a0 con 0x7fecd0078290 2024-09-21T07:21:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.637+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd01a6b20 con 0x7fecd0078290 2024-09-21T07:21:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.637+0000 7fecccc00640 1 -- 172.21.15.27:0/1053278559 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0047a0 con 0x7fecd0078290 2024-09-21T07:21:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.637+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd01ad280 con 0x7fecd0078290 2024-09-21T07:21:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 Processor -- start 2024-09-21T07:21:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 -- start start 2024-09-21T07:21:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d5e070 0x55ecda104f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda1054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda1059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ecda07bfb0 con 0x55ecd9d3d390 2024-09-21T07:21:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ecda06b3e0 con 0x55ecd9d351b0 2024-09-21T07:21:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ecda06f1f0 con 0x55ecd9d5e070 2024-09-21T07:21:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.641+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d5e070 0x55ecda104f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7fecccc00640 1 -- 172.21.15.27:0/1053278559 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fecbc004940 con 0x7fecd0078290 2024-09-21T07:21:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7feccfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda1054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d5e070 0x55ecda104f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52130/0 (socket says 172.21.15.27:52130) 2024-09-21T07:21:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7fecd4e00640 1 -- 172.21.15.27:0/715796538 learned_addr learned my addr 172.21.15.27:0/715796538 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7feccf400640 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda1059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7feccfe00640 1 -- 172.21.15.27:0/715796538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d5e070 msgr2=0x55ecda104f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7feccfe00640 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d5e070 0x55ecda104f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7feccfe00640 1 -- 172.21.15.27:0/715796538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 msgr2=0x55ecda1059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7feccfe00640 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda1059f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7feccfe00640 1 -- 172.21.15.27:0/715796538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecda035cd0 con 0x55ecd9d351b0 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.642+0000 7fecd4e00640 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d5e070 0x55ecda104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.643+0000 7feccfe00640 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda1054b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009c20 tx=0x7fecc0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=965d375d2a493f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.643+0000 7fecc5600640 1 -- 172.21.15.27:0/715796538 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc001e070 con 0x55ecd9d351b0 2024-09-21T07:21:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.643+0000 7fecc5600640 1 -- 172.21.15.27:0/715796538 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc0009dd0 con 0x55ecd9d351b0 2024-09-21T07:21:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.643+0000 7fecc5600640 1 -- 172.21.15.27:0/715796538 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0018710 con 0x55ecd9d351b0 2024-09-21T07:21:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecccc00640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb40b6700 0x7fecb40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 -- 172.21.15.27:0/715796538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 msgr2=0x55ecda1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda1054b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009c20 tx=0x7fecc0011810 comp rx=0 tx=0).stop 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecccc00640 1 -- 172.21.15.27:0/1053278559 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fecbc0e7630 con 0x7fecd0078290 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 -- 172.21.15.27:0/715796538 shutdown_connections 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecd4e00640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb40b6700 0x7fecb40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda1054b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/715796538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d5e070 0x55ecda104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 -- 172.21.15.27:0/715796538 >> 172.21.15.27:0/715796538 conn(0x55ecd9d52460 msgr2=0x55ecda102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 -- 172.21.15.27:0/715796538 shutdown_connections 2024-09-21T07:21:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.644+0000 7fecdd9ec740 1 -- 172.21.15.27:0/715796538 wait complete. 2024-09-21T07:21:34.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.645+0000 7fecd4e00640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb40b6700 0x7fecb40b8c10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fecc80094c0 tx=0x7fecc8005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:34.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.645+0000 7fecdd9ec740 1 -- start start 2024-09-21T07:21:34.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.645+0000 7fecdd9ec740 1 Processor -- start 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda1106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda110c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d5e070 0x55ecda11ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ecd9ecc700 con 0x55ecd9d3d390 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ecda006580 con 0x55ecd9d5e070 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ecda113750 con 0x55ecd9d351b0 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecdd9ec740 10 client.0 _ll_register_callbacks cb 0x55ecda016470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda110c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda1106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda110c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39174/0 (socket says 172.21.15.27:39174) 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccf400640 1 -- 172.21.15.27:0/2979632520 learned_addr learned my addr 172.21.15.27:0/2979632520 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda1106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52138/0 (socket says 172.21.15.27:52138) 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7fecd4e00640 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d5e070 0x55ecda11ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccfe00640 1 -- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d5e070 msgr2=0x55ecda11ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccfe00640 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d5e070 0x55ecda11ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccfe00640 1 -- 172.21.15.27:0/2979632520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 msgr2=0x55ecda110c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.646+0000 7feccfe00640 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda110c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.647+0000 7feccfe00640 1 -- 172.21.15.27:0/2979632520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecd9dc9650 con 0x55ecd9d351b0 2024-09-21T07:21:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.647+0000 7feccfe00640 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda1106c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecc001c040 tx=0x7fecc00098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e32e4f3eddf39ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.647+0000 7fecba000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.647+0000 7fecba000640 1 -- 172.21.15.27:0/2979632520 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc001e040 con 0x55ecd9d351b0 2024-09-21T07:21:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.647+0000 7fecba000640 1 -- 172.21.15.27:0/2979632520 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fecc0005040 con 0x55ecd9d351b0 2024-09-21T07:21:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.647+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ecda008b90 con 0x55ecd9d351b0 2024-09-21T07:21:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecba000640 1 -- 172.21.15.27:0/2979632520 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecc0032050 con 0x55ecd9d351b0 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecdd9ec740 10 client.27958 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecda023950 con 0x55ecd9d351b0 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecba000640 1 -- 172.21.15.27:0/2979632520 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ecda008b90 con 0x55ecd9d351b0 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecba000640 1 -- 172.21.15.27:0/2979632520 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fecc0021030 con 0x55ecd9d351b0 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecdd9ec740 10 client.27958 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecdd9ec740 10 client.27958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.648+0000 7fecdd9ec740 20 client.27958 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.649+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9d1c9f0 0x55ecda122740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.649+0000 7fecdd9ec740 4 client.27958 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.649+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecda07df00 con 0x55ecd9d1c9f0 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.649+0000 7feccf400640 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9d1c9f0 0x55ecda122740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.649+0000 7feccf400640 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9d1c9f0 0x55ecda122740 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e070ad8a8963c1f server_cookie=4241237dfbff06fb in_seq=0 out_seq=0 2024-09-21T07:21:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.649+0000 7fecba000640 10 client.27958 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:34.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.650+0000 7fecba000640 1 -- 172.21.15.27:0/2979632520 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ecda07df00 con 0x55ecd9d1c9f0 2024-09-21T07:21:34.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.650+0000 7fecba000640 10 client.27958 handle_command_reply: tid=0 2024-09-21T07:21:34.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.650+0000 7fecdd9ec740 1 client.27958 shutdown 2024-09-21T07:21:34.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 msgr2=0x55ecda1106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda1106c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecc001c040 tx=0x7fecc00098d0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 shutdown_connections 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9d1c9f0 0x55ecda122740 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d5e070 0x55ecda11ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d3d390 0x55ecda110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2979632520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda1106c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecba000640 0 client.27958 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 >> 172.21.15.27:0/2979632520 conn(0x55ecd9d52460 msgr2=0x55ecda108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 shutdown_connections 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2979632520 wait complete. 2024-09-21T07:21:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.651+0000 7fecdd9ec740 20 client.27958 trim_cache size 0 max 16384 2024-09-21T07:21:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 Processor -- start 2024-09-21T07:21:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 -- start start 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda101d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d3d390 0x55ecda11c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecd01b9c10 con 0x55ecd9d5e070 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecd01b74e0 con 0x55ecd9d3d390 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.666+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecd01b5350 con 0x55ecd9d351b0 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d3d390 0x55ecda11c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52140/0 (socket says 172.21.15.27:52140) 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 -- 172.21.15.27:0/542360330 learned_addr learned my addr 172.21.15.27:0/542360330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d3d390 0x55ecda11c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53464/0 (socket says 172.21.15.27:53464) 2024-09-21T07:21:34.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7fecd4e00640 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 -- 172.21.15.27:0/542360330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d3d390 msgr2=0x55ecda11c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d3d390 0x55ecda11c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 -- 172.21.15.27:0/542360330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 msgr2=0x55ecda10d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda10d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccfe00640 1 -- 172.21.15.27:0/542360330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecda107dd0 con 0x55ecd9d351b0 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.667+0000 7feccf400640 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d3d390 0x55ecda11c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.668+0000 7feccfe00640 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda101d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fecc00035f0 tx=0x7fecc0009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c9a94dbfef467a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.668+0000 7fecc4c00640 1 -- 172.21.15.27:0/542360330 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc001e040 con 0x55ecd9d351b0 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.668+0000 7fecc4c00640 1 -- 172.21.15.27:0/542360330 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc0029070 con 0x55ecd9d351b0 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.668+0000 7fecc4c00640 1 -- 172.21.15.27:0/542360330 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0005240 con 0x55ecd9d351b0 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 -- 172.21.15.27:0/542360330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 msgr2=0x55ecda101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda101d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fecc00035f0 tx=0x7fecc0009f60 comp rx=0 tx=0).stop 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 -- 172.21.15.27:0/542360330 shutdown_connections 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d3d390 0x55ecda11c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/542360330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d351b0 0x55ecda101d10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 -- 172.21.15.27:0/542360330 >> 172.21.15.27:0/542360330 conn(0x55ecd9d52460 msgr2=0x55ecda101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 -- 172.21.15.27:0/542360330 shutdown_connections 2024-09-21T07:21:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 -- 172.21.15.27:0/542360330 wait complete. 2024-09-21T07:21:34.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.669+0000 7fecdd9ec740 1 -- start start 2024-09-21T07:21:34.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 1 Processor -- start 2024-09-21T07:21:34.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda118bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d3d390 0x55ecda1190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda113960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecd01b59a0 con 0x55ecd9d5e070 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecd01b6e10 con 0x55ecd9d351b0 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecd01b6070 con 0x55ecd9d3d390 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecdd9ec740 10 client.0 _ll_register_callbacks cb 0x7fecc0009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7feccf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d3d390 0x55ecda1190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7feccfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda118bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39184/0 (socket says 172.21.15.27:39184) 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7feccf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d3d390 0x55ecda1190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52148/0 (socket says 172.21.15.27:52148) 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.670+0000 7fecd4e00640 1 -- 172.21.15.27:0/2972641156 learned_addr learned my addr 172.21.15.27:0/2972641156 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7feccfe00640 1 -- 172.21.15.27:0/2972641156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d3d390 msgr2=0x55ecda1190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7feccfe00640 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d3d390 0x55ecda1190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7feccfe00640 1 -- 172.21.15.27:0/2972641156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 msgr2=0x55ecda113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:34.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7feccfe00640 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7feccfe00640 1 -- 172.21.15.27:0/2972641156 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecda113fd0 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7feccfe00640 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda118bb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fecc0004e70 tx=0x7fecc0004350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f53bcdcffd9ebbcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7fecba000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.671+0000 7fecba000640 1 -- 172.21.15.27:0/2972641156 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc001e040 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecba000640 1 -- 172.21.15.27:0/2972641156 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fecc0004fe0 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc0029070 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecba000640 1 -- 172.21.15.27:0/2972641156 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecc0005220 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecdd9ec740 10 client.28010 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecda18d880 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecba000640 1 -- 172.21.15.27:0/2972641156 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc0029070 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecba000640 1 -- 172.21.15.27:0/2972641156 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fecc0021070 con 0x55ecd9d351b0 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecdd9ec740 10 client.28010 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecdd9ec740 10 client.28010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.672+0000 7fecdd9ec740 20 client.28010 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:34.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.673+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9cf1da0 0x55ecda18fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:34.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.673+0000 7fecdd9ec740 4 client.28010 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:21:34.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.673+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecda0799b0 con 0x55ecd9cf1da0 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.673+0000 7feccf400640 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9cf1da0 0x55ecda18fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.673+0000 7feccf400640 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9cf1da0 0x55ecda18fd60 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72580f9e1d741b34 server_cookie=7fa104b549f60820 in_seq=0 out_seq=0 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.673+0000 7fecba000640 10 client.28010 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecba000640 1 -- 172.21.15.27:0/2972641156 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecda0799b0 con 0x55ecd9cf1da0 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecba000640 10 client.28010 handle_command_reply: tid=0 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 client.28010 shutdown 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 msgr2=0x55ecda118bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda118bb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fecc0004e70 tx=0x7fecc0004350 comp rx=0 tx=0).stop 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 shutdown_connections 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ecd9cf1da0 0x55ecda18fd60 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ecd9d3d390 0x55ecda1190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ecd9d5e070 0x55ecda113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 --2- 172.21.15.27:0/2972641156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ecd9d351b0 0x55ecda118bb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecba000640 0 client.28010 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 >> 172.21.15.27:0/2972641156 conn(0x55ecd9d52460 msgr2=0x55ecda106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 shutdown_connections 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 1 -- 172.21.15.27:0/2972641156 wait complete. 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.674+0000 7fecdd9ec740 20 client.28010 trim_cache size 0 max 16384 2024-09-21T07:21:34.679 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb40b6700 msgr2=0x7fecb40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb40b6700 0x7fecb40b8c10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fecc80094c0 tx=0x7fecc8005c50 comp rx=0 tx=0).stop 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0078290 msgr2=0x7fecd01a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0078290 0x7fecd01a0e60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fecd006c630 tx=0x7fecbc00dbb0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 shutdown_connections 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecd0114570 0x7fecd01a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb40b6700 0x7fecb40b8c10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd0078290 0x7fecd01a0e60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 --2- 172.21.15.27:0/1053278559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecd0077940 0x7fecd01a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:34.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 >> 172.21.15.27:0/1053278559 conn(0x7fecd0101680 msgr2=0x7fecd01165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:34.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 shutdown_connections 2024-09-21T07:21:34.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:34.675+0000 7fecd5800640 1 -- 172.21.15.27:0/1053278559 wait complete. 2024-09-21T07:21:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:34 smithi119 ceph-mon[34519]: pgmap v640: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-21T07:21:35.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:34 smithi027 ceph-mon[30779]: pgmap v640: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-21T07:21:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:34 smithi153 ceph-mon[34747]: pgmap v640: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-21T07:21:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 -- 172.21.15.27:0/2456737877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4103910 msgr2=0x7f11a4103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 --2- 172.21.15.27:0/2456737877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4103910 0x7f11a4103d20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1194009900 tx=0x7f119402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 -- 172.21.15.27:0/2456737877 shutdown_connections 2024-09-21T07:21:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 --2- 172.21.15.27:0/2456737877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11a4109b90 0x7f11a41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 --2- 172.21.15.27:0/2456737877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4109240 0x7f11a4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 --2- 172.21.15.27:0/2456737877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4103910 0x7f11a4103d20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 -- 172.21.15.27:0/2456737877 >> 172.21.15.27:0/2456737877 conn(0x7f11a40ff330 msgr2=0x7f11a4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 -- 172.21.15.27:0/2456737877 shutdown_connections 2024-09-21T07:21:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.489+0000 7f11a3e00640 1 -- 172.21.15.27:0/2456737877 wait complete. 2024-09-21T07:21:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.490+0000 7f11a3e00640 1 Processor -- start 2024-09-21T07:21:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.490+0000 7f11a3e00640 1 -- start start 2024-09-21T07:21:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4103910 0x7f11a419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4109240 0x7f11a419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11a4109b90 0x7f11a41a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4113850 con 0x7f11a4109b90 2024-09-21T07:21:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41136d0 con 0x7f11a4103910 2024-09-21T07:21:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41139d0 con 0x7f11a4109240 2024-09-21T07:21:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4103910 0x7f11a419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11a4109b90 0x7f11a41a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4103910 0x7f11a419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53488/0 (socket says 172.21.15.27:53488) 2024-09-21T07:21:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a2a00640 1 -- 172.21.15.27:0/3410453714 learned_addr learned my addr 172.21.15.27:0/3410453714 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.491+0000 7f11a2000640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4109240 0x7f11a419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3400640 1 -- 172.21.15.27:0/3410453714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4109240 msgr2=0x7f11a419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3400640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4109240 0x7f11a419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3400640 1 -- 172.21.15.27:0/3410453714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4103910 msgr2=0x7f11a419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3400640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4103910 0x7f11a419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3400640 1 -- 172.21.15.27:0/3410453714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a41a2420 con 0x7f11a4109b90 2024-09-21T07:21:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4103910 0x7f11a419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3400640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11a4109b90 0x7f11a41a1d20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f119c00ec50 tx=0x7f119c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f119b400640 1 -- 172.21.15.27:0/3410453714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c003b70 con 0x7f11a4109b90 2024-09-21T07:21:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f119b400640 1 -- 172.21.15.27:0/3410453714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f119c003d10 con 0x7f11a4109b90 2024-09-21T07:21:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41a2710 con 0x7f11a4109b90 2024-09-21T07:21:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f119b400640 1 -- 172.21.15.27:0/3410453714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c010830 con 0x7f11a4109b90 2024-09-21T07:21:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.492+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a41a8ea0 con 0x7f11a4109b90 2024-09-21T07:21:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.495+0000 7f11b0c72740 1 Processor -- start 2024-09-21T07:21:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.496+0000 7f119b400640 1 -- 172.21.15.27:0/3410453714 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f119c003ec0 con 0x7f11a4109b90 2024-09-21T07:21:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.496+0000 7f11b0c72740 1 -- start start 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454291b0 0x5648457cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454002f0 0x5648457ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454084d0 0x5648457ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11b0c72740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5648457436b0 con 0x5648454291b0 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11b0c72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564845732ae0 con 0x5648454002f0 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11b0c72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5648457368f0 con 0x5648454084d0 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454291b0 0x5648457cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454291b0 0x5648457cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39216/0 (socket says 172.21.15.27:39216) 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2000640 1 -- 172.21.15.27:0/3479929710 learned_addr learned my addr 172.21.15.27:0/3479929710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454002f0 0x5648457ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2000640 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454084d0 0x5648457ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2a00640 1 -- 172.21.15.27:0/3479929710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454084d0 msgr2=0x5648457ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454084d0 0x5648457ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2a00640 1 -- 172.21.15.27:0/3479929710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454291b0 msgr2=0x5648457cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454291b0 0x5648457cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2a00640 1 -- 172.21.15.27:0/3479929710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648456ff780 con 0x5648454002f0 2024-09-21T07:21:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.497+0000 7f11a2000640 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454084d0 0x5648457ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.498+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454002f0 0x5648457ce350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1194005940 tx=0x7f1194005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78761a6941ddb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.498+0000 7f119b400640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f11880b6700 0x7f11880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.498+0000 7f119b400640 1 -- 172.21.15.27:0/3410453714 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f119c0e7ff0 con 0x7f11a4109b90 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.498+0000 7f118fe00640 1 -- 172.21.15.27:0/3479929710 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035040 con 0x5648454002f0 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.498+0000 7f11a2000640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f11880b6700 0x7f11880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.498+0000 7f118fe00640 1 -- 172.21.15.27:0/3479929710 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194031820 con 0x5648454002f0 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.498+0000 7f118fe00640 1 -- 172.21.15.27:0/3479929710 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11940049a0 con 0x5648454002f0 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 -- 172.21.15.27:0/3479929710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454002f0 msgr2=0x5648457ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454002f0 0x5648457ce350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1194005940 tx=0x7f1194005ed0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11a2000640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f11880b6700 0x7f11880b8c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f11900058c0 tx=0x7f1190005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 -- 172.21.15.27:0/3479929710 shutdown_connections 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454084d0 0x5648457ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454002f0 0x5648457ce350 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3479929710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454291b0 0x5648457cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 -- 172.21.15.27:0/3479929710 >> 172.21.15.27:0/3479929710 conn(0x56484541d300 msgr2=0x5648457cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 -- 172.21.15.27:0/3479929710 shutdown_connections 2024-09-21T07:21:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.499+0000 7f11b0c72740 1 -- 172.21.15.27:0/3479929710 wait complete. 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.500+0000 7f11b0c72740 1 -- start start 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.500+0000 7f11b0c72740 1 Processor -- start 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457d9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11b0c72740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56484566aab0 con 0x5648454002f0 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11b0c72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5648456d9570 con 0x5648454084d0 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11b0c72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5648457dc530 con 0x5648454291b0 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11b0c72740 10 client.0 _ll_register_callbacks cb 0x5648456e8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457d9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39224/0 (socket says 172.21.15.27:39224) 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 -- 172.21.15.27:0/3593501807 learned_addr learned my addr 172.21.15.27:0/3593501807 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2000640 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a3400640 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 -- 172.21.15.27:0/3593501807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 msgr2=0x5648457d9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 -- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 msgr2=0x5648457d99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.501+0000 7f11a2a00640 1 -- 172.21.15.27:0/3593501807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564845494790 con 0x5648454002f0 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f11a2a00640 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457d9470 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f11940042f0 tx=0x7f119402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e626810ebb2f77e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f118cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f118cc00640 1 -- 172.21.15.27:0/3593501807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119404e020 con 0x5648454002f0 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1194031820 con 0x5648454002f0 2024-09-21T07:21:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f118cc00640 1 -- 172.21.15.27:0/3593501807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f1194003800 con 0x5648454002f0 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f118cc00640 1 -- 172.21.15.27:0/3593501807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1194059050 con 0x5648454002f0 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f11b0c72740 10 client.17988 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.502+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56484550d620 con 0x5648454002f0 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f118cc00640 1 -- 172.21.15.27:0/3593501807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194031820 con 0x5648454002f0 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f118cc00640 1 -- 172.21.15.27:0/3593501807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1194046870 con 0x5648454002f0 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f11b0c72740 10 client.17988 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f11b0c72740 10 client.17988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f11b0c72740 20 client.17988 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453e7d80 0x5648457eb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f11b0c72740 4 client.17988 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564845745600 con 0x5648453e7d80 2024-09-21T07:21:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.503+0000 7f11a2000640 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453e7d80 0x5648457eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.504+0000 7f11a2000640 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453e7d80 0x5648457eb280 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f019eeb4acbdaa0 server_cookie=9ea0880b8007a414 in_seq=0 out_seq=0 2024-09-21T07:21:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.504+0000 7f118cc00640 10 client.17988 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f118cc00640 1 -- 172.21.15.27:0/3593501807 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564845745600 con 0x5648453e7d80 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f118cc00640 10 client.17988 handle_command_reply: tid=0 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 client.17988 shutdown 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 msgr2=0x5648457d9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457d9470 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f11940042f0 tx=0x7f119402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 shutdown_connections 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453e7d80 0x5648457eb280 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f118cc00640 0 client.17988 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 --2- 172.21.15.27:0/3593501807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457d9470 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 >> 172.21.15.27:0/3593501807 conn(0x56484541d300 msgr2=0x5648457cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 shutdown_connections 2024-09-21T07:21:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 1 -- 172.21.15.27:0/3593501807 wait complete. 2024-09-21T07:21:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.505+0000 7f11b0c72740 20 client.17988 trim_cache size 0 max 16384 2024-09-21T07:21:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.520+0000 7f11b0c72740 1 Processor -- start 2024-09-21T07:21:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.520+0000 7f11b0c72740 1 -- start start 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11b0c72740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b5790 con 0x5648454002f0 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11b0c72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b3060 con 0x5648454084d0 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11b0c72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b0ed0 con 0x5648454291b0 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39238/0 (socket says 172.21.15.27:39238) 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11a2a00640 1 -- 172.21.15.27:0/328644841 learned_addr learned my addr 172.21.15.27:0/328644841 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11a2000640 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11a3400640 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11a2a00640 1 -- 172.21.15.27:0/328644841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 msgr2=0x5648457d6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.521+0000 7f11a2a00640 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.522+0000 7f11a2a00640 1 -- 172.21.15.27:0/328644841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 msgr2=0x5648457d6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.522+0000 7f11a2a00640 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.522+0000 7f11a2a00640 1 -- 172.21.15.27:0/328644841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648457dbde0 con 0x5648454002f0 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.522+0000 7f11a2a00640 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457caa70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1194000c00 tx=0x7f1194001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dc9f07a5291e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.522+0000 7f118f400640 1 -- 172.21.15.27:0/328644841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119404e070 con 0x5648454002f0 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.522+0000 7f118f400640 1 -- 172.21.15.27:0/328644841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194046450 con 0x5648454002f0 2024-09-21T07:21:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.522+0000 7f118f400640 1 -- 172.21.15.27:0/328644841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035040 con 0x5648454002f0 2024-09-21T07:21:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 -- 172.21.15.27:0/328644841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 msgr2=0x5648457caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457caa70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1194000c00 tx=0x7f1194001e70 comp rx=0 tx=0).stop 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 -- 172.21.15.27:0/328644841 shutdown_connections 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x5648457d6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457d6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 --2- 172.21.15.27:0/328644841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457caa70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 -- 172.21.15.27:0/328644841 >> 172.21.15.27:0/328644841 conn(0x56484541d300 msgr2=0x5648457ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 -- 172.21.15.27:0/328644841 shutdown_connections 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 -- 172.21.15.27:0/328644841 wait complete. 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.523+0000 7f11b0c72740 1 -- start start 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 1 Processor -- start 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x564845853a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b1520 con 0x5648454002f0 2024-09-21T07:21:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b2990 con 0x5648454084d0 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b1bf0 con 0x5648454291b0 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11b0c72740 10 client.0 _ll_register_callbacks cb 0x7f1194040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.524+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457ed990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39240/0 (socket says 172.21.15.27:39240) 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2a00640 1 -- 172.21.15.27:0/892252076 learned_addr learned my addr 172.21.15.27:0/892252076 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a3400640 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x564845853a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2000640 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2a00640 1 -- 172.21.15.27:0/892252076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 msgr2=0x564845853a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2a00640 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x564845853a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2a00640 1 -- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 msgr2=0x5648457eded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2a00640 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457eded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2a00640 1 -- 172.21.15.27:0/892252076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564845853f40 con 0x5648454002f0 2024-09-21T07:21:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f11a2a00640 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457ed990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1194004520 tx=0x7f1194005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46c8d4313addacf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f118cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.525+0000 7f118cc00640 1 -- 172.21.15.27:0/892252076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035070 con 0x5648454002f0 2024-09-21T07:21:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f118cc00640 1 -- 172.21.15.27:0/892252076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f1194003800 con 0x5648454002f0 2024-09-21T07:21:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1194046450 con 0x5648454002f0 2024-09-21T07:21:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f118cc00640 1 -- 172.21.15.27:0/892252076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f119406a050 con 0x5648454002f0 2024-09-21T07:21:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 10 client.17994 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564845855160 con 0x5648454002f0 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f118cc00640 1 -- 172.21.15.27:0/892252076 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194046450 con 0x5648454002f0 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f118cc00640 1 -- 172.21.15.27:0/892252076 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1194065030 con 0x5648454002f0 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 10 client.17994 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 20 client.17994 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453bd1c0 0x5648458577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 4 client.17994 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.526+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648457410b0 con 0x5648453bd1c0 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.527+0000 7f11a2000640 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453bd1c0 0x5648458577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.527+0000 7f11a2000640 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453bd1c0 0x5648458577d0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f8d9c1f35faaca server_cookie=ea29b5be08128540 in_seq=0 out_seq=0 2024-09-21T07:21:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.527+0000 7f118cc00640 10 client.17994 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f118cc00640 1 -- 172.21.15.27:0/892252076 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648457410b0 con 0x5648453bd1c0 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f118cc00640 10 client.17994 handle_command_reply: tid=0 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 client.17994 shutdown 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 msgr2=0x5648457ed990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457ed990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1194004520 tx=0x7f1194005700 comp rx=0 tx=0).stop 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 shutdown_connections 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5648453bd1c0 0x5648458577d0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648454291b0 0x564845853a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648454084d0 0x5648457eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 --2- 172.21.15.27:0/892252076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5648454002f0 0x5648457ed990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f118cc00640 0 client.17994 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.537+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 >> 172.21.15.27:0/892252076 conn(0x56484541d300 msgr2=0x5648457ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.538+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 shutdown_connections 2024-09-21T07:21:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.538+0000 7f11b0c72740 1 -- 172.21.15.27:0/892252076 wait complete. 2024-09-21T07:21:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.538+0000 7f11b0c72740 20 client.17994 trim_cache size 0 max 16384 2024-09-21T07:21:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:21:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:21:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:21:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.538+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f11880b6700 msgr2=0x7f11880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.538+0000 7f11a3e00640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f11880b6700 0x7f11880b8c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f11900058c0 tx=0x7f1190005850 comp rx=0 tx=0).stop 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.538+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11a4109b90 msgr2=0x7f11a41a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.538+0000 7f11a3e00640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11a4109b90 0x7f11a41a1d20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f119c00ec50 tx=0x7f119c00c6a0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 shutdown_connections 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11a4109240 0x7f11a419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f11880b6700 0x7f11880b8c10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11a4109b90 0x7f11a41a1d20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 --2- 172.21.15.27:0/3410453714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a4103910 0x7f11a419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 >> 172.21.15.27:0/3410453714 conn(0x7f11a40ff330 msgr2=0x7f11a4108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 shutdown_connections 2024-09-21T07:21:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:35.539+0000 7f11a3e00640 1 -- 172.21.15.27:0/3410453714 wait complete. 2024-09-21T07:21:36.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:21:36.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:21:36.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:36 smithi027 ceph-mon[30779]: pgmap v641: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-21T07:21:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:36 smithi119 ceph-mon[34519]: pgmap v641: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-21T07:21:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:36 smithi153 ceph-mon[34747]: pgmap v641: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-21T07:21:37.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 -- 172.21.15.27:0/440968367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 msgr2=0x7f4644105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 --2- 172.21.15.27:0/440968367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f4644105fc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4634009900 tx=0x7f463402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 -- 172.21.15.27:0/440968367 shutdown_connections 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 --2- 172.21.15.27:0/440968367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f464410be40 0x7f4644112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 --2- 172.21.15.27:0/440968367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f464410b4f0 0x7f464410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 --2- 172.21.15.27:0/440968367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f4644105fc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 -- 172.21.15.27:0/440968367 >> 172.21.15.27:0/440968367 conn(0x7f4644101680 msgr2=0x7f4644103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 -- 172.21.15.27:0/440968367 shutdown_connections 2024-09-21T07:21:37.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.488+0000 7f464a000640 1 -- 172.21.15.27:0/440968367 wait complete. 2024-09-21T07:21:37.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.489+0000 7f464a000640 1 Processor -- start 2024-09-21T07:21:37.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.490+0000 7f464a000640 1 -- start start 2024-09-21T07:21:37.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.490+0000 7f464a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f464419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.490+0000 7f464a000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f464410b4f0 0x7f464419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.490+0000 7f464a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f464410be40 0x7f46441a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.490+0000 7f464a000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4644115b50 con 0x7f4644105bb0 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.490+0000 7f464a000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46441159d0 con 0x7f464410be40 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.490+0000 7f464a000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4644115cd0 con 0x7f464410b4f0 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f464419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f464419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39266/0 (socket says 172.21.15.27:39266) 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4649600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f464410be40 0x7f46441a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 -- 172.21.15.27:0/3154222772 learned_addr learned my addr 172.21.15.27:0/3154222772 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:37.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f463be00640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f464410b4f0 0x7f464419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 -- 172.21.15.27:0/3154222772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f464410b4f0 msgr2=0x7f464419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f464410b4f0 0x7f464419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 -- 172.21.15.27:0/3154222772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f464410be40 msgr2=0x7f46441a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f464410be40 0x7f46441a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.491+0000 7f4648c00640 1 -- 172.21.15.27:0/3154222772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46441a24b0 con 0x7f4644105bb0 2024-09-21T07:21:37.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.492+0000 7f4648c00640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f464419c540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4634005ab0 tx=0x7f4634003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:37.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.492+0000 7f4642000640 1 -- 172.21.15.27:0/3154222772 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463403f070 con 0x7f4644105bb0 2024-09-21T07:21:37.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.492+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46441a2740 con 0x7f4644105bb0 2024-09-21T07:21:37.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.492+0000 7f4642000640 1 -- 172.21.15.27:0/3154222772 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4634031d60 con 0x7f4644105bb0 2024-09-21T07:21:37.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.492+0000 7f4642000640 1 -- 172.21.15.27:0/3154222772 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634035030 con 0x7f4644105bb0 2024-09-21T07:21:37.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.492+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46441a8f00 con 0x7f4644105bb0 2024-09-21T07:21:37.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.495+0000 7f465240d740 1 Processor -- start 2024-09-21T07:21:37.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.495+0000 7f465240d740 1 -- start start 2024-09-21T07:21:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f465240d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56853f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f465240d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c56483490 0x563c56854440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f465240d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5648b670 0x563c56854980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f465240d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563c567c9e90 con 0x563c56483490 2024-09-21T07:21:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f465240d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563c567b92c0 con 0x563c5648b670 2024-09-21T07:21:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f465240d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563c567bd0d0 con 0x563c564ac350 2024-09-21T07:21:37.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f4642000640 1 -- 172.21.15.27:0/3154222772 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4634042070 con 0x7f4644105bb0 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f4649600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56853f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f4648c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c56483490 0x563c56854440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f4649600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56853f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52222/0 (socket says 172.21.15.27:52222) 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f4649600640 1 -- 172.21.15.27:0/273751312 learned_addr learned my addr 172.21.15.27:0/273751312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.496+0000 7f463be00640 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5648b670 0x563c56854980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.497+0000 7f4649600640 1 -- 172.21.15.27:0/273751312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5648b670 msgr2=0x563c56854980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.497+0000 7f4649600640 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5648b670 0x563c56854980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.497+0000 7f4649600640 1 -- 172.21.15.27:0/273751312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c56483490 msgr2=0x563c56854440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.497+0000 7f4649600640 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c56483490 0x563c56854440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.497+0000 7f4649600640 1 -- 172.21.15.27:0/273751312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c56517930 con 0x563c564ac350 2024-09-21T07:21:37.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.497+0000 7f4649600640 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56853f00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f463c009f90 tx=0x7f463c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731b2aa6acaf45f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.497+0000 7f4639c00640 1 -- 172.21.15.27:0/273751312 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c011700 con 0x563c564ac350 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.498+0000 7f4639c00640 1 -- 172.21.15.27:0/273751312 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f463c011d50 con 0x563c564ac350 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.498+0000 7f4639c00640 1 -- 172.21.15.27:0/273751312 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c019d40 con 0x563c564ac350 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.498+0000 7f4642000640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f46280b6680 0x7f46280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.498+0000 7f4642000640 1 -- 172.21.15.27:0/3154222772 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f4634108390 con 0x7f4644105bb0 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.498+0000 7f4648c00640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f46280b6680 0x7f46280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.498+0000 7f465240d740 1 -- 172.21.15.27:0/273751312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 msgr2=0x563c56853f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.498+0000 7f465240d740 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56853f00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f463c009f90 tx=0x7f463c010860 comp rx=0 tx=0).stop 2024-09-21T07:21:37.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f465240d740 1 -- 172.21.15.27:0/273751312 shutdown_connections 2024-09-21T07:21:37.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f465240d740 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c5648b670 0x563c56854980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f465240d740 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c56483490 0x563c56854440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f465240d740 1 --2- 172.21.15.27:0/273751312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56853f00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f465240d740 1 -- 172.21.15.27:0/273751312 >> 172.21.15.27:0/273751312 conn(0x563c564a0740 msgr2=0x563c56851df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:37.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f465240d740 1 -- 172.21.15.27:0/273751312 shutdown_connections 2024-09-21T07:21:37.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f465240d740 1 -- 172.21.15.27:0/273751312 wait complete. 2024-09-21T07:21:37.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.499+0000 7f4648c00640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f46280b6680 0x7f46280b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f463410bfd0 tx=0x7f463410be40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 -- start start 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 Processor -- start 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c5685fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5648b670 0x563c56860000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c564ac350 0x563c5686b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563c5661a890 con 0x563c564ac350 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563c56754160 con 0x563c56483490 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.500+0000 7f465240d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563c568626e0 con 0x563c5648b670 2024-09-21T07:21:37.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f465240d740 10 client.0 _ll_register_callbacks cb 0x563c56770a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f4649600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c5685fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f4649600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c564ac350 0x563c5686b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5648b670 0x563c56860000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5648b670 0x563c56860000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52236/0 (socket says 172.21.15.27:52236) 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 -- 172.21.15.27:0/1035524944 learned_addr learned my addr 172.21.15.27:0/1035524944 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 -- 172.21.15.27:0/1035524944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 msgr2=0x563c5685fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c5685fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 -- 172.21.15.27:0/1035524944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c564ac350 msgr2=0x563c5686b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c564ac350 0x563c5686b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f463be00640 1 -- 172.21.15.27:0/1035524944 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5686bfc0 con 0x563c5648b670 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.501+0000 7f4649600640 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c564ac350 0x563c5686b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f463be00640 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5648b670 0x563c56860000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f462c00bf50 tx=0x7f462c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c734909a7b3438fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f4632000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f4632000640 1 -- 172.21.15.27:0/1035524944 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c004020 con 0x563c5648b670 2024-09-21T07:21:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f4632000640 1 -- 172.21.15.27:0/1035524944 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f462c0041c0 con 0x563c5648b670 2024-09-21T07:21:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f463c011d50 con 0x563c5648b670 2024-09-21T07:21:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f4632000640 1 -- 172.21.15.27:0/1035524944 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f462c024050 con 0x563c5648b670 2024-09-21T07:21:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f465240d740 10 client.27991 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.502+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c5686e8d0 con 0x563c5648b670 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f4632000640 1 -- 172.21.15.27:0/1035524944 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f463c011d50 con 0x563c5648b670 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f4632000640 1 -- 172.21.15.27:0/1035524944 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f462c003790 con 0x563c5648b670 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f465240d740 10 client.27991 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f465240d740 10 client.27991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f465240d740 20 client.27991 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f465240d740 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c5646acd0 0x563c56871ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f465240d740 4 client.27991 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.503+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c567cbde0 con 0x563c5646acd0 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.504+0000 7f4649600640 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c5646acd0 0x563c56871ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.504+0000 7f4649600640 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c5646acd0 0x563c56871ee0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef8d21f0f72dea85 server_cookie=f32a38dae5504bde in_seq=0 out_seq=0 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.504+0000 7f4632000640 10 client.27991 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:37.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f4632000640 1 -- 172.21.15.27:0/1035524944 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c567cbde0 con 0x563c5646acd0 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f4632000640 10 client.27991 handle_command_reply: tid=0 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 client.27991 shutdown 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5648b670 msgr2=0x563c56860000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5648b670 0x563c56860000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f462c00bf50 tx=0x7f462c0118d0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 shutdown_connections 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c5646acd0 0x563c56871ee0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c5648b670 0x563c56860000 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c564ac350 0x563c5686b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 --2- 172.21.15.27:0/1035524944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c5685fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 >> 172.21.15.27:0/1035524944 conn(0x563c564a0740 msgr2=0x563c568570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.505+0000 7f4632000640 0 client.27991 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.506+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 shutdown_connections 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.506+0000 7f465240d740 1 -- 172.21.15.27:0/1035524944 wait complete. 2024-09-21T07:21:37.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.506+0000 7f465240d740 20 client.27991 trim_cache size 0 max 16384 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 Processor -- start 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 -- start start 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c56483490 0x563c5686b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c5685f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c564ac350 0x563c568614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b5830 con 0x563c5648b670 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b3100 con 0x563c564ac350 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.525+0000 7f465240d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b0f70 con 0x563c56483490 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c5685f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c5685f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39302/0 (socket says 172.21.15.27:39302) 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 -- 172.21.15.27:0/1471665771 learned_addr learned my addr 172.21.15.27:0/1471665771 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f4649600640 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c56483490 0x563c5686b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f4649600640 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c564ac350 0x563c568614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 -- 172.21.15.27:0/1471665771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c56483490 msgr2=0x563c5686b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c56483490 0x563c5686b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 -- 172.21.15.27:0/1471665771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c564ac350 msgr2=0x563c568614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c564ac350 0x563c568614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 -- 172.21.15.27:0/1471665771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5685b150 con 0x563c5648b670 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f4649600640 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c56483490 0x563c5686b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.526+0000 7f463be00640 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c5685f150 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f462c003c30 tx=0x7f462c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e593fa2b95ce50b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.527+0000 7f4639200640 1 -- 172.21.15.27:0/1471665771 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c02c070 con 0x563c5648b670 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.527+0000 7f4639200640 1 -- 172.21.15.27:0/1471665771 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462c005040 con 0x563c5648b670 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.527+0000 7f4639200640 1 -- 172.21.15.27:0/1471665771 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0042e0 con 0x563c5648b670 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 -- 172.21.15.27:0/1471665771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 msgr2=0x563c5685f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c5685f150 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f462c003c30 tx=0x7f462c005850 comp rx=0 tx=0).stop 2024-09-21T07:21:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 -- 172.21.15.27:0/1471665771 shutdown_connections 2024-09-21T07:21:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c564ac350 0x563c568614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c5685f150 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 --2- 172.21.15.27:0/1471665771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c56483490 0x563c5686b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 -- 172.21.15.27:0/1471665771 >> 172.21.15.27:0/1471665771 conn(0x563c564a0740 msgr2=0x563c56860de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 -- 172.21.15.27:0/1471665771 shutdown_connections 2024-09-21T07:21:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 -- 172.21.15.27:0/1471665771 wait complete. 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.528+0000 7f465240d740 1 -- start start 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 1 Processor -- start 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c56868220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c568773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56877a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b15c0 con 0x563c5648b670 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b2a30 con 0x563c56483490 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b1c90 con 0x563c564ac350 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f465240d740 10 client.0 _ll_register_callbacks cb 0x7f462c00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f463be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c568773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f463be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c568773b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39318/0 (socket says 172.21.15.27:39318) 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.529+0000 7f463be00640 1 -- 172.21.15.27:0/4041666642 learned_addr learned my addr 172.21.15.27:0/4041666642 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f4649600640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c56868220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f4649600640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56877a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f463be00640 1 -- 172.21.15.27:0/4041666642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 msgr2=0x563c56877a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f463be00640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56877a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f463be00640 1 -- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 msgr2=0x563c56868220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f463be00640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c56868220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f463be00640 1 -- 172.21.15.27:0/4041666642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c56878190 con 0x563c5648b670 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f4649600640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56877a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f463be00640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c568773b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f462c003f00 tx=0x7f462c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4d2e5cad0b69c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.530+0000 7f4632000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.531+0000 7f4632000640 1 -- 172.21.15.27:0/4041666642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c02c030 con 0x563c5648b670 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.531+0000 7f4632000640 1 -- 172.21.15.27:0/4041666642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f462c003f60 con 0x563c5648b670 2024-09-21T07:21:37.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.531+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462c005040 con 0x563c5648b670 2024-09-21T07:21:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.531+0000 7f4632000640 1 -- 172.21.15.27:0/4041666642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f462c043050 con 0x563c5648b670 2024-09-21T07:21:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.531+0000 7f465240d740 10 client.18012 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.531+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c568dc900 con 0x563c5648b670 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.531+0000 7f4632000640 1 -- 172.21.15.27:0/4041666642 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462c005040 con 0x563c5648b670 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f4632000640 1 -- 172.21.15.27:0/4041666642 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f462c004660 con 0x563c5648b670 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f465240d740 10 client.18012 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f465240d740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f465240d740 20 client.18012 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f465240d740 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c56440080 0x563c568df060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f465240d740 4 client.18012 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c567c7890 con 0x563c56440080 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.532+0000 7f4649600640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c56440080 0x563c568df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.533+0000 7f4649600640 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c56440080 0x563c568df060 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6925121a5d3a8f server_cookie=cd3925aa10ce4423 in_seq=0 out_seq=0 2024-09-21T07:21:37.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.533+0000 7f4632000640 10 client.18012 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:37.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.534+0000 7f4632000640 1 -- 172.21.15.27:0/4041666642 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c567c7890 con 0x563c56440080 2024-09-21T07:21:37.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.534+0000 7f4632000640 10 client.18012 handle_command_reply: tid=0 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.534+0000 7f465240d740 1 client.18012 shutdown 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 msgr2=0x563c568773b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c568773b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f462c003f00 tx=0x7f462c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 shutdown_connections 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563c56440080 0x563c568df060 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563c564ac350 0x563c56877a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c5648b670 0x563c568773b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 --2- 172.21.15.27:0/4041666642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c56483490 0x563c56868220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 >> 172.21.15.27:0/4041666642 conn(0x563c564a0740 msgr2=0x563c56860d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f4632000640 0 client.18012 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 shutdown_connections 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 1 -- 172.21.15.27:0/4041666642 wait complete. 2024-09-21T07:21:37.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.535+0000 7f465240d740 20 client.18012 trim_cache size 0 max 16384 2024-09-21T07:21:37.539 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:37.539 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:21:37.539 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "277625f4-5a50-4dd0-9451-0842adf228cf", 2024-09-21T07:21:37.539 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:21:37.539 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:37.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f46280b6680 msgr2=0x7f46280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f46280b6680 0x7f46280b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f463410bfd0 tx=0x7f463410be40 comp rx=0 tx=0).stop 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 msgr2=0x7f464419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f464419c540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4634005ab0 tx=0x7f4634003cd0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 shutdown_connections 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f464410b4f0 0x7f464419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f46280b6680 0x7f46280b8b90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f464410be40 0x7f46441a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 --2- 172.21.15.27:0/3154222772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4644105bb0 0x7f464419c540 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 >> 172.21.15.27:0/3154222772 conn(0x7f4644101680 msgr2=0x7f4644103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 shutdown_connections 2024-09-21T07:21:37.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:37.536+0000 7f464a000640 1 -- 172.21.15.27:0/3154222772 wait complete. 2024-09-21T07:21:38.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:21:38.809 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:38 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:21:39.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:38 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:39.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:38 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:21:39.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:38 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:39.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:38 smithi119 ceph-mon[34519]: pgmap v642: 129 pgs: 129 active+clean; 2.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:21:39.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:38 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:21:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:38 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:21:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:38 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:38 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:21:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:38 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:38 smithi027 ceph-mon[30779]: pgmap v642: 129 pgs: 129 active+clean; 2.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:21:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:38 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:21:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:38 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:21:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:38 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:38 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:21:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:38 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:39.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:38 smithi153 ceph-mon[34747]: pgmap v642: 129 pgs: 129 active+clean; 2.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:21:39.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:38 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.449+0000 7f60a0e00640 1 -- 172.21.15.27:0/3329590482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c10b4f0 msgr2=0x7f609c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.449+0000 7f60a0e00640 1 --2- 172.21.15.27:0/3329590482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c10b4f0 0x7f609c10b900 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6088009900 tx=0x7f608802a410 comp rx=0 tx=0).stop 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.449+0000 7f60a0e00640 1 -- 172.21.15.27:0/3329590482 shutdown_connections 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.449+0000 7f60a0e00640 1 --2- 172.21.15.27:0/3329590482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 0x7f609c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.449+0000 7f60a0e00640 1 --2- 172.21.15.27:0/3329590482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c10b4f0 0x7f609c10b900 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.449+0000 7f60a0e00640 1 --2- 172.21.15.27:0/3329590482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f609c105bb0 0x7f609c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.449+0000 7f60a0e00640 1 -- 172.21.15.27:0/3329590482 >> 172.21.15.27:0/3329590482 conn(0x7f609c101680 msgr2=0x7f609c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.450+0000 7f60a0e00640 1 -- 172.21.15.27:0/3329590482 shutdown_connections 2024-09-21T07:21:39.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.450+0000 7f60a0e00640 1 -- 172.21.15.27:0/3329590482 wait complete. 2024-09-21T07:21:39.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 Processor -- start 2024-09-21T07:21:39.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 -- start start 2024-09-21T07:21:39.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c105bb0 0x7f609c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f609c10b4f0 0x7f609c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 0x7f609c1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f609c115b00 con 0x7f609c105bb0 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f609c115980 con 0x7f609c10be40 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.451+0000 7f60a0e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f609c115c80 con 0x7f609c10b4f0 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 0x7f609c1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 0x7f609c1a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47980/0 (socket says 172.21.15.27:47980) 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 -- 172.21.15.27:0/1596564924 learned_addr learned my addr 172.21.15.27:0/1596564924 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609b400640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c105bb0 0x7f609c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609aa00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f609c10b4f0 0x7f609c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 -- 172.21.15.27:0/1596564924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f609c10b4f0 msgr2=0x7f609c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f609c10b4f0 0x7f609c19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 -- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c105bb0 msgr2=0x7f609c19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c105bb0 0x7f609c19c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609be00640 1 -- 172.21.15.27:0/1596564924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c1a2440 con 0x7f609c10be40 2024-09-21T07:21:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.452+0000 7f609b400640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c105bb0 0x7f609c19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.453+0000 7f609be00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 0x7f609c1a1d10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f609400eaa0 tx=0x7f609400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.453+0000 7f6093e00640 1 -- 172.21.15.27:0/1596564924 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609400cdb0 con 0x7f609c10be40 2024-09-21T07:21:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.453+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c1a2730 con 0x7f609c10be40 2024-09-21T07:21:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.453+0000 7f6093e00640 1 -- 172.21.15.27:0/1596564924 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6094005020 con 0x7f609c10be40 2024-09-21T07:21:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.453+0000 7f6093e00640 1 -- 172.21.15.27:0/1596564924 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940116d0 con 0x7f609c10be40 2024-09-21T07:21:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.453+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c1a8e90 con 0x7f609c10be40 2024-09-21T07:21:39.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.456+0000 7f60a91c4740 1 Processor -- start 2024-09-21T07:21:39.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.457+0000 7f6093e00640 1 -- 172.21.15.27:0/1596564924 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f60940051d0 con 0x7f609c10be40 2024-09-21T07:21:39.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.457+0000 7f60a91c4740 1 -- start start 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934342cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934343230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934343770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f60a91c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5599342b9f70 con 0x559933f9c070 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f60a91c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5599342a93a0 con 0x559933f7b390 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f60a91c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5599342ad1b0 con 0x559933f731b0 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f609aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934342cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f609aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934342cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41106/0 (socket says 172.21.15.27:41106) 2024-09-21T07:21:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f609aa00640 1 -- 172.21.15.27:0/3464952535 learned_addr learned my addr 172.21.15.27:0/3464952535 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f609b400640 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934343230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.458+0000 7f609aa00640 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934343770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f609b400640 1 -- 172.21.15.27:0/3464952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 msgr2=0x559934343770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f609b400640 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934343770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f609b400640 1 -- 172.21.15.27:0/3464952535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 msgr2=0x559934342cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f609b400640 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934342cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f609b400640 1 -- 172.21.15.27:0/3464952535 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559934273c90 con 0x559933f731b0 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f6093e00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60800b6680 0x7f60800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f609b400640 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934343230 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f608c009fd0 tx=0x7f608c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa01ba3806dfae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f6093e00640 1 -- 172.21.15.27:0/1596564924 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f60940e7c80 con 0x7f609c10be40 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f609aa00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60800b6680 0x7f60800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.459+0000 7f6090c00640 1 -- 172.21.15.27:0/3464952535 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c004310 con 0x559933f731b0 2024-09-21T07:21:39.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.460+0000 7f6090c00640 1 -- 172.21.15.27:0/3464952535 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c0044b0 con 0x559933f731b0 2024-09-21T07:21:39.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.460+0000 7f6090c00640 1 -- 172.21.15.27:0/3464952535 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c004e50 con 0x559933f731b0 2024-09-21T07:21:39.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.460+0000 7f60a91c4740 1 -- 172.21.15.27:0/3464952535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 msgr2=0x559934343230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.460+0000 7f609aa00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60800b6680 0x7f60800b8b90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f60880096c0 tx=0x7f6088000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:39.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.460+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934343230 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f608c009fd0 tx=0x7f608c0118d0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 -- 172.21.15.27:0/3464952535 shutdown_connections 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934343230 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934343770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3464952535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934342cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 -- 172.21.15.27:0/3464952535 >> 172.21.15.27:0/3464952535 conn(0x559933f90460 msgr2=0x559934340b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 -- 172.21.15.27:0/3464952535 shutdown_connections 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 -- 172.21.15.27:0/3464952535 wait complete. 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.461+0000 7f60a91c4740 1 -- start start 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 1 Processor -- start 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f731b0 0x55993434e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x55993434eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f9c070 0x55993435a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55993410a700 con 0x559933f731b0 2024-09-21T07:21:39.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559934244540 con 0x559933f7b390 2024-09-21T07:21:39.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5599343514d0 con 0x559933f9c070 2024-09-21T07:21:39.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f60a91c4740 10 client.0 _ll_register_callbacks cb 0x559934254430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:39.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.462+0000 7f609b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f731b0 0x55993434e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x55993434eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f731b0 0x55993434e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41112/0 (socket says 172.21.15.27:41112) 2024-09-21T07:21:39.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609b400640 1 -- 172.21.15.27:0/2716765063 learned_addr learned my addr 172.21.15.27:0/2716765063 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:39.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609be00640 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f9c070 0x55993435a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609aa00640 1 -- 172.21.15.27:0/2716765063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f9c070 msgr2=0x55993435a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609aa00640 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f9c070 0x55993435a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609aa00640 1 -- 172.21.15.27:0/2716765063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f731b0 msgr2=0x55993434e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609aa00640 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f731b0 0x55993434e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609aa00640 1 -- 172.21.15.27:0/2716765063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559934007650 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f609aa00640 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x55993434eda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f60880057c0 tx=0x7f6088031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9539bb21adac883 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.463+0000 7f6085600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.464+0000 7f6085600640 1 -- 172.21.15.27:0/2716765063 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608804f020 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.464+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559934246b50 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.464+0000 7f6085600640 1 -- 172.21.15.27:0/2716765063 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f6088005030 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.464+0000 7f6085600640 1 -- 172.21.15.27:0/2716765063 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f608805a050 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.464+0000 7f60a91c4740 10 client.28058 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.464+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559934261910 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.464+0000 7f6085600640 1 -- 172.21.15.27:0/2716765063 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559934246b50 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f6085600640 1 -- 172.21.15.27:0/2716765063 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f60880035f0 con 0x559933f7b390 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f60a91c4740 10 client.28058 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f60a91c4740 10 client.28058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f60a91c4740 20 client.28058 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f60a91c4740 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f5a9f0 0x559934360810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f60a91c4740 4 client.28058 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:39.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599342bbec0 con 0x559933f5a9f0 2024-09-21T07:21:39.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.465+0000 7f609b400640 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f5a9f0 0x559934360810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.466+0000 7f609b400640 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f5a9f0 0x559934360810 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d88d6596f48958 server_cookie=c5ecde696c4ba282 in_seq=0 out_seq=0 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.466+0000 7f6085600640 10 client.28058 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f6085600640 1 -- 172.21.15.27:0/2716765063 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599342bbec0 con 0x559933f5a9f0 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f6085600640 10 client.28058 handle_command_reply: tid=0 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 client.28058 shutdown 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 msgr2=0x55993434eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x55993434eda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f60880057c0 tx=0x7f6088031400 comp rx=0 tx=0).stop 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 shutdown_connections 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f5a9f0 0x559934360810 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f9c070 0x55993435a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x55993434eda0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 --2- 172.21.15.27:0/2716765063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f731b0 0x55993434e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 >> 172.21.15.27:0/2716765063 conn(0x559933f90460 msgr2=0x559934345ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:39.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f6085600640 0 client.28058 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:39.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 shutdown_connections 2024-09-21T07:21:39.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.467+0000 7f60a91c4740 1 -- 172.21.15.27:0/2716765063 wait complete. 2024-09-21T07:21:39.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.468+0000 7f60a91c4740 20 client.28058 trim_cache size 0 max 16384 2024-09-21T07:21:39.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 Processor -- start 2024-09-21T07:21:39.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 -- start start 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f731b0 0x55993433fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f7b390 0x559934344430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x55993435a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b56a0 con 0x559933f9c070 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b2f70 con 0x559933f731b0 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.486+0000 7f60a91c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b0de0 con 0x559933f7b390 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f731b0 0x55993433fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f7b390 0x559934344430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f731b0 0x55993433fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48016/0 (socket says 172.21.15.27:48016) 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 -- 172.21.15.27:0/3102397051 learned_addr learned my addr 172.21.15.27:0/3102397051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:39.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609be00640 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x55993435a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 -- 172.21.15.27:0/3102397051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f7b390 msgr2=0x559934344430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f7b390 0x559934344430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 -- 172.21.15.27:0/3102397051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 msgr2=0x55993435a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x55993435a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.487+0000 7f609b400640 1 -- 172.21.15.27:0/3102397051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559934344fe0 con 0x559933f731b0 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.488+0000 7f609b400640 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f731b0 0x55993433fb80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f608c004a60 tx=0x7f608c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=390f47c35026527 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.488+0000 7f6087e00640 1 -- 172.21.15.27:0/3102397051 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c009b30 con 0x559933f731b0 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.488+0000 7f6087e00640 1 -- 172.21.15.27:0/3102397051 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c004260 con 0x559933f731b0 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.488+0000 7f6087e00640 1 -- 172.21.15.27:0/3102397051 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c00f740 con 0x559933f731b0 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 -- 172.21.15.27:0/3102397051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f731b0 msgr2=0x55993433fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f731b0 0x55993433fb80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f608c004a60 tx=0x7f608c005f90 comp rx=0 tx=0).stop 2024-09-21T07:21:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 -- 172.21.15.27:0/3102397051 shutdown_connections 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f7b390 0x559934344430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x55993435a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3102397051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f731b0 0x55993433fb80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 -- 172.21.15.27:0/3102397051 >> 172.21.15.27:0/3102397051 conn(0x559933f90460 msgr2=0x559934343d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 -- 172.21.15.27:0/3102397051 shutdown_connections 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.489+0000 7f60a91c4740 1 -- 172.21.15.27:0/3102397051 wait complete. 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.490+0000 7f60a91c4740 1 -- start start 2024-09-21T07:21:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.490+0000 7f60a91c4740 1 Processor -- start 2024-09-21T07:21:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934356c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934351740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f60a91c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934351c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f60a91c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b1430 con 0x559933f9c070 2024-09-21T07:21:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f60a91c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b28a0 con 0x559933f7b390 2024-09-21T07:21:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f60a91c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b1b00 con 0x559933f731b0 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f60a91c4740 10 client.0 _ll_register_callbacks cb 0x7f6088031800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934351740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934351c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934356c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934351740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48024/0 (socket says 172.21.15.27:48024) 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 -- 172.21.15.27:0/3916417067 learned_addr learned my addr 172.21.15.27:0/3916417067 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 -- 172.21.15.27:0/3916417067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 msgr2=0x559934356c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934356c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 -- 172.21.15.27:0/3916417067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 msgr2=0x559934351c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934351c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.491+0000 7f609aa00640 1 -- 172.21.15.27:0/3916417067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599343521c0 con 0x559933f7b390 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f609aa00640 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934351740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f608802a450 tx=0x7f60880043d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4248031c8269c653 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f6085600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:39.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f6085600640 1 -- 172.21.15.27:0/3916417067 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608804f040 con 0x559933f7b390 2024-09-21T07:21:39.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c004260 con 0x559933f7b390 2024-09-21T07:21:39.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f6085600640 1 -- 172.21.15.27:0/3916417067 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f6088031b10 con 0x559933f7b390 2024-09-21T07:21:39.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f6085600640 1 -- 172.21.15.27:0/3916417067 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f608806c050 con 0x559933f7b390 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f60a91c4740 10 client.28064 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.492+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599343cbe50 con 0x559933f7b390 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f6085600640 1 -- 172.21.15.27:0/3916417067 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c004260 con 0x559933f7b390 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f6085600640 1 -- 172.21.15.27:0/3916417067 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f60880046c0 con 0x559933f7b390 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f60a91c4740 10 client.28064 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f60a91c4740 10 client.28064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f60a91c4740 20 client.28064 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f2fda0 0x5599343ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f60a91c4740 4 client.28064 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.493+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599342b7970 con 0x559933f2fda0 2024-09-21T07:21:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.494+0000 7f609b400640 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f2fda0 0x5599343ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.494+0000 7f609b400640 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f2fda0 0x5599343ce690 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5323eca06a79e3c6 server_cookie=f1c5f4632f4b17c0 in_seq=0 out_seq=0 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.494+0000 7f6085600640 10 client.28064 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.494+0000 7f6085600640 1 -- 172.21.15.27:0/3916417067 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x5599342b7970 con 0x559933f2fda0 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.494+0000 7f6085600640 10 client.28064 handle_command_reply: tid=0 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.494+0000 7f60a91c4740 1 client.28064 shutdown 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 msgr2=0x559934351740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934351740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f608802a450 tx=0x7f60880043d0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 shutdown_connections 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559933f2fda0 0x5599343ce690 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559933f9c070 0x559934351c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559933f7b390 0x559934351740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 --2- 172.21.15.27:0/3916417067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559933f731b0 0x559934356c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f6085600640 0 client.28064 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 >> 172.21.15.27:0/3916417067 conn(0x559933f90460 msgr2=0x55993433f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 shutdown_connections 2024-09-21T07:21:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 1 -- 172.21.15.27:0/3916417067 wait complete. 2024-09-21T07:21:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.495+0000 7f60a91c4740 20 client.28064 trim_cache size 0 max 16384 2024-09-21T07:21:39.500 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:39.500 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:21:39.500 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:21:39.500 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:21:39.500 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:21:39.500 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.501 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:21:39.503 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.504 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:21:39.505 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.506 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:39.508 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60800b6680 msgr2=0x7f60800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60800b6680 0x7f60800b8b90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f60880096c0 tx=0x7f6088000bc0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 msgr2=0x7f609c1a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 0x7f609c1a1d10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f609400eaa0 tx=0x7f609400ef60 comp rx=0 tx=0).stop 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 shutdown_connections 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f609c10b4f0 0x7f609c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f60800b6680 0x7f60800b8b90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f609c10be40 0x7f609c1a1d10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 --2- 172.21.15.27:0/1596564924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f609c105bb0 0x7f609c19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 >> 172.21.15.27:0/1596564924 conn(0x7f609c101680 msgr2=0x7f609c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 shutdown_connections 2024-09-21T07:21:39.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:39.496+0000 7f60a0e00640 1 -- 172.21.15.27:0/1596564924 wait complete. 2024-09-21T07:21:40.290 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:21:40.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:21:40.675 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:41.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:40 smithi027 ceph-mon[30779]: pgmap v643: 129 pgs: 129 active+clean; 2.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:21:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:40 smithi119 ceph-mon[34519]: pgmap v643: 129 pgs: 129 active+clean; 2.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:21:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:40 smithi153 ceph-mon[34747]: pgmap v643: 129 pgs: 129 active+clean; 2.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-21T07:21:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 -- 172.21.15.27:0/68805329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10b500 msgr2=0x7fc5cc10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/68805329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10b500 0x7fc5cc10b910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009900 tx=0x7fc5b402a410 comp rx=0 tx=0).stop 2024-09-21T07:21:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 -- 172.21.15.27:0/68805329 shutdown_connections 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/68805329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc10be50 0x7fc5cc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/68805329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10b500 0x7fc5cc10b910 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/68805329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 0x7fc5cc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 -- 172.21.15.27:0/68805329 >> 172.21.15.27:0/68805329 conn(0x7fc5cc101670 msgr2=0x7fc5cc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 -- 172.21.15.27:0/68805329 shutdown_connections 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.303+0000 7fc5d1c00640 1 -- 172.21.15.27:0/68805329 wait complete. 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.304+0000 7fc5d1c00640 1 Processor -- start 2024-09-21T07:21:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.304+0000 7fc5d1c00640 1 -- start start 2024-09-21T07:21:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 0x7fc5cc19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc10b500 0x7fc5cc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10be50 0x7fc5cc1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc115b60 con 0x7fc5cc10be50 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc1159e0 con 0x7fc5cc10b500 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc115ce0 con 0x7fc5cc105bc0 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10be50 0x7fc5cc1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5cbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 0x7fc5cc19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5c3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc10b500 0x7fc5cc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5cbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 0x7fc5cc19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41812/0 (socket says 172.21.15.27:41812) 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5cbe00640 1 -- 172.21.15.27:0/2596964330 learned_addr learned my addr 172.21.15.27:0/2596964330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.305+0000 7fc5d1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10be50 0x7fc5cc1a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41152/0 (socket says 172.21.15.27:41152) 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5cbe00640 1 -- 172.21.15.27:0/2596964330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc10b500 msgr2=0x7fc5cc19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5cbe00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc10b500 0x7fc5cc19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5cbe00640 1 -- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10be50 msgr2=0x7fc5cc1a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5cbe00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10be50 0x7fc5cc1a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5cbe00640 1 -- 172.21.15.27:0/2596964330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc1a24c0 con 0x7fc5cc105bc0 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5d1200640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10be50 0x7fc5cc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5cbe00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 0x7fc5cc19c550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00e9f0 tx=0x7fc5bc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.306+0000 7fc5c9600640 1 -- 172.21.15.27:0/2596964330 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00cd80 con 0x7fc5cc105bc0 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.307+0000 7fc5c9600640 1 -- 172.21.15.27:0/2596964330 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5bc0054d0 con 0x7fc5cc105bc0 2024-09-21T07:21:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.307+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc1a27b0 con 0x7fc5cc105bc0 2024-09-21T07:21:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.307+0000 7fc5c9600640 1 -- 172.21.15.27:0/2596964330 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc010690 con 0x7fc5cc105bc0 2024-09-21T07:21:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.307+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc1a8f10 con 0x7fc5cc105bc0 2024-09-21T07:21:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.308+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc105fd0 con 0x7fc5cc105bc0 2024-09-21T07:21:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.310+0000 7fc5c9600640 1 -- 172.21.15.27:0/2596964330 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc5bc004e30 con 0x7fc5cc105bc0 2024-09-21T07:21:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.312+0000 7fc5c9600640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc5b00b6680 0x7fc5b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.312+0000 7fc5c9600640 1 -- 172.21.15.27:0/2596964330 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fc5bc0e80e0 con 0x7fc5cc105bc0 2024-09-21T07:21:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.312+0000 7fc5c3400640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc5b00b6680 0x7fc5b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.313+0000 7fc5c3400640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc5b00b6680 0x7fc5b00b8b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5b402aa00 tx=0x7fc5b4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.314+0000 7fc5c9600640 1 -- 172.21.15.27:0/2596964330 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc5bc0ae220 con 0x7fc5cc105bc0 2024-09-21T07:21:41.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.555+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5cc10dc80 con 0x7fc5cc105bc0 2024-09-21T07:21:41.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.556+0000 7fc5c9600640 1 -- 172.21.15.27:0/2596964330 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fc5bc0b2600 con 0x7fc5cc105bc0 2024-09-21T07:21:41.558 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:21:41.558 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:21:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:21:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc5b00b6680 msgr2=0x7fc5b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc5b00b6680 0x7fc5b00b8b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5b402aa00 tx=0x7fc5b4005a20 comp rx=0 tx=0).stop 2024-09-21T07:21:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 msgr2=0x7fc5cc19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:41.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 0x7fc5cc19c550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00e9f0 tx=0x7fc5bc00edf0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 shutdown_connections 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc5b00b6680 0x7fc5b00b8b90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5cc10be50 0x7fc5cc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc10b500 0x7fc5cc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 --2- 172.21.15.27:0/2596964330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5cc105bc0 0x7fc5cc19c550 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 >> 172.21.15.27:0/2596964330 conn(0x7fc5cc101670 msgr2=0x7fc5cc103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 shutdown_connections 2024-09-21T07:21:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:41.558+0000 7fc5d1c00640 1 -- 172.21.15.27:0/2596964330 wait complete. 2024-09-21T07:21:41.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2596964330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2596964330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2596964330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:21:42.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:21:42.718 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:43.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:42 smithi027 ceph-mon[30779]: pgmap v644: 129 pgs: 129 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-21T07:21:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:42 smithi119 ceph-mon[34519]: pgmap v644: 129 pgs: 129 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-21T07:21:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:42 smithi153 ceph-mon[34747]: pgmap v644: 129 pgs: 129 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-21T07:21:43.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 -- 172.21.15.27:0/3596790290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 msgr2=0x7f8d50116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 --2- 172.21.15.27:0/3596790290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 0x7f8d50116990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009ca0 tx=0x7f8d4802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 -- 172.21.15.27:0/3596790290 shutdown_connections 2024-09-21T07:21:43.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 --2- 172.21.15.27:0/3596790290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 0x7f8d50116990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 --2- 172.21.15.27:0/3596790290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50078290 0x7f8d50114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 --2- 172.21.15.27:0/3596790290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d50077940 0x7f8d50077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 -- 172.21.15.27:0/3596790290 >> 172.21.15.27:0/3596790290 conn(0x7f8d50101680 msgr2=0x7f8d50103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:43.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 -- 172.21.15.27:0/3596790290 shutdown_connections 2024-09-21T07:21:43.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.308+0000 7f8d56600640 1 -- 172.21.15.27:0/3596790290 wait complete. 2024-09-21T07:21:43.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.309+0000 7f8d56600640 1 Processor -- start 2024-09-21T07:21:43.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.309+0000 7f8d56600640 1 -- start start 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d56600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50077940 0x7f8d501a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d56600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d50078290 0x7f8d501a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d56600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 0x7f8d501a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d56600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d50119ed0 con 0x7f8d50114570 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d56600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d50119d50 con 0x7f8d50077940 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d56600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5011a050 con 0x7f8d50078290 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d55200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50077940 0x7f8d501a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d55c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 0x7f8d501a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d55200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50077940 0x7f8d501a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48062/0 (socket says 172.21.15.27:48062) 2024-09-21T07:21:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d55200640 1 -- 172.21.15.27:0/3980858821 learned_addr learned my addr 172.21.15.27:0/3980858821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.310+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d50078290 0x7f8d501a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d55200640 1 -- 172.21.15.27:0/3980858821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d50078290 msgr2=0x7f8d501a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d55200640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d50078290 0x7f8d501a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d55200640 1 -- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 msgr2=0x7f8d501a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d55200640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 0x7f8d501a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d55200640 1 -- 172.21.15.27:0/3980858821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d501a67f0 con 0x7f8d50077940 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d55200640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50077940 0x7f8d501a0880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400e9f0 tx=0x7f8d4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d4d600640 1 -- 172.21.15.27:0/3980858821 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400cd80 con 0x7f8d50077940 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d501a6ae0 con 0x7f8d50077940 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d4d600640 1 -- 172.21.15.27:0/3980858821 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d440108e0 con 0x7f8d50077940 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d4d600640 1 -- 172.21.15.27:0/3980858821 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44018980 con 0x7f8d50077940 2024-09-21T07:21:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.311+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d501ad240 con 0x7f8d50077940 2024-09-21T07:21:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 Processor -- start 2024-09-21T07:21:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 -- start start 2024-09-21T07:21:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddf08070 0x55d4de2aed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddee7390 0x55d4de2af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d4de225fb0 con 0x55d4ddf08070 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d4de2153e0 con 0x55d4ddedf1b0 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d4de2191f0 con 0x55d4ddee7390 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.315+0000 7f8d55c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddf08070 0x55d4de2aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d55c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddf08070 0x55d4de2aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41168/0 (socket says 172.21.15.27:41168) 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d55c00640 1 -- 172.21.15.27:0/623150047 learned_addr learned my addr 172.21.15.27:0/623150047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d4d600640 1 -- 172.21.15.27:0/3980858821 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8d44010430 con 0x7f8d50077940 2024-09-21T07:21:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddee7390 0x55d4de2af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d55c00640 1 -- 172.21.15.27:0/623150047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddee7390 msgr2=0x55d4de2af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d55c00640 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddee7390 0x55d4de2af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d55c00640 1 -- 172.21.15.27:0/623150047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 msgr2=0x55d4de2af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d55c00640 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.316+0000 7f8d55c00640 1 -- 172.21.15.27:0/623150047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4de1dfcd0 con 0x55d4ddf08070 2024-09-21T07:21:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.317+0000 7f8d55c00640 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddf08070 0x55d4de2aed30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8d4802f410 tx=0x7f8d4802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9094c273ebec3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.317+0000 7f8d42000640 1 -- 172.21.15.27:0/623150047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48031cf0 con 0x55d4ddf08070 2024-09-21T07:21:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.317+0000 7f8d42000640 1 -- 172.21.15.27:0/623150047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d48036030 con 0x55d4ddf08070 2024-09-21T07:21:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.317+0000 7f8d42000640 1 -- 172.21.15.27:0/623150047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d480042a0 con 0x55d4ddf08070 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d4d600640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d380b6680 0x7f8d380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/623150047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddf08070 msgr2=0x55d4de2aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddf08070 0x55d4de2aed30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8d4802f410 tx=0x7f8d4802f440 comp rx=0 tx=0).stop 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d4d600640 1 -- 172.21.15.27:0/3980858821 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f8d440e7e60 con 0x7f8d50077940 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/623150047 shutdown_connections 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddee7390 0x55d4de2af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/623150047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddf08070 0x55d4de2aed30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/623150047 >> 172.21.15.27:0/623150047 conn(0x55d4ddefc460 msgr2=0x55d4de2acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d55200640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d380b6680 0x7f8d380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/623150047 shutdown_connections 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.318+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/623150047 wait complete. 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.319+0000 7f8d55200640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d380b6680 0x7f8d380b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55d4de2ad9b0 tx=0x7f8d4400cf20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.319+0000 7f8d5e8b1740 1 -- start start 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.319+0000 7f8d5e8b1740 1 Processor -- start 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2baff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d4de076700 con 0x55d4ddedf1b0 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d4de1b0580 con 0x55d4ddee7390 2024-09-21T07:21:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d4de2bd330 con 0x55d4ddf08070 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d5e8b1740 10 client.0 _ll_register_callbacks cb 0x55d4de1c0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d55c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2baff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d55c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2ba9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41174/0 (socket says 172.21.15.27:41174) 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d55c00640 1 -- 172.21.15.27:0/3137066851 learned_addr learned my addr 172.21.15.27:0/3137066851 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.320+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d4fe00640 1 -- 172.21.15.27:0/3137066851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 msgr2=0x55d4de2c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d4fe00640 1 -- 172.21.15.27:0/3137066851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 msgr2=0x55d4de2ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d4fe00640 1 -- 172.21.15.27:0/3137066851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ddf73650 con 0x55d4ddee7390 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:43.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2baff0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8d5006c630 tx=0x7f8d3c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310808f248e76331 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d36a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d36a00640 1 -- 172.21.15.27:0/3137066851 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00f890 con 0x55d4ddee7390 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d4de1b2b90 con 0x55d4ddee7390 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.321+0000 7f8d36a00640 1 -- 172.21.15.27:0/3137066851 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f8d3c00fa30 con 0x55d4ddee7390 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.322+0000 7f8d36a00640 1 -- 172.21.15.27:0/3137066851 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d3c00fc30 con 0x55d4ddee7390 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.322+0000 7f8d5e8b1740 10 client.28079 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.322+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d4de1cd950 con 0x55d4ddee7390 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.322+0000 7f8d36a00640 1 -- 172.21.15.27:0/3137066851 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d4de1b2b90 con 0x55d4ddee7390 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.322+0000 7f8d36a00640 1 -- 172.21.15.27:0/3137066851 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d3c01c420 con 0x55d4ddee7390 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d5e8b1740 10 client.28079 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d5e8b1740 10 client.28079 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d5e8b1740 20 client.28079 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4ddec69f0 0x55d4de2cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d5e8b1740 4 client.28079 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:43.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4de227f00 con 0x55d4ddec69f0 2024-09-21T07:21:43.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4ddec69f0 0x55d4de2cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4ddec69f0 0x55d4de2cc860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcf7d2e0b08f8b30 server_cookie=a0e34d1c522c5050 in_seq=0 out_seq=0 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.323+0000 7f8d36a00640 10 client.28079 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.324+0000 7f8d36a00640 1 -- 172.21.15.27:0/3137066851 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4de227f00 con 0x55d4ddec69f0 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.324+0000 7f8d36a00640 10 client.28079 handle_command_reply: tid=0 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 client.28079 shutdown 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 msgr2=0x55d4de2baff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2baff0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8d5006c630 tx=0x7f8d3c012b60 comp rx=0 tx=0).stop 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 shutdown_connections 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4ddec69f0 0x55d4de2cc860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2baff0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d36a00640 0 client.28079 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3137066851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 >> 172.21.15.27:0/3137066851 conn(0x55d4ddefc460 msgr2=0x55d4de2afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 shutdown_connections 2024-09-21T07:21:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3137066851 wait complete. 2024-09-21T07:21:43.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.325+0000 7f8d5e8b1740 20 client.28079 trim_cache size 0 max 16384 2024-09-21T07:21:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.339+0000 7f8d5e8b1740 1 Processor -- start 2024-09-21T07:21:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.339+0000 7f8d5e8b1740 1 -- start start 2024-09-21T07:21:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2bb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d501b9ad0 con 0x55d4ddedf1b0 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d501b73a0 con 0x55d4ddee7390 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d501b5210 con 0x55d4ddf08070 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d55c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2bb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d55c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2bb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41190/0 (socket says 172.21.15.27:41190) 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d55c00640 1 -- 172.21.15.27:0/2546921301 learned_addr learned my addr 172.21.15.27:0/2546921301 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.340+0000 7f8d55c00640 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d55c00640 1 -- 172.21.15.27:0/2546921301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 msgr2=0x55d4de2b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d55c00640 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d55c00640 1 -- 172.21.15.27:0/2546921301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 msgr2=0x55d4de2b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d55c00640 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d55c00640 1 -- 172.21.15.27:0/2546921301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4de2b8cf0 con 0x55d4ddedf1b0 2024-09-21T07:21:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d55c00640 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2bb100 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005cc0 tx=0x7f8d4802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cde9c4fe257d70b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d41600640 1 -- 172.21.15.27:0/2546921301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48061070 con 0x55d4ddedf1b0 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d41600640 1 -- 172.21.15.27:0/2546921301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d48036040 con 0x55d4ddedf1b0 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.341+0000 7f8d41600640 1 -- 172.21.15.27:0/2546921301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4802f950 con 0x55d4ddedf1b0 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/2546921301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 msgr2=0x55d4de2bb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2bb100 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005cc0 tx=0x7f8d4802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/2546921301 shutdown_connections 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddee7390 0x55d4de2b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/2546921301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddedf1b0 0x55d4de2bb100 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/2546921301 >> 172.21.15.27:0/2546921301 conn(0x55d4ddefc460 msgr2=0x55d4de2ba9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/2546921301 shutdown_connections 2024-09-21T07:21:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.342+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/2546921301 wait complete. 2024-09-21T07:21:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.343+0000 7f8d5e8b1740 1 -- start start 2024-09-21T07:21:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.343+0000 7f8d5e8b1740 1 Processor -- start 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddee7390 0x55d4de2c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d5e8b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d501b5860 con 0x55d4ddee7390 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d501b6cd0 con 0x55d4ddedf1b0 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d5e8b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d501b5f30 con 0x55d4ddf08070 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d5e8b1740 10 client.0 _ll_register_callbacks cb 0x7f8d3c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddee7390 0x55d4de2c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddee7390 0x55d4de2c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41196/0 (socket says 172.21.15.27:41196) 2024-09-21T07:21:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d4fe00640 1 -- 172.21.15.27:0/3680499801 learned_addr learned my addr 172.21.15.27:0/3680499801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.344+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d4fe00640 1 -- 172.21.15.27:0/3680499801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 msgr2=0x55d4de2bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d4fe00640 1 -- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 msgr2=0x55d4de2c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d4fe00640 1 -- 172.21.15.27:0/3680499801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4de2bfe90 con 0x55d4ddee7390 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d4fe00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddee7390 0x55d4de2c2fa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c01ca80 tx=0x7f8d3c01c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cb4fbe9494d9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d36a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d36a00640 1 -- 172.21.15.27:0/3680499801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c02f060 con 0x55d4ddee7390 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d36a00640 1 -- 172.21.15.27:0/3680499801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f8d3c00f9a0 con 0x55d4ddee7390 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d48036040 con 0x55d4ddee7390 2024-09-21T07:21:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d36a00640 1 -- 172.21.15.27:0/3680499801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d3c0050d0 con 0x55d4ddee7390 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d5e8b1740 10 client.18054 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.345+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d4de337e60 con 0x55d4ddee7390 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d36a00640 1 -- 172.21.15.27:0/3680499801 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d48036040 con 0x55d4ddee7390 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d36a00640 1 -- 172.21.15.27:0/3680499801 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d3c003cb0 con 0x55d4ddee7390 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d5e8b1740 10 client.18054 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d5e8b1740 10 client.18054 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d5e8b1740 20 client.18054 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4dde9bda0 0x55d4de33a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d5e8b1740 4 client.18054 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:21:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4de2239b0 con 0x55d4dde9bda0 2024-09-21T07:21:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.346+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4dde9bda0 0x55d4de33a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.347+0000 7f8d55c00640 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4dde9bda0 0x55d4de33a680 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f12e022c4286d4c server_cookie=33ef3831ece88d06 in_seq=0 out_seq=0 2024-09-21T07:21:43.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.347+0000 7f8d36a00640 10 client.18054 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.347+0000 7f8d36a00640 1 -- 172.21.15.27:0/3680499801 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4de2239b0 con 0x55d4dde9bda0 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.347+0000 7f8d36a00640 10 client.18054 handle_command_reply: tid=0 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.347+0000 7f8d5e8b1740 1 client.18054 shutdown 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.347+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddee7390 msgr2=0x55d4de2c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.347+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddee7390 0x55d4de2c2fa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c01ca80 tx=0x7f8d3c01c6d0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 shutdown_connections 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55d4dde9bda0 0x55d4de33a680 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddf08070 0x55d4de2bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d4ddee7390 0x55d4de2c2fa0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 --2- 172.21.15.27:0/3680499801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d4ddedf1b0 0x55d4de2c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d36a00640 0 client.18054 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 >> 172.21.15.27:0/3680499801 conn(0x55d4ddefc460 msgr2=0x55d4de2afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 shutdown_connections 2024-09-21T07:21:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 1 -- 172.21.15.27:0/3680499801 wait complete. 2024-09-21T07:21:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d5e8b1740 20 client.18054 trim_cache size 0 max 16384 2024-09-21T07:21:43.353 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:21:43.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d380b6680 msgr2=0x7f8d380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d56600640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d380b6680 0x7f8d380b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55d4de2ad9b0 tx=0x7f8d4400cf20 comp rx=0 tx=0).stop 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50077940 msgr2=0x7f8d501a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.348+0000 7f8d56600640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50077940 0x7f8d501a0880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400e9f0 tx=0x7f8d4400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 shutdown_connections 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d50078290 0x7f8d501a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d380b6680 0x7f8d380b8b90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d50114570 0x7f8d501a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 --2- 172.21.15.27:0/3980858821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d50077940 0x7f8d501a0880 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 >> 172.21.15.27:0/3980858821 conn(0x7f8d50101680 msgr2=0x7f8d501165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 shutdown_connections 2024-09-21T07:21:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:43.349+0000 7f8d56600640 1 -- 172.21.15.27:0/3980858821 wait complete. 2024-09-21T07:21:44.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:21:44.416 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:44.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:44 smithi027 ceph-mon[30779]: pgmap v645: 129 pgs: 129 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-21T07:21:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:44 smithi119 ceph-mon[34519]: pgmap v645: 129 pgs: 129 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-21T07:21:45.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 -- 172.21.15.27:0/3071526114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8114570 msgr2=0x7efdf8116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 --2- 172.21.15.27:0/3071526114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8114570 0x7efdf8116990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009ca0 tx=0x7efdf002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 -- 172.21.15.27:0/3071526114 shutdown_connections 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 --2- 172.21.15.27:0/3071526114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf8078290 0x7efdf8114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 --2- 172.21.15.27:0/3071526114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8114570 0x7efdf8116990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 --2- 172.21.15.27:0/3071526114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf8077940 0x7efdf8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 -- 172.21.15.27:0/3071526114 >> 172.21.15.27:0/3071526114 conn(0x7efdf8101680 msgr2=0x7efdf8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 -- 172.21.15.27:0/3071526114 shutdown_connections 2024-09-21T07:21:45.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.042+0000 7efdfee00640 1 -- 172.21.15.27:0/3071526114 wait complete. 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.043+0000 7efdfee00640 1 Processor -- start 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.043+0000 7efdfee00640 1 -- start start 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8077940 0x7efdf81a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf8078290 0x7efdf81a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfee00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf8114570 0x7efdf81a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8119f40 con 0x7efdf8077940 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8119dc0 con 0x7efdf8078290 2024-09-21T07:21:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfee00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdf811a0c0 con 0x7efdf8114570 2024-09-21T07:21:45.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8077940 0x7efdf81a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8077940 0x7efdf81a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41220/0 (socket says 172.21.15.27:41220) 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfda00640 1 -- 172.21.15.27:0/1525123790 learned_addr learned my addr 172.21.15.27:0/1525123790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfd000640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf8078290 0x7efdf81a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.044+0000 7efdfe400640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf8114570 0x7efdf81a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdfda00640 1 -- 172.21.15.27:0/1525123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf8114570 msgr2=0x7efdf81a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdfda00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf8114570 0x7efdf81a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdfda00640 1 -- 172.21.15.27:0/1525123790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf8078290 msgr2=0x7efdf81a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdfda00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf8078290 0x7efdf81a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdfda00640 1 -- 172.21.15.27:0/1525123790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf81a68c0 con 0x7efdf8077940 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdfda00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8077940 0x7efdf81a0950 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efdec00ea60 tx=0x7efdec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdf6000640 1 -- 172.21.15.27:0/1525123790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec00cde0 con 0x7efdf8077940 2024-09-21T07:21:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdf6000640 1 -- 172.21.15.27:0/1525123790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec0035f0 con 0x7efdf8077940 2024-09-21T07:21:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdf6000640 1 -- 172.21.15.27:0/1525123790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec010690 con 0x7efdf8077940 2024-09-21T07:21:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.045+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf81a6bb0 con 0x7efdf8077940 2024-09-21T07:21:45.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.046+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf81ad310 con 0x7efdf8077940 2024-09-21T07:21:45.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 Processor -- start 2024-09-21T07:21:45.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 -- start start 2024-09-21T07:21:45.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efdf6000640 1 -- 172.21.15.27:0/1525123790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efdec0037a0 con 0x7efdf8077940 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f60f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5634b8ed7fb0 con 0x5634b8b99390 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5634b8ec73e0 con 0x5634b8b911b0 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.049+0000 7efe0704c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634b8ecb1f0 con 0x5634b8bba070 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.050+0000 7efdfd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.050+0000 7efdfd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41224/0 (socket says 172.21.15.27:41224) 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.050+0000 7efdfda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.050+0000 7efdfd000640 1 -- 172.21.15.27:0/1491869228 learned_addr learned my addr 172.21.15.27:0/1491869228 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.050+0000 7efdfe400640 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f60f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.050+0000 7efdfda00640 1 -- 172.21.15.27:0/1491869228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 msgr2=0x5634b8f60f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.050+0000 7efdfda00640 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f60f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdfda00640 1 -- 172.21.15.27:0/1491869228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 msgr2=0x5634b8f619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdfda00640 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdfda00640 1 -- 172.21.15.27:0/1491869228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634b8e91cd0 con 0x5634b8b911b0 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdfd000640 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdfe400640 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f60f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdfda00640 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f614b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e5170 tx=0x7efdec0f04b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863433c78198635f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdeaa00640 1 -- 172.21.15.27:0/1491869228 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0fb070 con 0x5634b8b911b0 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdf6000640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efde00b6700 0x7efde00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdeaa00640 1 -- 172.21.15.27:0/1491869228 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec0f68b0 con 0x5634b8b911b0 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.051+0000 7efdeaa00640 1 -- 172.21.15.27:0/1491869228 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a91c0 con 0x5634b8b911b0 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efdf6000640 1 -- 172.21.15.27:0/1525123790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7efdec0e7e30 con 0x7efdf8077940 2024-09-21T07:21:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efdfe400640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efde00b6700 0x7efde00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efe0704c740 1 -- 172.21.15.27:0/1491869228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 msgr2=0x5634b8f614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efe0704c740 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f614b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e5170 tx=0x7efdec0f04b0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efdfe400640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efde00b6700 0x7efde00b8c10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efdf0004af0 tx=0x7efdf0005da0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efe0704c740 1 -- 172.21.15.27:0/1491869228 shutdown_connections 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efe0704c740 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efe0704c740 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f614b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.052+0000 7efe0704c740 1 --2- 172.21.15.27:0/1491869228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f60f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.053+0000 7efe0704c740 1 -- 172.21.15.27:0/1491869228 >> 172.21.15.27:0/1491869228 conn(0x5634b8bae460 msgr2=0x5634b8f5ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.053+0000 7efe0704c740 1 -- 172.21.15.27:0/1491869228 shutdown_connections 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.053+0000 7efe0704c740 1 -- 172.21.15.27:0/1491869228 wait complete. 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.053+0000 7efe0704c740 1 -- start start 2024-09-21T07:21:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.053+0000 7efe0704c740 1 Processor -- start 2024-09-21T07:21:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efe0704c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b911b0 0x5634b8f6c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efe0704c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b99390 0x5634b8f6cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efe0704c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efe0704c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5634b8d28700 con 0x5634b8b911b0 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efe0704c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5634b8e62580 con 0x5634b8b99390 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efe0704c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634b8f6f750 con 0x5634b8bba070 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efe0704c740 10 client.0 _ll_register_callbacks cb 0x5634b8e72470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efdfda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b911b0 0x5634b8f6c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efdfd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b99390 0x5634b8f6cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efdfda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b911b0 0x5634b8f6c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41238/0 (socket says 172.21.15.27:41238) 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.054+0000 7efdfda00640 1 -- 172.21.15.27:0/1970893662 learned_addr learned my addr 172.21.15.27:0/1970893662 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfe400640 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfd000640 1 -- 172.21.15.27:0/1970893662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 msgr2=0x5634b8f78a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfd000640 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f78a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfd000640 1 -- 172.21.15.27:0/1970893662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b911b0 msgr2=0x5634b8f6c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfd000640 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b911b0 0x5634b8f6c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfd000640 1 -- 172.21.15.27:0/1970893662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634b8c25650 con 0x5634b8b99390 2024-09-21T07:21:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfe400640 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efdfd000640 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b99390 0x5634b8f6cbc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efde4012700 tx=0x7efde4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204ffcbb4573637f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efddf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.055+0000 7efddf400640 1 -- 172.21.15.27:0/1970893662 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4004020 con 0x5634b8b99390 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.056+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5634b8e64b90 con 0x5634b8b99390 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.056+0000 7efddf400640 1 -- 172.21.15.27:0/1970893662 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7efde40041c0 con 0x5634b8b99390 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.056+0000 7efddf400640 1 -- 172.21.15.27:0/1970893662 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efde4025050 con 0x5634b8b99390 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.056+0000 7efe0704c740 10 client.28097 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.056+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5634b8e7f950 con 0x5634b8b99390 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.056+0000 7efddf400640 1 -- 172.21.15.27:0/1970893662 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5634b8e64b90 con 0x5634b8b99390 2024-09-21T07:21:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efddf400640 1 -- 172.21.15.27:0/1970893662 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efde4003990 con 0x5634b8b99390 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efe0704c740 10 client.28097 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efe0704c740 10 client.28097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efe0704c740 20 client.28097 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efe0704c740 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b789f0 0x5634b8f7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efe0704c740 4 client.28097 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634b8ed9f00 con 0x5634b8b789f0 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.057+0000 7efdfda00640 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b789f0 0x5634b8f7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.058+0000 7efdfda00640 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b789f0 0x5634b8f7e890 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6ff3f940354ec46 server_cookie=f77c8103c0062727 in_seq=0 out_seq=0 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.058+0000 7efddf400640 10 client.28097 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.058+0000 7efddf400640 1 -- 172.21.15.27:0/1970893662 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5634b8ed9f00 con 0x5634b8b789f0 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.058+0000 7efddf400640 10 client.28097 handle_command_reply: tid=0 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.058+0000 7efe0704c740 1 client.28097 shutdown 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b99390 msgr2=0x5634b8f6cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b99390 0x5634b8f6cbc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efde4012700 tx=0x7efde4012bc0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 shutdown_connections 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b789f0 0x5634b8f7e890 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b99390 0x5634b8f6cbc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 --2- 172.21.15.27:0/1970893662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b911b0 0x5634b8f6c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efddf400640 0 client.28097 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 >> 172.21.15.27:0/1970893662 conn(0x5634b8bae460 msgr2=0x5634b8f5d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 shutdown_connections 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 1 -- 172.21.15.27:0/1970893662 wait complete. 2024-09-21T07:21:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.059+0000 7efe0704c740 20 client.28097 trim_cache size 0 max 16384 2024-09-21T07:21:45.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.073+0000 7efe0704c740 1 Processor -- start 2024-09-21T07:21:45.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.073+0000 7efe0704c740 1 -- start start 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efe0704c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8b911b0 0x5634b8f6e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efe0704c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f78970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efe0704c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8bba070 0x5634b8f69870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efe0704c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b9c30 con 0x5634b8b99390 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efe0704c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b7500 con 0x5634b8bba070 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efe0704c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b5370 con 0x5634b8b911b0 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efdfd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f78970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efdfd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f78970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41240/0 (socket says 172.21.15.27:41240) 2024-09-21T07:21:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efdfd000640 1 -- 172.21.15.27:0/2525441577 learned_addr learned my addr 172.21.15.27:0/2525441577 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efdfe400640 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8bba070 0x5634b8f69870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efdfda00640 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8b911b0 0x5634b8f6e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.074+0000 7efdfd000640 1 -- 172.21.15.27:0/2525441577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8b911b0 msgr2=0x5634b8f6e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdfd000640 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8b911b0 0x5634b8f6e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdfd000640 1 -- 172.21.15.27:0/2525441577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8bba070 msgr2=0x5634b8f69870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdfd000640 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8bba070 0x5634b8f69870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdfd000640 1 -- 172.21.15.27:0/2525441577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634b8f63260 con 0x5634b8b99390 2024-09-21T07:21:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdfd000640 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f78970 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efde4003db0 tx=0x7efde401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a5b913d19d20f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdea000640 1 -- 172.21.15.27:0/2525441577 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde40035f0 con 0x5634b8b99390 2024-09-21T07:21:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdea000640 1 -- 172.21.15.27:0/2525441577 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde400f920 con 0x5634b8b99390 2024-09-21T07:21:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.075+0000 7efdea000640 1 -- 172.21.15.27:0/2525441577 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4005290 con 0x5634b8b99390 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 -- 172.21.15.27:0/2525441577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 msgr2=0x5634b8f78970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f78970 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efde4003db0 tx=0x7efde401c470 comp rx=0 tx=0).stop 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 -- 172.21.15.27:0/2525441577 shutdown_connections 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8bba070 0x5634b8f69870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f78970 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 --2- 172.21.15.27:0/2525441577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8b911b0 0x5634b8f6e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 -- 172.21.15.27:0/2525441577 >> 172.21.15.27:0/2525441577 conn(0x5634b8bae460 msgr2=0x5634b8f6dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 -- 172.21.15.27:0/2525441577 shutdown_connections 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.076+0000 7efe0704c740 1 -- 172.21.15.27:0/2525441577 wait complete. 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.077+0000 7efe0704c740 1 -- start start 2024-09-21T07:21:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.077+0000 7efe0704c740 1 Processor -- start 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efe0704c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f74ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efe0704c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f6f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efe0704c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f6fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efe0704c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b59c0 con 0x5634b8b99390 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efe0704c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b6e30 con 0x5634b8b911b0 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efe0704c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b6090 con 0x5634b8bba070 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efe0704c740 10 client.0 _ll_register_callbacks cb 0x7efde4009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efdfd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f6f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efdfd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f6f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41246/0 (socket says 172.21.15.27:41246) 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efdfd000640 1 -- 172.21.15.27:0/2928485708 learned_addr learned my addr 172.21.15.27:0/2928485708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efdfda00640 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f74ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efdfe400640 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f6fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efdfd000640 1 -- 172.21.15.27:0/2928485708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 msgr2=0x5634b8f6fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.078+0000 7efdfd000640 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f6fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efdfd000640 1 -- 172.21.15.27:0/2928485708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 msgr2=0x5634b8f74ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efdfd000640 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f74ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efdfd000640 1 -- 172.21.15.27:0/2928485708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634b8f70490 con 0x5634b8b99390 2024-09-21T07:21:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efdfd000640 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f6f990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efde4012c00 tx=0x7efde400fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25287cd075fa45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efddf400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efddf400640 1 -- 172.21.15.27:0/2928485708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4003800 con 0x5634b8b99390 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efddf400640 1 -- 172.21.15.27:0/2928485708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7efde4018040 con 0x5634b8b99390 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.079+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde400f920 con 0x5634b8b99390 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efddf400640 1 -- 172.21.15.27:0/2928485708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efde403b050 con 0x5634b8b99390 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 10 client.18072 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5634b8fe9fc0 con 0x5634b8b99390 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efddf400640 1 -- 172.21.15.27:0/2928485708 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde400f920 con 0x5634b8b99390 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efddf400640 1 -- 172.21.15.27:0/2928485708 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efde401c6f0 con 0x5634b8b99390 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 10 client.18072 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 10 client.18072 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 20 client.18072 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b4dda0 0x5634b8fec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 4 client.18072 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634b8ed59b0 con 0x5634b8b4dda0 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.080+0000 7efdfda00640 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b4dda0 0x5634b8fec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.081+0000 7efdfda00640 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b4dda0 0x5634b8fec640 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2cbb37cfbbae01a9 server_cookie=3806c607a94c1315 in_seq=0 out_seq=0 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.081+0000 7efddf400640 10 client.18072 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.081+0000 7efddf400640 1 -- 172.21.15.27:0/2928485708 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634b8ed59b0 con 0x5634b8b4dda0 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.081+0000 7efddf400640 10 client.18072 handle_command_reply: tid=0 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.081+0000 7efe0704c740 1 client.18072 shutdown 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.081+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 msgr2=0x5634b8f6f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.081+0000 7efe0704c740 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f6f990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efde4012c00 tx=0x7efde400fd00 comp rx=0 tx=0).stop 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 shutdown_connections 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5634b8b4dda0 0x5634b8fec640 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634b8bba070 0x5634b8f6fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5634b8b99390 0x5634b8f6f990 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 --2- 172.21.15.27:0/2928485708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634b8b911b0 0x5634b8f74ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 >> 172.21.15.27:0/2928485708 conn(0x5634b8bae460 msgr2=0x5634b8f5d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efddf400640 0 client.18072 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 shutdown_connections 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 1 -- 172.21.15.27:0/2928485708 wait complete. 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.082+0000 7efe0704c740 20 client.18072 trim_cache size 0 max 16384 2024-09-21T07:21:45.086 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efde00b6700 msgr2=0x7efde00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efde00b6700 0x7efde00b8c10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efdf0004af0 tx=0x7efdf0005da0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8077940 msgr2=0x7efdf81a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8077940 0x7efdf81a0950 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efdec00ea60 tx=0x7efdec00ef20 comp rx=0 tx=0).stop 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 shutdown_connections 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdf8114570 0x7efdf81a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efde00b6700 0x7efde00b8c10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdf8078290 0x7efdf81a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 --2- 172.21.15.27:0/1525123790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdf8077940 0x7efdf81a0950 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 >> 172.21.15.27:0/1525123790 conn(0x7efdf8101680 msgr2=0x7efdf8116580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 shutdown_connections 2024-09-21T07:21:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:45.083+0000 7efdfee00640 1 -- 172.21.15.27:0/1525123790 wait complete. 2024-09-21T07:21:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:44 smithi153 ceph-mon[34747]: pgmap v645: 129 pgs: 129 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-21T07:21:45.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:21:46.341 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 -- 172.21.15.27:0/1795542919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 msgr2=0x7f04dc116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 --2- 172.21.15.27:0/1795542919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 0x7f04dc116920 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009ca0 tx=0x7f04d402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 -- 172.21.15.27:0/1795542919 shutdown_connections 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 --2- 172.21.15.27:0/1795542919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc075dd0 0x7f04dc0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 --2- 172.21.15.27:0/1795542919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 0x7f04dc116920 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 --2- 172.21.15.27:0/1795542919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc077380 0x7f04dc0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 -- 172.21.15.27:0/1795542919 >> 172.21.15.27:0/1795542919 conn(0x7f04dc0ff610 msgr2=0x7f04dc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 -- 172.21.15.27:0/1795542919 shutdown_connections 2024-09-21T07:21:47.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.011+0000 7f04e4400640 1 -- 172.21.15.27:0/1795542919 wait complete. 2024-09-21T07:21:47.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.012+0000 7f04e4400640 1 Processor -- start 2024-09-21T07:21:47.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e4400640 1 -- start start 2024-09-21T07:21:47.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e4400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc075dd0 0x7f04dc1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e4400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc077380 0x7f04dc1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e4400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 0x7f04dc1a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e4400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc11a0a0 con 0x7f04dc114450 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e4400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc119f20 con 0x7f04dc075dd0 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e4400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc11a220 con 0x7f04dc077380 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc075dd0 0x7f04dc1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc075dd0 0x7f04dc1a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48160/0 (socket says 172.21.15.27:48160) 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.013+0000 7f04e1a00640 1 -- 172.21.15.27:0/3252035216 learned_addr learned my addr 172.21.15.27:0/3252035216 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e1000640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc077380 0x7f04dc1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e2400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 0x7f04dc1a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e1a00640 1 -- 172.21.15.27:0/3252035216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc077380 msgr2=0x7f04dc1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc077380 0x7f04dc1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e1a00640 1 -- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 msgr2=0x7f04dc1a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 0x7f04dc1a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e1a00640 1 -- 172.21.15.27:0/3252035216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc1a69a0 con 0x7f04dc075dd0 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc075dd0 0x7f04dc1a0a30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9f0 tx=0x7f04d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.014+0000 7f04da000640 1 -- 172.21.15.27:0/3252035216 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d000cd80 con 0x7f04dc075dd0 2024-09-21T07:21:47.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.015+0000 7f04da000640 1 -- 172.21.15.27:0/3252035216 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d00108e0 con 0x7f04dc075dd0 2024-09-21T07:21:47.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.015+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc1a6c90 con 0x7f04dc075dd0 2024-09-21T07:21:47.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.015+0000 7f04da000640 1 -- 172.21.15.27:0/3252035216 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0018980 con 0x7f04dc075dd0 2024-09-21T07:21:47.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.015+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1ad3f0 con 0x7f04dc075dd0 2024-09-21T07:21:47.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 Processor -- start 2024-09-21T07:21:47.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 -- start start 2024-09-21T07:21:47.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec661da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2921b0 0x5595ec6622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec662820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04da000640 1 -- 172.21.15.27:0/3252035216 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f04d0010430 con 0x7f04dc075dd0 2024-09-21T07:21:47.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5595ec5d8fb0 con 0x5595ec2921b0 2024-09-21T07:21:47.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5595ec5c83e0 con 0x5595ec29a390 2024-09-21T07:21:47.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.018+0000 7f04eb097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595ec5cc1f0 con 0x5595ec2bb070 2024-09-21T07:21:47.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec661da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec661da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41932/0 (socket says 172.21.15.27:41932) 2024-09-21T07:21:47.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 -- 172.21.15.27:0/1417287219 learned_addr learned my addr 172.21.15.27:0/1417287219 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:47.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e1a00640 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2921b0 0x5595ec6622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e1000640 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec662820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 -- 172.21.15.27:0/1417287219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 msgr2=0x5595ec662820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec662820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 -- 172.21.15.27:0/1417287219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2921b0 msgr2=0x5595ec6622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2921b0 0x5595ec6622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e2400640 1 -- 172.21.15.27:0/1417287219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ec592cd0 con 0x5595ec2bb070 2024-09-21T07:21:47.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.019+0000 7f04e1000640 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec662820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:47.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.020+0000 7f04e2400640 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec661da0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005cc0 tx=0x7f04d402f420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4e4f35cb2e56fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:47.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.020+0000 7f04cea00640 1 -- 172.21.15.27:0/1417287219 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4031cf0 con 0x5595ec2bb070 2024-09-21T07:21:47.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.020+0000 7f04cea00640 1 -- 172.21.15.27:0/1417287219 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d4036040 con 0x5595ec2bb070 2024-09-21T07:21:47.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.020+0000 7f04cea00640 1 -- 172.21.15.27:0/1417287219 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4004040 con 0x5595ec2bb070 2024-09-21T07:21:47.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 -- 172.21.15.27:0/1417287219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 msgr2=0x5595ec661da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec661da0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005cc0 tx=0x7f04d402f420 comp rx=0 tx=0).stop 2024-09-21T07:21:47.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04da000640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04c40b6680 0x7f04c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04da000640 1 -- 172.21.15.27:0/3252035216 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f04d00e7f10 con 0x7f04dc075dd0 2024-09-21T07:21:47.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 -- 172.21.15.27:0/1417287219 shutdown_connections 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec662820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2921b0 0x5595ec6622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 --2- 172.21.15.27:0/1417287219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec661da0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 -- 172.21.15.27:0/1417287219 >> 172.21.15.27:0/1417287219 conn(0x5595ec2af460 msgr2=0x5595ec65fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04e2400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04c40b6680 0x7f04c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.021+0000 7f04eb097740 1 -- 172.21.15.27:0/1417287219 shutdown_connections 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.022+0000 7f04eb097740 1 -- 172.21.15.27:0/1417287219 wait complete. 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.022+0000 7f04e2400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04c40b6680 0x7f04c40b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f04d4003ab0 tx=0x7f04d403e040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.022+0000 7f04eb097740 1 -- start start 2024-09-21T07:21:47.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.022+0000 7f04eb097740 1 Processor -- start 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04eb097740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec66d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04eb097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec29a390 0x5595ec66db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04eb097740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec679810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04eb097740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5595ec429700 con 0x5595ec2bb070 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04eb097740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5595ec563580 con 0x5595ec2921b0 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04eb097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595ec670520 con 0x5595ec29a390 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04eb097740 10 client.0 _ll_register_callbacks cb 0x5595ec573470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04e2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec679810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04e1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec66d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04e2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec679810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41264/0 (socket says 172.21.15.27:41264) 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04e2400640 1 -- 172.21.15.27:0/3148555406 learned_addr learned my addr 172.21.15.27:0/3148555406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.023+0000 7f04e1000640 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec29a390 0x5595ec66db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.024+0000 7f04e2400640 1 -- 172.21.15.27:0/3148555406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec29a390 msgr2=0x5595ec66db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.024+0000 7f04e2400640 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec29a390 0x5595ec66db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.024+0000 7f04e2400640 1 -- 172.21.15.27:0/3148555406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 msgr2=0x5595ec66d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.024+0000 7f04e2400640 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec66d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.024+0000 7f04e2400640 1 -- 172.21.15.27:0/3148555406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ec326650 con 0x5595ec2bb070 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.024+0000 7f04e2400640 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec679810 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f04d4045000 tx=0x7f04d4003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71748dde704b9a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.024+0000 7f04c3400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.025+0000 7f04c3400640 1 -- 172.21.15.27:0/3148555406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d404a040 con 0x5595ec2bb070 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.025+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5595ec565b90 con 0x5595ec2bb070 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.025+0000 7f04c3400640 1 -- 172.21.15.27:0/3148555406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f04d402f660 con 0x5595ec2bb070 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.025+0000 7f04c3400640 1 -- 172.21.15.27:0/3148555406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04d4064050 con 0x5595ec2bb070 2024-09-21T07:21:47.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.025+0000 7f04eb097740 10 client.18084 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:47.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.025+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595ec580950 con 0x5595ec2bb070 2024-09-21T07:21:47.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.025+0000 7f04c3400640 1 -- 172.21.15.27:0/3148555406 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5595ec565b90 con 0x5595ec2bb070 2024-09-21T07:21:47.029 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:46 smithi027 ceph-mon[30779]: pgmap v646: 129 pgs: 129 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:21:47.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04c3400640 1 -- 172.21.15.27:0/3148555406 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f04d404d030 con 0x5595ec2bb070 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04eb097740 10 client.18084 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04eb097740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04eb097740 20 client.18084 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04eb097740 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec2799f0 0x5595ec67f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04eb097740 4 client.18084 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595ec5daf00 con 0x5595ec2799f0 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.026+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec2799f0 0x5595ec67f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.027+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec2799f0 0x5595ec67f510 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e6cd6c2fd9b53c server_cookie=ce65495cfdd53015 in_seq=0 out_seq=0 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.027+0000 7f04c3400640 10 client.18084 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.027+0000 7f04c3400640 1 -- 172.21.15.27:0/3148555406 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5595ec5daf00 con 0x5595ec2799f0 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.027+0000 7f04c3400640 10 client.18084 handle_command_reply: tid=0 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 client.18084 shutdown 2024-09-21T07:21:47.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 msgr2=0x5595ec679810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec679810 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f04d4045000 tx=0x7f04d4003b10 comp rx=0 tx=0).stop 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 shutdown_connections 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec2799f0 0x5595ec67f510 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec29a390 0x5595ec66db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec679810 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 --2- 172.21.15.27:0/3148555406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec66d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 >> 172.21.15.27:0/3148555406 conn(0x5595ec2af460 msgr2=0x5595ec664f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04c3400640 0 client.18084 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 shutdown_connections 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 1 -- 172.21.15.27:0/3148555406 wait complete. 2024-09-21T07:21:47.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.028+0000 7f04eb097740 20 client.18084 trim_cache size 0 max 16384 2024-09-21T07:21:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:46 smithi119 ceph-mon[34519]: pgmap v646: 129 pgs: 129 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:21:47.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.042+0000 7f04eb097740 1 Processor -- start 2024-09-21T07:21:47.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04eb097740 1 -- start start 2024-09-21T07:21:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04eb097740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec65fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04eb097740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec29a390 0x5595ec679780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04eb097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec679cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04eb097740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1b9d20 con 0x5595ec29a390 2024-09-21T07:21:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04eb097740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1b75f0 con 0x5595ec2921b0 2024-09-21T07:21:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04eb097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1b5460 con 0x5595ec2bb070 2024-09-21T07:21:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04e1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec65fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04e1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec29a390 0x5595ec679780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04e1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec65fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48188/0 (socket says 172.21.15.27:48188) 2024-09-21T07:21:47.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04e1a00640 1 -- 172.21.15.27:0/3721004377 learned_addr learned my addr 172.21.15.27:0/3721004377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.043+0000 7f04e2400640 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec679cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04e1a00640 1 -- 172.21.15.27:0/3721004377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 msgr2=0x5595ec679cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec679cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04e1a00640 1 -- 172.21.15.27:0/3721004377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec29a390 msgr2=0x5595ec679780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec29a390 0x5595ec679780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04e1a00640 1 -- 172.21.15.27:0/3721004377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ec66cdc0 con 0x5595ec2921b0 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04e1000640 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec29a390 0x5595ec679780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04e1a00640 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec65fcc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04d00aabc0 tx=0x7f04d000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2813e3f7c39e545 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.044+0000 7f04ce000640 1 -- 172.21.15.27:0/3721004377 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00ad720 con 0x5595ec2921b0 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04ce000640 1 -- 172.21.15.27:0/3721004377 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d0010720 con 0x5595ec2921b0 2024-09-21T07:21:47.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04ce000640 1 -- 172.21.15.27:0/3721004377 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00a90c0 con 0x5595ec2921b0 2024-09-21T07:21:47.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04eb097740 1 -- 172.21.15.27:0/3721004377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 msgr2=0x5595ec65fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04eb097740 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec65fcc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04d00aabc0 tx=0x7f04d000cf20 comp rx=0 tx=0).stop 2024-09-21T07:21:47.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04eb097740 1 -- 172.21.15.27:0/3721004377 shutdown_connections 2024-09-21T07:21:47.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04eb097740 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2bb070 0x5595ec679cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04eb097740 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec29a390 0x5595ec679780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04eb097740 1 --2- 172.21.15.27:0/3721004377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec2921b0 0x5595ec65fcc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.045+0000 7f04eb097740 1 -- 172.21.15.27:0/3721004377 >> 172.21.15.27:0/3721004377 conn(0x5595ec2af460 msgr2=0x5595ec65f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:47.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.046+0000 7f04eb097740 1 -- 172.21.15.27:0/3721004377 shutdown_connections 2024-09-21T07:21:47.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.046+0000 7f04eb097740 1 -- 172.21.15.27:0/3721004377 wait complete. 2024-09-21T07:21:47.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.046+0000 7f04eb097740 1 -- start start 2024-09-21T07:21:47.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.046+0000 7f04eb097740 1 Processor -- start 2024-09-21T07:21:47.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04eb097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2921b0 0x5595ec680780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04eb097740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec680cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04eb097740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec681240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04eb097740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1b5ab0 con 0x5595ec2bb070 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04eb097740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1b6f20 con 0x5595ec29a390 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04eb097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1b6180 con 0x5595ec2921b0 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04eb097740 10 client.0 _ll_register_callbacks cb 0x7f04d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec680cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec680cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48204/0 (socket says 172.21.15.27:48204) 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 -- 172.21.15.27:0/2469160802 learned_addr learned my addr 172.21.15.27:0/2469160802 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e2400640 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec681240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1a00640 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2921b0 0x5595ec680780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 -- 172.21.15.27:0/2469160802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2921b0 msgr2=0x5595ec680780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2921b0 0x5595ec680780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 -- 172.21.15.27:0/2469160802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 msgr2=0x5595ec681240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec681240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.047+0000 7f04e1000640 1 -- 172.21.15.27:0/2469160802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ec6e8ee0 con 0x5595ec29a390 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04e1000640 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec680cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04c8012700 tx=0x7f04c8012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=446ae6c07afc81ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04c3400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04c3400640 1 -- 172.21.15.27:0/2469160802 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8022020 con 0x5595ec29a390 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5595ec5d69b0 con 0x5595ec29a390 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04c3400640 1 -- 172.21.15.27:0/2469160802 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f04c800fb20 con 0x5595ec29a390 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04c3400640 1 -- 172.21.15.27:0/2469160802 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04c800fd20 con 0x5595ec29a390 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04eb097740 10 client.28121 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.048+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595ec6ea210 con 0x5595ec29a390 2024-09-21T07:21:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04c3400640 1 -- 172.21.15.27:0/2469160802 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5595ec5d69b0 con 0x5595ec29a390 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04c3400640 1 -- 172.21.15.27:0/2469160802 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f04c8004a70 con 0x5595ec29a390 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04eb097740 10 client.28121 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04eb097740 10 client.28121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04eb097740 20 client.28121 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04eb097740 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec24eda0 0x5595ec6ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04eb097740 4 client.28121 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.049+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595ec5d3de0 con 0x5595ec24eda0 2024-09-21T07:21:47.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04e1a00640 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec24eda0 0x5595ec6ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04e1a00640 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec24eda0 0x5595ec6ec9d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b185fc3ce25f66 server_cookie=522b2e9ed95831d8 in_seq=0 out_seq=0 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04c3400640 10 client.28121 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04c3400640 1 -- 172.21.15.27:0/2469160802 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595ec5d3de0 con 0x5595ec24eda0 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04c3400640 10 client.28121 handle_command_reply: tid=0 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04eb097740 1 client.28121 shutdown 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 msgr2=0x5595ec680cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.050+0000 7f04eb097740 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec680cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04c8012700 tx=0x7f04c8012bc0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 shutdown_connections 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5595ec24eda0 0x5595ec6ec9d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5595ec2bb070 0x5595ec681240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595ec29a390 0x5595ec680cc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04c3400640 0 client.28121 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 --2- 172.21.15.27:0/2469160802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595ec2921b0 0x5595ec680780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 >> 172.21.15.27:0/2469160802 conn(0x5595ec2af460 msgr2=0x5595ec664ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:47.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 shutdown_connections 2024-09-21T07:21:47.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 1 -- 172.21.15.27:0/2469160802 wait complete. 2024-09-21T07:21:47.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.051+0000 7f04eb097740 20 client.28121 trim_cache size 0 max 16384 2024-09-21T07:21:47.056 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04c40b6680 msgr2=0x7f04c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04c40b6680 0x7f04c40b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f04d4003ab0 tx=0x7f04d403e040 comp rx=0 tx=0).stop 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc075dd0 msgr2=0x7f04dc1a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc075dd0 0x7f04dc1a0a30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9f0 tx=0x7f04d000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 shutdown_connections 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04dc077380 0x7f04dc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04c40b6680 0x7f04c40b8b90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04dc114450 0x7f04dc1a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 --2- 172.21.15.27:0/3252035216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04dc075dd0 0x7f04dc1a0a30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 >> 172.21.15.27:0/3252035216 conn(0x7f04dc0ff610 msgr2=0x7f04dc110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 shutdown_connections 2024-09-21T07:21:47.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:47.052+0000 7f04e4400640 1 -- 172.21.15.27:0/3252035216 wait complete. 2024-09-21T07:21:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:46 smithi153 ceph-mon[34747]: pgmap v646: 129 pgs: 129 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:21:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 -- 172.21.15.27:0/3645115190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 msgr2=0x7f7368103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 --2- 172.21.15.27:0/3645115190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f7368103d20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f735c009900 tx=0x7f735c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 -- 172.21.15.27:0/3645115190 shutdown_connections 2024-09-21T07:21:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 --2- 172.21.15.27:0/3645115190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7368109b90 0x7f73681103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 --2- 172.21.15.27:0/3645115190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7368109240 0x7f7368109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 --2- 172.21.15.27:0/3645115190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f7368103d20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 -- 172.21.15.27:0/3645115190 >> 172.21.15.27:0/3645115190 conn(0x7f73680ff330 msgr2=0x7f7368101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 -- 172.21.15.27:0/3645115190 shutdown_connections 2024-09-21T07:21:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.737+0000 7f736e600640 1 -- 172.21.15.27:0/3645115190 wait complete. 2024-09-21T07:21:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.738+0000 7f736e600640 1 Processor -- start 2024-09-21T07:21:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.738+0000 7f736e600640 1 -- start start 2024-09-21T07:21:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.738+0000 7f736e600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f736819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.738+0000 7f736e600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7368109240 0x7f736819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7368109b90 0x7f736819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736e600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73681138a0 con 0x7f7368103910 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7368113720 con 0x7f7368109b90 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736e600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7368113a20 con 0x7f7368109240 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f736819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f736819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56830/0 (socket says 172.21.15.27:56830) 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 -- 172.21.15.27:0/668176208 learned_addr learned my addr 172.21.15.27:0/668176208 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f7367e00640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7368109240 0x7f736819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736dc00640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7368109b90 0x7f736819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 -- 172.21.15.27:0/668176208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7368109240 msgr2=0x7f736819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7368109240 0x7f736819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 -- 172.21.15.27:0/668176208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7368109b90 msgr2=0x7f736819fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7368109b90 0x7f736819fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.739+0000 7f736d200640 1 -- 172.21.15.27:0/668176208 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73681a01d0 con 0x7f7368103910 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.740+0000 7f736d200640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f736819a2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f735c005ab0 tx=0x7f735c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.740+0000 7f7365600640 1 -- 172.21.15.27:0/668176208 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c03f070 con 0x7f7368103910 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.740+0000 7f7365600640 1 -- 172.21.15.27:0/668176208 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f735c031d60 con 0x7f7368103910 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.740+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73681a0460 con 0x7f7368103910 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.740+0000 7f7365600640 1 -- 172.21.15.27:0/668176208 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c035030 con 0x7f7368103910 2024-09-21T07:21:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.740+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73681a6c50 con 0x7f7368103910 2024-09-21T07:21:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.743+0000 7f7376c25740 1 Processor -- start 2024-09-21T07:21:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.743+0000 7f7376c25740 1 -- start start 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7365600640 1 -- 172.21.15.27:0/668176208 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f735c042070 con 0x7f7368103910 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7376c25740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff30e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7376c25740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff31350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7376c25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff31890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7376c25740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560e8fea66b0 con 0x560e8fb632f0 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7376c25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560e8fe95ae0 con 0x560e8fb6b4d0 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7376c25740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e8fe998f0 con 0x560e8fb8c1b0 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff31350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff31350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56844/0 (socket says 172.21.15.27:56844) 2024-09-21T07:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f736d200640 1 -- 172.21.15.27:0/3579627449 learned_addr learned my addr 172.21.15.27:0/3579627449 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f736dc00640 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff30e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.744+0000 7f7367e00640 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff31890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f736d200640 1 -- 172.21.15.27:0/3579627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 msgr2=0x560e8ff30e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f736d200640 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff30e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f736d200640 1 -- 172.21.15.27:0/3579627449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 msgr2=0x560e8ff31890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f736d200640 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff31890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f736d200640 1 -- 172.21.15.27:0/3579627449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e8fe62780 con 0x560e8fb632f0 2024-09-21T07:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f736d200640 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff31350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f735c105230 tx=0x7f735c110270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa77c1160e6ece68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f735a000640 1 -- 172.21.15.27:0/3579627449 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c11a470 con 0x560e8fb632f0 2024-09-21T07:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f735a000640 1 -- 172.21.15.27:0/3579627449 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f735c115070 con 0x560e8fb632f0 2024-09-21T07:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.745+0000 7f735a000640 1 -- 172.21.15.27:0/3579627449 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c108800 con 0x560e8fb632f0 2024-09-21T07:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7365600640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73500b6680 0x7f73500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7365600640 1 -- 172.21.15.27:0/668176208 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f735c107fb0 con 0x7f7368103910 2024-09-21T07:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f736dc00640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73500b6680 0x7f73500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 -- 172.21.15.27:0/3579627449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 msgr2=0x560e8ff31350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff31350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f735c105230 tx=0x7f735c110270 comp rx=0 tx=0).stop 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 -- 172.21.15.27:0/3579627449 shutdown_connections 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff31890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff31350 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 --2- 172.21.15.27:0/3579627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff30e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 -- 172.21.15.27:0/3579627449 >> 172.21.15.27:0/3579627449 conn(0x560e8fb80300 msgr2=0x560e8ff2ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.746+0000 7f7376c25740 1 -- 172.21.15.27:0/3579627449 shutdown_connections 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.747+0000 7f7376c25740 1 -- 172.21.15.27:0/3579627449 wait complete. 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.747+0000 7f736dc00640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73500b6680 0x7f73500b8b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7360005fd0 tx=0x7f7360009600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.747+0000 7f7376c25740 1 -- start start 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.747+0000 7f7376c25740 1 Processor -- start 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7376c25740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff40790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7376c25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff40cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7376c25740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7376c25740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560e8fdcdab0 con 0x560e8fb632f0 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7376c25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560e8fe3c570 con 0x560e8fb6b4d0 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7376c25740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e8ff43730 con 0x560e8fb8c1b0 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7376c25740 10 client.0 _ll_register_callbacks cb 0x560e8fe4b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff40790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f7367e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff40cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff40790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56852/0 (socket says 172.21.15.27:56852) 2024-09-21T07:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f736d200640 1 -- 172.21.15.27:0/3448321956 learned_addr learned my addr 172.21.15.27:0/3448321956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.748+0000 7f736dc00640 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7367e00640 1 -- 172.21.15.27:0/3448321956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 msgr2=0x560e8ff41210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7367e00640 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff41210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7367e00640 1 -- 172.21.15.27:0/3448321956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 msgr2=0x560e8ff40790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7367e00640 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff40790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7367e00640 1 -- 172.21.15.27:0/3448321956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e8fbf7790 con 0x560e8fb6b4d0 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7367e00640 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff40cd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f736806c400 tx=0x7f7354012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ebf7a113eeab68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f734ea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f734ea00640 1 -- 172.21.15.27:0/3448321956 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735400f890 con 0x560e8fb6b4d0 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f735c115070 con 0x560e8fb6b4d0 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f734ea00640 1 -- 172.21.15.27:0/3448321956 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f735400fa30 con 0x560e8fb6b4d0 2024-09-21T07:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f734ea00640 1 -- 172.21.15.27:0/3448321956 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f735400fc30 con 0x560e8fb6b4d0 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.749+0000 7f7376c25740 10 client.28133 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e8fc70620 con 0x560e8fb6b4d0 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f734ea00640 1 -- 172.21.15.27:0/3448321956 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f735c115070 con 0x560e8fb6b4d0 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f734ea00640 1 -- 172.21.15.27:0/3448321956 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f735401c420 con 0x560e8fb6b4d0 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f7376c25740 10 client.28133 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f7376c25740 10 client.28133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f7376c25740 20 client.28133 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f7376c25740 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb4ad80 0x560e8ff526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f7376c25740 4 client.28133 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e8fea8600 con 0x560e8fb4ad80 2024-09-21T07:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.750+0000 7f736d200640 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb4ad80 0x560e8ff526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.751+0000 7f736d200640 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb4ad80 0x560e8ff526b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c496ff12182f4b58 server_cookie=74da9fd9a700417a in_seq=0 out_seq=0 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.751+0000 7f734ea00640 10 client.28133 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f734ea00640 1 -- 172.21.15.27:0/3448321956 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e8fea8600 con 0x560e8fb4ad80 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f734ea00640 10 client.28133 handle_command_reply: tid=0 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 client.28133 shutdown 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 msgr2=0x560e8ff40cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff40cd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f736806c400 tx=0x7f7354012b60 comp rx=0 tx=0).stop 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 shutdown_connections 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb4ad80 0x560e8ff526b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ff41210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f734ea00640 0 client.28133 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff40cd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 --2- 172.21.15.27:0/3448321956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff40790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 >> 172.21.15.27:0/3448321956 conn(0x560e8fb80300 msgr2=0x560e8ff32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.752+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 shutdown_connections 2024-09-21T07:21:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.753+0000 7f7376c25740 1 -- 172.21.15.27:0/3448321956 wait complete. 2024-09-21T07:21:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.753+0000 7f7376c25740 20 client.28133 trim_cache size 0 max 16384 2024-09-21T07:21:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 Processor -- start 2024-09-21T07:21:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 -- start start 2024-09-21T07:21:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb632f0 0x560e8ff3bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb6b4d0 0x560e8ff32550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb8c1b0 0x560e8ff2e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73681b1410 con 0x560e8fb8c1b0 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73681aece0 con 0x560e8fb632f0 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7376c25740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73681acb50 con 0x560e8fb6b4d0 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f7367e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb6b4d0 0x560e8ff32550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.768+0000 7f736dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb8c1b0 0x560e8ff2e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb6b4d0 0x560e8ff32550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52978/0 (socket says 172.21.15.27:52978) 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 -- 172.21.15.27:0/3170710401 learned_addr learned my addr 172.21.15.27:0/3170710401 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f736d200640 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb632f0 0x560e8ff3bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 -- 172.21.15.27:0/3170710401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb632f0 msgr2=0x560e8ff3bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb632f0 0x560e8ff3bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 -- 172.21.15.27:0/3170710401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb8c1b0 msgr2=0x560e8ff2e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb8c1b0 0x560e8ff2e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 -- 172.21.15.27:0/3170710401 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e8ff33c70 con 0x560e8fb6b4d0 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7367e00640 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb6b4d0 0x560e8ff32550 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f735401ca80 tx=0x7f73540096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3008ef4bf509c48a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.769+0000 7f7359600640 1 -- 172.21.15.27:0/3170710401 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735401cbc0 con 0x560e8fb6b4d0 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7359600640 1 -- 172.21.15.27:0/3170710401 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7354005560 con 0x560e8fb6b4d0 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7359600640 1 -- 172.21.15.27:0/3170710401 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735400f9f0 con 0x560e8fb6b4d0 2024-09-21T07:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 -- 172.21.15.27:0/3170710401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb6b4d0 msgr2=0x560e8ff32550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb6b4d0 0x560e8ff32550 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f735401ca80 tx=0x7f73540096f0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 -- 172.21.15.27:0/3170710401 shutdown_connections 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb6b4d0 0x560e8ff32550 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb8c1b0 0x560e8ff2e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 --2- 172.21.15.27:0/3170710401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb632f0 0x560e8ff3bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 -- 172.21.15.27:0/3170710401 >> 172.21.15.27:0/3170710401 conn(0x560e8fb80300 msgr2=0x560e8ff31e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 -- 172.21.15.27:0/3170710401 shutdown_connections 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.770+0000 7f7376c25740 1 -- 172.21.15.27:0/3170710401 wait complete. 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.771+0000 7f7376c25740 1 -- start start 2024-09-21T07:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.771+0000 7f7376c25740 1 Processor -- start 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7376c25740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7376c25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7376c25740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ffbc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7376c25740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ad1a0 con 0x560e8fb632f0 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7376c25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ae610 con 0x560e8fb6b4d0 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7376c25740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ad870 con 0x560e8fb8c1b0 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7376c25740 10 client.0 _ll_register_callbacks cb 0x7f7354009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7367e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736dc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ffbc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff45080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56874/0 (socket says 172.21.15.27:56874) 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7367e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35344/0 (socket says 172.21.15.27:35344) 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 -- 172.21.15.27:0/3390221232 learned_addr learned my addr 172.21.15.27:0/3390221232 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 -- 172.21.15.27:0/3390221232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 msgr2=0x560e8ffbc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ffbc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 -- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 msgr2=0x560e8ff455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736d200640 1 -- 172.21.15.27:0/3390221232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e8ffbc5f0 con 0x560e8fb632f0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f736dc00640 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ffbc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.772+0000 7f7367e00640 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f736d200640 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff45080 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f735c1102b0 tx=0x7f735c031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9eea766df03ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f734ea00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f734ea00640 1 -- 172.21.15.27:0/3390221232 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c049040 con 0x560e8fb632f0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f734ea00640 1 -- 172.21.15.27:0/3390221232 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f735c03b4f0 con 0x560e8fb632f0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7354005560 con 0x560e8fb632f0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f734ea00640 1 -- 172.21.15.27:0/3390221232 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f735c0ca0f0 con 0x560e8fb632f0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f7376c25740 10 client.18108 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.773+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e8ffbd840 con 0x560e8fb632f0 2024-09-21T07:21:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f734ea00640 1 -- 172.21.15.27:0/3390221232 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7354005560 con 0x560e8fb632f0 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f734ea00640 1 -- 172.21.15.27:0/3390221232 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f735c04c060 con 0x560e8fb632f0 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f7376c25740 10 client.18108 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f7376c25740 10 client.18108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f7376c25740 20 client.18108 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f7376c25740 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb201c0 0x560e8ffbfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f7376c25740 4 client.18108 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e8fea40b0 con 0x560e8fb201c0 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.774+0000 7f7367e00640 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb201c0 0x560e8ffbfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.775+0000 7f7367e00640 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb201c0 0x560e8ffbfc20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5150d011dc00c39 server_cookie=a9ee3c0b0799b7ad in_seq=0 out_seq=0 2024-09-21T07:21:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.775+0000 7f734ea00640 10 client.18108 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f734ea00640 1 -- 172.21.15.27:0/3390221232 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e8fea40b0 con 0x560e8fb201c0 2024-09-21T07:21:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f734ea00640 10 client.18108 handle_command_reply: tid=0 2024-09-21T07:21:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 client.18108 shutdown 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 msgr2=0x560e8ff45080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff45080 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f735c1102b0 tx=0x7f735c031f10 comp rx=0 tx=0).stop 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 shutdown_connections 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e8fb201c0 0x560e8ffbfc20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e8fb8c1b0 0x560e8ffbc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e8fb6b4d0 0x560e8ff455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 --2- 172.21.15.27:0/3390221232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e8fb632f0 0x560e8ff45080 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f734ea00640 0 client.18108 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 >> 172.21.15.27:0/3390221232 conn(0x560e8fb80300 msgr2=0x560e8ff32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 shutdown_connections 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.791+0000 7f7376c25740 1 -- 172.21.15.27:0/3390221232 wait complete. 2024-09-21T07:21:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f7376c25740 20 client.18108 trim_cache size 0 max 16384 2024-09-21T07:21:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:21:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:21:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:21:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:21:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73500b6680 msgr2=0x7f73500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73500b6680 0x7f73500b8b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7360005fd0 tx=0x7f7360009600 comp rx=0 tx=0).stop 2024-09-21T07:21:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 msgr2=0x7f736819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f736819a2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f735c005ab0 tx=0x7f735c003c80 comp rx=0 tx=0).stop 2024-09-21T07:21:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 shutdown_connections 2024-09-21T07:21:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7368109240 0x7f736819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73500b6680 0x7f73500b8b90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7368109b90 0x7f736819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 --2- 172.21.15.27:0/668176208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368103910 0x7f736819a2a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.792+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 >> 172.21.15.27:0/668176208 conn(0x7f73680ff330 msgr2=0x7f7368101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.793+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 shutdown_connections 2024-09-21T07:21:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:21:48.793+0000 7f736e600640 1 -- 172.21.15.27:0/668176208 wait complete. 2024-09-21T07:21:48.981 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:21:48.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:21:49.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:48 smithi027 ceph-mon[30779]: pgmap v647: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-09-21T07:21:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:48 smithi119 ceph-mon[34519]: pgmap v647: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-09-21T07:21:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:48 smithi153 ceph-mon[34747]: pgmap v647: 129 pgs: 129 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-09-21T07:21:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:50.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.005+0000 7f0f80200640 1 -- 172.21.15.27:0/3095694339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f7810c010 msgr2=0x7f0f78112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.005+0000 7f0f80200640 1 --2- 172.21.15.27:0/3095694339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f7810c010 0x7f0f78112840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009ca0 tx=0x7f0f7002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.006+0000 7f0f80200640 1 -- 172.21.15.27:0/3095694339 shutdown_connections 2024-09-21T07:21:50.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.006+0000 7f0f80200640 1 --2- 172.21.15.27:0/3095694339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f7810b6c0 0x7f0f7810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.006+0000 7f0f80200640 1 --2- 172.21.15.27:0/3095694339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f7810c010 0x7f0f78112840 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.006+0000 7f0f80200640 1 --2- 172.21.15.27:0/3095694339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f78105d80 0x7f0f78106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.006+0000 7f0f80200640 1 -- 172.21.15.27:0/3095694339 >> 172.21.15.27:0/3095694339 conn(0x7f0f78101810 msgr2=0x7f0f78103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.006+0000 7f0f80200640 1 -- 172.21.15.27:0/3095694339 shutdown_connections 2024-09-21T07:21:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.006+0000 7f0f80200640 1 -- 172.21.15.27:0/3095694339 wait complete. 2024-09-21T07:21:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.007+0000 7f0f80200640 1 Processor -- start 2024-09-21T07:21:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.007+0000 7f0f80200640 1 -- start start 2024-09-21T07:21:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.007+0000 7f0f80200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f78105d80 0x7f0f7819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.007+0000 7f0f80200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f7810b6c0 0x7f0f7819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.007+0000 7f0f80200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7810c010 0x7f0f781a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f80200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78115d20 con 0x7f0f78105d80 2024-09-21T07:21:50.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f80200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78115ba0 con 0x7f0f7810c010 2024-09-21T07:21:50.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f80200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78115ea0 con 0x7f0f7810b6c0 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f78105d80 0x7f0f7819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7ce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f7810b6c0 0x7f0f7819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f78105d80 0x7f0f7819c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56886/0 (socket says 172.21.15.27:56886) 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 -- 172.21.15.27:0/2465736779 learned_addr learned my addr 172.21.15.27:0/2465736779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7e200640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7810c010 0x7f0f781a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 -- 172.21.15.27:0/2465736779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f7810b6c0 msgr2=0x7f0f7819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f7810b6c0 0x7f0f7819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 -- 172.21.15.27:0/2465736779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7810c010 msgr2=0x7f0f781a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7810c010 0x7f0f781a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.008+0000 7f0f7d800640 1 -- 172.21.15.27:0/2465736779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f781a26a0 con 0x7f0f78105d80 2024-09-21T07:21:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.009+0000 7f0f7d800640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f78105d80 0x7f0f7819c730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00ea60 tx=0x7f0f6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.009+0000 7f0f76000640 1 -- 172.21.15.27:0/2465736779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00cde0 con 0x7f0f78105d80 2024-09-21T07:21:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.009+0000 7f0f76000640 1 -- 172.21.15.27:0/2465736779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f6c0035f0 con 0x7f0f78105d80 2024-09-21T07:21:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.009+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f781a2990 con 0x7f0f78105d80 2024-09-21T07:21:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.009+0000 7f0f76000640 1 -- 172.21.15.27:0/2465736779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c010690 con 0x7f0f78105d80 2024-09-21T07:21:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.009+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f781a90f0 con 0x7f0f78105d80 2024-09-21T07:21:50.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.013+0000 7f0f86e1c740 1 Processor -- start 2024-09-21T07:21:50.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.013+0000 7f0f86e1c740 1 -- start start 2024-09-21T07:21:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192501d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x561192502250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x561192502790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561192477e90 con 0x561192131490 2024-09-21T07:21:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5611924672c0 con 0x56119215a350 2024-09-21T07:21:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56119246b0d0 con 0x561192139670 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f7e200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192501d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f7e200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192501d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35370/0 (socket says 172.21.15.27:35370) 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f7e200640 1 -- 172.21.15.27:0/1232105286 learned_addr learned my addr 172.21.15.27:0/1232105286 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.014+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x561192502790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.015+0000 7f0f76000640 1 -- 172.21.15.27:0/2465736779 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0f6c0037a0 con 0x7f0f78105d80 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.015+0000 7f0f7e200640 1 -- 172.21.15.27:0/1232105286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 msgr2=0x561192502790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.015+0000 7f0f7e200640 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x561192502790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.015+0000 7f0f7e200640 1 -- 172.21.15.27:0/1232105286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 msgr2=0x561192502250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.015+0000 7f0f7e200640 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x561192502250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.015+0000 7f0f7e200640 1 -- 172.21.15.27:0/1232105286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611921c5930 con 0x56119215a350 2024-09-21T07:21:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.015+0000 7f0f7e200640 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192501d10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009ca0 tx=0x7f0f70009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d6c8798eafdc7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.016+0000 7f0f6aa00640 1 -- 172.21.15.27:0/1232105286 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70031cf0 con 0x56119215a350 2024-09-21T07:21:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.016+0000 7f0f6aa00640 1 -- 172.21.15.27:0/1232105286 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f70036030 con 0x56119215a350 2024-09-21T07:21:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.016+0000 7f0f6aa00640 1 -- 172.21.15.27:0/1232105286 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70004280 con 0x56119215a350 2024-09-21T07:21:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f76000640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f600b6700 0x7f0f600b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 -- 172.21.15.27:0/1232105286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 msgr2=0x561192501d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192501d10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009ca0 tx=0x7f0f70009430 comp rx=0 tx=0).stop 2024-09-21T07:21:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f76000640 1 -- 172.21.15.27:0/2465736779 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0f6c0e7d80 con 0x7f0f78105d80 2024-09-21T07:21:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f7d800640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f600b6700 0x7f0f600b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 -- 172.21.15.27:0/1232105286 shutdown_connections 2024-09-21T07:21:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x561192502790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x561192502250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/1232105286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192501d10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 -- 172.21.15.27:0/1232105286 >> 172.21.15.27:0/1232105286 conn(0x56119214e740 msgr2=0x5611924ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 -- 172.21.15.27:0/1232105286 shutdown_connections 2024-09-21T07:21:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.017+0000 7f0f86e1c740 1 -- 172.21.15.27:0/1232105286 wait complete. 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.018+0000 7f0f7d800640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f600b6700 0x7f0f600b8c10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x561192500990 tx=0x7f0f6c0a9770 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.018+0000 7f0f86e1c740 1 -- start start 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.018+0000 7f0f86e1c740 1 Processor -- start 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561192131490 0x56119250d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56119215a350 0x561192519630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5611922c8890 con 0x56119215a350 2024-09-21T07:21:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561192402160 con 0x561192131490 2024-09-21T07:21:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561192510310 con 0x561192139670 2024-09-21T07:21:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f86e1c740 10 client.0 _ll_register_callbacks cb 0x56119241ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f7e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56119215a350 0x561192519630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f7ce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f7e200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561192131490 0x56119250d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f7ce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53028/0 (socket says 172.21.15.27:53028) 2024-09-21T07:21:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.019+0000 7f0f7ce00640 1 -- 172.21.15.27:0/2255245114 learned_addr learned my addr 172.21.15.27:0/2255245114 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f7ce00640 1 -- 172.21.15.27:0/2255245114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561192131490 msgr2=0x56119250d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561192131490 0x56119250d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f7ce00640 1 -- 172.21.15.27:0/2255245114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56119215a350 msgr2=0x561192519630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56119215a350 0x561192519630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f7ce00640 1 -- 172.21.15.27:0/2255245114 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561192519ba0 con 0x561192139670 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250de80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f64012700 tx=0x7f0f64012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ecf01ef75dfd7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f5f400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f5f400640 1 -- 172.21.15.27:0/2255245114 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400f920 con 0x561192139670 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.020+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f70036030 con 0x561192139670 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.021+0000 7f0f5f400640 1 -- 172.21.15.27:0/2255245114 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0f6400fac0 con 0x561192139670 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.021+0000 7f0f5f400640 1 -- 172.21.15.27:0/2255245114 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f64025050 con 0x561192139670 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.021+0000 7f0f86e1c740 10 client.28105 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.021+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56119251c520 con 0x561192139670 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.021+0000 7f0f5f400640 1 -- 172.21.15.27:0/2255245114 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f70036030 con 0x561192139670 2024-09-21T07:21:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f5f400640 1 -- 172.21.15.27:0/2255245114 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f640049d0 con 0x561192139670 2024-09-21T07:21:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f86e1c740 10 client.28105 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f86e1c740 10 client.28105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f86e1c740 20 client.28105 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561192118cd0 0x56119251fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f86e1c740 4 client.28105 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561192479de0 con 0x561192118cd0 2024-09-21T07:21:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.022+0000 7f0f7e200640 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561192118cd0 0x56119251fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.023+0000 7f0f7e200640 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561192118cd0 0x56119251fa80 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d9d6af91d698a5 server_cookie=ea6a81ce4339b222 in_seq=0 out_seq=0 2024-09-21T07:21:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.023+0000 7f0f5f400640 10 client.28105 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f5f400640 1 -- 172.21.15.27:0/2255245114 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561192479de0 con 0x561192118cd0 2024-09-21T07:21:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f5f400640 10 client.28105 handle_command_reply: tid=0 2024-09-21T07:21:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 client.28105 shutdown 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 msgr2=0x56119250de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250de80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f64012700 tx=0x7f0f64012bc0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 shutdown_connections 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561192118cd0 0x56119251fa80 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250de80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f5f400640 0 client.28105 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56119215a350 0x561192519630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/2255245114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561192131490 0x56119250d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 >> 172.21.15.27:0/2255245114 conn(0x56119214e740 msgr2=0x5611924fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 shutdown_connections 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.024+0000 7f0f86e1c740 1 -- 172.21.15.27:0/2255245114 wait complete. 2024-09-21T07:21:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.025+0000 7f0f86e1c740 20 client.28105 trim_cache size 0 max 16384 2024-09-21T07:21:50.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.043+0000 7f0f86e1c740 1 Processor -- start 2024-09-21T07:21:50.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.043+0000 7f0f86e1c740 1 -- start start 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x5611924ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192506f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781b59c0 con 0x561192131490 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781b3290 con 0x56119215a350 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781b1100 con 0x561192139670 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f7e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x5611924ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f7ce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f7e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x5611924ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56922/0 (socket says 172.21.15.27:56922) 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f7ce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53038/0 (socket says 172.21.15.27:53038) 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f7e200640 1 -- 172.21.15.27:0/3658253834 learned_addr learned my addr 172.21.15.27:0/3658253834 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.044+0000 7f0f7e200640 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192506f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f7ce00640 1 -- 172.21.15.27:0/3658253834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 msgr2=0x561192506f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192506f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f7ce00640 1 -- 172.21.15.27:0/3658253834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 msgr2=0x5611924ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x5611924ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f7ce00640 1 -- 172.21.15.27:0/3658253834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611925045a0 con 0x561192139670 2024-09-21T07:21:50.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f7e200640 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x5611924ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:50.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250cea0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f64004df0 tx=0x7f0f6400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797d67a9ca4dba01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f6a000640 1 -- 172.21.15.27:0/3658253834 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400f9e0 con 0x561192139670 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f6a000640 1 -- 172.21.15.27:0/3658253834 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f64005000 con 0x561192139670 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.045+0000 7f0f6a000640 1 -- 172.21.15.27:0/3658253834 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64003770 con 0x561192139670 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3658253834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 msgr2=0x56119250cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250cea0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f64004df0 tx=0x7f0f6400f790 comp rx=0 tx=0).stop 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3658253834 shutdown_connections 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192139670 0x56119250cea0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192506f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3658253834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192131490 0x5611924ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3658253834 >> 172.21.15.27:0/3658253834 conn(0x56119214e740 msgr2=0x5611924ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3658253834 shutdown_connections 2024-09-21T07:21:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.046+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3658253834 wait complete. 2024-09-21T07:21:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.047+0000 7f0f86e1c740 1 -- start start 2024-09-21T07:21:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.047+0000 7f0f86e1c740 1 Processor -- start 2024-09-21T07:21:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192131490 0x5611925155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192139670 0x561192515ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f86e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192510550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781b1750 con 0x561192139670 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781b2bc0 con 0x56119215a350 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f86e1c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781b1e20 con 0x561192131490 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f86e1c740 10 client.0 _ll_register_callbacks cb 0x7f0f64009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192139670 0x561192515ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7e200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192131490 0x5611925155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192139670 0x561192515ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56936/0 (socket says 172.21.15.27:56936) 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7ce00640 1 -- 172.21.15.27:0/3434099203 learned_addr learned my addr 172.21.15.27:0/3434099203 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7e200640 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192510550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7e200640 1 -- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 msgr2=0x561192510550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7e200640 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192510550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7e200640 1 -- 172.21.15.27:0/3434099203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192139670 msgr2=0x561192515ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7e200640 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192139670 0x561192515ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7e200640 1 -- 172.21.15.27:0/3434099203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561192516180 con 0x561192131490 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.048+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192139670 0x561192515ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:50.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.049+0000 7f0f7e200640 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192131490 0x5611925155a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0f70003d50 tx=0x7f0f70003e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74945b3d1d945675 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.049+0000 7f0f5f400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.049+0000 7f0f5f400640 1 -- 172.21.15.27:0/3434099203 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7004a040 con 0x561192131490 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.049+0000 7f0f5f400640 1 -- 172.21.15.27:0/3434099203 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f0f70031650 con 0x561192131490 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.049+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f64005000 con 0x561192131490 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.049+0000 7f0f5f400640 1 -- 172.21.15.27:0/3434099203 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f70031850 con 0x561192131490 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.049+0000 7f0f86e1c740 10 client.28111 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.050+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56119258a8e0 con 0x561192131490 2024-09-21T07:21:50.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f5f400640 1 -- 172.21.15.27:0/3434099203 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f64005000 con 0x561192131490 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f5f400640 1 -- 172.21.15.27:0/3434099203 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f70043070 con 0x561192131490 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f86e1c740 10 client.28111 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f86e1c740 10 client.28111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f86e1c740 20 client.28111 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611920ee080 0x56119258cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f86e1c740 4 client.28111 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561192475890 con 0x5611920ee080 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.051+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611920ee080 0x56119258cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.052+0000 7f0f7ce00640 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611920ee080 0x56119258cce0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae6d3f965324fde server_cookie=d2d2e872dded72fc in_seq=0 out_seq=0 2024-09-21T07:21:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.052+0000 7f0f5f400640 10 client.28111 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:50.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f5f400640 1 -- 172.21.15.27:0/3434099203 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561192475890 con 0x5611920ee080 2024-09-21T07:21:50.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f5f400640 10 client.28111 handle_command_reply: tid=0 2024-09-21T07:21:50.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f86e1c740 1 client.28111 shutdown 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192131490 msgr2=0x5611925155a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192131490 0x5611925155a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0f70003d50 tx=0x7f0f70003e20 comp rx=0 tx=0).stop 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 shutdown_connections 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611920ee080 0x56119258cce0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.054+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56119215a350 0x561192510550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.055+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561192139670 0x561192515ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.055+0000 7f0f86e1c740 1 --2- 172.21.15.27:0/3434099203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561192131490 0x5611925155a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.055+0000 7f0f5f400640 0 client.28111 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.055+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 >> 172.21.15.27:0/3434099203 conn(0x56119214e740 msgr2=0x561192502d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.055+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 shutdown_connections 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.055+0000 7f0f86e1c740 1 -- 172.21.15.27:0/3434099203 wait complete. 2024-09-21T07:21:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.055+0000 7f0f86e1c740 20 client.28111 trim_cache size 0 max 16384 2024-09-21T07:21:50.058 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:50.058 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:21:50.058 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "0ad2e08c-dd6f-42b7-b797-c85976aaee82", 2024-09-21T07:21:50.058 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:21:50.058 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f600b6700 msgr2=0x7f0f600b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f600b6700 0x7f0f600b8c10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x561192500990 tx=0x7f0f6c0a9770 comp rx=0 tx=0).stop 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f78105d80 msgr2=0x7f0f7819c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f78105d80 0x7f0f7819c730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00ea60 tx=0x7f0f6c00ef20 comp rx=0 tx=0).stop 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 shutdown_connections 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f7810b6c0 0x7f0f7819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0f600b6700 0x7f0f600b8c10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7810c010 0x7f0f781a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 --2- 172.21.15.27:0/2465736779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f78105d80 0x7f0f7819c730 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 >> 172.21.15.27:0/2465736779 conn(0x7f0f78101810 msgr2=0x7f0f7810acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 shutdown_connections 2024-09-21T07:21:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:50.056+0000 7f0f80200640 1 -- 172.21.15.27:0/2465736779 wait complete. 2024-09-21T07:21:50.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:21:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:50 smithi119 ceph-mon[34519]: pgmap v648: 129 pgs: 129 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:21:51.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:50 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:21:51.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:50 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:51.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:50 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:21:51.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:50 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:50 smithi027 ceph-mon[30779]: pgmap v648: 129 pgs: 129 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:21:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:50 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:21:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:50 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:50 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:21:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:50 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:50 smithi153 ceph-mon[34747]: pgmap v648: 129 pgs: 129 active+clean; 3.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:21:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:50 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:21:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:50 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:50 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:21:51.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:50 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:21:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:21:51.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 -- 172.21.15.27:0/1156423007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 msgr2=0x7f7378109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 --2- 172.21.15.27:0/1156423007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f7378109700 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7364009900 tx=0x7f736402a410 comp rx=0 tx=0).stop 2024-09-21T07:21:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 -- 172.21.15.27:0/1156423007 shutdown_connections 2024-09-21T07:21:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 --2- 172.21.15.27:0/1156423007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7378109c40 0x7f7378110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 --2- 172.21.15.27:0/1156423007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f7378109700 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 --2- 172.21.15.27:0/1156423007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73781039b0 0x7f7378103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 -- 172.21.15.27:0/1156423007 >> 172.21.15.27:0/1156423007 conn(0x7f73780ff440 msgr2=0x7f7378101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 -- 172.21.15.27:0/1156423007 shutdown_connections 2024-09-21T07:21:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.907+0000 7f737e400640 1 -- 172.21.15.27:0/1156423007 wait complete. 2024-09-21T07:21:51.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.908+0000 7f737e400640 1 Processor -- start 2024-09-21T07:21:51.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.908+0000 7f737e400640 1 -- start start 2024-09-21T07:21:51.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737e400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73781039b0 0x7f737819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737e400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f737819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737e400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7378109c40 0x7f737819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737e400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7378113900 con 0x7f73781092f0 2024-09-21T07:21:51.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737e400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7378113780 con 0x7f7378109c40 2024-09-21T07:21:51.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737e400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7378113a80 con 0x7f73781039b0 2024-09-21T07:21:51.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f7377e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f737819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f7377e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f737819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56946/0 (socket says 172.21.15.27:56946) 2024-09-21T07:21:51.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f7377e00640 1 -- 172.21.15.27:0/471393403 learned_addr learned my addr 172.21.15.27:0/471393403 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:51.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737d000640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73781039b0 0x7f737819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.909+0000 7f737da00640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7378109c40 0x7f737819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.910+0000 7f7377e00640 1 -- 172.21.15.27:0/471393403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73781039b0 msgr2=0x7f737819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.910+0000 7f7377e00640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73781039b0 0x7f737819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.910+0000 7f7377e00640 1 -- 172.21.15.27:0/471393403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7378109c40 msgr2=0x7f737819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.910+0000 7f7377e00640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7378109c40 0x7f737819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.910+0000 7f7377e00640 1 -- 172.21.15.27:0/471393403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73781a02a0 con 0x7f73781092f0 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.910+0000 7f7377e00640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f737819a8d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f736402a930 tx=0x7f7364031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.910+0000 7f7375600640 1 -- 172.21.15.27:0/471393403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736403f030 con 0x7f73781092f0 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.911+0000 7f7375600640 1 -- 172.21.15.27:0/471393403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73640043d0 con 0x7f73781092f0 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.911+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73781a0530 con 0x7f73781092f0 2024-09-21T07:21:51.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.911+0000 7f7375600640 1 -- 172.21.15.27:0/471393403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364035070 con 0x7f73781092f0 2024-09-21T07:21:51.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.911+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73781a6cf0 con 0x7f73781092f0 2024-09-21T07:21:51.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.914+0000 7f73865d5740 1 Processor -- start 2024-09-21T07:21:51.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.914+0000 7f73865d5740 1 -- start start 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.914+0000 7f7375600640 1 -- 172.21.15.27:0/471393403 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7364042030 con 0x7f73781092f0 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f73865d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557656620070 0x5576569c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f73865d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565f71b0 0x5576569c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f73865d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569c7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f73865d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55765693df70 con 0x557656620070 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f73865d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55765692d3a0 con 0x5576565ff390 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f73865d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5576569311b0 con 0x5576565f71b0 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f737da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557656620070 0x5576569c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f737da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557656620070 0x5576569c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56956/0 (socket says 172.21.15.27:56956) 2024-09-21T07:21:51.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f737da00640 1 -- 172.21.15.27:0/1122127771 learned_addr learned my addr 172.21.15.27:0/1122127771 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:51.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f737d000640 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565f71b0 0x5576569c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.915+0000 7f7377e00640 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f737d000640 1 -- 172.21.15.27:0/1122127771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 msgr2=0x5576569c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f737d000640 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f737d000640 1 -- 172.21.15.27:0/1122127771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557656620070 msgr2=0x5576569c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f737d000640 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557656620070 0x5576569c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f737d000640 1 -- 172.21.15.27:0/1122127771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576568f7c90 con 0x5576565f71b0 2024-09-21T07:21:51.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f737d000640 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565f71b0 0x5576569c7230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f736c009c20 tx=0x7f736c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78718152e63c6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:51.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f736a000640 1 -- 172.21.15.27:0/1122127771 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c01e070 con 0x5576565f71b0 2024-09-21T07:21:51.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f736a000640 1 -- 172.21.15.27:0/1122127771 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736c009dd0 con 0x5576565f71b0 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f736a000640 1 -- 172.21.15.27:0/1122127771 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c018710 con 0x5576565f71b0 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.916+0000 7f7375600640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73600b6680 0x7f73600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.917+0000 7f7375600640 1 -- 172.21.15.27:0/471393403 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f7364107dd0 con 0x7f73781092f0 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.917+0000 7f737da00640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73600b6680 0x7f73600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 -- 172.21.15.27:0/1122127771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565f71b0 msgr2=0x5576569c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f737da00640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73600b6680 0x7f73600b8b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7370012680 tx=0x7f7370009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565f71b0 0x5576569c7230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f736c009c20 tx=0x7f736c011810 comp rx=0 tx=0).stop 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 -- 172.21.15.27:0/1122127771 shutdown_connections 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565f71b0 0x5576569c7230 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 --2- 172.21.15.27:0/1122127771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557656620070 0x5576569c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 -- 172.21.15.27:0/1122127771 >> 172.21.15.27:0/1122127771 conn(0x557656614460 msgr2=0x5576569c4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 -- 172.21.15.27:0/1122127771 shutdown_connections 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 -- 172.21.15.27:0/1122127771 wait complete. 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.918+0000 7f73865d5740 1 -- start start 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 1 Processor -- start 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557656620070 0x5576569dcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55765678e700 con 0x5576565f71b0 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5576568c8540 con 0x5576565ff390 2024-09-21T07:21:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5576569d97f0 con 0x557656620070 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f73865d5740 10 client.0 _ll_register_callbacks cb 0x5576568d8430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f737d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f737d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569d29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56970/0 (socket says 172.21.15.27:56970) 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.919+0000 7f737d000640 1 -- 172.21.15.27:0/2106345481 learned_addr learned my addr 172.21.15.27:0/2106345481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f737da00640 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557656620070 0x5576569dcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f7377e00640 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f737d000640 1 -- 172.21.15.27:0/2106345481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557656620070 msgr2=0x5576569dcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f737d000640 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557656620070 0x5576569dcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f737d000640 1 -- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 msgr2=0x5576569d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f737d000640 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569d2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f737d000640 1 -- 172.21.15.27:0/2106345481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765668b650 con 0x5576565f71b0 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f737d000640 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569d29e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f736c00a000 tx=0x7f736c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e6a1c5437443bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.920+0000 7f735ea00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:51.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.921+0000 7f735ea00640 1 -- 172.21.15.27:0/2106345481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c01e050 con 0x5576565f71b0 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.921+0000 7f735ea00640 1 -- 172.21.15.27:0/2106345481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f736c005240 con 0x5576565f71b0 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.921+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576568cab50 con 0x5576565f71b0 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.921+0000 7f735ea00640 1 -- 172.21.15.27:0/2106345481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f736c032050 con 0x5576565f71b0 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.921+0000 7f73865d5740 10 client.18135 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.921+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576568e5910 con 0x5576565f71b0 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.921+0000 7f735ea00640 1 -- 172.21.15.27:0/2106345481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5576568cab50 con 0x5576565f71b0 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f735ea00640 1 -- 172.21.15.27:0/2106345481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f736c02d030 con 0x5576565f71b0 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f73865d5740 10 client.18135 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f73865d5740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f73865d5740 20 client.18135 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f73865d5740 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565de9f0 0x5576569e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f73865d5740 4 client.18135 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:21:51.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55765693fec0 con 0x5576565de9f0 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.922+0000 7f7377e00640 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565de9f0 0x5576569e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.923+0000 7f7377e00640 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565de9f0 0x5576569e8c40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e59a403c4e01b2 server_cookie=f60c6244f5e18577 in_seq=0 out_seq=0 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.923+0000 7f735ea00640 10 client.18135 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.923+0000 7f735ea00640 1 -- 172.21.15.27:0/2106345481 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55765693fec0 con 0x5576565de9f0 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.923+0000 7f735ea00640 10 client.18135 handle_command_reply: tid=0 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 client.18135 shutdown 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 msgr2=0x5576569d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569d29e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f736c00a000 tx=0x7f736c005730 comp rx=0 tx=0).stop 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 shutdown_connections 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565de9f0 0x5576569e8c40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557656620070 0x5576569dcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5576565ff390 0x5576569d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 --2- 172.21.15.27:0/2106345481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569d29e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 >> 172.21.15.27:0/2106345481 conn(0x557656614460 msgr2=0x5576569c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f735ea00640 0 client.18135 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 shutdown_connections 2024-09-21T07:21:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 1 -- 172.21.15.27:0/2106345481 wait complete. 2024-09-21T07:21:51.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.924+0000 7f73865d5740 20 client.18135 trim_cache size 0 max 16384 2024-09-21T07:21:51.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 Processor -- start 2024-09-21T07:21:51.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 -- start start 2024-09-21T07:21:51.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x5576569cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73781b1400 con 0x5576565f71b0 2024-09-21T07:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73781aecd0 con 0x557656620070 2024-09-21T07:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f73865d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73781acb40 con 0x5576565ff390 2024-09-21T07:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f737d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f737d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569dcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56986/0 (socket says 172.21.15.27:56986) 2024-09-21T07:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f737d000640 1 -- 172.21.15.27:0/187984631 learned_addr learned my addr 172.21.15.27:0/187984631 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.943+0000 7f7377e00640 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737da00640 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x5576569cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737d000640 1 -- 172.21.15.27:0/187984631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 msgr2=0x5576569cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737d000640 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737d000640 1 -- 172.21.15.27:0/187984631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 msgr2=0x5576569cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737d000640 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x5576569cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737d000640 1 -- 172.21.15.27:0/187984631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576569d2590 con 0x5576565f71b0 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737da00640 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x5576569cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f7377e00640 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f737d000640 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569dcb30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f736c011d10 tx=0x7f736c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d273cef4186edc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.944+0000 7f7369600640 1 -- 172.21.15.27:0/187984631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c01e040 con 0x5576565f71b0 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f7369600640 1 -- 172.21.15.27:0/187984631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736c01f040 con 0x5576565f71b0 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f7369600640 1 -- 172.21.15.27:0/187984631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c025870 con 0x5576565f71b0 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 -- 172.21.15.27:0/187984631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 msgr2=0x5576569dcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569dcb30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f736c011d10 tx=0x7f736c011810 comp rx=0 tx=0).stop 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 -- 172.21.15.27:0/187984631 shutdown_connections 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x5576569cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 --2- 172.21.15.27:0/187984631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569dcb30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 -- 172.21.15.27:0/187984631 >> 172.21.15.27:0/187984631 conn(0x557656614460 msgr2=0x5576569d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.945+0000 7f73865d5740 1 -- 172.21.15.27:0/187984631 shutdown_connections 2024-09-21T07:21:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.946+0000 7f73865d5740 1 -- 172.21.15.27:0/187984631 wait complete. 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.946+0000 7f73865d5740 1 -- start start 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.946+0000 7f73865d5740 1 Processor -- start 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f73865d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f73865d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f73865d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x557656a525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f73865d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73781ad190 con 0x5576565f71b0 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f73865d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73781ae600 con 0x557656620070 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f73865d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73781ad860 con 0x5576565ff390 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f73865d5740 10 client.0 _ll_register_callbacks cb 0x7f736c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f737d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f737d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569df5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56990/0 (socket says 172.21.15.27:56990) 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f737d000640 1 -- 172.21.15.27:0/254296879 learned_addr learned my addr 172.21.15.27:0/254296879 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f737da00640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x557656a525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.947+0000 7f7377e00640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f737d000640 1 -- 172.21.15.27:0/254296879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 msgr2=0x5576569dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f737d000640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569dfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f737d000640 1 -- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 msgr2=0x557656a525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f737d000640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x557656a525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f737d000640 1 -- 172.21.15.27:0/254296879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557656a52b00 con 0x5576565f71b0 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f7377e00640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f737d000640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569df5b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f736c005bc0 tx=0x7f736c003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107bed522e43e243 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:21:51.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f735ea00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:21:51.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f735ea00640 1 -- 172.21.15.27:0/254296879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c042020 con 0x5576565f71b0 2024-09-21T07:21:51.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f735ea00640 1 -- 172.21.15.27:0/254296879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7f736c005650 con 0x5576565f71b0 2024-09-21T07:21:51.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.948+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736c01f040 con 0x5576565f71b0 2024-09-21T07:21:51.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f735ea00640 1 -- 172.21.15.27:0/254296879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f736c04d050 con 0x5576565f71b0 2024-09-21T07:21:51.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 10 client.18141 fetch_fsmap learned FSMap version 27 2024-09-21T07:21:51.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557656a53cf0 con 0x5576565f71b0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f735ea00640 1 -- 172.21.15.27:0/254296879 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736c01f040 con 0x5576565f71b0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f735ea00640 1 -- 172.21.15.27:0/254296879 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f736c02d040 con 0x5576565f71b0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 10 client.18141 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 20 client.18141 populate_metadata read hostname 'smithi027' 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565b3da0 0x557656a56440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 4 client.18141 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.949+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55765693b970 con 0x5576565b3da0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.950+0000 7f7377e00640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565b3da0 0x557656a56440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.950+0000 7f7377e00640 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565b3da0 0x557656a56440 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=718877e326cd8d51 server_cookie=8fb53ad1477e9403 in_seq=0 out_seq=0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.950+0000 7f735ea00640 10 client.18141 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f735ea00640 1 -- 172.21.15.27:0/254296879 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x55765693b970 con 0x5576565b3da0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f735ea00640 10 client.18141 handle_command_reply: tid=0 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 client.18141 shutdown 2024-09-21T07:21:51.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 msgr2=0x5576569df5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569df5b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f736c005bc0 tx=0x7f736c003bb0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 shutdown_connections 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5576565b3da0 0x557656a56440 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576565ff390 0x5576569dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557656620070 0x557656a525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 --2- 172.21.15.27:0/254296879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576565f71b0 0x5576569df5b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 >> 172.21.15.27:0/254296879 conn(0x557656614460 msgr2=0x5576569d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f735ea00640 0 client.18141 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 shutdown_connections 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 1 -- 172.21.15.27:0/254296879 wait complete. 2024-09-21T07:21:51.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.951+0000 7f73865d5740 20 client.18141 trim_cache size 0 max 16384 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "0ad2e08c-dd6f-42b7-b797-c85976aaee82", 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:21:51.956 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.958 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:21:51.959 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:21:51.961 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.962 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:51.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73600b6680 msgr2=0x7f73600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73600b6680 0x7f73600b8b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7370012680 tx=0x7f7370009500 comp rx=0 tx=0).stop 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 msgr2=0x7f737819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f737819a8d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f736402a930 tx=0x7f7364031a20 comp rx=0 tx=0).stop 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 shutdown_connections 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f73600b6680 0x7f73600b8b90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7378109c40 0x7f737819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73781092f0 0x7f737819a8d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 --2- 172.21.15.27:0/471393403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73781039b0 0x7f737819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 >> 172.21.15.27:0/471393403 conn(0x7f73780ff440 msgr2=0x7f7378101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 shutdown_connections 2024-09-21T07:21:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:21:51.952+0000 7f737e400640 1 -- 172.21.15.27:0/471393403 wait complete. 2024-09-21T07:21:52.925 INFO:tasks.cephfs.filesystem:scrub status for tag:0ad2e08c-dd6f-42b7-b797-c85976aaee82 - {'path': '/', 'tag': '0ad2e08c-dd6f-42b7-b797-c85976aaee82', 'options': 'recursive,force'} 2024-09-21T07:21:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:52 smithi027 ceph-mon[30779]: pgmap v649: 129 pgs: 129 active+clean; 3.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-21T07:21:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:52 smithi153 ceph-mon[34747]: pgmap v649: 129 pgs: 129 active+clean; 3.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-21T07:21:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:52 smithi119 ceph-mon[34519]: pgmap v649: 129 pgs: 129 active+clean; 3.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-21T07:21:54.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:53 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:21:54.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:53 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:21:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:53 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:21:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:54 smithi027 ceph-mon[30779]: pgmap v650: 129 pgs: 129 active+clean; 3.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:21:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:54 smithi153 ceph-mon[34747]: pgmap v650: 129 pgs: 129 active+clean; 3.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:21:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:54 smithi119 ceph-mon[34519]: pgmap v650: 129 pgs: 129 active+clean; 3.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:21:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:56 smithi027 ceph-mon[30779]: pgmap v651: 129 pgs: 129 active+clean; 3.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:21:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:56 smithi153 ceph-mon[34747]: pgmap v651: 129 pgs: 129 active+clean; 3.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:21:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:56 smithi119 ceph-mon[34519]: pgmap v651: 129 pgs: 129 active+clean; 3.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:21:59.094 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:21:58 smithi027 ceph-mon[30779]: pgmap v652: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:21:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:21:58 smithi153 ceph-mon[34747]: pgmap v652: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:21:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:21:58 smithi119 ceph-mon[34519]: pgmap v652: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:22:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:00 smithi027 ceph-mon[30779]: pgmap v653: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:22:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:00 smithi153 ceph-mon[34747]: pgmap v653: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:22:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:00 smithi119 ceph-mon[34519]: pgmap v653: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:22:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 -- 172.21.15.27:0/3046527227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450103920 msgr2=0x7fd450103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 --2- 172.21.15.27:0/3046527227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450103920 0x7fd450103d30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd440009900 tx=0x7fd44002a410 comp rx=0 tx=0).stop 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 -- 172.21.15.27:0/3046527227 shutdown_connections 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 --2- 172.21.15.27:0/3046527227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450109250 0x7fd450109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 --2- 172.21.15.27:0/3046527227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd450109ba0 0x7fd4501103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 --2- 172.21.15.27:0/3046527227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450103920 0x7fd450103d30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 -- 172.21.15.27:0/3046527227 >> 172.21.15.27:0/3046527227 conn(0x7fd4500ff320 msgr2=0x7fd450101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 -- 172.21.15.27:0/3046527227 shutdown_connections 2024-09-21T07:22:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.997+0000 7fd44f200640 1 -- 172.21.15.27:0/3046527227 wait complete. 2024-09-21T07:22:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.998+0000 7fd44f200640 1 Processor -- start 2024-09-21T07:22:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.998+0000 7fd44f200640 1 -- start start 2024-09-21T07:22:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd450103920 0x7fd450075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450109250 0x7fd450075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450109ba0 0x7fd450076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd450113860 con 0x7fd450109ba0 2024-09-21T07:22:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501136e0 con 0x7fd450103920 2024-09-21T07:22:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501139e0 con 0x7fd450109250 2024-09-21T07:22:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450109ba0 0x7fd450076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450109ba0 0x7fd450076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35388/0 (socket says 172.21.15.27:35388) 2024-09-21T07:22:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44e800640 1 -- 172.21.15.27:0/1334913608 learned_addr learned my addr 172.21.15.27:0/1334913608 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44d400640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450109250 0x7fd450075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:01.999+0000 7fd44de00640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd450103920 0x7fd450075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd44e800640 1 -- 172.21.15.27:0/1334913608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450109250 msgr2=0x7fd450075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd44e800640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450109250 0x7fd450075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd44e800640 1 -- 172.21.15.27:0/1334913608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd450103920 msgr2=0x7fd450075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd44e800640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd450103920 0x7fd450075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd44e800640 1 -- 172.21.15.27:0/1334913608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4501a8570 con 0x7fd450109ba0 2024-09-21T07:22:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd44e800640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450109ba0 0x7fd450076190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd44800eb10 tx=0x7fd44800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd446a00640 1 -- 172.21.15.27:0/1334913608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448004b20 con 0x7fd450109ba0 2024-09-21T07:22:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd446a00640 1 -- 172.21.15.27:0/1334913608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd448004cc0 con 0x7fd450109ba0 2024-09-21T07:22:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4501a8860 con 0x7fd450109ba0 2024-09-21T07:22:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.000+0000 7fd446a00640 1 -- 172.21.15.27:0/1334913608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448010670 con 0x7fd450109ba0 2024-09-21T07:22:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.001+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501a8d60 con 0x7fd450109ba0 2024-09-21T07:22:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.003+0000 7fd45ba8b740 1 Processor -- start 2024-09-21T07:22:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.004+0000 7fd446a00640 1 -- 172.21.15.27:0/1334913608 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd448005000 con 0x7fd450109ba0 2024-09-21T07:22:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd45ba8b740 1 -- start start 2024-09-21T07:22:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4de1b0 0x5585ea882e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea883350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4bd4d0 0x5585ea883890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5585ea7f86b0 con 0x5585ea4bd4d0 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5585ea7e7ae0 con 0x5585ea4de1b0 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5585ea7eb8f0 con 0x5585ea4b52f0 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd44d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4bd4d0 0x5585ea883890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd44de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea883350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.005+0000 7fd44d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4de1b0 0x5585ea882e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4de1b0 0x5585ea882e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59340/0 (socket says 172.21.15.27:59340) 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea883350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34500/0 (socket says 172.21.15.27:34500) 2024-09-21T07:22:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 -- 172.21.15.27:0/483541006 learned_addr learned my addr 172.21.15.27:0/483541006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 -- 172.21.15.27:0/483541006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 msgr2=0x5585ea883350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 --2- 172.21.15.27:0/483541006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea883350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 -- 172.21.15.27:0/483541006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4bd4d0 msgr2=0x5585ea883890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 --2- 172.21.15.27:0/483541006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4bd4d0 0x5585ea883890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 -- 172.21.15.27:0/483541006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ea7b4780 con 0x5585ea4de1b0 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd44d400640 1 --2- 172.21.15.27:0/483541006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4de1b0 0x5585ea882e10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd43c010940 tx=0x7fd43c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39c3c759e1d0403 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.006+0000 7fd43b000640 1 -- 172.21.15.27:0/483541006 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c016070 con 0x5585ea4de1b0 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.007+0000 7fd43b000640 1 -- 172.21.15.27:0/483541006 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43c00e3c0 con 0x5585ea4de1b0 2024-09-21T07:22:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.007+0000 7fd43b000640 1 -- 172.21.15.27:0/483541006 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c019040 con 0x5585ea4de1b0 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.007+0000 7fd446a00640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4340b66b0 0x7fd4340b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.007+0000 7fd44de00640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4340b66b0 0x7fd4340b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.007+0000 7fd446a00640 1 -- 172.21.15.27:0/1334913608 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fd4480e80d0 con 0x7fd450109ba0 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 -- 172.21.15.27:0/483541006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4de1b0 msgr2=0x5585ea882e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/483541006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4de1b0 0x5585ea882e10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd43c010940 tx=0x7fd43c005f90 comp rx=0 tx=0).stop 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 -- 172.21.15.27:0/483541006 shutdown_connections 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/483541006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea883350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/483541006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4bd4d0 0x5585ea883890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/483541006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4de1b0 0x5585ea882e10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 -- 172.21.15.27:0/483541006 >> 172.21.15.27:0/483541006 conn(0x5585ea4d2300 msgr2=0x5585ea880d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd44de00640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4340b66b0 0x7fd4340b8bc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd440000c00 tx=0x7fd440031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 -- 172.21.15.27:0/483541006 shutdown_connections 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 -- 172.21.15.27:0/483541006 wait complete. 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.008+0000 7fd45ba8b740 1 -- start start 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 1 Processor -- start 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea88e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea88ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5585ea71fab0 con 0x5585ea4b52f0 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5585ea78e570 con 0x5585ea4bd4d0 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5585ea891590 con 0x5585ea4de1b0 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd45ba8b740 10 client.0 _ll_register_callbacks cb 0x5585ea79d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd44de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd44de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35400/0 (socket says 172.21.15.27:35400) 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd44de00640 1 -- 172.21.15.27:0/1348265378 learned_addr learned my addr 172.21.15.27:0/1348265378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.009+0000 7fd44d400640 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea88e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd44e800640 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea88ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd44de00640 1 -- 172.21.15.27:0/1348265378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 msgr2=0x5585ea88ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd44de00640 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea88ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd44de00640 1 -- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 msgr2=0x5585ea88e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd44de00640 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea88e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd44de00640 1 -- 172.21.15.27:0/1348265378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ea549790 con 0x5585ea4b52f0 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd44de00640 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88e4a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd440033000 tx=0x7fd440009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a189d0b4b7a56490 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.010+0000 7fd42f400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd42f400640 1 -- 172.21.15.27:0/1348265378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44004d030 con 0x5585ea4b52f0 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd42f400640 1 -- 172.21.15.27:0/1348265378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fd440004250 con 0x5585ea4b52f0 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43c00e3c0 con 0x5585ea4b52f0 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd42f400640 1 -- 172.21.15.27:0/1348265378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd44005a050 con 0x5585ea4b52f0 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd45ba8b740 10 client.18153 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585ea5c2620 con 0x5585ea4b52f0 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd42f400640 1 -- 172.21.15.27:0/1348265378 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43c00e3c0 con 0x5585ea4b52f0 2024-09-21T07:22:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.011+0000 7fd42f400640 1 -- 172.21.15.27:0/1348265378 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd44003d570 con 0x5585ea4b52f0 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.012+0000 7fd45ba8b740 10 client.18153 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.012+0000 7fd45ba8b740 10 client.18153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.012+0000 7fd45ba8b740 20 client.18153 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.012+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea49cd80 0x5585ea8a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.012+0000 7fd45ba8b740 4 client.18153 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.012+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585ea7fa600 con 0x5585ea49cd80 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.012+0000 7fd44d400640 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea49cd80 0x5585ea8a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.013+0000 7fd44d400640 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea49cd80 0x5585ea8a05b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc412eba65faf29 server_cookie=eeff1dcfa20393c4 in_seq=0 out_seq=0 2024-09-21T07:22:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.013+0000 7fd42f400640 10 client.18153 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd42f400640 1 -- 172.21.15.27:0/1348265378 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5585ea7fa600 con 0x5585ea49cd80 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd42f400640 10 client.18153 handle_command_reply: tid=0 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 client.18153 shutdown 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 msgr2=0x5585ea88e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88e4a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd440033000 tx=0x7fd440009260 comp rx=0 tx=0).stop 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 shutdown_connections 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea49cd80 0x5585ea8a05b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea88ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea88e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/1348265378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88e4a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd42f400640 0 client.18153 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 >> 172.21.15.27:0/1348265378 conn(0x5585ea4d2300 msgr2=0x5585ea883e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 shutdown_connections 2024-09-21T07:22:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 1 -- 172.21.15.27:0/1348265378 wait complete. 2024-09-21T07:22:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.014+0000 7fd45ba8b740 20 client.18153 trim_cache size 0 max 16384 2024-09-21T07:22:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 Processor -- start 2024-09-21T07:22:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 -- start start 2024-09-21T07:22:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea8805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea880af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b57a0 con 0x5585ea4b52f0 2024-09-21T07:22:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b3070 con 0x5585ea4bd4d0 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b0ee0 con 0x5585ea4de1b0 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd44de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd44de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35408/0 (socket says 172.21.15.27:35408) 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.030+0000 7fd44de00640 1 -- 172.21.15.27:0/3848387433 learned_addr learned my addr 172.21.15.27:0/3848387433 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44e800640 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea880af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44d400640 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea8805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44de00640 1 -- 172.21.15.27:0/3848387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 msgr2=0x5585ea880af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44de00640 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea880af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44de00640 1 -- 172.21.15.27:0/3848387433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 msgr2=0x5585ea8805b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44de00640 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea8805b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44de00640 1 -- 172.21.15.27:0/3848387433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ea885c20 con 0x5585ea4b52f0 2024-09-21T07:22:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44e800640 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea880af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44d400640 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea8805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd44de00640 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88bf90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd44002a930 tx=0x7fd440031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4519f95b441cf2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd43a600640 1 -- 172.21.15.27:0/3848387433 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44004d040 con 0x5585ea4b52f0 2024-09-21T07:22:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd43a600640 1 -- 172.21.15.27:0/3848387433 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44003ddd0 con 0x5585ea4b52f0 2024-09-21T07:22:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.031+0000 7fd43a600640 1 -- 172.21.15.27:0/3848387433 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440035040 con 0x5585ea4b52f0 2024-09-21T07:22:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 -- 172.21.15.27:0/3848387433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 msgr2=0x5585ea88bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88bf90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd44002a930 tx=0x7fd440031990 comp rx=0 tx=0).stop 2024-09-21T07:22:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 -- 172.21.15.27:0/3848387433 shutdown_connections 2024-09-21T07:22:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4de1b0 0x5585ea880af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea8805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/3848387433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4b52f0 0x5585ea88bf90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 -- 172.21.15.27:0/3848387433 >> 172.21.15.27:0/3848387433 conn(0x5585ea4d2300 msgr2=0x5585ea88b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 -- 172.21.15.27:0/3848387433 shutdown_connections 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.032+0000 7fd45ba8b740 1 -- 172.21.15.27:0/3848387433 wait complete. 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 -- start start 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 Processor -- start 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea892940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea892e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4de1b0 0x5585ea8933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b1530 con 0x5585ea4de1b0 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b29a0 con 0x5585ea4bd4d0 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b1c00 con 0x5585ea4b52f0 2024-09-21T07:22:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.033+0000 7fd45ba8b740 10 client.0 _ll_register_callbacks cb 0x7fd440009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea892e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4de1b0 0x5585ea8933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea892e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59356/0 (socket says 172.21.15.27:59356) 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 -- 172.21.15.27:0/2000645813 learned_addr learned my addr 172.21.15.27:0/2000645813 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44de00640 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea892940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 -- 172.21.15.27:0/2000645813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 msgr2=0x5585ea892940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea892940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 -- 172.21.15.27:0/2000645813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4de1b0 msgr2=0x5585ea8933c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4de1b0 0x5585ea8933c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 -- 172.21.15.27:0/2000645813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ea89a780 con 0x5585ea4bd4d0 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd44d400640 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea892e80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd43c000c00 tx=0x7fd43c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af4cbcef70dfecbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.034+0000 7fd42f400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd42f400640 1 -- 172.21.15.27:0/2000645813 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c037060 con 0x5585ea4bd4d0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44003ddd0 con 0x5585ea4bd4d0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd42f400640 1 -- 172.21.15.27:0/2000645813 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9461+0+0 (secure 0 0 0) 0x7fd43c005040 con 0x5585ea4bd4d0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd42f400640 1 -- 172.21.15.27:0/2000645813 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd43c03e050 con 0x5585ea4bd4d0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd45ba8b740 10 client.28193 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585ea90b1e0 con 0x5585ea4bd4d0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd42f400640 1 -- 172.21.15.27:0/2000645813 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44003ddd0 con 0x5585ea4bd4d0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.035+0000 7fd42f400640 1 -- 172.21.15.27:0/2000645813 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd43c00e3c0 con 0x5585ea4bd4d0 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd45ba8b740 10 client.28193 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd45ba8b740 10 client.28193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd45ba8b740 20 client.28193 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea4721c0 0x5585ea90d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd45ba8b740 4 client.28193 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:22:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585ea7f60b0 con 0x5585ea4721c0 2024-09-21T07:22:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd44de00640 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea4721c0 0x5585ea90d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd44de00640 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea4721c0 0x5585ea90d6f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a3c8ac9b0902b5 server_cookie=e624af5fcc5c4e4f in_seq=0 out_seq=0 2024-09-21T07:22:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.036+0000 7fd42f400640 10 client.28193 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.049+0000 7fd42f400640 1 -- 172.21.15.27:0/2000645813 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585ea7f60b0 con 0x5585ea4721c0 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.049+0000 7fd42f400640 10 client.28193 handle_command_reply: tid=0 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.049+0000 7fd45ba8b740 1 client.28193 shutdown 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 msgr2=0x5585ea892e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea892e80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd43c000c00 tx=0x7fd43c005850 comp rx=0 tx=0).stop 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 shutdown_connections 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585ea4721c0 0x5585ea90d6f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585ea4de1b0 0x5585ea8933c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ea4bd4d0 0x5585ea892e80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 --2- 172.21.15.27:0/2000645813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585ea4b52f0 0x5585ea892940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd42f400640 0 client.28193 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 >> 172.21.15.27:0/2000645813 conn(0x5585ea4d2300 msgr2=0x5585ea8805b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 shutdown_connections 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 1 -- 172.21.15.27:0/2000645813 wait complete. 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.050+0000 7fd45ba8b740 20 client.28193 trim_cache size 0 max 16384 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:22:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:22:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4340b66b0 msgr2=0x7fd4340b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4340b66b0 0x7fd4340b8bc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd440000c00 tx=0x7fd440031400 comp rx=0 tx=0).stop 2024-09-21T07:22:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450109ba0 msgr2=0x7fd450076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450109ba0 0x7fd450076190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd44800eb10 tx=0x7fd44800c2d0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 shutdown_connections 2024-09-21T07:22:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450109250 0x7fd450075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4340b66b0 0x7fd4340b8bc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd450109ba0 0x7fd450076190 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 --2- 172.21.15.27:0/1334913608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd450103920 0x7fd450075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 >> 172.21.15.27:0/1334913608 conn(0x7fd4500ff320 msgr2=0x7fd450108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 shutdown_connections 2024-09-21T07:22:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:02.051+0000 7fd44f200640 1 -- 172.21.15.27:0/1334913608 wait complete. 2024-09-21T07:22:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:02 smithi027 ceph-mon[30779]: pgmap v654: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:22:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:02 smithi153 ceph-mon[34747]: pgmap v654: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:22:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:02 smithi119 ceph-mon[34519]: pgmap v654: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:22:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:04 smithi027 ceph-mon[30779]: pgmap v655: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-09-21T07:22:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:04 smithi027 ceph-mon[30779]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T07:22:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:04 smithi153 ceph-mon[34747]: pgmap v655: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-09-21T07:22:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:04 smithi153 ceph-mon[34747]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T07:22:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:04 smithi119 ceph-mon[34519]: pgmap v655: 129 pgs: 129 active+clean; 4.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-09-21T07:22:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:04 smithi119 ceph-mon[34519]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T07:22:06.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:06 smithi153 ceph-mon[34747]: pgmap v657: 129 pgs: 129 active+clean; 4.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 KiB/s wr, 32 op/s 2024-09-21T07:22:06.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:22:06.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:06.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:06 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:06 smithi119 ceph-mon[34519]: pgmap v657: 129 pgs: 129 active+clean; 4.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 KiB/s wr, 32 op/s 2024-09-21T07:22:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:22:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:06 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:06 smithi027 ceph-mon[30779]: pgmap v657: 129 pgs: 129 active+clean; 4.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 KiB/s wr, 32 op/s 2024-09-21T07:22:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:22:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:07.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:06 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:08 smithi119 ceph-mon[34519]: pgmap v658: 129 pgs: 129 active+clean; 4.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:08.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:08 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:08 smithi027 ceph-mon[30779]: pgmap v658: 129 pgs: 129 active+clean; 4.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:08 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:08 smithi153 ceph-mon[34747]: pgmap v658: 129 pgs: 129 active+clean; 4.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:08.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:08.928 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:08 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:09.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:22:09.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:22:09.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:09.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:22:09.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:22:09.927 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:10.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:22:10.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:22:10.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:22:10.829 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:10 smithi153 ceph-mon[34747]: pgmap v659: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:10.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:10 smithi027 ceph-mon[30779]: pgmap v659: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:10 smithi119 ceph-mon[34519]: pgmap v659: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:12 smithi119 ceph-mon[34519]: pgmap v660: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:12 smithi119 ceph-mon[34519]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T07:22:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:12 smithi027 ceph-mon[30779]: pgmap v660: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:12 smithi027 ceph-mon[30779]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T07:22:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:12 smithi153 ceph-mon[34747]: pgmap v660: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:22:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:12 smithi153 ceph-mon[34747]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T07:22:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:14 smithi119 ceph-mon[34519]: pgmap v662: 129 pgs: 129 active+clean; 4.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-21T07:22:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:14 smithi027 ceph-mon[30779]: pgmap v662: 129 pgs: 129 active+clean; 4.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-21T07:22:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:14 smithi153 ceph-mon[34747]: pgmap v662: 129 pgs: 129 active+clean; 4.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-21T07:22:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 -- 172.21.15.27:0/446795101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7010fe70 msgr2=0x7fee701122d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 --2- 172.21.15.27:0/446795101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7010fe70 0x7fee701122d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fee68009be0 tx=0x7fee6802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 -- 172.21.15.27:0/446795101 shutdown_connections 2024-09-21T07:22:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 --2- 172.21.15.27:0/446795101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7010fe70 0x7fee701122d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 --2- 172.21.15.27:0/446795101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee70076100 0x7fee7010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 --2- 172.21.15.27:0/446795101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee700757b0 0x7fee70075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 -- 172.21.15.27:0/446795101 >> 172.21.15.27:0/446795101 conn(0x7fee700ff320 msgr2=0x7fee70101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 -- 172.21.15.27:0/446795101 shutdown_connections 2024-09-21T07:22:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.257+0000 7fee74e00640 1 -- 172.21.15.27:0/446795101 wait complete. 2024-09-21T07:22:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.258+0000 7fee74e00640 1 Processor -- start 2024-09-21T07:22:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.258+0000 7fee74e00640 1 -- start start 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee74e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee700757b0 0x7fee7019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee74e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee70076100 0x7fee7019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee74e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7010fe70 0x7fee701a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee74e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee70115a40 con 0x7fee7010fe70 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee74e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee701158c0 con 0x7fee700757b0 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee74e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee70115bc0 con 0x7fee70076100 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7010fe70 0x7fee701a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7010fe70 0x7fee701a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51786/0 (socket says 172.21.15.27:51786) 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee6fe00640 1 -- 172.21.15.27:0/3056724412 learned_addr learned my addr 172.21.15.27:0/3056724412 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee6ea00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee70076100 0x7fee7019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.259+0000 7fee6f400640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee700757b0 0x7fee7019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6fe00640 1 -- 172.21.15.27:0/3056724412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee70076100 msgr2=0x7fee7019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6fe00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee70076100 0x7fee7019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6fe00640 1 -- 172.21.15.27:0/3056724412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee700757b0 msgr2=0x7fee7019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6fe00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee700757b0 0x7fee7019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6fe00640 1 -- 172.21.15.27:0/3056724412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee701a2330 con 0x7fee7010fe70 2024-09-21T07:22:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6ea00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee70076100 0x7fee7019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6f400640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee700757b0 0x7fee7019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee6fe00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7010fe70 0x7fee701a1c30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fee68009b30 tx=0x7fee68003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee67e00640 1 -- 172.21.15.27:0/3056724412 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68040020 con 0x7fee7010fe70 2024-09-21T07:22:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee67e00640 1 -- 172.21.15.27:0/3056724412 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee68030970 con 0x7fee7010fe70 2024-09-21T07:22:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee701a25c0 con 0x7fee7010fe70 2024-09-21T07:22:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee67e00640 1 -- 172.21.15.27:0/3056724412 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68036040 con 0x7fee7010fe70 2024-09-21T07:22:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.260+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee701a8db0 con 0x7fee7010fe70 2024-09-21T07:22:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.264+0000 7fee7d465740 1 Processor -- start 2024-09-21T07:22:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.264+0000 7fee67e00640 1 -- 172.21.15.27:0/3056724412 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fee68043020 con 0x7fee7010fe70 2024-09-21T07:22:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.264+0000 7fee7d465740 1 -- start start 2024-09-21T07:22:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee7d465740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b173e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee7d465740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b174350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee7d465740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adae4d0 0x55e41b174890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee7d465740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e41b0e96b0 con 0x55e41ada62f0 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee7d465740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e41b0d8ae0 con 0x55e41adcf1b0 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee7d465740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e41b0dc8f0 con 0x55e41adae4d0 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b174350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b173e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b174350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51792/0 (socket says 172.21.15.27:51792) 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b173e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44574/0 (socket says 172.21.15.27:44574) 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee6f400640 1 -- 172.21.15.27:0/747484827 learned_addr learned my addr 172.21.15.27:0/747484827 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.265+0000 7fee6ea00640 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adae4d0 0x55e41b174890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee67e00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fee540b6680 0x7fee540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6ea00640 1 -- 172.21.15.27:0/747484827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adae4d0 msgr2=0x55e41b174890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6ea00640 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adae4d0 0x55e41b174890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6ea00640 1 -- 172.21.15.27:0/747484827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 msgr2=0x55e41b174350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6ea00640 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b174350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6ea00640 1 -- 172.21.15.27:0/747484827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41b0a5780 con 0x55e41adcf1b0 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6f400640 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b174350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee67e00640 1 -- 172.21.15.27:0/3056724412 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fee68108720 con 0x7fee7010fe70 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6fe00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fee540b6680 0x7fee540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee6ea00640 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b173e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009530 tx=0x7fee5c0095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72cb7bc7cf2addcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee64c00640 1 -- 172.21.15.27:0/747484827 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00fc20 con 0x55e41adcf1b0 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.266+0000 7fee64c00640 1 -- 172.21.15.27:0/747484827 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5c00fdc0 con 0x55e41adcf1b0 2024-09-21T07:22:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee64c00640 1 -- 172.21.15.27:0/747484827 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004a00 con 0x55e41adcf1b0 2024-09-21T07:22:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 -- 172.21.15.27:0/747484827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 msgr2=0x55e41b173e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b173e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009530 tx=0x7fee5c0095b0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee6fe00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fee540b6680 0x7fee540b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee68034040 tx=0x7fee68009370 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 -- 172.21.15.27:0/747484827 shutdown_connections 2024-09-21T07:22:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adae4d0 0x55e41b174890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b174350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 --2- 172.21.15.27:0/747484827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b173e10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 -- 172.21.15.27:0/747484827 >> 172.21.15.27:0/747484827 conn(0x55e41adc3300 msgr2=0x55e41b171d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.267+0000 7fee7d465740 1 -- 172.21.15.27:0/747484827 shutdown_connections 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.268+0000 7fee7d465740 1 -- 172.21.15.27:0/747484827 wait complete. 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.268+0000 7fee7d465740 1 -- start start 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.268+0000 7fee7d465740 1 Processor -- start 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee7d465740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41ada62f0 0x55e41b17f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee7d465740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b17fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee7d465740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b18b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee7d465740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e41b010ab0 con 0x55e41adae4d0 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee7d465740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e41b07f570 con 0x55e41ada62f0 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee7d465740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e41b182590 con 0x55e41adcf1b0 2024-09-21T07:22:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee7d465740 10 client.0 _ll_register_callbacks cb 0x55e41b08e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b17fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b17fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51800/0 (socket says 172.21.15.27:51800) 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee6f400640 1 -- 172.21.15.27:0/64473975 learned_addr learned my addr 172.21.15.27:0/64473975 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee6ea00640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41ada62f0 0x55e41b17f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.269+0000 7fee6ea00640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b18b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee6f400640 1 -- 172.21.15.27:0/64473975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 msgr2=0x55e41b18b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee6f400640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b18b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee6f400640 1 -- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41ada62f0 msgr2=0x55e41b17f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee6f400640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41ada62f0 0x55e41b17f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee6f400640 1 -- 172.21.15.27:0/64473975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41ae3a790 con 0x55e41adae4d0 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee6ea00640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b18b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee6f400640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b17fa20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fee60012d20 tx=0x7fee6000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64ca044282ccd9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee59600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee59600640 1 -- 172.21.15.27:0/64473975 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60004030 con 0x55e41adae4d0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee59600640 1 -- 172.21.15.27:0/64473975 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fee600041d0 con 0x55e41adae4d0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5c00fdc0 con 0x55e41adae4d0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee59600640 1 -- 172.21.15.27:0/64473975 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee600043d0 con 0x55e41adae4d0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee7d465740 10 client.18171 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.270+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e41aeb3620 con 0x55e41adae4d0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee59600640 1 -- 172.21.15.27:0/64473975 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5c00fdc0 con 0x55e41adae4d0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee59600640 1 -- 172.21.15.27:0/64473975 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fee6001c460 con 0x55e41adae4d0 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee7d465740 10 client.18171 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee7d465740 10 client.18171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee7d465740 20 client.18171 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee7d465740 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad8dd80 0x55e41b191640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee7d465740 4 client.18171 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e41b0eb600 con 0x55e41ad8dd80 2024-09-21T07:22:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.271+0000 7fee6ea00640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad8dd80 0x55e41b191640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.272+0000 7fee6ea00640 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad8dd80 0x55e41b191640 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de51919281158061 server_cookie=2a094b62e8442209 in_seq=0 out_seq=0 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.272+0000 7fee59600640 10 client.18171 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee59600640 1 -- 172.21.15.27:0/64473975 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e41b0eb600 con 0x55e41ad8dd80 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee59600640 10 client.18171 handle_command_reply: tid=0 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 client.18171 shutdown 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 msgr2=0x55e41b17fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b17fa20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fee60012d20 tx=0x7fee6000f790 comp rx=0 tx=0).stop 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 shutdown_connections 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad8dd80 0x55e41b191640 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b18b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b17fa20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee59600640 0 client.18171 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 --2- 172.21.15.27:0/64473975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41ada62f0 0x55e41b17f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 >> 172.21.15.27:0/64473975 conn(0x55e41adc3300 msgr2=0x55e41b174e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 shutdown_connections 2024-09-21T07:22:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 1 -- 172.21.15.27:0/64473975 wait complete. 2024-09-21T07:22:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.273+0000 7fee7d465740 20 client.18171 trim_cache size 0 max 16384 2024-09-21T07:22:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 Processor -- start 2024-09-21T07:22:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 -- start start 2024-09-21T07:22:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b170d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adae4d0 0x55e41b175550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b196e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b57b0 con 0x55e41ada62f0 2024-09-21T07:22:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b3080 con 0x55e41adae4d0 2024-09-21T07:22:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee7d465740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b0ef0 con 0x55e41adcf1b0 2024-09-21T07:22:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b170d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b170d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51814/0 (socket says 172.21.15.27:51814) 2024-09-21T07:22:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee6ea00640 1 -- 172.21.15.27:0/2307675214 learned_addr learned my addr 172.21.15.27:0/2307675214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.289+0000 7fee6f400640 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adae4d0 0x55e41b175550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6ea00640 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6ea00640 1 -- 172.21.15.27:0/2307675214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 msgr2=0x55e41b196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6ea00640 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6ea00640 1 -- 172.21.15.27:0/2307675214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adae4d0 msgr2=0x55e41b175550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6ea00640 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adae4d0 0x55e41b175550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6ea00640 1 -- 172.21.15.27:0/2307675214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41b176c20 con 0x55e41ada62f0 2024-09-21T07:22:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6f400640 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adae4d0 0x55e41b175550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee6ea00640 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b170d20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fee5c005590 tx=0x7fee5c0055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ef8105be41b1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee5be00640 1 -- 172.21.15.27:0/2307675214 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00fd40 con 0x55e41ada62f0 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.290+0000 7fee5be00640 1 -- 172.21.15.27:0/2307675214 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5c004020 con 0x55e41ada62f0 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee5be00640 1 -- 172.21.15.27:0/2307675214 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c003870 con 0x55e41ada62f0 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 -- 172.21.15.27:0/2307675214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 msgr2=0x55e41b170d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b170d20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fee5c005590 tx=0x7fee5c0055c0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 -- 172.21.15.27:0/2307675214 shutdown_connections 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41adcf1b0 0x55e41b196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adae4d0 0x55e41b175550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 --2- 172.21.15.27:0/2307675214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41ada62f0 0x55e41b170d20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 -- 172.21.15.27:0/2307675214 >> 172.21.15.27:0/2307675214 conn(0x55e41adc3300 msgr2=0x55e41b174e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 -- 172.21.15.27:0/2307675214 shutdown_connections 2024-09-21T07:22:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.291+0000 7fee7d465740 1 -- 172.21.15.27:0/2307675214 wait complete. 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.292+0000 7fee7d465740 1 -- start start 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.292+0000 7fee7d465740 1 Processor -- start 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.292+0000 7fee7d465740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41ada62f0 0x55e41b187cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.292+0000 7fee7d465740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b185190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee7d465740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b1856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee7d465740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b1540 con 0x55e41adae4d0 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee7d465740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b29b0 con 0x55e41adcf1b0 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee7d465740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b1c10 con 0x55e41ada62f0 2024-09-21T07:22:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee7d465740 10 client.0 _ll_register_callbacks cb 0x7fee60009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b185190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41ada62f0 0x55e41b187cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b185190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51820/0 (socket says 172.21.15.27:51820) 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 -- 172.21.15.27:0/85704000 learned_addr learned my addr 172.21.15.27:0/85704000 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6ea00640 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b1856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 -- 172.21.15.27:0/85704000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41ada62f0 msgr2=0x55e41b187cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41ada62f0 0x55e41b187cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 -- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 msgr2=0x55e41b1856d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b1856d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6f400640 1 -- 172.21.15.27:0/85704000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41b185c10 con 0x55e41adae4d0 2024-09-21T07:22:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.293+0000 7fee6ea00640 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b1856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee6f400640 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b185190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fee6000a010 tx=0x7fee6000fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f39aa44e8fa3c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee59600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee59600640 1 -- 172.21.15.27:0/85704000 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6002e060 con 0x55e41adae4d0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee59600640 1 -- 172.21.15.27:0/85704000 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fee60005030 con 0x55e41adae4d0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5c004020 con 0x55e41adae4d0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee59600640 1 -- 172.21.15.27:0/85704000 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee60005230 con 0x55e41adae4d0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee7d465740 10 client.18177 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.294+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e41b1fbcf0 con 0x55e41adae4d0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee59600640 1 -- 172.21.15.27:0/85704000 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5c004020 con 0x55e41adae4d0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee59600640 1 -- 172.21.15.27:0/85704000 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fee60029450 con 0x55e41adae4d0 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee7d465740 10 client.18177 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee7d465740 10 client.18177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee7d465740 20 client.18177 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee7d465740 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad631c0 0x55e41b1fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee7d465740 4 client.18177 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:22:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e41b0e70b0 con 0x55e41ad631c0 2024-09-21T07:22:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.295+0000 7fee6ea00640 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad631c0 0x55e41b1fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.296+0000 7fee6ea00640 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad631c0 0x55e41b1fe4e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3710b02df4fb3314 server_cookie=922608a3b56c8b37 in_seq=0 out_seq=0 2024-09-21T07:22:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.296+0000 7fee59600640 10 client.18177 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.309+0000 7fee59600640 1 -- 172.21.15.27:0/85704000 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e41b0e70b0 con 0x55e41ad631c0 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.309+0000 7fee59600640 10 client.18177 handle_command_reply: tid=0 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.309+0000 7fee7d465740 1 client.18177 shutdown 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 msgr2=0x55e41b185190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b185190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fee6000a010 tx=0x7fee6000fee0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 shutdown_connections 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e41ad631c0 0x55e41b1fe4e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e41adcf1b0 0x55e41b1856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e41adae4d0 0x55e41b185190 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 --2- 172.21.15.27:0/85704000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e41ada62f0 0x55e41b187cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee59600640 0 client.18177 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 >> 172.21.15.27:0/85704000 conn(0x55e41adc3300 msgr2=0x55e41b170af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 shutdown_connections 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 1 -- 172.21.15.27:0/85704000 wait complete. 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.310+0000 7fee7d465740 20 client.18177 trim_cache size 0 max 16384 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:22:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:22:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fee540b6680 msgr2=0x7fee540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fee540b6680 0x7fee540b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee68034040 tx=0x7fee68009370 comp rx=0 tx=0).stop 2024-09-21T07:22:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7010fe70 msgr2=0x7fee701a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7010fe70 0x7fee701a1c30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fee68009b30 tx=0x7fee68003c10 comp rx=0 tx=0).stop 2024-09-21T07:22:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 shutdown_connections 2024-09-21T07:22:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee70076100 0x7fee7019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fee540b6680 0x7fee540b8b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7010fe70 0x7fee701a1c30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 --2- 172.21.15.27:0/3056724412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee700757b0 0x7fee7019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 >> 172.21.15.27:0/3056724412 conn(0x7fee700ff320 msgr2=0x7fee70109980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 shutdown_connections 2024-09-21T07:22:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:15.311+0000 7fee74e00640 1 -- 172.21.15.27:0/3056724412 wait complete. 2024-09-21T07:22:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:16 smithi119 ceph-mon[34519]: pgmap v663: 129 pgs: 129 active+clean; 4.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 68 KiB/s wr, 15 op/s 2024-09-21T07:22:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:16 smithi027 ceph-mon[30779]: pgmap v663: 129 pgs: 129 active+clean; 4.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 68 KiB/s wr, 15 op/s 2024-09-21T07:22:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:16 smithi153 ceph-mon[34747]: pgmap v663: 129 pgs: 129 active+clean; 4.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 68 KiB/s wr, 15 op/s 2024-09-21T07:22:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:18 smithi119 ceph-mon[34519]: pgmap v664: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-09-21T07:22:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:18 smithi027 ceph-mon[30779]: pgmap v664: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-09-21T07:22:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:18 smithi153 ceph-mon[34747]: pgmap v664: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-09-21T07:22:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:20 smithi119 ceph-mon[34519]: pgmap v665: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-09-21T07:22:21.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:20 smithi027 ceph-mon[30779]: pgmap v665: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-09-21T07:22:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:20 smithi153 ceph-mon[34747]: pgmap v665: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-09-21T07:22:22.064 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.064 INFO:tasks.workunit.client.0.smithi027.stdout:251/600: open fd = 3 2024-09-21T07:22:22.064 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:252/600: open fd = 3 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:253/600: open fd = 3 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:254/600: open fd = 3 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:255/600: open fd = 3 2024-09-21T07:22:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:256/600: open fd = 3 2024-09-21T07:22:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:257/600: open fd = 3 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:258/600: open fd = 3 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:259/600: open fd = 3 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:260/600: open fd = 3 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:261/600: open fd = 3 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:262/600: open fd = 3 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:263/600: open fd = 3 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:264/600: open fd = 3 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:265/600: open fd = 3 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:266/600: open fd = 3 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:267/600: open fd = 3 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:268/600: open fd = 3 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:269/600: open fd = 3 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:270/600: open fd = 3 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:271/600: open fd = 3 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.071 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:272/600: open fd = 3 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:273/600: open fd = 3 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:274/600: open fd = 3 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.072 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:275/600: open fd = 3 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:276/600: open fd = 3 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:277/600: open fd = 3 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:278/600: open fd = 3 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:279/600: open fd = 3 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:280/600: open fd = 3 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:281/600: open fd = 3 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:282/600: open fd = 3 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:283/600: open fd = 3 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:284/600: open fd = 3 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:285/600: open fd = 3 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:286/600: open fd = 3 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:287/600: open fd = 3 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:288/600: open fd = 3 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:289/600: open fd = 3 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:290/600: open fd = 3 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:291/600: open fd = 3 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:292/600: open fd = 3 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:293/600: open fd = 3 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.078 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:294/600: open fd = 3 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:295/600: open fd = 3 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:296/600: open fd = 3 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:297/600: open fd = 3 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:298/600: open fd = 3 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:299/600: open fd = 3 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:300/600: open fd = 3 2024-09-21T07:22:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:301/600: open fd = 3 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:302/600: open fd = 3 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:303/600: open fd = 3 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:304/600: open fd = 3 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:305/600: open fd = 3 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:306/600: open fd = 3 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:307/600: open fd = 3 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:308/600: open fd = 3 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:309/600: open fd = 3 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:310/600: open fd = 3 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:311/600: open fd = 3 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:312/600: open fd = 3 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:22:22.926 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:22:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:22 smithi119 ceph-mon[34519]: pgmap v666: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:22:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:22 smithi027 ceph-mon[30779]: pgmap v666: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:22:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:22 smithi153 ceph-mon[34747]: pgmap v666: 129 pgs: 129 active+clean; 4.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:22:23.324 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:23.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.922+0000 7f0639c00640 1 -- 172.21.15.27:0/510682067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0634105d80 msgr2=0x7f0634106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.922+0000 7f0639c00640 1 --2- 172.21.15.27:0/510682067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0634105d80 0x7f0634106190 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0624009900 tx=0x7f062402a410 comp rx=0 tx=0).stop 2024-09-21T07:22:23.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.923+0000 7f0639c00640 1 -- 172.21.15.27:0/510682067 shutdown_connections 2024-09-21T07:22:23.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.923+0000 7f0639c00640 1 --2- 172.21.15.27:0/510682067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f063410c010 0x7f0634112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.923+0000 7f0639c00640 1 --2- 172.21.15.27:0/510682067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f063410b6c0 0x7f063410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.923+0000 7f0639c00640 1 --2- 172.21.15.27:0/510682067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0634105d80 0x7f0634106190 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.923+0000 7f0639c00640 1 -- 172.21.15.27:0/510682067 >> 172.21.15.27:0/510682067 conn(0x7f0634101810 msgr2=0x7f0634103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:23.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.923+0000 7f0639c00640 1 -- 172.21.15.27:0/510682067 shutdown_connections 2024-09-21T07:22:23.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.923+0000 7f0639c00640 1 -- 172.21.15.27:0/510682067 wait complete. 2024-09-21T07:22:23.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 Processor -- start 2024-09-21T07:22:23.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 -- start start 2024-09-21T07:22:23.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0634105d80 0x7f063419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063410b6c0 0x7f063419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f063410c010 0x7f06341a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0634115cd0 con 0x7f063410b6c0 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0634115b50 con 0x7f063410c010 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.924+0000 7f0639c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0634115e50 con 0x7f0634105d80 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f062be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063410b6c0 0x7f063419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f0632a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0634105d80 0x7f063419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f062be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063410b6c0 0x7f063419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38978/0 (socket says 172.21.15.27:38978) 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f062be00640 1 -- 172.21.15.27:0/2092320489 learned_addr learned my addr 172.21.15.27:0/2092320489 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f0633400640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f063410c010 0x7f06341a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f0632a00640 1 -- 172.21.15.27:0/2092320489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f063410c010 msgr2=0x7f06341a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f0632a00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f063410c010 0x7f06341a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f0632a00640 1 -- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063410b6c0 msgr2=0x7f063419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f0632a00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063410b6c0 0x7f063419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f0632a00640 1 -- 172.21.15.27:0/2092320489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341a2650 con 0x7f0634105d80 2024-09-21T07:22:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.925+0000 7f062be00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063410b6c0 0x7f063419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.926+0000 7f0632a00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0634105d80 0x7f063419c6e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0624006010 tx=0x7f0624004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.926+0000 7f062b400640 1 -- 172.21.15.27:0/2092320489 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062403f070 con 0x7f0634105d80 2024-09-21T07:22:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.926+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06341a28e0 con 0x7f0634105d80 2024-09-21T07:22:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.926+0000 7f062b400640 1 -- 172.21.15.27:0/2092320489 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0624003ca0 con 0x7f0634105d80 2024-09-21T07:22:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.926+0000 7f062b400640 1 -- 172.21.15.27:0/2092320489 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624035070 con 0x7f0634105d80 2024-09-21T07:22:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.926+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341a90a0 con 0x7f0634105d80 2024-09-21T07:22:23.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.929+0000 7f06407c3740 1 Processor -- start 2024-09-21T07:22:23.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.929+0000 7f06407c3740 1 -- start start 2024-09-21T07:22:23.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f06407c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2670ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f06407c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d2671002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f06407c3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f06407c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d267076f70 con 0x55d266d59070 2024-09-21T07:22:23.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f06407c3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d2670663a0 con 0x55d266d38390 2024-09-21T07:22:23.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f06407c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d26706a1b0 con 0x55d266d301b0 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f0633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2670ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f0633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2670ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38986/0 (socket says 172.21.15.27:38986) 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f0633400640 1 -- 172.21.15.27:0/307204168 learned_addr learned my addr 172.21.15.27:0/307204168 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f062b400640 1 -- 172.21.15.27:0/2092320489 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f062403b460 con 0x7f0634105d80 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.930+0000 7f062be00640 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f062be00640 1 -- 172.21.15.27:0/307204168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 msgr2=0x55d2671002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f062be00640 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d2671002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f062be00640 1 -- 172.21.15.27:0/307204168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 msgr2=0x55d2670ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f062be00640 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2670ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f062be00640 1 -- 172.21.15.27:0/307204168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d267030c90 con 0x55d266d38390 2024-09-21T07:22:23.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f0633400640 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2670ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:23.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f062be00640 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671007e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f061c0114c0 tx=0x7f061c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9beab22c2f076d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f0623e00640 1 -- 172.21.15.27:0/307204168 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c01e070 con 0x55d266d38390 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.931+0000 7f0623e00640 1 -- 172.21.15.27:0/307204168 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c0035f0 con 0x55d266d38390 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.932+0000 7f0623e00640 1 -- 172.21.15.27:0/307204168 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c00f760 con 0x55d266d38390 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.932+0000 7f062b400640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06180b6680 0x7f06180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.932+0000 7f062b400640 1 -- 172.21.15.27:0/2092320489 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f0624107be0 con 0x7f0634105d80 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.932+0000 7f06407c3740 1 -- 172.21.15.27:0/307204168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 msgr2=0x55d2671007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.932+0000 7f06407c3740 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671007e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f061c0114c0 tx=0x7f061c009f90 comp rx=0 tx=0).stop 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.932+0000 7f0633400640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06180b6680 0x7f06180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 -- 172.21.15.27:0/307204168 shutdown_connections 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d2671002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671007e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 --2- 172.21.15.27:0/307204168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2670ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 -- 172.21.15.27:0/307204168 >> 172.21.15.27:0/307204168 conn(0x55d266d4d460 msgr2=0x55d2670fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:23.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 -- 172.21.15.27:0/307204168 shutdown_connections 2024-09-21T07:22:23.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 -- 172.21.15.27:0/307204168 wait complete. 2024-09-21T07:22:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f0633400640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06180b6680 0x7f06180b8b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f062c005f40 tx=0x7f062c009600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.933+0000 7f06407c3740 1 -- start start 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 1 Processor -- start 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d301b0 0x55d26710b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d38390 0x55d26710ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d59070 0x55d267117860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d266ec7700 con 0x55d266d301b0 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d267001540 con 0x55d266d59070 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d26710e540 con 0x55d266d38390 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.934+0000 7f06407c3740 10 client.0 _ll_register_callbacks cb 0x55d267011430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0632a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d301b0 0x55d26710b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f062be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d38390 0x55d26710ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f062be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d38390 0x55d26710ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34984/0 (socket says 172.21.15.27:34984) 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f062be00640 1 -- 172.21.15.27:0/3785632725 learned_addr learned my addr 172.21.15.27:0/3785632725 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0633400640 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d59070 0x55d267117860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0632a00640 1 -- 172.21.15.27:0/3785632725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d38390 msgr2=0x55d26710ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0632a00640 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d38390 0x55d26710ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0632a00640 1 -- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d59070 msgr2=0x55d267117860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0632a00640 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d59070 0x55d267117860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0632a00640 1 -- 172.21.15.27:0/3785632725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d266dc4650 con 0x55d266d301b0 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f062be00640 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d38390 0x55d26710ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0632a00640 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d301b0 0x55d26710b4e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0624104f80 tx=0x7f06240c9df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2120d63ce11f8844 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.935+0000 7f0620c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f0620c00640 1 -- 172.21.15.27:0/3785632725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624048040 con 0x55d266d301b0 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f0620c00640 1 -- 172.21.15.27:0/3785632725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f06240037f0 con 0x55d266d301b0 2024-09-21T07:22:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d267003b50 con 0x55d266d301b0 2024-09-21T07:22:23.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f0620c00640 1 -- 172.21.15.27:0/3785632725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06240cd080 con 0x55d266d301b0 2024-09-21T07:22:23.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f06407c3740 10 client.18189 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d26701e910 con 0x55d266d301b0 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f0620c00640 1 -- 172.21.15.27:0/3785632725 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d267003b50 con 0x55d266d301b0 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.936+0000 7f0620c00640 1 -- 172.21.15.27:0/3785632725 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f06240c91a0 con 0x55d266d301b0 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.937+0000 7f06407c3740 10 client.18189 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.937+0000 7f06407c3740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.937+0000 7f06407c3740 20 client.18189 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.937+0000 7f06407c3740 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266d179f0 0x55d26711d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.937+0000 7f06407c3740 4 client.18189 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.937+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d267078ec0 con 0x55d266d179f0 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.937+0000 7f062be00640 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266d179f0 0x55d26711d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.938+0000 7f062be00640 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266d179f0 0x55d26711d630 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff7db49f2dd9a04 server_cookie=eddcc7c13c0d46e9 in_seq=0 out_seq=0 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.938+0000 7f0620c00640 10 client.18189 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.938+0000 7f0620c00640 1 -- 172.21.15.27:0/3785632725 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d267078ec0 con 0x55d266d179f0 2024-09-21T07:22:23.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.938+0000 7f0620c00640 10 client.18189 handle_command_reply: tid=0 2024-09-21T07:22:23.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 client.18189 shutdown 2024-09-21T07:22:23.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d301b0 msgr2=0x55d26710b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d301b0 0x55d26710b4e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0624104f80 tx=0x7f06240c9df0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 shutdown_connections 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266d179f0 0x55d26711d630 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d38390 0x55d26710ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d59070 0x55d267117860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 --2- 172.21.15.27:0/3785632725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d301b0 0x55d26710b4e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 >> 172.21.15.27:0/3785632725 conn(0x55d266d4d460 msgr2=0x55d267102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f0620c00640 0 client.18189 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 shutdown_connections 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 1 -- 172.21.15.27:0/3785632725 wait complete. 2024-09-21T07:22:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.939+0000 7f06407c3740 20 client.18189 trim_cache size 0 max 16384 2024-09-21T07:22:23.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 Processor -- start 2024-09-21T07:22:23.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 -- start start 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d26710bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d267103570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2671086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06341b59a0 con 0x55d266d59070 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f06341b3270 con 0x55d266d38390 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.958+0000 7f06407c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f06341b10e0 con 0x55d266d301b0 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d267103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d267103570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53134/0 (socket says 172.21.15.27:53134) 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f0632a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d26710bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 -- 172.21.15.27:0/1280865256 learned_addr learned my addr 172.21.15.27:0/1280865256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f0633400640 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2671086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 -- 172.21.15.27:0/1280865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 msgr2=0x55d26710bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d26710bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 -- 172.21.15.27:0/1280865256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 msgr2=0x55d2671086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2671086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 -- 172.21.15.27:0/1280865256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d267102050 con 0x55d266d38390 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.959+0000 7f062be00640 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d267103570 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f061c012fd0 tx=0x7f061c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fac8ba15356601a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.960+0000 7f0623400640 1 -- 172.21.15.27:0/1280865256 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c032070 con 0x55d266d38390 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.960+0000 7f0623400640 1 -- 172.21.15.27:0/1280865256 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c0043c0 con 0x55d266d38390 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.960+0000 7f0623400640 1 -- 172.21.15.27:0/1280865256 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c01e040 con 0x55d266d38390 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 -- 172.21.15.27:0/1280865256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 msgr2=0x55d267103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d267103570 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f061c012fd0 tx=0x7f061c005f90 comp rx=0 tx=0).stop 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 -- 172.21.15.27:0/1280865256 shutdown_connections 2024-09-21T07:22:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d2671086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d267103570 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 --2- 172.21.15.27:0/1280865256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d26710bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 -- 172.21.15.27:0/1280865256 >> 172.21.15.27:0/1280865256 conn(0x55d266d4d460 msgr2=0x55d267102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 -- 172.21.15.27:0/1280865256 shutdown_connections 2024-09-21T07:22:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 -- 172.21.15.27:0/1280865256 wait complete. 2024-09-21T07:22:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 -- start start 2024-09-21T07:22:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.961+0000 7f06407c3740 1 Processor -- start 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f06407c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d267113c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f06407c3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f06407c3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d26710f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f06407c3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06341b1730 con 0x55d266d59070 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f06407c3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f06341b2ba0 con 0x55d266d38390 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f06407c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f06341b1e00 con 0x55d266d301b0 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f06407c3740 10 client.0 _ll_register_callbacks cb 0x7f0624009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f0633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d26710f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f062be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f0632a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d267113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f0633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d26710f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38998/0 (socket says 172.21.15.27:38998) 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.962+0000 7f0633400640 1 -- 172.21.15.27:0/3586892777 learned_addr learned my addr 172.21.15.27:0/3586892777 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.963+0000 7f062be00640 1 -- 172.21.15.27:0/3586892777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 msgr2=0x55d267113c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.963+0000 7f062be00640 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d267113c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.963+0000 7f062be00640 1 -- 172.21.15.27:0/3586892777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 msgr2=0x55d26710f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.963+0000 7f062be00640 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d26710f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.963+0000 7f062be00640 1 -- 172.21.15.27:0/3586892777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26710faa0 con 0x55d266d38390 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.963+0000 7f062be00640 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671141d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55d26711a290 tx=0x7f061c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583d5e6fe911fd7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.963+0000 7f0620c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.964+0000 7f0620c00640 1 -- 172.21.15.27:0/3586892777 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c032070 con 0x55d266d38390 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.964+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061c0043c0 con 0x55d266d38390 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.964+0000 7f0620c00640 1 -- 172.21.15.27:0/3586892777 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f061c005280 con 0x55d266d38390 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.964+0000 7f0620c00640 1 -- 172.21.15.27:0/3586892777 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f061c042050 con 0x55d266d38390 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.964+0000 7f06407c3740 10 client.28229 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.964+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d267187b90 con 0x55d266d38390 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f0620c00640 1 -- 172.21.15.27:0/3586892777 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c0043c0 con 0x55d266d38390 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f0620c00640 1 -- 172.21.15.27:0/3586892777 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f061c004720 con 0x55d266d38390 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f06407c3740 10 client.28229 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f06407c3740 10 client.28229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f06407c3740 20 client.28229 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f06407c3740 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266cecda0 0x55d26718a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f06407c3740 4 client.28229 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d267074970 con 0x55d266cecda0 2024-09-21T07:22:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.965+0000 7f0632a00640 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266cecda0 0x55d26718a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f0632a00640 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266cecda0 0x55d26718a020 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63123a3af74b604 server_cookie=12a485f6941c3187 in_seq=0 out_seq=0 2024-09-21T07:22:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f0620c00640 10 client.28229 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f0620c00640 1 -- 172.21.15.27:0/3586892777 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x55d267074970 con 0x55d266cecda0 2024-09-21T07:22:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f0620c00640 10 client.28229 handle_command_reply: tid=0 2024-09-21T07:22:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 client.28229 shutdown 2024-09-21T07:22:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 msgr2=0x55d2671141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671141d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55d26711a290 tx=0x7f061c005d90 comp rx=0 tx=0).stop 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 shutdown_connections 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d266cecda0 0x55d26718a020 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d266d59070 0x55d26710f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d266d38390 0x55d2671141d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f0620c00640 0 client.28229 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.966+0000 7f06407c3740 1 --2- 172.21.15.27:0/3586892777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d266d301b0 0x55d267113c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.967+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 >> 172.21.15.27:0/3586892777 conn(0x55d266d4d460 msgr2=0x55d267109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.967+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 shutdown_connections 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.967+0000 7f06407c3740 1 -- 172.21.15.27:0/3586892777 wait complete. 2024-09-21T07:22:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.967+0000 7f06407c3740 20 client.28229 trim_cache size 0 max 16384 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:22:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:23.979 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06180b6680 msgr2=0x7f06180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06180b6680 0x7f06180b8b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f062c005f40 tx=0x7f062c009600 comp rx=0 tx=0).stop 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0634105d80 msgr2=0x7f063419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0634105d80 0x7f063419c6e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0624006010 tx=0x7f0624004060 comp rx=0 tx=0).stop 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 shutdown_connections 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f06180b6680 0x7f06180b8b90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f063410c010 0x7f06341a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063410b6c0 0x7f063419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 --2- 172.21.15.27:0/2092320489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0634105d80 0x7f063419c6e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 >> 172.21.15.27:0/2092320489 conn(0x7f0634101810 msgr2=0x7f0634103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 shutdown_connections 2024-09-21T07:22:23.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:23.968+0000 7f0639c00640 1 -- 172.21.15.27:0/2092320489 wait complete. 2024-09-21T07:22:24.971 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:22:24.972 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:22:25.023 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:24 smithi027 ceph-mon[30779]: pgmap v667: 129 pgs: 129 active+clean; 4.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:22:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:24 smithi119 ceph-mon[34519]: pgmap v667: 129 pgs: 129 active+clean; 4.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:22:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:24 smithi153 ceph-mon[34747]: pgmap v667: 129 pgs: 129 active+clean; 4.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:22:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.973+0000 7f4114400640 1 -- 172.21.15.27:0/3202047593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 msgr2=0x7f410c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.973+0000 7f4114400640 1 --2- 172.21.15.27:0/3202047593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 0x7f410c112840 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4104009ca0 tx=0x7f410402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:22:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.973+0000 7f4114400640 1 -- 172.21.15.27:0/3202047593 shutdown_connections 2024-09-21T07:22:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.973+0000 7f4114400640 1 --2- 172.21.15.27:0/3202047593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f410c10b6c0 0x7f410c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.973+0000 7f4114400640 1 --2- 172.21.15.27:0/3202047593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 0x7f410c112840 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.973+0000 7f4114400640 1 --2- 172.21.15.27:0/3202047593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 0x7f410c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.973+0000 7f4114400640 1 -- 172.21.15.27:0/3202047593 >> 172.21.15.27:0/3202047593 conn(0x7f410c101810 msgr2=0x7f410c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.974+0000 7f4114400640 1 -- 172.21.15.27:0/3202047593 shutdown_connections 2024-09-21T07:22:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.974+0000 7f4114400640 1 -- 172.21.15.27:0/3202047593 wait complete. 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 Processor -- start 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 -- start start 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 0x7f410c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f410c10b6c0 0x7f410c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 0x7f410c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f410c115cd0 con 0x7f410c10c010 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f410c115b50 con 0x7f410c105d80 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.975+0000 7f4114400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f410c115e50 con 0x7f410c10b6c0 2024-09-21T07:22:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 0x7f410c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 0x7f410c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53164/0 (socket says 172.21.15.27:53164) 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f410c10b6c0 0x7f410c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 -- 172.21.15.27:0/3991132272 learned_addr learned my addr 172.21.15.27:0/3991132272 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4112400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 0x7f410c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 -- 172.21.15.27:0/3991132272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f410c10b6c0 msgr2=0x7f410c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f410c10b6c0 0x7f410c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 -- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 msgr2=0x7f410c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 0x7f410c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 -- 172.21.15.27:0/3991132272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410c1a2690 con 0x7f410c105d80 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4112400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 0x7f410c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111000640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f410c10b6c0 0x7f410c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.976+0000 7f4111a00640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 0x7f410c19c720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f410000e9f0 tx=0x7f410000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.977+0000 7f410a000640 1 -- 172.21.15.27:0/3991132272 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410000cd80 con 0x7f410c105d80 2024-09-21T07:22:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.977+0000 7f410a000640 1 -- 172.21.15.27:0/3991132272 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41000108e0 con 0x7f410c105d80 2024-09-21T07:22:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.977+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410c1a2980 con 0x7f410c105d80 2024-09-21T07:22:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.977+0000 7f410a000640 1 -- 172.21.15.27:0/3991132272 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100018980 con 0x7f410c105d80 2024-09-21T07:22:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.977+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f410c1a90e0 con 0x7f410c105d80 2024-09-21T07:22:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.978+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410c106190 con 0x7f410c105d80 2024-09-21T07:22:25.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.981+0000 7f410a000640 1 -- 172.21.15.27:0/3991132272 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4100010430 con 0x7f410c105d80 2024-09-21T07:22:25.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.982+0000 7f410a000640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40f40b6700 0x7f40f40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:25.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.983+0000 7f410a000640 1 -- 172.21.15.27:0/3991132272 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f41000e8310 con 0x7f410c105d80 2024-09-21T07:22:25.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.983+0000 7f4111000640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40f40b6700 0x7f40f40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:25.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.983+0000 7f4111000640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40f40b6700 0x7f40f40b8c10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f410c104f10 tx=0x7f40f8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:25.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:25.984+0000 7f410a000640 1 -- 172.21.15.27:0/3991132272 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41000ae450 con 0x7f410c105d80 2024-09-21T07:22:26.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.223+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f410c19d850 con 0x7f410c105d80 2024-09-21T07:22:26.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.224+0000 7f410a000640 1 -- 172.21.15.27:0/3991132272 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f410c19d850 con 0x7f410c105d80 2024-09-21T07:22:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:22:26.227 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:22:26.227 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:22:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40f40b6700 msgr2=0x7f40f40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40f40b6700 0x7f40f40b8c10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f410c104f10 tx=0x7f40f8005f90 comp rx=0 tx=0).stop 2024-09-21T07:22:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 msgr2=0x7f410c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 0x7f410c19c720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f410000e9f0 tx=0x7f410000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:22:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 shutdown_connections 2024-09-21T07:22:26.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f410c10b6c0 0x7f410c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40f40b6700 0x7f40f40b8c10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c10c010 0x7f410c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 --2- 172.21.15.27:0/3991132272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c105d80 0x7f410c19c720 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 >> 172.21.15.27:0/3991132272 conn(0x7f410c101810 msgr2=0x7f410c10ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 shutdown_connections 2024-09-21T07:22:26.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:26.227+0000 7f4114400640 1 -- 172.21.15.27:0/3991132272 wait complete. 2024-09-21T07:22:26.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:26 smithi027 ceph-mon[30779]: pgmap v668: 129 pgs: 129 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:22:26.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:26 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3991132272' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:22:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:26 smithi119 ceph-mon[34519]: pgmap v668: 129 pgs: 129 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:22:27.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:26 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3991132272' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:22:27.065 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:22:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:26 smithi153 ceph-mon[34747]: pgmap v668: 129 pgs: 129 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:22:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:26 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3991132272' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:22:27.452 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.069+0000 7f4094600640 1 -- 172.21.15.27:0/3348673254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10b6d0 msgr2=0x7f408c10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.069+0000 7f4094600640 1 --2- 172.21.15.27:0/3348673254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10b6d0 0x7f408c10bae0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4078009900 tx=0x7f407802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.069+0000 7f4094600640 1 -- 172.21.15.27:0/3348673254 shutdown_connections 2024-09-21T07:22:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.069+0000 7f4094600640 1 --2- 172.21.15.27:0/3348673254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c10c020 0x7f408c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.069+0000 7f4094600640 1 --2- 172.21.15.27:0/3348673254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10b6d0 0x7f408c10bae0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.069+0000 7f4094600640 1 --2- 172.21.15.27:0/3348673254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f408c105d90 0x7f408c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.070+0000 7f4094600640 1 -- 172.21.15.27:0/3348673254 >> 172.21.15.27:0/3348673254 conn(0x7f408c101860 msgr2=0x7f408c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.070+0000 7f4094600640 1 -- 172.21.15.27:0/3348673254 shutdown_connections 2024-09-21T07:22:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.070+0000 7f4094600640 1 -- 172.21.15.27:0/3348673254 wait complete. 2024-09-21T07:22:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 Processor -- start 2024-09-21T07:22:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 -- start start 2024-09-21T07:22:28.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c105d90 0x7f408c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f408c10b6d0 0x7f408c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10c020 0x7f408c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f408c115ce0 con 0x7f408c10c020 2024-09-21T07:22:28.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f408c115b60 con 0x7f408c10b6d0 2024-09-21T07:22:28.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.071+0000 7f4094600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f408c115e60 con 0x7f408c105d90 2024-09-21T07:22:28.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c105d90 0x7f408c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f408c10b6d0 0x7f408c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c105d90 0x7f408c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41106/0 (socket says 172.21.15.27:41106) 2024-09-21T07:22:28.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 -- 172.21.15.27:0/3228418900 learned_addr learned my addr 172.21.15.27:0/3228418900 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4092600640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10c020 0x7f408c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 -- 172.21.15.27:0/3228418900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f408c10b6d0 msgr2=0x7f408c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f408c10b6d0 0x7f408c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 -- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10c020 msgr2=0x7f408c1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10c020 0x7f408c1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 -- 172.21.15.27:0/3228418900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c1a2640 con 0x7f408c105d90 2024-09-21T07:22:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.072+0000 7f4091c00640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c105d90 0x7f408c19c6d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4080009d50 tx=0x7f408000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.073+0000 7f408a000640 1 -- 172.21.15.27:0/3228418900 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080003b70 con 0x7f408c105d90 2024-09-21T07:22:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.073+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c1a2930 con 0x7f408c105d90 2024-09-21T07:22:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.073+0000 7f408a000640 1 -- 172.21.15.27:0/3228418900 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4080003d10 con 0x7f408c105d90 2024-09-21T07:22:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.073+0000 7f408a000640 1 -- 172.21.15.27:0/3228418900 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800108b0 con 0x7f408c105d90 2024-09-21T07:22:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.073+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c1a9090 con 0x7f408c105d90 2024-09-21T07:22:28.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.077+0000 7f409b254740 1 Processor -- start 2024-09-21T07:22:28.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.077+0000 7f409b254740 1 -- start start 2024-09-21T07:22:28.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f409b254740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f409b254740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f409b254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f409b254740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560876757fb0 con 0x5608764111b0 2024-09-21T07:22:28.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f409b254740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5608767473e0 con 0x56087643a070 2024-09-21T07:22:28.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f409b254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56087674b1f0 con 0x560876419390 2024-09-21T07:22:28.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4092600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4091200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4092600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52692/0 (socket says 172.21.15.27:52692) 2024-09-21T07:22:28.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4092600640 1 -- 172.21.15.27:0/2975331331 learned_addr learned my addr 172.21.15.27:0/2975331331 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4091200640 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41108/0 (socket says 172.21.15.27:41108) 2024-09-21T07:22:28.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f408a000640 1 -- 172.21.15.27:0/3228418900 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4080004000 con 0x7f408c105d90 2024-09-21T07:22:28.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4092600640 1 -- 172.21.15.27:0/2975331331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 msgr2=0x5608767e1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4092600640 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4092600640 1 -- 172.21.15.27:0/2975331331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 msgr2=0x5608767e10e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:22:28.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.078+0000 7f4092600640 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767e10e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.079+0000 7f4092600640 1 -- 172.21.15.27:0/2975331331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560876711cd0 con 0x56087643a070 2024-09-21T07:22:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.079+0000 7f4092600640 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e0ba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4084009ca0 tx=0x7f4084011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07c34cd32ef8030 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.079+0000 7f407ea00640 1 -- 172.21.15.27:0/2975331331 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408401e070 con 0x56087643a070 2024-09-21T07:22:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.079+0000 7f407ea00640 1 -- 172.21.15.27:0/2975331331 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4084009e50 con 0x56087643a070 2024-09-21T07:22:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.079+0000 7f407ea00640 1 -- 172.21.15.27:0/2975331331 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408400ea10 con 0x56087643a070 2024-09-21T07:22:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f408a000640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40740b6680 0x7f40740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f409b254740 1 -- 172.21.15.27:0/2975331331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 msgr2=0x5608767e0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f409b254740 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e0ba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4084009ca0 tx=0x7f4084011810 comp rx=0 tx=0).stop 2024-09-21T07:22:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f409b254740 1 -- 172.21.15.27:0/2975331331 shutdown_connections 2024-09-21T07:22:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f409b254740 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f409b254740 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767e10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f409b254740 1 --2- 172.21.15.27:0/2975331331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e0ba0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f409b254740 1 -- 172.21.15.27:0/2975331331 >> 172.21.15.27:0/2975331331 conn(0x56087642e460 msgr2=0x5608767dea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.080+0000 7f408a000640 1 -- 172.21.15.27:0/3228418900 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f40800e8160 con 0x7f408c105d90 2024-09-21T07:22:28.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.081+0000 7f409b254740 1 -- 172.21.15.27:0/2975331331 shutdown_connections 2024-09-21T07:22:28.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.081+0000 7f4091c00640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40740b6680 0x7f40740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.081+0000 7f409b254740 1 -- 172.21.15.27:0/2975331331 wait complete. 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.081+0000 7f409b254740 1 -- start start 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.081+0000 7f4091c00640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40740b6680 0x7f40740b8b90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5608767df820 tx=0x7f4080004830 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 1 Processor -- start 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608764111b0 0x5608767ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608765a8700 con 0x56087643a070 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5608766e2580 con 0x5608764111b0 2024-09-21T07:22:28.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5608767ef380 con 0x560876419390 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f409b254740 10 client.0 _ll_register_callbacks cb 0x5608766f2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.082+0000 7f4092600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608764111b0 0x5608767ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4092600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608764111b0 0x5608767ec800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52704/0 (socket says 172.21.15.27:52704) 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4092600640 1 -- 172.21.15.27:0/3191206682 learned_addr learned my addr 172.21.15.27:0/3191206682 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4091200640 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4092600640 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4091200640 1 -- 172.21.15.27:0/3191206682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608764111b0 msgr2=0x5608767ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4091200640 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608764111b0 0x5608767ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4091200640 1 -- 172.21.15.27:0/3191206682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 msgr2=0x5608767f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4091200640 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.083+0000 7f4091200640 1 -- 172.21.15.27:0/3191206682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608764a5650 con 0x560876419390 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f4091200640 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767ecd80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f407802a910 tx=0x7f4078032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5f7620430d31d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f4073400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f4073400640 1 -- 172.21.15.27:0/3191206682 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078035040 con 0x560876419390 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f4073400640 1 -- 172.21.15.27:0/3191206682 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f4078031760 con 0x560876419390 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608766e4b90 con 0x560876419390 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f4073400640 1 -- 172.21.15.27:0/3191206682 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f407804f050 con 0x560876419390 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f409b254740 10 client.28198 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.084+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608766ff950 con 0x560876419390 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f4073400640 1 -- 172.21.15.27:0/3191206682 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5608766e4b90 con 0x560876419390 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f4073400640 1 -- 172.21.15.27:0/3191206682 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4078004750 con 0x560876419390 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f409b254740 10 client.28198 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f409b254740 10 client.28198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f409b254740 20 client.28198 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f409b254740 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763f89f0 0x5608767fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f409b254740 4 client.28198 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.085+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560876759f00 con 0x5608763f89f0 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.086+0000 7f4092600640 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763f89f0 0x5608767fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.086+0000 7f4092600640 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763f89f0 0x5608767fe730 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83e078b4fca9d623 server_cookie=333bd783ee959002 in_seq=0 out_seq=0 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.086+0000 7f4073400640 10 client.28198 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.087+0000 7f4073400640 1 -- 172.21.15.27:0/3191206682 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560876759f00 con 0x5608763f89f0 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.087+0000 7f4073400640 10 client.28198 handle_command_reply: tid=0 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.087+0000 7f409b254740 1 client.28198 shutdown 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 msgr2=0x5608767ecd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767ecd80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f407802a910 tx=0x7f4078032e90 comp rx=0 tx=0).stop 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 shutdown_connections 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763f89f0 0x5608767fe730 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767ecd80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f4073400640 0 client.28198 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 --2- 172.21.15.27:0/3191206682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608764111b0 0x5608767ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 >> 172.21.15.27:0/3191206682 conn(0x56087642e460 msgr2=0x5608767e3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 shutdown_connections 2024-09-21T07:22:28.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 1 -- 172.21.15.27:0/3191206682 wait complete. 2024-09-21T07:22:28.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.088+0000 7f409b254740 20 client.28198 trim_cache size 0 max 16384 2024-09-21T07:22:28.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.102+0000 7f409b254740 1 Processor -- start 2024-09-21T07:22:28.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.102+0000 7f409b254740 1 -- start start 2024-09-21T07:22:28.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f409b254740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f409b254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f409b254740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f409b254740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b5890 con 0x5608764111b0 2024-09-21T07:22:28.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f409b254740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b3160 con 0x56087643a070 2024-09-21T07:22:28.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f409b254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b0fd0 con 0x560876419390 2024-09-21T07:22:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767f8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37264/0 (socket says 172.21.15.27:37264) 2024-09-21T07:22:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 -- 172.21.15.27:0/3982112620 learned_addr learned my addr 172.21.15.27:0/3982112620 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4091200640 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 -- 172.21.15.27:0/3982112620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 msgr2=0x5608767e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 -- 172.21.15.27:0/3982112620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 msgr2=0x5608767e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.103+0000 7f4092600640 1 -- 172.21.15.27:0/3982112620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608767e3a00 con 0x5608764111b0 2024-09-21T07:22:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.104+0000 7f4092600640 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767f8570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f408400e520 tx=0x7f4084005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5e977844db86a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.104+0000 7f407e000640 1 -- 172.21.15.27:0/3982112620 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408401e040 con 0x5608764111b0 2024-09-21T07:22:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.104+0000 7f407e000640 1 -- 172.21.15.27:0/3982112620 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4084033040 con 0x5608764111b0 2024-09-21T07:22:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.104+0000 7f407e000640 1 -- 172.21.15.27:0/3982112620 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084003c30 con 0x5608764111b0 2024-09-21T07:22:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 -- 172.21.15.27:0/3982112620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 msgr2=0x5608767f8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767f8570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f408400e520 tx=0x7f4084005f90 comp rx=0 tx=0).stop 2024-09-21T07:22:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 -- 172.21.15.27:0/3982112620 shutdown_connections 2024-09-21T07:22:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560876419390 0x5608767e63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56087643a070 0x5608767e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 --2- 172.21.15.27:0/3982112620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608764111b0 0x5608767f8570 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 -- 172.21.15.27:0/3982112620 >> 172.21.15.27:0/3982112620 conn(0x56087642e460 msgr2=0x5608767e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 -- 172.21.15.27:0/3982112620 shutdown_connections 2024-09-21T07:22:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.105+0000 7f409b254740 1 -- 172.21.15.27:0/3982112620 wait complete. 2024-09-21T07:22:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 -- start start 2024-09-21T07:22:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 Processor -- start 2024-09-21T07:22:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608764111b0 0x5608767f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560876419390 0x5608767f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b1620 con 0x56087643a070 2024-09-21T07:22:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b2a90 con 0x560876419390 2024-09-21T07:22:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.106+0000 7f409b254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b1cf0 con 0x5608764111b0 2024-09-21T07:22:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f409b254740 10 client.0 _ll_register_callbacks cb 0x7f4078032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560876419390 0x5608767f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4092600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608764111b0 0x5608767f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560876419390 0x5608767f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52720/0 (socket says 172.21.15.27:52720) 2024-09-21T07:22:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 -- 172.21.15.27:0/4215017906 learned_addr learned my addr 172.21.15.27:0/4215017906 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4092600640 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 -- 172.21.15.27:0/4215017906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608764111b0 msgr2=0x5608767f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608764111b0 0x5608767f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 -- 172.21.15.27:0/4215017906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 msgr2=0x5608767f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.107+0000 7f4091200640 1 -- 172.21.15.27:0/4215017906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608767f1b10 con 0x560876419390 2024-09-21T07:22:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f4091200640 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560876419390 0x5608767f1090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4078000c00 tx=0x7f4078004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b32b6576ffc043 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f4073400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:22:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f4073400640 1 -- 172.21.15.27:0/4215017906 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078035070 con 0x560876419390 2024-09-21T07:22:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f4073400640 1 -- 172.21.15.27:0/4215017906 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f40780048d0 con 0x560876419390 2024-09-21T07:22:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4084033040 con 0x560876419390 2024-09-21T07:22:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f4073400640 1 -- 172.21.15.27:0/4215017906 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4078000e60 con 0x560876419390 2024-09-21T07:22:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f409b254740 10 client.28253 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.108+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560876869dc0 con 0x560876419390 2024-09-21T07:22:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f4073400640 1 -- 172.21.15.27:0/4215017906 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4084033040 con 0x560876419390 2024-09-21T07:22:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f4073400640 1 -- 172.21.15.27:0/4215017906 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f40780035f0 con 0x560876419390 2024-09-21T07:22:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f409b254740 10 client.28253 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f409b254740 10 client.28253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:22:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f409b254740 20 client.28253 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f409b254740 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763cdda0 0x56087686c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f409b254740 4 client.28253 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:22:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608767559b0 con 0x5608763cdda0 2024-09-21T07:22:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.109+0000 7f4092600640 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763cdda0 0x56087686c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.110+0000 7f4092600640 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763cdda0 0x56087686c600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16423eea555887c5 server_cookie=3950721cbba69179 in_seq=0 out_seq=0 2024-09-21T07:22:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.110+0000 7f4073400640 10 client.28253 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:22:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.110+0000 7f4073400640 1 -- 172.21.15.27:0/4215017906 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608767559b0 con 0x5608763cdda0 2024-09-21T07:22:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.110+0000 7f4073400640 10 client.28253 handle_command_reply: tid=0 2024-09-21T07:22:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.110+0000 7f409b254740 1 client.28253 shutdown 2024-09-21T07:22:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.110+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560876419390 msgr2=0x5608767f1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.110+0000 7f409b254740 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560876419390 0x5608767f1090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4078000c00 tx=0x7f4078004ef0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 shutdown_connections 2024-09-21T07:22:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5608763cdda0 0x56087686c600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087643a070 0x5608767f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560876419390 0x5608767f1090 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 --2- 172.21.15.27:0/4215017906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608764111b0 0x5608767f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f4073400640 0 client.28253 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:22:28.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 >> 172.21.15.27:0/4215017906 conn(0x56087642e460 msgr2=0x5608767eae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 shutdown_connections 2024-09-21T07:22:28.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 1 -- 172.21.15.27:0/4215017906 wait complete. 2024-09-21T07:22:28.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.111+0000 7f409b254740 20 client.28253 trim_cache size 0 max 16384 2024-09-21T07:22:28.124 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:22:28.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40740b6680 msgr2=0x7f40740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40740b6680 0x7f40740b8b90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5608767df820 tx=0x7f4080004830 comp rx=0 tx=0).stop 2024-09-21T07:22:28.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c105d90 msgr2=0x7f408c19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c105d90 0x7f408c19c6d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4080009d50 tx=0x7f408000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 shutdown_connections 2024-09-21T07:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f40740b6680 0x7f40740b8b90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c10c020 0x7f408c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f408c10b6d0 0x7f408c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 --2- 172.21.15.27:0/3228418900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f408c105d90 0x7f408c19c6d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 >> 172.21.15.27:0/3228418900 conn(0x7f408c101860 msgr2=0x7f408c10ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 shutdown_connections 2024-09-21T07:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:28.112+0000 7f4094600640 1 -- 172.21.15.27:0/3228418900 wait complete. 2024-09-21T07:22:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3491077951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d4076100 msgr2=0x7fa5d410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3491077951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d4076100 0x7fa5d410f920 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009900 tx=0x7fa5c002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3491077951 shutdown_connections 2024-09-21T07:22:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3491077951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5d410fe60 0x7fa5d41122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3491077951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d4076100 0x7fa5d410f920 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3491077951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5d40757b0 0x7fa5d4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3491077951 >> 172.21.15.27:0/3491077951 conn(0x7fa5d40ff330 msgr2=0x7fa5d4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3491077951 shutdown_connections 2024-09-21T07:22:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.511+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3491077951 wait complete. 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.512+0000 7fa5d9e00640 1 Processor -- start 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.512+0000 7fa5d9e00640 1 -- start start 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d40757b0 0x7fa5d419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5d4076100 0x7fa5d419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5d410fe60 0x7fa5d41a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d9e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4115a30 con 0x7fa5d40757b0 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d9e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41158b0 con 0x7fa5d410fe60 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d9e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4115bb0 con 0x7fa5d4076100 2024-09-21T07:22:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d40757b0 0x7fa5d419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d40757b0 0x7fa5d419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37286/0 (socket says 172.21.15.27:37286) 2024-09-21T07:22:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d8a00640 1 -- 172.21.15.27:0/3568415338 learned_addr learned my addr 172.21.15.27:0/3568415338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5d410fe60 0x7fa5d41a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.513+0000 7fa5d3e00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5d4076100 0x7fa5d419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d9400640 1 -- 172.21.15.27:0/3568415338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5d4076100 msgr2=0x7fa5d419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5d4076100 0x7fa5d419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d9400640 1 -- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d40757b0 msgr2=0x7fa5d419c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d40757b0 0x7fa5d419c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d9400640 1 -- 172.21.15.27:0/3568415338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d41a23c0 con 0x7fa5d410fe60 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d40757b0 0x7fa5d419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5d410fe60 0x7fa5d41a1cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009fd0 tx=0x7fa5cc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d1600640 1 -- 172.21.15.27:0/3568415338 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00ce20 con 0x7fa5d410fe60 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d41a26b0 con 0x7fa5d410fe60 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d1600640 1 -- 172.21.15.27:0/3568415338 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5cc005020 con 0x7fa5d410fe60 2024-09-21T07:22:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.514+0000 7fa5d1600640 1 -- 172.21.15.27:0/3568415338 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc010690 con 0x7fa5d410fe60 2024-09-21T07:22:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.515+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d41a8e40 con 0x7fa5d410fe60 2024-09-21T07:22:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.518+0000 7fa5d1600640 1 -- 172.21.15.27:0/3568415338 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa5cc0054d0 con 0x7fa5d410fe60 2024-09-21T07:22:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.518+0000 7fa5e2605740 1 Processor -- start 2024-09-21T07:22:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.519+0000 7fa5e2605740 1 -- start start 2024-09-21T07:22:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.519+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca73e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.519+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca74350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.519+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca74890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.519+0000 7fa5e2605740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5594fc9e96b0 con 0x5594fc6a62f0 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.519+0000 7fa5e2605740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5594fc9d8ae0 con 0x5594fc6ae4d0 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.519+0000 7fa5e2605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594fc9dc8f0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca73e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca74350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca73e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41176/0 (socket says 172.21.15.27:41176) 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 -- 172.21.15.27:0/707127497 learned_addr learned my addr 172.21.15.27:0/707127497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca74890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 -- 172.21.15.27:0/707127497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 msgr2=0x5594fca74890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca74890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 -- 172.21.15.27:0/707127497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 msgr2=0x5594fca74350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca74350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 -- 172.21.15.27:0/707127497 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594fc9a5780 con 0x5594fc6cf1b0 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5d3e00640 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca73e10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40777d0 tx=0x7fa5c0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4f61256bac1d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.520+0000 7fa5c5c00640 1 -- 172.21.15.27:0/707127497 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0035070 con 0x5594fc6cf1b0 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5c5c00640 1 -- 172.21.15.27:0/707127497 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c0004030 con 0x5594fc6cf1b0 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5c5c00640 1 -- 172.21.15.27:0/707127497 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0005320 con 0x5594fc6cf1b0 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5d1600640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5bc0b6680 0x7fa5bc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5d1600640 1 -- 172.21.15.27:0/3568415338 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fa5cc0e8210 con 0x7fa5d410fe60 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5bc0b6680 0x7fa5bc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5e2605740 1 -- 172.21.15.27:0/707127497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 msgr2=0x5594fca73e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5e2605740 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca73e10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40777d0 tx=0x7fa5c0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.521+0000 7fa5e2605740 1 -- 172.21.15.27:0/707127497 shutdown_connections 2024-09-21T07:22:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5e2605740 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca74890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5e2605740 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca74350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5e2605740 1 --2- 172.21.15.27:0/707127497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca73e10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5e2605740 1 -- 172.21.15.27:0/707127497 >> 172.21.15.27:0/707127497 conn(0x5594fc6c3300 msgr2=0x5594fca71d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5e2605740 1 -- 172.21.15.27:0/707127497 shutdown_connections 2024-09-21T07:22:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5e2605740 1 -- 172.21.15.27:0/707127497 wait complete. 2024-09-21T07:22:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5bc0b6680 0x7fa5bc0b8b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5594fca72a90 tx=0x7fa5c8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.522+0000 7fa5e2605740 1 -- start start 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 1 Processor -- start 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca7fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5594fc910ab0 con 0x5594fc6a62f0 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5594fc97f570 con 0x5594fc6ae4d0 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594fca82530 con 0x5594fc6cf1b0 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5e2605740 10 client.0 _ll_register_callbacks cb 0x5594fc98e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5d9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca7fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca7f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37306/0 (socket says 172.21.15.27:37306) 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5d8a00640 1 -- 172.21.15.27:0/3490042925 learned_addr learned my addr 172.21.15.27:0/3490042925 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.523+0000 7fa5d3e00640 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5d9400640 1 -- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 msgr2=0x5594fca7f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca7f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5d9400640 1 -- 172.21.15.27:0/3490042925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 msgr2=0x5594fca7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5d9400640 1 -- 172.21.15.27:0/3490042925 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594fc73a790 con 0x5594fc6cf1b0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca7fe10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0e5550 tx=0x7fa5cc0a93c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94e967ed7ba8f771 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5ba000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5ba000640 1 -- 172.21.15.27:0/3490042925 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0d1310 con 0x5594fc6cf1b0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c0004030 con 0x5594fc6cf1b0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.524+0000 7fa5ba000640 1 -- 172.21.15.27:0/3490042925 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fa5cc0aceb0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5ba000640 1 -- 172.21.15.27:0/3490042925 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5cc0b2730 con 0x5594fc6cf1b0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 10 client.28216 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594fc7b3620 con 0x5594fc6cf1b0 2024-09-21T07:22:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5ba000640 1 -- 172.21.15.27:0/3490042925 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c0004030 con 0x5594fc6cf1b0 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5ba000640 1 -- 172.21.15.27:0/3490042925 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa5cc0a95a0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 10 client.28216 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 10 client.28216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 20 client.28216 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc68dd80 0x5594fca914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 4 client.28216 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.525+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594fc9eb600 con 0x5594fc68dd80 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.526+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc68dd80 0x5594fca914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.526+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc68dd80 0x5594fca914a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987af4e2fede1f9 server_cookie=91ea73c52705659 in_seq=0 out_seq=0 2024-09-21T07:22:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.526+0000 7fa5ba000640 10 client.28216 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.527+0000 7fa5ba000640 1 -- 172.21.15.27:0/3490042925 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594fc9eb600 con 0x5594fc68dd80 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.527+0000 7fa5ba000640 10 client.28216 handle_command_reply: tid=0 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.527+0000 7fa5e2605740 1 client.28216 shutdown 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 msgr2=0x5594fca7fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca7fe10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0e5550 tx=0x7fa5cc0a93c0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 shutdown_connections 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc68dd80 0x5594fca914a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6cf1b0 0x5594fca7fe10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5ba000640 0 client.28216 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3490042925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6a62f0 0x5594fca7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 >> 172.21.15.27:0/3490042925 conn(0x5594fc6c3300 msgr2=0x5594fca74e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 shutdown_connections 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 1 -- 172.21.15.27:0/3490042925 wait complete. 2024-09-21T07:22:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.528+0000 7fa5e2605740 20 client.28216 trim_cache size 0 max 16384 2024-09-21T07:22:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.543+0000 7fa5e2605740 1 Processor -- start 2024-09-21T07:22:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.543+0000 7fa5e2605740 1 -- start start 2024-09-21T07:22:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6a62f0 0x5594fca80fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6ae4d0 0x5594fca7c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca7caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5e2605740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41b5770 con 0x5594fc6cf1b0 2024-09-21T07:22:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5e2605740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41b3040 con 0x5594fc6a62f0 2024-09-21T07:22:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5e2605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41b0eb0 con 0x5594fc6ae4d0 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6a62f0 0x5594fca80fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6a62f0 0x5594fca80fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52778/0 (socket says 172.21.15.27:52778) 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 -- 172.21.15.27:0/2808716500 learned_addr learned my addr 172.21.15.27:0/2808716500 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca7caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d3e00640 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6ae4d0 0x5594fca7c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 -- 172.21.15.27:0/2808716500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6ae4d0 msgr2=0x5594fca7c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6ae4d0 0x5594fca7c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 -- 172.21.15.27:0/2808716500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 msgr2=0x5594fca7caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca7caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.544+0000 7fa5d8a00640 1 -- 172.21.15.27:0/2808716500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594fca7aee0 con 0x5594fc6a62f0 2024-09-21T07:22:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.545+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6a62f0 0x5594fca80fc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800f730 tx=0x7fa5c801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3136ff51de8d797c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.545+0000 7fa5c5200640 1 -- 172.21.15.27:0/2808716500 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8005020 con 0x5594fc6a62f0 2024-09-21T07:22:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.545+0000 7fa5c5200640 1 -- 172.21.15.27:0/2808716500 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c80047a0 con 0x5594fc6a62f0 2024-09-21T07:22:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.545+0000 7fa5c5200640 1 -- 172.21.15.27:0/2808716500 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c802b6d0 con 0x5594fc6a62f0 2024-09-21T07:22:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 -- 172.21.15.27:0/2808716500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6a62f0 msgr2=0x5594fca80fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6a62f0 0x5594fca80fc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800f730 tx=0x7fa5c801a000 comp rx=0 tx=0).stop 2024-09-21T07:22:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 -- 172.21.15.27:0/2808716500 shutdown_connections 2024-09-21T07:22:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6ae4d0 0x5594fca7c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca7caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 --2- 172.21.15.27:0/2808716500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6a62f0 0x5594fca80fc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 -- 172.21.15.27:0/2808716500 >> 172.21.15.27:0/2808716500 conn(0x5594fc6c3300 msgr2=0x5594fca808b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 -- 172.21.15.27:0/2808716500 shutdown_connections 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 -- 172.21.15.27:0/2808716500 wait complete. 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 -- start start 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.546+0000 7fa5e2605740 1 Processor -- start 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6a62f0 0x5594fca87d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5e2605740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca85730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5e2605740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41b1500 con 0x5594fc6cf1b0 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5e2605740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41b2970 con 0x5594fc6ae4d0 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5e2605740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41b1bd0 con 0x5594fc6a62f0 2024-09-21T07:22:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5e2605740 10 client.0 _ll_register_callbacks cb 0x7fa5cc0ad3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5d9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca85730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6a62f0 0x5594fca87d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6a62f0 0x5594fca87d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41204/0 (socket says 172.21.15.27:41204) 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.547+0000 7fa5d8a00640 1 -- 172.21.15.27:0/3586705884 learned_addr learned my addr 172.21.15.27:0/3586705884 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5d9400640 1 -- 172.21.15.27:0/3586705884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6a62f0 msgr2=0x5594fca87d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6a62f0 0x5594fca87d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5d9400640 1 -- 172.21.15.27:0/3586705884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 msgr2=0x5594fca851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca851f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5d9400640 1 -- 172.21.15.27:0/3586705884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594fca85c70 con 0x5594fc6cf1b0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5d9400640 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca85730 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c760 tx=0x7fa5cc0a97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18162effd076f05c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5ba000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5ba000640 1 -- 172.21.15.27:0/3586705884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0d50d0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5ba000640 1 -- 172.21.15.27:0/3586705884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fa5cc0ad0c0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c80047a0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5ba000640 1 -- 172.21.15.27:0/3586705884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5cc0ad5a0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5e2605740 10 client.18234 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.548+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594fcafbb40 con 0x5594fc6cf1b0 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5ba000640 1 -- 172.21.15.27:0/3586705884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c80047a0 con 0x5594fc6cf1b0 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5ba000640 1 -- 172.21.15.27:0/3586705884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa5cc0ac920 con 0x5594fc6cf1b0 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5e2605740 10 client.18234 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5e2605740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5e2605740 20 client.18234 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc6631c0 0x5594fcafe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5e2605740 4 client.18234 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594fc9e70b0 con 0x5594fc6631c0 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.549+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc6631c0 0x5594fcafe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.550+0000 7fa5d8a00640 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc6631c0 0x5594fcafe350 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631cd161e57b1897 server_cookie=967bfbd4f1b73079 in_seq=0 out_seq=0 2024-09-21T07:22:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.550+0000 7fa5ba000640 10 client.18234 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.564+0000 7fa5ba000640 1 -- 172.21.15.27:0/3586705884 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594fc9e70b0 con 0x5594fc6631c0 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.564+0000 7fa5ba000640 10 client.18234 handle_command_reply: tid=0 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.564+0000 7fa5e2605740 1 client.18234 shutdown 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.564+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 msgr2=0x5594fca85730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.564+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca85730 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c760 tx=0x7fa5cc0a97b0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 shutdown_connections 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5594fc6631c0 0x5594fcafe350 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594fc6cf1b0 0x5594fca85730 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594fc6ae4d0 0x5594fca851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5ba000640 0 client.18234 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 --2- 172.21.15.27:0/3586705884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594fc6a62f0 0x5594fca87d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 >> 172.21.15.27:0/3586705884 conn(0x5594fc6c3300 msgr2=0x5594fca74e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 shutdown_connections 2024-09-21T07:22:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 1 -- 172.21.15.27:0/3586705884 wait complete. 2024-09-21T07:22:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.565+0000 7fa5e2605740 20 client.18234 trim_cache size 0 max 16384 2024-09-21T07:22:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:22:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:22:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:22:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5bc0b6680 msgr2=0x7fa5bc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5bc0b6680 0x7fa5bc0b8b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5594fca72a90 tx=0x7fa5c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5d410fe60 msgr2=0x7fa5d41a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5d410fe60 0x7fa5d41a1cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009fd0 tx=0x7fa5cc00ef10 comp rx=0 tx=0).stop 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 shutdown_connections 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5d4076100 0x7fa5d419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa5bc0b6680 0x7fa5bc0b8b90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5d410fe60 0x7fa5d41a1cc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 --2- 172.21.15.27:0/3568415338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d40757b0 0x7fa5d419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 >> 172.21.15.27:0/3568415338 conn(0x7fa5d40ff330 msgr2=0x7fa5d4109a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 shutdown_connections 2024-09-21T07:22:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:28.566+0000 7fa5d9e00640 1 -- 172.21.15.27:0/3568415338 wait complete. 2024-09-21T07:22:28.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:22:29.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:28 smithi119 ceph-mon[34519]: pgmap v669: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-21T07:22:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:28 smithi027 ceph-mon[30779]: pgmap v669: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-21T07:22:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:28 smithi153 ceph-mon[34747]: pgmap v669: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-21T07:22:29.322 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.943+0000 7f15af200640 1 -- 172.21.15.27:0/1379304525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010b500 msgr2=0x7f15b010b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.943+0000 7f15af200640 1 --2- 172.21.15.27:0/1379304525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010b500 0x7f15b010b910 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c02a410 comp rx=0 tx=0).stop 2024-09-21T07:22:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.944+0000 7f15af200640 1 -- 172.21.15.27:0/1379304525 shutdown_connections 2024-09-21T07:22:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.944+0000 7f15af200640 1 --2- 172.21.15.27:0/1379304525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b010be50 0x7f15b0112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.944+0000 7f15af200640 1 --2- 172.21.15.27:0/1379304525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010b500 0x7f15b010b910 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.944+0000 7f15af200640 1 --2- 172.21.15.27:0/1379304525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15b0105bc0 0x7f15b0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.944+0000 7f15af200640 1 -- 172.21.15.27:0/1379304525 >> 172.21.15.27:0/1379304525 conn(0x7f15b0101630 msgr2=0x7f15b0103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:29.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.944+0000 7f15af200640 1 -- 172.21.15.27:0/1379304525 shutdown_connections 2024-09-21T07:22:29.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.944+0000 7f15af200640 1 -- 172.21.15.27:0/1379304525 wait complete. 2024-09-21T07:22:29.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 Processor -- start 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 -- start start 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0105bc0 0x7f15b019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15b010b500 0x7f15b019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010be50 0x7f15b01a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0115b10 con 0x7f15b010be50 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0115990 con 0x7f15b0105bc0 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.945+0000 7f15af200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0115c90 con 0x7f15b010b500 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0105bc0 0x7f15b019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0105bc0 0x7f15b019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52814/0 (socket says 172.21.15.27:52814) 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 -- 172.21.15.27:0/3663679519 learned_addr learned my addr 172.21.15.27:0/3663679519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ad400640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15b010b500 0x7f15b019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ae800640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010be50 0x7f15b01a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 -- 172.21.15.27:0/3663679519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15b010b500 msgr2=0x7f15b019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15b010b500 0x7f15b019cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 -- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010be50 msgr2=0x7f15b01a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010be50 0x7f15b01a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 -- 172.21.15.27:0/3663679519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b01a24f0 con 0x7f15b0105bc0 2024-09-21T07:22:29.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.946+0000 7f15ade00640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0105bc0 0x7f15b019c580 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f15a000e9f0 tx=0x7f15a000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:29.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.947+0000 7f15a6a00640 1 -- 172.21.15.27:0/3663679519 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a000cd80 con 0x7f15b0105bc0 2024-09-21T07:22:29.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.947+0000 7f15a6a00640 1 -- 172.21.15.27:0/3663679519 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a00054d0 con 0x7f15b0105bc0 2024-09-21T07:22:29.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.947+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b01a27e0 con 0x7f15b0105bc0 2024-09-21T07:22:29.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.947+0000 7f15a6a00640 1 -- 172.21.15.27:0/3663679519 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0010690 con 0x7f15b0105bc0 2024-09-21T07:22:29.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.947+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b01a8f40 con 0x7f15b0105bc0 2024-09-21T07:22:29.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.950+0000 7f15bbc01740 1 Processor -- start 2024-09-21T07:22:29.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.950+0000 7f15bbc01740 1 -- start start 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.950+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56748d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.950+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56749270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.950+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e567497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15bbc01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562e566bffb0 con 0x562e563791b0 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15bbc01740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562e566af3e0 con 0x562e563a2070 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15bbc01740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e566b31f0 con 0x562e56381390 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15a6a00640 1 -- 172.21.15.27:0/3663679519 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f15a0004e90 con 0x7f15b0105bc0 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56748d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56748d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52818/0 (socket says 172.21.15.27:52818) 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 -- 172.21.15.27:0/2746779463 learned_addr learned my addr 172.21.15.27:0/2746779463 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:29.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ad400640 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e567497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ade00640 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56749270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 -- 172.21.15.27:0/2746779463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 msgr2=0x562e567497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e567497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 -- 172.21.15.27:0/2746779463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 msgr2=0x562e56749270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56749270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.951+0000 7f15ae800640 1 -- 172.21.15.27:0/2746779463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e56679cd0 con 0x562e563a2070 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.952+0000 7f15ae800640 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56748d30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009f50 tx=0x7f15a80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e64671a89ff0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.952+0000 7f159b400640 1 -- 172.21.15.27:0/2746779463 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8004d20 con 0x562e563a2070 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.952+0000 7f159b400640 1 -- 172.21.15.27:0/2746779463 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a8004050 con 0x562e563a2070 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.952+0000 7f159b400640 1 -- 172.21.15.27:0/2746779463 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8005220 con 0x562e563a2070 2024-09-21T07:22:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15a6a00640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15940b6700 0x7f15940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15a6a00640 1 -- 172.21.15.27:0/3663679519 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f15a00e7ea0 con 0x7f15b0105bc0 2024-09-21T07:22:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 -- 172.21.15.27:0/2746779463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 msgr2=0x562e56748d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56748d30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009f50 tx=0x7f15a80118d0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15ade00640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15940b6700 0x7f15940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 -- 172.21.15.27:0/2746779463 shutdown_connections 2024-09-21T07:22:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e567497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56749270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2746779463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56748d30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 -- 172.21.15.27:0/2746779463 >> 172.21.15.27:0/2746779463 conn(0x562e56396460 msgr2=0x562e56746bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 -- 172.21.15.27:0/2746779463 shutdown_connections 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.953+0000 7f15bbc01740 1 -- 172.21.15.27:0/2746779463 wait complete. 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.954+0000 7f15ade00640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15940b6700 0x7f15940b8c10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x562e567479b0 tx=0x7f15a0004a90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.954+0000 7f15bbc01740 1 -- start start 2024-09-21T07:22:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.954+0000 7f15bbc01740 1 Processor -- start 2024-09-21T07:22:29.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56754970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e56754eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56760830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15bbc01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562e56510700 con 0x562e563791b0 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15bbc01740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562e5664a580 con 0x562e563a2070 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15bbc01740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e56757510 con 0x562e56381390 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15bbc01740 10 client.0 _ll_register_callbacks cb 0x562e5665a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15ae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56754970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15ad400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e56754eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15ae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56754970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37356/0 (socket says 172.21.15.27:37356) 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15ae800640 1 -- 172.21.15.27:0/2349288647 learned_addr learned my addr 172.21.15.27:0/2349288647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.955+0000 7f15ae800640 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56760830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f15ad400640 1 -- 172.21.15.27:0/2349288647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 msgr2=0x562e56760830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f15ad400640 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56760830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f15ad400640 1 -- 172.21.15.27:0/2349288647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 msgr2=0x562e56754970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f15ad400640 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56754970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f15ad400640 1 -- 172.21.15.27:0/2349288647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5640d650 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f15ad400640 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e56754eb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f159c02a930 tx=0x7f159c032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96f8fa310c3afb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f158fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f158fe00640 1 -- 172.21.15.27:0/2349288647 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c035040 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e5664cb90 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.956+0000 7f158fe00640 1 -- 172.21.15.27:0/2349288647 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f159c031760 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f158fe00640 1 -- 172.21.15.27:0/2349288647 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f159c04f050 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 10 client.28234 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e56667950 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f158fe00640 1 -- 172.21.15.27:0/2349288647 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e5664cb90 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f158fe00640 1 -- 172.21.15.27:0/2349288647 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f159c004750 con 0x562e56381390 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 10 client.28234 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 10 client.28234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:22:29.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 20 client.28234 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:29.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e563609f0 0x562e567667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 4 client.28234 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:29.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.957+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e566c1f00 con 0x562e563609f0 2024-09-21T07:22:29.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.958+0000 7f15ae800640 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e563609f0 0x562e567667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.958+0000 7f15ae800640 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e563609f0 0x562e567667d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dbadcda57095f76 server_cookie=e59063783ea6e64c in_seq=0 out_seq=0 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.958+0000 7f158fe00640 10 client.28234 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.959+0000 7f158fe00640 1 -- 172.21.15.27:0/2349288647 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e566c1f00 con 0x562e563609f0 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.959+0000 7f158fe00640 10 client.28234 handle_command_reply: tid=0 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.959+0000 7f15bbc01740 1 client.28234 shutdown 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.959+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 msgr2=0x562e56754eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.959+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e56754eb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f159c02a930 tx=0x7f159c032e90 comp rx=0 tx=0).stop 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.959+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 shutdown_connections 2024-09-21T07:22:29.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.959+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e563609f0 0x562e567667d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e56381390 0x562e56754eb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563a2070 0x562e56760830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f15bbc01740 1 --2- 172.21.15.27:0/2349288647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e56754970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f158fe00640 0 client.28234 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 >> 172.21.15.27:0/2349288647 conn(0x562e56396460 msgr2=0x562e5674bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 shutdown_connections 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f15bbc01740 1 -- 172.21.15.27:0/2349288647 wait complete. 2024-09-21T07:22:29.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.960+0000 7f15bbc01740 20 client.28234 trim_cache size 0 max 16384 2024-09-21T07:22:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 Processor -- start 2024-09-21T07:22:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 -- start start 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e5674ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e56381390 0x562e56746c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56751960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01b5810 con 0x562e563791b0 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01b30e0 con 0x562e56381390 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.974+0000 7f15bbc01740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01b0f50 con 0x562e563a2070 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e5674ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e5674ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37358/0 (socket says 172.21.15.27:37358) 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 -- 172.21.15.27:0/1051226448 learned_addr learned my addr 172.21.15.27:0/1051226448 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ad400640 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e56381390 0x562e56746c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56751960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 -- 172.21.15.27:0/1051226448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 msgr2=0x562e56751960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56751960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 -- 172.21.15.27:0/1051226448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e56381390 msgr2=0x562e56746c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e56381390 0x562e56746c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 -- 172.21.15.27:0/1051226448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e56740dc0 con 0x562e563791b0 2024-09-21T07:22:29.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.975+0000 7f15ae800640 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e5674ff20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f15a8000f40 tx=0x7f15a80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b457f485238f006 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:29.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.976+0000 7f159aa00640 1 -- 172.21.15.27:0/1051226448 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8004c80 con 0x562e563791b0 2024-09-21T07:22:29.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.976+0000 7f159aa00640 1 -- 172.21.15.27:0/1051226448 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a8033070 con 0x562e563791b0 2024-09-21T07:22:29.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.976+0000 7f159aa00640 1 -- 172.21.15.27:0/1051226448 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a80053b0 con 0x562e563791b0 2024-09-21T07:22:29.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 -- 172.21.15.27:0/1051226448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 msgr2=0x562e5674ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e5674ff20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f15a8000f40 tx=0x7f15a80096c0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 -- 172.21.15.27:0/1051226448 shutdown_connections 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56751960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e56381390 0x562e56746c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 --2- 172.21.15.27:0/1051226448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e563791b0 0x562e5674ff20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 -- 172.21.15.27:0/1051226448 >> 172.21.15.27:0/1051226448 conn(0x562e56396460 msgr2=0x562e567464f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 -- 172.21.15.27:0/1051226448 shutdown_connections 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 -- 172.21.15.27:0/1051226448 wait complete. 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.977+0000 7f15bbc01740 1 -- start start 2024-09-21T07:22:29.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 1 Processor -- start 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563791b0 0x562e5675c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e56381390 0x562e5675cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56757750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01b15a0 con 0x562e56381390 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01b2a10 con 0x562e563791b0 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01b1c70 con 0x562e563a2070 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15bbc01740 10 client.0 _ll_register_callbacks cb 0x7f159c032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15ad400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e56381390 0x562e5675cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15ad400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e56381390 0x562e5675cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37374/0 (socket says 172.21.15.27:37374) 2024-09-21T07:22:29.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.978+0000 7f15ad400640 1 -- 172.21.15.27:0/3022257705 learned_addr learned my addr 172.21.15.27:0/3022257705 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ae800640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563791b0 0x562e5675c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ae800640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56757750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ad400640 1 -- 172.21.15.27:0/3022257705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 msgr2=0x562e56757750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ad400640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56757750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ad400640 1 -- 172.21.15.27:0/3022257705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563791b0 msgr2=0x562e5675c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ad400640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563791b0 0x562e5675c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ad400640 1 -- 172.21.15.27:0/3022257705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e56757c90 con 0x562e56381390 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ae800640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56757750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f15ad400640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e56381390 0x562e5675cce0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f159c004cc0 tx=0x7f159c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a167d1aac8567a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f158fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.979+0000 7f158fe00640 1 -- 172.21.15.27:0/3022257705 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c058060 con 0x562e56381390 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.980+0000 7f158fe00640 1 -- 172.21.15.27:0/3022257705 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f159c004f50 con 0x562e56381390 2024-09-21T07:22:29.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.980+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a8033070 con 0x562e56381390 2024-09-21T07:22:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.980+0000 7f158fe00640 1 -- 172.21.15.27:0/3022257705 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f159c05f050 con 0x562e56381390 2024-09-21T07:22:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.980+0000 7f15bbc01740 10 client.18252 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.980+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e567d1830 con 0x562e56381390 2024-09-21T07:22:29.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.980+0000 7f158fe00640 1 -- 172.21.15.27:0/3022257705 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a8033070 con 0x562e56381390 2024-09-21T07:22:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.980+0000 7f158fe00640 1 -- 172.21.15.27:0/3022257705 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f159c0319f0 con 0x562e56381390 2024-09-21T07:22:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15bbc01740 10 client.18252 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:29.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15bbc01740 10 client.18252 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15bbc01740 20 client.18252 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15bbc01740 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e56335da0 0x562e567d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15bbc01740 4 client.18252 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e566bd9b0 con 0x562e56335da0 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15ae800640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e56335da0 0x562e567d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f158fe00640 10 client.18252 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.981+0000 7f15ae800640 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e56335da0 0x562e567d3fc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28d9fc00daf7c6d8 server_cookie=a5c49b750664860c in_seq=0 out_seq=0 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f158fe00640 1 -- 172.21.15.27:0/3022257705 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e566bd9b0 con 0x562e56335da0 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f158fe00640 10 client.18252 handle_command_reply: tid=0 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 client.18252 shutdown 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e56381390 msgr2=0x562e5675cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e56381390 0x562e5675cce0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f159c004cc0 tx=0x7f159c0048f0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 shutdown_connections 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562e56335da0 0x562e567d3fc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e563a2070 0x562e56757750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e56381390 0x562e5675cce0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 --2- 172.21.15.27:0/3022257705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e563791b0 0x562e5675c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 >> 172.21.15.27:0/3022257705 conn(0x562e56396460 msgr2=0x562e56749d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f158fe00640 0 client.18252 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 shutdown_connections 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.982+0000 7f15bbc01740 1 -- 172.21.15.27:0/3022257705 wait complete. 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15bbc01740 20 client.18252 trim_cache size 0 max 16384 2024-09-21T07:22:29.987 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:22:29.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15940b6700 msgr2=0x7f15940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15940b6700 0x7f15940b8c10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x562e567479b0 tx=0x7f15a0004a90 comp rx=0 tx=0).stop 2024-09-21T07:22:29.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0105bc0 msgr2=0x7f15b019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:29.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0105bc0 0x7f15b019c580 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f15a000e9f0 tx=0x7f15a000edf0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 shutdown_connections 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15b010b500 0x7f15b019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f15940b6700 0x7f15940b8c10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b010be50 0x7f15b01a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 --2- 172.21.15.27:0/3663679519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0105bc0 0x7f15b019c580 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.983+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 >> 172.21.15.27:0/3663679519 conn(0x7f15b0101630 msgr2=0x7f15b0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.984+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 shutdown_connections 2024-09-21T07:22:29.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:29.984+0000 7f15af200640 1 -- 172.21.15.27:0/3663679519 wait complete. 2024-09-21T07:22:30.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:22:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:30 smithi119 ceph-mon[34519]: pgmap v670: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-09-21T07:22:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:30 smithi027 ceph-mon[30779]: pgmap v670: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-09-21T07:22:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:30 smithi153 ceph-mon[34747]: pgmap v670: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-09-21T07:22:31.324 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:31.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 -- 172.21.15.27:0/2218935191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 msgr2=0x7f89c01061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 --2- 172.21.15.27:0/2218935191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c01061c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009900 tx=0x7f89b002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 -- 172.21.15.27:0/2218935191 shutdown_connections 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 --2- 172.21.15.27:0/2218935191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c010b6f0 0x7f89c010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 --2- 172.21.15.27:0/2218935191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c010c040 0x7f89c0112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 --2- 172.21.15.27:0/2218935191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c01061c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 -- 172.21.15.27:0/2218935191 >> 172.21.15.27:0/2218935191 conn(0x7f89c01017e0 msgr2=0x7f89c0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 -- 172.21.15.27:0/2218935191 shutdown_connections 2024-09-21T07:22:31.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.955+0000 7f89c6200640 1 -- 172.21.15.27:0/2218935191 wait complete. 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.956+0000 7f89c6200640 1 Processor -- start 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89c6200640 1 -- start start 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89c6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89c6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c010b6f0 0x7f89c019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89c6200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c010c040 0x7f89c01a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89c6200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0115d00 con 0x7f89c0105db0 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89c6200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0115b80 con 0x7f89c010b6f0 2024-09-21T07:22:31.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89c6200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0115e80 con 0x7f89c010c040 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89bf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89bf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37402/0 (socket says 172.21.15.27:37402) 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.957+0000 7f89bf400640 1 -- 172.21.15.27:0/793279114 learned_addr learned my addr 172.21.15.27:0/793279114 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89b6a00640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c010b6f0 0x7f89c019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89bfe00640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c010c040 0x7f89c01a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89bf400640 1 -- 172.21.15.27:0/793279114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c010c040 msgr2=0x7f89c01a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89bf400640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c010c040 0x7f89c01a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89bf400640 1 -- 172.21.15.27:0/793279114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c010b6f0 msgr2=0x7f89c019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89bf400640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c010b6f0 0x7f89c019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89bf400640 1 -- 172.21.15.27:0/793279114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c01a26a0 con 0x7f89c0105db0 2024-09-21T07:22:31.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.958+0000 7f89bf400640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c019c730 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f89b0005940 tx=0x7f89b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:31.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.959+0000 7f89bcc00640 1 -- 172.21.15.27:0/793279114 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b003f030 con 0x7f89c0105db0 2024-09-21T07:22:31.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.959+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c01a2930 con 0x7f89c0105db0 2024-09-21T07:22:31.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.959+0000 7f89bcc00640 1 -- 172.21.15.27:0/793279114 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b0003aa0 con 0x7f89c0105db0 2024-09-21T07:22:31.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.959+0000 7f89bcc00640 1 -- 172.21.15.27:0/793279114 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0035070 con 0x7f89c0105db0 2024-09-21T07:22:31.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.959+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c01a90f0 con 0x7f89c0105db0 2024-09-21T07:22:31.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.962+0000 7f89cceb5740 1 Processor -- start 2024-09-21T07:22:31.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89cceb5740 1 -- start start 2024-09-21T07:22:31.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x561545151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x5615451520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545152620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89cceb5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5615450c8fb0 con 0x561544dab070 2024-09-21T07:22:31.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89cceb5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5615450b83e0 con 0x561544d8a390 2024-09-21T07:22:31.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89cceb5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615450bc1f0 con 0x561544d821b0 2024-09-21T07:22:31.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89bcc00640 1 -- 172.21.15.27:0/793279114 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f89b0042030 con 0x7f89c0105db0 2024-09-21T07:22:31.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x561545151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x561545151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37414/0 (socket says 172.21.15.27:37414) 2024-09-21T07:22:31.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89bfe00640 1 -- 172.21.15.27:0/1614921716 learned_addr learned my addr 172.21.15.27:0/1614921716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:31.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.963+0000 7f89b6a00640 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89bf400640 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x5615451520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89bfe00640 1 -- 172.21.15.27:0/1614921716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 msgr2=0x5615451520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89bfe00640 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x5615451520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89bfe00640 1 -- 172.21.15.27:0/1614921716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 msgr2=0x561545152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89bfe00640 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89bfe00640 1 -- 172.21.15.27:0/1614921716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561545082cd0 con 0x561544dab070 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89b6a00640 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.964+0000 7f89bfe00640 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x561545151ba0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f89b80039f0 tx=0x7f89b8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c116cb2c7c19f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:31.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.965+0000 7f89afe00640 1 -- 172.21.15.27:0/1614921716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b801a470 con 0x561544dab070 2024-09-21T07:22:31.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.965+0000 7f89afe00640 1 -- 172.21.15.27:0/1614921716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b8015070 con 0x561544dab070 2024-09-21T07:22:31.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.965+0000 7f89afe00640 1 -- 172.21.15.27:0/1614921716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8010780 con 0x561544dab070 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89bcc00640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89a40b6680 0x7f89a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 -- 172.21.15.27:0/1614921716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 msgr2=0x561545151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x561545151ba0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f89b80039f0 tx=0x7f89b8003eb0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89bcc00640 1 -- 172.21.15.27:0/793279114 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f89b0107f30 con 0x7f89c0105db0 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89bf400640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89a40b6680 0x7f89a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 -- 172.21.15.27:0/1614921716 shutdown_connections 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x5615451520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 --2- 172.21.15.27:0/1614921716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x561545151ba0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 -- 172.21.15.27:0/1614921716 >> 172.21.15.27:0/1614921716 conn(0x561544d9f460 msgr2=0x56154514fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 -- 172.21.15.27:0/1614921716 shutdown_connections 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.966+0000 7f89cceb5740 1 -- 172.21.15.27:0/1614921716 wait complete. 2024-09-21T07:22:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.967+0000 7f89bf400640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89a40b6680 0x7f89a40b8b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f89b0105380 tx=0x7f89b002a3f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:31.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.967+0000 7f89cceb5740 1 -- start start 2024-09-21T07:22:31.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.967+0000 7f89cceb5740 1 Processor -- start 2024-09-21T07:22:31.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x56154515d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x561545169670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89cceb5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561544f19700 con 0x561544d821b0 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89cceb5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561545053580 con 0x561544d8a390 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89cceb5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561545160380 con 0x561544dab070 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89cceb5740 10 client.0 _ll_register_callbacks cb 0x561545063470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x56154515d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x56154515d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37428/0 (socket says 172.21.15.27:37428) 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89bfe00640 1 -- 172.21.15.27:0/534076373 learned_addr learned my addr 172.21.15.27:0/534076373 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89b6a00640 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.968+0000 7f89bfe00640 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x561545169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89bfe00640 1 -- 172.21.15.27:0/534076373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 msgr2=0x561545169670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89bfe00640 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x561545169670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89bfe00640 1 -- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 msgr2=0x56154515dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89bfe00640 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89bfe00640 1 -- 172.21.15.27:0/534076373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561544e16650 con 0x561544d821b0 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89bfe00640 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x56154515d7d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f89b8011e90 tx=0x7f89b8010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c81afd91aae124 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89acc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89acc00640 1 -- 172.21.15.27:0/534076373 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8004e50 con 0x561544d821b0 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89acc00640 1 -- 172.21.15.27:0/534076373 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f89b8011670 con 0x561544d821b0 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.969+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561545055b90 con 0x561544d821b0 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89acc00640 1 -- 172.21.15.27:0/534076373 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89b8030050 con 0x561544d821b0 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89cceb5740 10 client.18264 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561545070950 con 0x561544d821b0 2024-09-21T07:22:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89acc00640 1 -- 172.21.15.27:0/534076373 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561545055b90 con 0x561544d821b0 2024-09-21T07:22:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89acc00640 1 -- 172.21.15.27:0/534076373 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f89b8010520 con 0x561544d821b0 2024-09-21T07:22:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89cceb5740 10 client.18264 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89cceb5740 10 client.18264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.970+0000 7f89cceb5740 20 client.18264 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.971+0000 7f89cceb5740 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d699f0 0x56154516f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.971+0000 7f89cceb5740 4 client.18264 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.971+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615450caf00 con 0x561544d699f0 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.971+0000 7f89b6a00640 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d699f0 0x56154516f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.971+0000 7f89b6a00640 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d699f0 0x56154516f5a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb306b2d42dac563 server_cookie=85c65ddb2d608711 in_seq=0 out_seq=0 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.971+0000 7f89acc00640 10 client.18264 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.972+0000 7f89acc00640 1 -- 172.21.15.27:0/534076373 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615450caf00 con 0x561544d699f0 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.972+0000 7f89acc00640 10 client.18264 handle_command_reply: tid=0 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.972+0000 7f89cceb5740 1 client.18264 shutdown 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 msgr2=0x56154515d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x56154515d7d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f89b8011e90 tx=0x7f89b8010440 comp rx=0 tx=0).stop 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 shutdown_connections 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d699f0 0x56154516f5a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x561545169670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89acc00640 0 client.18264 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 --2- 172.21.15.27:0/534076373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x56154515d7d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 >> 172.21.15.27:0/534076373 conn(0x561544d9f460 msgr2=0x561545154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:31.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 shutdown_connections 2024-09-21T07:22:31.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 1 -- 172.21.15.27:0/534076373 wait complete. 2024-09-21T07:22:31.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.973+0000 7f89cceb5740 20 client.18264 trim_cache size 0 max 16384 2024-09-21T07:22:31.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.987+0000 7f89cceb5740 1 Processor -- start 2024-09-21T07:22:31.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.987+0000 7f89cceb5740 1 -- start start 2024-09-21T07:22:31.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x5615451695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x56154515c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:31.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89cceb5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01b59f0 con 0x561544d821b0 2024-09-21T07:22:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89cceb5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01b32c0 con 0x561544d8a390 2024-09-21T07:22:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89cceb5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01b1130 con 0x561544dab070 2024-09-21T07:22:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x5615451695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x5615451695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37430/0 (socket says 172.21.15.27:37430) 2024-09-21T07:22:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 -- 172.21.15.27:0/2976861380 learned_addr learned my addr 172.21.15.27:0/2976861380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89b6a00640 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x56154515c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 -- 172.21.15.27:0/2976861380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 msgr2=0x56154515c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x56154515c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 -- 172.21.15.27:0/2976861380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 msgr2=0x56154515be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.988+0000 7f89bfe00640 1 -- 172.21.15.27:0/2976861380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154515e890 con 0x561544d821b0 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.989+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x5615451695e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f89b8003ef0 tx=0x7f89b8005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93a1e578858c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.989+0000 7f89af400640 1 -- 172.21.15.27:0/2976861380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b803b020 con 0x561544d821b0 2024-09-21T07:22:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.989+0000 7f89af400640 1 -- 172.21.15.27:0/2976861380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b802d030 con 0x561544d821b0 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.989+0000 7f89af400640 1 -- 172.21.15.27:0/2976861380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8010710 con 0x561544d821b0 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 -- 172.21.15.27:0/2976861380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 msgr2=0x5615451695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x5615451695e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f89b8003ef0 tx=0x7f89b8005c60 comp rx=0 tx=0).stop 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 -- 172.21.15.27:0/2976861380 shutdown_connections 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544dab070 0x56154515c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x56154515be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2976861380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544d821b0 0x5615451695e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 -- 172.21.15.27:0/2976861380 >> 172.21.15.27:0/2976861380 conn(0x561544d9f460 msgr2=0x561545158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 -- 172.21.15.27:0/2976861380 shutdown_connections 2024-09-21T07:22:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.990+0000 7f89cceb5740 1 -- 172.21.15.27:0/2976861380 wait complete. 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 -- start start 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 Processor -- start 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x561545172dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545170290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x5615451707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01b1780 con 0x561544dab070 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01b2bf0 con 0x561544d8a390 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01b1e50 con 0x561544d821b0 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.991+0000 7f89cceb5740 10 client.0 _ll_register_callbacks cb 0x7f89b8004740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545170290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545170290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52880/0 (socket says 172.21.15.27:52880) 2024-09-21T07:22:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 -- 172.21.15.27:0/2974209609 learned_addr learned my addr 172.21.15.27:0/2974209609 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x561545172dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x5615451707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 -- 172.21.15.27:0/2974209609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 msgr2=0x561545172dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x561545172dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 -- 172.21.15.27:0/2974209609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 msgr2=0x5615451707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x5615451707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89b6a00640 1 -- 172.21.15.27:0/2974209609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561545170f60 con 0x561544d8a390 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.992+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x5615451707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89b6a00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545170290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009c90 tx=0x7f89a8010b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6eafb839ea96ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89acc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:22:32.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89acc00640 1 -- 172.21.15.27:0/2974209609 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8005220 con 0x561544d8a390 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b802d030 con 0x561544d8a390 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89acc00640 1 -- 172.21.15.27:0/2974209609 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f89a80053c0 con 0x561544d8a390 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89acc00640 1 -- 172.21.15.27:0/2974209609 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89a80055c0 con 0x561544d8a390 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89cceb5740 10 client.28307 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.993+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615451d9d00 con 0x561544d8a390 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89acc00640 1 -- 172.21.15.27:0/2974209609 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b802d030 con 0x561544d8a390 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89acc00640 1 -- 172.21.15.27:0/2974209609 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f89a8024400 con 0x561544d8a390 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89cceb5740 10 client.28307 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89cceb5740 10 client.28307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89cceb5740 20 client.28307 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d3eda0 0x5615451dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:32.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89cceb5740 4 client.28307 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615450c69b0 con 0x561544d3eda0 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.994+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d3eda0 0x5615451dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89bfe00640 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d3eda0 0x5615451dc4c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f896f8c21cf2ff34 server_cookie=5b594c3e99d5ab83 in_seq=0 out_seq=0 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89acc00640 10 client.28307 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89acc00640 1 -- 172.21.15.27:0/2974209609 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615450c69b0 con 0x561544d3eda0 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89acc00640 10 client.28307 handle_command_reply: tid=0 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 client.28307 shutdown 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 msgr2=0x561545170290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545170290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009c90 tx=0x7f89a8010b30 comp rx=0 tx=0).stop 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 shutdown_connections 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561544d3eda0 0x5615451dc4c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561544dab070 0x5615451707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561544d8a390 0x561545170290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89acc00640 0 client.28307 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 --2- 172.21.15.27:0/2974209609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561544d821b0 0x561545172dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 >> 172.21.15.27:0/2974209609 conn(0x561544d9f460 msgr2=0x56154514f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.995+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 shutdown_connections 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89cceb5740 1 -- 172.21.15.27:0/2974209609 wait complete. 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89cceb5740 20 client.28307 trim_cache size 0 max 16384 2024-09-21T07:22:32.005 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:22:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89a40b6680 msgr2=0x7f89a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89a40b6680 0x7f89a40b8b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f89b0105380 tx=0x7f89b002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 msgr2=0x7f89c019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c019c730 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f89b0005940 tx=0x7f89b0004270 comp rx=0 tx=0).stop 2024-09-21T07:22:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 shutdown_connections 2024-09-21T07:22:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c010c040 0x7f89c01a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f89a40b6680 0x7f89a40b8b90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c010b6f0 0x7f89c019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 --2- 172.21.15.27:0/793279114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c0105db0 0x7f89c019c730 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.996+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 >> 172.21.15.27:0/793279114 conn(0x7f89c01017e0 msgr2=0x7f89c010add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.997+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 shutdown_connections 2024-09-21T07:22:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:31.997+0000 7f89c6200640 1 -- 172.21.15.27:0/793279114 wait complete. 2024-09-21T07:22:33.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:32 smithi119 ceph-mon[34519]: pgmap v671: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-21T07:22:33.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:32 smithi027 ceph-mon[30779]: pgmap v671: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-21T07:22:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:32 smithi153 ceph-mon[34747]: pgmap v671: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-21T07:22:33.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:22:33.815 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:22:34.211 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.803+0000 7f7985200640 1 -- 172.21.15.27:0/726250507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980077940 msgr2=0x7f7980077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.803+0000 7f7985200640 1 --2- 172.21.15.27:0/726250507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980077940 0x7f7980077d50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7970009900 tx=0x7f797002a410 comp rx=0 tx=0).stop 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.803+0000 7f7985200640 1 -- 172.21.15.27:0/726250507 shutdown_connections 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.803+0000 7f7985200640 1 --2- 172.21.15.27:0/726250507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980114740 0x7f7980116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.803+0000 7f7985200640 1 --2- 172.21.15.27:0/726250507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7980078290 0x7f7980114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.803+0000 7f7985200640 1 --2- 172.21.15.27:0/726250507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980077940 0x7f7980077d50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.803+0000 7f7985200640 1 -- 172.21.15.27:0/726250507 >> 172.21.15.27:0/726250507 conn(0x7f7980101810 msgr2=0x7f7980103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.804+0000 7f7985200640 1 -- 172.21.15.27:0/726250507 shutdown_connections 2024-09-21T07:22:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.804+0000 7f7985200640 1 -- 172.21.15.27:0/726250507 wait complete. 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 Processor -- start 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 -- start start 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980077940 0x7f79801a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980078290 0x7f79801a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7980114740 0x7f79801a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f798011a120 con 0x7f7980078290 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7980119fa0 con 0x7f7980114740 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.805+0000 7f7985200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f798011a2a0 con 0x7f7980077940 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980077940 0x7f79801a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980077940 0x7f79801a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41326/0 (socket says 172.21.15.27:41326) 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 -- 172.21.15.27:0/3249512235 learned_addr learned my addr 172.21.15.27:0/3249512235 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f7975400640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980078290 0x7f79801a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797e800640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7980114740 0x7f79801a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 -- 172.21.15.27:0/3249512235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7980114740 msgr2=0x7f79801a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7980114740 0x7f79801a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 -- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980078290 msgr2=0x7f79801a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980078290 0x7f79801a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f797de00640 1 -- 172.21.15.27:0/3249512235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79801a6a40 con 0x7f7980077940 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.806+0000 7f7975400640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980078290 0x7f79801a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.807+0000 7f797de00640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980077940 0x7f79801a0ad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7970005ab0 tx=0x7f7970003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.807+0000 7f7977400640 1 -- 172.21.15.27:0/3249512235 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797003f070 con 0x7f7980077940 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.807+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79801a6cd0 con 0x7f7980077940 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.807+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801ad490 con 0x7f7980077940 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.807+0000 7f7977400640 1 -- 172.21.15.27:0/3249512235 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7970031b60 con 0x7f7980077940 2024-09-21T07:22:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.807+0000 7f7977400640 1 -- 172.21.15.27:0/3249512235 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970049060 con 0x7f7980077940 2024-09-21T07:22:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.810+0000 7f798bb22740 1 Processor -- start 2024-09-21T07:22:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.810+0000 7f798bb22740 1 -- start start 2024-09-21T07:22:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f798bb22740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b400d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f798bb22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b401250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f7977400640 1 -- 172.21.15.27:0/3249512235 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7970042070 con 0x7f7980077940 2024-09-21T07:22:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f798bb22740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b038670 0x560e2b401790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f798bb22740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560e2b376e90 con 0x560e2b038670 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f798bb22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560e2b3662c0 con 0x560e2b030490 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f798bb22740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e2b36a0d0 con 0x560e2b059350 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f7975400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b038670 0x560e2b401790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f797e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b400d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f797e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b400d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41342/0 (socket says 172.21.15.27:41342) 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f797e800640 1 -- 172.21.15.27:0/2148335148 learned_addr learned my addr 172.21.15.27:0/2148335148 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f7975400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b038670 0x560e2b401790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37468/0 (socket says 172.21.15.27:37468) 2024-09-21T07:22:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.811+0000 7f797de00640 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b401250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f797e800640 1 -- 172.21.15.27:0/2148335148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 msgr2=0x560e2b401250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f797e800640 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b401250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f797e800640 1 -- 172.21.15.27:0/2148335148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b038670 msgr2=0x560e2b401790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f797e800640 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b038670 0x560e2b401790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f797e800640 1 -- 172.21.15.27:0/2148335148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2b0c4930 con 0x560e2b059350 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f797e800640 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b400d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7978009fd0 tx=0x7f79780118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=649097398f2fd285 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f796f400640 1 -- 172.21.15.27:0/2148335148 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978004d20 con 0x560e2b059350 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f796f400640 1 -- 172.21.15.27:0/2148335148 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978004050 con 0x560e2b059350 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.812+0000 7f796f400640 1 -- 172.21.15.27:0/2148335148 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978005280 con 0x560e2b059350 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f7977400640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79640b6680 0x7f79640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f7977400640 1 -- 172.21.15.27:0/3249512235 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f7970107960 con 0x7f7980077940 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f797de00640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79640b6680 0x7f79640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f798bb22740 1 -- 172.21.15.27:0/2148335148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 msgr2=0x560e2b400d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f798bb22740 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b400d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7978009fd0 tx=0x7f79780118d0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f798bb22740 1 -- 172.21.15.27:0/2148335148 shutdown_connections 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f798bb22740 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b038670 0x560e2b401790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f798bb22740 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b401250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f798bb22740 1 --2- 172.21.15.27:0/2148335148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b400d10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.813+0000 7f798bb22740 1 -- 172.21.15.27:0/2148335148 >> 172.21.15.27:0/2148335148 conn(0x560e2b04d740 msgr2=0x560e2b3fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.814+0000 7f798bb22740 1 -- 172.21.15.27:0/2148335148 shutdown_connections 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.814+0000 7f798bb22740 1 -- 172.21.15.27:0/2148335148 wait complete. 2024-09-21T07:22:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.814+0000 7f797de00640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79640b6680 0x7f79640b8b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7970104db0 tx=0x7f797010d550 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.814+0000 7f798bb22740 1 -- start start 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 1 Processor -- start 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b40c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b038670 0x560e2b40cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b418840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560e2b1c7890 con 0x560e2b030490 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560e2b301160 con 0x560e2b038670 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e2b40f4f0 con 0x560e2b059350 2024-09-21T07:22:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.815+0000 7f798bb22740 10 client.0 _ll_register_callbacks cb 0x560e2b31da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f797e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b40c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f7975400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b038670 0x560e2b40cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f797e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b40c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37484/0 (socket says 172.21.15.27:37484) 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f797e800640 1 -- 172.21.15.27:0/548646759 learned_addr learned my addr 172.21.15.27:0/548646759 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f7975400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b038670 0x560e2b40cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52934/0 (socket says 172.21.15.27:52934) 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f797e800640 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b418840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f7975400640 1 -- 172.21.15.27:0/548646759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 msgr2=0x560e2b418840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f7975400640 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b418840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f7975400640 1 -- 172.21.15.27:0/548646759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 msgr2=0x560e2b40c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f7975400640 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b40c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f7975400640 1 -- 172.21.15.27:0/548646759 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e2b418d80 con 0x560e2b038670 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.816+0000 7f797e800640 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b418840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f7975400640 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b038670 0x560e2b40cd90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f798006c810 tx=0x7f796800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fadbfc9515f57f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f7963e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:22:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f7963e00640 1 -- 172.21.15.27:0/548646759 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968012110 con 0x560e2b038670 2024-09-21T07:22:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7978004050 con 0x560e2b038670 2024-09-21T07:22:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f7963e00640 1 -- 172.21.15.27:0/548646759 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f7968004050 con 0x560e2b038670 2024-09-21T07:22:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f7963e00640 1 -- 172.21.15.27:0/548646759 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7968025050 con 0x560e2b038670 2024-09-21T07:22:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f798bb22740 10 client.28319 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.817+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e2b41b740 con 0x560e2b038670 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f7963e00640 1 -- 172.21.15.27:0/548646759 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978004050 con 0x560e2b038670 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f7963e00640 1 -- 172.21.15.27:0/548646759 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7968010430 con 0x560e2b038670 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f798bb22740 10 client.28319 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f798bb22740 10 client.28319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f798bb22740 20 client.28319 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f798bb22740 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2b017cd0 0x560e2b41ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f798bb22740 4 client.28319 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e2b378de0 con 0x560e2b017cd0 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.818+0000 7f797e800640 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2b017cd0 0x560e2b41ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.819+0000 7f797e800640 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2b017cd0 0x560e2b41ebf0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232c429ed0e304b1 server_cookie=556a5d4d9e7eff1d in_seq=0 out_seq=0 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.819+0000 7f7963e00640 10 client.28319 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f7963e00640 1 -- 172.21.15.27:0/548646759 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e2b378de0 con 0x560e2b017cd0 2024-09-21T07:22:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f7963e00640 10 client.28319 handle_command_reply: tid=0 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 client.28319 shutdown 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b038670 msgr2=0x560e2b40cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b038670 0x560e2b40cd90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f798006c810 tx=0x7f796800f790 comp rx=0 tx=0).stop 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 shutdown_connections 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2b017cd0 0x560e2b41ebf0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b059350 0x560e2b418840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b038670 0x560e2b40cd90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 --2- 172.21.15.27:0/548646759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b40c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 >> 172.21.15.27:0/548646759 conn(0x560e2b04d740 msgr2=0x560e2b403ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f7963e00640 0 client.28319 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:34.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.820+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 shutdown_connections 2024-09-21T07:22:34.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.821+0000 7f798bb22740 1 -- 172.21.15.27:0/548646759 wait complete. 2024-09-21T07:22:34.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.821+0000 7f798bb22740 20 client.28319 trim_cache size 0 max 16384 2024-09-21T07:22:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.839+0000 7f798bb22740 1 Processor -- start 2024-09-21T07:22:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.839+0000 7f798bb22740 1 -- start start 2024-09-21T07:22:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f798bb22740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b403e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f798bb22740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b404350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f798bb22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b059350 0x560e2b409840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f798bb22740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f795c012830 con 0x560e2b030490 2024-09-21T07:22:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f798bb22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f795c010100 con 0x560e2b059350 2024-09-21T07:22:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f798bb22740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f795c00df70 con 0x560e2b038670 2024-09-21T07:22:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f797e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b403e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f797e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b403e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37494/0 (socket says 172.21.15.27:37494) 2024-09-21T07:22:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f797e800640 1 -- 172.21.15.27:0/2850632375 learned_addr learned my addr 172.21.15.27:0/2850632375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.840+0000 7f7975400640 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b404350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f7975400640 1 -- 172.21.15.27:0/2850632375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b059350 msgr2=0x560e2b409840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f7975400640 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b059350 0x560e2b409840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f7975400640 1 -- 172.21.15.27:0/2850632375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 msgr2=0x560e2b403e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f7975400640 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b403e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f7975400640 1 -- 172.21.15.27:0/2850632375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2b407ee0 con 0x560e2b038670 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f797e800640 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b403e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f7975400640 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b404350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f79680109c0 tx=0x7f79680123a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4bac0f12ef3c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f796ea00640 1 -- 172.21.15.27:0/2850632375 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796802e070 con 0x560e2b038670 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f796ea00640 1 -- 172.21.15.27:0/2850632375 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7968005560 con 0x560e2b038670 2024-09-21T07:22:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.841+0000 7f796ea00640 1 -- 172.21.15.27:0/2850632375 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79680035f0 con 0x560e2b038670 2024-09-21T07:22:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 -- 172.21.15.27:0/2850632375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 msgr2=0x560e2b404350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b404350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f79680109c0 tx=0x7f79680123a0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 -- 172.21.15.27:0/2850632375 shutdown_connections 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b404350 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b059350 0x560e2b409840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 --2- 172.21.15.27:0/2850632375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b030490 0x560e2b403e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 -- 172.21.15.27:0/2850632375 >> 172.21.15.27:0/2850632375 conn(0x560e2b04d740 msgr2=0x560e2b40aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 -- 172.21.15.27:0/2850632375 shutdown_connections 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.842+0000 7f798bb22740 1 -- 172.21.15.27:0/2850632375 wait complete. 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 -- start start 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 Processor -- start 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b410a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b410f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b059350 0x560e2b4885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f795c00e5c0 con 0x560e2b059350 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f795c00fa30 con 0x560e2b030490 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.843+0000 7f798bb22740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f795c00ec90 con 0x560e2b038670 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f798bb22740 10 client.0 _ll_register_callbacks cb 0x7f7968009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f797e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b410a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b410f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f797e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b059350 0x560e2b4885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b410f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41378/0 (socket says 172.21.15.27:41378) 2024-09-21T07:22:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 -- 172.21.15.27:0/676311840 learned_addr learned my addr 172.21.15.27:0/676311840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 -- 172.21.15.27:0/676311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 msgr2=0x560e2b410a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b410a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 -- 172.21.15.27:0/676311840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b059350 msgr2=0x560e2b4885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b059350 0x560e2b4885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 -- 172.21.15.27:0/676311840 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e2b4117d0 con 0x560e2b038670 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7975400640 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b410f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f79680059f0 tx=0x7f7968010660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8f0d9ff352f8d40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.844+0000 7f7963e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.845+0000 7f7963e00640 1 -- 172.21.15.27:0/676311840 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796802e070 con 0x560e2b038670 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.845+0000 7f7963e00640 1 -- 172.21.15.27:0/676311840 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f79680050d0 con 0x560e2b038670 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.845+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7968005560 con 0x560e2b038670 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.845+0000 7f7963e00640 1 -- 172.21.15.27:0/676311840 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f79680052d0 con 0x560e2b038670 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.845+0000 7f798bb22740 10 client.28276 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.845+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e2b489b80 con 0x560e2b038670 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.845+0000 7f7963e00640 1 -- 172.21.15.27:0/676311840 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7968005560 con 0x560e2b038670 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f7963e00640 1 -- 172.21.15.27:0/676311840 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f796801c420 con 0x560e2b038670 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f798bb22740 10 client.28276 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f798bb22740 10 client.28276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f798bb22740 20 client.28276 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f798bb22740 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2afed080 0x560e2b48c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f798bb22740 4 client.28276 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e2b374890 con 0x560e2afed080 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.846+0000 7f797e800640 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2afed080 0x560e2b48c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.847+0000 7f797e800640 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2afed080 0x560e2b48c2a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee71cda829263148 server_cookie=223959c7bd85b092 in_seq=0 out_seq=0 2024-09-21T07:22:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.847+0000 7f7963e00640 10 client.28276 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.848+0000 7f7963e00640 1 -- 172.21.15.27:0/676311840 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e2b374890 con 0x560e2afed080 2024-09-21T07:22:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.848+0000 7f7963e00640 10 client.28276 handle_command_reply: tid=0 2024-09-21T07:22:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.848+0000 7f798bb22740 1 client.28276 shutdown 2024-09-21T07:22:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.848+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 msgr2=0x560e2b410f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.848+0000 7f798bb22740 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b410f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f79680059f0 tx=0x7f7968010660 comp rx=0 tx=0).stop 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 shutdown_connections 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560e2afed080 0x560e2b48c2a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e2b038670 0x560e2b410f80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560e2b059350 0x560e2b4885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 --2- 172.21.15.27:0/676311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e2b030490 0x560e2b410a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 >> 172.21.15.27:0/676311840 conn(0x560e2b04d740 msgr2=0x560e2b403e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f7963e00640 0 client.28276 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 shutdown_connections 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 1 -- 172.21.15.27:0/676311840 wait complete. 2024-09-21T07:22:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.849+0000 7f798bb22740 20 client.28276 trim_cache size 0 max 16384 2024-09-21T07:22:34.853 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:22:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:22:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543", 2024-09-21T07:22:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:22:34.853 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:22:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79640b6680 msgr2=0x7f79640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79640b6680 0x7f79640b8b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7970104db0 tx=0x7f797010d550 comp rx=0 tx=0).stop 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980077940 msgr2=0x7f79801a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980077940 0x7f79801a0ad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7970005ab0 tx=0x7f7970003c10 comp rx=0 tx=0).stop 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 shutdown_connections 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79640b6680 0x7f79640b8b90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7980114740 0x7f79801a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7980078290 0x7f79801a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 --2- 172.21.15.27:0/3249512235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7980077940 0x7f79801a0ad0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 >> 172.21.15.27:0/3249512235 conn(0x7f7980101810 msgr2=0x7f79801167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 shutdown_connections 2024-09-21T07:22:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:34.850+0000 7f7985200640 1 -- 172.21.15.27:0/3249512235 wait complete. 2024-09-21T07:22:34.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:34 smithi027 ceph-mon[30779]: pgmap v672: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-09-21T07:22:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:34 smithi119 ceph-mon[34519]: pgmap v672: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-09-21T07:22:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:34 smithi153 ceph-mon[34747]: pgmap v672: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-09-21T07:22:35.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:22:36.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:35 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:22:36.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:35 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:22:36.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:35 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:22:36.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:35 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:22:36.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:35 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:22:36.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:35 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:22:36.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:35 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:22:36.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:35 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:22:36.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:35 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:22:36.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:35 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:22:36.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:35 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:22:36.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:35 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:22:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.871+0000 7f1c12600640 1 -- 172.21.15.27:0/1078101560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 msgr2=0x7f1c0c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.871+0000 7f1c12600640 1 --2- 172.21.15.27:0/1078101560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c110620 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009ca0 tx=0x7f1c0402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.871+0000 7f1c12600640 1 -- 172.21.15.27:0/1078101560 shutdown_connections 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.871+0000 7f1c12600640 1 --2- 172.21.15.27:0/1078101560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c110620 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.871+0000 7f1c12600640 1 --2- 172.21.15.27:0/1078101560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1094a0 0x7f1c0c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.871+0000 7f1c12600640 1 --2- 172.21.15.27:0/1078101560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103b60 0x7f1c0c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.871+0000 7f1c12600640 1 -- 172.21.15.27:0/1078101560 >> 172.21.15.27:0/1078101560 conn(0x7f1c0c07b850 msgr2=0x7f1c0c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.872+0000 7f1c12600640 1 -- 172.21.15.27:0/1078101560 shutdown_connections 2024-09-21T07:22:36.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.872+0000 7f1c12600640 1 -- 172.21.15.27:0/1078101560 wait complete. 2024-09-21T07:22:36.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.873+0000 7f1c12600640 1 Processor -- start 2024-09-21T07:22:36.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.873+0000 7f1c12600640 1 -- start start 2024-09-21T07:22:36.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c12600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103b60 0x7f1c0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c12600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c1094a0 0x7f1c0c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c12600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c12600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c113ab0 con 0x7f1c0c109df0 2024-09-21T07:22:36.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c12600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c113930 con 0x7f1c0c103b60 2024-09-21T07:22:36.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c12600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c113c30 con 0x7f1c0c1094a0 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37530/0 (socket says 172.21.15.27:37530) 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c0be00640 1 -- 172.21.15.27:0/100319108 learned_addr learned my addr 172.21.15.27:0/100319108 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c0b400640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103b60 0x7f1c0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.874+0000 7f1c03e00640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c1094a0 0x7f1c0c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c0be00640 1 -- 172.21.15.27:0/100319108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c1094a0 msgr2=0x7f1c0c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c0be00640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c1094a0 0x7f1c0c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c0be00640 1 -- 172.21.15.27:0/100319108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103b60 msgr2=0x7f1c0c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c0be00640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103b60 0x7f1c0c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c0be00640 1 -- 172.21.15.27:0/100319108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c1a86c0 con 0x7f1c0c109df0 2024-09-21T07:22:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c0b400640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103b60 0x7f1c0c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c0be00640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c076190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1c04005cc0 tx=0x7f1c0402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.875+0000 7f1c08c00640 1 -- 172.21.15.27:0/100319108 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04003bf0 con 0x7f1c0c109df0 2024-09-21T07:22:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.876+0000 7f1c08c00640 1 -- 172.21.15.27:0/100319108 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c04003d90 con 0x7f1c0c109df0 2024-09-21T07:22:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.876+0000 7f1c08c00640 1 -- 172.21.15.27:0/100319108 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0402fbc0 con 0x7f1c0c109df0 2024-09-21T07:22:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.876+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c1a8950 con 0x7f1c0c109df0 2024-09-21T07:22:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.876+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c1a8ee0 con 0x7f1c0c109df0 2024-09-21T07:22:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.879+0000 7f1c192b5740 1 Processor -- start 2024-09-21T07:22:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.879+0000 7f1c08c00640 1 -- 172.21.15.27:0/100319108 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1c0402fd60 con 0x7f1c0c109df0 2024-09-21T07:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c192b5740 1 -- start start 2024-09-21T07:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc6d070 0x56435d013b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc441b0 0x56435d0140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c192b5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56435cf8af70 con 0x56435cc4c390 2024-09-21T07:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c192b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56435cf7a3a0 con 0x56435cc441b0 2024-09-21T07:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c192b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56435cf7e1b0 con 0x56435cc6d070 2024-09-21T07:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc6d070 0x56435d013b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc6d070 0x56435d013b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41420/0 (socket says 172.21.15.27:41420) 2024-09-21T07:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc441b0 0x56435d0140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.880+0000 7f1c03e00640 1 -- 172.21.15.27:0/3092430423 learned_addr learned my addr 172.21.15.27:0/3092430423 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.881+0000 7f1c03e00640 1 -- 172.21.15.27:0/3092430423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc441b0 msgr2=0x56435d0140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.881+0000 7f1c03e00640 1 --2- 172.21.15.27:0/3092430423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc441b0 0x56435d0140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.881+0000 7f1c03e00640 1 -- 172.21.15.27:0/3092430423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 msgr2=0x56435d0145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.881+0000 7f1c03e00640 1 --2- 172.21.15.27:0/3092430423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.881+0000 7f1c03e00640 1 -- 172.21.15.27:0/3092430423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435cf44c90 con 0x56435cc6d070 2024-09-21T07:22:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.881+0000 7f1c08c00640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1bf00b6700 0x7f1bf00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.881+0000 7f1c03e00640 1 --2- 172.21.15.27:0/3092430423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc6d070 0x56435d013b60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c06c9f0 tx=0x7f1bf400b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6bec23e7096ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.882+0000 7f1c08c00640 1 -- 172.21.15.27:0/100319108 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f1c04108ef0 con 0x7f1c0c109df0 2024-09-21T07:22:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.882+0000 7f1c0b400640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1bf00b6700 0x7f1bf00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.882+0000 7f1c00c00640 1 -- 172.21.15.27:0/3092430423 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4004020 con 0x56435cc6d070 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.882+0000 7f1c00c00640 1 -- 172.21.15.27:0/3092430423 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf400fc10 con 0x56435cc6d070 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.882+0000 7f1c00c00640 1 -- 172.21.15.27:0/3092430423 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4003920 con 0x56435cc6d070 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 -- 172.21.15.27:0/3092430423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc6d070 msgr2=0x56435d013b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 --2- 172.21.15.27:0/3092430423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc6d070 0x56435d013b60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c06c9f0 tx=0x7f1bf400b410 comp rx=0 tx=0).stop 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c0b400640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1bf00b6700 0x7f1bf00b8c10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x56435d0127e0 tx=0x7f1bfc00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 -- 172.21.15.27:0/3092430423 shutdown_connections 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 --2- 172.21.15.27:0/3092430423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 --2- 172.21.15.27:0/3092430423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc441b0 0x56435d0140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 --2- 172.21.15.27:0/3092430423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc6d070 0x56435d013b60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 -- 172.21.15.27:0/3092430423 >> 172.21.15.27:0/3092430423 conn(0x56435cc61460 msgr2=0x56435d011a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 -- 172.21.15.27:0/3092430423 shutdown_connections 2024-09-21T07:22:36.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.883+0000 7f1c192b5740 1 -- 172.21.15.27:0/3092430423 wait complete. 2024-09-21T07:22:36.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 -- start start 2024-09-21T07:22:36.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 Processor -- start 2024-09-21T07:22:36.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc4c390 0x56435d01fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc6d070 0x56435d02b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56435cddb700 con 0x56435cc6d070 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56435cf15540 con 0x56435cc4c390 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.884+0000 7f1c192b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56435d022340 con 0x56435cc441b0 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c192b5740 10 client.0 _ll_register_callbacks cb 0x56435cf25430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc6d070 0x56435d02b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41430/0 (socket says 172.21.15.27:41430) 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc4c390 0x56435d01fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 -- 172.21.15.27:0/1198646743 learned_addr learned my addr 172.21.15.27:0/1198646743 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 -- 172.21.15.27:0/1198646743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc4c390 msgr2=0x56435d01fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc4c390 0x56435d01fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 -- 172.21.15.27:0/1198646743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc6d070 msgr2=0x56435d02b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc6d070 0x56435d02b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0b400640 1 -- 172.21.15.27:0/1198646743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435ccd8650 con 0x56435cc441b0 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.885+0000 7f1c0be00640 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc6d070 0x56435d02b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1c0b400640 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01f770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc012010 tx=0x7f1bfc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3248ccb1c9fec56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1bf9600640 1 -- 172.21.15.27:0/1198646743 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc01f930 con 0x56435cc441b0 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1bf9600640 1 -- 172.21.15.27:0/1198646743 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f1bfc017030 con 0x56435cc441b0 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56435cf17b50 con 0x56435cc441b0 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1bf9600640 1 -- 172.21.15.27:0/1198646743 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc032050 con 0x56435cc441b0 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1c192b5740 10 client.28288 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.886+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56435cf32910 con 0x56435cc441b0 2024-09-21T07:22:36.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1bf9600640 1 -- 172.21.15.27:0/1198646743 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56435cf17b50 con 0x56435cc441b0 2024-09-21T07:22:36.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1bf9600640 1 -- 172.21.15.27:0/1198646743 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1bfc027cd0 con 0x56435cc441b0 2024-09-21T07:22:36.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1c192b5740 10 client.28288 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:36.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1c192b5740 10 client.28288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1c192b5740 20 client.28288 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1c192b5740 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc2b9f0 0x56435d0318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1c192b5740 4 client.28288 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.887+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56435cf8cec0 con 0x56435cc2b9f0 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.888+0000 7f1c03e00640 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc2b9f0 0x56435d0318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.888+0000 7f1c03e00640 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc2b9f0 0x56435d0318d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816514ed20efa3c7 server_cookie=431f707dfda849df in_seq=0 out_seq=0 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.888+0000 7f1bf9600640 10 client.28288 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1bf9600640 1 -- 172.21.15.27:0/1198646743 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56435cf8cec0 con 0x56435cc2b9f0 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1bf9600640 10 client.28288 handle_command_reply: tid=0 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 client.28288 shutdown 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 msgr2=0x56435d01f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01f770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc012010 tx=0x7f1bfc004730 comp rx=0 tx=0).stop 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 shutdown_connections 2024-09-21T07:22:36.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc2b9f0 0x56435d0318d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc6d070 0x56435d02b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc4c390 0x56435d01fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 --2- 172.21.15.27:0/1198646743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01f770 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1bf9600640 0 client.28288 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.889+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 >> 172.21.15.27:0/1198646743 conn(0x56435cc61460 msgr2=0x56435d016cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.890+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 shutdown_connections 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.890+0000 7f1c192b5740 1 -- 172.21.15.27:0/1198646743 wait complete. 2024-09-21T07:22:36.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.890+0000 7f1c192b5740 20 client.28288 trim_cache size 0 max 16384 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 Processor -- start 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 -- start start 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d011000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b5900 con 0x56435cc4c390 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b31d0 con 0x56435cc6d070 2024-09-21T07:22:36.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.908+0000 7f1c192b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b1040 con 0x56435cc441b0 2024-09-21T07:22:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37556/0 (socket says 172.21.15.27:37556) 2024-09-21T07:22:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 -- 172.21.15.27:0/87541697 learned_addr learned my addr 172.21.15.27:0/87541697 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c0b400640 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c0be00640 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d011000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 -- 172.21.15.27:0/87541697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 msgr2=0x56435d01ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 -- 172.21.15.27:0/87541697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 msgr2=0x56435d011000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d011000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c03e00640 1 -- 172.21.15.27:0/87541697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435d020830 con 0x56435cc4c390 2024-09-21T07:22:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.909+0000 7f1c0b400640 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.910+0000 7f1c03e00640 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0152a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4012440 tx=0x7f1bf40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c5335b564b0d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.910+0000 7f1bfbe00640 1 -- 172.21.15.27:0/87541697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf402e040 con 0x56435cc4c390 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.910+0000 7f1bfbe00640 1 -- 172.21.15.27:0/87541697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf4005440 con 0x56435cc4c390 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.910+0000 7f1bfbe00640 1 -- 172.21.15.27:0/87541697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf40049e0 con 0x56435cc4c390 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 -- 172.21.15.27:0/87541697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 msgr2=0x56435d0152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0152a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4012440 tx=0x7f1bf40044c0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 -- 172.21.15.27:0/87541697 shutdown_connections 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d011000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d0152a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 --2- 172.21.15.27:0/87541697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d01ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 -- 172.21.15.27:0/87541697 >> 172.21.15.27:0/87541697 conn(0x56435cc61460 msgr2=0x56435d014b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 -- 172.21.15.27:0/87541697 shutdown_connections 2024-09-21T07:22:36.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 -- 172.21.15.27:0/87541697 wait complete. 2024-09-21T07:22:36.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.911+0000 7f1c192b5740 1 -- start start 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 1 Processor -- start 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d027e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d023340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d023990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b1690 con 0x56435cc4c390 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b2b00 con 0x56435cc6d070 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b1d60 con 0x56435cc441b0 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.912+0000 7f1c192b5740 10 client.0 _ll_register_callbacks cb 0x7f1bfc0107c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d027e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d023340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d027e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41450/0 (socket says 172.21.15.27:41450) 2024-09-21T07:22:36.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 -- 172.21.15.27:0/2886896181 learned_addr learned my addr 172.21.15.27:0/2886896181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0be00640 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d023990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 -- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 msgr2=0x56435d023990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d023990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 -- 172.21.15.27:0/2886896181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 msgr2=0x56435d023340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d023340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 -- 172.21.15.27:0/2886896181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435d024100 con 0x56435cc441b0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c03e00640 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d023340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1c0b400640 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d027e90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc01c3f0 tx=0x7f1bfc01c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166a587ce605e3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.913+0000 7f1bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.914+0000 7f1bf9600640 1 -- 172.21.15.27:0/2886896181 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc03b060 con 0x56435cc441b0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.914+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf4005440 con 0x56435cc441b0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.914+0000 7f1bf9600640 1 -- 172.21.15.27:0/2886896181 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f1bfc01fa00 con 0x56435cc441b0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.914+0000 7f1bf9600640 1 -- 172.21.15.27:0/2886896181 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc042050 con 0x56435cc441b0 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.914+0000 7f1c192b5740 10 client.28294 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:36.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.914+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56435d09c2e0 con 0x56435cc441b0 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.914+0000 7f1bf9600640 1 -- 172.21.15.27:0/2886896181 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf4005440 con 0x56435cc441b0 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1bf9600640 1 -- 172.21.15.27:0/2886896181 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1bfc005040 con 0x56435cc441b0 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1c192b5740 10 client.28294 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1c192b5740 10 client.28294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1c192b5740 20 client.28294 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1c192b5740 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc00da0 0x56435d09ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1c192b5740 4 client.28294 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:22:36.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56435cf88970 con 0x56435cc00da0 2024-09-21T07:22:36.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.915+0000 7f1c03e00640 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc00da0 0x56435d09ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:36.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1c03e00640 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc00da0 0x56435d09ead0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902c078c349931e5 server_cookie=a37442723d9e3fce in_seq=0 out_seq=0 2024-09-21T07:22:36.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1bf9600640 10 client.28294 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1bf9600640 1 -- 172.21.15.27:0/2886896181 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x56435cf88970 con 0x56435cc00da0 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1bf9600640 10 client.28294 handle_command_reply: tid=0 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1c192b5740 1 client.28294 shutdown 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 msgr2=0x56435d027e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1c192b5740 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d027e90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc01c3f0 tx=0x7f1bfc01c980 comp rx=0 tx=0).stop 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 shutdown_connections 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.916+0000 7f1c192b5740 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56435cc00da0 0x56435d09ead0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c192b5740 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56435cc6d070 0x56435d023990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c192b5740 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56435cc4c390 0x56435d023340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c192b5740 1 --2- 172.21.15.27:0/2886896181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56435cc441b0 0x56435d027e90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1bf9600640 0 client.28294 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 >> 172.21.15.27:0/2886896181 conn(0x56435cc61460 msgr2=0x56435d014b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 shutdown_connections 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c192b5740 1 -- 172.21.15.27:0/2886896181 wait complete. 2024-09-21T07:22:36.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c192b5740 20 client.28294 trim_cache size 0 max 16384 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543", 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:22:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:22:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:22:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:36.929 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1bf00b6700 msgr2=0x7f1bf00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c12600640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1bf00b6700 0x7f1bf00b8c10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x56435d0127e0 tx=0x7f1bfc00b390 comp rx=0 tx=0).stop 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 msgr2=0x7f1c0c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.917+0000 7f1c12600640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c076190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1c04005cc0 tx=0x7f1c0402f380 comp rx=0 tx=0).stop 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 shutdown_connections 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c1094a0 0x7f1c0c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1bf00b6700 0x7f1bf00b8c10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0c109df0 0x7f1c0c076190 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 --2- 172.21.15.27:0/100319108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103b60 0x7f1c0c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 >> 172.21.15.27:0/100319108 conn(0x7f1c0c07b850 msgr2=0x7f1c0c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 shutdown_connections 2024-09-21T07:22:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:36.918+0000 7f1c12600640 1 -- 172.21.15.27:0/100319108 wait complete. 2024-09-21T07:22:36.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:36 smithi027 ceph-mon[30779]: pgmap v673: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:22:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:36 smithi119 ceph-mon[34519]: pgmap v673: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:22:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:36 smithi153 ceph-mon[34747]: pgmap v673: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:22:37.859 INFO:tasks.cephfs.filesystem:scrub status for tag:ecab2dc3-aa2b-40f9-bc04-9dcbbf928543 - {'path': '/', 'tag': 'ecab2dc3-aa2b-40f9-bc04-9dcbbf928543', 'options': 'recursive,force'} 2024-09-21T07:22:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:38 smithi119 ceph-mon[34519]: pgmap v674: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:22:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:38 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:22:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:38 smithi027 ceph-mon[30779]: pgmap v674: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:22:39.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:38 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:22:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:38 smithi153 ceph-mon[34747]: pgmap v674: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:22:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:38 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:22:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:40 smithi119 ceph-mon[34519]: pgmap v675: 129 pgs: 129 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-21T07:22:41.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:40 smithi027 ceph-mon[30779]: pgmap v675: 129 pgs: 129 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-21T07:22:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:40 smithi153 ceph-mon[34747]: pgmap v675: 129 pgs: 129 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-09-21T07:22:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 -- 172.21.15.27:0/3566840496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 msgr2=0x7feccc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 --2- 172.21.15.27:0/3566840496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc103d20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009900 tx=0x7fecbc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 -- 172.21.15.27:0/3566840496 shutdown_connections 2024-09-21T07:22:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 --2- 172.21.15.27:0/3566840496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feccc109240 0x7feccc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 --2- 172.21.15.27:0/3566840496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feccc109b90 0x7feccc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 --2- 172.21.15.27:0/3566840496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc103d20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 -- 172.21.15.27:0/3566840496 >> 172.21.15.27:0/3566840496 conn(0x7feccc0ff330 msgr2=0x7feccc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 -- 172.21.15.27:0/3566840496 shutdown_connections 2024-09-21T07:22:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.770+0000 7fecd1800640 1 -- 172.21.15.27:0/3566840496 wait complete. 2024-09-21T07:22:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.771+0000 7fecd1800640 1 Processor -- start 2024-09-21T07:22:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.771+0000 7fecd1800640 1 -- start start 2024-09-21T07:22:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd1800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd1800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feccc109240 0x7feccc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feccc109b90 0x7feccc19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd1800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feccc113850 con 0x7feccc103910 2024-09-21T07:22:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1136d0 con 0x7feccc109b90 2024-09-21T07:22:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd1800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1139d0 con 0x7feccc109240 2024-09-21T07:22:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53822/0 (socket says 172.21.15.27:53822) 2024-09-21T07:22:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 -- 172.21.15.27:0/3907337098 learned_addr learned my addr 172.21.15.27:0/3907337098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecc3e00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feccc109240 0x7feccc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd0e00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feccc109b90 0x7feccc19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 -- 172.21.15.27:0/3907337098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feccc109240 msgr2=0x7feccc19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feccc109240 0x7feccc19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 -- 172.21.15.27:0/3907337098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feccc109b90 msgr2=0x7feccc19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feccc109b90 0x7feccc19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7feccbe00640 1 -- 172.21.15.27:0/3907337098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc1a0200 con 0x7feccc103910 2024-09-21T07:22:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.772+0000 7fecd0e00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feccc109b90 0x7feccc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.773+0000 7feccbe00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc19a2d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005940 tx=0x7fecbc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.773+0000 7fecc9600640 1 -- 172.21.15.27:0/3907337098 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc03f030 con 0x7feccc103910 2024-09-21T07:22:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.773+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1a0490 con 0x7feccc103910 2024-09-21T07:22:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.773+0000 7fecc9600640 1 -- 172.21.15.27:0/3907337098 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc0044c0 con 0x7feccc103910 2024-09-21T07:22:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.773+0000 7fecc9600640 1 -- 172.21.15.27:0/3907337098 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc035070 con 0x7feccc103910 2024-09-21T07:22:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.773+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a6c80 con 0x7feccc103910 2024-09-21T07:22:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.776+0000 7fecda036740 1 Processor -- start 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.776+0000 7fecda036740 1 -- start start 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecda036740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x563334879220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecda036740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334879760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecda036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x563334879ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecda036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5633347ea6b0 con 0x5633344af4d0 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecda036740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5633347d9ae0 con 0x5633344a72f0 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecda036740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5633347dd8f0 con 0x5633344d01b0 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecc9600640 1 -- 172.21.15.27:0/3907337098 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fecbc042030 con 0x7feccc103910 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x563334879ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x563334879ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53832/0 (socket says 172.21.15.27:53832) 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecc3e00640 1 -- 172.21.15.27:0/2554238526 learned_addr learned my addr 172.21.15.27:0/2554238526 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7fecd0e00640 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x563334879220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.777+0000 7feccbe00640 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334879760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc3e00640 1 -- 172.21.15.27:0/2554238526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 msgr2=0x563334879220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc3e00640 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x563334879220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc3e00640 1 -- 172.21.15.27:0/2554238526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 msgr2=0x563334879760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc3e00640 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334879760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc3e00640 1 -- 172.21.15.27:0/2554238526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633347a6780 con 0x5633344af4d0 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecd0e00640 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x563334879220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7feccbe00640 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334879760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc3e00640 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x563334879ca0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009c20 tx=0x7fecb4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c70aaa11ff6fac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc1600640 1 -- 172.21.15.27:0/2554238526 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4004230 con 0x5633344af4d0 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc1600640 1 -- 172.21.15.27:0/2554238526 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb40043d0 con 0x5633344af4d0 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.778+0000 7fecc1600640 1 -- 172.21.15.27:0/2554238526 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4004cd0 con 0x5633344af4d0 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecc9600640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb00b6680 0x7fecb00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecc9600640 1 -- 172.21.15.27:0/3907337098 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fecbc107c20 con 0x7feccc103910 2024-09-21T07:22:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecd0e00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb00b6680 0x7fecb00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecda036740 1 -- 172.21.15.27:0/2554238526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 msgr2=0x563334879ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecda036740 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x563334879ca0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009c20 tx=0x7fecb4011810 comp rx=0 tx=0).stop 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecda036740 1 -- 172.21.15.27:0/2554238526 shutdown_connections 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecda036740 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x563334879ca0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecda036740 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334879760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.779+0000 7fecda036740 1 --2- 172.21.15.27:0/2554238526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x563334879220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.780+0000 7fecda036740 1 -- 172.21.15.27:0/2554238526 >> 172.21.15.27:0/2554238526 conn(0x5633344c4300 msgr2=0x563334877110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.780+0000 7fecda036740 1 -- 172.21.15.27:0/2554238526 shutdown_connections 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.780+0000 7fecda036740 1 -- 172.21.15.27:0/2554238526 wait complete. 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.780+0000 7fecd0e00640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb00b6680 0x7fecb00b8b90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fecc40099a0 tx=0x7fecc4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.780+0000 7fecda036740 1 -- start start 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 1 Processor -- start 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334884830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344af4d0 0x563334884d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344d01b0 0x5633348852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563334711ab0 con 0x5633344d01b0 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563334780570 con 0x5633344a72f0 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5633348879a0 con 0x5633344af4d0 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7fecda036740 10 client.0 _ll_register_callbacks cb 0x56333478f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344af4d0 0x563334884d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.781+0000 7feccbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334884830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344af4d0 0x563334884d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52348/0 (socket says 172.21.15.27:52348) 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecd0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344d01b0 0x5633348852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 -- 172.21.15.27:0/765264782 learned_addr learned my addr 172.21.15.27:0/765264782 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 -- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 msgr2=0x563334884830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334884830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 -- 172.21.15.27:0/765264782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344d01b0 msgr2=0x5633348852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344d01b0 0x5633348852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 -- 172.21.15.27:0/765264782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333453b790 con 0x5633344af4d0 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7feccbe00640 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334884830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecc3e00640 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344af4d0 0x563334884d70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fecb4019b00 tx=0x7fecb4019b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b98467406137024 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.782+0000 7fecba000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.784+0000 7fecba000640 1 -- 172.21.15.27:0/765264782 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4028020 con 0x5633344af4d0 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.784+0000 7fecba000640 1 -- 172.21.15.27:0/765264782 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fecb4024620 con 0x5633344af4d0 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.784+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb40043d0 con 0x5633344af4d0 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.784+0000 7fecba000640 1 -- 172.21.15.27:0/765264782 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecb4033050 con 0x5633344af4d0 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.784+0000 7fecda036740 10 client.28306 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.784+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633345b4620 con 0x5633344af4d0 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecba000640 1 -- 172.21.15.27:0/765264782 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb40043d0 con 0x5633344af4d0 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecba000640 1 -- 172.21.15.27:0/765264782 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fecb4018070 con 0x5633344af4d0 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecda036740 10 client.28306 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecda036740 10 client.28306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecda036740 20 client.28306 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecda036740 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56333448ed80 0x5633348968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecda036740 4 client.28306 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633347ec600 con 0x56333448ed80 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.785+0000 7feccbe00640 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56333448ed80 0x5633348968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.786+0000 7feccbe00640 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56333448ed80 0x5633348968f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419d4595f212e79d server_cookie=4375a128b17c6d3 in_seq=0 out_seq=0 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.786+0000 7fecba000640 10 client.28306 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecba000640 1 -- 172.21.15.27:0/765264782 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633347ec600 con 0x56333448ed80 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecba000640 10 client.28306 handle_command_reply: tid=0 2024-09-21T07:22:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 client.28306 shutdown 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344af4d0 msgr2=0x563334884d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344af4d0 0x563334884d70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fecb4019b00 tx=0x7fecb4019b30 comp rx=0 tx=0).stop 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 shutdown_connections 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56333448ed80 0x5633348968f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344af4d0 0x563334884d70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecba000640 0 client.28306 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344d01b0 0x5633348852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 --2- 172.21.15.27:0/765264782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344a72f0 0x563334884830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 >> 172.21.15.27:0/765264782 conn(0x5633344c4300 msgr2=0x56333487a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 shutdown_connections 2024-09-21T07:22:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 1 -- 172.21.15.27:0/765264782 wait complete. 2024-09-21T07:22:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.787+0000 7fecda036740 20 client.28306 trim_cache size 0 max 16384 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 Processor -- start 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 -- start start 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344a72f0 0x56333487ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344af4d0 0x563334886460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x56333489bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1b1290 con 0x5633344a72f0 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1aeb60 con 0x5633344af4d0 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecda036740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1ac9d0 con 0x5633344d01b0 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7feccbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344a72f0 0x56333487ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344af4d0 0x563334886460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7feccbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344a72f0 0x56333487ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53858/0 (socket says 172.21.15.27:53858) 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7feccbe00640 1 -- 172.21.15.27:0/3412667964 learned_addr learned my addr 172.21.15.27:0/3412667964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344af4d0 0x563334886460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37802/0 (socket says 172.21.15.27:37802) 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.803+0000 7fecd0e00640 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x56333489bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7fecc3e00640 1 -- 172.21.15.27:0/3412667964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 msgr2=0x56333489bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7fecc3e00640 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x56333489bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7fecc3e00640 1 -- 172.21.15.27:0/3412667964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344a72f0 msgr2=0x56333487ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7fecc3e00640 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344a72f0 0x56333487ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7fecc3e00640 1 -- 172.21.15.27:0/3412667964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563334890bc0 con 0x5633344af4d0 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7feccbe00640 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344a72f0 0x56333487ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7fecd0e00640 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x56333489bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:22:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.804+0000 7fecc3e00640 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344af4d0 0x563334886460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fecb4011d10 tx=0x7fecb4009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4e08019aaadc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.805+0000 7fecc0c00640 1 -- 172.21.15.27:0/3412667964 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4028040 con 0x5633344af4d0 2024-09-21T07:22:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.805+0000 7fecc0c00640 1 -- 172.21.15.27:0/3412667964 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb401f040 con 0x5633344af4d0 2024-09-21T07:22:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.805+0000 7fecc0c00640 1 -- 172.21.15.27:0/3412667964 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40042d0 con 0x5633344af4d0 2024-09-21T07:22:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.805+0000 7fecda036740 1 -- 172.21.15.27:0/3412667964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344af4d0 msgr2=0x563334886460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.805+0000 7fecda036740 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344af4d0 0x563334886460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fecb4011d10 tx=0x7fecb4009af0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 -- 172.21.15.27:0/3412667964 shutdown_connections 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344d01b0 0x56333489bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344af4d0 0x563334886460 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 --2- 172.21.15.27:0/3412667964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344a72f0 0x56333487ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 -- 172.21.15.27:0/3412667964 >> 172.21.15.27:0/3412667964 conn(0x5633344c4300 msgr2=0x563334885d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 -- 172.21.15.27:0/3412667964 shutdown_connections 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 -- 172.21.15.27:0/3412667964 wait complete. 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 -- start start 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.806+0000 7fecda036740 1 Processor -- start 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecda036740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344a72f0 0x56333488d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecda036740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x56333488a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecda036740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344d01b0 0x56333488ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecda036740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1ad020 con 0x5633344af4d0 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecda036740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1ae490 con 0x5633344d01b0 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecda036740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1ad6f0 con 0x5633344a72f0 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecda036740 10 client.0 _ll_register_callbacks cb 0x7fecc4009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x56333488a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7feccbe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344a72f0 0x56333488d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x56333488a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53860/0 (socket says 172.21.15.27:53860) 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 -- 172.21.15.27:0/4137846301 learned_addr learned my addr 172.21.15.27:0/4137846301 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecd0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344d01b0 0x56333488ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 -- 172.21.15.27:0/4137846301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344a72f0 msgr2=0x56333488d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344a72f0 0x56333488d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 -- 172.21.15.27:0/4137846301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344d01b0 msgr2=0x56333488ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344d01b0 0x56333488ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.807+0000 7fecc3e00640 1 -- 172.21.15.27:0/4137846301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333488b450 con 0x5633344af4d0 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecc3e00640 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x56333488a820 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fecb40059e0 tx=0x7fecb4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586bd79dbe838ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecba000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecba000640 1 -- 172.21.15.27:0/4137846301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4028070 con 0x5633344af4d0 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecba000640 1 -- 172.21.15.27:0/4137846301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fecb4004270 con 0x5633344af4d0 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb401f040 con 0x5633344af4d0 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecba000640 1 -- 172.21.15.27:0/4137846301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecb40044b0 con 0x5633344af4d0 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecda036740 10 client.18324 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.808+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563334901010 con 0x5633344af4d0 2024-09-21T07:22:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecba000640 1 -- 172.21.15.27:0/4137846301 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb401f040 con 0x5633344af4d0 2024-09-21T07:22:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecba000640 1 -- 172.21.15.27:0/4137846301 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fecb403e030 con 0x5633344af4d0 2024-09-21T07:22:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecda036740 10 client.18324 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecda036740 10 client.18324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecda036740 20 client.18324 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecda036740 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5633344641c0 0x563334903720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecda036740 4 client.18324 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:22:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633347e80b0 con 0x5633344641c0 2024-09-21T07:22:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.809+0000 7feccbe00640 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5633344641c0 0x563334903720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.810+0000 7feccbe00640 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5633344641c0 0x563334903720 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea620fd16e74899d server_cookie=1615dbbb91c4dc4 in_seq=0 out_seq=0 2024-09-21T07:22:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.810+0000 7fecba000640 10 client.18324 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.823+0000 7fecba000640 1 -- 172.21.15.27:0/4137846301 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633347e80b0 con 0x5633344641c0 2024-09-21T07:22:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.823+0000 7fecba000640 10 client.18324 handle_command_reply: tid=0 2024-09-21T07:22:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.823+0000 7fecda036740 1 client.18324 shutdown 2024-09-21T07:22:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.823+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 msgr2=0x56333488a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.823+0000 7fecda036740 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x56333488a820 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fecb40059e0 tx=0x7fecb4009260 comp rx=0 tx=0).stop 2024-09-21T07:22:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 shutdown_connections 2024-09-21T07:22:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5633344641c0 0x563334903720 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5633344d01b0 0x56333488ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633344af4d0 0x56333488a820 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecba000640 0 client.18324 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 --2- 172.21.15.27:0/4137846301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5633344a72f0 0x56333488d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 >> 172.21.15.27:0/4137846301 conn(0x5633344c4300 msgr2=0x56333487a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 shutdown_connections 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 1 -- 172.21.15.27:0/4137846301 wait complete. 2024-09-21T07:22:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.824+0000 7fecda036740 20 client.18324 trim_cache size 0 max 16384 2024-09-21T07:22:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:22:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:22:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:22:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:22:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb00b6680 msgr2=0x7fecb00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb00b6680 0x7fecb00b8b90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fecc40099a0 tx=0x7fecc4009500 comp rx=0 tx=0).stop 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 msgr2=0x7feccc19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc19a2d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005940 tx=0x7fecbc0040c0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 shutdown_connections 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feccc109240 0x7feccc19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecb00b6680 0x7fecb00b8b90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feccc109b90 0x7feccc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 --2- 172.21.15.27:0/3907337098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feccc103910 0x7feccc19a2d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 >> 172.21.15.27:0/3907337098 conn(0x7feccc0ff330 msgr2=0x7feccc101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 shutdown_connections 2024-09-21T07:22:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:41.825+0000 7fecd1800640 1 -- 172.21.15.27:0/3907337098 wait complete. 2024-09-21T07:22:43.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:42 smithi027 ceph-mon[30779]: pgmap v676: 129 pgs: 129 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-21T07:22:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:42 smithi153 ceph-mon[34747]: pgmap v676: 129 pgs: 129 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-21T07:22:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:42 smithi119 ceph-mon[34519]: pgmap v676: 129 pgs: 129 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-09-21T07:22:45.087 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:44 smithi119 ceph-mon[34519]: pgmap v677: 129 pgs: 129 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 70 KiB/s wr, 19 op/s 2024-09-21T07:22:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:44 smithi027 ceph-mon[30779]: pgmap v677: 129 pgs: 129 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 70 KiB/s wr, 19 op/s 2024-09-21T07:22:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:44 smithi153 ceph-mon[34747]: pgmap v677: 129 pgs: 129 active+clean; 1.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 70 KiB/s wr, 19 op/s 2024-09-21T07:22:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:46 smithi027 ceph-mon[30779]: pgmap v678: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-21T07:22:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:46 smithi153 ceph-mon[34747]: pgmap v678: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-21T07:22:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:46 smithi119 ceph-mon[34519]: pgmap v678: 129 pgs: 129 active+clean; 1.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-21T07:22:47.758 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:22:47.758 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T07:22:47.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:22:48.158 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 -- 172.21.15.27:0/1316467648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d4105bb0 msgr2=0x7fe0d4105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 --2- 172.21.15.27:0/1316467648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d4105bb0 0x7fe0d4105fc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009900 tx=0x7fe0c402a410 comp rx=0 tx=0).stop 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 -- 172.21.15.27:0/1316467648 shutdown_connections 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 --2- 172.21.15.27:0/1316467648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d410b4f0 0x7fe0d410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 --2- 172.21.15.27:0/1316467648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d410be40 0x7fe0d4112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 --2- 172.21.15.27:0/1316467648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d4105bb0 0x7fe0d4105fc0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 -- 172.21.15.27:0/1316467648 >> 172.21.15.27:0/1316467648 conn(0x7fe0d4101680 msgr2=0x7fe0d4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 -- 172.21.15.27:0/1316467648 shutdown_connections 2024-09-21T07:22:48.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.815+0000 7fe0d9200640 1 -- 172.21.15.27:0/1316467648 wait complete. 2024-09-21T07:22:48.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.816+0000 7fe0d9200640 1 Processor -- start 2024-09-21T07:22:48.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.817+0000 7fe0d9200640 1 -- start start 2024-09-21T07:22:48.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.817+0000 7fe0d9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d4105bb0 0x7fe0d419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.817+0000 7fe0d9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d410b4f0 0x7fe0d419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.817+0000 7fe0d9200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d410be40 0x7fe0d41a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.817+0000 7fe0d9200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d4115b50 con 0x7fe0d410b4f0 2024-09-21T07:22:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.817+0000 7fe0d9200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d41159d0 con 0x7fe0d4105bb0 2024-09-21T07:22:48.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.817+0000 7fe0d9200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d4115cd0 con 0x7fe0d410be40 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d4105bb0 0x7fe0d419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0caa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d410b4f0 0x7fe0d419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d4105bb0 0x7fe0d419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41144/0 (socket says 172.21.15.27:41144) 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3400640 1 -- 172.21.15.27:0/2130943212 learned_addr learned my addr 172.21.15.27:0/2130943212 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3e00640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d410be40 0x7fe0d41a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3e00640 1 -- 172.21.15.27:0/2130943212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d4105bb0 msgr2=0x7fe0d419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3e00640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d4105bb0 0x7fe0d419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3e00640 1 -- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d410b4f0 msgr2=0x7fe0d419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:48.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3e00640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d410b4f0 0x7fe0d419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3e00640 1 -- 172.21.15.27:0/2130943212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d41a2530 con 0x7fe0d410be40 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0d3400640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d4105bb0 0x7fe0d419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.818+0000 7fe0caa00640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d410b4f0 0x7fe0d419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.819+0000 7fe0d3e00640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d410be40 0x7fe0d41a1e00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00ec50 tx=0x7fe0cc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.819+0000 7fe0d0c00640 1 -- 172.21.15.27:0/2130943212 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc003b70 con 0x7fe0d410be40 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.819+0000 7fe0d0c00640 1 -- 172.21.15.27:0/2130943212 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0cc003d10 con 0x7fe0d410be40 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.819+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d41a2820 con 0x7fe0d410be40 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.819+0000 7fe0d0c00640 1 -- 172.21.15.27:0/2130943212 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc010830 con 0x7fe0d410be40 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.819+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d41a8f80 con 0x7fe0d410be40 2024-09-21T07:22:48.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.820+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d4105fc0 con 0x7fe0d410be40 2024-09-21T07:22:48.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.823+0000 7fe0d0c00640 1 -- 172.21.15.27:0/2130943212 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe0cc003ec0 con 0x7fe0d410be40 2024-09-21T07:22:48.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.824+0000 7fe0d0c00640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0b80b6680 0x7fe0b80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:48.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.824+0000 7fe0d0c00640 1 -- 172.21.15.27:0/2130943212 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7fe0cc0e7ff0 con 0x7fe0d410be40 2024-09-21T07:22:48.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.824+0000 7fe0d3400640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0b80b6680 0x7fe0b80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.825+0000 7fe0d3400640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0b80b6680 0x7fe0b80b8b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40096c0 tx=0x7fe0c4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:48.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:48.828+0000 7fe0d0c00640 1 -- 172.21.15.27:0/2130943212 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0cc0ae150 con 0x7fe0d410be40 2024-09-21T07:22:49.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.067+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0d402e640 con 0x7fe0d410be40 2024-09-21T07:22:49.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:48 smithi027 ceph-mon[30779]: pgmap v679: 129 pgs: 129 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:22:49.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.068+0000 7fe0d0c00640 1 -- 172.21.15.27:0/2130943212 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fe0cc0b2530 con 0x7fe0d410be40 2024-09-21T07:22:49.070 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:22:49.070 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:22:49.071 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:22:49.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0b80b6680 msgr2=0x7fe0b80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:49.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0b80b6680 0x7fe0b80b8b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40096c0 tx=0x7fe0c4005a20 comp rx=0 tx=0).stop 2024-09-21T07:22:49.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d410be40 msgr2=0x7fe0d41a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:49.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d410be40 0x7fe0d41a1e00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00ec50 tx=0x7fe0cc00c6a0 comp rx=0 tx=0).stop 2024-09-21T07:22:49.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 shutdown_connections 2024-09-21T07:22:49.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0d410be40 0x7fe0d41a1e00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:49.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0b80b6680 0x7fe0b80b8b90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:49.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0d410b4f0 0x7fe0d419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:49.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 --2- 172.21.15.27:0/2130943212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0d4105bb0 0x7fe0d419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:49.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 >> 172.21.15.27:0/2130943212 conn(0x7fe0d4101680 msgr2=0x7fe0d4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:49.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 shutdown_connections 2024-09-21T07:22:49.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:49.071+0000 7fe0d9200640 1 -- 172.21.15.27:0/2130943212 wait complete. 2024-09-21T07:22:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:48 smithi153 ceph-mon[34747]: pgmap v679: 129 pgs: 129 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:22:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:48 smithi119 ceph-mon[34519]: pgmap v679: 129 pgs: 129 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:22:49.753 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T07:22:49.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T07:22:50.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:49 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2130943212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:22:50.143 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:22:50.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:49 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2130943212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:22:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:49 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2130943212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 -- 172.21.15.27:0/585258345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6410dc50 msgr2=0x7f2a64110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 --2- 172.21.15.27:0/585258345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6410dc50 0x7f2a64110140 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009ca0 tx=0x7f2a5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 -- 172.21.15.27:0/585258345 shutdown_connections 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 --2- 172.21.15.27:0/585258345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a64105140 0x7f2a6410d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 --2- 172.21.15.27:0/585258345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6410dc50 0x7f2a64110140 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 --2- 172.21.15.27:0/585258345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 0x7f2a64104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 -- 172.21.15.27:0/585258345 >> 172.21.15.27:0/585258345 conn(0x7f2a640fe330 msgr2=0x7f2a64100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.776+0000 7f2a69000640 1 -- 172.21.15.27:0/585258345 shutdown_connections 2024-09-21T07:22:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.777+0000 7f2a69000640 1 -- 172.21.15.27:0/585258345 wait complete. 2024-09-21T07:22:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 Processor -- start 2024-09-21T07:22:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 -- start start 2024-09-21T07:22:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 0x7f2a6419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a64105140 0x7f2a6419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a6410dc50 0x7f2a641a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64113900 con 0x7f2a64105140 2024-09-21T07:22:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64113780 con 0x7f2a641047f0 2024-09-21T07:22:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.778+0000 7f2a69000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64113a80 con 0x7f2a6410dc50 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a64105140 0x7f2a6419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 0x7f2a6419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a6410dc50 0x7f2a641a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 0x7f2a6419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41154/0 (socket says 172.21.15.27:41154) 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 -- 172.21.15.27:0/159431230 learned_addr learned my addr 172.21.15.27:0/159431230 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 -- 172.21.15.27:0/159431230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a6410dc50 msgr2=0x7f2a641a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a6410dc50 0x7f2a641a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 -- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a64105140 msgr2=0x7f2a6419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a64105140 0x7f2a6419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 -- 172.21.15.27:0/159431230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641a2460 con 0x7f2a641047f0 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.779+0000 7f2a63400640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 0x7f2a6419c4f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e9f0 tx=0x7f2a5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.780+0000 7f2a60c00640 1 -- 172.21.15.27:0/159431230 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400cd80 con 0x7f2a641047f0 2024-09-21T07:22:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.780+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641a2750 con 0x7f2a641047f0 2024-09-21T07:22:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.780+0000 7f2a60c00640 1 -- 172.21.15.27:0/159431230 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a540108e0 con 0x7f2a641047f0 2024-09-21T07:22:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.780+0000 7f2a60c00640 1 -- 172.21.15.27:0/159431230 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54018980 con 0x7f2a641047f0 2024-09-21T07:22:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.780+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a8eb0 con 0x7f2a641047f0 2024-09-21T07:22:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.783+0000 7f2a712d9740 1 Processor -- start 2024-09-21T07:22:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.783+0000 7f2a712d9740 1 -- start start 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80dca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a712d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5628a8051ca0 con 0x5628a7d13b60 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a712d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5628a80410d0 con 0x5628a7d0b980 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a712d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5628a8044ee0 con 0x5628a7d34840 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80dcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37672/0 (socket says 172.21.15.27:37672) 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 -- 172.21.15.27:0/164703785 learned_addr learned my addr 172.21.15.27:0/164703785 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a63e00640 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a60c00640 1 -- 172.21.15.27:0/159431230 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2a54010430 con 0x7f2a641047f0 2024-09-21T07:22:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 -- 172.21.15.27:0/164703785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 msgr2=0x5628a80dc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80dc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 -- 172.21.15.27:0/164703785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 msgr2=0x5628a80dca50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:22:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80dca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.784+0000 7f2a5be00640 1 -- 172.21.15.27:0/164703785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a7ff3400 con 0x5628a7d13b60 2024-09-21T07:22:50.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.785+0000 7f2a5be00640 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80dcf90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009ba0 tx=0x7f2a4c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8bbb5475442ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:50.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.785+0000 7f2a58c00640 1 -- 172.21.15.27:0/164703785 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004230 con 0x5628a7d13b60 2024-09-21T07:22:50.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.785+0000 7f2a58c00640 1 -- 172.21.15.27:0/164703785 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a4c009e10 con 0x5628a7d13b60 2024-09-21T07:22:50.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.785+0000 7f2a58c00640 1 -- 172.21.15.27:0/164703785 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004d10 con 0x5628a7d13b60 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 -- 172.21.15.27:0/164703785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 msgr2=0x5628a80dcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80dcf90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009ba0 tx=0x7f2a4c011810 comp rx=0 tx=0).stop 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 -- 172.21.15.27:0/164703785 shutdown_connections 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80dcf90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 --2- 172.21.15.27:0/164703785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80dc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 -- 172.21.15.27:0/164703785 >> 172.21.15.27:0/164703785 conn(0x5628a7d28990 msgr2=0x5628a80da400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a60c00640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a480b6670 0x7f2a480b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 -- 172.21.15.27:0/164703785 shutdown_connections 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a60c00640 1 -- 172.21.15.27:0/159431230 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f2a540e7e00 con 0x7f2a641047f0 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a712d9740 1 -- 172.21.15.27:0/164703785 wait complete. 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.786+0000 7f2a63e00640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a480b6670 0x7f2a480b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.787+0000 7f2a712d9740 1 -- start start 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.787+0000 7f2a63e00640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a480b6670 0x7f2a480b8b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006010 tx=0x7f2a5c005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.787+0000 7f2a712d9740 1 Processor -- start 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d0b980 0x5628a80e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d13b60 0x5628a80e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d34840 0x5628a80e8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a712d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5628a7ea27a0 con 0x5628a7d0b980 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a712d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5628a7fe6a10 con 0x5628a7d34840 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a712d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5628a80ead40 con 0x5628a7d13b60 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a712d9740 10 client.0 _ll_register_callbacks cb 0x5628a7ff6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a63400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d0b980 0x5628a80e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a63400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d0b980 0x5628a80e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37680/0 (socket says 172.21.15.27:37680) 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a63400640 1 -- 172.21.15.27:0/1666578653 learned_addr learned my addr 172.21.15.27:0/1666578653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a5be00640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d13b60 0x5628a80e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.788+0000 7f2a63e00640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d34840 0x5628a80e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a5be00640 1 -- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d34840 msgr2=0x5628a80e8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a5be00640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d34840 0x5628a80e8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a5be00640 1 -- 172.21.15.27:0/1666578653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d0b980 msgr2=0x5628a80e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a5be00640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d0b980 0x5628a80e7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a5be00640 1 -- 172.21.15.27:0/1666578653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a7daeb90 con 0x5628a7d13b60 2024-09-21T07:22:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a63400640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d0b980 0x5628a80e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a5be00640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d13b60 0x5628a80e8030 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c599303686a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a51600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.789+0000 7f2a51600640 1 -- 172.21.15.27:0/1666578653 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c027020 con 0x5628a7d13b60 2024-09-21T07:22:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.790+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a4c009e10 con 0x5628a7d13b60 2024-09-21T07:22:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.790+0000 7f2a51600640 1 -- 172.21.15.27:0/1666578653 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f2a4c003c90 con 0x5628a7d13b60 2024-09-21T07:22:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.790+0000 7f2a51600640 1 -- 172.21.15.27:0/1666578653 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a4c032050 con 0x5628a7d13b60 2024-09-21T07:22:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.790+0000 7f2a712d9740 10 client.28330 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.790+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628a80f6970 con 0x5628a7d13b60 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.790+0000 7f2a51600640 1 -- 172.21.15.27:0/1666578653 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a4c009e10 con 0x5628a7d13b60 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a51600640 1 -- 172.21.15.27:0/1666578653 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2a4c018070 con 0x5628a7d13b60 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a712d9740 10 client.28330 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a712d9740 10 client.28330 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a712d9740 20 client.28330 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cf31c0 0x5628a80f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a712d9740 4 client.28330 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628a8053bf0 con 0x5628a7cf31c0 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.791+0000 7f2a63400640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cf31c0 0x5628a80f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.792+0000 7f2a63400640 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cf31c0 0x5628a80f9d30 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=569864970387b812 server_cookie=ef2b407cc4846299 in_seq=0 out_seq=0 2024-09-21T07:22:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.792+0000 7f2a51600640 10 client.28330 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.793+0000 7f2a51600640 1 -- 172.21.15.27:0/1666578653 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628a8053bf0 con 0x5628a7cf31c0 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.793+0000 7f2a51600640 10 client.28330 handle_command_reply: tid=0 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.793+0000 7f2a712d9740 1 client.28330 shutdown 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.793+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d13b60 msgr2=0x5628a80e8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.793+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d13b60 0x5628a80e8030 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.793+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 shutdown_connections 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.793+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cf31c0 0x5628a80f9d30 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d13b60 0x5628a80e8030 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d34840 0x5628a80e8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1666578653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d0b980 0x5628a80e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a51600640 0 client.28330 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 >> 172.21.15.27:0/1666578653 conn(0x5628a7d28990 msgr2=0x5628a80df6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 shutdown_connections 2024-09-21T07:22:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a712d9740 1 -- 172.21.15.27:0/1666578653 wait complete. 2024-09-21T07:22:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.794+0000 7f2a712d9740 20 client.28330 trim_cache size 0 max 16384 2024-09-21T07:22:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.812+0000 7f2a712d9740 1 Processor -- start 2024-09-21T07:22:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a712d9740 1 -- start start 2024-09-21T07:22:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a712d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b57e0 con 0x5628a7d13b60 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a712d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b30b0 con 0x5628a7d0b980 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a712d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b0f20 con 0x5628a7d34840 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a63400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a63400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80ddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41180/0 (socket says 172.21.15.27:41180) 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a63400640 1 -- 172.21.15.27:0/3285215225 learned_addr learned my addr 172.21.15.27:0/3285215225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.813+0000 7f2a63e00640 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a63400640 1 -- 172.21.15.27:0/3285215225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 msgr2=0x5628a80e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a63400640 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a63400640 1 -- 172.21.15.27:0/3285215225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 msgr2=0x5628a80e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a63400640 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80e4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a63400640 1 -- 172.21.15.27:0/3285215225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a80dacf0 con 0x5628a7d0b980 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a5be00640 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a63400640 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80ddc50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ad8e0 tx=0x7f2a54018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46387c4734288b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a53e00640 1 -- 172.21.15.27:0/3285215225 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a540b2380 con 0x5628a7d0b980 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a53e00640 1 -- 172.21.15.27:0/3285215225 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a540b72c0 con 0x5628a7d0b980 2024-09-21T07:22:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.814+0000 7f2a53e00640 1 -- 172.21.15.27:0/3285215225 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a540aee30 con 0x5628a7d0b980 2024-09-21T07:22:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.815+0000 7f2a712d9740 1 -- 172.21.15.27:0/3285215225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 msgr2=0x5628a80ddc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.815+0000 7f2a712d9740 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80ddc50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ad8e0 tx=0x7f2a54018420 comp rx=0 tx=0).stop 2024-09-21T07:22:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.815+0000 7f2a712d9740 1 -- 172.21.15.27:0/3285215225 shutdown_connections 2024-09-21T07:22:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.815+0000 7f2a712d9740 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.815+0000 7f2a712d9740 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.815+0000 7f2a712d9740 1 --2- 172.21.15.27:0/3285215225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80ddc50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.815+0000 7f2a712d9740 1 -- 172.21.15.27:0/3285215225 >> 172.21.15.27:0/3285215225 conn(0x5628a7d28990 msgr2=0x5628a80dd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.816+0000 7f2a712d9740 1 -- 172.21.15.27:0/3285215225 shutdown_connections 2024-09-21T07:22:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.816+0000 7f2a712d9740 1 -- 172.21.15.27:0/3285215225 wait complete. 2024-09-21T07:22:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.816+0000 7f2a712d9740 1 -- start start 2024-09-21T07:22:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.816+0000 7f2a712d9740 1 Processor -- start 2024-09-21T07:22:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a712d9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a712d9740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b1570 con 0x5628a7d13b60 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a712d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b29e0 con 0x5628a7d0b980 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a712d9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b1c40 con 0x5628a7d34840 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a712d9740 10 client.0 _ll_register_callbacks cb 0x7f2a4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a63400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37692/0 (socket says 172.21.15.27:37692) 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a5be00640 1 -- 172.21.15.27:0/1071357509 learned_addr learned my addr 172.21.15.27:0/1071357509 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.817+0000 7f2a63e00640 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a63400640 1 -- 172.21.15.27:0/1071357509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 msgr2=0x5628a80ee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a63400640 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80ee4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a63400640 1 -- 172.21.15.27:0/1071357509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 msgr2=0x5628a80eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a63400640 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a63400640 1 -- 172.21.15.27:0/1071357509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a81635d0 con 0x5628a7d0b980 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a5be00640 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a63400640 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80f08f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2a540e57a0 tx=0x7f2a540aa050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49445f8baaa95293 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a51600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a51600640 1 -- 172.21.15.27:0/1071357509 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a540b2380 con 0x5628a7d0b980 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a540b72c0 con 0x5628a7d0b980 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.818+0000 7f2a51600640 1 -- 172.21.15.27:0/1071357509 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f2a540aebd0 con 0x5628a7d0b980 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a51600640 1 -- 172.21.15.27:0/1071357509 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a540aedd0 con 0x5628a7d0b980 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 10 client.28379 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628a8164670 con 0x5628a7d0b980 2024-09-21T07:22:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a51600640 1 -- 172.21.15.27:0/1071357509 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a540b72c0 con 0x5628a7d0b980 2024-09-21T07:22:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a51600640 1 -- 172.21.15.27:0/1071357509 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2a540ac7b0 con 0x5628a7d0b980 2024-09-21T07:22:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 10 client.28379 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 10 client.28379 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:22:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 20 client.28379 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cc8570 0x5628a8166e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 4 client.28379 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T07:22:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.819+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x5628a804f6a0 con 0x5628a7cc8570 2024-09-21T07:22:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.820+0000 7f2a5be00640 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cc8570 0x5628a8166e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.820+0000 7f2a5be00640 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cc8570 0x5628a8166e70 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84314fcb72188677 server_cookie=cd027fc90a202de1 in_seq=0 out_seq=0 2024-09-21T07:22:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.820+0000 7f2a51600640 10 client.28379 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:50.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a51600640 1 -- 172.21.15.27:0/1071357509 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5628a804f6a0 con 0x5628a7cc8570 2024-09-21T07:22:50.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a51600640 10 client.28379 handle_command_reply: tid=0 2024-09-21T07:22:50.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 client.28379 shutdown 2024-09-21T07:22:50.835 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:50 smithi027 ceph-mon[30779]: pgmap v680: 129 pgs: 129 active+clean; 1.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-09-21T07:22:50.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 msgr2=0x5628a80f08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80f08f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2a540e57a0 tx=0x7f2a540aa050 comp rx=0 tx=0).stop 2024-09-21T07:22:50.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 shutdown_connections 2024-09-21T07:22:50.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5628a7cc8570 0x5628a8166e70 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628a7d34840 0x5628a80ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5628a7d13b60 0x5628a80eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 --2- 172.21.15.27:0/1071357509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5628a7d0b980 0x5628a80f08f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 >> 172.21.15.27:0/1071357509 conn(0x5628a7d28990 msgr2=0x5628a80dd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.833+0000 7f2a51600640 0 client.28379 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.834+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 shutdown_connections 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.834+0000 7f2a712d9740 1 -- 172.21.15.27:0/1071357509 wait complete. 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.834+0000 7f2a712d9740 20 client.28379 trim_cache size 0 max 16384 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:22:50.837 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 38, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "fbaed821": { 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 38, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 144.5, 2024-09-21T07:22:50.838 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:22:50.839 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a480b6670 msgr2=0x7f2a480b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a480b6670 0x7f2a480b8b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006010 tx=0x7f2a5c005d80 comp rx=0 tx=0).stop 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 msgr2=0x7f2a6419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 0x7f2a6419c4f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e9f0 tx=0x7f2a5400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 shutdown_connections 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a6410dc50 0x7f2a641a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2a480b6670 0x7f2a480b8b80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a64105140 0x7f2a6419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 --2- 172.21.15.27:0/159431230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a641047f0 0x7f2a6419c4f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 >> 172.21.15.27:0/159431230 conn(0x7f2a640fe330 msgr2=0x7f2a64102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:50.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 shutdown_connections 2024-09-21T07:22:50.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:22:50.835+0000 7f2a69000640 1 -- 172.21.15.27:0/159431230 wait complete. 2024-09-21T07:22:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:50 smithi153 ceph-mon[34747]: pgmap v680: 129 pgs: 129 active+clean; 1.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-09-21T07:22:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:50 smithi119 ceph-mon[34519]: pgmap v680: 129 pgs: 129 active+clean; 1.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-09-21T07:22:51.585 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fbaed821', quiesce took 3.8 seconds. Will release after: 20.7 2024-09-21T07:22:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:52 smithi027 ceph-mon[30779]: pgmap v681: 129 pgs: 129 active+clean; 1.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:22:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:52 smithi153 ceph-mon[34747]: pgmap v681: 129 pgs: 129 active+clean; 1.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:22:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:52 smithi119 ceph-mon[34519]: pgmap v681: 129 pgs: 129 active+clean; 1.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 -- 172.21.15.27:0/2869081697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 msgr2=0x7f58741103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 --2- 172.21.15.27:0/2869081697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741103c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f586c009ca0 tx=0x7f586c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 -- 172.21.15.27:0/2869081697 shutdown_connections 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 --2- 172.21.15.27:0/2869081697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5874109240 0x7f5874109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 --2- 172.21.15.27:0/2869081697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741103c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 --2- 172.21.15.27:0/2869081697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5874103910 0x7f5874103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 -- 172.21.15.27:0/2869081697 >> 172.21.15.27:0/2869081697 conn(0x7f58740ff330 msgr2=0x7f5874101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 -- 172.21.15.27:0/2869081697 shutdown_connections 2024-09-21T07:22:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.035+0000 7f5873e00640 1 -- 172.21.15.27:0/2869081697 wait complete. 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.036+0000 7f5873e00640 1 Processor -- start 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.036+0000 7f5873e00640 1 -- start start 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.036+0000 7f5873e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5874103910 0x7f587419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5874109240 0x7f587419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5874113850 con 0x7f5874109b90 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58741136d0 con 0x7f5874103910 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58741139d0 con 0x7f5874109240 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37704/0 (socket says 172.21.15.27:37704) 2024-09-21T07:22:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5872a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5874103910 0x7f587419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873400640 1 -- 172.21.15.27:0/3766469751 learned_addr learned my addr 172.21.15.27:0/3766469751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5872000640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5874109240 0x7f587419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5872a00640 1 -- 172.21.15.27:0/3766469751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5874109240 msgr2=0x7f587419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5872a00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5874109240 0x7f587419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5872a00640 1 -- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 msgr2=0x7f58741a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5872a00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5872a00640 1 -- 172.21.15.27:0/3766469751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58741a23a0 con 0x7f5874103910 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.037+0000 7f5873400640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.038+0000 7f5872a00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5874103910 0x7f587419c470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f586400e9f0 tx=0x7f586400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.038+0000 7f586b400640 1 -- 172.21.15.27:0/3766469751 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586400ccc0 con 0x7f5874103910 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.038+0000 7f586b400640 1 -- 172.21.15.27:0/3766469751 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58640108e0 con 0x7f5874103910 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.038+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58741a2690 con 0x7f5874103910 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.038+0000 7f586b400640 1 -- 172.21.15.27:0/3766469751 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864018880 con 0x7f5874103910 2024-09-21T07:22:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.038+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58741a8e20 con 0x7f5874103910 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.041+0000 7f5880c08740 1 Processor -- start 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5880c08740 1 -- start start 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5880c08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc0de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5880c08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc0e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5880c08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc0e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5880c08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55acddb836b0 con 0x55acdd8402f0 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5880c08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55acddb72ae0 con 0x55acdd8691b0 2024-09-21T07:22:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5880c08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55acddb768f0 con 0x55acdd8484d0 2024-09-21T07:22:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f586b400640 1 -- 172.21.15.27:0/3766469751 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5864010430 con 0x7f5874103910 2024-09-21T07:22:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5873400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc0de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5873400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc0de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41224/0 (socket says 172.21.15.27:41224) 2024-09-21T07:22:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5872000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc0e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.042+0000 7f5873400640 1 -- 172.21.15.27:0/2654935794 learned_addr learned my addr 172.21.15.27:0/2654935794 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5872a00640 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc0e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5872000640 1 -- 172.21.15.27:0/2654935794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 msgr2=0x55acddc0de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5872000640 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc0de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5872000640 1 -- 172.21.15.27:0/2654935794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 msgr2=0x55acddc0e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5872000640 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc0e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5872000640 1 -- 172.21.15.27:0/2654935794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acddb3f780 con 0x55acdd8484d0 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5873400640 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc0de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.043+0000 7f5872000640 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc0e890 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f587406c690 tx=0x7f58600107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4aa5242e156d6e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.044+0000 7f585fe00640 1 -- 172.21.15.27:0/2654935794 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004030 con 0x55acdd8484d0 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.044+0000 7f585fe00640 1 -- 172.21.15.27:0/2654935794 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58600035f0 con 0x55acdd8484d0 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.044+0000 7f585fe00640 1 -- 172.21.15.27:0/2654935794 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600116a0 con 0x55acdd8484d0 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.044+0000 7f586b400640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58580b6680 0x7f58580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.044+0000 7f586b400640 1 -- 172.21.15.27:0/3766469751 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f58640e7e60 con 0x7f5874103910 2024-09-21T07:22:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.044+0000 7f5873400640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58580b6680 0x7f58580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 -- 172.21.15.27:0/2654935794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 msgr2=0x55acddc0e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc0e890 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f587406c690 tx=0x7f58600107a0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5873400640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58580b6680 0x7f58580b8b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f586c033040 tx=0x7f586c005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 -- 172.21.15.27:0/2654935794 shutdown_connections 2024-09-21T07:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc0e890 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc0e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 --2- 172.21.15.27:0/2654935794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc0de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 -- 172.21.15.27:0/2654935794 >> 172.21.15.27:0/2654935794 conn(0x55acdd85d300 msgr2=0x55acddc0bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 -- 172.21.15.27:0/2654935794 shutdown_connections 2024-09-21T07:22:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.045+0000 7f5880c08740 1 -- 172.21.15.27:0/2654935794 wait complete. 2024-09-21T07:22:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.046+0000 7f5880c08740 1 -- start start 2024-09-21T07:22:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.046+0000 7f5880c08740 1 Processor -- start 2024-09-21T07:22:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5880c08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc19430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5880c08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc19970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5880c08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5880c08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55acddaaaab0 con 0x55acdd8484d0 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5880c08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55acddb19570 con 0x55acdd8691b0 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5880c08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55acddc1c590 con 0x55acdd8402f0 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5880c08740 10 client.0 _ll_register_callbacks cb 0x55acddb28100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.047+0000 7f5872000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc19970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc19970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37712/0 (socket says 172.21.15.27:37712) 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 -- 172.21.15.27:0/2747620658 learned_addr learned my addr 172.21.15.27:0/2747620658 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5873400640 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872a00640 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc19430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 -- 172.21.15.27:0/2747620658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 msgr2=0x55acddc19430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc19430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 -- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 msgr2=0x55acddc19ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc19ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 -- 172.21.15.27:0/2747620658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdd8d4790 con 0x55acdd8484d0 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f5872000640 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc19970 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f586000bf70 tx=0x7f586000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e112224956e4532 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.048+0000 7f585cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.049+0000 7f585cc00640 1 -- 172.21.15.27:0/2747620658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860029060 con 0x55acdd8484d0 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.049+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58600035f0 con 0x55acdd8484d0 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.049+0000 7f585cc00640 1 -- 172.21.15.27:0/2747620658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f5860003a30 con 0x55acdd8484d0 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.049+0000 7f585cc00640 1 -- 172.21.15.27:0/2747620658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5860030050 con 0x55acdd8484d0 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.049+0000 7f5880c08740 10 client.18360 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.049+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acdd94d620 con 0x55acdd8484d0 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.049+0000 7f585cc00640 1 -- 172.21.15.27:0/2747620658 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58600035f0 con 0x55acdd8484d0 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f585cc00640 1 -- 172.21.15.27:0/2747620658 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f58600046c0 con 0x55acdd8484d0 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f5880c08740 10 client.18360 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f5880c08740 10 client.18360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f5880c08740 20 client.18360 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f5880c08740 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd827d80 0x55acddc2b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f5880c08740 4 client.18360 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:22:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acddb85600 con 0x55acdd827d80 2024-09-21T07:22:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.050+0000 7f5872a00640 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd827d80 0x55acddc2b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.051+0000 7f5872a00640 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd827d80 0x55acddc2b470 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc087d2ff288bc0 server_cookie=ac26cbde1d8c45f2 in_seq=0 out_seq=0 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.051+0000 7f585cc00640 10 client.18360 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f585cc00640 1 -- 172.21.15.27:0/2747620658 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55acddb85600 con 0x55acdd827d80 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f585cc00640 10 client.18360 handle_command_reply: tid=0 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 client.18360 shutdown 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 msgr2=0x55acddc19970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc19970 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f586000bf70 tx=0x7f586000bfa0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 shutdown_connections 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd827d80 0x55acddc2b470 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc19ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc19970 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 --2- 172.21.15.27:0/2747620658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc19430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 >> 172.21.15.27:0/2747620658 conn(0x55acdd85d300 msgr2=0x55acddc0ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f585cc00640 0 client.18360 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 shutdown_connections 2024-09-21T07:22:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 1 -- 172.21.15.27:0/2747620658 wait complete. 2024-09-21T07:22:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.052+0000 7f5880c08740 20 client.18360 trim_cache size 0 max 16384 2024-09-21T07:22:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 Processor -- start 2024-09-21T07:22:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 -- start start 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc0f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc16600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc16b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58741b57e0 con 0x55acdd8484d0 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58741b30b0 con 0x55acdd8691b0 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5880c08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58741b0f20 con 0x55acdd8402f0 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.068+0000 7f5872000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc16600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc16600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37718/0 (socket says 172.21.15.27:37718) 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 -- 172.21.15.27:0/51941761 learned_addr learned my addr 172.21.15.27:0/51941761 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5873400640 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc16b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872a00640 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc0f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 -- 172.21.15.27:0/51941761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 msgr2=0x55acddc0f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc0f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 -- 172.21.15.27:0/51941761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 msgr2=0x55acddc16b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc16b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 -- 172.21.15.27:0/51941761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdd827fb0 con 0x55acdd8484d0 2024-09-21T07:22:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f5872000640 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc16600 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f586000c010 tx=0x7f58600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7b88ddfcbfec89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.069+0000 7f585f400640 1 -- 172.21.15.27:0/51941761 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860029070 con 0x55acdd8484d0 2024-09-21T07:22:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f585f400640 1 -- 172.21.15.27:0/51941761 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58600035f0 con 0x55acdd8484d0 2024-09-21T07:22:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f585f400640 1 -- 172.21.15.27:0/51941761 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004230 con 0x55acdd8484d0 2024-09-21T07:22:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 -- 172.21.15.27:0/51941761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 msgr2=0x55acddc16600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc16600 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f586000c010 tx=0x7f58600107a0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 -- 172.21.15.27:0/51941761 shutdown_connections 2024-09-21T07:22:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc16b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8484d0 0x55acddc16600 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 --2- 172.21.15.27:0/51941761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8402f0 0x55acddc0f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 -- 172.21.15.27:0/51941761 >> 172.21.15.27:0/51941761 conn(0x55acdd85d300 msgr2=0x55acddc0ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.070+0000 7f5880c08740 1 -- 172.21.15.27:0/51941761 shutdown_connections 2024-09-21T07:22:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.071+0000 7f5880c08740 1 -- 172.21.15.27:0/51941761 wait complete. 2024-09-21T07:22:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.071+0000 7f5880c08740 1 -- start start 2024-09-21T07:22:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.071+0000 7f5880c08740 1 Processor -- start 2024-09-21T07:22:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5880c08740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc1eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5880c08740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc1efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5880c08740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5880c08740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58741b1570 con 0x55acdd8402f0 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5880c08740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58741b29e0 con 0x55acdd8691b0 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5880c08740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58741b1c40 con 0x55acdd8484d0 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5880c08740 10 client.0 _ll_register_callbacks cb 0x7f586c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc1efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc1eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc1efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59534/0 (socket says 172.21.15.27:59534) 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 -- 172.21.15.27:0/644625392 learned_addr learned my addr 172.21.15.27:0/644625392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5873400640 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 -- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 msgr2=0x55acddc93c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc93c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 -- 172.21.15.27:0/644625392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 msgr2=0x55acddc1eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:22:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc1eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872000640 1 -- 172.21.15.27:0/644625392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acddc1f800 con 0x55acdd8484d0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.072+0000 7f5872a00640 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc1eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.073+0000 7f5872000640 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc1efe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5860005900 tx=0x7f5860004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c1250697c8b3e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.073+0000 7f585cc00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.073+0000 7f585cc00640 1 -- 172.21.15.27:0/644625392 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860029040 con 0x55acdd8484d0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.073+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58600035f0 con 0x55acdd8484d0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.073+0000 7f585cc00640 1 -- 172.21.15.27:0/644625392 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f5860005230 con 0x55acdd8484d0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.073+0000 7f585cc00640 1 -- 172.21.15.27:0/644625392 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5860005430 con 0x55acdd8484d0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.073+0000 7f5880c08740 10 client.28354 fetch_fsmap learned FSMap version 27 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acddc95160 con 0x55acdd8484d0 2024-09-21T07:22:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f585cc00640 1 -- 172.21.15.27:0/644625392 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58600035f0 con 0x55acdd8484d0 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f585cc00640 1 -- 172.21.15.27:0/644625392 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5860034430 con 0x55acdd8484d0 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f5880c08740 10 client.28354 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f5880c08740 10 client.28354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f5880c08740 20 client.28354 populate_metadata read hostname 'smithi027' 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f5880c08740 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd7fd1c0 0x55acddc97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f5880c08740 4 client.28354 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.074+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acddb810b0 con 0x55acdd7fd1c0 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.075+0000 7f5872a00640 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd7fd1c0 0x55acddc97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.075+0000 7f5872a00640 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd7fd1c0 0x55acddc97780 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=366d5cb0b20c194b server_cookie=3e80cb6a79530fb2 in_seq=0 out_seq=0 2024-09-21T07:22:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.075+0000 7f585cc00640 10 client.28354 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f585cc00640 1 -- 172.21.15.27:0/644625392 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acddb810b0 con 0x55acdd7fd1c0 2024-09-21T07:22:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f585cc00640 10 client.28354 handle_command_reply: tid=0 2024-09-21T07:22:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 client.28354 shutdown 2024-09-21T07:22:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 msgr2=0x55acddc1efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc1efe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5860005900 tx=0x7f5860004cb0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:54 smithi027 ceph-mon[30779]: pgmap v682: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-21T07:22:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 shutdown_connections 2024-09-21T07:22:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55acdd7fd1c0 0x55acddc97780 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acdd8484d0 0x55acddc1efe0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acdd8691b0 0x55acddc93c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 --2- 172.21.15.27:0/644625392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acdd8402f0 0x55acddc1eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f585cc00640 0 client.28354 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:22:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 >> 172.21.15.27:0/644625392 conn(0x55acdd85d300 msgr2=0x55acddc24770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 shutdown_connections 2024-09-21T07:22:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 1 -- 172.21.15.27:0/644625392 wait complete. 2024-09-21T07:22:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.089+0000 7f5880c08740 20 client.28354 trim_cache size 0 max 16384 2024-09-21T07:22:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:22:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:22:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:22:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58580b6680 msgr2=0x7f58580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58580b6680 0x7f58580b8b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f586c033040 tx=0x7f586c005c50 comp rx=0 tx=0).stop 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5874103910 msgr2=0x7f587419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5874103910 0x7f587419c470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f586400e9f0 tx=0x7f586400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 shutdown_connections 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5874109240 0x7f587419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58580b6680 0x7f58580b8b90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5874109b90 0x7f58741a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 --2- 172.21.15.27:0/3766469751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5874103910 0x7f587419c470 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:22:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 >> 172.21.15.27:0/3766469751 conn(0x7f58740ff330 msgr2=0x7f58741088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:22:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 shutdown_connections 2024-09-21T07:22:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:22:55.090+0000 7f5873e00640 1 -- 172.21.15.27:0/3766469751 wait complete. 2024-09-21T07:22:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:54 smithi153 ceph-mon[34747]: pgmap v682: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-21T07:22:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:54 smithi119 ceph-mon[34519]: pgmap v682: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-21T07:22:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:56 smithi027 ceph-mon[30779]: pgmap v683: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-09-21T07:22:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:56 smithi153 ceph-mon[34747]: pgmap v683: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-09-21T07:22:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:56 smithi119 ceph-mon[34519]: pgmap v683: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-09-21T07:22:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:22:58 smithi027 ceph-mon[30779]: pgmap v684: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-09-21T07:22:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:22:58 smithi153 ceph-mon[34747]: pgmap v684: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-09-21T07:22:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:22:58 smithi119 ceph-mon[34519]: pgmap v684: 129 pgs: 129 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-09-21T07:23:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:00 smithi027 ceph-mon[30779]: pgmap v685: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:23:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:00 smithi153 ceph-mon[34747]: pgmap v685: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:23:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:00 smithi119 ceph-mon[34519]: pgmap v685: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:23:03.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:02 smithi027 ceph-mon[30779]: pgmap v686: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:23:03.180 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:02 smithi153 ceph-mon[34747]: pgmap v686: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:23:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:02 smithi119 ceph-mon[34519]: pgmap v686: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:23:05.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:04 smithi027 ceph-mon[30779]: pgmap v687: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-09-21T07:23:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:04 smithi153 ceph-mon[34747]: pgmap v687: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-09-21T07:23:05.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:04 smithi119 ceph-mon[34519]: pgmap v687: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 25 op/s 2024-09-21T07:23:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:06 smithi027 ceph-mon[30779]: pgmap v688: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T07:23:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:06 smithi153 ceph-mon[34747]: pgmap v688: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T07:23:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:06 smithi119 ceph-mon[34519]: pgmap v688: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T07:23:07.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:23:08.279 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.308+0000 7f4f03200640 1 -- 172.21.15.27:0/3937572850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc10dba0 msgr2=0x7f4efc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.308+0000 7f4f03200640 1 --2- 172.21.15.27:0/3937572850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc10dba0 0x7f4efc110090 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009ca0 tx=0x7f4ef802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.308+0000 7f4f03200640 1 -- 172.21.15.27:0/3937572850 shutdown_connections 2024-09-21T07:23:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.308+0000 7f4f03200640 1 --2- 172.21.15.27:0/3937572850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc10dba0 0x7f4efc110090 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.308+0000 7f4f03200640 1 --2- 172.21.15.27:0/3937572850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4efc105090 0x7f4efc10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.308+0000 7f4f03200640 1 --2- 172.21.15.27:0/3937572850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 0x7f4efc104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.308+0000 7f4f03200640 1 -- 172.21.15.27:0/3937572850 >> 172.21.15.27:0/3937572850 conn(0x7f4efc0fe230 msgr2=0x7f4efc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.309+0000 7f4f03200640 1 -- 172.21.15.27:0/3937572850 shutdown_connections 2024-09-21T07:23:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.309+0000 7f4f03200640 1 -- 172.21.15.27:0/3937572850 wait complete. 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 Processor -- start 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 -- start start 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 0x7f4efc102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc105090 0x7f4efc1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4efc10dba0 0x7f4efc102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc113860 con 0x7f4efc105090 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1136e0 con 0x7f4efc10dba0 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.310+0000 7f4f03200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1139e0 con 0x7f4efc104740 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 0x7f4efc102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc105090 0x7f4efc1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc105090 0x7f4efc1029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49738/0 (socket says 172.21.15.27:49738) 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 0x7f4efc102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46848/0 (socket says 172.21.15.27:46848) 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f02800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4efc10dba0 0x7f4efc102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01400640 1 -- 172.21.15.27:0/3605639427 learned_addr learned my addr 172.21.15.27:0/3605639427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 -- 172.21.15.27:0/3605639427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4efc10dba0 msgr2=0x7f4efc102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4efc10dba0 0x7f4efc102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 -- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc105090 msgr2=0x7f4efc1029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc105090 0x7f4efc1029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 -- 172.21.15.27:0/3605639427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc1a8530 con 0x7f4efc104740 2024-09-21T07:23:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.311+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 0x7f4efc102470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000e9f0 tx=0x7f4ef000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.312+0000 7f4ef6a00640 1 -- 172.21.15.27:0/3605639427 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000ccc0 con 0x7f4efc104740 2024-09-21T07:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.312+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4efc1a8820 con 0x7f4efc104740 2024-09-21T07:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.312+0000 7f4ef6a00640 1 -- 172.21.15.27:0/3605639427 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef00108e0 con 0x7f4efc104740 2024-09-21T07:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.312+0000 7f4ef6a00640 1 -- 172.21.15.27:0/3605639427 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0018880 con 0x7f4efc104740 2024-09-21T07:23:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.312+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4efc1a8d70 con 0x7f4efc104740 2024-09-21T07:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.315+0000 7f4f0b892740 1 Processor -- start 2024-09-21T07:23:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.315+0000 7f4f0b892740 1 -- start start 2024-09-21T07:23:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb33e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe7662f0 0x5643feb34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f0b892740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5643feaa96b0 con 0x5643fe7662f0 2024-09-21T07:23:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f0b892740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5643fea98ae0 con 0x5643fe76e4d0 2024-09-21T07:23:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f0b892740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643fea9c8f0 con 0x5643fe78f1b0 2024-09-21T07:23:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f01400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f02800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb33e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f01400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb34890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59442/0 (socket says 172.21.15.27:59442) 2024-09-21T07:23:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.316+0000 7f4f01400640 1 -- 172.21.15.27:0/1687602428 learned_addr learned my addr 172.21.15.27:0/1687602428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4f01400640 1 -- 172.21.15.27:0/1687602428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 msgr2=0x5643feb33e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4f01400640 1 --2- 172.21.15.27:0/1687602428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb33e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4f01400640 1 -- 172.21.15.27:0/1687602428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe7662f0 msgr2=0x5643feb34350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4f01400640 1 --2- 172.21.15.27:0/1687602428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe7662f0 0x5643feb34350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4f01400640 1 -- 172.21.15.27:0/1687602428 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643fea65780 con 0x5643fe76e4d0 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4f01400640 1 --2- 172.21.15.27:0/1687602428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb34890 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009d30 tx=0x7f4eec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a8116ed1843d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4eeb000640 1 -- 172.21.15.27:0/1687602428 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec005220 con 0x5643fe76e4d0 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4eeb000640 1 -- 172.21.15.27:0/1687602428 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eec004c50 con 0x5643fe76e4d0 2024-09-21T07:23:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.317+0000 7f4eeb000640 1 -- 172.21.15.27:0/1687602428 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec019850 con 0x5643fe76e4d0 2024-09-21T07:23:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.318+0000 7f4ef6a00640 1 -- 172.21.15.27:0/3605639427 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4ef0018a20 con 0x7f4efc104740 2024-09-21T07:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 -- 172.21.15.27:0/1687602428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 msgr2=0x5643feb34890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 --2- 172.21.15.27:0/1687602428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb34890 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009d30 tx=0x7f4eec011810 comp rx=0 tx=0).stop 2024-09-21T07:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 -- 172.21.15.27:0/1687602428 shutdown_connections 2024-09-21T07:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 --2- 172.21.15.27:0/1687602428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb34890 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 --2- 172.21.15.27:0/1687602428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe7662f0 0x5643feb34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 --2- 172.21.15.27:0/1687602428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb33e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 -- 172.21.15.27:0/1687602428 >> 172.21.15.27:0/1687602428 conn(0x5643fe783300 msgr2=0x5643feb31d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 -- 172.21.15.27:0/1687602428 shutdown_connections 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.319+0000 7f4f0b892740 1 -- 172.21.15.27:0/1687602428 wait complete. 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.320+0000 7f4f0b892740 1 -- start start 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.320+0000 7f4ef6a00640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ee40b65c0 0x7f4ee40b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.320+0000 7f4f0b892740 1 Processor -- start 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.320+0000 7f4ef6a00640 1 -- 172.21.15.27:0/3605639427 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f4ef00e7e30 con 0x7f4efc104740 2024-09-21T07:23:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.320+0000 7f4f02800640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ee40b65c0 0x7f4ee40b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb3f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe76e4d0 0x5643feb3f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb3ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f0b892740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5643fe9d0ab0 con 0x5643fe76e4d0 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f0b892740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5643fea3f570 con 0x5643fe7662f0 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f0b892740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643feb42590 con 0x5643fe78f1b0 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f0b892740 10 client.0 _ll_register_callbacks cb 0x5643fea4e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f02800640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ee40b65c0 0x7f4ee40b8ad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006010 tx=0x7f4ef8005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f01400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe76e4d0 0x5643feb3f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f01400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe76e4d0 0x5643feb3f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49754/0 (socket says 172.21.15.27:49754) 2024-09-21T07:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f01400640 1 -- 172.21.15.27:0/3795861728 learned_addr learned my addr 172.21.15.27:0/3795861728 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f02800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb3ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.321+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb3f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4f01400640 1 -- 172.21.15.27:0/3795861728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 msgr2=0x5643feb3ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4f01400640 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb3ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4f01400640 1 -- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 msgr2=0x5643feb3f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4f01400640 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb3f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4f01400640 1 -- 172.21.15.27:0/3795861728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643fe7fa790 con 0x5643fe76e4d0 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4f02800640 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb3ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4f01400640 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe76e4d0 0x5643feb3f9f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4eec005030 tx=0x7f4eec005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c8adc754f1b340 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4edf400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.322+0000 7f4edf400640 1 -- 172.21.15.27:0/3795861728 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec028070 con 0x5643fe76e4d0 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.323+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eec004c50 con 0x5643fe76e4d0 2024-09-21T07:23:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.323+0000 7f4edf400640 1 -- 172.21.15.27:0/3795861728 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f4eec0195d0 con 0x5643fe76e4d0 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.323+0000 7f4edf400640 1 -- 172.21.15.27:0/3795861728 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4eec034050 con 0x5643fe76e4d0 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.323+0000 7f4f0b892740 10 client.18375 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.323+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643fe873620 con 0x5643fe76e4d0 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.323+0000 7f4edf400640 1 -- 172.21.15.27:0/3795861728 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eec004c50 con 0x5643fe76e4d0 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.323+0000 7f4edf400640 1 -- 172.21.15.27:0/3795861728 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4eec024430 con 0x5643fe76e4d0 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.324+0000 7f4f0b892740 10 client.18375 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.324+0000 7f4f0b892740 10 client.18375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.324+0000 7f4f0b892740 20 client.18375 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.324+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe74dd80 0x5643feb51490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.324+0000 7f4f0b892740 4 client.18375 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.324+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643feaab600 con 0x5643fe74dd80 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.324+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe74dd80 0x5643feb51490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.325+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe74dd80 0x5643feb51490 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8defa7ccabf72f server_cookie=6ceb7136f6ff96de in_seq=0 out_seq=0 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.325+0000 7f4edf400640 10 client.18375 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.325+0000 7f4edf400640 1 -- 172.21.15.27:0/3795861728 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643feaab600 con 0x5643fe74dd80 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.325+0000 7f4edf400640 10 client.18375 handle_command_reply: tid=0 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.325+0000 7f4f0b892740 1 client.18375 shutdown 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe76e4d0 msgr2=0x5643feb3f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe76e4d0 0x5643feb3f9f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4eec005030 tx=0x7f4eec005060 comp rx=0 tx=0).stop 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 shutdown_connections 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe74dd80 0x5643feb51490 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe78f1b0 0x5643feb3ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe76e4d0 0x5643feb3f9f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3795861728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb3f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 >> 172.21.15.27:0/3795861728 conn(0x5643fe783300 msgr2=0x5643feb34e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4edf400640 0 client.18375 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 shutdown_connections 2024-09-21T07:23:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 1 -- 172.21.15.27:0/3795861728 wait complete. 2024-09-21T07:23:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.326+0000 7f4f0b892740 20 client.18375 trim_cache size 0 max 16384 2024-09-21T07:23:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.342+0000 7f4f0b892740 1 Processor -- start 2024-09-21T07:23:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.342+0000 7f4f0b892740 1 -- start start 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb35550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe76e4d0 0x5643feb3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643feb3cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f0b892740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1b5890 con 0x5643fe78f1b0 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f0b892740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1b3160 con 0x5643fe7662f0 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f0b892740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1b0fd0 con 0x5643fe76e4d0 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643feb3cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb35550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643feb3cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49762/0 (socket says 172.21.15.27:49762) 2024-09-21T07:23:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f02800640 1 -- 172.21.15.27:0/890692621 learned_addr learned my addr 172.21.15.27:0/890692621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.343+0000 7f4f01400640 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe76e4d0 0x5643feb3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4f01e00640 1 -- 172.21.15.27:0/890692621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe76e4d0 msgr2=0x5643feb3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4f01e00640 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe76e4d0 0x5643feb3c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4f01e00640 1 -- 172.21.15.27:0/890692621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 msgr2=0x5643feb3cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4f01e00640 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643feb3cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4f01e00640 1 -- 172.21.15.27:0/890692621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643feb36110 con 0x5643fe7662f0 2024-09-21T07:23:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4f01e00640 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb35550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00e5170 tx=0x7f4ef000cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a524af29023f5370 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4eea600640 1 -- 172.21.15.27:0/890692621 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00b2350 con 0x5643fe7662f0 2024-09-21T07:23:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4eea600640 1 -- 172.21.15.27:0/890692621 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef00b7290 con 0x5643fe7662f0 2024-09-21T07:23:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.344+0000 7f4eea600640 1 -- 172.21.15.27:0/890692621 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00a9270 con 0x5643fe7662f0 2024-09-21T07:23:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 -- 172.21.15.27:0/890692621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 msgr2=0x5643feb35550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb35550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00e5170 tx=0x7f4ef000cef0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 -- 172.21.15.27:0/890692621 shutdown_connections 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe76e4d0 0x5643feb3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643feb3cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 --2- 172.21.15.27:0/890692621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe7662f0 0x5643feb35550 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 -- 172.21.15.27:0/890692621 >> 172.21.15.27:0/890692621 conn(0x5643fe783300 msgr2=0x5643feb34e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 -- 172.21.15.27:0/890692621 shutdown_connections 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.345+0000 7f4f0b892740 1 -- 172.21.15.27:0/890692621 wait complete. 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.346+0000 7f4f0b892740 1 -- start start 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.346+0000 7f4f0b892740 1 Processor -- start 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.346+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe7662f0 0x5643feb53f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.346+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.346+0000 7f4f0b892740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643febb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.346+0000 7f4f0b892740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1b1620 con 0x5643fe78f1b0 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f0b892740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1b2a90 con 0x5643fe76e4d0 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f0b892740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1b1cf0 con 0x5643fe7662f0 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f0b892740 10 client.0 _ll_register_callbacks cb 0x7f4ef0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe7662f0 0x5643feb53f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643febb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe7662f0 0x5643feb53f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46886/0 (socket says 172.21.15.27:46886) 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 -- 172.21.15.27:0/3749295762 learned_addr learned my addr 172.21.15.27:0/3749295762 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643febb3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49768/0 (socket says 172.21.15.27:49768) 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01400640 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 -- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 msgr2=0x5643feb544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 -- 172.21.15.27:0/3749295762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 msgr2=0x5643febb3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643febb3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 -- 172.21.15.27:0/3749295762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643febb42f0 con 0x5643fe7662f0 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.347+0000 7f4f01e00640 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe7662f0 0x5643feb53f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ad330 tx=0x7f4ef00e50e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0c0b6faf592204 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4edf400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4edf400640 1 -- 172.21.15.27:0/3749295762 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0010d90 con 0x5643fe7662f0 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef00b7290 con 0x5643fe7662f0 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4edf400640 1 -- 172.21.15.27:0/3749295762 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9461+0+0 (secure 0 0 0) 0x7f4ef0010a90 con 0x5643fe7662f0 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4edf400640 1 -- 172.21.15.27:0/3749295762 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ef00e53f0 con 0x5643fe7662f0 2024-09-21T07:23:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4f0b892740 10 client.28372 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643febbb730 con 0x5643fe7662f0 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4edf400640 1 -- 172.21.15.27:0/3749295762 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef00b7290 con 0x5643fe7662f0 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4edf400640 1 -- 172.21.15.27:0/3749295762 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef00a9270 con 0x5643fe7662f0 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4f0b892740 10 client.28372 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4f0b892740 10 client.28372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.348+0000 7f4f0b892740 20 client.28372 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.349+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe7231c0 0x5643febbddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.349+0000 7f4f0b892740 4 client.28372 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.349+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643feaa70b0 con 0x5643fe7231c0 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.349+0000 7f4f01400640 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe7231c0 0x5643febbddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.349+0000 7f4f01400640 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe7231c0 0x5643febbddb0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fda81dd459ea753 server_cookie=d60cfc5384e316b0 in_seq=0 out_seq=0 2024-09-21T07:23:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.349+0000 7f4edf400640 10 client.28372 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4edf400640 1 -- 172.21.15.27:0/3749295762 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643feaa70b0 con 0x5643fe7231c0 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4edf400640 10 client.28372 handle_command_reply: tid=0 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 client.28372 shutdown 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe7662f0 msgr2=0x5643feb53f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe7662f0 0x5643feb53f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ad330 tx=0x7f4ef00e50e0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 shutdown_connections 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5643fe7231c0 0x5643febbddb0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5643fe78f1b0 0x5643febb3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643fe76e4d0 0x5643feb544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 --2- 172.21.15.27:0/3749295762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fe7662f0 0x5643feb53f00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4edf400640 0 client.28372 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.361+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 >> 172.21.15.27:0/3749295762 conn(0x5643fe783300 msgr2=0x5643feb30360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.362+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 shutdown_connections 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.362+0000 7f4f0b892740 1 -- 172.21.15.27:0/3749295762 wait complete. 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.362+0000 7f4f0b892740 20 client.28372 trim_cache size 0 max 16384 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:23:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:23:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.362+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ee40b65c0 msgr2=0x7f4ee40b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.362+0000 7f4f03200640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ee40b65c0 0x7f4ee40b8ad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006010 tx=0x7f4ef8005d80 comp rx=0 tx=0).stop 2024-09-21T07:23:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.362+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 msgr2=0x7f4efc102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.362+0000 7f4f03200640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 0x7f4efc102470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000e9f0 tx=0x7f4ef000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 shutdown_connections 2024-09-21T07:23:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4ee40b65c0 0x7f4ee40b8ad0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4efc10dba0 0x7f4efc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4efc105090 0x7f4efc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 --2- 172.21.15.27:0/3605639427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4efc104740 0x7f4efc102470 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 >> 172.21.15.27:0/3605639427 conn(0x7f4efc0fe230 msgr2=0x7f4efc107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 shutdown_connections 2024-09-21T07:23:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:08.363+0000 7f4f03200640 1 -- 172.21.15.27:0/3605639427 wait complete. 2024-09-21T07:23:08.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.915+0000 7f31a8400640 1 -- 172.21.15.27:0/4223129767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a010c010 msgr2=0x7f31a0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.915+0000 7f31a8400640 1 --2- 172.21.15.27:0/4223129767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a010c010 0x7f31a0112840 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3198009ca0 tx=0x7f319802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.916+0000 7f31a8400640 1 -- 172.21.15.27:0/4223129767 shutdown_connections 2024-09-21T07:23:08.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.916+0000 7f31a8400640 1 --2- 172.21.15.27:0/4223129767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a010b6c0 0x7f31a010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.916+0000 7f31a8400640 1 --2- 172.21.15.27:0/4223129767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a010c010 0x7f31a0112840 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.916+0000 7f31a8400640 1 --2- 172.21.15.27:0/4223129767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a0105d80 0x7f31a0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.916+0000 7f31a8400640 1 -- 172.21.15.27:0/4223129767 >> 172.21.15.27:0/4223129767 conn(0x7f31a0101810 msgr2=0x7f31a0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.916+0000 7f31a8400640 1 -- 172.21.15.27:0/4223129767 shutdown_connections 2024-09-21T07:23:08.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.916+0000 7f31a8400640 1 -- 172.21.15.27:0/4223129767 wait complete. 2024-09-21T07:23:08.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 Processor -- start 2024-09-21T07:23:08.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 -- start start 2024-09-21T07:23:08.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a0105d80 0x7f31a019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a010b6c0 0x7f31a019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a010c010 0x7f31a01a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0115cd0 con 0x7f31a0105d80 2024-09-21T07:23:08.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0115b50 con 0x7f31a010b6c0 2024-09-21T07:23:08.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.917+0000 7f31a8400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0115e50 con 0x7f31a010c010 2024-09-21T07:23:08.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a010b6c0 0x7f31a019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a0105d80 0x7f31a019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a010b6c0 0x7f31a019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59494/0 (socket says 172.21.15.27:59494) 2024-09-21T07:23:08.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5000640 1 -- 172.21.15.27:0/2524980021 learned_addr learned my addr 172.21.15.27:0/2524980021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a6400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a010c010 0x7f31a01a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5a00640 1 -- 172.21.15.27:0/2524980021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a010c010 msgr2=0x7f31a01a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a010c010 0x7f31a01a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5a00640 1 -- 172.21.15.27:0/2524980021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a010b6c0 msgr2=0x7f31a019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a010b6c0 0x7f31a019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.918+0000 7f31a5a00640 1 -- 172.21.15.27:0/2524980021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a01a2690 con 0x7f31a0105d80 2024-09-21T07:23:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.919+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a0105d80 0x7f31a019c720 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f319400e9f0 tx=0x7f319400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.919+0000 7f319e000640 1 -- 172.21.15.27:0/2524980021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319400ccc0 con 0x7f31a0105d80 2024-09-21T07:23:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.919+0000 7f319e000640 1 -- 172.21.15.27:0/2524980021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31940108e0 con 0x7f31a0105d80 2024-09-21T07:23:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.919+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a01a2980 con 0x7f31a0105d80 2024-09-21T07:23:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.919+0000 7f319e000640 1 -- 172.21.15.27:0/2524980021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194018880 con 0x7f31a0105d80 2024-09-21T07:23:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.919+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a01a90e0 con 0x7f31a0105d80 2024-09-21T07:23:08.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.922+0000 7f31af098740 1 Processor -- start 2024-09-21T07:23:08.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.922+0000 7f31af098740 1 -- start start 2024-09-21T07:23:08.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31af098740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da35070 0x56050dddbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f319e000640 1 -- 172.21.15.27:0/2524980021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3194010430 con 0x7f31a0105d80 2024-09-21T07:23:08.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31af098740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da0c1b0 0x56050dddc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31af098740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dddc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31af098740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56050dd52f70 con 0x56050da14390 2024-09-21T07:23:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31af098740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56050dd423a0 con 0x56050da0c1b0 2024-09-21T07:23:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31af098740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56050dd461b0 con 0x56050da35070 2024-09-21T07:23:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da0c1b0 0x56050dddc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da0c1b0 0x56050dddc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59496/0 (socket says 172.21.15.27:59496) 2024-09-21T07:23:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31a5a00640 1 -- 172.21.15.27:0/1781784621 learned_addr learned my addr 172.21.15.27:0/1781784621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31a5000640 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dddc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.923+0000 7f31a6400640 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da35070 0x56050dddbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.924+0000 7f31a5a00640 1 -- 172.21.15.27:0/1781784621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da35070 msgr2=0x56050dddbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.924+0000 7f31a5a00640 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da35070 0x56050dddbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.924+0000 7f31a5a00640 1 -- 172.21.15.27:0/1781784621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 msgr2=0x56050dddc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.924+0000 7f31a5a00640 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dddc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.924+0000 7f31a5a00640 1 -- 172.21.15.27:0/1781784621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050dd0cc90 con 0x56050da0c1b0 2024-09-21T07:23:08.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.924+0000 7f31a6400640 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da35070 0x56050dddbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:08.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.924+0000 7f31a5a00640 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da0c1b0 0x56050dddc2a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f31940e50e0 tx=0x7f31940ebf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcdd36e44b0ca110 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.925+0000 7f319e000640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31880b6680 0x7f31880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.925+0000 7f3192a00640 1 -- 172.21.15.27:0/1781784621 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940fb070 con 0x56050da0c1b0 2024-09-21T07:23:08.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.925+0000 7f31a6400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31880b6680 0x7f31880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.925+0000 7f3192a00640 1 -- 172.21.15.27:0/1781784621 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3194010720 con 0x56050da0c1b0 2024-09-21T07:23:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.925+0000 7f3192a00640 1 -- 172.21.15.27:0/1781784621 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940ad720 con 0x56050da0c1b0 2024-09-21T07:23:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.925+0000 7f319e000640 1 -- 172.21.15.27:0/2524980021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f31940e7eb0 con 0x7f31a0105d80 2024-09-21T07:23:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 -- 172.21.15.27:0/1781784621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da0c1b0 msgr2=0x56050dddc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da0c1b0 0x56050dddc2a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f31940e50e0 tx=0x7f31940ebf90 comp rx=0 tx=0).stop 2024-09-21T07:23:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31a6400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31880b6680 0x7f31880b8b90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f319802f500 tx=0x7f319802f490 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 -- 172.21.15.27:0/1781784621 shutdown_connections 2024-09-21T07:23:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dddc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da0c1b0 0x56050dddc2a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 --2- 172.21.15.27:0/1781784621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da35070 0x56050dddbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 -- 172.21.15.27:0/1781784621 >> 172.21.15.27:0/1781784621 conn(0x56050da29460 msgr2=0x56050ddd9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 -- 172.21.15.27:0/1781784621 shutdown_connections 2024-09-21T07:23:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.926+0000 7f31af098740 1 -- 172.21.15.27:0/1781784621 wait complete. 2024-09-21T07:23:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.927+0000 7f31af098740 1 -- start start 2024-09-21T07:23:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.927+0000 7f31af098740 1 Processor -- start 2024-09-21T07:23:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31af098740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31af098740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dde7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31af098740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddf3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31af098740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56050dba3700 con 0x56050da14390 2024-09-21T07:23:08.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31af098740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56050dcdd540 con 0x56050da35070 2024-09-21T07:23:08.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31af098740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56050ddea540 con 0x56050da0c1b0 2024-09-21T07:23:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31af098740 10 client.0 _ll_register_callbacks cb 0x56050dced430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dde7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dde7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49794/0 (socket says 172.21.15.27:49794) 2024-09-21T07:23:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31a5000640 1 -- 172.21.15.27:0/2581598317 learned_addr learned my addr 172.21.15.27:0/2581598317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.928+0000 7f31a6400640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddf3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31a5000640 1 -- 172.21.15.27:0/2581598317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 msgr2=0x56050dde7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31a5000640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31a5000640 1 -- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 msgr2=0x56050ddf3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31a5000640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddf3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31a5000640 1 -- 172.21.15.27:0/2581598317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050daa0650 con 0x56050da14390 2024-09-21T07:23:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31a6400640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddf3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31a5000640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dde7a40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f318c012770 tx=0x7f318c012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125a97d9a9e485d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f3187400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f3187400640 1 -- 172.21.15.27:0/2581598317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00f8a0 con 0x56050da14390 2024-09-21T07:23:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f3187400640 1 -- 172.21.15.27:0/2581598317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f318c00fa40 con 0x56050da14390 2024-09-21T07:23:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.929+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56050dcdfb50 con 0x56050da14390 2024-09-21T07:23:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f3187400640 1 -- 172.21.15.27:0/2581598317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f318c00fc40 con 0x56050da14390 2024-09-21T07:23:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 10 client.18393 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56050dcfa910 con 0x56050da14390 2024-09-21T07:23:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f3187400640 1 -- 172.21.15.27:0/2581598317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56050dcdfb50 con 0x56050da14390 2024-09-21T07:23:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f3187400640 1 -- 172.21.15.27:0/2581598317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f318c01c420 con 0x56050da14390 2024-09-21T07:23:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 10 client.18393 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 10 client.18393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 20 client.18393 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9f39f0 0x56050ddf9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 4 client.18393 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.930+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050dd54ec0 con 0x56050d9f39f0 2024-09-21T07:23:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.931+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9f39f0 0x56050ddf9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.931+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9f39f0 0x56050ddf9770 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d165ce8da02384ae server_cookie=1f8dc1e429325b92 in_seq=0 out_seq=0 2024-09-21T07:23:08.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.931+0000 7f3187400640 10 client.18393 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.932+0000 7f3187400640 1 -- 172.21.15.27:0/2581598317 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56050dd54ec0 con 0x56050d9f39f0 2024-09-21T07:23:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.932+0000 7f3187400640 10 client.18393 handle_command_reply: tid=0 2024-09-21T07:23:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.932+0000 7f31af098740 1 client.18393 shutdown 2024-09-21T07:23:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 msgr2=0x56050dde7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dde7a40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f318c012770 tx=0x7f318c012c30 comp rx=0 tx=0).stop 2024-09-21T07:23:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 shutdown_connections 2024-09-21T07:23:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9f39f0 0x56050ddf9770 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddf3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da14390 0x56050dde7a40 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 --2- 172.21.15.27:0/2581598317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 >> 172.21.15.27:0/2581598317 conn(0x56050da29460 msgr2=0x56050dddeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f3187400640 0 client.18393 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 shutdown_connections 2024-09-21T07:23:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 1 -- 172.21.15.27:0/2581598317 wait complete. 2024-09-21T07:23:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.933+0000 7f31af098740 20 client.18393 trim_cache size 0 max 16384 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 Processor -- start 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 -- start start 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da14390 0x56050ddf3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da35070 0x56050ddf3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01b5bd0 con 0x56050da35070 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01b34a0 con 0x56050da14390 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.952+0000 7f31af098740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01b1310 con 0x56050da0c1b0 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a6400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da35070 0x56050ddf3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46948/0 (socket says 172.21.15.27:46948) 2024-09-21T07:23:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 -- 172.21.15.27:0/2845258101 learned_addr learned my addr 172.21.15.27:0/2845258101 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5000640 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da14390 0x56050ddf3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 -- 172.21.15.27:0/2845258101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da14390 msgr2=0x56050ddf3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da14390 0x56050ddf3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 -- 172.21.15.27:0/2845258101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da35070 msgr2=0x56050ddf3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da35070 0x56050ddf3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 -- 172.21.15.27:0/2845258101 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050dde87c0 con 0x56050da0c1b0 2024-09-21T07:23:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.953+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde79c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31940f0a00 tx=0x7f3194009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e579b31b9bd895f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.954+0000 7f3192000640 1 -- 172.21.15.27:0/2845258101 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940ad0c0 con 0x56050da0c1b0 2024-09-21T07:23:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.954+0000 7f3192000640 1 -- 172.21.15.27:0/2845258101 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31940ad460 con 0x56050da0c1b0 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.954+0000 7f3192000640 1 -- 172.21.15.27:0/2845258101 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940ad2a0 con 0x56050da0c1b0 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 -- 172.21.15.27:0/2845258101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 msgr2=0x56050dde79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde79c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31940f0a00 tx=0x7f3194009700 comp rx=0 tx=0).stop 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 -- 172.21.15.27:0/2845258101 shutdown_connections 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da35070 0x56050ddf3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da14390 0x56050ddf3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 --2- 172.21.15.27:0/2845258101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da0c1b0 0x56050dde79c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 -- 172.21.15.27:0/2845258101 >> 172.21.15.27:0/2845258101 conn(0x56050da29460 msgr2=0x56050dde72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 -- 172.21.15.27:0/2845258101 shutdown_connections 2024-09-21T07:23:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 -- 172.21.15.27:0/2845258101 wait complete. 2024-09-21T07:23:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 -- start start 2024-09-21T07:23:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.955+0000 7f31af098740 1 Processor -- start 2024-09-21T07:23:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31af098740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da0c1b0 0x56050ddf0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31af098740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da14390 0x56050ddeb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31af098740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddebd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31af098740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01b1960 con 0x56050da0c1b0 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31af098740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01b2dd0 con 0x56050da35070 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31af098740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01b2030 con 0x56050da14390 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31af098740 10 client.0 _ll_register_callbacks cb 0x7f318c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da0c1b0 0x56050ddf0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da0c1b0 0x56050ddf0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49800/0 (socket says 172.21.15.27:49800) 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31a5a00640 1 -- 172.21.15.27:0/2864582431 learned_addr learned my addr 172.21.15.27:0/2864582431 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31a6400640 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.956+0000 7f31a5000640 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da14390 0x56050ddeb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f31a5a00640 1 -- 172.21.15.27:0/2864582431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da14390 msgr2=0x56050ddeb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da14390 0x56050ddeb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f31a5a00640 1 -- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 msgr2=0x56050ddebd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddebd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f31a5a00640 1 -- 172.21.15.27:0/2864582431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050de63120 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f31a5a00640 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da0c1b0 0x56050ddf0180 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f31940f9040 tx=0x7f31940e5280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa2f4e2b66c6699 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f3187400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f3187400640 1 -- 172.21.15.27:0/2864582431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940ad2a0 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f3187400640 1 -- 172.21.15.27:0/2864582431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f3194010a90 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.957+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31940ad460 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f3187400640 1 -- 172.21.15.27:0/2864582431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f31940a9fa0 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 10 client.18399 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56050de64250 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f3187400640 1 -- 172.21.15.27:0/2864582431 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31940ad460 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f3187400640 1 -- 172.21.15.27:0/2864582431 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f31940e5420 con 0x56050da0c1b0 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 10 client.18399 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 20 client.18399 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9c8da0 0x56050de66560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 4 client.18399 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.958+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56050dd50970 con 0x56050d9c8da0 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.959+0000 7f31a5000640 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9c8da0 0x56050de66560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.959+0000 7f31a5000640 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9c8da0 0x56050de66560 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bfc80d6cceba182 server_cookie=9d3bb9c8035778a3 in_seq=0 out_seq=0 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.959+0000 7f3187400640 10 client.18399 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.959+0000 7f3187400640 1 -- 172.21.15.27:0/2864582431 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x56050dd50970 con 0x56050d9c8da0 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.959+0000 7f3187400640 10 client.18399 handle_command_reply: tid=0 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 client.18399 shutdown 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da0c1b0 msgr2=0x56050ddf0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da0c1b0 0x56050ddf0180 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f31940f9040 tx=0x7f31940e5280 comp rx=0 tx=0).stop 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 shutdown_connections 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56050d9c8da0 0x56050de66560 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56050da14390 0x56050ddeb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56050da35070 0x56050ddebd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 --2- 172.21.15.27:0/2864582431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56050da0c1b0 0x56050ddf0180 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f3187400640 0 client.18399 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 >> 172.21.15.27:0/2864582431 conn(0x56050da29460 msgr2=0x56050dddcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 shutdown_connections 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 1 -- 172.21.15.27:0/2864582431 wait complete. 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.960+0000 7f31af098740 20 client.18399 trim_cache size 0 max 16384 2024-09-21T07:23:08.966 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:23:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:08.976 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:23:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31880b6680 msgr2=0x7f31880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31880b6680 0x7f31880b8b90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f319802f500 tx=0x7f319802f490 comp rx=0 tx=0).stop 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a0105d80 msgr2=0x7f31a019c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a0105d80 0x7f31a019c720 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f319400e9f0 tx=0x7f319400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 shutdown_connections 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a010c010 0x7f31a01a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f31880b6680 0x7f31880b8b90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a010b6c0 0x7f31a019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 --2- 172.21.15.27:0/2524980021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a0105d80 0x7f31a019c720 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 >> 172.21.15.27:0/2524980021 conn(0x7f31a0101810 msgr2=0x7f31a0109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 shutdown_connections 2024-09-21T07:23:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:08.961+0000 7f31a8400640 1 -- 172.21.15.27:0/2524980021 wait complete. 2024-09-21T07:23:08.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:08 smithi027 ceph-mon[30779]: pgmap v689: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T07:23:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:08 smithi153 ceph-mon[34747]: pgmap v689: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T07:23:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:08 smithi119 ceph-mon[34519]: pgmap v689: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-09-21T07:23:09.764 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:23:09.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:23:10.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:09 smithi027 ceph-mon[30779]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T07:23:10.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:23:10.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:09 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:10.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:09 smithi153 ceph-mon[34747]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T07:23:10.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:23:10.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:09 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:09 smithi119 ceph-mon[34519]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T07:23:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:23:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:09 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.835+0000 7f6d35000640 1 -- 172.21.15.27:0/102498789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 msgr2=0x7f6d30110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.835+0000 7f6d35000640 1 --2- 172.21.15.27:0/102498789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 0x7f6d30110470 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009ca0 tx=0x7f6d2802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:23:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.836+0000 7f6d35000640 1 -- 172.21.15.27:0/102498789 shutdown_connections 2024-09-21T07:23:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.836+0000 7f6d35000640 1 --2- 172.21.15.27:0/102498789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 0x7f6d30110470 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.836+0000 7f6d35000640 1 --2- 172.21.15.27:0/102498789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d301092f0 0x7f6d30109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.836+0000 7f6d35000640 1 --2- 172.21.15.27:0/102498789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d301039b0 0x7f6d30103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.836+0000 7f6d35000640 1 -- 172.21.15.27:0/102498789 >> 172.21.15.27:0/102498789 conn(0x7f6d300ff440 msgr2=0x7f6d30101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.836+0000 7f6d35000640 1 -- 172.21.15.27:0/102498789 shutdown_connections 2024-09-21T07:23:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.836+0000 7f6d35000640 1 -- 172.21.15.27:0/102498789 wait complete. 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.837+0000 7f6d35000640 1 Processor -- start 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.837+0000 7f6d35000640 1 -- start start 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.837+0000 7f6d35000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d301039b0 0x7f6d3019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.837+0000 7f6d35000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d301092f0 0x7f6d3019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d35000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 0x7f6d3019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d35000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30113900 con 0x7f6d301092f0 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d35000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30113780 con 0x7f6d301039b0 2024-09-21T07:23:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d35000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30113a80 con 0x7f6d30109c40 2024-09-21T07:23:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d301092f0 0x7f6d3019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d301039b0 0x7f6d3019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 0x7f6d3019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d301092f0 0x7f6d3019a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49824/0 (socket says 172.21.15.27:49824) 2024-09-21T07:23:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2ea00640 1 -- 172.21.15.27:0/3573577163 learned_addr learned my addr 172.21.15.27:0/3573577163 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2f400640 1 -- 172.21.15.27:0/3573577163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 msgr2=0x7f6d3019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2f400640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 0x7f6d3019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2f400640 1 -- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d301092f0 msgr2=0x7f6d3019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2f400640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d301092f0 0x7f6d3019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.838+0000 7f6d2f400640 1 -- 172.21.15.27:0/3573577163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d301a0200 con 0x7f6d301039b0 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.839+0000 7f6d2f400640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d301039b0 0x7f6d3019a290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009bc0 tx=0x7f6d2000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.839+0000 7f6d27e00640 1 -- 172.21.15.27:0/3573577163 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2001e070 con 0x7f6d301039b0 2024-09-21T07:23:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.839+0000 7f6d27e00640 1 -- 172.21.15.27:0/3573577163 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d200198c0 con 0x7f6d301039b0 2024-09-21T07:23:10.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.839+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d301a04f0 con 0x7f6d301039b0 2024-09-21T07:23:10.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.839+0000 7f6d27e00640 1 -- 172.21.15.27:0/3573577163 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200187d0 con 0x7f6d301039b0 2024-09-21T07:23:10.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.839+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d300755f0 con 0x7f6d301039b0 2024-09-21T07:23:10.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.840+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d30103dc0 con 0x7f6d301039b0 2024-09-21T07:23:10.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.843+0000 7f6d27e00640 1 -- 172.21.15.27:0/3573577163 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6d20021070 con 0x7f6d301039b0 2024-09-21T07:23:10.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.844+0000 7f6d27e00640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6d140b6680 0x7f6d140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:10.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.844+0000 7f6d27e00640 1 -- 172.21.15.27:0/3573577163 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f6d200eb7c0 con 0x7f6d301039b0 2024-09-21T07:23:10.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.844+0000 7f6d2ea00640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6d140b6680 0x7f6d140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:10.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.845+0000 7f6d2ea00640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6d140b6680 0x7f6d140b8b90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6d30102d20 tx=0x7f6d1c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:10.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:10.846+0000 7f6d27e00640 1 -- 172.21.15.27:0/3573577163 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6d200b5c90 con 0x7f6d301039b0 2024-09-21T07:23:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.085+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d3019b3c0 con 0x7f6d301039b0 2024-09-21T07:23:11.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:10 smithi027 ceph-mon[30779]: pgmap v691: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-21T07:23:11.088 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:23:11.088 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:23:11.088 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.086+0000 7f6d27e00640 1 -- 172.21.15.27:0/3573577163 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f6d200babd0 con 0x7f6d301039b0 2024-09-21T07:23:11.089 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:23:11.089 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:23:11.089 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:23:11.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6d140b6680 msgr2=0x7f6d140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6d140b6680 0x7f6d140b8b90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6d30102d20 tx=0x7f6d1c005f90 comp rx=0 tx=0).stop 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d301039b0 msgr2=0x7f6d3019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d301039b0 0x7f6d3019a290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009bc0 tx=0x7f6d2000b690 comp rx=0 tx=0).stop 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 shutdown_connections 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d30109c40 0x7f6d3019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6d140b6680 0x7f6d140b8b90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d301092f0 0x7f6d3019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.089+0000 7f6d35000640 1 --2- 172.21.15.27:0/3573577163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d301039b0 0x7f6d3019a290 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.090+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 >> 172.21.15.27:0/3573577163 conn(0x7f6d300ff440 msgr2=0x7f6d301089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.090+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 shutdown_connections 2024-09-21T07:23:11.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:11.090+0000 7f6d35000640 1 -- 172.21.15.27:0/3573577163 wait complete. 2024-09-21T07:23:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:10 smithi153 ceph-mon[34747]: pgmap v691: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-21T07:23:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:23:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:23:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:10 smithi119 ceph-mon[34519]: pgmap v691: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-21T07:23:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:23:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:23:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:23:11.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:23:12.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:11 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3573577163' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:12.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:11 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3573577163' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:12.253 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:11 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3573577163' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:12.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:23:12.888 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 -- 172.21.15.27:0/2915880102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0103b80 msgr2=0x7f87a0103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 --2- 172.21.15.27:0/2915880102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0103b80 0x7f87a0103f90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8794009900 tx=0x7f879402a410 comp rx=0 tx=0).stop 2024-09-21T07:23:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 -- 172.21.15.27:0/2915880102 shutdown_connections 2024-09-21T07:23:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 --2- 172.21.15.27:0/2915880102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87a0109e10 0x7f87a0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 --2- 172.21.15.27:0/2915880102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 0x7f87a01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 --2- 172.21.15.27:0/2915880102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0103b80 0x7f87a0103f90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 -- 172.21.15.27:0/2915880102 >> 172.21.15.27:0/2915880102 conn(0x7f87a00ff590 msgr2=0x7f87a01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.893+0000 7f87a8000640 1 -- 172.21.15.27:0/2915880102 shutdown_connections 2024-09-21T07:23:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.894+0000 7f87a8000640 1 -- 172.21.15.27:0/2915880102 wait complete. 2024-09-21T07:23:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.895+0000 7f87a8000640 1 Processor -- start 2024-09-21T07:23:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.895+0000 7f87a8000640 1 -- start start 2024-09-21T07:23:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a8000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87a0103b80 0x7f87a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a8000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 0x7f87a0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a8000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0109e10 0x7f87a0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a8000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0113b20 con 0x7f87a0109e10 2024-09-21T07:23:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a8000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01139a0 con 0x7f87a01094c0 2024-09-21T07:23:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a8000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0113ca0 con 0x7f87a0103b80 2024-09-21T07:23:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 0x7f87a0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 0x7f87a0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59536/0 (socket says 172.21.15.27:59536) 2024-09-21T07:23:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a4c00640 1 -- 172.21.15.27:0/805997520 learned_addr learned my addr 172.21.15.27:0/805997520 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a5600640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87a0103b80 0x7f87a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.896+0000 7f87a6000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0109e10 0x7f87a0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a4c00640 1 -- 172.21.15.27:0/805997520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87a0103b80 msgr2=0x7f87a0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a4c00640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87a0103b80 0x7f87a0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a4c00640 1 -- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0109e10 msgr2=0x7f87a0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a4c00640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0109e10 0x7f87a0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a4c00640 1 -- 172.21.15.27:0/805997520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a01a87b0 con 0x7f87a01094c0 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a6000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0109e10 0x7f87a0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a4c00640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 0x7f87a0075c50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e9f0 tx=0x7f878c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f879e000640 1 -- 172.21.15.27:0/805997520 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c00ccc0 con 0x7f87a01094c0 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f879e000640 1 -- 172.21.15.27:0/805997520 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c0108e0 con 0x7f87a01094c0 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f879e000640 1 -- 172.21.15.27:0/805997520 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c018860 con 0x7f87a01094c0 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.897+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a01a8aa0 con 0x7f87a01094c0 2024-09-21T07:23:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.898+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01a9000 con 0x7f87a01094c0 2024-09-21T07:23:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.901+0000 7f87aec50740 1 Processor -- start 2024-09-21T07:23:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f87aec50740 1 -- start start 2024-09-21T07:23:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f879e000640 1 -- 172.21.15.27:0/805997520 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f878c010430 con 0x7f87a01094c0 2024-09-21T07:23:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f87aec50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0683ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f87aec50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f87aec50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0684620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f87aec50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5572b05fafb0 con 0x5572b02dd070 2024-09-21T07:23:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f87aec50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5572b05ea3e0 con 0x5572b02b41b0 2024-09-21T07:23:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.902+0000 7f87aec50740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5572b05ee1f0 con 0x5572b02bc390 2024-09-21T07:23:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0683ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0684620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0684620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46990/0 (socket says 172.21.15.27:46990) 2024-09-21T07:23:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 -- 172.21.15.27:0/2685010067 learned_addr learned my addr 172.21.15.27:0/2685010067 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0683ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49836/0 (socket says 172.21.15.27:49836) 2024-09-21T07:23:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 -- 172.21.15.27:0/2685010067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 msgr2=0x5572b06840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 -- 172.21.15.27:0/2685010067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 msgr2=0x5572b0683ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0683ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a4c00640 1 -- 172.21.15.27:0/2685010067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b05b4cd0 con 0x5572b02bc390 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.903+0000 7f87a5600640 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.904+0000 7f87a4c00640 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0684620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f878c0e50b0 tx=0x7f878c0f0570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c7f2cc670d3ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.904+0000 7f8792600640 1 -- 172.21.15.27:0/2685010067 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0fb070 con 0x5572b02bc390 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.904+0000 7f8792600640 1 -- 172.21.15.27:0/2685010067 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c0f6ab0 con 0x5572b02bc390 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.904+0000 7f8792600640 1 -- 172.21.15.27:0/2685010067 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0f5a00 con 0x5572b02bc390 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.904+0000 7f879e000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87880b6700 0x7f87880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.905+0000 7f879e000640 1 -- 172.21.15.27:0/805997520 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f878c0e7df0 con 0x7f87a01094c0 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.905+0000 7f87a6000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87880b6700 0x7f87880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.905+0000 7f87aec50740 1 -- 172.21.15.27:0/2685010067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 msgr2=0x5572b0684620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.905+0000 7f87aec50740 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0684620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f878c0e50b0 tx=0x7f878c0f0570 comp rx=0 tx=0).stop 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87a6000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87880b6700 0x7f87880b8c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8798005fd0 tx=0x7f8798009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87aec50740 1 -- 172.21.15.27:0/2685010067 shutdown_connections 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87aec50740 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0684620 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87aec50740 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87aec50740 1 --2- 172.21.15.27:0/2685010067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0683ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87aec50740 1 -- 172.21.15.27:0/2685010067 >> 172.21.15.27:0/2685010067 conn(0x5572b02d1460 msgr2=0x5572b0681a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:12.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87aec50740 1 -- 172.21.15.27:0/2685010067 shutdown_connections 2024-09-21T07:23:12.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.906+0000 7f87aec50740 1 -- 172.21.15.27:0/2685010067 wait complete. 2024-09-21T07:23:12.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 -- start start 2024-09-21T07:23:12.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 Processor -- start 2024-09-21T07:23:12.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02b41b0 0x5572b068f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b068fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02dd070 0x5572b069b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5572b044b700 con 0x5572b02b41b0 2024-09-21T07:23:12.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5572b0585580 con 0x5572b02dd070 2024-09-21T07:23:12.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5572b0692380 con 0x5572b02bc390 2024-09-21T07:23:12.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.907+0000 7f87aec50740 10 client.0 _ll_register_callbacks cb 0x5572b0595470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02b41b0 0x5572b068f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02dd070 0x5572b069b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02b41b0 0x5572b068f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49846/0 (socket says 172.21.15.27:49846) 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02dd070 0x5572b069b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59542/0 (socket says 172.21.15.27:59542) 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a5600640 1 -- 172.21.15.27:0/4206950055 learned_addr learned my addr 172.21.15.27:0/4206950055 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a4c00640 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b068fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a4c00640 1 -- 172.21.15.27:0/4206950055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02dd070 msgr2=0x5572b069b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a4c00640 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02dd070 0x5572b069b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a4c00640 1 -- 172.21.15.27:0/4206950055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02b41b0 msgr2=0x5572b068f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a4c00640 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02b41b0 0x5572b068f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a4c00640 1 -- 172.21.15.27:0/4206950055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b0348650 con 0x5572b02bc390 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.908+0000 7f87a5600640 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02b41b0 0x5572b068f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f87a4c00640 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b068fd70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c6a0 tx=0x7f878c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22dea2b4019dcb19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f8786a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f8786a00640 1 -- 172.21.15.27:0/4206950055 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0a9e90 con 0x5572b02bc390 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572b0587b90 con 0x5572b02bc390 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f8786a00640 1 -- 172.21.15.27:0/4206950055 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f878c0b6dd0 con 0x5572b02bc390 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f8786a00640 1 -- 172.21.15.27:0/4206950055 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f878c0acfb0 con 0x5572b02bc390 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f87aec50740 10 client.28408 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:12.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572b05a2950 con 0x5572b02bc390 2024-09-21T07:23:12.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.909+0000 7f8786a00640 1 -- 172.21.15.27:0/4206950055 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5572b0587b90 con 0x5572b02bc390 2024-09-21T07:23:12.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f8786a00640 1 -- 172.21.15.27:0/4206950055 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f878c0aeb70 con 0x5572b02bc390 2024-09-21T07:23:12.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f87aec50740 10 client.28408 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:12.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f87aec50740 10 client.28408 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:23:12.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f87aec50740 20 client.28408 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:12.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f87aec50740 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b029b9f0 0x5572b06a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f87aec50740 4 client.28408 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:12.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b05fcf00 con 0x5572b029b9f0 2024-09-21T07:23:12.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.910+0000 7f87a5600640 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b029b9f0 0x5572b06a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.911+0000 7f87a5600640 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b029b9f0 0x5572b06a15a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5c439369402c89a server_cookie=ab464d267bd28fb in_seq=0 out_seq=0 2024-09-21T07:23:12.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.911+0000 7f8786a00640 10 client.28408 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f8786a00640 1 -- 172.21.15.27:0/4206950055 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572b05fcf00 con 0x5572b029b9f0 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f8786a00640 10 client.28408 handle_command_reply: tid=0 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 client.28408 shutdown 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 msgr2=0x5572b068fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b068fd70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c6a0 tx=0x7f878c01f000 comp rx=0 tx=0).stop 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 shutdown_connections 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b029b9f0 0x5572b06a15a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b068fd70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02dd070 0x5572b069b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 --2- 172.21.15.27:0/4206950055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02b41b0 0x5572b068f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f8786a00640 0 client.28408 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:23:12.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 >> 172.21.15.27:0/4206950055 conn(0x5572b02d1460 msgr2=0x5572b0686d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:12.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 shutdown_connections 2024-09-21T07:23:12.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.912+0000 7f87aec50740 1 -- 172.21.15.27:0/4206950055 wait complete. 2024-09-21T07:23:12.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.913+0000 7f87aec50740 20 client.28408 trim_cache size 0 max 16384 2024-09-21T07:23:12.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.927+0000 7f87aec50740 1 Processor -- start 2024-09-21T07:23:12.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.927+0000 7f87aec50740 1 -- start start 2024-09-21T07:23:12.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87aec50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87aec50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0681040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87aec50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0681580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87aec50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b5b80 con 0x5572b02dd070 2024-09-21T07:23:12.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87aec50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b3450 con 0x5572b02b41b0 2024-09-21T07:23:12.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87aec50740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b12c0 con 0x5572b02bc390 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0681580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87a4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0681040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0681580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49860/0 (socket says 172.21.15.27:49860) 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87a6000640 1 -- 172.21.15.27:0/1874503587 learned_addr learned my addr 172.21.15.27:0/1874503587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.928+0000 7f87a5600640 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f87a4c00640 1 -- 172.21.15.27:0/1874503587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 msgr2=0x5572b06893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f87a4c00640 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f87a4c00640 1 -- 172.21.15.27:0/1874503587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 msgr2=0x5572b0681580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f87a4c00640 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0681580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f87a4c00640 1 -- 172.21.15.27:0/1874503587 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b06a7890 con 0x5572b02bc390 2024-09-21T07:23:12.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f87a5600640 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:12.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f87a4c00640 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0681040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f878c0aef10 tx=0x7f878c00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85dd1014d15ffd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:12.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.929+0000 7f8791c00640 1 -- 172.21.15.27:0/1874503587 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0ad190 con 0x5572b02bc390 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.930+0000 7f8791c00640 1 -- 172.21.15.27:0/1874503587 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c0f0a70 con 0x5572b02bc390 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.930+0000 7f8791c00640 1 -- 172.21.15.27:0/1874503587 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0ad610 con 0x5572b02bc390 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 -- 172.21.15.27:0/1874503587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 msgr2=0x5572b0681040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0681040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f878c0aef10 tx=0x7f878c00cef0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 -- 172.21.15.27:0/1874503587 shutdown_connections 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02bc390 0x5572b0681040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02dd070 0x5572b0681580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 --2- 172.21.15.27:0/1874503587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 -- 172.21.15.27:0/1874503587 >> 172.21.15.27:0/1874503587 conn(0x5572b02d1460 msgr2=0x5572b0688cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 -- 172.21.15.27:0/1874503587 shutdown_connections 2024-09-21T07:23:12.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 -- 172.21.15.27:0/1874503587 wait complete. 2024-09-21T07:23:12.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.931+0000 7f87aec50740 1 -- start start 2024-09-21T07:23:12.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.932+0000 7f87aec50740 1 Processor -- start 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.932+0000 7f87aec50740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87aec50740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02bc390 0x5572b06a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87aec50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02dd070 0x5572b06a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87aec50740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b1910 con 0x5572b02bc390 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87aec50740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b2d80 con 0x5572b02b41b0 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87aec50740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b1fe0 con 0x5572b02dd070 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87aec50740 10 client.0 _ll_register_callbacks cb 0x7f87980093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:12.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87a5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02bc390 0x5572b06a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87a5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59560/0 (socket says 172.21.15.27:59560) 2024-09-21T07:23:12.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87a5600640 1 -- 172.21.15.27:0/1072070991 learned_addr learned my addr 172.21.15.27:0/1072070991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:12.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87a6000640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02dd070 0x5572b06a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.933+0000 7f87a5600640 1 -- 172.21.15.27:0/1072070991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02dd070 msgr2=0x5572b06a2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f87a5600640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02dd070 0x5572b06a2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f87a5600640 1 -- 172.21.15.27:0/1072070991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02bc390 msgr2=0x5572b06a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:12.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f87a5600640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02bc390 0x5572b06a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f87a5600640 1 -- 172.21.15.27:0/1072070991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b070ad20 con 0x5572b02b41b0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f87a4c00640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02bc390 0x5572b06a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f87a6000640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02dd070 0x5572b06a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f87a5600640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06a2550 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f879402a930 tx=0x7f8794004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5faef90eb2081415 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.934+0000 7f8786a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.935+0000 7f8786a00640 1 -- 172.21.15.27:0/1072070991 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794035040 con 0x5572b02b41b0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.935+0000 7f8786a00640 1 -- 172.21.15.27:0/1072070991 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f8794005510 con 0x5572b02b41b0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.935+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878c0f0a70 con 0x5572b02b41b0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.935+0000 7f8786a00640 1 -- 172.21.15.27:0/1072070991 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8794004770 con 0x5572b02b41b0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.935+0000 7f87aec50740 10 client.28457 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.935+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572b070bf90 con 0x5572b02b41b0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.935+0000 7f8786a00640 1 -- 172.21.15.27:0/1072070991 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c0f0a70 con 0x5572b02b41b0 2024-09-21T07:23:12.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.936+0000 7f8786a00640 1 -- 172.21.15.27:0/1072070991 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8794003c10 con 0x5572b02b41b0 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.936+0000 7f87aec50740 10 client.28457 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.936+0000 7f87aec50740 10 client.28457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.936+0000 7f87aec50740 20 client.28457 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.936+0000 7f87aec50740 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b0270da0 0x5572b070e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.936+0000 7f87aec50740 4 client.28457 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.936+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572b05f89b0 con 0x5572b0270da0 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.937+0000 7f87a4c00640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b0270da0 0x5572b070e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.937+0000 7f87a4c00640 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b0270da0 0x5572b070e780 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4dd71619107c84ab server_cookie=c192e2cf9e891587 in_seq=0 out_seq=0 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.937+0000 7f8786a00640 10 client.28457 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:23:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.937+0000 7f8786a00640 1 -- 172.21.15.27:0/1072070991 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572b05f89b0 con 0x5572b0270da0 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.937+0000 7f8786a00640 10 client.28457 handle_command_reply: tid=0 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.937+0000 7f87aec50740 1 client.28457 shutdown 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 msgr2=0x5572b06a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06a2550 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f879402a930 tx=0x7f8794004570 comp rx=0 tx=0).stop 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 shutdown_connections 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5572b0270da0 0x5572b070e780 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572b02dd070 0x5572b06a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5572b02bc390 0x5572b06a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 --2- 172.21.15.27:0/1072070991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572b02b41b0 0x5572b06a2550 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 >> 172.21.15.27:0/1072070991 conn(0x5572b02d1460 msgr2=0x5572b0690a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f8786a00640 0 client.28457 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 shutdown_connections 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 1 -- 172.21.15.27:0/1072070991 wait complete. 2024-09-21T07:23:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.938+0000 7f87aec50740 20 client.28457 trim_cache size 0 max 16384 2024-09-21T07:23:12.942 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87880b6700 msgr2=0x7f87880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87880b6700 0x7f87880b8c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8798005fd0 tx=0x7f8798009500 comp rx=0 tx=0).stop 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 msgr2=0x7f87a0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 0x7f87a0075c50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e9f0 tx=0x7f878c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 shutdown_connections 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f87880b6700 0x7f87880b8c10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a0109e10 0x7f87a0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a01094c0 0x7f87a0075c50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 --2- 172.21.15.27:0/805997520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87a0103b80 0x7f87a0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 >> 172.21.15.27:0/805997520 conn(0x7f87a00ff590 msgr2=0x7f87a0102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 shutdown_connections 2024-09-21T07:23:12.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:12.939+0000 7f87a8000640 1 -- 172.21.15.27:0/805997520 wait complete. 2024-09-21T07:23:12.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:12 smithi027 ceph-mon[30779]: pgmap v692: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-09-21T07:23:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:12 smithi153 ceph-mon[34747]: pgmap v692: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-09-21T07:23:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:12 smithi119 ceph-mon[34519]: pgmap v692: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-09-21T07:23:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 -- 172.21.15.27:0/1140229208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c077940 msgr2=0x7f819c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140229208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c077940 0x7f819c077d50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f818c009900 tx=0x7f818c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:23:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 -- 172.21.15.27:0/1140229208 shutdown_connections 2024-09-21T07:23:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140229208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819c114740 0x7f819c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140229208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c078290 0x7f819c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140229208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c077940 0x7f819c077d50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 -- 172.21.15.27:0/1140229208 >> 172.21.15.27:0/1140229208 conn(0x7f819c101810 msgr2=0x7f819c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 -- 172.21.15.27:0/1140229208 shutdown_connections 2024-09-21T07:23:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.828+0000 7f81a1000640 1 -- 172.21.15.27:0/1140229208 wait complete. 2024-09-21T07:23:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 Processor -- start 2024-09-21T07:23:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 -- start start 2024-09-21T07:23:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c077940 0x7f819c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c078290 0x7f819c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819c114740 0x7f819c1a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f819c11a100 con 0x7f819c078290 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f819c119f80 con 0x7f819c077940 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.830+0000 7f81a1000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f819c11a280 con 0x7f819c114740 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c077940 0x7f819c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8193e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c078290 0x7f819c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c077940 0x7f819c1a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59588/0 (socket says 172.21.15.27:59588) 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 -- 172.21.15.27:0/1140258718 learned_addr learned my addr 172.21.15.27:0/1140258718 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f819a800640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819c114740 0x7f819c1a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:13.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 -- 172.21.15.27:0/1140258718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819c114740 msgr2=0x7f819c1a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:13.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819c114740 0x7f819c1a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:13.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 -- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c078290 msgr2=0x7f819c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:13.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c078290 0x7f819c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:13.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.831+0000 7f8199e00640 1 -- 172.21.15.27:0/1140258718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c1a6a60 con 0x7f819c077940 2024-09-21T07:23:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.832+0000 7f8199e00640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c077940 0x7f819c1a0af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f818c005ab0 tx=0x7f818c003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.832+0000 7f8192a00640 1 -- 172.21.15.27:0/1140258718 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c03f070 con 0x7f819c077940 2024-09-21T07:23:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.832+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c1a6cf0 con 0x7f819c077940 2024-09-21T07:23:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.832+0000 7f8192a00640 1 -- 172.21.15.27:0/1140258718 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818c031d60 con 0x7f819c077940 2024-09-21T07:23:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.832+0000 7f8192a00640 1 -- 172.21.15.27:0/1140258718 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c035030 con 0x7f819c077940 2024-09-21T07:23:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.832+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1ad4b0 con 0x7f819c077940 2024-09-21T07:23:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.833+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c077d50 con 0x7f819c077940 2024-09-21T07:23:13.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.836+0000 7f8192a00640 1 -- 172.21.15.27:0/1140258718 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f818c042070 con 0x7f819c077940 2024-09-21T07:23:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.838+0000 7f8192a00640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f81800b6680 0x7f81800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:13.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.838+0000 7f8192a00640 1 -- 172.21.15.27:0/1140258718 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f818c108590 con 0x7f819c077940 2024-09-21T07:23:13.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.838+0000 7f8193e00640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f81800b6680 0x7f81800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:13.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.839+0000 7f8193e00640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f81800b6680 0x7f81800b8b90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f819c079a90 tx=0x7f8184005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:13.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:13.839+0000 7f8192a00640 1 -- 172.21.15.27:0/1140258718 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f818c0d2a60 con 0x7f819c077940 2024-09-21T07:23:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.080+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f819c10f930 con 0x7f819c077940 2024-09-21T07:23:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.081+0000 7f8192a00640 1 -- 172.21.15.27:0/1140258718 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f818c0d79a0 con 0x7f819c077940 2024-09-21T07:23:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:23:14.084 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:23:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f81800b6680 msgr2=0x7f81800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f81800b6680 0x7f81800b8b90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f819c079a90 tx=0x7f8184005f90 comp rx=0 tx=0).stop 2024-09-21T07:23:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c077940 msgr2=0x7f819c1a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c077940 0x7f819c1a0af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f818c005ab0 tx=0x7f818c003c80 comp rx=0 tx=0).stop 2024-09-21T07:23:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 shutdown_connections 2024-09-21T07:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819c114740 0x7f819c1a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f81800b6680 0x7f81800b8b90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f819c078290 0x7f819c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 --2- 172.21.15.27:0/1140258718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819c077940 0x7f819c1a0af0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 >> 172.21.15.27:0/1140258718 conn(0x7f819c101810 msgr2=0x7f819c116820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 shutdown_connections 2024-09-21T07:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:14.084+0000 7f81a1000640 1 -- 172.21.15.27:0/1140258718 wait complete. 2024-09-21T07:23:14.313 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:23:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:14 smithi027 ceph-mon[30779]: pgmap v693: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:23:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:14 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1140258718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:14 smithi153 ceph-mon[34747]: pgmap v693: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:23:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:14 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1140258718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:15.202 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id fbaed821 --release --await' 2024-09-21T07:23:15.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id fbaed821 --release --await 2024-09-21T07:23:15.229 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:14 smithi119 ceph-mon[34519]: pgmap v693: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:23:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:14 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1140258718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:15.844 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.864+0000 7f42a9200640 1 -- 172.21.15.27:0/3700694144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a4105d90 msgr2=0x7f42a41061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.864+0000 7f42a9200640 1 --2- 172.21.15.27:0/3700694144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a4105d90 0x7f42a41061a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4294009900 tx=0x7f429402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.865+0000 7f42a9200640 1 -- 172.21.15.27:0/3700694144 shutdown_connections 2024-09-21T07:23:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.865+0000 7f42a9200640 1 --2- 172.21.15.27:0/3700694144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42a410c020 0x7f42a4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.865+0000 7f42a9200640 1 --2- 172.21.15.27:0/3700694144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42a410b6d0 0x7f42a410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.865+0000 7f42a9200640 1 --2- 172.21.15.27:0/3700694144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a4105d90 0x7f42a41061a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.865+0000 7f42a9200640 1 -- 172.21.15.27:0/3700694144 >> 172.21.15.27:0/3700694144 conn(0x7f42a4101860 msgr2=0x7f42a4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.865+0000 7f42a9200640 1 -- 172.21.15.27:0/3700694144 shutdown_connections 2024-09-21T07:23:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.865+0000 7f42a9200640 1 -- 172.21.15.27:0/3700694144 wait complete. 2024-09-21T07:23:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 Processor -- start 2024-09-21T07:23:15.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 -- start start 2024-09-21T07:23:15.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42a410b6d0 0x7f42a419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42a410c020 0x7f42a419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a41a1fe0 0x7f42a41a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4115d30 con 0x7f42a41a1fe0 2024-09-21T07:23:15.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4115bb0 con 0x7f42a410c020 2024-09-21T07:23:15.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a9200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4115eb0 con 0x7f42a410b6d0 2024-09-21T07:23:15.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.867+0000 7f42a1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42a410c020 0x7f42a419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42a410b6d0 0x7f42a419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42a410c020 0x7f42a419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59600/0 (socket says 172.21.15.27:59600) 2024-09-21T07:23:15.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1400640 1 -- 172.21.15.27:0/4246756530 learned_addr learned my addr 172.21.15.27:0/4246756530 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:15.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a2800640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a41a1fe0 0x7f42a41a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1e00640 1 -- 172.21.15.27:0/4246756530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42a410c020 msgr2=0x7f42a419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1e00640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42a410c020 0x7f42a419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1e00640 1 -- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a41a1fe0 msgr2=0x7f42a41a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1e00640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a41a1fe0 0x7f42a41a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1e00640 1 -- 172.21.15.27:0/4246756530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a41a2bc0 con 0x7f42a410b6d0 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a2800640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a41a1fe0 0x7f42a41a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.868+0000 7f42a1e00640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42a410b6d0 0x7f42a419c860 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4294008000 tx=0x7f4294003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.869+0000 7f429aa00640 1 -- 172.21.15.27:0/4246756530 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429403f020 con 0x7f42a410b6d0 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.869+0000 7f429aa00640 1 -- 172.21.15.27:0/4246756530 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4294031b50 con 0x7f42a410b6d0 2024-09-21T07:23:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.869+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a41a9160 con 0x7f42a410b6d0 2024-09-21T07:23:15.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.869+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a41a9500 con 0x7f42a410b6d0 2024-09-21T07:23:15.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.869+0000 7f429aa00640 1 -- 172.21.15.27:0/4246756530 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294035070 con 0x7f42a410b6d0 2024-09-21T07:23:15.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.872+0000 7f42afe28740 1 Processor -- start 2024-09-21T07:23:15.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f42afe28740 1 -- start start 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f429aa00640 1 -- 172.21.15.27:0/4246756530 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4294042020 con 0x7f42a410b6d0 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f42afe28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da3017da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f42afe28740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da30182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f42afe28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da3018820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f42afe28740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560da2f8efb0 con 0x560da2c481b0 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f42afe28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560da2f7e3e0 con 0x560da2c71070 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.873+0000 7f42afe28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560da2f821f0 con 0x560da2c50390 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da3017da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da3017da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59614/0 (socket says 172.21.15.27:59614) 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 -- 172.21.15.27:0/2299281141 learned_addr learned my addr 172.21.15.27:0/2299281141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a1400640 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da3018820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a1e00640 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da30182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 -- 172.21.15.27:0/2299281141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 msgr2=0x560da3018820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da3018820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 -- 172.21.15.27:0/2299281141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 msgr2=0x560da30182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da30182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.874+0000 7f42a2800640 1 -- 172.21.15.27:0/2299281141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da2f48cd0 con 0x560da2c71070 2024-09-21T07:23:15.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.875+0000 7f42a1e00640 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da30182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:15.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.875+0000 7f42a2800640 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da3017da0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f429c009fd0 tx=0x7f429c010820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67453ec649252968 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.875+0000 7f428f400640 1 -- 172.21.15.27:0/2299281141 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c011700 con 0x560da2c71070 2024-09-21T07:23:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.875+0000 7f428f400640 1 -- 172.21.15.27:0/2299281141 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f429c011d50 con 0x560da2c71070 2024-09-21T07:23:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.875+0000 7f428f400640 1 -- 172.21.15.27:0/2299281141 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c01bdf0 con 0x560da2c71070 2024-09-21T07:23:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.875+0000 7f429aa00640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f42880b6680 0x7f42880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.875+0000 7f429aa00640 1 -- 172.21.15.27:0/4246756530 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f4294107c40 con 0x7f42a410b6d0 2024-09-21T07:23:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42a1e00640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f42880b6680 0x7f42880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42afe28740 1 -- 172.21.15.27:0/2299281141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 msgr2=0x560da3017da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42afe28740 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da3017da0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f429c009fd0 tx=0x7f429c010820 comp rx=0 tx=0).stop 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42a1e00640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f42880b6680 0x7f42880b8b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x560da3016a20 tx=0x7f4294003f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42afe28740 1 -- 172.21.15.27:0/2299281141 shutdown_connections 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42afe28740 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da3018820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42afe28740 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da30182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42afe28740 1 --2- 172.21.15.27:0/2299281141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da3017da0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.876+0000 7f42afe28740 1 -- 172.21.15.27:0/2299281141 >> 172.21.15.27:0/2299281141 conn(0x560da2c65460 msgr2=0x560da3015c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.877+0000 7f42afe28740 1 -- 172.21.15.27:0/2299281141 shutdown_connections 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.877+0000 7f42afe28740 1 -- 172.21.15.27:0/2299281141 wait complete. 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.877+0000 7f42afe28740 1 -- start start 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 1 Processor -- start 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c481b0 0x560da3023540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c50390 0x560da3023a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da3038110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560da2ddf700 con 0x560da2c50390 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560da2f19580 con 0x560da2c481b0 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560da302ed90 con 0x560da2c71070 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.878+0000 7f42afe28740 10 client.0 _ll_register_callbacks cb 0x560da2f29470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c481b0 0x560da3023540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c481b0 0x560da3023540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59626/0 (socket says 172.21.15.27:59626) 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 -- 172.21.15.27:0/2352466676 learned_addr learned my addr 172.21.15.27:0/2352466676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da3038110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a1400640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c50390 0x560da3023a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 -- 172.21.15.27:0/2352466676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 msgr2=0x560da3038110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da3038110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 -- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c50390 msgr2=0x560da3023a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c50390 0x560da3023a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a2800640 1 -- 172.21.15.27:0/2352466676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da2cdc650 con 0x560da2c481b0 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.879+0000 7f42a1400640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c50390 0x560da3023a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f42a2800640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c481b0 0x560da3023540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f429c01bfc0 tx=0x7f429c01b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d10d75a056bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f4283e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f4283e00640 1 -- 172.21.15.27:0/2352466676 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c024040 con 0x560da2c481b0 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f4283e00640 1 -- 172.21.15.27:0/2352466676 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f429c004c40 con 0x560da2c481b0 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560da2f1bb90 con 0x560da2c481b0 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f4283e00640 1 -- 172.21.15.27:0/2352466676 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f429c032050 con 0x560da2c481b0 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f42afe28740 10 client.28475 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:15.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.880+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560da2f36950 con 0x560da2c481b0 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f4283e00640 1 -- 172.21.15.27:0/2352466676 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560da2f1bb90 con 0x560da2c481b0 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f4283e00640 1 -- 172.21.15.27:0/2352466676 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f429c0050b0 con 0x560da2c481b0 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f42afe28740 10 client.28475 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f42afe28740 10 client.28475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f42afe28740 20 client.28475 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f42afe28740 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c2f9f0 0x560da303dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f42afe28740 4 client.28475 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.881+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da2f90f00 con 0x560da2c2f9f0 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.882+0000 7f42a1400640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c2f9f0 0x560da303dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.883+0000 7f42a1400640 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c2f9f0 0x560da303dd40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=407608198844ac97 server_cookie=377447f4886fa65b in_seq=0 out_seq=0 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.883+0000 7f4283e00640 10 client.28475 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:23:15.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f4283e00640 1 -- 172.21.15.27:0/2352466676 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560da2f90f00 con 0x560da2c2f9f0 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f4283e00640 10 client.28475 handle_command_reply: tid=0 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 client.28475 shutdown 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c481b0 msgr2=0x560da3023540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c481b0 0x560da3023540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f429c01bfc0 tx=0x7f429c01b9d0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 shutdown_connections 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c2f9f0 0x560da303dd40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da3038110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c50390 0x560da3023a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 --2- 172.21.15.27:0/2352466676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c481b0 0x560da3023540 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f4283e00640 0 client.28475 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.884+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 >> 172.21.15.27:0/2352466676 conn(0x560da2c65460 msgr2=0x560da301af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:15.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.885+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 shutdown_connections 2024-09-21T07:23:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.885+0000 7f42afe28740 1 -- 172.21.15.27:0/2352466676 wait complete. 2024-09-21T07:23:15.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.885+0000 7f42afe28740 20 client.28475 trim_cache size 0 max 16384 2024-09-21T07:23:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.899+0000 7f42afe28740 1 Processor -- start 2024-09-21T07:23:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42afe28740 1 -- start start 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42afe28740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3023a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42afe28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c50390 0x560da301aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42afe28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da301b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42afe28740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4284012940 con 0x560da2c481b0 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42afe28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4284010210 con 0x560da2c50390 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42afe28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f428400e080 con 0x560da2c71070 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.900+0000 7f42a2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3023a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3023a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49912/0 (socket says 172.21.15.27:49912) 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a2800640 1 -- 172.21.15.27:0/249120176 learned_addr learned my addr 172.21.15.27:0/249120176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a1400640 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c50390 0x560da301aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a2800640 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da301b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a1400640 1 -- 172.21.15.27:0/249120176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 msgr2=0x560da301b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a1400640 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da301b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a1400640 1 -- 172.21.15.27:0/249120176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 msgr2=0x560da3023a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a1400640 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3023a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a1400640 1 -- 172.21.15.27:0/249120176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da302ec10 con 0x560da2c50390 2024-09-21T07:23:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.901+0000 7f42a1400640 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c50390 0x560da301aea0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f42a406b1e0 tx=0x7f4290009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4c128683786405f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.902+0000 7f428ea00640 1 -- 172.21.15.27:0/249120176 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290028070 con 0x560da2c50390 2024-09-21T07:23:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.902+0000 7f428ea00640 1 -- 172.21.15.27:0/249120176 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4290003e40 con 0x560da2c50390 2024-09-21T07:23:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.902+0000 7f428ea00640 1 -- 172.21.15.27:0/249120176 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429001e870 con 0x560da2c50390 2024-09-21T07:23:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 -- 172.21.15.27:0/249120176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c50390 msgr2=0x560da301aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c50390 0x560da301aea0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f42a406b1e0 tx=0x7f4290009750 comp rx=0 tx=0).stop 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 -- 172.21.15.27:0/249120176 shutdown_connections 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c71070 0x560da301b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c50390 0x560da301aea0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 --2- 172.21.15.27:0/249120176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3023a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 -- 172.21.15.27:0/249120176 >> 172.21.15.27:0/249120176 conn(0x560da2c65460 msgr2=0x560da30232f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.903+0000 7f42afe28740 1 -- 172.21.15.27:0/249120176 shutdown_connections 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.904+0000 7f42afe28740 1 -- 172.21.15.27:0/249120176 wait complete. 2024-09-21T07:23:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.904+0000 7f42afe28740 1 -- start start 2024-09-21T07:23:15.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 1 Processor -- start 2024-09-21T07:23:15.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3030480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da30309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da30a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f428400e6d0 con 0x560da2c481b0 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f428400fb40 con 0x560da2c71070 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f428400eda0 con 0x560da2c50390 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.905+0000 7f42afe28740 10 client.0 _ll_register_callbacks cb 0x7f429c01b640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da30309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3030480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da30309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:47082/0 (socket says 172.21.15.27:47082) 2024-09-21T07:23:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 -- 172.21.15.27:0/110199164 learned_addr learned my addr 172.21.15.27:0/110199164 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 -- 172.21.15.27:0/110199164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 msgr2=0x560da30a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da30a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 -- 172.21.15.27:0/110199164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 msgr2=0x560da3030480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3030480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 -- 172.21.15.27:0/110199164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da30a7b70 con 0x560da2c50390 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f42a1400640 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da30309e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4290009ef0 tx=0x7f4290009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf731569db59094 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.906+0000 7f4283e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:15.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.907+0000 7f4283e00640 1 -- 172.21.15.27:0/110199164 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290031060 con 0x560da2c50390 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.907+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560da2f8c9b0 con 0x560da2c50390 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.907+0000 7f4283e00640 1 -- 172.21.15.27:0/110199164 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9549+0+0 (secure 0 0 0) 0x7f429001e7f0 con 0x560da2c50390 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.907+0000 7f4283e00640 1 -- 172.21.15.27:0/110199164 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4290038050 con 0x560da2c50390 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.907+0000 7f42afe28740 10 client.28438 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.907+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560da30a8ca0 con 0x560da2c50390 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.908+0000 7f4283e00640 1 -- 172.21.15.27:0/110199164 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560da2f8c9b0 con 0x560da2c50390 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.908+0000 7f4283e00640 1 -- 172.21.15.27:0/110199164 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4290010bb0 con 0x560da2c50390 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.908+0000 7f42afe28740 10 client.28438 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.908+0000 7f42afe28740 10 client.28438 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.908+0000 7f42afe28740 20 client.28438 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.908+0000 7f42afe28740 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c04da0 0x560da30ab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.908+0000 7f42afe28740 4 client.28438 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.909+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560da2f89de0 con 0x560da2c04da0 2024-09-21T07:23:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.909+0000 7f42a2800640 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c04da0 0x560da30ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.909+0000 7f42a2800640 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c04da0 0x560da30ab410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=517c8479c5ca1373 server_cookie=4c4f3a3099aef864 in_seq=0 out_seq=0 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.909+0000 7f4283e00640 10 client.28438 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f4283e00640 1 -- 172.21.15.27:0/110199164 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560da2f89de0 con 0x560da2c04da0 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f4283e00640 10 client.28438 handle_command_reply: tid=0 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 client.28438 shutdown 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 msgr2=0x560da30309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da30309e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4290009ef0 tx=0x7f4290009f20 comp rx=0 tx=0).stop 2024-09-21T07:23:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 shutdown_connections 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x560da2c04da0 0x560da30ab410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560da2c50390 0x560da30309e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f4283e00640 0 client.28438 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560da2c71070 0x560da30a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 --2- 172.21.15.27:0/110199164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560da2c481b0 0x560da3030480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 >> 172.21.15.27:0/110199164 conn(0x560da2c65460 msgr2=0x560da301aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 shutdown_connections 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.910+0000 7f42afe28740 1 -- 172.21.15.27:0/110199164 wait complete. 2024-09-21T07:23:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42afe28740 20 client.28438 trim_cache size 0 max 16384 2024-09-21T07:23:15.915 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:23:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f42880b6680 msgr2=0x7f42880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f42880b6680 0x7f42880b8b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x560da3016a20 tx=0x7f4294003f90 comp rx=0 tx=0).stop 2024-09-21T07:23:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42a410b6d0 msgr2=0x7f42a419c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42a410b6d0 0x7f42a419c860 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4294008000 tx=0x7f4294003b90 comp rx=0 tx=0).stop 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 shutdown_connections 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f42880b6680 0x7f42880b8b90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42a41a1fe0 0x7f42a41a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42a410c020 0x7f42a419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 --2- 172.21.15.27:0/4246756530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42a410b6d0 0x7f42a419c860 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.911+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 >> 172.21.15.27:0/4246756530 conn(0x7f42a4101860 msgr2=0x7f42a4103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.912+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 shutdown_connections 2024-09-21T07:23:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:15.912+0000 7f42a9200640 1 -- 172.21.15.27:0/4246756530 wait complete. 2024-09-21T07:23:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 -- 172.21.15.27:0/3912078329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c01047c0 msgr2=0x7f59c0104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 --2- 172.21.15.27:0/3912078329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c01047c0 0x7f59c0104bd0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009900 tx=0x7f59b002a410 comp rx=0 tx=0).stop 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 -- 172.21.15.27:0/3912078329 shutdown_connections 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 --2- 172.21.15.27:0/3912078329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59c0105110 0x7f59c010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 --2- 172.21.15.27:0/3912078329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c010dc20 0x7f59c0110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 --2- 172.21.15.27:0/3912078329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c01047c0 0x7f59c0104bd0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 -- 172.21.15.27:0/3912078329 >> 172.21.15.27:0/3912078329 conn(0x7f59c00fe2c0 msgr2=0x7f59c0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 -- 172.21.15.27:0/3912078329 shutdown_connections 2024-09-21T07:23:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.787+0000 7f59bf200640 1 -- 172.21.15.27:0/3912078329 wait complete. 2024-09-21T07:23:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.788+0000 7f59bf200640 1 Processor -- start 2024-09-21T07:23:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.788+0000 7f59bf200640 1 -- start start 2024-09-21T07:23:16.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c01047c0 0x7f59c019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bf200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59c0105110 0x7f59c019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bf200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c010dc20 0x7f59c01a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bf200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0113880 con 0x7f59c010dc20 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0113700 con 0x7f59c01047c0 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bf200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0113a00 con 0x7f59c0105110 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c01047c0 0x7f59c019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c01047c0 0x7f59c019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59654/0 (socket says 172.21.15.27:59654) 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59bde00640 1 -- 172.21.15.27:0/2606682042 learned_addr learned my addr 172.21.15.27:0/2606682042 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.789+0000 7f59be800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c010dc20 0x7f59c01a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59bd400640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59c0105110 0x7f59c019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59be800640 1 -- 172.21.15.27:0/2606682042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59c0105110 msgr2=0x7f59c019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59be800640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59c0105110 0x7f59c019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59be800640 1 -- 172.21.15.27:0/2606682042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c01047c0 msgr2=0x7f59c019c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59be800640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c01047c0 0x7f59c019c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59be800640 1 -- 172.21.15.27:0/2606682042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c01a2440 con 0x7f59c010dc20 2024-09-21T07:23:16.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59bde00640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c01047c0 0x7f59c019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:16.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.790+0000 7f59be800640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c010dc20 0x7f59c01a1d10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009fd0 tx=0x7f59b800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:16.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.791+0000 7f59b6a00640 1 -- 172.21.15.27:0/2606682042 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b800ce20 con 0x7f59c010dc20 2024-09-21T07:23:16.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.791+0000 7f59b6a00640 1 -- 172.21.15.27:0/2606682042 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b8005020 con 0x7f59c010dc20 2024-09-21T07:23:16.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.791+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c01a2730 con 0x7f59c010dc20 2024-09-21T07:23:16.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.791+0000 7f59b6a00640 1 -- 172.21.15.27:0/2606682042 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8010690 con 0x7f59c010dc20 2024-09-21T07:23:16.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.791+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c01a8e90 con 0x7f59c010dc20 2024-09-21T07:23:16.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.794+0000 7f59cb9c6740 1 Processor -- start 2024-09-21T07:23:16.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.794+0000 7f59b6a00640 1 -- 172.21.15.27:0/2606682042 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f59b80054d0 con 0x7f59c010dc20 2024-09-21T07:23:16.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.795+0000 7f59cb9c6740 1 -- start start 2024-09-21T07:23:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.795+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ec9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.795+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776af9660 0x564776ec9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.795+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b01840 0x564776ec9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.795+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564776e40290 con 0x564776b01840 2024-09-21T07:23:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.795+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564776e2f6c0 con 0x564776b22520 2024-09-21T07:23:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.795+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564776e334d0 con 0x564776af9660 2024-09-21T07:23:16.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776af9660 0x564776ec9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ec9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776af9660 0x564776ec9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:47118/0 (socket says 172.21.15.27:47118) 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 -- 172.21.15.27:0/154719893 learned_addr learned my addr 172.21.15.27:0/154719893 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bd400640 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b01840 0x564776ec9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 -- 172.21.15.27:0/154719893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 msgr2=0x564776ec9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ec9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 -- 172.21.15.27:0/154719893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b01840 msgr2=0x564776ec9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b01840 0x564776ec9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bde00640 1 -- 172.21.15.27:0/154719893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564776dbc460 con 0x564776af9660 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59bd400640 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b01840 0x564776ec9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59b6a00640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f59a40b6680 0x7f59a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.796+0000 7f59b6a00640 1 -- 172.21.15.27:0/2606682042 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f59b80e81b0 con 0x7f59c010dc20 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.797+0000 7f59bde00640 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776af9660 0x564776ec9880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f59b0000c00 tx=0x7f59b0005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ff74392a5210d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:16.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.797+0000 7f59bd400640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f59a40b6680 0x7f59a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.797+0000 7f59ab000640 1 -- 172.21.15.27:0/154719893 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0035040 con 0x564776af9660 2024-09-21T07:23:16.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.797+0000 7f59ab000640 1 -- 172.21.15.27:0/154719893 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b0031820 con 0x564776af9660 2024-09-21T07:23:16.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.797+0000 7f59ab000640 1 -- 172.21.15.27:0/154719893 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b00049e0 con 0x564776af9660 2024-09-21T07:23:16.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 -- 172.21.15.27:0/154719893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776af9660 msgr2=0x564776ec9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59bd400640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f59a40b6680 0x7f59a40b8b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009730 tx=0x7f59ac013040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776af9660 0x564776ec9880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f59b0000c00 tx=0x7f59b0005ed0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 -- 172.21.15.27:0/154719893 shutdown_connections 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776af9660 0x564776ec9880 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b01840 0x564776ec9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/154719893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ec9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 -- 172.21.15.27:0/154719893 >> 172.21.15.27:0/154719893 conn(0x564776b16910 msgr2=0x564776ec7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 -- 172.21.15.27:0/154719893 shutdown_connections 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.798+0000 7f59cb9c6740 1 -- 172.21.15.27:0/154719893 wait complete. 2024-09-21T07:23:16.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 -- start start 2024-09-21T07:23:16.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 Processor -- start 2024-09-21T07:23:16.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776ed4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b01840 0x564776ed53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b22520 0x564776ee0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564776c90bb0 con 0x564776af9660 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564776dca6c0 con 0x564776b01840 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.799+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564776ed7ac0 con 0x564776b22520 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59cb9c6740 10 client.0 _ll_register_callbacks cb 0x564776de5490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b01840 0x564776ed53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b01840 0x564776ed53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59666/0 (socket says 172.21.15.27:59666) 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 -- 172.21.15.27:0/143052763 learned_addr learned my addr 172.21.15.27:0/143052763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776ed4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59be800640 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b22520 0x564776ee0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 -- 172.21.15.27:0/143052763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b22520 msgr2=0x564776ee0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b22520 0x564776ee0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 -- 172.21.15.27:0/143052763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 msgr2=0x564776ed4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776ed4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.800+0000 7f59bd400640 1 -- 172.21.15.27:0/143052763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564776b8db00 con 0x564776b01840 2024-09-21T07:23:16.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f59bd400640 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b01840 0x564776ed53f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0038a0 tx=0x7f59ac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49970e9f1226e90b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:16.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f599f400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f599f400640 1 -- 172.21.15.27:0/143052763 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00fa80 con 0x564776b01840 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f599f400640 1 -- 172.21.15.27:0/143052763 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f59ac01a070 con 0x564776b01840 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b0031820 con 0x564776b01840 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f599f400640 1 -- 172.21.15.27:0/143052763 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59ac034050 con 0x564776b01840 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f59cb9c6740 10 client.28490 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.801+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564776c04960 con 0x564776b01840 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f599f400640 1 -- 172.21.15.27:0/143052763 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b0031820 con 0x564776b01840 2024-09-21T07:23:16.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f599f400640 1 -- 172.21.15.27:0/143052763 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f59ac029020 con 0x564776b01840 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f59cb9c6740 10 client.28490 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f59cb9c6740 10 client.28490 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f59cb9c6740 20 client.28490 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ae0ea0 0x564776ee6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f59cb9c6740 4 client.28490 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.802+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564776e421e0 con 0x564776ae0ea0 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.803+0000 7f59bde00640 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ae0ea0 0x564776ee6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.803+0000 7f59bde00640 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ae0ea0 0x564776ee6f20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f0c96b86ace1b0 server_cookie=ffbc2287b2116bc5 in_seq=0 out_seq=0 2024-09-21T07:23:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.803+0000 7f599f400640 10 client.28490 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.804+0000 7f599f400640 1 -- 172.21.15.27:0/143052763 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564776e421e0 con 0x564776ae0ea0 2024-09-21T07:23:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.804+0000 7f599f400640 10 client.28490 handle_command_reply: tid=0 2024-09-21T07:23:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.804+0000 7f59cb9c6740 1 client.28490 shutdown 2024-09-21T07:23:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b01840 msgr2=0x564776ed53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b01840 0x564776ed53f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0038a0 tx=0x7f59ac004730 comp rx=0 tx=0).stop 2024-09-21T07:23:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 shutdown_connections 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ae0ea0 0x564776ee6f20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b22520 0x564776ee0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b01840 0x564776ed53f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/143052763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776ed4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 >> 172.21.15.27:0/143052763 conn(0x564776b16910 msgr2=0x564776ecc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f599f400640 0 client.28490 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 shutdown_connections 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 1 -- 172.21.15.27:0/143052763 wait complete. 2024-09-21T07:23:16.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.805+0000 7f59cb9c6740 20 client.28490 trim_cache size 0 max 16384 2024-09-21T07:23:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 Processor -- start 2024-09-21T07:23:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 -- start start 2024-09-21T07:23:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776af9660 0x564776eccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776ee0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b22520 0x564776ee1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01b5750 con 0x564776b22520 2024-09-21T07:23:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01b3020 con 0x564776af9660 2024-09-21T07:23:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.824+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01b0e90 con 0x564776b01840 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59be800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b22520 0x564776ee1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776af9660 0x564776eccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776af9660 0x564776eccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59678/0 (socket says 172.21.15.27:59678) 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59be800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b22520 0x564776ee1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49958/0 (socket says 172.21.15.27:49958) 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 -- 172.21.15.27:0/1216107285 learned_addr learned my addr 172.21.15.27:0/1216107285 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bd400640 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776ee0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 -- 172.21.15.27:0/1216107285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 msgr2=0x564776ee0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776ee0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 -- 172.21.15.27:0/1216107285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b22520 msgr2=0x564776ee1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b22520 0x564776ee1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 -- 172.21.15.27:0/1216107285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564776ed0540 con 0x564776af9660 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.825+0000 7f59bde00640 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776af9660 0x564776eccb50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009a20 tx=0x7f59b00054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9d3f1d26c1ce10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.826+0000 7f59aa600640 1 -- 172.21.15.27:0/1216107285 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0035040 con 0x564776af9660 2024-09-21T07:23:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.826+0000 7f59aa600640 1 -- 172.21.15.27:0/1216107285 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b0004040 con 0x564776af9660 2024-09-21T07:23:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.826+0000 7f59aa600640 1 -- 172.21.15.27:0/1216107285 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0003b20 con 0x564776af9660 2024-09-21T07:23:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 -- 172.21.15.27:0/1216107285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776af9660 msgr2=0x564776eccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776af9660 0x564776eccb50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009a20 tx=0x7f59b00054e0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 -- 172.21.15.27:0/1216107285 shutdown_connections 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776ee0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776b22520 0x564776ee1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/1216107285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776af9660 0x564776eccb50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 -- 172.21.15.27:0/1216107285 >> 172.21.15.27:0/1216107285 conn(0x564776b16910 msgr2=0x564776ecc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 -- 172.21.15.27:0/1216107285 shutdown_connections 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 -- 172.21.15.27:0/1216107285 wait complete. 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.827+0000 7f59cb9c6740 1 -- start start 2024-09-21T07:23:16.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 1 Processor -- start 2024-09-21T07:23:16.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776edcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776edd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ed7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01b14e0 con 0x564776af9660 2024-09-21T07:23:16.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01b2950 con 0x564776b22520 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01b1bb0 con 0x564776b01840 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59cb9c6740 10 client.0 _ll_register_callbacks cb 0x7f59ac004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.828+0000 7f59bde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776edcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776edcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49966/0 (socket says 172.21.15.27:49966) 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 -- 172.21.15.27:0/200781780 learned_addr learned my addr 172.21.15.27:0/200781780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bd400640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776edd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59be800640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ed7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 -- 172.21.15.27:0/200781780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 msgr2=0x564776edd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776edd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 -- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 msgr2=0x564776ed7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ed7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 -- 172.21.15.27:0/200781780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564776edd930 con 0x564776af9660 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bd400640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776edd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f59bde00640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776edcd80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f59b0032fd0 tx=0x7f59b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686b204e0080f3e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.829+0000 7f599f400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.830+0000 7f599f400640 1 -- 172.21.15.27:0/200781780 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0035070 con 0x564776af9660 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.830+0000 7f599f400640 1 -- 172.21.15.27:0/200781780 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f59b0004400 con 0x564776af9660 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.830+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b0004040 con 0x564776af9660 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.830+0000 7f599f400640 1 -- 172.21.15.27:0/200781780 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59b0031520 con 0x564776af9660 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.830+0000 7f59cb9c6740 10 client.18462 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.830+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564776f51d10 con 0x564776af9660 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.830+0000 7f599f400640 1 -- 172.21.15.27:0/200781780 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b0004040 con 0x564776af9660 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f599f400640 1 -- 172.21.15.27:0/200781780 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f59b0046570 con 0x564776af9660 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f59cb9c6740 10 client.18462 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f59cb9c6740 10 client.18462 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f59cb9c6740 20 client.18462 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ab6250 0x564776f53ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:16.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f59cb9c6740 4 client.18462 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fbaed821", "release": true, "await": true}] 2024-09-21T07:23:16.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fbaed821", "release": true, "await": true}) -- 0x564776e3dc90 con 0x564776ab6250 2024-09-21T07:23:16.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.831+0000 7f59bd400640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ab6250 0x564776f53ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:16.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.832+0000 7f59bd400640 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ab6250 0x564776f53ff0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e908b0e24f0a887f server_cookie=2bb310bf43554455 in_seq=0 out_seq=0 2024-09-21T07:23:16.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.832+0000 7f599f400640 10 client.18462 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:16.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f599f400640 1 -- 172.21.15.27:0/200781780 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564776e3dc90 con 0x564776ab6250 2024-09-21T07:23:16.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f599f400640 10 client.18462 handle_command_reply: tid=0 2024-09-21T07:23:16.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 client.18462 shutdown 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 msgr2=0x564776edcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776edcd80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f59b0032fd0 tx=0x7f59b0005fd0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 shutdown_connections 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564776ab6250 0x564776f53ff0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564776b01840 0x564776edd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564776b22520 0x564776ed7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 --2- 172.21.15.27:0/200781780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564776af9660 0x564776edcd80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 >> 172.21.15.27:0/200781780 conn(0x564776b16910 msgr2=0x564776ecc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.834+0000 7f599f400640 0 client.18462 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.835+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 shutdown_connections 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.835+0000 7f59cb9c6740 1 -- 172.21.15.27:0/200781780 wait complete. 2024-09-21T07:23:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.835+0000 7f59cb9c6740 20 client.18462 trim_cache size 0 max 16384 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 40, 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "fbaed821": { 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 40, 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:23:16.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 144.5, 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:16.838 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:23:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.836+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f59a40b6680 msgr2=0x7f59a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.836+0000 7f59bf200640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f59a40b6680 0x7f59a40b8b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009730 tx=0x7f59ac013040 comp rx=0 tx=0).stop 2024-09-21T07:23:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.836+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c010dc20 msgr2=0x7f59c01a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.836+0000 7f59bf200640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c010dc20 0x7f59c01a1d10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009fd0 tx=0x7f59b800ef10 comp rx=0 tx=0).stop 2024-09-21T07:23:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.838+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 shutdown_connections 2024-09-21T07:23:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.838+0000 7f59bf200640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59c0105110 0x7f59c019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.838+0000 7f59bf200640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f59a40b6680 0x7f59a40b8b90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.838+0000 7f59bf200640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59c010dc20 0x7f59c01a1d10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.838+0000 7f59bf200640 1 --2- 172.21.15.27:0/2606682042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59c01047c0 0x7f59c019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.838+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 >> 172.21.15.27:0/2606682042 conn(0x7f59c00fe2c0 msgr2=0x7f59c01024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.839+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 shutdown_connections 2024-09-21T07:23:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:16.839+0000 7f59bf200640 1 -- 172.21.15.27:0/2606682042 wait complete. 2024-09-21T07:23:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:16 smithi027 ceph-mon[30779]: pgmap v694: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:23:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:16 smithi027 ceph-mon[30779]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T07:23:17.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:23:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:16 smithi153 ceph-mon[34747]: pgmap v694: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:23:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:16 smithi153 ceph-mon[34747]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T07:23:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:16 smithi119 ceph-mon[34519]: pgmap v694: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:23:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:16 smithi119 ceph-mon[34519]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T07:23:18.036 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fbaed821', total seconds elapsed: 30.3 2024-09-21T07:23:18.037 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T07:23:18.037 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:23:18.153 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:18.806 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 -- 172.21.15.27:0/629796523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 msgr2=0x7f1d4c112350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 --2- 172.21.15.27:0/629796523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 0x7f1d4c112350 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009ca0 tx=0x7f1d4402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 -- 172.21.15.27:0/629796523 shutdown_connections 2024-09-21T07:23:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 --2- 172.21.15.27:0/629796523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d4c076100 0x7f1d4c10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 --2- 172.21.15.27:0/629796523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 0x7f1d4c112350 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 --2- 172.21.15.27:0/629796523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c0757b0 0x7f1d4c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 -- 172.21.15.27:0/629796523 >> 172.21.15.27:0/629796523 conn(0x7f1d4c0ff440 msgr2=0x7f1d4c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 -- 172.21.15.27:0/629796523 shutdown_connections 2024-09-21T07:23:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.807+0000 7f1d4be00640 1 -- 172.21.15.27:0/629796523 wait complete. 2024-09-21T07:23:18.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.808+0000 7f1d4be00640 1 Processor -- start 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.808+0000 7f1d4be00640 1 -- start start 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d4c0757b0 0x7f1d4c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c076100 0x7f1d4c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 0x7f1d4c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c115b10 con 0x7f1d4c10ff10 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c115990 con 0x7f1d4c076100 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c115c90 con 0x7f1d4c0757b0 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c076100 0x7f1d4c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 0x7f1d4c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d4c0757b0 0x7f1d4c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c076100 0x7f1d4c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50622/0 (socket says 172.21.15.27:50622) 2024-09-21T07:23:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.809+0000 7f1d4a000640 1 -- 172.21.15.27:0/2410922840 learned_addr learned my addr 172.21.15.27:0/2410922840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4a000640 1 -- 172.21.15.27:0/2410922840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d4c0757b0 msgr2=0x7f1d4c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4a000640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d4c0757b0 0x7f1d4c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4a000640 1 -- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 msgr2=0x7f1d4c1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4a000640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 0x7f1d4c1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4a000640 1 -- 172.21.15.27:0/2410922840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c1a2490 con 0x7f1d4c076100 2024-09-21T07:23:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4b400640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 0x7f1d4c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4a000640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c076100 0x7f1d4c19cac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e9c0 tx=0x7f1d3800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d43400640 1 -- 172.21.15.27:0/2410922840 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800cd80 con 0x7f1d4c076100 2024-09-21T07:23:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c1a2780 con 0x7f1d4c076100 2024-09-21T07:23:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.810+0000 7f1d43400640 1 -- 172.21.15.27:0/2410922840 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d380035f0 con 0x7f1d4c076100 2024-09-21T07:23:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.811+0000 7f1d43400640 1 -- 172.21.15.27:0/2410922840 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38010690 con 0x7f1d4c076100 2024-09-21T07:23:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.811+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c1a8ee0 con 0x7f1d4c076100 2024-09-21T07:23:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.814+0000 7f1d582cb740 1 Processor -- start 2024-09-21T07:23:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.814+0000 7f1d582cb740 1 -- start start 2024-09-21T07:23:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 0x559c8d6dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d582cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559c8d652fb0 con 0x559c8d314390 2024-09-21T07:23:18.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d582cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559c8d6423e0 con 0x559c8d30c1b0 2024-09-21T07:23:18.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d582cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559c8d6461f0 con 0x559c8d335070 2024-09-21T07:23:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d4b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6dc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50636/0 (socket says 172.21.15.27:50636) 2024-09-21T07:23:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d4b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58768/0 (socket says 172.21.15.27:58768) 2024-09-21T07:23:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.815+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1582442684 learned_addr learned my addr 172.21.15.27:0/1582442684 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.816+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1582442684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 msgr2=0x559c8d6dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.816+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.816+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1582442684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 msgr2=0x559c8d6dc7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:23:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.816+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 0x559c8d6dc7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.816+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1582442684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c8d60ccd0 con 0x559c8d30c1b0 2024-09-21T07:23:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.816+0000 7f1d4b400640 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.816+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6dc270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009cf0 tx=0x7f1d3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d172eafa5c3349 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.817+0000 7f1d37400640 1 -- 172.21.15.27:0/1582442684 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c004230 con 0x559c8d30c1b0 2024-09-21T07:23:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.817+0000 7f1d43400640 1 -- 172.21.15.27:0/2410922840 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1d38004000 con 0x7f1d4c076100 2024-09-21T07:23:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.817+0000 7f1d37400640 1 -- 172.21.15.27:0/1582442684 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c0043d0 con 0x559c8d30c1b0 2024-09-21T07:23:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.817+0000 7f1d37400640 1 -- 172.21.15.27:0/1582442684 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0197c0 con 0x559c8d30c1b0 2024-09-21T07:23:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 -- 172.21.15.27:0/1582442684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 msgr2=0x559c8d6dc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6dc270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009cf0 tx=0x7f1d3c011810 comp rx=0 tx=0).stop 2024-09-21T07:23:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 -- 172.21.15.27:0/1582442684 shutdown_connections 2024-09-21T07:23:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 0x559c8d6dc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6dc270 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1582442684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 -- 172.21.15.27:0/1582442684 >> 172.21.15.27:0/1582442684 conn(0x559c8d329460 msgr2=0x559c8d6d9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 -- 172.21.15.27:0/1582442684 shutdown_connections 2024-09-21T07:23:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.818+0000 7f1d582cb740 1 -- 172.21.15.27:0/1582442684 wait complete. 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.819+0000 7f1d43400640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1d300b6590 0x7f1d300b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.819+0000 7f1d582cb740 1 -- start start 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.819+0000 7f1d43400640 1 -- 172.21.15.27:0/2410922840 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f1d380e7fd0 con 0x7f1d4c076100 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.819+0000 7f1d4b400640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1d300b6590 0x7f1d300b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.819+0000 7f1d582cb740 1 Processor -- start 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d314390 0x559c8d6e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d335070 0x559c8d6ed7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d582cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559c8d4a3700 con 0x559c8d335070 2024-09-21T07:23:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d582cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559c8d5dd580 con 0x559c8d30c1b0 2024-09-21T07:23:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d582cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559c8d6ea510 con 0x559c8d314390 2024-09-21T07:23:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d582cb740 10 client.0 _ll_register_callbacks cb 0x559c8d5ed470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d4b400640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1d300b6590 0x7f1d300b8aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d4402f540 tx=0x7f1d4402f310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d4b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d335070 0x559c8d6ed7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6e79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50652/0 (socket says 172.21.15.27:50652) 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d4b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d335070 0x559c8d6ed7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51584/0 (socket says 172.21.15.27:51584) 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1690788196 learned_addr learned my addr 172.21.15.27:0/1690788196 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.820+0000 7f1d4a000640 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d314390 0x559c8d6e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.821+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1690788196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d314390 msgr2=0x559c8d6e7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.821+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d314390 0x559c8d6e7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.821+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1690788196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d335070 msgr2=0x559c8d6ed7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.821+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d335070 0x559c8d6ed7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.821+0000 7f1d4aa00640 1 -- 172.21.15.27:0/1690788196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c8d3a0650 con 0x559c8d30c1b0 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.821+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6e79d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009340 tx=0x7f1d3c004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e40a4b7b315c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.821+0000 7f1d2be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.822+0000 7f1d2be00640 1 -- 172.21.15.27:0/1690788196 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c027070 con 0x559c8d30c1b0 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.822+0000 7f1d2be00640 1 -- 172.21.15.27:0/1690788196 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f1d3c005240 con 0x559c8d30c1b0 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.822+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c8d5dfb90 con 0x559c8d30c1b0 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.822+0000 7f1d2be00640 1 -- 172.21.15.27:0/1690788196 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c033050 con 0x559c8d30c1b0 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.822+0000 7f1d582cb740 10 client.28508 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.822+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c8d5fa950 con 0x559c8d30c1b0 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.822+0000 7f1d2be00640 1 -- 172.21.15.27:0/1690788196 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559c8d5dfb90 con 0x559c8d30c1b0 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d2be00640 1 -- 172.21.15.27:0/1690788196 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1d3c004b70 con 0x559c8d30c1b0 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d582cb740 10 client.28508 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d582cb740 10 client.28508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d582cb740 20 client.28508 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2f39f0 0x559c8d6f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d582cb740 4 client.28508 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c8d654f00 con 0x559c8d2f39f0 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.823+0000 7f1d4a000640 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2f39f0 0x559c8d6f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.824+0000 7f1d2be00640 10 client.28508 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.824+0000 7f1d4a000640 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2f39f0 0x559c8d6f97f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff12ffa5f453afc3 server_cookie=482eec89609c0ce3 in_seq=0 out_seq=0 2024-09-21T07:23:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d2be00640 1 -- 172.21.15.27:0/1690788196 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c8d654f00 con 0x559c8d2f39f0 2024-09-21T07:23:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d2be00640 10 client.28508 handle_command_reply: tid=0 2024-09-21T07:23:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 client.28508 shutdown 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 msgr2=0x559c8d6e79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6e79d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009340 tx=0x7f1d3c004170 comp rx=0 tx=0).stop 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 shutdown_connections 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2f39f0 0x559c8d6f97f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d314390 0x559c8d6e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d335070 0x559c8d6ed7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 --2- 172.21.15.27:0/1690788196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d30c1b0 0x559c8d6e79d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 >> 172.21.15.27:0/1690788196 conn(0x559c8d329460 msgr2=0x559c8d6deec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.825+0000 7f1d2be00640 0 client.28508 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.826+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 shutdown_connections 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.826+0000 7f1d582cb740 1 -- 172.21.15.27:0/1690788196 wait complete. 2024-09-21T07:23:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.826+0000 7f1d582cb740 20 client.28508 trim_cache size 0 max 16384 2024-09-21T07:23:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 Processor -- start 2024-09-21T07:23:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 -- start start 2024-09-21T07:23:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d30c1b0 0x559c8d6e1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d314390 0x559c8d6d9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6d9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1b58a0 con 0x559c8d30c1b0 2024-09-21T07:23:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1b3170 con 0x559c8d314390 2024-09-21T07:23:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.841+0000 7f1d582cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1b0fe0 con 0x559c8d335070 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d30c1b0 0x559c8d6e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d30c1b0 0x559c8d6e1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51590/0 (socket says 172.21.15.27:51590) 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4aa00640 1 -- 172.21.15.27:0/195267137 learned_addr learned my addr 172.21.15.27:0/195267137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d314390 0x559c8d6d9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4b400640 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6d9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4a000640 1 -- 172.21.15.27:0/195267137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 msgr2=0x559c8d6d9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4a000640 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6d9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4a000640 1 -- 172.21.15.27:0/195267137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d30c1b0 msgr2=0x559c8d6e1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4a000640 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d30c1b0 0x559c8d6e1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4a000640 1 -- 172.21.15.27:0/195267137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c8d6de020 con 0x559c8d314390 2024-09-21T07:23:18.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.842+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d30c1b0 0x559c8d6e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:18.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.843+0000 7f1d4a000640 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d314390 0x559c8d6d9450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d380e5370 tx=0x7f1d38003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6e99c370aa3b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:18.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.843+0000 7f1d36a00640 1 -- 172.21.15.27:0/195267137 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d380a9040 con 0x559c8d314390 2024-09-21T07:23:18.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.843+0000 7f1d36a00640 1 -- 172.21.15.27:0/195267137 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d380ae170 con 0x559c8d314390 2024-09-21T07:23:18.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.843+0000 7f1d36a00640 1 -- 172.21.15.27:0/195267137 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d380a9ce0 con 0x559c8d314390 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 -- 172.21.15.27:0/195267137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d314390 msgr2=0x559c8d6d9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d314390 0x559c8d6d9450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d380e5370 tx=0x7f1d38003e70 comp rx=0 tx=0).stop 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 -- 172.21.15.27:0/195267137 shutdown_connections 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d335070 0x559c8d6d9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d314390 0x559c8d6d9450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 --2- 172.21.15.27:0/195267137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d30c1b0 0x559c8d6e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 -- 172.21.15.27:0/195267137 >> 172.21.15.27:0/195267137 conn(0x559c8d329460 msgr2=0x559c8d6e0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 -- 172.21.15.27:0/195267137 shutdown_connections 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.844+0000 7f1d582cb740 1 -- 172.21.15.27:0/195267137 wait complete. 2024-09-21T07:23:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.845+0000 7f1d582cb740 1 -- start start 2024-09-21T07:23:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.845+0000 7f1d582cb740 1 Processor -- start 2024-09-21T07:23:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d30c1b0 0x559c8d6ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 0x559c8d6ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d582cb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d335070 0x559c8d6eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d582cb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1b1630 con 0x559c8d314390 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d582cb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1b2aa0 con 0x559c8d335070 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d582cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1b1d00 con 0x559c8d30c1b0 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d582cb740 10 client.0 _ll_register_callbacks cb 0x7f1d3c003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d30c1b0 0x559c8d6ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d4a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 0x559c8d6ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d4aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d30c1b0 0x559c8d6ee370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58794/0 (socket says 172.21.15.27:58794) 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d4aa00640 1 -- 172.21.15.27:0/2601421693 learned_addr learned my addr 172.21.15.27:0/2601421693 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.846+0000 7f1d4b400640 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d335070 0x559c8d6eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d4aa00640 1 -- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d335070 msgr2=0x559c8d6eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:18.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d335070 0x559c8d6eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d4aa00640 1 -- 172.21.15.27:0/2601421693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 msgr2=0x559c8d6ee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:18.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 0x559c8d6ee8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d4aa00640 1 -- 172.21.15.27:0/2601421693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c8d6f33c0 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d4b400640 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d335070 0x559c8d6eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d4aa00640 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d30c1b0 0x559c8d6ee370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c004f70 tx=0x7f1d3c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c5ccd0510073e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.847+0000 7f1d2be00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.848+0000 7f1d2be00640 1 -- 172.21.15.27:0/2601421693 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c03b020 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.848+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d380ae170 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.848+0000 7f1d2be00640 1 -- 172.21.15.27:0/2601421693 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f1d3c005000 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.848+0000 7f1d2be00640 1 -- 172.21.15.27:0/2601421693 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c046050 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.848+0000 7f1d582cb740 10 client.28474 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.848+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c8d764230 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d2be00640 1 -- 172.21.15.27:0/2601421693 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d380ae170 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d2be00640 1 -- 172.21.15.27:0/2601421693 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1d3c005200 con 0x559c8d30c1b0 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d582cb740 10 client.28474 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d582cb740 10 client.28474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d582cb740 20 client.28474 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:18.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d582cb740 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2c8da0 0x559c8d766980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:18.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d582cb740 4 client.28474 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:23:18.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c8d6509b0 con 0x559c8d2c8da0 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.849+0000 7f1d4a000640 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2c8da0 0x559c8d766980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.850+0000 7f1d2be00640 10 client.28474 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.850+0000 7f1d4a000640 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2c8da0 0x559c8d766980 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5186ef9da03c3d server_cookie=54f78235372e61b8 in_seq=0 out_seq=0 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.850+0000 7f1d2be00640 1 -- 172.21.15.27:0/2601421693 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c8d6509b0 con 0x559c8d2c8da0 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.850+0000 7f1d2be00640 10 client.28474 handle_command_reply: tid=0 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.850+0000 7f1d582cb740 1 client.28474 shutdown 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d30c1b0 msgr2=0x559c8d6ee370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d30c1b0 0x559c8d6ee370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c004f70 tx=0x7f1d3c009f60 comp rx=0 tx=0).stop 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 shutdown_connections 2024-09-21T07:23:18.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559c8d2c8da0 0x559c8d766980 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c8d335070 0x559c8d6eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559c8d314390 0x559c8d6ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 --2- 172.21.15.27:0/2601421693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c8d30c1b0 0x559c8d6ee370 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d2be00640 0 client.28474 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 >> 172.21.15.27:0/2601421693 conn(0x559c8d329460 msgr2=0x559c8d6d9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 shutdown_connections 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 1 -- 172.21.15.27:0/2601421693 wait complete. 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.851+0000 7f1d582cb740 20 client.28474 trim_cache size 0 max 16384 2024-09-21T07:23:18.855 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1d300b6590 msgr2=0x7f1d300b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1d300b6590 0x7f1d300b8aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d4402f540 tx=0x7f1d4402f310 comp rx=0 tx=0).stop 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c076100 msgr2=0x7f1d4c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c076100 0x7f1d4c19cac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e9c0 tx=0x7f1d3800ee80 comp rx=0 tx=0).stop 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 shutdown_connections 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1d300b6590 0x7f1d300b8aa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d4c10ff10 0x7f1d4c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d4c076100 0x7f1d4c19cac0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 --2- 172.21.15.27:0/2410922840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d4c0757b0 0x7f1d4c19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.852+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 >> 172.21.15.27:0/2410922840 conn(0x7f1d4c0ff440 msgr2=0x7f1d4c109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.853+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 shutdown_connections 2024-09-21T07:23:18.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:18.853+0000 7f1d4be00640 1 -- 172.21.15.27:0/2410922840 wait complete. 2024-09-21T07:23:19.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:18 smithi027 ceph-mon[30779]: pgmap v696: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-21T07:23:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:18 smithi153 ceph-mon[34747]: pgmap v696: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-21T07:23:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:18 smithi119 ceph-mon[34519]: pgmap v696: 129 pgs: 129 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.698+0000 7fca7b200640 1 -- 172.21.15.27:0/4230121738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 msgr2=0x7fca7c0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.698+0000 7fca7b200640 1 --2- 172.21.15.27:0/4230121738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 0x7fca7c0757b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009900 tx=0x7fca6c02a410 comp rx=0 tx=0).stop 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.699+0000 7fca7b200640 1 -- 172.21.15.27:0/4230121738 shutdown_connections 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.699+0000 7fca7b200640 1 --2- 172.21.15.27:0/4230121738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7c075dd0 0x7fca7c0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.699+0000 7fca7b200640 1 --2- 172.21.15.27:0/4230121738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c114280 0x7fca7c116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.699+0000 7fca7b200640 1 --2- 172.21.15.27:0/4230121738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 0x7fca7c0757b0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.699+0000 7fca7b200640 1 -- 172.21.15.27:0/4230121738 >> 172.21.15.27:0/4230121738 conn(0x7fca7c0ff440 msgr2=0x7fca7c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.699+0000 7fca7b200640 1 -- 172.21.15.27:0/4230121738 shutdown_connections 2024-09-21T07:23:19.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.699+0000 7fca7b200640 1 -- 172.21.15.27:0/4230121738 wait complete. 2024-09-21T07:23:19.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 Processor -- start 2024-09-21T07:23:19.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 -- start start 2024-09-21T07:23:19.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c075dd0 0x7fca7c1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:19.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 0x7fca7c1a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:19.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7c114280 0x7fca7c1a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:19.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c119ec0 con 0x7fca7c077380 2024-09-21T07:23:19.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c119d40 con 0x7fca7c075dd0 2024-09-21T07:23:19.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.700+0000 7fca7b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c11a040 con 0x7fca7c114280 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c075dd0 0x7fca7c1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca7a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7c114280 0x7fca7c1a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c075dd0 0x7fca7c1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50690/0 (socket says 172.21.15.27:50690) 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 -- 172.21.15.27:0/1604488305 learned_addr learned my addr 172.21.15.27:0/1604488305 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79400640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 0x7fca7c1a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 -- 172.21.15.27:0/1604488305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7c114280 msgr2=0x7fca7c1a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7c114280 0x7fca7c1a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 -- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 msgr2=0x7fca7c1a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 0x7fca7c1a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 -- 172.21.15.27:0/1604488305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c1a6860 con 0x7fca7c075dd0 2024-09-21T07:23:19.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79400640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 0x7fca7c1a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.701+0000 7fca79e00640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c075dd0 0x7fca7c1a08f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fca6c005940 tx=0x7fca6c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.702+0000 7fca72a00640 1 -- 172.21.15.27:0/1604488305 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c03f030 con 0x7fca7c075dd0 2024-09-21T07:23:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.702+0000 7fca72a00640 1 -- 172.21.15.27:0/1604488305 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6c003aa0 con 0x7fca7c075dd0 2024-09-21T07:23:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.702+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c1a6af0 con 0x7fca7c075dd0 2024-09-21T07:23:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.702+0000 7fca72a00640 1 -- 172.21.15.27:0/1604488305 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c035070 con 0x7fca7c075dd0 2024-09-21T07:23:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.702+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c1ad2b0 con 0x7fca7c075dd0 2024-09-21T07:23:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.703+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c0761e0 con 0x7fca7c075dd0 2024-09-21T07:23:19.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.706+0000 7fca72a00640 1 -- 172.21.15.27:0/1604488305 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fca6c042030 con 0x7fca7c075dd0 2024-09-21T07:23:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.707+0000 7fca72a00640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca600b6680 0x7fca600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.708+0000 7fca72a00640 1 -- 172.21.15.27:0/1604488305 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fca6c108510 con 0x7fca7c075dd0 2024-09-21T07:23:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.708+0000 7fca79400640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca600b6680 0x7fca600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.708+0000 7fca79400640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca600b6680 0x7fca600b8b90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca68005d00 tx=0x7fca68005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:19.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.709+0000 7fca72a00640 1 -- 172.21.15.27:0/1604488305 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fca6c0ce6b0 con 0x7fca7c075dd0 2024-09-21T07:23:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.951+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca7c1a1a20 con 0x7fca7c075dd0 2024-09-21T07:23:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.952+0000 7fca72a00640 1 -- 172.21.15.27:0/1604488305 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fca6c0d2a90 con 0x7fca7c075dd0 2024-09-21T07:23:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:23:19.955 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:23:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca600b6680 msgr2=0x7fca600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca600b6680 0x7fca600b8b90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca68005d00 tx=0x7fca68005c70 comp rx=0 tx=0).stop 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c075dd0 msgr2=0x7fca7c1a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c075dd0 0x7fca7c1a08f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fca6c005940 tx=0x7fca6c004270 comp rx=0 tx=0).stop 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 shutdown_connections 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca7c114280 0x7fca7c1a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fca600b6680 0x7fca600b8b90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c077380 0x7fca7c1a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 --2- 172.21.15.27:0/1604488305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca7c075dd0 0x7fca7c1a08f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 >> 172.21.15.27:0/1604488305 conn(0x7fca7c0ff440 msgr2=0x7fca7c110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 shutdown_connections 2024-09-21T07:23:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:19.955+0000 7fca7b200640 1 -- 172.21.15.27:0/1604488305 wait complete. 2024-09-21T07:23:21.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:20 smithi027 ceph-mon[30779]: pgmap v697: 129 pgs: 129 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-09-21T07:23:21.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:20 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1604488305' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:21.070 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T07:23:21.071 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T07:23:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:20 smithi153 ceph-mon[34747]: pgmap v697: 129 pgs: 129 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-09-21T07:23:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:20 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1604488305' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:21.256 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:23:21.256 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:23:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:20 smithi119 ceph-mon[34519]: pgmap v697: 129 pgs: 129 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-09-21T07:23:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:20 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1604488305' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:21.476 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 -- 172.21.15.27:0/4032950746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e40760a0 msgr2=0x7fb5e410fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/4032950746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e40760a0 0x7fb5e410fb70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009900 tx=0x7fb5cc02a410 comp rx=0 tx=0).stop 2024-09-21T07:23:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 -- 172.21.15.27:0/4032950746 shutdown_connections 2024-09-21T07:23:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/4032950746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e41100b0 0x7fb5e41124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/4032950746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e40760a0 0x7fb5e410fb70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/4032950746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e4075750 0x7fb5e4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 -- 172.21.15.27:0/4032950746 >> 172.21.15.27:0/4032950746 conn(0x7fb5e40ff500 msgr2=0x7fb5e4101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 -- 172.21.15.27:0/4032950746 shutdown_connections 2024-09-21T07:23:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.567+0000 7fb5e8e00640 1 -- 172.21.15.27:0/4032950746 wait complete. 2024-09-21T07:23:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 Processor -- start 2024-09-21T07:23:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 -- start start 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4075750 0x7fb5e419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e40760a0 0x7fb5e419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e41100b0 0x7fb5e41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4115c70 con 0x7fb5e4075750 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4115af0 con 0x7fb5e40760a0 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e8e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4115df0 con 0x7fb5e41100b0 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5e1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4075750 0x7fb5e419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.569+0000 7fb5dbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e40760a0 0x7fb5e419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e40760a0 0x7fb5e419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:50708/0 (socket says 172.21.15.27:50708) 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 -- 172.21.15.27:0/1108898427 learned_addr learned my addr 172.21.15.27:0/1108898427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5e2800640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e41100b0 0x7fb5e41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 -- 172.21.15.27:0/1108898427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e41100b0 msgr2=0x7fb5e41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e41100b0 0x7fb5e41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 -- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4075750 msgr2=0x7fb5e419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4075750 0x7fb5e419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 -- 172.21.15.27:0/1108898427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e41a2580 con 0x7fb5e40760a0 2024-09-21T07:23:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4075750 0x7fb5e419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.570+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e40760a0 0x7fb5e419cbf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc005ab0 tx=0x7fb5cc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.571+0000 7fb5daa00640 1 -- 172.21.15.27:0/1108898427 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc03f070 con 0x7fb5e40760a0 2024-09-21T07:23:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.571+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e41a2810 con 0x7fb5e40760a0 2024-09-21T07:23:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.571+0000 7fb5daa00640 1 -- 172.21.15.27:0/1108898427 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5cc031b50 con 0x7fb5e40760a0 2024-09-21T07:23:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.571+0000 7fb5daa00640 1 -- 172.21.15.27:0/1108898427 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc035030 con 0x7fb5e40760a0 2024-09-21T07:23:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.571+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e41a9000 con 0x7fb5e40760a0 2024-09-21T07:23:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 Processor -- start 2024-09-21T07:23:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 -- start start 2024-09-21T07:23:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2a51b0 0x5606fc649c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc64a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc64a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5606fc5bf6b0 con 0x5606fc2a51b0 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5606fc5aeae0 con 0x5606fc2844d0 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.574+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606fc5b28f0 con 0x5606fc27c2f0 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2a51b0 0x5606fc649c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e1e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc64a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2a51b0 0x5606fc649c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51642/0 (socket says 172.21.15.27:51642) 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e2800640 1 -- 172.21.15.27:0/2299547240 learned_addr learned my addr 172.21.15.27:0/2299547240 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5daa00640 1 -- 172.21.15.27:0/1108898427 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb5cc042070 con 0x7fb5e40760a0 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc64a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e1e00640 1 -- 172.21.15.27:0/2299547240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 msgr2=0x5606fc64a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc64a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e1e00640 1 -- 172.21.15.27:0/2299547240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2a51b0 msgr2=0x5606fc649c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2a51b0 0x5606fc649c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e1e00640 1 -- 172.21.15.27:0/2299547240 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606fc57b780 con 0x5606fc27c2f0 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc64a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e2800640 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2a51b0 0x5606fc649c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.575+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc64a170 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011440 tx=0x7fb5d4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233eeb6f62aa8289 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.576+0000 7fb5d3400640 1 -- 172.21.15.27:0/2299547240 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4004310 con 0x5606fc27c2f0 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.576+0000 7fb5d3400640 1 -- 172.21.15.27:0/2299547240 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d40044b0 con 0x5606fc27c2f0 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.576+0000 7fb5d3400640 1 -- 172.21.15.27:0/2299547240 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40035f0 con 0x5606fc27c2f0 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2299547240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 msgr2=0x5606fc64a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc64a170 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011440 tx=0x7fb5d4011900 comp rx=0 tx=0).stop 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5daa00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6680 0x7fb5c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2299547240 shutdown_connections 2024-09-21T07:23:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc64a170 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc64a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2299547240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2a51b0 0x5606fc649c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2299547240 >> 172.21.15.27:0/2299547240 conn(0x5606fc299300 msgr2=0x5606fc647b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5daa00640 1 -- 172.21.15.27:0/1108898427 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fb5cc108150 con 0x7fb5e40760a0 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5e2800640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6680 0x7fb5c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2299547240 shutdown_connections 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.577+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2299547240 wait complete. 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.578+0000 7fb5efdf3740 1 -- start start 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.578+0000 7fb5e2800640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6680 0x7fb5c80b8b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc005fd0 tx=0x7fb5dc009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.578+0000 7fb5efdf3740 1 Processor -- start 2024-09-21T07:23:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc655270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc6557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc6616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5606fc4e6ab0 con 0x5606fc2844d0 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5606fc555570 con 0x5606fc2a51b0 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606fc6583b0 con 0x5606fc27c2f0 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5efdf3740 10 client.0 _ll_register_callbacks cb 0x5606fc564100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc6557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc6557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51644/0 (socket says 172.21.15.27:51644) 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4114757846 learned_addr learned my addr 172.21.15.27:0/4114757846 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5e2800640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc6616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc655270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4114757846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 msgr2=0x5606fc655270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc655270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 msgr2=0x5606fc6616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc6616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4114757846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606fc310790 con 0x5606fc2844d0 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.579+0000 7fb5e2800640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc6616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc6557b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc031440 tx=0x7fb5cc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6139fbac468d6fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5c7e00640 1 -- 172.21.15.27:0/4114757846 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc049030 con 0x5606fc2844d0 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d40044b0 con 0x5606fc2844d0 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5c7e00640 1 -- 172.21.15.27:0/4114757846 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fb5cc105710 con 0x5606fc2844d0 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5c7e00640 1 -- 172.21.15.27:0/4114757846 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb5cc0c93b0 con 0x5606fc2844d0 2024-09-21T07:23:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5efdf3740 10 client.18495 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5606fc389620 con 0x5606fc2844d0 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.580+0000 7fb5c7e00640 1 -- 172.21.15.27:0/4114757846 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d40044b0 con 0x5606fc2844d0 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5c7e00640 1 -- 172.21.15.27:0/4114757846 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb5cc105be0 con 0x5606fc2844d0 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5efdf3740 10 client.18495 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5efdf3740 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5efdf3740 20 client.18495 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc263d80 0x5606fc667330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5efdf3740 4 client.18495 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606fc5c1600 con 0x5606fc263d80 2024-09-21T07:23:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.581+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc263d80 0x5606fc667330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.582+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc263d80 0x5606fc667330 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29fcf630951aa9e3 server_cookie=f72ab946cf2cb331 in_seq=0 out_seq=0 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.582+0000 7fb5c7e00640 10 client.18495 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5c7e00640 1 -- 172.21.15.27:0/4114757846 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606fc5c1600 con 0x5606fc263d80 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5c7e00640 10 client.18495 handle_command_reply: tid=0 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 client.18495 shutdown 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 msgr2=0x5606fc6557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc6557b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc031440 tx=0x7fb5cc003f90 comp rx=0 tx=0).stop 2024-09-21T07:23:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 shutdown_connections 2024-09-21T07:23:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc263d80 0x5606fc667330 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc6616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc6557b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5c7e00640 0 client.18495 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4114757846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc655270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 >> 172.21.15.27:0/4114757846 conn(0x5606fc299300 msgr2=0x5606fc64ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 shutdown_connections 2024-09-21T07:23:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4114757846 wait complete. 2024-09-21T07:23:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.583+0000 7fb5efdf3740 20 client.18495 trim_cache size 0 max 16384 2024-09-21T07:23:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 Processor -- start 2024-09-21T07:23:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 -- start start 2024-09-21T07:23:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc646aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc656ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc652440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e41b5920 con 0x5606fc2844d0 2024-09-21T07:23:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e41b31f0 con 0x5606fc2a51b0 2024-09-21T07:23:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e41b1060 con 0x5606fc27c2f0 2024-09-21T07:23:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc656ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5dbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc656ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51660/0 (socket says 172.21.15.27:51660) 2024-09-21T07:23:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.599+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4158662941 learned_addr learned my addr 172.21.15.27:0/4158662941 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5e2800640 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc652440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc646aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4158662941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 msgr2=0x5606fc646aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc646aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4158662941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 msgr2=0x5606fc652440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc652440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5dbe00640 1 -- 172.21.15.27:0/4158662941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606fc656530 con 0x5606fc2844d0 2024-09-21T07:23:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc656ea0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc031fb0 tx=0x7fb5cc005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27001d47d504b878 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5d2a00640 1 -- 172.21.15.27:0/4158662941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc049070 con 0x5606fc2844d0 2024-09-21T07:23:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5d2a00640 1 -- 172.21.15.27:0/4158662941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5cc0d7380 con 0x5606fc2844d0 2024-09-21T07:23:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.600+0000 7fb5d2a00640 1 -- 172.21.15.27:0/4158662941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0d75b0 con 0x5606fc2844d0 2024-09-21T07:23:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4158662941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 msgr2=0x5606fc656ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc656ea0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc031fb0 tx=0x7fb5cc005d70 comp rx=0 tx=0).stop 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4158662941 shutdown_connections 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2a51b0 0x5606fc652440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc2844d0 0x5606fc656ea0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/4158662941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc27c2f0 0x5606fc646aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4158662941 >> 172.21.15.27:0/4158662941 conn(0x5606fc299300 msgr2=0x5606fc656790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4158662941 shutdown_connections 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.601+0000 7fb5efdf3740 1 -- 172.21.15.27:0/4158662941 wait complete. 2024-09-21T07:23:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 -- start start 2024-09-21T07:23:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 Processor -- start 2024-09-21T07:23:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc27c2f0 0x5606fc65de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc65b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc2a51b0 0x5606fc65b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e41b16b0 con 0x5606fc27c2f0 2024-09-21T07:23:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e41b2b20 con 0x5606fc2844d0 2024-09-21T07:23:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e41b1d80 con 0x5606fc2a51b0 2024-09-21T07:23:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.602+0000 7fb5efdf3740 10 client.0 _ll_register_callbacks cb 0x7fb5cc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc27c2f0 0x5606fc65de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc27c2f0 0x5606fc65de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51662/0 (socket says 172.21.15.27:51662) 2024-09-21T07:23:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 -- 172.21.15.27:0/2460183482 learned_addr learned my addr 172.21.15.27:0/2460183482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc65b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e2800640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc2a51b0 0x5606fc65b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 -- 172.21.15.27:0/2460183482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc2a51b0 msgr2=0x5606fc65b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc2a51b0 0x5606fc65b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 -- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 msgr2=0x5606fc65b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc65b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 -- 172.21.15.27:0/2460183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606fc65bff0 con 0x5606fc27c2f0 2024-09-21T07:23:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc65b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5e1e00640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc27c2f0 0x5606fc65de20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011e00 tx=0x7fb5d40045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce9c0997920982e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.603+0000 7fb5c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.604+0000 7fb5c7e00640 1 -- 172.21.15.27:0/2460183482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d401d040 con 0x5606fc27c2f0 2024-09-21T07:23:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.604+0000 7fb5c7e00640 1 -- 172.21.15.27:0/2460183482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fb5d40037f0 con 0x5606fc27c2f0 2024-09-21T07:23:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.604+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5cc0d7380 con 0x5606fc27c2f0 2024-09-21T07:23:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.604+0000 7fb5c7e00640 1 -- 172.21.15.27:0/2460183482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb5d403d050 con 0x5606fc27c2f0 2024-09-21T07:23:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.604+0000 7fb5efdf3740 10 client.18501 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.604+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5606fc6d1ca0 con 0x5606fc27c2f0 2024-09-21T07:23:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.604+0000 7fb5c7e00640 1 -- 172.21.15.27:0/2460183482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5cc0d7380 con 0x5606fc27c2f0 2024-09-21T07:23:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5c7e00640 1 -- 172.21.15.27:0/2460183482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb5d400fbe0 con 0x5606fc27c2f0 2024-09-21T07:23:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5efdf3740 10 client.18501 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5efdf3740 10 client.18501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5efdf3740 20 client.18501 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc2391c0 0x5606fc6d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5efdf3740 4 client.18501 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:23:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606fc5bd0b0 con 0x5606fc2391c0 2024-09-21T07:23:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc2391c0 0x5606fc6d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5dbe00640 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc2391c0 0x5606fc6d42a0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7743bab3221f5352 server_cookie=d828afa5cda82633 in_seq=0 out_seq=0 2024-09-21T07:23:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.605+0000 7fb5c7e00640 10 client.18501 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5c7e00640 1 -- 172.21.15.27:0/2460183482 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606fc5bd0b0 con 0x5606fc2391c0 2024-09-21T07:23:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5c7e00640 10 client.18501 handle_command_reply: tid=0 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 client.18501 shutdown 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc27c2f0 msgr2=0x5606fc65de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc27c2f0 0x5606fc65de20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011e00 tx=0x7fb5d40045a0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 shutdown_connections 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5606fc2391c0 0x5606fc6d42a0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606fc2a51b0 0x5606fc65b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5c7e00640 0 client.18501 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606fc2844d0 0x5606fc65b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 --2- 172.21.15.27:0/2460183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5606fc27c2f0 0x5606fc65de20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.618+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 >> 172.21.15.27:0/2460183482 conn(0x5606fc299300 msgr2=0x5606fc646870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 shutdown_connections 2024-09-21T07:23:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5efdf3740 1 -- 172.21.15.27:0/2460183482 wait complete. 2024-09-21T07:23:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5efdf3740 20 client.18501 trim_cache size 0 max 16384 2024-09-21T07:23:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:23:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:23:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:23:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6680 msgr2=0x7fb5c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6680 0x7fb5c80b8b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc005fd0 tx=0x7fb5dc009500 comp rx=0 tx=0).stop 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e40760a0 msgr2=0x7fb5e419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e40760a0 0x7fb5e419cbf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc005ab0 tx=0x7fb5cc003b90 comp rx=0 tx=0).stop 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 shutdown_connections 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5e41100b0 0x7fb5e41a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb5c80b6680 0x7fb5c80b8b90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e40760a0 0x7fb5e419cbf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 --2- 172.21.15.27:0/1108898427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e4075750 0x7fb5e419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.619+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 >> 172.21.15.27:0/1108898427 conn(0x7fb5e40ff500 msgr2=0x7fb5e41010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.620+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 shutdown_connections 2024-09-21T07:23:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:21.620+0000 7fb5e8e00640 1 -- 172.21.15.27:0/1108898427 wait complete. 2024-09-21T07:23:22.094 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:22.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 -- 172.21.15.27:0/1181143505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb0105dc0 msgr2=0x7f1cb01061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 --2- 172.21.15.27:0/1181143505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb0105dc0 0x7f1cb01061d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009900 tx=0x7f1ca002a410 comp rx=0 tx=0).stop 2024-09-21T07:23:22.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 -- 172.21.15.27:0/1181143505 shutdown_connections 2024-09-21T07:23:22.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 --2- 172.21.15.27:0/1181143505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010c050 0x7f1cb0112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 --2- 172.21.15.27:0/1181143505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb010b700 0x7f1cb010bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 --2- 172.21.15.27:0/1181143505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb0105dc0 0x7f1cb01061d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 -- 172.21.15.27:0/1181143505 >> 172.21.15.27:0/1181143505 conn(0x7f1cb01017d0 msgr2=0x7f1cb0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 -- 172.21.15.27:0/1181143505 shutdown_connections 2024-09-21T07:23:22.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.128+0000 7f1cb5200640 1 -- 172.21.15.27:0/1181143505 wait complete. 2024-09-21T07:23:22.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.129+0000 7f1cb5200640 1 Processor -- start 2024-09-21T07:23:22.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.130+0000 7f1cb5200640 1 -- start start 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.130+0000 7f1cb5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0105dc0 0x7f1cb019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.130+0000 7f1cb5200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010b700 0x7f1cb019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.130+0000 7f1cb5200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb010c050 0x7f1cb01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.130+0000 7f1cb5200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0115d10 con 0x7f1cb010c050 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.130+0000 7f1cb5200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0115b90 con 0x7f1cb0105dc0 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.130+0000 7f1cb5200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0115e90 con 0x7f1cb010b700 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb010c050 0x7f1cb01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010b700 0x7f1cb019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010b700 0x7f1cb019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58902/0 (socket says 172.21.15.27:58902) 2024-09-21T07:23:22.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 -- 172.21.15.27:0/2406177931 learned_addr learned my addr 172.21.15.27:0/2406177931 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb010c050 0x7f1cb01a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51684/0 (socket says 172.21.15.27:51684) 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 -- 172.21.15.27:0/2406177931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0105dc0 msgr2=0x7f1cb019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cade00640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0105dc0 0x7f1cb019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0105dc0 0x7f1cb019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 -- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb010c050 msgr2=0x7f1cb01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb010c050 0x7f1cb01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.131+0000 7f1cad400640 1 -- 172.21.15.27:0/2406177931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb01a2630 con 0x7f1cb010b700 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1cade00640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0105dc0 0x7f1cb019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1cae800640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb010c050 0x7f1cb01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1cad400640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010b700 0x7f1cb019cc60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00e9f0 tx=0x7f1c9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1ca6a00640 1 -- 172.21.15.27:0/2406177931 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00ccc0 con 0x7f1cb010b700 2024-09-21T07:23:22.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb01a2920 con 0x7f1cb010b700 2024-09-21T07:23:22.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1ca6a00640 1 -- 172.21.15.27:0/2406177931 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c9c0108e0 con 0x7f1cb010b700 2024-09-21T07:23:22.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb0077800 con 0x7f1cb010b700 2024-09-21T07:23:22.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.132+0000 7f1ca6a00640 1 -- 172.21.15.27:0/2406177931 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c018860 con 0x7f1cb010b700 2024-09-21T07:23:22.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.136+0000 7f1cbbe13740 1 Processor -- start 2024-09-21T07:23:22.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1cbbe13740 1 -- start start 2024-09-21T07:23:22.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1ca6a00640 1 -- 172.21.15.27:0/2406177931 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1c9c010430 con 0x7f1cb010b700 2024-09-21T07:23:22.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec401350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec401890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec401dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5586ec376ca0 con 0x5586ec059840 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5586ec3660d0 con 0x5586ec030980 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.137+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5586ec369ee0 con 0x5586ec038b60 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec401350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec401350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51686/0 (socket says 172.21.15.27:51686) 2024-09-21T07:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 -- 172.21.15.27:0/2764120309 learned_addr learned my addr 172.21.15.27:0/2764120309 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cad400640 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec401dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cade00640 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec401890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 -- 172.21.15.27:0/2764120309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 msgr2=0x5586ec401dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec401dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 -- 172.21.15.27:0/2764120309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 msgr2=0x5586ec401890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec401890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.138+0000 7f1cae800640 1 -- 172.21.15.27:0/2764120309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ec318400 con 0x5586ec059840 2024-09-21T07:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.139+0000 7f1cade00640 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec401890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.139+0000 7f1cad400640 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec401dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.139+0000 7f1cae800640 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec401350 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80117e0 tx=0x7f1ca8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb9014aaf25255d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.139+0000 7f1c9b400640 1 -- 172.21.15.27:0/2764120309 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8011310 con 0x5586ec059840 2024-09-21T07:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.139+0000 7f1c9b400640 1 -- 172.21.15.27:0/2764120309 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca8003e20 con 0x5586ec059840 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.139+0000 7f1c9b400640 1 -- 172.21.15.27:0/2764120309 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8004490 con 0x5586ec059840 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.140+0000 7f1ca6a00640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c940b6680 0x7f1c940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.140+0000 7f1ca6a00640 1 -- 172.21.15.27:0/2406177931 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f1c9c0e7e70 con 0x7f1cb010b700 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.140+0000 7f1cade00640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c940b6680 0x7f1c940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.140+0000 7f1cbbe13740 1 -- 172.21.15.27:0/2764120309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 msgr2=0x5586ec401350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.140+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec401350 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80117e0 tx=0x7f1ca8011ca0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cbbe13740 1 -- 172.21.15.27:0/2764120309 shutdown_connections 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec401dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec401890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/2764120309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec401350 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cbbe13740 1 -- 172.21.15.27:0/2764120309 >> 172.21.15.27:0/2764120309 conn(0x5586ec04d990 msgr2=0x5586ec3ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cbbe13740 1 -- 172.21.15.27:0/2764120309 shutdown_connections 2024-09-21T07:23:22.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cade00640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c940b6680 0x7f1c940b8b90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5586ec3fffd0 tx=0x7f1ca0005ee0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.141+0000 7f1cbbe13740 1 -- 172.21.15.27:0/2764120309 wait complete. 2024-09-21T07:23:22.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.142+0000 7f1cbbe13740 1 -- start start 2024-09-21T07:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.142+0000 7f1cbbe13740 1 Processor -- start 2024-09-21T07:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec030980 0x5586ec40c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec038b60 0x5586ec40cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec059840 0x5586ec40d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5586ec1c77a0 con 0x5586ec030980 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5586ec30ba10 con 0x5586ec038b60 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5586ec40fb30 con 0x5586ec059840 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cbbe13740 10 client.0 _ll_register_callbacks cb 0x5586ec31bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cade00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec030980 0x5586ec40c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cad400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec038b60 0x5586ec40cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cade00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec030980 0x5586ec40c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51698/0 (socket says 172.21.15.27:51698) 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.143+0000 7f1cade00640 1 -- 172.21.15.27:0/86676596 learned_addr learned my addr 172.21.15.27:0/86676596 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cae800640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec059840 0x5586ec40d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cad400640 1 -- 172.21.15.27:0/86676596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec059840 msgr2=0x5586ec40d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cad400640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec059840 0x5586ec40d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cad400640 1 -- 172.21.15.27:0/86676596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec030980 msgr2=0x5586ec40c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cad400640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec030980 0x5586ec40c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cad400640 1 -- 172.21.15.27:0/86676596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ec0d3b90 con 0x5586ec038b60 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cae800640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec059840 0x5586ec40d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cade00640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec030980 0x5586ec40c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1cad400640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec038b60 0x5586ec40cef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0e51b0 tx=0x7f1c9c00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a95c584d3600e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1c8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.144+0000 7f1c8fe00640 1 -- 172.21.15.27:0/86676596 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0a9750 con 0x5586ec038b60 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca8003e20 con 0x5586ec038b60 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1c8fe00640 1 -- 172.21.15.27:0/86676596 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f1c9c0ac580 con 0x5586ec038b60 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1c8fe00640 1 -- 172.21.15.27:0/86676596 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c9c0ad020 con 0x5586ec038b60 2024-09-21T07:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1cbbe13740 10 client.28544 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586ec41b830 con 0x5586ec038b60 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1c8fe00640 1 -- 172.21.15.27:0/86676596 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca8003e20 con 0x5586ec038b60 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1c8fe00640 1 -- 172.21.15.27:0/86676596 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1c9c0ac780 con 0x5586ec038b60 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1cbbe13740 10 client.28544 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1cbbe13740 10 client.28544 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1cbbe13740 20 client.28544 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.145+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ec0181c0 0x5586ec41ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.146+0000 7f1cbbe13740 4 client.28544 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.146+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ec378bf0 con 0x5586ec0181c0 2024-09-21T07:23:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.146+0000 7f1cade00640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ec0181c0 0x5586ec41ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.147+0000 7f1cade00640 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ec0181c0 0x5586ec41ed20 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77932ceee08b0cfc server_cookie=a7157dbfcb92106e in_seq=0 out_seq=0 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.147+0000 7f1c8fe00640 10 client.28544 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.147+0000 7f1c8fe00640 1 -- 172.21.15.27:0/86676596 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5586ec378bf0 con 0x5586ec0181c0 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1c8fe00640 10 client.28544 handle_command_reply: tid=0 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 client.28544 shutdown 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec038b60 msgr2=0x5586ec40cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec038b60 0x5586ec40cef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0e51b0 tx=0x7f1c9c00cef0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 shutdown_connections 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ec0181c0 0x5586ec41ed20 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec059840 0x5586ec40d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec038b60 0x5586ec40cef0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/86676596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec030980 0x5586ec40c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 >> 172.21.15.27:0/86676596 conn(0x5586ec04d990 msgr2=0x5586ec404530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1c8fe00640 0 client.28544 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 shutdown_connections 2024-09-21T07:23:22.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 1 -- 172.21.15.27:0/86676596 wait complete. 2024-09-21T07:23:22.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.148+0000 7f1cbbe13740 20 client.28544 trim_cache size 0 max 16384 2024-09-21T07:23:22.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.167+0000 7f1cbbe13740 1 Processor -- start 2024-09-21T07:23:22.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.167+0000 7f1cbbe13740 1 -- start start 2024-09-21T07:23:22.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.167+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec40e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.167+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec409ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.167+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec40a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b9f00 con 0x5586ec059840 2024-09-21T07:23:22.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b77d0 con 0x5586ec030980 2024-09-21T07:23:22.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b5640 con 0x5586ec038b60 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec40a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cae800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec40a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51700/0 (socket says 172.21.15.27:51700) 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cae800640 1 -- 172.21.15.27:0/3203494584 learned_addr learned my addr 172.21.15.27:0/3203494584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cade00640 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec40e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cad400640 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec409ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cad400640 1 -- 172.21.15.27:0/3203494584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 msgr2=0x5586ec40e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cad400640 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec40e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cad400640 1 -- 172.21.15.27:0/3203494584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 msgr2=0x5586ec40a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cad400640 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec40a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.168+0000 7f1cad400640 1 -- 172.21.15.27:0/3203494584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ec3faa40 con 0x5586ec038b60 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.169+0000 7f1cae800640 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec40a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.169+0000 7f1cad400640 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec409ba0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0ad5b0 tx=0x7f1c9c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be43dbef392bf216 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.169+0000 7f1c9aa00640 1 -- 172.21.15.27:0/3203494584 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0ad200 con 0x5586ec038b60 2024-09-21T07:23:22.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.169+0000 7f1c9aa00640 1 -- 172.21.15.27:0/3203494584 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c9c010720 con 0x5586ec038b60 2024-09-21T07:23:22.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.169+0000 7f1c9aa00640 1 -- 172.21.15.27:0/3203494584 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0ad680 con 0x5586ec038b60 2024-09-21T07:23:22.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3203494584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 msgr2=0x5586ec409ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec409ba0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0ad5b0 tx=0x7f1c9c009740 comp rx=0 tx=0).stop 2024-09-21T07:23:22.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3203494584 shutdown_connections 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec038b60 0x5586ec409ba0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec059840 0x5586ec40a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3203494584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec030980 0x5586ec40e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3203494584 >> 172.21.15.27:0/3203494584 conn(0x5586ec04d990 msgr2=0x5586ec40ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3203494584 shutdown_connections 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.170+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3203494584 wait complete. 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.171+0000 7f1cbbe13740 1 -- start start 2024-09-21T07:23:22.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.171+0000 7f1cbbe13740 1 Processor -- start 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec030980 0x5586ec4154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec038b60 0x5586ec410990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cbbe13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec059840 0x5586ec410ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b5c90 con 0x5586ec038b60 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b7100 con 0x5586ec059840 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cbbe13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b6360 con 0x5586ec030980 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cbbe13740 10 client.0 _ll_register_callbacks cb 0x7f1c9c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:22.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cade00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec030980 0x5586ec4154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cade00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec030980 0x5586ec4154e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58932/0 (socket says 172.21.15.27:58932) 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cade00640 1 -- 172.21.15.27:0/3682697003 learned_addr learned my addr 172.21.15.27:0/3682697003 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cad400640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec038b60 0x5586ec410990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.172+0000 7f1cae800640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec059840 0x5586ec410ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cade00640 1 -- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec059840 msgr2=0x5586ec410ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cade00640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec059840 0x5586ec410ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cade00640 1 -- 172.21.15.27:0/3682697003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec038b60 msgr2=0x5586ec410990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cade00640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec038b60 0x5586ec410990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cade00640 1 -- 172.21.15.27:0/3682697003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ec411610 con 0x5586ec030980 2024-09-21T07:23:22.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cad400640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec038b60 0x5586ec410990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cae800640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec059840 0x5586ec410ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cade00640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec030980 0x5586ec4154e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009230 tx=0x7f1ca003c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=262e8670583ed6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1c8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1c8fe00640 1 -- 172.21.15.27:0/3682697003 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0035050 con 0x5586ec030980 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1c8fe00640 1 -- 172.21.15.27:0/3682697003 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f1ca0004450 con 0x5586ec030980 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.173+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586ec3746a0 con 0x5586ec030980 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.174+0000 7f1c8fe00640 1 -- 172.21.15.27:0/3682697003 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ca0005040 con 0x5586ec030980 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.174+0000 7f1cbbe13740 10 client.28516 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.174+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586ec4895d0 con 0x5586ec030980 2024-09-21T07:23:22.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.174+0000 7f1c8fe00640 1 -- 172.21.15.27:0/3682697003 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5586ec3746a0 con 0x5586ec030980 2024-09-21T07:23:22.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.174+0000 7f1c8fe00640 1 -- 172.21.15.27:0/3682697003 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1ca00035f0 con 0x5586ec030980 2024-09-21T07:23:22.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.175+0000 7f1cbbe13740 10 client.28516 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:22.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.175+0000 7f1cbbe13740 10 client.28516 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:23:22.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.175+0000 7f1cbbe13740 20 client.28516 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:22.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.175+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ebfed570 0x5586ec48bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.175+0000 7f1cbbe13740 4 client.28516 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T07:23:22.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.175+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x5586ec371ad0 con 0x5586ebfed570 2024-09-21T07:23:22.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.175+0000 7f1cad400640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ebfed570 0x5586ec48bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.176+0000 7f1cad400640 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ebfed570 0x5586ec48bde0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61af39b9e7324890 server_cookie=46b75562c1b1651b in_seq=0 out_seq=0 2024-09-21T07:23:22.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.176+0000 7f1c8fe00640 10 client.28516 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1c8fe00640 1 -- 172.21.15.27:0/3682697003 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5586ec371ad0 con 0x5586ebfed570 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1c8fe00640 10 client.28516 handle_command_reply: tid=0 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 client.28516 shutdown 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec030980 msgr2=0x5586ec4154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec030980 0x5586ec4154e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009230 tx=0x7f1ca003c000 comp rx=0 tx=0).stop 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 shutdown_connections 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5586ebfed570 0x5586ec48bde0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5586ec059840 0x5586ec410ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5586ec038b60 0x5586ec410990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 --2- 172.21.15.27:0/3682697003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5586ec030980 0x5586ec4154e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1c8fe00640 0 client.28516 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 >> 172.21.15.27:0/3682697003 conn(0x5586ec04d990 msgr2=0x5586ec40dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 shutdown_connections 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 1 -- 172.21.15.27:0/3682697003 wait complete. 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.188+0000 7f1cbbe13740 20 client.28516 trim_cache size 0 max 16384 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 42, 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "68c6c9a7": { 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 42, 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:23:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 144.5, 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:22.194 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c940b6680 msgr2=0x7f1c940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c940b6680 0x7f1c940b8b90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5586ec3fffd0 tx=0x7f1ca0005ee0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010b700 msgr2=0x7f1cb019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010b700 0x7f1cb019cc60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00e9f0 tx=0x7f1c9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 shutdown_connections 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cb010b700 0x7f1cb019cc60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1c940b6680 0x7f1c940b8b90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cb010c050 0x7f1cb01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 --2- 172.21.15.27:0/2406177931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0105dc0 0x7f1cb019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 >> 172.21.15.27:0/2406177931 conn(0x7f1cb01017d0 msgr2=0x7f1cb0104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.189+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 shutdown_connections 2024-09-21T07:23:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.190+0000 7f1cb5200640 1 -- 172.21.15.27:0/2406177931 wait complete. 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.784+0000 7f3e5e400640 1 -- 172.21.15.27:0/974891022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 msgr2=0x7f3e58077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.784+0000 7f3e5e400640 1 --2- 172.21.15.27:0/974891022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 0x7f3e58077d50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009900 tx=0x7f3e4802a410 comp rx=0 tx=0).stop 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.784+0000 7f3e5e400640 1 -- 172.21.15.27:0/974891022 shutdown_connections 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.784+0000 7f3e5e400640 1 --2- 172.21.15.27:0/974891022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 0x7f3e58116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.784+0000 7f3e5e400640 1 --2- 172.21.15.27:0/974891022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58078290 0x7f3e58114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.784+0000 7f3e5e400640 1 --2- 172.21.15.27:0/974891022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 0x7f3e58077d50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.784+0000 7f3e5e400640 1 -- 172.21.15.27:0/974891022 >> 172.21.15.27:0/974891022 conn(0x7f3e58101810 msgr2=0x7f3e58103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.785+0000 7f3e5e400640 1 -- 172.21.15.27:0/974891022 shutdown_connections 2024-09-21T07:23:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.785+0000 7f3e5e400640 1 -- 172.21.15.27:0/974891022 wait complete. 2024-09-21T07:23:22.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.786+0000 7f3e5e400640 1 Processor -- start 2024-09-21T07:23:22.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.786+0000 7f3e5e400640 1 -- start start 2024-09-21T07:23:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e5e400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 0x7f3e581a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e5e400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58078290 0x7f3e581a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e5e400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 0x7f3e581a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e5e400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5811a0b0 con 0x7f3e58077940 2024-09-21T07:23:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e5e400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58119f30 con 0x7f3e58078290 2024-09-21T07:23:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e5e400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5811a230 con 0x7f3e58114740 2024-09-21T07:23:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e57400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 0x7f3e581a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e57e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 0x7f3e581a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e56a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58078290 0x7f3e581a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e57e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 0x7f3e581a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58948/0 (socket says 172.21.15.27:58948) 2024-09-21T07:23:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e57e00640 1 -- 172.21.15.27:0/1662068916 learned_addr learned my addr 172.21.15.27:0/1662068916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.787+0000 7f3e57400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 0x7f3e581a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51740/0 (socket says 172.21.15.27:51740) 2024-09-21T07:23:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e57e00640 1 -- 172.21.15.27:0/1662068916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58078290 msgr2=0x7f3e581a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e57e00640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58078290 0x7f3e581a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e57e00640 1 -- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 msgr2=0x7f3e581a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e57e00640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 0x7f3e581a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e57e00640 1 -- 172.21.15.27:0/1662068916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e581a6a30 con 0x7f3e58114740 2024-09-21T07:23:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e57e00640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 0x7f3e581a6300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000eb10 tx=0x7f3e5000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e4fe00640 1 -- 172.21.15.27:0/1662068916 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50004b20 con 0x7f3e58114740 2024-09-21T07:23:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e581a6d20 con 0x7f3e58114740 2024-09-21T07:23:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e4fe00640 1 -- 172.21.15.27:0/1662068916 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e50004cc0 con 0x7f3e58114740 2024-09-21T07:23:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e4fe00640 1 -- 172.21.15.27:0/1662068916 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50010670 con 0x7f3e58114740 2024-09-21T07:23:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.788+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e581ad480 con 0x7f3e58114740 2024-09-21T07:23:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.792+0000 7f3e65009740 1 Processor -- start 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.793+0000 7f3e4fe00640 1 -- 172.21.15.27:0/1662068916 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3e50005020 con 0x7f3e58114740 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e65009740 1 -- start start 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e65009740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d49d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e65009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e65009740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828981670 0x55a828d4a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e65009740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a828cbfe90 con 0x55a8289a2350 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e65009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a828caf2c0 con 0x55a828979490 2024-09-21T07:23:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e65009740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a828cb30d0 con 0x55a828981670 2024-09-21T07:23:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e56a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d49d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e56a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d49d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51748/0 (socket says 172.21.15.27:51748) 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e57400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d4a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.794+0000 7f3e56a00640 1 -- 172.21.15.27:0/1165625761 learned_addr learned my addr 172.21.15.27:0/1165625761 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e56a00640 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828981670 0x55a828d4a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e57400640 1 -- 172.21.15.27:0/1165625761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828981670 msgr2=0x55a828d4a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e57400640 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828981670 0x55a828d4a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e57400640 1 -- 172.21.15.27:0/1165625761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 msgr2=0x55a828d49d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e57400640 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d49d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e57400640 1 -- 172.21.15.27:0/1165625761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a828a0d930 con 0x55a828979490 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e56a00640 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d49d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.795+0000 7f3e4fe00640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.796+0000 7f3e57400640 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d4a250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009230 tx=0x7f3e48032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67aca7ecd1d84273 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.796+0000 7f3e4fe00640 1 -- 172.21.15.27:0/1662068916 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f3e500e8100 con 0x7f3e58114740 2024-09-21T07:23:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.796+0000 7f3e56a00640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.796+0000 7f3e4cc00640 1 -- 172.21.15.27:0/1165625761 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48040070 con 0x55a828979490 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.796+0000 7f3e4cc00640 1 -- 172.21.15.27:0/1165625761 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e48004030 con 0x55a828979490 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.796+0000 7f3e4cc00640 1 -- 172.21.15.27:0/1165625761 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48035040 con 0x55a828979490 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 -- 172.21.15.27:0/1165625761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 msgr2=0x55a828d4a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e56a00640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3e440058c0 tx=0x7f3e44005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d4a250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009230 tx=0x7f3e48032b90 comp rx=0 tx=0).stop 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 -- 172.21.15.27:0/1165625761 shutdown_connections 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828981670 0x55a828d4a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d4a250 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 --2- 172.21.15.27:0/1165625761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d49d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 -- 172.21.15.27:0/1165625761 >> 172.21.15.27:0/1165625761 conn(0x55a828996740 msgr2=0x55a828d47c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 -- 172.21.15.27:0/1165625761 shutdown_connections 2024-09-21T07:23:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.797+0000 7f3e65009740 1 -- 172.21.15.27:0/1165625761 wait complete. 2024-09-21T07:23:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.798+0000 7f3e65009740 1 -- start start 2024-09-21T07:23:22.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.798+0000 7f3e65009740 1 Processor -- start 2024-09-21T07:23:22.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e65009740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828979490 0x55a828d558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e65009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d55e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e65009740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d61780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e65009740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a828b10890 con 0x55a828979490 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e65009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a828c4a160 con 0x55a828981670 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e65009740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a828d58490 con 0x55a8289a2350 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e65009740 10 client.0 _ll_register_callbacks cb 0x55a828c66a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e57400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828979490 0x55a828d558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e57400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828979490 0x55a828d558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51760/0 (socket says 172.21.15.27:51760) 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e57400640 1 -- 172.21.15.27:0/4125765319 learned_addr learned my addr 172.21.15.27:0/4125765319 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e57e00640 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d61780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.799+0000 7f3e56a00640 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d55e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e57400640 1 -- 172.21.15.27:0/4125765319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 msgr2=0x55a828d61780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e57400640 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d61780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e57400640 1 -- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 msgr2=0x55a828d55e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e57400640 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d55e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e57400640 1 -- 172.21.15.27:0/4125765319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a828d61d10 con 0x55a828979490 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e57400640 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828979490 0x55a828d558e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3e48032f10 tx=0x7f3e48032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0320ab909f78cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e41600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e41600640 1 -- 172.21.15.27:0/4125765319 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48035070 con 0x55a828979490 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e41600640 1 -- 172.21.15.27:0/4125765319 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f3e480314a0 con 0x55a828979490 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.800+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e48004030 con 0x55a828979490 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e41600640 1 -- 172.21.15.27:0/4125765319 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e48058050 con 0x55a828979490 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 10 client.18531 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:22.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a828d64620 con 0x55a828979490 2024-09-21T07:23:22.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e41600640 1 -- 172.21.15.27:0/4125765319 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e48004030 con 0x55a828979490 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e41600640 1 -- 172.21.15.27:0/4125765319 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3e480049f0 con 0x55a828979490 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 10 client.18531 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 10 client.18531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 20 client.18531 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828960cd0 0x55a828d67980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 4 client.18531 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.801+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a828cc1de0 con 0x55a828960cd0 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.802+0000 7f3e56a00640 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828960cd0 0x55a828d67980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.802+0000 7f3e56a00640 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828960cd0 0x55a828d67980 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d55010e321cc75e9 server_cookie=4f0d61d432ef1af7 in_seq=0 out_seq=0 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.802+0000 7f3e41600640 10 client.18531 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.803+0000 7f3e41600640 1 -- 172.21.15.27:0/4125765319 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a828cc1de0 con 0x55a828960cd0 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e41600640 10 client.18531 handle_command_reply: tid=0 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 client.18531 shutdown 2024-09-21T07:23:22.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828979490 msgr2=0x55a828d558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828979490 0x55a828d558e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3e48032f10 tx=0x7f3e48032f40 comp rx=0 tx=0).stop 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 shutdown_connections 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828960cd0 0x55a828d67980 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d61780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d55e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 --2- 172.21.15.27:0/4125765319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828979490 0x55a828d558e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 >> 172.21.15.27:0/4125765319 conn(0x55a828996740 msgr2=0x55a828d46280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e41600640 0 client.18531 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 shutdown_connections 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 1 -- 172.21.15.27:0/4125765319 wait complete. 2024-09-21T07:23:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.804+0000 7f3e65009740 20 client.18531 trim_cache size 0 max 16384 2024-09-21T07:23:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 Processor -- start 2024-09-21T07:23:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 -- start start 2024-09-21T07:23:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d46990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828981670 0x55a828d616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d4ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581b9cc0 con 0x55a828981670 2024-09-21T07:23:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581b7590 con 0x55a828979490 2024-09-21T07:23:22.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e65009740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581b5400 con 0x55a8289a2350 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e56a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828981670 0x55a828d616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e56a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828981670 0x55a828d616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51766/0 (socket says 172.21.15.27:51766) 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e56a00640 1 -- 172.21.15.27:0/207930496 learned_addr learned my addr 172.21.15.27:0/207930496 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.824+0000 7f3e57400640 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d46990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e57e00640 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d4ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e57400640 1 -- 172.21.15.27:0/207930496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 msgr2=0x55a828d4ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e57400640 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d4ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e57400640 1 -- 172.21.15.27:0/207930496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828981670 msgr2=0x55a828d616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e57400640 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828981670 0x55a828d616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e57400640 1 -- 172.21.15.27:0/207930496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a828d4c5a0 con 0x55a828979490 2024-09-21T07:23:22.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e56a00640 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828981670 0x55a828d616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:22.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e57400640 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d46990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3e48000c00 tx=0x7f3e48005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927f46fefa216486 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.825+0000 7f3e43e00640 1 -- 172.21.15.27:0/207930496 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48035040 con 0x55a828979490 2024-09-21T07:23:22.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e43e00640 1 -- 172.21.15.27:0/207930496 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e48004b70 con 0x55a828979490 2024-09-21T07:23:22.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e43e00640 1 -- 172.21.15.27:0/207930496 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48040040 con 0x55a828979490 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e65009740 1 -- 172.21.15.27:0/207930496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 msgr2=0x55a828d46990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e65009740 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d46990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3e48000c00 tx=0x7f3e48005c90 comp rx=0 tx=0).stop 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e65009740 1 -- 172.21.15.27:0/207930496 shutdown_connections 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e65009740 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8289a2350 0x55a828d4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e65009740 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a828981670 0x55a828d616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e65009740 1 --2- 172.21.15.27:0/207930496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828979490 0x55a828d46990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.826+0000 7f3e65009740 1 -- 172.21.15.27:0/207930496 >> 172.21.15.27:0/207930496 conn(0x55a828996740 msgr2=0x55a828d46280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.827+0000 7f3e65009740 1 -- 172.21.15.27:0/207930496 shutdown_connections 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.827+0000 7f3e65009740 1 -- 172.21.15.27:0/207930496 wait complete. 2024-09-21T07:23:22.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.827+0000 7f3e65009740 1 -- start start 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.827+0000 7f3e65009740 1 Processor -- start 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e65009740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828979490 0x55a828d67bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e65009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d68130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e65009740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d68670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e65009740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581b5a50 con 0x55a8289a2350 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e65009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581b6ec0 con 0x55a828981670 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e65009740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581b6120 con 0x55a828979490 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e65009740 10 client.0 _ll_register_callbacks cb 0x7f3e44009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d68670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e56a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d68130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d68670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51770/0 (socket says 172.21.15.27:51770) 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e57e00640 1 -- 172.21.15.27:0/3438945704 learned_addr learned my addr 172.21.15.27:0/3438945704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.828+0000 7f3e57400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828979490 0x55a828d67bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e57400640 1 -- 172.21.15.27:0/3438945704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 msgr2=0x55a828d68130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e57400640 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d68130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e57400640 1 -- 172.21.15.27:0/3438945704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 msgr2=0x55a828d68670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:22.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e57400640 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d68670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e57400640 1 -- 172.21.15.27:0/3438945704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a828dd1250 con 0x55a828979490 2024-09-21T07:23:22.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e57400640 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828979490 0x55a828d67bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3e48004e40 tx=0x7f3e48005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e60a4ad2c3471672 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:22.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e41600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:22.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e41600640 1 -- 172.21.15.27:0/3438945704 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48040040 con 0x55a828979490 2024-09-21T07:23:22.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e48004b70 con 0x55a828979490 2024-09-21T07:23:22.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.829+0000 7f3e41600640 1 -- 172.21.15.27:0/3438945704 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f3e480051b0 con 0x55a828979490 2024-09-21T07:23:22.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e41600640 1 -- 172.21.15.27:0/3438945704 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e48071050 con 0x55a828979490 2024-09-21T07:23:22.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 10 client.28534 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:22.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a828dd24f0 con 0x55a828979490 2024-09-21T07:23:22.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e41600640 1 -- 172.21.15.27:0/3438945704 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e48004b70 con 0x55a828979490 2024-09-21T07:23:22.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e41600640 1 -- 172.21.15.27:0/3438945704 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3e4806c030 con 0x55a828979490 2024-09-21T07:23:22.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 10 client.28534 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:22.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 10 client.28534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:22.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 20 client.28534 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:22.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828936080 0x55a828dd4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:22.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 4 client.28534 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:23:22.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.830+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a828cbd890 con 0x55a828936080 2024-09-21T07:23:22.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.831+0000 7f3e56a00640 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828936080 0x55a828dd4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:22.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.831+0000 7f3e56a00640 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828936080 0x55a828dd4a20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542e14d569fca329 server_cookie=ebc92fa19b8b2d2d in_seq=0 out_seq=0 2024-09-21T07:23:22.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.831+0000 7f3e41600640 10 client.28534 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e41600640 1 -- 172.21.15.27:0/3438945704 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a828cbd890 con 0x55a828936080 2024-09-21T07:23:22.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e41600640 10 client.28534 handle_command_reply: tid=0 2024-09-21T07:23:22.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 client.28534 shutdown 2024-09-21T07:23:22.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828979490 msgr2=0x55a828d67bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828979490 0x55a828d67bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3e48004e40 tx=0x7f3e48005980 comp rx=0 tx=0).stop 2024-09-21T07:23:22.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 shutdown_connections 2024-09-21T07:23:22.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a828936080 0x55a828dd4a20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8289a2350 0x55a828d68670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a828981670 0x55a828d68130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 --2- 172.21.15.27:0/3438945704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a828979490 0x55a828d67bf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e41600640 0 client.28534 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.833+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 >> 172.21.15.27:0/3438945704 conn(0x55a828996740 msgr2=0x55a828d4ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.834+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 shutdown_connections 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.834+0000 7f3e65009740 1 -- 172.21.15.27:0/3438945704 wait complete. 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.834+0000 7f3e65009740 20 client.28534 trim_cache size 0 max 16384 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "f52934f5-6079-4ed1-a0f3-77cab733e4d5", 2024-09-21T07:23:22.841 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:23:22.842 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:23:22.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e3c0b6680 msgr2=0x7f3e3c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3e440058c0 tx=0x7f3e44005850 comp rx=0 tx=0).stop 2024-09-21T07:23:22.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 msgr2=0x7f3e581a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 0x7f3e581a6300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000eb10 tx=0x7f3e5000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 shutdown_connections 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e58114740 0x7f3e581a6300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58078290 0x7f3e581a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 --2- 172.21.15.27:0/1662068916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e58077940 0x7f3e581a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 >> 172.21.15.27:0/1662068916 conn(0x7f3e58101810 msgr2=0x7f3e581167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 shutdown_connections 2024-09-21T07:23:22.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:22.835+0000 7f3e5e400640 1 -- 172.21.15.27:0/1662068916 wait complete. 2024-09-21T07:23:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:22 smithi027 ceph-mon[30779]: pgmap v698: 129 pgs: 129 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-21T07:23:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:22 smithi153 ceph-mon[34747]: pgmap v698: 129 pgs: 129 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-21T07:23:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:22 smithi119 ceph-mon[34519]: pgmap v698: 129 pgs: 129 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-21T07:23:23.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:23:24.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:23 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:23:24.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:23 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:23:24.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:23 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:23:24.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:23 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:23:24.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:23 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:23:24.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:23 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:23:24.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:23 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:23:24.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:23 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:23:24.248 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '68c6c9a7', quiesce took 6.2 seconds. Will release after: 18.3 2024-09-21T07:23:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:23 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:23:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:23 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:23:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:23 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:23:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:23 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:23:24.344 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:25.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.033+0000 7fea03000640 1 -- 172.21.15.27:0/1162161053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc1039b0 msgr2=0x7fe9fc103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.033+0000 7fea03000640 1 --2- 172.21.15.27:0/1162161053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc1039b0 0x7fe9fc103dc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009900 tx=0x7fe9f002a410 comp rx=0 tx=0).stop 2024-09-21T07:23:25.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.034+0000 7fea03000640 1 -- 172.21.15.27:0/1162161053 shutdown_connections 2024-09-21T07:23:25.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.034+0000 7fea03000640 1 --2- 172.21.15.27:0/1162161053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9fc109c40 0x7fe9fc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.034+0000 7fea03000640 1 --2- 172.21.15.27:0/1162161053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9fc1092f0 0x7fe9fc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.034+0000 7fea03000640 1 --2- 172.21.15.27:0/1162161053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc1039b0 0x7fe9fc103dc0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.034+0000 7fea03000640 1 -- 172.21.15.27:0/1162161053 >> 172.21.15.27:0/1162161053 conn(0x7fe9fc0ff440 msgr2=0x7fe9fc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:25.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.034+0000 7fea03000640 1 -- 172.21.15.27:0/1162161053 shutdown_connections 2024-09-21T07:23:25.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.034+0000 7fea03000640 1 -- 172.21.15.27:0/1162161053 wait complete. 2024-09-21T07:23:25.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 Processor -- start 2024-09-21T07:23:25.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 -- start start 2024-09-21T07:23:25.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9fc1039b0 0x7fe9fc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9fc1092f0 0x7fe9fc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc109c40 0x7fe9fc19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc113900 con 0x7fe9fc109c40 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc113780 con 0x7fe9fc1039b0 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.035+0000 7fea03000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc113a80 con 0x7fe9fc1092f0 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc109c40 0x7fe9fc19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc109c40 0x7fe9fc19fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51776/0 (socket says 172.21.15.27:51776) 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 -- 172.21.15.27:0/2970771852 learned_addr learned my addr 172.21.15.27:0/2970771852 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea01c00640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9fc1039b0 0x7fe9fc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea01200640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9fc1092f0 0x7fe9fc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 -- 172.21.15.27:0/2970771852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9fc1092f0 msgr2=0x7fe9fc19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9fc1092f0 0x7fe9fc19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 -- 172.21.15.27:0/2970771852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9fc1039b0 msgr2=0x7fe9fc19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9fc1039b0 0x7fe9fc19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.036+0000 7fea02600640 1 -- 172.21.15.27:0/2970771852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc1a02a0 con 0x7fe9fc109c40 2024-09-21T07:23:25.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.037+0000 7fea02600640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc109c40 0x7fe9fc19fb70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400eb10 tx=0x7fe9f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:25.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.037+0000 7fe9fa000640 1 -- 172.21.15.27:0/2970771852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4004b20 con 0x7fe9fc109c40 2024-09-21T07:23:25.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.037+0000 7fe9fa000640 1 -- 172.21.15.27:0/2970771852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f4004cc0 con 0x7fe9fc109c40 2024-09-21T07:23:25.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.037+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc1a0590 con 0x7fe9fc109c40 2024-09-21T07:23:25.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.037+0000 7fe9fa000640 1 -- 172.21.15.27:0/2970771852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4010670 con 0x7fe9fc109c40 2024-09-21T07:23:25.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.037+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1a6cf0 con 0x7fe9fc109c40 2024-09-21T07:23:25.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.040+0000 7fea0b24d740 1 Processor -- start 2024-09-21T07:23:25.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.040+0000 7fe9fa000640 1 -- 172.21.15.27:0/2970771852 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe9f4005000 con 0x7fe9fc109c40 2024-09-21T07:23:25.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea0b24d740 1 -- start start 2024-09-21T07:23:25.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051aa070 0x55bf05550cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051811b0 0x55bf05551230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05551770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea0b24d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf054c7f70 con 0x55bf05189390 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea0b24d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bf054b73a0 con 0x55bf051811b0 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea0b24d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf054bb1b0 con 0x55bf051aa070 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.041+0000 7fea01200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051aa070 0x55bf05550cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051aa070 0x55bf05550cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59018/0 (socket says 172.21.15.27:59018) 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01200640 1 -- 172.21.15.27:0/720444762 learned_addr learned my addr 172.21.15.27:0/720444762 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01200640 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05551770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01c00640 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051811b0 0x55bf05551230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01c00640 1 -- 172.21.15.27:0/720444762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051aa070 msgr2=0x55bf05550cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01c00640 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051aa070 0x55bf05550cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01c00640 1 -- 172.21.15.27:0/720444762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 msgr2=0x55bf05551770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01c00640 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05551770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.042+0000 7fea01c00640 1 -- 172.21.15.27:0/720444762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf05481c90 con 0x55bf051811b0 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.043+0000 7fe9fa000640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e40b6680 0x7fe9e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.043+0000 7fea01c00640 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051811b0 0x55bf05551230 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009230 tx=0x7fe9f0032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf809248c6362b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.043+0000 7fe9fa000640 1 -- 172.21.15.27:0/2970771852 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fe9f40e8080 con 0x7fe9fc109c40 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.043+0000 7fea01200640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e40b6680 0x7fe9e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.043+0000 7fe9eea00640 1 -- 172.21.15.27:0/720444762 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0040070 con 0x55bf051811b0 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.043+0000 7fe9eea00640 1 -- 172.21.15.27:0/720444762 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f0004030 con 0x55bf051811b0 2024-09-21T07:23:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.043+0000 7fe9eea00640 1 -- 172.21.15.27:0/720444762 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0035040 con 0x55bf051811b0 2024-09-21T07:23:25.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 -- 172.21.15.27:0/720444762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051811b0 msgr2=0x55bf05551230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051811b0 0x55bf05551230 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009230 tx=0x7fe9f0032b90 comp rx=0 tx=0).stop 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea01200640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e40b6680 0x7fe9e40b8b90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80058c0 tx=0x7fe9e8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 -- 172.21.15.27:0/720444762 shutdown_connections 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05551770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051811b0 0x55bf05551230 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 --2- 172.21.15.27:0/720444762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051aa070 0x55bf05550cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 -- 172.21.15.27:0/720444762 >> 172.21.15.27:0/720444762 conn(0x55bf0519e460 msgr2=0x55bf0554eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 -- 172.21.15.27:0/720444762 shutdown_connections 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.044+0000 7fea0b24d740 1 -- 172.21.15.27:0/720444762 wait complete. 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.045+0000 7fea0b24d740 1 -- start start 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.045+0000 7fea0b24d740 1 Processor -- start 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf051811b0 0x55bf05560cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf05189390 0x55bf055611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea0b24d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bf05318700 con 0x55bf051811b0 2024-09-21T07:23:25.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea0b24d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bf05452540 con 0x55bf051aa070 2024-09-21T07:23:25.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea0b24d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf055637a0 con 0x55bf05189390 2024-09-21T07:23:25.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea0b24d740 10 client.0 _ll_register_callbacks cb 0x55bf05462430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:25.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea01c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf051811b0 0x55bf05560cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea01c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf051811b0 0x55bf05560cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51796/0 (socket says 172.21.15.27:51796) 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea01c00640 1 -- 172.21.15.27:0/1485768445 learned_addr learned my addr 172.21.15.27:0/1485768445 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea02600640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.046+0000 7fea01200640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf05189390 0x55bf055611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea01c00640 1 -- 172.21.15.27:0/1485768445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf05189390 msgr2=0x55bf055611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea01c00640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf05189390 0x55bf055611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea01c00640 1 -- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 msgr2=0x55bf0556cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea01c00640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea01c00640 1 -- 172.21.15.27:0/1485768445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf05215650 con 0x55bf051811b0 2024-09-21T07:23:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea02600640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea01c00640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf051811b0 0x55bf05560cb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0032f60 tx=0x7fe9f0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ea72a40e70d09a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fe9e3400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fe9e3400640 1 -- 172.21.15.27:0/1485768445 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0035040 con 0x55bf051811b0 2024-09-21T07:23:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf05454b50 con 0x55bf051811b0 2024-09-21T07:23:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.047+0000 7fe9e3400640 1 -- 172.21.15.27:0/1485768445 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fe9f00318b0 con 0x55bf051811b0 2024-09-21T07:23:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fe9e3400640 1 -- 172.21.15.27:0/1485768445 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9f0056050 con 0x55bf051811b0 2024-09-21T07:23:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 10 client.18549 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf0546f910 con 0x55bf051811b0 2024-09-21T07:23:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fe9e3400640 1 -- 172.21.15.27:0/1485768445 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bf05454b50 con 0x55bf051811b0 2024-09-21T07:23:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fe9e3400640 1 -- 172.21.15.27:0/1485768445 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9f0004870 con 0x55bf051811b0 2024-09-21T07:23:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 10 client.18549 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 10 client.18549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 20 client.18549 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf051689f0 0x55bf05572980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 4 client.18549 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.048+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf054c9ec0 con 0x55bf051689f0 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.049+0000 7fea01200640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf051689f0 0x55bf05572980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.049+0000 7fea01200640 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf051689f0 0x55bf05572980 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a3e650083d4824 server_cookie=ec17d44e708b4e80 in_seq=0 out_seq=0 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.049+0000 7fe9e3400640 10 client.18549 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.050+0000 7fe9e3400640 1 -- 172.21.15.27:0/1485768445 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf054c9ec0 con 0x55bf051689f0 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.050+0000 7fe9e3400640 10 client.18549 handle_command_reply: tid=0 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.050+0000 7fea0b24d740 1 client.18549 shutdown 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.050+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf051811b0 msgr2=0x55bf05560cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.050+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf051811b0 0x55bf05560cb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0032f60 tx=0x7fe9f0032f90 comp rx=0 tx=0).stop 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 shutdown_connections 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf051689f0 0x55bf05572980 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf05189390 0x55bf055611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fe9e3400640 0 client.18549 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:25.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1485768445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf051811b0 0x55bf05560cb0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 >> 172.21.15.27:0/1485768445 conn(0x55bf0519e460 msgr2=0x55bf0554d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:25.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 shutdown_connections 2024-09-21T07:23:25.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 1 -- 172.21.15.27:0/1485768445 wait complete. 2024-09-21T07:23:25.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.051+0000 7fea0b24d740 20 client.18549 trim_cache size 0 max 16384 2024-09-21T07:23:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.069+0000 7fea0b24d740 1 Processor -- start 2024-09-21T07:23:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.069+0000 7fea0b24d740 1 -- start start 2024-09-21T07:23:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf0554d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf0556ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea0b24d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1b13e0 con 0x55bf05189390 2024-09-21T07:23:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea0b24d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1aecb0 con 0x55bf051aa070 2024-09-21T07:23:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea0b24d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1acb20 con 0x55bf051811b0 2024-09-21T07:23:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea01200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf0556ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea01200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf0556ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51806/0 (socket says 172.21.15.27:51806) 2024-09-21T07:23:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea01200640 1 -- 172.21.15.27:0/1179295183 learned_addr learned my addr 172.21.15.27:0/1179295183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea02600640 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.070+0000 7fea01c00640 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf0554d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fea01200640 1 -- 172.21.15.27:0/1179295183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 msgr2=0x55bf0554d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fea01200640 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf0554d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fea01200640 1 -- 172.21.15.27:0/1179295183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 msgr2=0x55bf0556cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fea01200640 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fea01200640 1 -- 172.21.15.27:0/1179295183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0555fa10 con 0x55bf05189390 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fea01c00640 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf0554d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fea01200640 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf0556ca40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x55bf055637a0 tx=0x7fe9e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bf8f66fdc4e172 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fe9ee000640 1 -- 172.21.15.27:0/1179295183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8011cc0 con 0x55bf05189390 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fe9ee000640 1 -- 172.21.15.27:0/1179295183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e8011e60 con 0x55bf05189390 2024-09-21T07:23:25.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.071+0000 7fe9ee000640 1 -- 172.21.15.27:0/1179295183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8020790 con 0x55bf05189390 2024-09-21T07:23:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 -- 172.21.15.27:0/1179295183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 msgr2=0x55bf0556ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf0556ca40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x55bf055637a0 tx=0x7fe9e80046c0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 -- 172.21.15.27:0/1179295183 shutdown_connections 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf0556cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf0556ca40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 --2- 172.21.15.27:0/1179295183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf0554d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 -- 172.21.15.27:0/1179295183 >> 172.21.15.27:0/1179295183 conn(0x55bf0519e460 msgr2=0x55bf0554d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.072+0000 7fea0b24d740 1 -- 172.21.15.27:0/1179295183 shutdown_connections 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.073+0000 7fea0b24d740 1 -- 172.21.15.27:0/1179295183 wait complete. 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.073+0000 7fea0b24d740 1 -- start start 2024-09-21T07:23:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.073+0000 7fea0b24d740 1 Processor -- start 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf055718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05571e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea0b24d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf05572370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea0b24d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1ad170 con 0x55bf05189390 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea0b24d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1ae5e0 con 0x55bf051aa070 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea0b24d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1ad840 con 0x55bf051811b0 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea0b24d740 10 client.0 _ll_register_callbacks cb 0x7fe9f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea01200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05571e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea01200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05571e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51814/0 (socket says 172.21.15.27:51814) 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea01200640 1 -- 172.21.15.27:0/2593347026 learned_addr learned my addr 172.21.15.27:0/2593347026 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea01c00640 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf055718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.074+0000 7fea02600640 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf05572370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fea01200640 1 -- 172.21.15.27:0/2593347026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 msgr2=0x55bf055718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fea01200640 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf055718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fea01200640 1 -- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 msgr2=0x55bf05572370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fea01200640 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf05572370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fea01200640 1 -- 172.21.15.27:0/2593347026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf055dc0f0 con 0x55bf05189390 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fea01200640 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05571e30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8011e00 tx=0x7fe9e8011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618b624105935617 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fe9e3400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fe9e3400640 1 -- 172.21.15.27:0/2593347026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8005050 con 0x55bf05189390 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fe9e3400640 1 -- 172.21.15.27:0/2593347026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fe9e80051f0 con 0x55bf05189390 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.075+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9e8011e60 con 0x55bf05189390 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fe9e3400640 1 -- 172.21.15.27:0/2593347026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9e80053f0 con 0x55bf05189390 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 10 client.18555 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:25.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf055dd2e0 con 0x55bf05189390 2024-09-21T07:23:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fe9e3400640 1 -- 172.21.15.27:0/2593347026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e8011e60 con 0x55bf05189390 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fe9e3400640 1 -- 172.21.15.27:0/2593347026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9e803c030 con 0x55bf05189390 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 10 client.18555 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 10 client.18555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 20 client.18555 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf0513dda0 0x55bf055dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 4 client.18555 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.076+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf054c5970 con 0x55bf0513dda0 2024-09-21T07:23:25.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.077+0000 7fea01c00640 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf0513dda0 0x55bf055dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.077+0000 7fea01c00640 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf0513dda0 0x55bf055dfa40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8483f4ad545b947a server_cookie=1d4c135565e2b6eb in_seq=0 out_seq=0 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.077+0000 7fe9e3400640 10 client.18555 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fe9e3400640 1 -- 172.21.15.27:0/2593347026 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55bf054c5970 con 0x55bf0513dda0 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fe9e3400640 10 client.18555 handle_command_reply: tid=0 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 client.18555 shutdown 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 msgr2=0x55bf05571e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05571e30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8011e00 tx=0x7fe9e8011040 comp rx=0 tx=0).stop 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 shutdown_connections 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bf0513dda0 0x55bf055dfa40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf051aa070 0x55bf05572370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf05189390 0x55bf05571e30 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 --2- 172.21.15.27:0/2593347026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf051811b0 0x55bf055718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 >> 172.21.15.27:0/2593347026 conn(0x55bf0519e460 msgr2=0x55bf05551d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fe9e3400640 0 client.18555 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 shutdown_connections 2024-09-21T07:23:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 1 -- 172.21.15.27:0/2593347026 wait complete. 2024-09-21T07:23:25.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.078+0000 7fea0b24d740 20 client.18555 trim_cache size 0 max 16384 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "f52934f5-6079-4ed1-a0f3-77cab733e4d5", 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:23:25.084 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.085 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.086 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.087 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:23:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e40b6680 msgr2=0x7fe9e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e40b6680 0x7fe9e40b8b90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80058c0 tx=0x7fe9e8005f90 comp rx=0 tx=0).stop 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc109c40 msgr2=0x7fe9fc19fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc109c40 0x7fe9fc19fb70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400eb10 tx=0x7fe9f400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 shutdown_connections 2024-09-21T07:23:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9fc1092f0 0x7fe9fc19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe9e40b6680 0x7fe9e40b8b90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9fc109c40 0x7fe9fc19fb70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 --2- 172.21.15.27:0/2970771852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9fc1039b0 0x7fe9fc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:25.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 >> 172.21.15.27:0/2970771852 conn(0x7fe9fc0ff440 msgr2=0x7fe9fc101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:25.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 shutdown_connections 2024-09-21T07:23:25.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:25.079+0000 7fea03000640 1 -- 172.21.15.27:0/2970771852 wait complete. 2024-09-21T07:23:25.102 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:24 smithi027 ceph-mon[30779]: pgmap v699: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-21T07:23:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:24 smithi153 ceph-mon[34747]: pgmap v699: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-21T07:23:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:24 smithi119 ceph-mon[34519]: pgmap v699: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-21T07:23:25.980 INFO:tasks.cephfs.filesystem:scrub status for tag:f52934f5-6079-4ed1-a0f3-77cab733e4d5 - {'path': '/', 'tag': 'f52934f5-6079-4ed1-a0f3-77cab733e4d5', 'options': 'recursive,force'} 2024-09-21T07:23:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:26 smithi153 ceph-mon[34747]: pgmap v700: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-21T07:23:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:26 smithi119 ceph-mon[34519]: pgmap v700: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-21T07:23:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:26 smithi027 ceph-mon[30779]: pgmap v700: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-09-21T07:23:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:28 smithi153 ceph-mon[34747]: pgmap v701: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T07:23:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:28 smithi119 ceph-mon[34519]: pgmap v701: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T07:23:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:28 smithi027 ceph-mon[30779]: pgmap v701: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T07:23:31.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:30 smithi027 ceph-mon[30779]: pgmap v702: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:23:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:30 smithi153 ceph-mon[34747]: pgmap v702: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:23:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:30 smithi119 ceph-mon[34519]: pgmap v702: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-21T07:23:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:32 smithi153 ceph-mon[34747]: pgmap v703: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-21T07:23:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:32 smithi119 ceph-mon[34519]: pgmap v703: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-21T07:23:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:32 smithi027 ceph-mon[30779]: pgmap v703: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-21T07:23:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 -- 172.21.15.27:0/3508212929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010bd80 msgr2=0x7f0b201125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 --2- 172.21.15.27:0/3508212929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010bd80 0x7f0b201125b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009be0 tx=0x7f0b1802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 -- 172.21.15.27:0/3508212929 shutdown_connections 2024-09-21T07:23:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 --2- 172.21.15.27:0/3508212929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010bd80 0x7f0b201125b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 --2- 172.21.15.27:0/3508212929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2010b430 0x7f0b2010b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 --2- 172.21.15.27:0/3508212929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20105b00 0x7f0b20105f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 -- 172.21.15.27:0/3508212929 >> 172.21.15.27:0/3508212929 conn(0x7f0b20101560 msgr2=0x7f0b201039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 -- 172.21.15.27:0/3508212929 shutdown_connections 2024-09-21T07:23:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.827+0000 7f0b1f200640 1 -- 172.21.15.27:0/3508212929 wait complete. 2024-09-21T07:23:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 Processor -- start 2024-09-21T07:23:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 -- start start 2024-09-21T07:23:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b20105b00 0x7f0b2019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010b430 0x7f0b2019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b2010bd80 0x7f0b201a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20115a40 con 0x7f0b2010bd80 2024-09-21T07:23:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201158c0 con 0x7f0b20105b00 2024-09-21T07:23:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.829+0000 7f0b1f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20115bc0 con 0x7f0b2010b430 2024-09-21T07:23:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b2010bd80 0x7f0b201a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b2010bd80 0x7f0b201a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37644/0 (socket says 172.21.15.27:37644) 2024-09-21T07:23:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b20105b00 0x7f0b2019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1e800640 1 -- 172.21.15.27:0/2746653630 learned_addr learned my addr 172.21.15.27:0/2746653630 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1d400640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010b430 0x7f0b2019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1de00640 1 -- 172.21.15.27:0/2746653630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010b430 msgr2=0x7f0b2019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1de00640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010b430 0x7f0b2019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1de00640 1 -- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b2010bd80 msgr2=0x7f0b201a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1de00640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b2010bd80 0x7f0b201a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1de00640 1 -- 172.21.15.27:0/2746653630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b201a2390 con 0x7f0b20105b00 2024-09-21T07:23:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1e800640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b2010bd80 0x7f0b201a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.830+0000 7f0b1de00640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b20105b00 0x7f0b2019c460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005fd0 tx=0x7f0b1000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.831+0000 7f0b16a00640 1 -- 172.21.15.27:0/2746653630 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000cd00 con 0x7f0b20105b00 2024-09-21T07:23:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.831+0000 7f0b16a00640 1 -- 172.21.15.27:0/2746653630 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b100035f0 con 0x7f0b20105b00 2024-09-21T07:23:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.831+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201a2680 con 0x7f0b20105b00 2024-09-21T07:23:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.831+0000 7f0b16a00640 1 -- 172.21.15.27:0/2746653630 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10010690 con 0x7f0b20105b00 2024-09-21T07:23:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.831+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a8e10 con 0x7f0b20105b00 2024-09-21T07:23:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.834+0000 7f0b2c066740 1 Processor -- start 2024-09-21T07:23:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.834+0000 7f0b2c066740 1 -- start start 2024-09-21T07:23:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72585fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725860350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b725860890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b2c066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b7257d56b0 con 0x55b72549a4d0 2024-09-21T07:23:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b2c066740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7257c4ae0 con 0x55b7254922f0 2024-09-21T07:23:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b2c066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b7257c88f0 con 0x55b7254bb1b0 2024-09-21T07:23:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b16a00640 1 -- 172.21.15.27:0/2746653630 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0b10003a80 con 0x7f0b20105b00 2024-09-21T07:23:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b725860890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72585fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b725860890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37660/0 (socket says 172.21.15.27:37660) 2024-09-21T07:23:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1d400640 1 -- 172.21.15.27:0/3546121176 learned_addr learned my addr 172.21.15.27:0/3546121176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72585fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36388/0 (socket says 172.21.15.27:36388) 2024-09-21T07:23:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1de00640 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725860350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1d400640 1 -- 172.21.15.27:0/3546121176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 msgr2=0x55b72585fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1d400640 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72585fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1d400640 1 -- 172.21.15.27:0/3546121176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 msgr2=0x55b725860350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.835+0000 7f0b1d400640 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725860350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.836+0000 7f0b1d400640 1 -- 172.21.15.27:0/3546121176 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b725791780 con 0x55b72549a4d0 2024-09-21T07:23:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.836+0000 7f0b1de00640 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725860350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:23:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.836+0000 7f0b1d400640 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b725860890 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009c20 tx=0x7f0b08011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded921d607180095 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.836+0000 7f0b0f400640 1 -- 172.21.15.27:0/3546121176 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004230 con 0x55b72549a4d0 2024-09-21T07:23:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.836+0000 7f0b0f400640 1 -- 172.21.15.27:0/3546121176 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b080043d0 con 0x55b72549a4d0 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.836+0000 7f0b0f400640 1 -- 172.21.15.27:0/3546121176 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004cd0 con 0x55b72549a4d0 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b16a00640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b040b6700 0x7f0b040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 -- 172.21.15.27:0/3546121176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 msgr2=0x55b725860890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b725860890 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009c20 tx=0x7f0b08011810 comp rx=0 tx=0).stop 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b16a00640 1 -- 172.21.15.27:0/2746653630 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f0b100e7ca0 con 0x7f0b20105b00 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 -- 172.21.15.27:0/3546121176 shutdown_connections 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b725860890 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725860350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 --2- 172.21.15.27:0/3546121176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72585fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 -- 172.21.15.27:0/3546121176 >> 172.21.15.27:0/3546121176 conn(0x55b7254af300 msgr2=0x55b72585dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b1e800640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b040b6700 0x7f0b040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 -- 172.21.15.27:0/3546121176 shutdown_connections 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.837+0000 7f0b2c066740 1 -- 172.21.15.27:0/3546121176 wait complete. 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.838+0000 7f0b2c066740 1 -- start start 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.838+0000 7f0b1e800640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b040b6700 0x7f0b040b8c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009370 tx=0x7f0b180093a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.838+0000 7f0b2c066740 1 Processor -- start 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b72586b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b72586b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72586be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b2c066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b7256fcab0 con 0x55b72549a4d0 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b2c066740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b72576b570 con 0x55b7254922f0 2024-09-21T07:23:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b2c066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b72586e590 con 0x55b7254bb1b0 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b2c066740 10 client.0 _ll_register_callbacks cb 0x55b72577a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b72586b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b72586b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b72586b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41066/0 (socket says 172.21.15.27:41066) 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 -- 172.21.15.27:0/1528340149 learned_addr learned my addr 172.21.15.27:0/1528340149 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1e800640 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72586be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 -- 172.21.15.27:0/1528340149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 msgr2=0x55b72586be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72586be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 -- 172.21.15.27:0/1528340149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 msgr2=0x55b72586b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b72586b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.839+0000 7f0b1de00640 1 -- 172.21.15.27:0/1528340149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b725526790 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b1de00640 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b72586b3f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c6a0 tx=0x7f0b100039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66dc5e7d800f3a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b03e00640 1 -- 172.21.15.27:0/1528340149 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100ad1f0 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b03e00640 1 -- 172.21.15.27:0/1528340149 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f0b100a9240 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b080043d0 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b03e00640 1 -- 172.21.15.27:0/1528340149 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b10009510 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b2c066740 10 client.28595 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.840+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b72559f620 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b03e00640 1 -- 172.21.15.27:0/1528340149 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b080043d0 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b03e00640 1 -- 172.21.15.27:0/1528340149 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0b100b33e0 con 0x55b7254922f0 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b2c066740 10 client.28595 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b2c066740 10 client.28595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b2c066740 20 client.28595 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b725479d80 0x55b72587d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b2c066740 4 client.28595 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7257d7600 con 0x55b725479d80 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.841+0000 7f0b1d400640 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b725479d80 0x55b72587d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.842+0000 7f0b1d400640 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b725479d80 0x55b72587d5b0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be67ccf227e42619 server_cookie=9b1744b53ce52798 in_seq=0 out_seq=0 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.842+0000 7f0b03e00640 10 client.28595 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.842+0000 7f0b03e00640 1 -- 172.21.15.27:0/1528340149 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7257d7600 con 0x55b725479d80 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.842+0000 7f0b03e00640 10 client.28595 handle_command_reply: tid=0 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 client.28595 shutdown 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 msgr2=0x55b72586b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b72586b3f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c6a0 tx=0x7f0b100039d0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 shutdown_connections 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b725479d80 0x55b72587d5b0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b72586be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b72549a4d0 0x55b72586b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b03e00640 0 client.28595 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1528340149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b72586b3f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 >> 172.21.15.27:0/1528340149 conn(0x55b7254af300 msgr2=0x55b725860e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 shutdown_connections 2024-09-21T07:23:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 1 -- 172.21.15.27:0/1528340149 wait complete. 2024-09-21T07:23:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.843+0000 7f0b2c066740 20 client.28595 trim_cache size 0 max 16384 2024-09-21T07:23:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 Processor -- start 2024-09-21T07:23:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 -- start start 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254922f0 0x55b72585cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b72549a4d0 0x55b725861550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b725882d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201b5700 con 0x55b7254922f0 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201b2fd0 con 0x55b72549a4d0 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b2c066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201b0e40 con 0x55b7254bb1b0 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254922f0 0x55b72585cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254922f0 0x55b72585cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37680/0 (socket says 172.21.15.27:37680) 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b1de00640 1 -- 172.21.15.27:0/949046304 learned_addr learned my addr 172.21.15.27:0/949046304 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b1d400640 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b72549a4d0 0x55b725861550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.859+0000 7f0b1e800640 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b725882d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b1de00640 1 -- 172.21.15.27:0/949046304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 msgr2=0x55b725882d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b1de00640 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b725882d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b1de00640 1 -- 172.21.15.27:0/949046304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b72549a4d0 msgr2=0x55b725861550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b1de00640 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b72549a4d0 0x55b725861550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b1de00640 1 -- 172.21.15.27:0/949046304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72585e680 con 0x55b7254922f0 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b1e800640 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b725882d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b1de00640 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254922f0 0x55b72585cd20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0b100aa950 tx=0x7f0b1000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52288acdfec0e46b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b0ea00640 1 -- 172.21.15.27:0/949046304 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100ad1f0 con 0x55b7254922f0 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b0ea00640 1 -- 172.21.15.27:0/949046304 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b100adde0 con 0x55b7254922f0 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.860+0000 7f0b0ea00640 1 -- 172.21.15.27:0/949046304 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100b6640 con 0x55b7254922f0 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 -- 172.21.15.27:0/949046304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254922f0 msgr2=0x55b72585cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254922f0 0x55b72585cd20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0b100aa950 tx=0x7f0b1000c7c0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 -- 172.21.15.27:0/949046304 shutdown_connections 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7254bb1b0 0x55b725882d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b72549a4d0 0x55b725861550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 --2- 172.21.15.27:0/949046304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254922f0 0x55b72585cd20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 -- 172.21.15.27:0/949046304 >> 172.21.15.27:0/949046304 conn(0x55b7254af300 msgr2=0x55b725860e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 -- 172.21.15.27:0/949046304 shutdown_connections 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.861+0000 7f0b2c066740 1 -- 172.21.15.27:0/949046304 wait complete. 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 -- start start 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 Processor -- start 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725871be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b72549a4d0 0x55b725872120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254bb1b0 0x55b7258e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201b1490 con 0x55b7254bb1b0 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201b2900 con 0x55b7254922f0 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201b1b60 con 0x55b72549a4d0 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b2c066740 10 client.0 _ll_register_callbacks cb 0x7f0b100046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b72549a4d0 0x55b725872120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.862+0000 7f0b1e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254bb1b0 0x55b7258e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725871be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b72549a4d0 0x55b725872120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36418/0 (socket says 172.21.15.27:36418) 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 -- 172.21.15.27:0/1058226798 learned_addr learned my addr 172.21.15.27:0/1058226798 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 -- 172.21.15.27:0/1058226798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 msgr2=0x55b725871be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725871be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 -- 172.21.15.27:0/1058226798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254bb1b0 msgr2=0x55b7258e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254bb1b0 0x55b7258e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 -- 172.21.15.27:0/1058226798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7258e74c0 con 0x55b72549a4d0 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b1d400640 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b72549a4d0 0x55b725872120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0b080099b0 tx=0x7f0b08004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11f2548cb150aa2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.863+0000 7f0b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.864+0000 7f0b03e00640 1 -- 172.21.15.27:0/1058226798 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0801e040 con 0x55b72549a4d0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.864+0000 7f0b03e00640 1 -- 172.21.15.27:0/1058226798 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f0b08026850 con 0x55b72549a4d0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.864+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b100adde0 con 0x55b72549a4d0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.864+0000 7f0b03e00640 1 -- 172.21.15.27:0/1058226798 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b0803d050 con 0x55b72549a4d0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.864+0000 7f0b2c066740 10 client.28570 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.864+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7258e85a0 con 0x55b72549a4d0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.864+0000 7f0b03e00640 1 -- 172.21.15.27:0/1058226798 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b100adde0 con 0x55b72549a4d0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b03e00640 1 -- 172.21.15.27:0/1058226798 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0b08024040 con 0x55b72549a4d0 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b2c066740 10 client.28570 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b2c066740 10 client.28570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b2c066740 20 client.28570 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b72544f1c0 0x55b7258eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b2c066740 4 client.28570 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:23:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7257d30b0 con 0x55b72544f1c0 2024-09-21T07:23:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.865+0000 7f0b1de00640 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b72544f1c0 0x55b7258eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.866+0000 7f0b1de00640 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b72544f1c0 0x55b7258eac50 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8bed72b2f4891d5 server_cookie=8e548d0a689f8213 in_seq=0 out_seq=0 2024-09-21T07:23:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.866+0000 7f0b03e00640 10 client.28570 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.877+0000 7f0b03e00640 1 -- 172.21.15.27:0/1058226798 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7257d30b0 con 0x55b72544f1c0 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.877+0000 7f0b03e00640 10 client.28570 handle_command_reply: tid=0 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 client.28570 shutdown 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b72549a4d0 msgr2=0x55b725872120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b72549a4d0 0x55b725872120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0b080099b0 tx=0x7f0b08004690 comp rx=0 tx=0).stop 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 shutdown_connections 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b72544f1c0 0x55b7258eac50 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b72549a4d0 0x55b725872120 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7254bb1b0 0x55b7258e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 --2- 172.21.15.27:0/1058226798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7254922f0 0x55b725871be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b03e00640 0 client.28570 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 >> 172.21.15.27:0/1058226798 conn(0x55b7254af300 msgr2=0x55b72585caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 shutdown_connections 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 1 -- 172.21.15.27:0/1058226798 wait complete. 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.878+0000 7f0b2c066740 20 client.28570 trim_cache size 0 max 16384 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:23:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:23:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:23:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b040b6700 msgr2=0x7f0b040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b040b6700 0x7f0b040b8c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009370 tx=0x7f0b180093a0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b20105b00 msgr2=0x7f0b2019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b20105b00 0x7f0b2019c460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005fd0 tx=0x7f0b1000ee50 comp rx=0 tx=0).stop 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 shutdown_connections 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2010b430 0x7f0b2019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b040b6700 0x7f0b040b8c10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b2010bd80 0x7f0b201a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 --2- 172.21.15.27:0/2746653630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b20105b00 0x7f0b2019c460 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 >> 172.21.15.27:0/2746653630 conn(0x7f0b20101560 msgr2=0x7f0b2010aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 shutdown_connections 2024-09-21T07:23:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:34.879+0000 7f0b1f200640 1 -- 172.21.15.27:0/2746653630 wait complete. 2024-09-21T07:23:35.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:34 smithi153 ceph-mon[34747]: pgmap v704: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-21T07:23:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:34 smithi119 ceph-mon[34519]: pgmap v704: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-21T07:23:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:34 smithi027 ceph-mon[30779]: pgmap v704: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-21T07:23:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:36 smithi153 ceph-mon[34747]: pgmap v705: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-21T07:23:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:36 smithi119 ceph-mon[34519]: pgmap v705: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-21T07:23:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:36 smithi027 ceph-mon[30779]: pgmap v705: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-21T07:23:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:38 smithi153 ceph-mon[34747]: pgmap v706: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-21T07:23:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:38 smithi119 ceph-mon[34519]: pgmap v706: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-21T07:23:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:38 smithi027 ceph-mon[30779]: pgmap v706: 129 pgs: 129 active+clean; 2.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-21T07:23:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:40 smithi153 ceph-mon[34747]: pgmap v707: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2024-09-21T07:23:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:40 smithi119 ceph-mon[34519]: pgmap v707: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2024-09-21T07:23:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:40 smithi027 ceph-mon[30779]: pgmap v707: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2024-09-21T07:23:42.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:23:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:42 smithi153 ceph-mon[34747]: pgmap v708: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-21T07:23:43.263 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:42 smithi027 ceph-mon[30779]: pgmap v708: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-21T07:23:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:42 smithi119 ceph-mon[34519]: pgmap v708: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-21T07:23:43.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.609+0000 7f8f19000640 1 -- 172.21.15.27:0/1986886078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410be40 msgr2=0x7f8f14112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:43.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.609+0000 7f8f19000640 1 --2- 172.21.15.27:0/1986886078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410be40 0x7f8f14112670 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009ca0 tx=0x7f8f0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.610+0000 7f8f19000640 1 -- 172.21.15.27:0/1986886078 shutdown_connections 2024-09-21T07:23:43.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.610+0000 7f8f19000640 1 --2- 172.21.15.27:0/1986886078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1410b4f0 0x7f8f1410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.610+0000 7f8f19000640 1 --2- 172.21.15.27:0/1986886078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410be40 0x7f8f14112670 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.610+0000 7f8f19000640 1 --2- 172.21.15.27:0/1986886078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 0x7f8f14105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.610+0000 7f8f19000640 1 -- 172.21.15.27:0/1986886078 >> 172.21.15.27:0/1986886078 conn(0x7f8f14101680 msgr2=0x7f8f14103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:43.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.610+0000 7f8f19000640 1 -- 172.21.15.27:0/1986886078 shutdown_connections 2024-09-21T07:23:43.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.610+0000 7f8f19000640 1 -- 172.21.15.27:0/1986886078 wait complete. 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 Processor -- start 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 -- start start 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 0x7f8f1419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410b4f0 0x7f8f1419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1410be40 0x7f8f141a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14115b50 con 0x7f8f1410b4f0 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141159d0 con 0x7f8f14105bb0 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.611+0000 7f8f19000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14115cd0 con 0x7f8f1410be40 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410b4f0 0x7f8f1419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 0x7f8f1419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1410be40 0x7f8f141a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:43.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 0x7f8f1419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33406/0 (socket says 172.21.15.27:33406) 2024-09-21T07:23:43.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 -- 172.21.15.27:0/1858002756 learned_addr learned my addr 172.21.15.27:0/1858002756 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:43.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410b4f0 0x7f8f1419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33040/0 (socket says 172.21.15.27:33040) 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 -- 172.21.15.27:0/1858002756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1410be40 msgr2=0x7f8f141a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1410be40 0x7f8f141a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 -- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410b4f0 msgr2=0x7f8f1419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410b4f0 0x7f8f1419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13400640 1 -- 172.21.15.27:0/1858002756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f141a2430 con 0x7f8f14105bb0 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.612+0000 7f8f13e00640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1410be40 0x7f8f141a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.613+0000 7f8f13400640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 0x7f8f1419c4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400e9f0 tx=0x7f8f0400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.613+0000 7f8f10c00640 1 -- 172.21.15.27:0/1858002756 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400cd80 con 0x7f8f14105bb0 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.613+0000 7f8f10c00640 1 -- 172.21.15.27:0/1858002756 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f040108e0 con 0x7f8f14105bb0 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.613+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f141a2720 con 0x7f8f14105bb0 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.613+0000 7f8f10c00640 1 -- 172.21.15.27:0/1858002756 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04018980 con 0x7f8f14105bb0 2024-09-21T07:23:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.613+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a8e80 con 0x7f8f14105bb0 2024-09-21T07:23:43.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.614+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f14105fc0 con 0x7f8f14105bb0 2024-09-21T07:23:43.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.617+0000 7f8f10c00640 1 -- 172.21.15.27:0/1858002756 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8f04010430 con 0x7f8f14105bb0 2024-09-21T07:23:43.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.618+0000 7f8f10c00640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8ef80b6680 0x7f8ef80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:43.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.619+0000 7f8f10c00640 1 -- 172.21.15.27:0/1858002756 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f8f040e82e0 con 0x7f8f14105bb0 2024-09-21T07:23:43.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.619+0000 7f8f0be00640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8ef80b6680 0x7f8ef80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:43.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.619+0000 7f8f0be00640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8ef80b6680 0x7f8ef80b8b90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f14104f40 tx=0x7f8efc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:43.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.620+0000 7f8f10c00640 1 -- 172.21.15.27:0/1858002756 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f040b2800 con 0x7f8f14105bb0 2024-09-21T07:23:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.860+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f1419d5f0 con 0x7f8f14105bb0 2024-09-21T07:23:43.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.861+0000 7f8f10c00640 1 -- 172.21.15.27:0/1858002756 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8f040b7740 con 0x7f8f14105bb0 2024-09-21T07:23:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:23:43.863 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:23:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:23:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8ef80b6680 msgr2=0x7f8ef80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8ef80b6680 0x7f8ef80b8b90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f14104f40 tx=0x7f8efc005f90 comp rx=0 tx=0).stop 2024-09-21T07:23:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 msgr2=0x7f8f1419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 0x7f8f1419c4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400e9f0 tx=0x7f8f0400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 shutdown_connections 2024-09-21T07:23:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1410be40 0x7f8f141a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8ef80b6680 0x7f8ef80b8b90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1410b4f0 0x7f8f1419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 --2- 172.21.15.27:0/1858002756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f14105bb0 0x7f8f1419c4c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 >> 172.21.15.27:0/1858002756 conn(0x7f8f14101680 msgr2=0x7f8f1410ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 shutdown_connections 2024-09-21T07:23:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:43.864+0000 7f8f19000640 1 -- 172.21.15.27:0/1858002756 wait complete. 2024-09-21T07:23:44.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:43 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1858002756' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:44.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:43 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1858002756' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:44.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:43 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1858002756' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:23:44.771 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 68c6c9a7 --release --await' 2024-09-21T07:23:44.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 68c6c9a7 --release --await 2024-09-21T07:23:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:44 smithi027 ceph-mon[30779]: pgmap v709: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-21T07:23:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:44 smithi153 ceph-mon[34747]: pgmap v709: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-21T07:23:45.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:44 smithi119 ceph-mon[34519]: pgmap v709: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-21T07:23:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 -- 172.21.15.27:0/1740444499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010b4d0 msgr2=0x7f762010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 --2- 172.21.15.27:0/1740444499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010b4d0 0x7f762010b8e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7608009900 tx=0x7f760802a410 comp rx=0 tx=0).stop 2024-09-21T07:23:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 -- 172.21.15.27:0/1740444499 shutdown_connections 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 --2- 172.21.15.27:0/1740444499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010be20 0x7f7620112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 --2- 172.21.15.27:0/1740444499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010b4d0 0x7f762010b8e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 --2- 172.21.15.27:0/1740444499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7620105b90 0x7f7620105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 -- 172.21.15.27:0/1740444499 >> 172.21.15.27:0/1740444499 conn(0x7f76201015e0 msgr2=0x7f7620103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 -- 172.21.15.27:0/1740444499 shutdown_connections 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.786+0000 7f761fe00640 1 -- 172.21.15.27:0/1740444499 wait complete. 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 Processor -- start 2024-09-21T07:23:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 -- start start 2024-09-21T07:23:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7620105b90 0x7f762019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010b4d0 0x7f762019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010be20 0x7f76201a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7620115ae0 con 0x7f762010be20 2024-09-21T07:23:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7620115960 con 0x7f762010b4d0 2024-09-21T07:23:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.788+0000 7f761fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7620115c60 con 0x7f7620105b90 2024-09-21T07:23:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010b4d0 0x7f762019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010b4d0 0x7f762019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33430/0 (socket says 172.21.15.27:33430) 2024-09-21T07:23:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 -- 172.21.15.27:0/4023128001 learned_addr learned my addr 172.21.15.27:0/4023128001 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f761ea00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7620105b90 0x7f762019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f761f400640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010be20 0x7f76201a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 -- 172.21.15.27:0/4023128001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7620105b90 msgr2=0x7f762019c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7620105b90 0x7f762019c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 -- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010be20 msgr2=0x7f76201a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010be20 0x7f76201a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f7617e00640 1 -- 172.21.15.27:0/4023128001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76201a24a0 con 0x7f762010b4d0 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.789+0000 7f761ea00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7620105b90 0x7f762019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.790+0000 7f7617e00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010b4d0 0x7f762019cad0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7608006010 tx=0x7f7608004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.790+0000 7f7624a00640 1 -- 172.21.15.27:0/4023128001 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760803f030 con 0x7f762010b4d0 2024-09-21T07:23:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.790+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76201a2730 con 0x7f762010b4d0 2024-09-21T07:23:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.790+0000 7f7624a00640 1 -- 172.21.15.27:0/4023128001 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608003aa0 con 0x7f762010b4d0 2024-09-21T07:23:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.790+0000 7f7624a00640 1 -- 172.21.15.27:0/4023128001 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608035070 con 0x7f762010b4d0 2024-09-21T07:23:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.790+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76201a8ef0 con 0x7f762010b4d0 2024-09-21T07:23:45.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.793+0000 7f762ca6a740 1 Processor -- start 2024-09-21T07:23:45.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.793+0000 7f762ca6a740 1 -- start start 2024-09-21T07:23:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516bce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585167fe660 0x558516bce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516bcedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f762ca6a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558516b45290 con 0x5585167fe660 2024-09-21T07:23:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f762ca6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558516b346c0 con 0x558516827520 2024-09-21T07:23:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f762ca6a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558516b384d0 con 0x558516806840 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f761ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585167fe660 0x558516bce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f761ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585167fe660 0x558516bce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33068/0 (socket says 172.21.15.27:33068) 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f761f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516bce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f761ea00640 1 -- 172.21.15.27:0/297247186 learned_addr learned my addr 172.21.15.27:0/297247186 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.794+0000 7f7624a00640 1 -- 172.21.15.27:0/4023128001 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7608042030 con 0x7f762010b4d0 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f761f400640 1 -- 172.21.15.27:0/297247186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 msgr2=0x558516bcedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f761f400640 1 --2- 172.21.15.27:0/297247186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516bcedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f761f400640 1 -- 172.21.15.27:0/297247186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585167fe660 msgr2=0x558516bce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f761f400640 1 --2- 172.21.15.27:0/297247186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585167fe660 0x558516bce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f761f400640 1 -- 172.21.15.27:0/297247186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558516ac1460 con 0x558516827520 2024-09-21T07:23:45.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f761f400640 1 --2- 172.21.15.27:0/297247186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516bce340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7618009f50 tx=0x7f7618003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160e514033b7d66c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f760fe00640 1 -- 172.21.15.27:0/297247186 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618011440 con 0x558516827520 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f760fe00640 1 -- 172.21.15.27:0/297247186 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7618015030 con 0x558516827520 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.795+0000 7f760fe00640 1 -- 172.21.15.27:0/297247186 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180108e0 con 0x558516827520 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f7624a00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76040b6700 0x7f76040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f762ca6a740 1 -- 172.21.15.27:0/297247186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 msgr2=0x558516bce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f762ca6a740 1 --2- 172.21.15.27:0/297247186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516bce340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7618009f50 tx=0x7f7618003e00 comp rx=0 tx=0).stop 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f7624a00640 1 -- 172.21.15.27:0/4023128001 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f7608107fd0 con 0x7f762010b4d0 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f762ca6a740 1 -- 172.21.15.27:0/297247186 shutdown_connections 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f762ca6a740 1 --2- 172.21.15.27:0/297247186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516bcedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f762ca6a740 1 --2- 172.21.15.27:0/297247186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5585167fe660 0x558516bce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.796+0000 7f762ca6a740 1 --2- 172.21.15.27:0/297247186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516bce340 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.797+0000 7f762ca6a740 1 -- 172.21.15.27:0/297247186 >> 172.21.15.27:0/297247186 conn(0x55851681b910 msgr2=0x558516bcc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.797+0000 7f761ea00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76040b6700 0x7f76040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.797+0000 7f762ca6a740 1 -- 172.21.15.27:0/297247186 shutdown_connections 2024-09-21T07:23:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.797+0000 7f762ca6a740 1 -- 172.21.15.27:0/297247186 wait complete. 2024-09-21T07:23:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f761ea00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76040b6700 0x7f76040b8c10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x558516bccfc0 tx=0x7f7610005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 -- start start 2024-09-21T07:23:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 Processor -- start 2024-09-21T07:23:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585167fe660 0x558516bd9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516806840 0x558516bda530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516be5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558516995bb0 con 0x558516806840 2024-09-21T07:23:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558516acf6c0 con 0x558516827520 2024-09-21T07:23:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.798+0000 7f762ca6a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558516bdcb20 con 0x5585167fe660 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f762ca6a740 10 client.0 _ll_register_callbacks cb 0x558516aea490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516be5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f7617e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516806840 0x558516bda530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516be5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33440/0 (socket says 172.21.15.27:33440) 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 -- 172.21.15.27:0/4232841378 learned_addr learned my addr 172.21.15.27:0/4232841378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761ea00640 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585167fe660 0x558516bd9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 -- 172.21.15.27:0/4232841378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585167fe660 msgr2=0x558516bd9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585167fe660 0x558516bd9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 -- 172.21.15.27:0/4232841378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516806840 msgr2=0x558516bda530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516806840 0x558516bda530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f761f400640 1 -- 172.21.15.27:0/4232841378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558516892b00 con 0x558516827520 2024-09-21T07:23:45.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.799+0000 7f7617e00640 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516806840 0x558516bda530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.800+0000 7f761f400640 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516be5e10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7618010540 tx=0x7f7618011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34bb24eb81fde7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.800+0000 7f760cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.800+0000 7f760cc00640 1 -- 172.21.15.27:0/4232841378 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618029060 con 0x558516827520 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.800+0000 7f760cc00640 1 -- 172.21.15.27:0/4232841378 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f7618011640 con 0x558516827520 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.800+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7618015030 con 0x558516827520 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f760cc00640 1 -- 172.21.15.27:0/4232841378 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7618030050 con 0x558516827520 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 10 client.28619 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558516909960 con 0x558516827520 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f760cc00640 1 -- 172.21.15.27:0/4232841378 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7618015030 con 0x558516827520 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f760cc00640 1 -- 172.21.15.27:0/4232841378 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7618023070 con 0x558516827520 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 10 client.28619 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 10 client.28619 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:23:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 20 client.28619 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167e5ea0 0x558516bebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 4 client.28619 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.801+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558516b471e0 con 0x5585167e5ea0 2024-09-21T07:23:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.802+0000 7f761ea00640 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167e5ea0 0x558516bebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.802+0000 7f761ea00640 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167e5ea0 0x558516bebd80 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52854dac1180371 server_cookie=34fe9fbc9d585ba5 in_seq=0 out_seq=0 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.802+0000 7f760cc00640 10 client.28619 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.803+0000 7f760cc00640 1 -- 172.21.15.27:0/4232841378 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558516b471e0 con 0x5585167e5ea0 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.803+0000 7f760cc00640 10 client.28619 handle_command_reply: tid=0 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.803+0000 7f762ca6a740 1 client.28619 shutdown 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 msgr2=0x558516be5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516be5e10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7618010540 tx=0x7f7618011e50 comp rx=0 tx=0).stop 2024-09-21T07:23:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 shutdown_connections 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167e5ea0 0x558516bebd80 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558516827520 0x558516be5e10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516806840 0x558516bda530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 --2- 172.21.15.27:0/4232841378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5585167fe660 0x558516bd9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f760cc00640 0 client.28619 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 >> 172.21.15.27:0/4232841378 conn(0x55851681b910 msgr2=0x558516bd1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 shutdown_connections 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 1 -- 172.21.15.27:0/4232841378 wait complete. 2024-09-21T07:23:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.804+0000 7f762ca6a740 20 client.28619 trim_cache size 0 max 16384 2024-09-21T07:23:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 Processor -- start 2024-09-21T07:23:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 -- start start 2024-09-21T07:23:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bd9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516bd34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516bd39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b57f0 con 0x558516827520 2024-09-21T07:23:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b30c0 con 0x5585167fe660 2024-09-21T07:23:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.822+0000 7f762ca6a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b0f30 con 0x558516806840 2024-09-21T07:23:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516bd39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bd9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516bd39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33096/0 (socket says 172.21.15.27:33096) 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761f400640 1 -- 172.21.15.27:0/793432850 learned_addr learned my addr 172.21.15.27:0/793432850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f7617e00640 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516bd34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761ea00640 1 -- 172.21.15.27:0/793432850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 msgr2=0x558516bd34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761ea00640 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516bd34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761ea00640 1 -- 172.21.15.27:0/793432850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 msgr2=0x558516bd39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761ea00640 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516bd39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761ea00640 1 -- 172.21.15.27:0/793432850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558516bd8d50 con 0x5585167fe660 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.823+0000 7f761ea00640 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bd9fd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558516acf6f0 tx=0x7f761001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb54bcaa830d5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.824+0000 7f760f400640 1 -- 172.21.15.27:0/793432850 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761000fe00 con 0x5585167fe660 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.824+0000 7f760f400640 1 -- 172.21.15.27:0/793432850 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761001dd10 con 0x5585167fe660 2024-09-21T07:23:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.824+0000 7f760f400640 1 -- 172.21.15.27:0/793432850 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761002bd10 con 0x5585167fe660 2024-09-21T07:23:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.824+0000 7f762ca6a740 1 -- 172.21.15.27:0/793432850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 msgr2=0x558516bd9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.824+0000 7f762ca6a740 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bd9fd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558516acf6f0 tx=0x7f761001b000 comp rx=0 tx=0).stop 2024-09-21T07:23:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 -- 172.21.15.27:0/793432850 shutdown_connections 2024-09-21T07:23:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516bd34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516bd39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 --2- 172.21.15.27:0/793432850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bd9fd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 -- 172.21.15.27:0/793432850 >> 172.21.15.27:0/793432850 conn(0x55851681b910 msgr2=0x558516bd4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 -- 172.21.15.27:0/793432850 shutdown_connections 2024-09-21T07:23:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 -- 172.21.15.27:0/793432850 wait complete. 2024-09-21T07:23:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 -- start start 2024-09-21T07:23:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.825+0000 7f762ca6a740 1 Processor -- start 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516beddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f762ca6a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516c4e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f762ca6a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b1580 con 0x558516827520 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f762ca6a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b29f0 con 0x5585167fe660 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f762ca6a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b1c50 con 0x558516806840 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f762ca6a740 10 client.0 _ll_register_callbacks cb 0x7f7610009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f761ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f761ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bed7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:33456/0 (socket says 172.21.15.27:33456) 2024-09-21T07:23:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f761f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516c4e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f761ea00640 1 -- 172.21.15.27:0/948123233 learned_addr learned my addr 172.21.15.27:0/948123233 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.826+0000 7f7617e00640 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516beddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f7617e00640 1 -- 172.21.15.27:0/948123233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 msgr2=0x558516bed7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f7617e00640 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bed7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f7617e00640 1 -- 172.21.15.27:0/948123233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 msgr2=0x558516c4e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f7617e00640 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516c4e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f7617e00640 1 -- 172.21.15.27:0/948123233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558516c4eb30 con 0x558516806840 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f7617e00640 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516beddd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7608105310 tx=0x7f76081058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50986bf74b503550 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f760cc00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:23:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.827+0000 7f760cc00640 1 -- 172.21.15.27:0/948123233 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608049040 con 0x558516806840 2024-09-21T07:23:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f761001dd10 con 0x558516806840 2024-09-21T07:23:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f760cc00640 1 -- 172.21.15.27:0/948123233 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f76080d71a0 con 0x558516806840 2024-09-21T07:23:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f760cc00640 1 -- 172.21.15.27:0/948123233 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76080cd410 con 0x558516806840 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 10 client.28591 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558516c570b0 con 0x558516806840 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f760cc00640 1 -- 172.21.15.27:0/948123233 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761001dd10 con 0x558516806840 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f760cc00640 1 -- 172.21.15.27:0/948123233 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f76080ccd20 con 0x558516806840 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 10 client.28591 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 10 client.28591 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 20 client.28591 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167bb250 0x558516c59870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 4 client.28591 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "68c6c9a7", "release": true, "await": true}] 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.828+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "68c6c9a7", "release": true, "await": true}) -- 0x558516b42c90 con 0x5585167bb250 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.829+0000 7f761ea00640 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167bb250 0x558516c59870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.829+0000 7f761ea00640 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167bb250 0x558516c59870 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c415189379afeec2 server_cookie=72feef07b203acde in_seq=0 out_seq=0 2024-09-21T07:23:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.829+0000 7f760cc00640 10 client.28591 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f760cc00640 1 -- 172.21.15.27:0/948123233 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558516b42c90 con 0x5585167bb250 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f760cc00640 10 client.28591 handle_command_reply: tid=0 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f762ca6a740 1 client.28591 shutdown 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 msgr2=0x558516beddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f762ca6a740 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516beddd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7608105310 tx=0x7f76081058d0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 shutdown_connections 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f762ca6a740 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5585167bb250 0x558516c59870 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f762ca6a740 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558516806840 0x558516beddd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.833+0000 7f762ca6a740 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558516827520 0x558516c4e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.834+0000 7f762ca6a740 1 --2- 172.21.15.27:0/948123233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585167fe660 0x558516bed7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.834+0000 7f760cc00640 0 client.28591 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.834+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 >> 172.21.15.27:0/948123233 conn(0x55851681b910 msgr2=0x558516bd4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.834+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 shutdown_connections 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.834+0000 7f762ca6a740 1 -- 172.21.15.27:0/948123233 wait complete. 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.834+0000 7f762ca6a740 20 client.28591 trim_cache size 0 max 16384 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 44, 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:23:45.837 INFO:teuthology.orchestra.run.smithi027.stdout: "68c6c9a7": { 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 44, 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 144.5, 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:23:45.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76040b6700 msgr2=0x7f76040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76040b6700 0x7f76040b8c10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x558516bccfc0 tx=0x7f7610005f90 comp rx=0 tx=0).stop 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010b4d0 msgr2=0x7f762019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010b4d0 0x7f762019cad0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7608006010 tx=0x7f7608004060 comp rx=0 tx=0).stop 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 shutdown_connections 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f76040b6700 0x7f76040b8c10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f762010be20 0x7f76201a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762010b4d0 0x7f762019cad0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 --2- 172.21.15.27:0/4023128001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7620105b90 0x7f762019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 >> 172.21.15.27:0/4023128001 conn(0x7f76201015e0 msgr2=0x7f7620103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 shutdown_connections 2024-09-21T07:23:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:45.835+0000 7f761fe00640 1 -- 172.21.15.27:0/4023128001 wait complete. 2024-09-21T07:23:46.912 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '68c6c9a7', total seconds elapsed: 28.9 2024-09-21T07:23:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:46 smithi119 ceph-mon[34519]: pgmap v710: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-09-21T07:23:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:46 smithi027 ceph-mon[30779]: pgmap v710: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-09-21T07:23:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:46 smithi153 ceph-mon[34747]: pgmap v710: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.088+0000 7fc8cde00640 1 -- 172.21.15.27:0/2876222611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 msgr2=0x7fc8c81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.088+0000 7fc8cde00640 1 --2- 172.21.15.27:0/2876222611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 0x7fc8c81103c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ca0 tx=0x7fc8c002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.088+0000 7fc8cde00640 1 -- 172.21.15.27:0/2876222611 shutdown_connections 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.089+0000 7fc8cde00640 1 --2- 172.21.15.27:0/2876222611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 0x7fc8c81103c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.089+0000 7fc8cde00640 1 --2- 172.21.15.27:0/2876222611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c8109240 0x7fc8c8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.089+0000 7fc8cde00640 1 --2- 172.21.15.27:0/2876222611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 0x7fc8c8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.089+0000 7fc8cde00640 1 -- 172.21.15.27:0/2876222611 >> 172.21.15.27:0/2876222611 conn(0x7fc8c80ff330 msgr2=0x7fc8c8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.089+0000 7fc8cde00640 1 -- 172.21.15.27:0/2876222611 shutdown_connections 2024-09-21T07:23:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.089+0000 7fc8cde00640 1 -- 172.21.15.27:0/2876222611 wait complete. 2024-09-21T07:23:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 Processor -- start 2024-09-21T07:23:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 -- start start 2024-09-21T07:23:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 0x7fc8c819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c8109240 0x7fc8c819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 0x7fc8c819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c8113850 con 0x7fc8c8109b90 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81136d0 con 0x7fc8c8109240 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.090+0000 7fc8cde00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81139d0 con 0x7fc8c8103910 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 0x7fc8c819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 0x7fc8c819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56652/0 (socket says 172.21.15.27:56652) 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 -- 172.21.15.27:0/3531434095 learned_addr learned my addr 172.21.15.27:0/3531434095 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c8109240 0x7fc8c819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 0x7fc8c819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 -- 172.21.15.27:0/3531434095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c8109240 msgr2=0x7fc8c819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c8109240 0x7fc8c819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 -- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 msgr2=0x7fc8c819faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 0x7fc8c819faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 -- 172.21.15.27:0/3531434095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c81a01a0 con 0x7fc8c8103910 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 0x7fc8c819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.091+0000 7fc8cca00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 0x7fc8c819a270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00e9f0 tx=0x7fc8bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.092+0000 7fc8c5600640 1 -- 172.21.15.27:0/3531434095 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00cd80 con 0x7fc8c8103910 2024-09-21T07:23:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.092+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c81a0490 con 0x7fc8c8103910 2024-09-21T07:23:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.092+0000 7fc8c5600640 1 -- 172.21.15.27:0/3531434095 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8bc0108e0 con 0x7fc8c8103910 2024-09-21T07:23:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.092+0000 7fc8c5600640 1 -- 172.21.15.27:0/3531434095 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc018980 con 0x7fc8c8103910 2024-09-21T07:23:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.092+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c81a6c20 con 0x7fc8c8103910 2024-09-21T07:23:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 Processor -- start 2024-09-21T07:23:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 -- start start 2024-09-21T07:23:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147991b0 0x563b14b3de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b3e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b3e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8c5600640 1 -- 172.21.15.27:0/3531434095 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc8bc018b20 con 0x7fc8c8103910 2024-09-21T07:23:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563b14ab36b0 con 0x563b147702f0 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b14aa2ae0 con 0x563b147784d0 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.095+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563b14aa68f0 con 0x563b147991b0 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147991b0 0x563b14b3de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147991b0 0x563b14b3de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56668/0 (socket says 172.21.15.27:56668) 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 -- 172.21.15.27:0/3404191299 learned_addr learned my addr 172.21.15.27:0/3404191299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b3e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b3e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 -- 172.21.15.27:0/3404191299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 msgr2=0x563b14b3e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b3e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 -- 172.21.15.27:0/3404191299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 msgr2=0x563b14b3e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b3e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8cd400640 1 -- 172.21.15.27:0/3404191299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b14a6f780 con 0x563b147991b0 2024-09-21T07:23:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.096+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b3e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.097+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147991b0 0x563b14b3de10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00094a0 tx=0x7fc8c0009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ddf0f66a75e4432 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.097+0000 7fc8c5600640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b00b6680 0x7fc8b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.097+0000 7fc8b9c00640 1 -- 172.21.15.27:0/3404191299 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0031db0 con 0x563b147991b0 2024-09-21T07:23:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.097+0000 7fc8b9c00640 1 -- 172.21.15.27:0/3404191299 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c0040070 con 0x563b147991b0 2024-09-21T07:23:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.097+0000 7fc8b9c00640 1 -- 172.21.15.27:0/3404191299 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00042a0 con 0x563b147991b0 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.097+0000 7fc8c5600640 1 -- 172.21.15.27:0/3531434095 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fc8bc0e7ef0 con 0x7fc8c8103910 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.097+0000 7fc8cca00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b00b6680 0x7fc8b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3404191299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147991b0 msgr2=0x563b14b3de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147991b0 0x563b14b3de10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00094a0 tx=0x7fc8c0009800 comp rx=0 tx=0).stop 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8cca00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b00b6680 0x7fc8b00b8b90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x563b14b3ca90 tx=0x7fc8bc0eb9a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3404191299 shutdown_connections 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b3e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b3e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3404191299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147991b0 0x563b14b3de10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3404191299 >> 172.21.15.27:0/3404191299 conn(0x563b1478d300 msgr2=0x563b14b3bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.098+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3404191299 shutdown_connections 2024-09-21T07:23:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.099+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3404191299 wait complete. 2024-09-21T07:23:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.099+0000 7fc8d65f2740 1 -- start start 2024-09-21T07:23:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.099+0000 7fc8d65f2740 1 Processor -- start 2024-09-21T07:23:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b49450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14b55820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563b149daab0 con 0x563b147702f0 2024-09-21T07:23:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b14a49570 con 0x563b147991b0 2024-09-21T07:23:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563b14b4c530 con 0x563b147784d0 2024-09-21T07:23:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8d65f2740 10 client.0 _ll_register_callbacks cb 0x563b14a58100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b49450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b49450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46368/0 (socket says 172.21.15.27:46368) 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8cd400640 1 -- 172.21.15.27:0/2062322737 learned_addr learned my addr 172.21.15.27:0/2062322737 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.100+0000 7fc8cd400640 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14b55820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8cd400640 1 -- 172.21.15.27:0/2062322737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 msgr2=0x563b14b49990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8cd400640 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b49990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8cd400640 1 -- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 msgr2=0x563b14b55820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8cd400640 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14b55820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8cd400640 1 -- 172.21.15.27:0/2062322737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b14804790 con 0x563b147702f0 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8cd400640 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b49450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0003ef0 tx=0x7fc8c0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf0b31d093e09e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8ae000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8ae000640 1 -- 172.21.15.27:0/2062322737 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0050070 con 0x563b147702f0 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8ae000640 1 -- 172.21.15.27:0/2062322737 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fc8c0031860 con 0x563b147702f0 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8c0040070 con 0x563b147702f0 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8ae000640 1 -- 172.21.15.27:0/2062322737 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8c002f660 con 0x563b147702f0 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8d65f2740 10 client.18609 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.101+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b1487d620 con 0x563b147702f0 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8ae000640 1 -- 172.21.15.27:0/2062322737 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c0040070 con 0x563b147702f0 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8ae000640 1 -- 172.21.15.27:0/2062322737 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8c0004330 con 0x563b147702f0 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8d65f2740 10 client.18609 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8d65f2740 10 client.18609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8d65f2740 20 client.18609 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b14757d80 0x563b14b5b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8d65f2740 4 client.18609 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b14ab5600 con 0x563b14757d80 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.102+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b14757d80 0x563b14b5b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.103+0000 7fc8ae000640 10 client.18609 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.103+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b14757d80 0x563b14b5b370 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad5ac88890de8ff7 server_cookie=39bd8e83b0b97caf in_seq=0 out_seq=0 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8ae000640 1 -- 172.21.15.27:0/2062322737 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b14ab5600 con 0x563b14757d80 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8ae000640 10 client.18609 handle_command_reply: tid=0 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 client.18609 shutdown 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 msgr2=0x563b14b49450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b49450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0003ef0 tx=0x7fc8c0009a20 comp rx=0 tx=0).stop 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 shutdown_connections 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b14757d80 0x563b14b5b370 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b49990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14b55820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/2062322737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b49450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8ae000640 0 client.18609 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 >> 172.21.15.27:0/2062322737 conn(0x563b1478d300 msgr2=0x563b14b3ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.104+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 shutdown_connections 2024-09-21T07:23:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.105+0000 7fc8d65f2740 1 -- 172.21.15.27:0/2062322737 wait complete. 2024-09-21T07:23:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.105+0000 7fc8d65f2740 20 client.18609 trim_cache size 0 max 16384 2024-09-21T07:23:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.120+0000 7fc8d65f2740 1 Processor -- start 2024-09-21T07:23:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.120+0000 7fc8d65f2740 1 -- start start 2024-09-21T07:23:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147702f0 0x563b14b3ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b4b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147991b0 0x563b14b46620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81b13a0 con 0x563b147991b0 2024-09-21T07:23:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81aec70 con 0x563b147784d0 2024-09-21T07:23:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81acae0 con 0x563b147702f0 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147991b0 0x563b14b46620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147702f0 0x563b14b3ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147702f0 0x563b14b3ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56682/0 (socket says 172.21.15.27:56682) 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 -- 172.21.15.27:0/3542955316 learned_addr learned my addr 172.21.15.27:0/3542955316 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 -- 172.21.15.27:0/3542955316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 msgr2=0x563b14b4b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b4b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 -- 172.21.15.27:0/3542955316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147991b0 msgr2=0x563b14b46620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147991b0 0x563b14b46620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8cd400640 1 -- 172.21.15.27:0/3542955316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b14b40c20 con 0x563b147702f0 2024-09-21T07:23:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.121+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.122+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147702f0 0x563b14b3ad20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00094e0 tx=0x7fc8c0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c08050ad510d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.122+0000 7fc8b9200640 1 -- 172.21.15.27:0/3542955316 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0050070 con 0x563b147702f0 2024-09-21T07:23:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.122+0000 7fc8b9200640 1 -- 172.21.15.27:0/3542955316 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c0040040 con 0x563b147702f0 2024-09-21T07:23:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.122+0000 7fc8b9200640 1 -- 172.21.15.27:0/3542955316 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c002f800 con 0x563b147702f0 2024-09-21T07:23:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3542955316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147702f0 msgr2=0x563b14b3ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147702f0 0x563b14b3ad20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00094e0 tx=0x7fc8c0000f40 comp rx=0 tx=0).stop 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3542955316 shutdown_connections 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147991b0 0x563b14b46620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147784d0 0x563b14b4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3542955316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147702f0 0x563b14b3ad20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3542955316 >> 172.21.15.27:0/3542955316 conn(0x563b1478d300 msgr2=0x563b14b4a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3542955316 shutdown_connections 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3542955316 wait complete. 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 -- start start 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.123+0000 7fc8d65f2740 1 Processor -- start 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b5dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8d65f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14bbdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81ad130 con 0x563b147702f0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81ae5a0 con 0x563b147991b0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8d65f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81ad800 con 0x563b147784d0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8d65f2740 10 client.0 _ll_register_callbacks cb 0x7fc8bc0eb870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b5dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b5dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46390/0 (socket says 172.21.15.27:46390) 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8cd400640 1 -- 172.21.15.27:0/3268253 learned_addr learned my addr 172.21.15.27:0/3268253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.124+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14bbdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8cd400640 1 -- 172.21.15.27:0/3268253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 msgr2=0x563b14b5e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b5e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8cd400640 1 -- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 msgr2=0x563b14bbdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14bbdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8cd400640 1 -- 172.21.15.27:0/3268253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b14bbe160 con 0x563b147702f0 2024-09-21T07:23:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8cd400640 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b5dd70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006010 tx=0x7fc8c0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acae5a252294814 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8ae000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8ae000640 1 -- 172.21.15.27:0/3268253 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0068070 con 0x563b147702f0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8ae000640 1 -- 172.21.15.27:0/3268253 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fc8c00307d0 con 0x563b147702f0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.125+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8c0040040 con 0x563b147702f0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8ae000640 1 -- 172.21.15.27:0/3268253 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8c0050040 con 0x563b147702f0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 10 client.18615 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b14bc5550 con 0x563b147702f0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8ae000640 1 -- 172.21.15.27:0/3268253 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c0040040 con 0x563b147702f0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8ae000640 1 -- 172.21.15.27:0/3268253 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8c0004150 con 0x563b147702f0 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 10 client.18615 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 10 client.18615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 20 client.18615 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1472d1c0 0x563b14bc7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 4 client.18615 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:23:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.126+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b14ab10b0 con 0x563b1472d1c0 2024-09-21T07:23:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.127+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1472d1c0 0x563b14bc7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.127+0000 7fc8c7e00640 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1472d1c0 0x563b14bc7b50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf7d9c93a171d065 server_cookie=3a233099d00b9973 in_seq=0 out_seq=0 2024-09-21T07:23:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.127+0000 7fc8ae000640 10 client.18615 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.138+0000 7fc8ae000640 1 -- 172.21.15.27:0/3268253 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b14ab10b0 con 0x563b1472d1c0 2024-09-21T07:23:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.138+0000 7fc8ae000640 10 client.18615 handle_command_reply: tid=0 2024-09-21T07:23:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.138+0000 7fc8d65f2740 1 client.18615 shutdown 2024-09-21T07:23:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 msgr2=0x563b14b5dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b5dd70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006010 tx=0x7fc8c0009a20 comp rx=0 tx=0).stop 2024-09-21T07:23:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 shutdown_connections 2024-09-21T07:23:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563b1472d1c0 0x563b14bc7b50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563b147784d0 0x563b14b5e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b147991b0 0x563b14bbdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 --2- 172.21.15.27:0/3268253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b147702f0 0x563b14b5dd70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 >> 172.21.15.27:0/3268253 conn(0x563b1478d300 msgr2=0x563b14b3aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8ae000640 0 client.18615 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 shutdown_connections 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 1 -- 172.21.15.27:0/3268253 wait complete. 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.139+0000 7fc8d65f2740 20 client.18615 trim_cache size 0 max 16384 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:23:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:23:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:23:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b00b6680 msgr2=0x7fc8b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b00b6680 0x7fc8b00b8b90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x563b14b3ca90 tx=0x7fc8bc0eb9a0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 msgr2=0x7fc8c819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 0x7fc8c819a270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00e9f0 tx=0x7fc8bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 shutdown_connections 2024-09-21T07:23:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc8b00b6680 0x7fc8b00b8b90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c8109b90 0x7fc8c819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c8109240 0x7fc8c819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 --2- 172.21.15.27:0/3531434095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c8103910 0x7fc8c819a270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 >> 172.21.15.27:0/3531434095 conn(0x7fc8c80ff330 msgr2=0x7fc8c8101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 shutdown_connections 2024-09-21T07:23:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:23:48.140+0000 7fc8cde00640 1 -- 172.21.15.27:0/3531434095 wait complete. 2024-09-21T07:23:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:48 smithi119 ceph-mon[34519]: pgmap v711: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-09-21T07:23:49.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:48 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:23:49.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:48 smithi027 ceph-mon[30779]: pgmap v711: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-09-21T07:23:49.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:48 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:23:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:48 smithi153 ceph-mon[34747]: pgmap v711: 129 pgs: 129 active+clean; 2.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-09-21T07:23:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:48 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:23:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:50 smithi119 ceph-mon[34519]: pgmap v712: 129 pgs: 129 active+clean; 2.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-21T07:23:51.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:50 smithi027 ceph-mon[30779]: pgmap v712: 129 pgs: 129 active+clean; 2.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-21T07:23:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:50 smithi153 ceph-mon[34747]: pgmap v712: 129 pgs: 129 active+clean; 2.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-21T07:23:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:52 smithi119 ceph-mon[34519]: pgmap v713: 129 pgs: 129 active+clean; 2.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:23:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:52 smithi027 ceph-mon[30779]: pgmap v713: 129 pgs: 129 active+clean; 2.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:23:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:52 smithi153 ceph-mon[34747]: pgmap v713: 129 pgs: 129 active+clean; 2.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:23:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:54 smithi119 ceph-mon[34519]: pgmap v714: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:54 smithi027 ceph-mon[30779]: pgmap v714: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:54 smithi153 ceph-mon[34747]: pgmap v714: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:55.981 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:23:56.380 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:57.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 -- 172.21.15.27:0/3524269279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28109c20 msgr2=0x7f0b28110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/3524269279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28109c20 0x7f0b28110450 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009ca0 tx=0x7f0b2002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 -- 172.21.15.27:0/3524269279 shutdown_connections 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/3524269279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b281092d0 0x7f0b281096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/3524269279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28109c20 0x7f0b28110450 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/3524269279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b28103990 0x7f0b28103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 -- 172.21.15.27:0/3524269279 >> 172.21.15.27:0/3524269279 conn(0x7f0b280ff3e0 msgr2=0x7f0b28101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 -- 172.21.15.27:0/3524269279 shutdown_connections 2024-09-21T07:23:57.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.019+0000 7f0b2cc00640 1 -- 172.21.15.27:0/3524269279 wait complete. 2024-09-21T07:23:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.020+0000 7f0b2cc00640 1 Processor -- start 2024-09-21T07:23:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.020+0000 7f0b2cc00640 1 -- start start 2024-09-21T07:23:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b2cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28103990 0x7f0b2819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b2cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b281092d0 0x7f0b2819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b2cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b28109c20 0x7f0b281a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b2cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281138e0 con 0x7f0b28103990 2024-09-21T07:23:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b2cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28113760 con 0x7f0b281092d0 2024-09-21T07:23:57.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b2cc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28113a60 con 0x7f0b28109c20 2024-09-21T07:23:57.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b27400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28103990 0x7f0b2819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b27e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b28109c20 0x7f0b281a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b27e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b28109c20 0x7f0b281a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56704/0 (socket says 172.21.15.27:56704) 2024-09-21T07:23:57.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b27e00640 1 -- 172.21.15.27:0/1560732314 learned_addr learned my addr 172.21.15.27:0/1560732314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:57.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.021+0000 7f0b26a00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b281092d0 0x7f0b2819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.022+0000 7f0b27e00640 1 -- 172.21.15.27:0/1560732314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b281092d0 msgr2=0x7f0b2819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.022+0000 7f0b27e00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b281092d0 0x7f0b2819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.022+0000 7f0b27e00640 1 -- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28103990 msgr2=0x7f0b2819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.022+0000 7f0b27e00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28103990 0x7f0b2819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.022+0000 7f0b27e00640 1 -- 172.21.15.27:0/1560732314 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b281a2460 con 0x7f0b28109c20 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.022+0000 7f0b27e00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b28109c20 0x7f0b281a1d30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005cc0 tx=0x7f0b20031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.022+0000 7f0b1fe00640 1 -- 172.21.15.27:0/1560732314 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20003b70 con 0x7f0b28109c20 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.023+0000 7f0b1fe00640 1 -- 172.21.15.27:0/1560732314 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b200318f0 con 0x7f0b28109c20 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.023+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b281a26f0 con 0x7f0b28109c20 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.023+0000 7f0b1fe00640 1 -- 172.21.15.27:0/1560732314 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2002f950 con 0x7f0b28109c20 2024-09-21T07:23:57.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.023+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b281a8eb0 con 0x7f0b28109c20 2024-09-21T07:23:57.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.026+0000 7f0b34e09740 1 Processor -- start 2024-09-21T07:23:57.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.026+0000 7f0b1fe00640 1 -- 172.21.15.27:0/1560732314 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0b2002fb50 con 0x7f0b28109c20 2024-09-21T07:23:57.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b34e09740 1 -- start start 2024-09-21T07:23:57.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x55632031f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632031f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b34e09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556320295f70 con 0x55631ff4f1b0 2024-09-21T07:23:57.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b34e09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5563202853a0 con 0x55631ff57390 2024-09-21T07:23:57.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b34e09740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5563202891b0 con 0x55631ff78070 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b27400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x55632031f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b26a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b27400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x55632031f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46420/0 (socket says 172.21.15.27:46420) 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b27400640 1 -- 172.21.15.27:0/3919199835 learned_addr learned my addr 172.21.15.27:0/3919199835 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.027+0000 7f0b26a00640 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632031f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b27400640 1 -- 172.21.15.27:0/3919199835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 msgr2=0x55632031ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b27400640 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b27400640 1 -- 172.21.15.27:0/3919199835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 msgr2=0x55632031f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b27400640 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632031f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b27400640 1 -- 172.21.15.27:0/3919199835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632024fc90 con 0x55631ff4f1b0 2024-09-21T07:23:57.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b27400640 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x55632031f470 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009cf0 tx=0x7f0b18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227cbfc2ce309c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b13e00640 1 -- 172.21.15.27:0/3919199835 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18004230 con 0x55631ff4f1b0 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.028+0000 7f0b1fe00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b0c0b6680 0x7f0b0c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.029+0000 7f0b13e00640 1 -- 172.21.15.27:0/3919199835 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b180043d0 con 0x55631ff4f1b0 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.029+0000 7f0b13e00640 1 -- 172.21.15.27:0/3919199835 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b180197c0 con 0x55631ff4f1b0 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.029+0000 7f0b1fe00640 1 -- 172.21.15.27:0/1560732314 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f0b20108c00 con 0x7f0b28109c20 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.029+0000 7f0b26a00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b0c0b6680 0x7f0b0c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 -- 172.21.15.27:0/3919199835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 msgr2=0x55632031f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x55632031f470 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009cf0 tx=0x7f0b18011810 comp rx=0 tx=0).stop 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b26a00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b0c0b6680 0x7f0b0c0b8b90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009fd0 tx=0x7f0b140046c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 -- 172.21.15.27:0/3919199835 shutdown_connections 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632031f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x55632031f470 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3919199835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 -- 172.21.15.27:0/3919199835 >> 172.21.15.27:0/3919199835 conn(0x55631ff6c460 msgr2=0x55632031ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 -- 172.21.15.27:0/3919199835 shutdown_connections 2024-09-21T07:23:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 -- 172.21.15.27:0/3919199835 wait complete. 2024-09-21T07:23:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.030+0000 7f0b34e09740 1 -- start start 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 1 Processor -- start 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff4f1b0 0x55632032a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff57390 0x55632032ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff78070 0x556320336a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5563200e6700 con 0x55631ff57390 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556320220540 con 0x55631ff78070 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55632032d710 con 0x55631ff4f1b0 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.031+0000 7f0b34e09740 10 client.0 _ll_register_callbacks cb 0x556320230430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff4f1b0 0x55632032a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff4f1b0 0x55632032a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56718/0 (socket says 172.21.15.27:56718) 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27400640 1 -- 172.21.15.27:0/3780081292 learned_addr learned my addr 172.21.15.27:0/3780081292 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27e00640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff78070 0x556320336a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b26a00640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff57390 0x55632032ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27e00640 1 -- 172.21.15.27:0/3780081292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff4f1b0 msgr2=0x55632032a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27e00640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff4f1b0 0x55632032a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27e00640 1 -- 172.21.15.27:0/3780081292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff57390 msgr2=0x55632032ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27e00640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff57390 0x55632032ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27e00640 1 -- 172.21.15.27:0/3780081292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55631ffe3650 con 0x55631ff78070 2024-09-21T07:23:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27400640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff4f1b0 0x55632032a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b27e00640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff78070 0x556320336a00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0b20105f40 tx=0x7f0b20003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394499a4a87496bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.032+0000 7f0b10c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.033+0000 7f0b10c00640 1 -- 172.21.15.27:0/3780081292 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b200d3120 con 0x55631ff78070 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.033+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556320222b50 con 0x55631ff78070 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.033+0000 7f0b10c00640 1 -- 172.21.15.27:0/3780081292 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f0b200d7bd0 con 0x55631ff78070 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.033+0000 7f0b10c00640 1 -- 172.21.15.27:0/3780081292 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b200d8060 con 0x55631ff78070 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.033+0000 7f0b34e09740 10 client.28652 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.033+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55632023d910 con 0x55631ff78070 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.033+0000 7f0b10c00640 1 -- 172.21.15.27:0/3780081292 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556320222b50 con 0x55631ff78070 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b10c00640 1 -- 172.21.15.27:0/3780081292 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0b200c9f90 con 0x55631ff78070 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b34e09740 10 client.28652 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b34e09740 10 client.28652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b34e09740 20 client.28652 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff369f0 0x55632033c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b34e09740 4 client.28652 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:23:57.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556320297ec0 con 0x55631ff369f0 2024-09-21T07:23:57.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.034+0000 7f0b27400640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff369f0 0x55632033c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.035+0000 7f0b27400640 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff369f0 0x55632033c7e0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e46e756e9072bc server_cookie=773728a52fc032df in_seq=0 out_seq=0 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.035+0000 7f0b10c00640 10 client.28652 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b10c00640 1 -- 172.21.15.27:0/3780081292 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556320297ec0 con 0x55631ff369f0 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b10c00640 10 client.28652 handle_command_reply: tid=0 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 client.28652 shutdown 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff78070 msgr2=0x556320336a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff78070 0x556320336a00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0b20105f40 tx=0x7f0b20003f90 comp rx=0 tx=0).stop 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 shutdown_connections 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff369f0 0x55632033c7e0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff78070 0x556320336a00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff57390 0x55632032ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b10c00640 0 client.28652 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 --2- 172.21.15.27:0/3780081292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff4f1b0 0x55632032a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 >> 172.21.15.27:0/3780081292 conn(0x55631ff6c460 msgr2=0x5563203220c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 shutdown_connections 2024-09-21T07:23:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 1 -- 172.21.15.27:0/3780081292 wait complete. 2024-09-21T07:23:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.036+0000 7f0b34e09740 20 client.28652 trim_cache size 0 max 16384 2024-09-21T07:23:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 Processor -- start 2024-09-21T07:23:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 -- start start 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320329520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632032c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b57e0 con 0x55631ff4f1b0 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b30b0 con 0x55631ff57390 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b34e09740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b0f20 con 0x55631ff78070 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b27400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320329520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b27400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320329520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46436/0 (socket says 172.21.15.27:46436) 2024-09-21T07:23:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b27400640 1 -- 172.21.15.27:0/1117195951 learned_addr learned my addr 172.21.15.27:0/1117195951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.055+0000 7f0b26a00640 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632032c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b27e00640 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b27400640 1 -- 172.21.15.27:0/1117195951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 msgr2=0x55632031bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b27400640 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b27400640 1 -- 172.21.15.27:0/1117195951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 msgr2=0x55632032c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b27400640 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632032c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b27400640 1 -- 172.21.15.27:0/1117195951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632032d5f0 con 0x55631ff4f1b0 2024-09-21T07:23:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b27400640 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320329520 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009cf0 tx=0x7f0b18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd8c39020bde8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.056+0000 7f0b13400640 1 -- 172.21.15.27:0/1117195951 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18031070 con 0x55631ff4f1b0 2024-09-21T07:23:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b13400640 1 -- 172.21.15.27:0/1117195951 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18003d30 con 0x55631ff4f1b0 2024-09-21T07:23:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b13400640 1 -- 172.21.15.27:0/1117195951 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18019ab0 con 0x55631ff4f1b0 2024-09-21T07:23:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b34e09740 1 -- 172.21.15.27:0/1117195951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 msgr2=0x556320329520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b34e09740 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320329520 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009cf0 tx=0x7f0b18011810 comp rx=0 tx=0).stop 2024-09-21T07:23:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b34e09740 1 -- 172.21.15.27:0/1117195951 shutdown_connections 2024-09-21T07:23:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b34e09740 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632031bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b34e09740 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x55632032c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b34e09740 1 --2- 172.21.15.27:0/1117195951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320329520 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.057+0000 7f0b34e09740 1 -- 172.21.15.27:0/1117195951 >> 172.21.15.27:0/1117195951 conn(0x55631ff6c460 msgr2=0x55632031b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.058+0000 7f0b34e09740 1 -- 172.21.15.27:0/1117195951 shutdown_connections 2024-09-21T07:23:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.058+0000 7f0b34e09740 1 -- 172.21.15.27:0/1117195951 wait complete. 2024-09-21T07:23:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.058+0000 7f0b34e09740 1 -- start start 2024-09-21T07:23:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.058+0000 7f0b34e09740 1 Processor -- start 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320332cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x556320333290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b34e09740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632032d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b34e09740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b1570 con 0x55631ff4f1b0 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b34e09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b29e0 con 0x55631ff57390 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b34e09740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b1c40 con 0x55631ff78070 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b34e09740 10 client.0 _ll_register_callbacks cb 0x7f0b200cdf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b27400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320332cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b27400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320332cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46440/0 (socket says 172.21.15.27:46440) 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b26a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x556320333290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b27400640 1 -- 172.21.15.27:0/2574152172 learned_addr learned my addr 172.21.15.27:0/2574152172 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.059+0000 7f0b27e00640 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632032d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b27400640 1 -- 172.21.15.27:0/2574152172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 msgr2=0x55632032d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b27400640 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632032d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b27400640 1 -- 172.21.15.27:0/2574152172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 msgr2=0x556320333290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b27400640 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x556320333290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b27400640 1 -- 172.21.15.27:0/2574152172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632031d010 con 0x55631ff4f1b0 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b27400640 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320332cd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009ab0 tx=0x7f0b18009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd9ea821801b545 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b10c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:23:57.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b10c00640 1 -- 172.21.15.27:0/2574152172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18031040 con 0x55631ff4f1b0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b10c00640 1 -- 172.21.15.27:0/2574152172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f0b18003980 con 0x55631ff4f1b0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.060+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b18003d30 con 0x55631ff4f1b0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b10c00640 1 -- 172.21.15.27:0/2574152172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b180050c0 con 0x55631ff4f1b0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 10 client.18633 fetch_fsmap learned FSMap version 27 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563203a7a60 con 0x55631ff4f1b0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b10c00640 1 -- 172.21.15.27:0/2574152172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18003d30 con 0x55631ff4f1b0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b10c00640 1 -- 172.21.15.27:0/2574152172 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0b1803d030 con 0x55631ff4f1b0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 10 client.18633 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 10 client.18633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 20 client.18633 populate_metadata read hostname 'smithi027' 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff0bda0 0x5563203aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 4 client.18633 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.061+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556320293970 con 0x55631ff0bda0 2024-09-21T07:23:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.062+0000 7f0b26a00640 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff0bda0 0x5563203aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:57.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.062+0000 7f0b26a00640 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff0bda0 0x5563203aa070 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edec7eb04b34b6b5 server_cookie=73ec6beb7d937a6 in_seq=0 out_seq=0 2024-09-21T07:23:57.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.062+0000 7f0b10c00640 10 client.18633 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:57.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b10c00640 1 -- 172.21.15.27:0/2574152172 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x556320293970 con 0x55631ff0bda0 2024-09-21T07:23:57.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b10c00640 10 client.18633 handle_command_reply: tid=0 2024-09-21T07:23:57.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 client.18633 shutdown 2024-09-21T07:23:57.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 msgr2=0x556320332cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320332cd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009ab0 tx=0x7f0b18009ea0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 shutdown_connections 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55631ff0bda0 0x5563203aa070 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55631ff78070 0x55632032d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55631ff57390 0x556320333290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 --2- 172.21.15.27:0/2574152172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55631ff4f1b0 0x556320332cd0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 >> 172.21.15.27:0/2574152172 conn(0x55631ff6c460 msgr2=0x55632031ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b10c00640 0 client.18633 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 shutdown_connections 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.063+0000 7f0b34e09740 1 -- 172.21.15.27:0/2574152172 wait complete. 2024-09-21T07:23:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b34e09740 20 client.18633 trim_cache size 0 max 16384 2024-09-21T07:23:57.068 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:23:57.068 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:23:57.068 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:23:57.068 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:23:57.068 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:23:57.068 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:23:57.069 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.069 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.069 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.070 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:23:57.071 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:23:57.072 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:23:57.073 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:23:57.074 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.075 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.076 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:23:57.077 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b0c0b6680 msgr2=0x7f0b0c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b0c0b6680 0x7f0b0c0b8b90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009fd0 tx=0x7f0b140046c0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b28109c20 msgr2=0x7f0b281a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b28109c20 0x7f0b281a1d30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005cc0 tx=0x7f0b20031700 comp rx=0 tx=0).stop 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 shutdown_connections 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b28109c20 0x7f0b281a1d30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b0c0b6680 0x7f0b0c0b8b90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b281092d0 0x7f0b2819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.064+0000 7f0b2cc00640 1 --2- 172.21.15.27:0/1560732314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b28103990 0x7f0b2819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.065+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 >> 172.21.15.27:0/1560732314 conn(0x7f0b280ff3e0 msgr2=0x7f0b2810e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.065+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 shutdown_connections 2024-09-21T07:23:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:57.065+0000 7f0b2cc00640 1 -- 172.21.15.27:0/1560732314 wait complete. 2024-09-21T07:23:57.088 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:56 smithi027 ceph-mon[30779]: pgmap v715: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:56 smithi119 ceph-mon[34519]: pgmap v715: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:56 smithi153 ceph-mon[34747]: pgmap v715: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:57.911 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:23:57.911 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:23:58.304 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 -- 172.21.15.27:0/2968789005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c8105bc0 msgr2=0x7f14c8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 --2- 172.21.15.27:0/2968789005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c8105bc0 0x7f14c8105fd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009900 tx=0x7f14b802a400 comp rx=0 tx=0).stop 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 -- 172.21.15.27:0/2968789005 shutdown_connections 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 --2- 172.21.15.27:0/2968789005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c810b500 0x7f14c810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 --2- 172.21.15.27:0/2968789005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c810be50 0x7f14c8112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 --2- 172.21.15.27:0/2968789005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c8105bc0 0x7f14c8105fd0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 -- 172.21.15.27:0/2968789005 >> 172.21.15.27:0/2968789005 conn(0x7f14c8101670 msgr2=0x7f14c8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 -- 172.21.15.27:0/2968789005 shutdown_connections 2024-09-21T07:23:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.963+0000 7f14ce000640 1 -- 172.21.15.27:0/2968789005 wait complete. 2024-09-21T07:23:58.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 Processor -- start 2024-09-21T07:23:58.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 -- start start 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c8105bc0 0x7f14c819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c810b500 0x7f14c819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c810be50 0x7f14c81a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8115b60 con 0x7f14c810b500 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81159e0 con 0x7f14c8105bc0 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.965+0000 7f14ce000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8115ce0 con 0x7f14c810be50 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14ccc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c8105bc0 0x7f14c819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c810b500 0x7f14c819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c810b500 0x7f14c819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39722/0 (socket says 172.21.15.27:39722) 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14ccc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c8105bc0 0x7f14c819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44082/0 (socket says 172.21.15.27:44082) 2024-09-21T07:23:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 -- 172.21.15.27:0/1207770479 learned_addr learned my addr 172.21.15.27:0/1207770479 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:23:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14cd600640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c810be50 0x7f14c81a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 -- 172.21.15.27:0/1207770479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c810be50 msgr2=0x7f14c81a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c810be50 0x7f14c81a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 -- 172.21.15.27:0/1207770479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c8105bc0 msgr2=0x7f14c819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c8105bc0 0x7f14c819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14bfe00640 1 -- 172.21.15.27:0/1207770479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c81a25a0 con 0x7f14c810b500 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14cd600640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c810be50 0x7f14c81a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.966+0000 7f14ccc00640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c8105bc0 0x7f14c819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.967+0000 7f14bfe00640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c810b500 0x7f14c819cbd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e9c0 tx=0x7f14b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.967+0000 7f14c6000640 1 -- 172.21.15.27:0/1207770479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b000ccc0 con 0x7f14c810b500 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.967+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c81a2890 con 0x7f14c810b500 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.967+0000 7f14c6000640 1 -- 172.21.15.27:0/1207770479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b00054d0 con 0x7f14c810b500 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.967+0000 7f14c6000640 1 -- 172.21.15.27:0/1207770479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0010430 con 0x7f14c810b500 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.967+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c81a8ff0 con 0x7f14c810b500 2024-09-21T07:23:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.969+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c8105fd0 con 0x7f14c810b500 2024-09-21T07:23:58.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.971+0000 7f14c6000640 1 -- 172.21.15.27:0/1207770479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f14b0005000 con 0x7f14c810b500 2024-09-21T07:23:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.972+0000 7f14c6000640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14ac0b6680 0x7f14ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:23:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.973+0000 7f14c6000640 1 -- 172.21.15.27:0/1207770479 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f14b00e7ee0 con 0x7f14c810b500 2024-09-21T07:23:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.973+0000 7f14ccc00640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14ac0b6680 0x7f14ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:23:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.974+0000 7f14ccc00640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14ac0b6680 0x7f14ac0b8b90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f14b8005ab0 tx=0x7f14b8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:23:58.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:58.977+0000 7f14c6000640 1 -- 172.21.15.27:0/1207770479 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14b00052f0 con 0x7f14c810b500 2024-09-21T07:23:59.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:58 smithi027 ceph-mon[30779]: pgmap v716: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:59.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.218+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14c810d690 con 0x7f14c810b500 2024-09-21T07:23:59.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.219+0000 7f14c6000640 1 -- 172.21.15.27:0/1207770479 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f14b00ae020 con 0x7f14c810b500 2024-09-21T07:23:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:23:59.221 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:23:59.222 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:23:59.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14ac0b6680 msgr2=0x7f14ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:59.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14ac0b6680 0x7f14ac0b8b90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f14b8005ab0 tx=0x7f14b8005a20 comp rx=0 tx=0).stop 2024-09-21T07:23:59.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c810b500 msgr2=0x7f14c819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:23:59.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c810b500 0x7f14c819cbd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e9c0 tx=0x7f14b000ee80 comp rx=0 tx=0).stop 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 shutdown_connections 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c810be50 0x7f14c81a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14ac0b6680 0x7f14ac0b8b90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14c810b500 0x7f14c819cbd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 --2- 172.21.15.27:0/1207770479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c8105bc0 0x7f14c819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 >> 172.21.15.27:0/1207770479 conn(0x7f14c8101670 msgr2=0x7f14c810ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.221+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 shutdown_connections 2024-09-21T07:23:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:23:59.222+0000 7f14ce000640 1 -- 172.21.15.27:0/1207770479 wait complete. 2024-09-21T07:23:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:58 smithi119 ceph-mon[34519]: pgmap v716: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:23:59.430 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:58 smithi153 ceph-mon[34747]: pgmap v716: 129 pgs: 129 active+clean; 2.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-21T07:24:00.081 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:24:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:23:59 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1207770479' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:00.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:23:59 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1207770479' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:00.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:23:59 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1207770479' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:00.475 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 -- 172.21.15.27:0/278045986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810b6d0 msgr2=0x7f4ea810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 --2- 172.21.15.27:0/278045986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810b6d0 0x7f4ea810bae0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009900 tx=0x7f4e9002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 -- 172.21.15.27:0/278045986 shutdown_connections 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 --2- 172.21.15.27:0/278045986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea810c020 0x7f4ea8112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 --2- 172.21.15.27:0/278045986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810b6d0 0x7f4ea810bae0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 --2- 172.21.15.27:0/278045986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ea8105d90 0x7f4ea81061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 -- 172.21.15.27:0/278045986 >> 172.21.15.27:0/278045986 conn(0x7f4ea8101860 msgr2=0x7f4ea8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 -- 172.21.15.27:0/278045986 shutdown_connections 2024-09-21T07:24:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.185+0000 7f4eafc00640 1 -- 172.21.15.27:0/278045986 wait complete. 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.186+0000 7f4eafc00640 1 Processor -- start 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4eafc00640 1 -- start start 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4eafc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea8105d90 0x7f4ea819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4eafc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ea810b6d0 0x7f4ea819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4eafc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810c020 0x7f4ea81a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4eafc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8115d30 con 0x7f4ea810c020 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4eafc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8115bb0 con 0x7f4ea810b6d0 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4eafc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8115eb0 con 0x7f4ea8105d90 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.187+0000 7f4ead200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea8105d90 0x7f4ea819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea8105d90 0x7f4ea819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46524/0 (socket says 172.21.15.27:46524) 2024-09-21T07:24:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 -- 172.21.15.27:0/2749879613 learned_addr learned my addr 172.21.15.27:0/2749879613 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ea810b6d0 0x7f4ea819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4eadc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810c020 0x7f4ea81a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 -- 172.21.15.27:0/2749879613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ea810b6d0 msgr2=0x7f4ea819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ea810b6d0 0x7f4ea819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 -- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810c020 msgr2=0x7f4ea81a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810c020 0x7f4ea81a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 -- 172.21.15.27:0/2749879613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea81a2630 con 0x7f4ea8105d90 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4eadc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810c020 0x7f4ea81a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.188+0000 7f4ead200640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea8105d90 0x7f4ea819c6c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800e9f0 tx=0x7f4e9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.189+0000 7f4ea6000640 1 -- 172.21.15.27:0/2749879613 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800cd80 con 0x7f4ea8105d90 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.189+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea81a2920 con 0x7f4ea8105d90 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.189+0000 7f4ea6000640 1 -- 172.21.15.27:0/2749879613 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e980108e0 con 0x7f4ea8105d90 2024-09-21T07:24:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.189+0000 7f4ea6000640 1 -- 172.21.15.27:0/2749879613 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98018980 con 0x7f4ea8105d90 2024-09-21T07:24:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.189+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81a9080 con 0x7f4ea8105d90 2024-09-21T07:24:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.192+0000 7f4eb6849740 1 Processor -- start 2024-09-21T07:24:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eb6849740 1 -- start start 2024-09-21T07:24:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4ea6000640 1 -- 172.21.15.27:0/2749879613 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4e98010430 con 0x7f4ea8105d90 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5217ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c52180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5218620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eb6849740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5627c518efb0 con 0x5627c4e481b0 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eb6849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5627c517e3e0 con 0x5627c4e71070 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eb6849740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627c51821f0 con 0x5627c4e50390 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.193+0000 7f4eadc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5217ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4ead200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c52180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5217ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44110/0 (socket says 172.21.15.27:44110) 2024-09-21T07:24:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4e9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5218620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 -- 172.21.15.27:0/2311149126 learned_addr learned my addr 172.21.15.27:0/2311149126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 -- 172.21.15.27:0/2311149126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 msgr2=0x5627c5218620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5218620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 -- 172.21.15.27:0/2311149126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 msgr2=0x5627c52180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c52180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 -- 172.21.15.27:0/2311149126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c5148cd0 con 0x5627c4e71070 2024-09-21T07:24:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4ead200640 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c52180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:01.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.194+0000 7f4eadc00640 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5217ba0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009fb0 tx=0x7f4ea0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ad56d27f8a2285 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.195+0000 7f4e9e000640 1 -- 172.21.15.27:0/2311149126 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0016070 con 0x5627c4e71070 2024-09-21T07:24:01.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.195+0000 7f4e9e000640 1 -- 172.21.15.27:0/2311149126 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea0012bc0 con 0x5627c4e71070 2024-09-21T07:24:01.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.195+0000 7f4e9e000640 1 -- 172.21.15.27:0/2311149126 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea001bc50 con 0x5627c4e71070 2024-09-21T07:24:01.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4ea6000640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4e8c0b6680 0x7f4e8c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 -- 172.21.15.27:0/2311149126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 msgr2=0x5627c5217ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5217ba0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009fb0 tx=0x7f4ea0010800 comp rx=0 tx=0).stop 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4ead200640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4e8c0b6680 0x7f4e8c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4ea6000640 1 -- 172.21.15.27:0/2749879613 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f4e980e7f10 con 0x7f4ea8105d90 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 -- 172.21.15.27:0/2311149126 shutdown_connections 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5218620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c52180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2311149126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5217ba0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 -- 172.21.15.27:0/2311149126 >> 172.21.15.27:0/2311149126 conn(0x5627c4e65460 msgr2=0x5627c5215a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.196+0000 7f4eb6849740 1 -- 172.21.15.27:0/2311149126 shutdown_connections 2024-09-21T07:24:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.197+0000 7f4eb6849740 1 -- 172.21.15.27:0/2311149126 wait complete. 2024-09-21T07:24:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.197+0000 7f4ead200640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4e8c0b6680 0x7f4e8c0b8b90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5627c5216820 tx=0x7f4e98000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.197+0000 7f4eb6849740 1 -- start start 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.197+0000 7f4eb6849740 1 Processor -- start 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e481b0 0x5627c52237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e50390 0x5627c5223d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c522f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eb6849740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5627c4fdf700 con 0x5627c4e50390 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eb6849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5627c5119580 con 0x5627c4e71070 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eb6849740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627c5226380 con 0x5627c4e481b0 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eb6849740 10 client.0 _ll_register_callbacks cb 0x5627c5129470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4e9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e50390 0x5627c5223d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4eadc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e481b0 0x5627c52237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4e9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e50390 0x5627c5223d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39760/0 (socket says 172.21.15.27:39760) 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.198+0000 7f4e9fe00640 1 -- 172.21.15.27:0/1954137054 learned_addr learned my addr 172.21.15.27:0/1954137054 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4eadc00640 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c522f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4e9fe00640 1 -- 172.21.15.27:0/1954137054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e481b0 msgr2=0x5627c52237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e481b0 0x5627c52237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4e9fe00640 1 -- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 msgr2=0x5627c522f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c522f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4e9fe00640 1 -- 172.21.15.27:0/1954137054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c4edc650 con 0x5627c4e50390 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e50390 0x5627c5223d00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4e90005ab0 tx=0x7f4e90004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c66e2437b86a2db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.199+0000 7f4e96a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.200+0000 7f4e96a00640 1 -- 172.21.15.27:0/1954137054 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90035040 con 0x5627c4e50390 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.200+0000 7f4e96a00640 1 -- 172.21.15.27:0/1954137054 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f4e90005510 con 0x5627c4e50390 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.200+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627c511bb90 con 0x5627c4e50390 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.200+0000 7f4e96a00640 1 -- 172.21.15.27:0/1954137054 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4e9004f050 con 0x5627c4e50390 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.200+0000 7f4eb6849740 10 client.18651 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.200+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627c5136950 con 0x5627c4e50390 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4e96a00640 1 -- 172.21.15.27:0/1954137054 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5627c511bb90 con 0x5627c4e50390 2024-09-21T07:24:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4e96a00640 1 -- 172.21.15.27:0/1954137054 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4e900038c0 con 0x5627c4e50390 2024-09-21T07:24:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4eb6849740 10 client.18651 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4eb6849740 10 client.18651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:24:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4eb6849740 20 client.18651 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4eb6849740 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e2f9f0 0x5627c52356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4eb6849740 4 client.18651 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627c5190f00 con 0x5627c4e2f9f0 2024-09-21T07:24:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.201+0000 7f4eadc00640 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e2f9f0 0x5627c52356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.202+0000 7f4eadc00640 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e2f9f0 0x5627c52356f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc10adb4a5b9478b server_cookie=2e147ba54d3220e8 in_seq=0 out_seq=0 2024-09-21T07:24:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.202+0000 7f4e96a00640 10 client.18651 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4e96a00640 1 -- 172.21.15.27:0/1954137054 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627c5190f00 con 0x5627c4e2f9f0 2024-09-21T07:24:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4e96a00640 10 client.18651 handle_command_reply: tid=0 2024-09-21T07:24:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 client.18651 shutdown 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e50390 msgr2=0x5627c5223d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e50390 0x5627c5223d00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4e90005ab0 tx=0x7f4e90004510 comp rx=0 tx=0).stop 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 shutdown_connections 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e2f9f0 0x5627c52356f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c522f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e50390 0x5627c5223d00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 --2- 172.21.15.27:0/1954137054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e481b0 0x5627c52237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4e96a00640 0 client.18651 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 >> 172.21.15.27:0/1954137054 conn(0x5627c4e65460 msgr2=0x5627c521ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 shutdown_connections 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.203+0000 7f4eb6849740 1 -- 172.21.15.27:0/1954137054 wait complete. 2024-09-21T07:24:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.204+0000 7f4eb6849740 20 client.18651 trim_cache size 0 max 16384 2024-09-21T07:24:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 Processor -- start 2024-09-21T07:24:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 -- start start 2024-09-21T07:24:01.220 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:00 smithi027 ceph-mon[30779]: pgmap v717: 129 pgs: 129 active+clean; 3.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e481b0 0x5627c5221e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5222380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e71070 0x5627c522f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81b59a0 con 0x5627c4e71070 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81b3270 con 0x5627c4e481b0 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.218+0000 7f4eb6849740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81b10e0 con 0x5627c4e50390 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4eadc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e481b0 0x5627c5221e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4e9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5222380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4eadc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e481b0 0x5627c5221e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44132/0 (socket says 172.21.15.27:44132) 2024-09-21T07:24:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4eadc00640 1 -- 172.21.15.27:0/2124429339 learned_addr learned my addr 172.21.15.27:0/2124429339 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4e9fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5222380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46554/0 (socket says 172.21.15.27:46554) 2024-09-21T07:24:01.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4eadc00640 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e71070 0x5627c522f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4e9fe00640 1 -- 172.21.15.27:0/2124429339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e481b0 msgr2=0x5627c5221e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e481b0 0x5627c5221e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4e9fe00640 1 -- 172.21.15.27:0/2124429339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e71070 msgr2=0x5627c522f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e71070 0x5627c522f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.219+0000 7f4e9fe00640 1 -- 172.21.15.27:0/2124429339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c5224880 con 0x5627c4e50390 2024-09-21T07:24:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.220+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5222380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4e9002a9e0 tx=0x7f4e90001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e262fcb317f05f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.220+0000 7f4e9d600640 1 -- 172.21.15.27:0/2124429339 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90035020 con 0x5627c4e50390 2024-09-21T07:24:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.220+0000 7f4e9d600640 1 -- 172.21.15.27:0/2124429339 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e90041070 con 0x5627c4e50390 2024-09-21T07:24:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.220+0000 7f4e9d600640 1 -- 172.21.15.27:0/2124429339 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90031a30 con 0x5627c4e50390 2024-09-21T07:24:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 -- 172.21.15.27:0/2124429339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 msgr2=0x5627c5222380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5222380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4e9002a9e0 tx=0x7f4e90001e50 comp rx=0 tx=0).stop 2024-09-21T07:24:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 -- 172.21.15.27:0/2124429339 shutdown_connections 2024-09-21T07:24:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5222380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e71070 0x5627c522f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 --2- 172.21.15.27:0/2124429339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e481b0 0x5627c5221e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 -- 172.21.15.27:0/2124429339 >> 172.21.15.27:0/2124429339 conn(0x5627c4e65460 msgr2=0x5627c521e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.221+0000 7f4eb6849740 1 -- 172.21.15.27:0/2124429339 shutdown_connections 2024-09-21T07:24:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.222+0000 7f4eb6849740 1 -- 172.21.15.27:0/2124429339 wait complete. 2024-09-21T07:24:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.222+0000 7f4eb6849740 1 -- start start 2024-09-21T07:24:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.222+0000 7f4eb6849740 1 Processor -- start 2024-09-21T07:24:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c5227590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5227ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eb6849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5228010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eb6849740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81b1730 con 0x5627c4e481b0 2024-09-21T07:24:01.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eb6849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81b2ba0 con 0x5627c4e71070 2024-09-21T07:24:01.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eb6849740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81b1e00 con 0x5627c4e50390 2024-09-21T07:24:01.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eb6849740 10 client.0 _ll_register_callbacks cb 0x7f4e900041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:01.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eadc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c5227590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eadc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c5227590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39780/0 (socket says 172.21.15.27:39780) 2024-09-21T07:24:01.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4eadc00640 1 -- 172.21.15.27:0/4099246722 learned_addr learned my addr 172.21.15.27:0/4099246722 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.223+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5227ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4eadc00640 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5228010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4e9fe00640 1 -- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 msgr2=0x5627c5228010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5228010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4e9fe00640 1 -- 172.21.15.27:0/4099246722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 msgr2=0x5627c5227590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c5227590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4e9fe00640 1 -- 172.21.15.27:0/4099246722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c52a78e0 con 0x5627c4e50390 2024-09-21T07:24:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4e9fe00640 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5227ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e90003cc0 tx=0x7f4e90005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10502dd9becdee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.224+0000 7f4e96a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:24:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.225+0000 7f4e96a00640 1 -- 172.21.15.27:0/4099246722 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90035070 con 0x5627c4e50390 2024-09-21T07:24:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.225+0000 7f4e96a00640 1 -- 172.21.15.27:0/4099246722 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f4e90005680 con 0x5627c4e50390 2024-09-21T07:24:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.225+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e90041070 con 0x5627c4e50390 2024-09-21T07:24:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.225+0000 7f4e96a00640 1 -- 172.21.15.27:0/4099246722 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4e900314a0 con 0x5627c4e50390 2024-09-21T07:24:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.225+0000 7f4eb6849740 10 client.28651 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.225+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627c52a8b40 con 0x5627c4e50390 2024-09-21T07:24:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.225+0000 7f4e96a00640 1 -- 172.21.15.27:0/4099246722 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e90041070 con 0x5627c4e50390 2024-09-21T07:24:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4e96a00640 1 -- 172.21.15.27:0/4099246722 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4e90052070 con 0x5627c4e50390 2024-09-21T07:24:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4eb6849740 10 client.28651 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4eb6849740 10 client.28651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:24:01.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4eb6849740 20 client.28651 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:01.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4eb6849740 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e04da0 0x5627c52ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4eb6849740 4 client.28651 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:24:01.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627c518c9b0 con 0x5627c4e04da0 2024-09-21T07:24:01.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.226+0000 7f4eadc00640 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e04da0 0x5627c52ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eadc00640 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e04da0 0x5627c52ab1f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6a778329eedf25b server_cookie=4947aa5f5ae76fdf in_seq=0 out_seq=0 2024-09-21T07:24:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4e96a00640 10 client.28651 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4e96a00640 1 -- 172.21.15.27:0/4099246722 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627c518c9b0 con 0x5627c4e04da0 2024-09-21T07:24:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4e96a00640 10 client.28651 handle_command_reply: tid=0 2024-09-21T07:24:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 client.28651 shutdown 2024-09-21T07:24:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 msgr2=0x5627c5227ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5227ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e90003cc0 tx=0x7f4e90005a40 comp rx=0 tx=0).stop 2024-09-21T07:24:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 shutdown_connections 2024-09-21T07:24:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5627c4e04da0 0x5627c52ab1f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627c4e50390 0x5627c5227ad0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5627c4e71070 0x5627c5228010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 --2- 172.21.15.27:0/4099246722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5627c4e481b0 0x5627c5227590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.227+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 >> 172.21.15.27:0/4099246722 conn(0x5627c4e65460 msgr2=0x5627c521e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.228+0000 7f4e96a00640 0 client.28651 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.228+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 shutdown_connections 2024-09-21T07:24:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.228+0000 7f4eb6849740 1 -- 172.21.15.27:0/4099246722 wait complete. 2024-09-21T07:24:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.228+0000 7f4eb6849740 20 client.28651 trim_cache size 0 max 16384 2024-09-21T07:24:01.240 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:24:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4e8c0b6680 msgr2=0x7f4e8c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4e8c0b6680 0x7f4e8c0b8b90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5627c5216820 tx=0x7f4e98000bc0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea8105d90 msgr2=0x7f4ea819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea8105d90 0x7f4ea819c6c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800e9f0 tx=0x7f4e9800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 shutdown_connections 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4e8c0b6680 0x7f4e8c0b8b90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea810c020 0x7f4ea81a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ea810b6d0 0x7f4ea819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 --2- 172.21.15.27:0/2749879613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ea8105d90 0x7f4ea819c6c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 >> 172.21.15.27:0/2749879613 conn(0x7f4ea8101860 msgr2=0x7f4ea8108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 shutdown_connections 2024-09-21T07:24:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:01.229+0000 7f4eafc00640 1 -- 172.21.15.27:0/2749879613 wait complete. 2024-09-21T07:24:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:00 smithi119 ceph-mon[34519]: pgmap v717: 129 pgs: 129 active+clean; 3.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:24:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.348+0000 7f97c1400640 1 -- 172.21.15.27:0/380022523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc103b00 msgr2=0x7f97bc103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.348+0000 7f97c1400640 1 --2- 172.21.15.27:0/380022523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc103b00 0x7f97bc103f10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009900 tx=0x7f97ac02a410 comp rx=0 tx=0).stop 2024-09-21T07:24:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.348+0000 7f97c1400640 1 -- 172.21.15.27:0/380022523 shutdown_connections 2024-09-21T07:24:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.348+0000 7f97c1400640 1 --2- 172.21.15.27:0/380022523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc109d80 0x7f97bc1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.348+0000 7f97c1400640 1 --2- 172.21.15.27:0/380022523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97bc109430 0x7f97bc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.348+0000 7f97c1400640 1 --2- 172.21.15.27:0/380022523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc103b00 0x7f97bc103f10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.348+0000 7f97c1400640 1 -- 172.21.15.27:0/380022523 >> 172.21.15.27:0/380022523 conn(0x7f97bc0ff500 msgr2=0x7f97bc101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.349+0000 7f97c1400640 1 -- 172.21.15.27:0/380022523 shutdown_connections 2024-09-21T07:24:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.349+0000 7f97c1400640 1 -- 172.21.15.27:0/380022523 wait complete. 2024-09-21T07:24:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 Processor -- start 2024-09-21T07:24:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 -- start start 2024-09-21T07:24:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc103b00 0x7f97bc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97bc109430 0x7f97bc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc109d80 0x7f97bc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc113a40 con 0x7f97bc103b00 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1138c0 con 0x7f97bc109430 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.350+0000 7f97c1400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc113bc0 con 0x7f97bc109d80 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc103b00 0x7f97bc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc103b00 0x7f97bc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39794/0 (socket says 172.21.15.27:39794) 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 -- 172.21.15.27:0/148769844 learned_addr learned my addr 172.21.15.27:0/148769844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97b1e00640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97bc109430 0x7f97bc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97bb200640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc109d80 0x7f97bc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 -- 172.21.15.27:0/148769844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc109d80 msgr2=0x7f97bc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc109d80 0x7f97bc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 -- 172.21.15.27:0/148769844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97bc109430 msgr2=0x7f97bc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97bc109430 0x7f97bc19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97ba800640 1 -- 172.21.15.27:0/148769844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc1a03b0 con 0x7f97bc103b00 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.351+0000 7f97b1e00640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97bc109430 0x7f97bc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.352+0000 7f97ba800640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc103b00 0x7f97bc19a480 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005940 tx=0x7f97ac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.352+0000 7f97b3e00640 1 -- 172.21.15.27:0/148769844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac03f030 con 0x7f97bc103b00 2024-09-21T07:24:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.352+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc1a0640 con 0x7f97bc103b00 2024-09-21T07:24:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.352+0000 7f97b3e00640 1 -- 172.21.15.27:0/148769844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac0044c0 con 0x7f97bc103b00 2024-09-21T07:24:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.352+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc1a6e30 con 0x7f97bc103b00 2024-09-21T07:24:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.352+0000 7f97b3e00640 1 -- 172.21.15.27:0/148769844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac049060 con 0x7f97bc103b00 2024-09-21T07:24:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.355+0000 7f97c84d7740 1 Processor -- start 2024-09-21T07:24:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.355+0000 7f97c84d7740 1 -- start start 2024-09-21T07:24:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fdc61b0 0x55d6a016ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a016b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fda54d0 0x55d6a016b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97c84d7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d6a00e06b0 con 0x55d69fdc61b0 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97c84d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d6a00cfae0 con 0x55d69fd9d2f0 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97c84d7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d6a00d38f0 con 0x55d69fda54d0 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97b3e00640 1 -- 172.21.15.27:0/148769844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f97ac042030 con 0x7f97bc103b00 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97bb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fdc61b0 0x55d6a016ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97bb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fdc61b0 0x55d6a016ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39800/0 (socket says 172.21.15.27:39800) 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97bb200640 1 -- 172.21.15.27:0/4279802048 learned_addr learned my addr 172.21.15.27:0/4279802048 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97ba800640 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a016b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97b1e00640 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fda54d0 0x55d6a016b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.356+0000 7f97bb200640 1 -- 172.21.15.27:0/4279802048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fda54d0 msgr2=0x55d6a016b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97bb200640 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fda54d0 0x55d6a016b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97bb200640 1 -- 172.21.15.27:0/4279802048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 msgr2=0x55d6a016b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97bb200640 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a016b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97bb200640 1 -- 172.21.15.27:0/4279802048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a009c780 con 0x55d69fdc61b0 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97b1e00640 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fda54d0 0x55d6a016b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97bb200640 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fdc61b0 0x55d6a016ac30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009fd0 tx=0x7f97b40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518420b3aceea899 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97ab400640 1 -- 172.21.15.27:0/4279802048 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4004d20 con 0x55d69fdc61b0 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97ab400640 1 -- 172.21.15.27:0/4279802048 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b4004050 con 0x55d69fdc61b0 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.357+0000 7f97ab400640 1 -- 172.21.15.27:0/4279802048 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4005260 con 0x55d69fdc61b0 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97b3e00640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97a00b6680 0x7f97a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97b3e00640 1 -- 172.21.15.27:0/148769844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f97ac107a90 con 0x7f97bc103b00 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97ba800640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97a00b6680 0x7f97a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 -- 172.21.15.27:0/4279802048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fdc61b0 msgr2=0x55d6a016ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fdc61b0 0x55d6a016ac30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009fd0 tx=0x7f97b40118d0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 -- 172.21.15.27:0/4279802048 shutdown_connections 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fda54d0 0x55d6a016b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a016b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 --2- 172.21.15.27:0/4279802048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fdc61b0 0x55d6a016ac30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 -- 172.21.15.27:0/4279802048 >> 172.21.15.27:0/4279802048 conn(0x55d69fdba300 msgr2=0x55d6a0168b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 -- 172.21.15.27:0/4279802048 shutdown_connections 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.358+0000 7f97c84d7740 1 -- 172.21.15.27:0/4279802048 wait complete. 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.359+0000 7f97ba800640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97a00b6680 0x7f97a00b8b90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55d6a01698b0 tx=0x7f97ac10b540 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.359+0000 7f97c84d7740 1 -- start start 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.359+0000 7f97c84d7740 1 Processor -- start 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a017a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97c84d7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d6a0007ab0 con 0x55d69fda54d0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97c84d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d6a0076570 con 0x55d69fdc61b0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97c84d7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d6a017d790 con 0x55d69fd9d2f0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97c84d7740 10 client.0 _ll_register_callbacks cb 0x55d6a0085100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39802/0 (socket says 172.21.15.27:39802) 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97b1e00640 1 -- 172.21.15.27:0/307491954 learned_addr learned my addr 172.21.15.27:0/307491954 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97bb200640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a017a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.360+0000 7f97bb200640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97b1e00640 1 -- 172.21.15.27:0/307491954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 msgr2=0x55d6a017a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97b1e00640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a017a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97b1e00640 1 -- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 msgr2=0x55d6a017b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97b1e00640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97b1e00640 1 -- 172.21.15.27:0/307491954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d69fe31790 con 0x55d69fda54d0 2024-09-21T07:24:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97bb200640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97b1e00640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017ab20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f97a4012a10 tx=0x7f97a4012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cc6dbb4b416f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f979fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f979fe00640 1 -- 172.21.15.27:0/307491954 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a400fa60 con 0x55d69fda54d0 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f979fe00640 1 -- 172.21.15.27:0/307491954 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f97a400fc00 con 0x55d69fda54d0 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b4004050 con 0x55d69fda54d0 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f979fe00640 1 -- 172.21.15.27:0/307491954 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97a400fe00 con 0x55d69fda54d0 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97c84d7740 10 client.18669 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.361+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d69feaa620 con 0x55d69fda54d0 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f979fe00640 1 -- 172.21.15.27:0/307491954 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b4004050 con 0x55d69fda54d0 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f979fe00640 1 -- 172.21.15.27:0/307491954 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f97a40105f0 con 0x55d69fda54d0 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f97c84d7740 10 client.18669 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f97c84d7740 10 client.18669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f97c84d7740 20 client.18669 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f97c84d7740 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd84d80 0x55d6a018c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f97c84d7740 4 client.18669 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6a00e2600 con 0x55d69fd84d80 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.362+0000 7f97bb200640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd84d80 0x55d6a018c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.363+0000 7f97bb200640 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd84d80 0x55d6a018c7a0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7437f7043b967442 server_cookie=89c1467ab8f167c1 in_seq=0 out_seq=0 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.363+0000 7f979fe00640 10 client.18669 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f979fe00640 1 -- 172.21.15.27:0/307491954 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d6a00e2600 con 0x55d69fd84d80 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f979fe00640 10 client.18669 handle_command_reply: tid=0 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 client.18669 shutdown 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 msgr2=0x55d6a017ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017ab20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f97a4012a10 tx=0x7f97a4012ed0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 shutdown_connections 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd84d80 0x55d6a018c7a0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017ab20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f979fe00640 0 client.18669 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 --2- 172.21.15.27:0/307491954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a017a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 >> 172.21.15.27:0/307491954 conn(0x55d69fdba300 msgr2=0x55d6a016c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 shutdown_connections 2024-09-21T07:24:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.364+0000 7f97c84d7740 1 -- 172.21.15.27:0/307491954 wait complete. 2024-09-21T07:24:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.365+0000 7f97c84d7740 20 client.18669 trim_cache size 0 max 16384 2024-09-21T07:24:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 Processor -- start 2024-09-21T07:24:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 -- start start 2024-09-21T07:24:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a0175a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fdc61b0 0x55d6a017b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1b15c0 con 0x55d69fda54d0 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1aee90 con 0x55d69fd9d2f0 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.380+0000 7f97c84d7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1acd00 con 0x55d69fdc61b0 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97bb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a0175a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39814/0 (socket says 172.21.15.27:39814) 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 -- 172.21.15.27:0/1141124374 learned_addr learned my addr 172.21.15.27:0/1141124374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97bb200640 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fdc61b0 0x55d6a017b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 -- 172.21.15.27:0/1141124374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fdc61b0 msgr2=0x55d6a017b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fdc61b0 0x55d6a017b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 -- 172.21.15.27:0/1141124374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 msgr2=0x55d6a0175a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a0175a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 -- 172.21.15.27:0/1141124374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a016d390 con 0x55d69fda54d0 2024-09-21T07:24:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97bb200640 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fdc61b0 0x55d6a017b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97b1e00640 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017aac0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f97a40129e0 tx=0x7f97a40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ce1bc793cee74f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.381+0000 7f97aaa00640 1 -- 172.21.15.27:0/1141124374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a400fb20 con 0x55d69fda54d0 2024-09-21T07:24:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97aaa00640 1 -- 172.21.15.27:0/1141124374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a4005040 con 0x55d69fda54d0 2024-09-21T07:24:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97aaa00640 1 -- 172.21.15.27:0/1141124374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4003830 con 0x55d69fda54d0 2024-09-21T07:24:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 -- 172.21.15.27:0/1141124374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 msgr2=0x55d6a017aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017aac0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f97a40129e0 tx=0x7f97a40092c0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 -- 172.21.15.27:0/1141124374 shutdown_connections 2024-09-21T07:24:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fdc61b0 0x55d6a017b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017aac0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 --2- 172.21.15.27:0/1141124374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fd9d2f0 0x55d6a0175a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 -- 172.21.15.27:0/1141124374 >> 172.21.15.27:0/1141124374 conn(0x55d69fdba300 msgr2=0x55d6a0175370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.382+0000 7f97c84d7740 1 -- 172.21.15.27:0/1141124374 shutdown_connections 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.383+0000 7f97c84d7740 1 -- 172.21.15.27:0/1141124374 wait complete. 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.383+0000 7f97c84d7740 1 -- start start 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.383+0000 7f97c84d7740 1 Processor -- start 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.383+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a0182fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97c84d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97c84d7740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1ad350 con 0x55d69fda54d0 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97c84d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1ae7c0 con 0x55d69fdc61b0 2024-09-21T07:24:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97c84d7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1ada20 con 0x55d69fd9d2f0 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97c84d7740 10 client.0 _ll_register_callbacks cb 0x7f97a4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39820/0 (socket says 172.21.15.27:39820) 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 -- 172.21.15.27:0/3227343637 learned_addr learned my addr 172.21.15.27:0/3227343637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97bb200640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a0182fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97bb200640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 -- 172.21.15.27:0/3227343637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 msgr2=0x55d6a0182fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a0182fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 -- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 msgr2=0x55d6a017e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 -- 172.21.15.27:0/3227343637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6a017ef20 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97bb200640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a0182fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f97b1e00640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017e470 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f97a4012f10 tx=0x7f97a40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3a45789a677c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.384+0000 7f979fe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.385+0000 7f979fe00640 1 -- 172.21.15.27:0/3227343637 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a402e070 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.385+0000 7f979fe00640 1 -- 172.21.15.27:0/3227343637 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7f97a4005500 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.385+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a4005040 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.385+0000 7f979fe00640 1 -- 172.21.15.27:0/3227343637 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97a40035f0 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.385+0000 7f97c84d7740 10 client.18675 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.385+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6a01f6ef0 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.385+0000 7f979fe00640 1 -- 172.21.15.27:0/3227343637 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a4005040 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f979fe00640 1 -- 172.21.15.27:0/3227343637 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f97a40334f0 con 0x55d69fda54d0 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97c84d7740 10 client.18675 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97c84d7740 10 client.18675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97c84d7740 20 client.18675 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97c84d7740 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd5a1c0 0x55d6a01f9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97c84d7740 4 client.18675 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:24:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6a00de0b0 con 0x55d69fd5a1c0 2024-09-21T07:24:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97bb200640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd5a1c0 0x55d6a01f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f97bb200640 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd5a1c0 0x55d6a01f9610 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb16abf4aec4bb2 server_cookie=6c5a0a01e527a1f7 in_seq=0 out_seq=0 2024-09-21T07:24:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.386+0000 7f979fe00640 10 client.18675 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.400+0000 7f979fe00640 1 -- 172.21.15.27:0/3227343637 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6a00de0b0 con 0x55d69fd5a1c0 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.400+0000 7f979fe00640 10 client.18675 handle_command_reply: tid=0 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.400+0000 7f97c84d7740 1 client.18675 shutdown 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 msgr2=0x55d6a017e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017e470 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f97a4012f10 tx=0x7f97a40096c0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 shutdown_connections 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d69fd5a1c0 0x55d6a01f9610 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69fdc61b0 0x55d6a017e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d69fda54d0 0x55d6a017e470 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 --2- 172.21.15.27:0/3227343637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d69fd9d2f0 0x55d6a0182fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f979fe00640 0 client.18675 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 >> 172.21.15.27:0/3227343637 conn(0x55d69fdba300 msgr2=0x55d6a0168af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 shutdown_connections 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 1 -- 172.21.15.27:0/3227343637 wait complete. 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.401+0000 7f97c84d7740 20 client.18675 trim_cache size 0 max 16384 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:24:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:24:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:24:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97a00b6680 msgr2=0x7f97a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97a00b6680 0x7f97a00b8b90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55d6a01698b0 tx=0x7f97ac10b540 comp rx=0 tx=0).stop 2024-09-21T07:24:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc103b00 msgr2=0x7f97bc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc103b00 0x7f97bc19a480 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005940 tx=0x7f97ac0040c0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 shutdown_connections 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97bc109d80 0x7f97bc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f97a00b6680 0x7f97a00b8b90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97bc109430 0x7f97bc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 --2- 172.21.15.27:0/148769844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc103b00 0x7f97bc19a480 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 >> 172.21.15.27:0/148769844 conn(0x7f97bc0ff500 msgr2=0x7f97bc108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 shutdown_connections 2024-09-21T07:24:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:01.402+0000 7f97c1400640 1 -- 172.21.15.27:0/148769844 wait complete. 2024-09-21T07:24:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:00 smithi153 ceph-mon[34747]: pgmap v717: 129 pgs: 129 active+clean; 3.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-09-21T07:24:02.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:24:02.470 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 -- 172.21.15.27:0/4199093444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1092f0 msgr2=0x7fd63c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 --2- 172.21.15.27:0/4199093444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1092f0 0x7fd63c109700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd628009900 tx=0x7fd62802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 -- 172.21.15.27:0/4199093444 shutdown_connections 2024-09-21T07:24:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 --2- 172.21.15.27:0/4199093444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd63c109c40 0x7fd63c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 --2- 172.21.15.27:0/4199093444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1092f0 0x7fd63c109700 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 --2- 172.21.15.27:0/4199093444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd63c1039b0 0x7fd63c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 -- 172.21.15.27:0/4199093444 >> 172.21.15.27:0/4199093444 conn(0x7fd63c0ff440 msgr2=0x7fd63c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 -- 172.21.15.27:0/4199093444 shutdown_connections 2024-09-21T07:24:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.117+0000 7fd640c00640 1 -- 172.21.15.27:0/4199093444 wait complete. 2024-09-21T07:24:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.118+0000 7fd640c00640 1 Processor -- start 2024-09-21T07:24:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd640c00640 1 -- start start 2024-09-21T07:24:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd640c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1039b0 0x7fd63c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd640c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd63c1092f0 0x7fd63c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd640c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd63c109c40 0x7fd63c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd640c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c113950 con 0x7fd63c1039b0 2024-09-21T07:24:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd640c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1137d0 con 0x7fd63c109c40 2024-09-21T07:24:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd640c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c113ad0 con 0x7fd63c1092f0 2024-09-21T07:24:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd63b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1039b0 0x7fd63c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd63b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1039b0 0x7fd63c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39840/0 (socket says 172.21.15.27:39840) 2024-09-21T07:24:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.119+0000 7fd63b400640 1 -- 172.21.15.27:0/3416617629 learned_addr learned my addr 172.21.15.27:0/3416617629 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd63c1092f0 0x7fd63c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63be00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd63c109c40 0x7fd63c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63aa00640 1 -- 172.21.15.27:0/3416617629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd63c109c40 msgr2=0x7fd63c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd63c109c40 0x7fd63c19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63aa00640 1 -- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1039b0 msgr2=0x7fd63c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1039b0 0x7fd63c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63aa00640 1 -- 172.21.15.27:0/3416617629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c1a0270 con 0x7fd63c1092f0 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63be00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd63c109c40 0x7fd63c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63b400640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1039b0 0x7fd63c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:03.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.120+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd63c1092f0 0x7fd63c19a8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd628009230 tx=0x7fd628003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:03.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.121+0000 7fd633e00640 1 -- 172.21.15.27:0/3416617629 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62803f070 con 0x7fd63c1092f0 2024-09-21T07:24:03.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.121+0000 7fd633e00640 1 -- 172.21.15.27:0/3416617629 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd628031b50 con 0x7fd63c1092f0 2024-09-21T07:24:03.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.121+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c1a0500 con 0x7fd63c1092f0 2024-09-21T07:24:03.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.121+0000 7fd633e00640 1 -- 172.21.15.27:0/3416617629 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628035030 con 0x7fd63c1092f0 2024-09-21T07:24:03.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.121+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1a6cc0 con 0x7fd63c1092f0 2024-09-21T07:24:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.124+0000 7fd648ddd740 1 Processor -- start 2024-09-21T07:24:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.124+0000 7fd648ddd740 1 -- start start 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b80070 0x55fce5f2b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f2b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b5f390 0x55fce5f2be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd648ddd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fce5e9dfb0 con 0x55fce5b80070 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd648ddd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fce5e8d3e0 con 0x55fce5b5f390 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd648ddd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fce5e911f0 con 0x55fce5b571b0 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd63be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b80070 0x55fce5f2b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd63be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b80070 0x55fce5f2b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39844/0 (socket says 172.21.15.27:39844) 2024-09-21T07:24:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd63be00640 1 -- 172.21.15.27:0/420082972 learned_addr learned my addr 172.21.15.27:0/420082972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.125+0000 7fd63b400640 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f2b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd63be00640 1 -- 172.21.15.27:0/420082972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 msgr2=0x55fce5f2b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:03.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd63be00640 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f2b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd63be00640 1 -- 172.21.15.27:0/420082972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b5f390 msgr2=0x55fce5f2be00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd63be00640 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b5f390 0x55fce5f2be00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd63be00640 1 -- 172.21.15.27:0/420082972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce5e57cd0 con 0x55fce5b80070 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd633e00640 1 -- 172.21.15.27:0/3416617629 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd628042070 con 0x7fd63c1092f0 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd63be00640 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b80070 0x55fce5f2b380 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd6340117e0 tx=0x7fd634011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ee8e37b7bc3808 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd627e00640 1 -- 172.21.15.27:0/420082972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634011310 con 0x55fce5b80070 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd627e00640 1 -- 172.21.15.27:0/420082972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd634003e20 con 0x55fce5b80070 2024-09-21T07:24:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.126+0000 7fd627e00640 1 -- 172.21.15.27:0/420082972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6340044f0 con 0x55fce5b80070 2024-09-21T07:24:03.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 -- 172.21.15.27:0/420082972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b80070 msgr2=0x55fce5f2b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b80070 0x55fce5f2b380 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd6340117e0 tx=0x7fd634011ca0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 -- 172.21.15.27:0/420082972 shutdown_connections 2024-09-21T07:24:03.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f2b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b5f390 0x55fce5f2be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 --2- 172.21.15.27:0/420082972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b80070 0x55fce5f2b380 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 -- 172.21.15.27:0/420082972 >> 172.21.15.27:0/420082972 conn(0x55fce5b74460 msgr2=0x55fce5f29270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 -- 172.21.15.27:0/420082972 shutdown_connections 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.127+0000 7fd648ddd740 1 -- 172.21.15.27:0/420082972 wait complete. 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.128+0000 7fd633e00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd6200b6590 0x7fd6200b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.128+0000 7fd633e00640 1 -- 172.21.15.27:0/3416617629 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fd6281080a0 con 0x7fd63c1092f0 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.128+0000 7fd63b400640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd6200b6590 0x7fd6200b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.128+0000 7fd648ddd740 1 -- start start 2024-09-21T07:24:03.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.128+0000 7fd648ddd740 1 Processor -- start 2024-09-21T07:24:03.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd63b400640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd6200b6590 0x7fd6200b8aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55fce5f2a000 tx=0x7fd62c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:03.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f36c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b80070 0x55fce5f42e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd648ddd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fce5cee700 con 0x55fce5b5f390 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd648ddd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fce5e28580 con 0x55fce5b80070 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd648ddd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fce5f39b60 con 0x55fce5b571b0 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd648ddd740 10 client.0 _ll_register_callbacks cb 0x55fce5e38470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd63b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f36c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd63b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f36c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46634/0 (socket says 172.21.15.27:46634) 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd63aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.129+0000 7fd63b400640 1 -- 172.21.15.27:0/3545714116 learned_addr learned my addr 172.21.15.27:0/3545714116 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63be00640 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b80070 0x55fce5f42e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63aa00640 1 -- 172.21.15.27:0/3545714116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 msgr2=0x55fce5f36c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f36c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63aa00640 1 -- 172.21.15.27:0/3545714116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b80070 msgr2=0x55fce5f42e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b80070 0x55fce5f42e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63aa00640 1 -- 172.21.15.27:0/3545714116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce5beb650 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63b400640 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f36c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f371c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd6281053e0 tx=0x7fd6281058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a2920649b8e3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.130+0000 7fd624c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd624c00640 1 -- 172.21.15.27:0/3545714116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628049070 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd624c00640 1 -- 172.21.15.27:0/3545714116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fd6280316a0 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fce5e2ab90 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd624c00640 1 -- 172.21.15.27:0/3545714116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6280cd4e0 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd648ddd740 10 client.18687 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fce5e45950 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd624c00640 1 -- 172.21.15.27:0/3545714116 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fce5e2ab90 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd624c00640 1 -- 172.21.15.27:0/3545714116 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd628105b30 con 0x55fce5b5f390 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd648ddd740 10 client.18687 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.131+0000 7fd648ddd740 10 client.18687 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:24:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.132+0000 7fd648ddd740 20 client.18687 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.132+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b3e9f0 0x55fce5f48c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.132+0000 7fd648ddd740 4 client.18687 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.132+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce5e9ff00 con 0x55fce5b3e9f0 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.132+0000 7fd63b400640 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b3e9f0 0x55fce5f48c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.132+0000 7fd63b400640 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b3e9f0 0x55fce5f48c30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e22ec467500b008 server_cookie=ae04aec0b2c9c4d5 in_seq=0 out_seq=0 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.132+0000 7fd624c00640 10 client.18687 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.133+0000 7fd624c00640 1 -- 172.21.15.27:0/3545714116 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fce5e9ff00 con 0x55fce5b3e9f0 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.133+0000 7fd624c00640 10 client.18687 handle_command_reply: tid=0 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.133+0000 7fd648ddd740 1 client.18687 shutdown 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 msgr2=0x55fce5f371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f371c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd6281053e0 tx=0x7fd6281058c0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 shutdown_connections 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b3e9f0 0x55fce5f48c30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b80070 0x55fce5f42e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd624c00640 0 client.18687 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f371c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3545714116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b571b0 0x55fce5f36c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 >> 172.21.15.27:0/3545714116 conn(0x55fce5b74460 msgr2=0x55fce5f2e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 shutdown_connections 2024-09-21T07:24:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 1 -- 172.21.15.27:0/3545714116 wait complete. 2024-09-21T07:24:03.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.134+0000 7fd648ddd740 20 client.18687 trim_cache size 0 max 16384 2024-09-21T07:24:03.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.148+0000 7fd648ddd740 1 Processor -- start 2024-09-21T07:24:03.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.148+0000 7fd648ddd740 1 -- start start 2024-09-21T07:24:03.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f33e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd648ddd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b1370 con 0x55fce5b5f390 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd648ddd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1aec40 con 0x55fce5b571b0 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd648ddd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1acab0 con 0x55fce5b80070 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd63b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd63b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f29190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44236/0 (socket says 172.21.15.27:44236) 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd63b400640 1 -- 172.21.15.27:0/3265134596 learned_addr learned my addr 172.21.15.27:0/3265134596 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd63be00640 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd63aa00640 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f33e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.149+0000 7fd63b400640 1 -- 172.21.15.27:0/3265134596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 msgr2=0x55fce5f34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:03.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd63b400640 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd63b400640 1 -- 172.21.15.27:0/3265134596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 msgr2=0x55fce5f33e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:03.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd63b400640 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f33e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd63b400640 1 -- 172.21.15.27:0/3265134596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce5f37ec0 con 0x55fce5b571b0 2024-09-21T07:24:03.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd63b400640 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f29190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55fce5f492f0 tx=0x7fd62c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193f72f3442a3070 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd627400640 1 -- 172.21.15.27:0/3265134596 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00f970 con 0x55fce5b571b0 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd627400640 1 -- 172.21.15.27:0/3265134596 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62c02b470 con 0x55fce5b571b0 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.150+0000 7fd627400640 1 -- 172.21.15.27:0/3265134596 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01f660 con 0x55fce5b571b0 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 -- 172.21.15.27:0/3265134596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 msgr2=0x55fce5f29190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f29190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55fce5f492f0 tx=0x7fd62c01b000 comp rx=0 tx=0).stop 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 -- 172.21.15.27:0/3265134596 shutdown_connections 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f33e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 --2- 172.21.15.27:0/3265134596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f29190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 -- 172.21.15.27:0/3265134596 >> 172.21.15.27:0/3265134596 conn(0x55fce5b74460 msgr2=0x55fce5f28a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 -- 172.21.15.27:0/3265134596 shutdown_connections 2024-09-21T07:24:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.151+0000 7fd648ddd740 1 -- 172.21.15.27:0/3265134596 wait complete. 2024-09-21T07:24:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.152+0000 7fd648ddd740 1 -- start start 2024-09-21T07:24:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.152+0000 7fd648ddd740 1 Processor -- start 2024-09-21T07:24:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.152+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f3efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f3f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd648ddd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd648ddd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1ad100 con 0x55fce5b5f390 2024-09-21T07:24:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd648ddd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1ae570 con 0x55fce5b571b0 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd648ddd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1ad7d0 con 0x55fce5b80070 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd648ddd740 10 client.0 _ll_register_callbacks cb 0x7fd628005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f3efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f3f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f3efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44244/0 (socket says 172.21.15.27:44244) 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63b400640 1 -- 172.21.15.27:0/759062003 learned_addr learned my addr 172.21.15.27:0/759062003 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63be00640 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63aa00640 1 -- 172.21.15.27:0/759062003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 msgr2=0x55fce5f39d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63aa00640 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f39d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63aa00640 1 -- 172.21.15.27:0/759062003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 msgr2=0x55fce5f3efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63aa00640 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f3efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63aa00640 1 -- 172.21.15.27:0/759062003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce5f3a330 con 0x55fce5b5f390 2024-09-21T07:24:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.153+0000 7fd63be00640 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd63aa00640 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f3f500 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd6280cdb20 tx=0x7fd628031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e19a962f553052 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd624c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd624c00640 1 -- 172.21.15.27:0/759062003 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628049070 con 0x55fce5b5f390 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd624c00640 1 -- 172.21.15.27:0/759062003 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9461+0+0 (secure 0 0 0) 0x7fd62803bb40 con 0x55fce5b5f390 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62c02b470 con 0x55fce5b5f390 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd624c00640 1 -- 172.21.15.27:0/759062003 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6280cd300 con 0x55fce5b5f390 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd648ddd740 10 client.18693 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fce5fb3d60 con 0x55fce5b5f390 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.154+0000 7fd624c00640 1 -- 172.21.15.27:0/759062003 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62c02b470 con 0x55fce5b5f390 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd624c00640 1 -- 172.21.15.27:0/759062003 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6280efaf0 con 0x55fce5b5f390 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd648ddd740 10 client.18693 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd648ddd740 10 client.18693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd648ddd740 20 client.18693 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd648ddd740 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b13da0 0x55fce5fb63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:03.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd648ddd740 4 client.18693 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:24:03.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fce5e9b9b0 con 0x55fce5b13da0 2024-09-21T07:24:03.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.155+0000 7fd63b400640 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b13da0 0x55fce5fb63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd63b400640 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b13da0 0x55fce5fb63e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28b4d1e54bad883f server_cookie=4958e351b005bbe4 in_seq=0 out_seq=0 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd624c00640 10 client.18693 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd624c00640 1 -- 172.21.15.27:0/759062003 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fce5e9b9b0 con 0x55fce5b13da0 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd624c00640 10 client.18693 handle_command_reply: tid=0 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 client.18693 shutdown 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 msgr2=0x55fce5f3f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f3f500 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd6280cdb20 tx=0x7fd628031400 comp rx=0 tx=0).stop 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 shutdown_connections 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55fce5b13da0 0x55fce5fb63e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fce5b80070 0x55fce5f39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fce5b5f390 0x55fce5f3f500 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 --2- 172.21.15.27:0/759062003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fce5b571b0 0x55fce5f3efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 >> 172.21.15.27:0/759062003 conn(0x55fce5b74460 msgr2=0x55fce5f2c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.156+0000 7fd624c00640 0 client.18693 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.157+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 shutdown_connections 2024-09-21T07:24:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.157+0000 7fd648ddd740 1 -- 172.21.15.27:0/759062003 wait complete. 2024-09-21T07:24:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.157+0000 7fd648ddd740 20 client.18693 trim_cache size 0 max 16384 2024-09-21T07:24:03.162 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.157+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd6200b6590 msgr2=0x7fd6200b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.157+0000 7fd640c00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd6200b6590 0x7fd6200b8aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55fce5f2a000 tx=0x7fd62c005f90 comp rx=0 tx=0).stop 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.157+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd63c1092f0 msgr2=0x7fd63c19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.157+0000 7fd640c00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd63c1092f0 0x7fd63c19a8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd628009230 tx=0x7fd628003b90 comp rx=0 tx=0).stop 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 shutdown_connections 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd63c1092f0 0x7fd63c19a8a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd6200b6590 0x7fd6200b8aa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd63c109c40 0x7fd63c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 --2- 172.21.15.27:0/3416617629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c1039b0 0x7fd63c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:03.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 >> 172.21.15.27:0/3416617629 conn(0x7fd63c0ff440 msgr2=0x7fd63c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:03.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 shutdown_connections 2024-09-21T07:24:03.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:03.158+0000 7fd640c00640 1 -- 172.21.15.27:0/3416617629 wait complete. 2024-09-21T07:24:03.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:02 smithi027 ceph-mon[30779]: pgmap v718: 129 pgs: 129 active+clean; 3.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:24:03.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:02 smithi119 ceph-mon[34519]: pgmap v718: 129 pgs: 129 active+clean; 3.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:24:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:02 smithi153 ceph-mon[34747]: pgmap v718: 129 pgs: 129 active+clean; 3.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:24:04.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:24:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 -- 172.21.15.27:0/4288537411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 msgr2=0x7f4b74112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 --2- 172.21.15.27:0/4288537411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 0x7f4b74112840 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009ca0 tx=0x7f4b6c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 -- 172.21.15.27:0/4288537411 shutdown_connections 2024-09-21T07:24:05.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 --2- 172.21.15.27:0/4288537411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7410b6c0 0x7f4b7410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 --2- 172.21.15.27:0/4288537411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 0x7f4b74112840 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 --2- 172.21.15.27:0/4288537411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74105d80 0x7f4b74106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 -- 172.21.15.27:0/4288537411 >> 172.21.15.27:0/4288537411 conn(0x7f4b74101810 msgr2=0x7f4b74103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:05.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 -- 172.21.15.27:0/4288537411 shutdown_connections 2024-09-21T07:24:05.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.077+0000 7f4b7b400640 1 -- 172.21.15.27:0/4288537411 wait complete. 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.078+0000 7f4b7b400640 1 Processor -- start 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b7b400640 1 -- start start 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b7b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74105d80 0x7f4b7419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b7b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7410b6c0 0x7f4b7419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b7b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 0x7f4b741a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b7b400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74115cd0 con 0x7f4b7410c010 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b7b400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74115b50 con 0x7f4b74105d80 2024-09-21T07:24:05.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b7b400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74115e50 con 0x7f4b7410b6c0 2024-09-21T07:24:05.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 0x7f4b741a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 0x7f4b741a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39886/0 (socket says 172.21.15.27:39886) 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b79400640 1 -- 172.21.15.27:0/2494704577 learned_addr learned my addr 172.21.15.27:0/2494704577 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b78a00640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74105d80 0x7f4b7419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.079+0000 7f4b6be00640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7410b6c0 0x7f4b7419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.080+0000 7f4b6be00640 1 -- 172.21.15.27:0/2494704577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74105d80 msgr2=0x7f4b7419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.080+0000 7f4b6be00640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74105d80 0x7f4b7419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.080+0000 7f4b6be00640 1 -- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 msgr2=0x7f4b741a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.080+0000 7f4b6be00640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 0x7f4b741a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.080+0000 7f4b6be00640 1 -- 172.21.15.27:0/2494704577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b741a2650 con 0x7f4b7410b6c0 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.080+0000 7f4b78a00640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74105d80 0x7f4b7419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.080+0000 7f4b6be00640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7410b6c0 0x7f4b7419cc80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b7406caa0 tx=0x7f4b5c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.081+0000 7f4b72000640 1 -- 172.21.15.27:0/2494704577 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00cc40 con 0x7f4b7410b6c0 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.081+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b741a2940 con 0x7f4b7410b6c0 2024-09-21T07:24:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.081+0000 7f4b72000640 1 -- 172.21.15.27:0/2494704577 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5c00cde0 con 0x7f4b7410b6c0 2024-09-21T07:24:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.081+0000 7f4b72000640 1 -- 172.21.15.27:0/2494704577 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c018760 con 0x7f4b7410b6c0 2024-09-21T07:24:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.081+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b741a90a0 con 0x7f4b7410b6c0 2024-09-21T07:24:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 Processor -- start 2024-09-21T07:24:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 -- start start 2024-09-21T07:24:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242a9d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426d5390 0x556242a9d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556242a13fb0 con 0x5562426f6070 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556242a033e0 con 0x5562426d5390 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.084+0000 7f4b8208a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556242a071f0 con 0x5562426cd1b0 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242a9d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39898/0 (socket says 172.21.15.27:39898) 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242a9d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46666/0 (socket says 172.21.15.27:46666) 2024-09-21T07:24:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b79400640 1 -- 172.21.15.27:0/1792710599 learned_addr learned my addr 172.21.15.27:0/1792710599 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 -- 172.21.15.27:0/1792710599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426d5390 msgr2=0x556242a9d620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426d5390 0x556242a9d620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 -- 172.21.15.27:0/1792710599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 msgr2=0x556242a9cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 -- 172.21.15.27:0/1792710599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562429cdcd0 con 0x5562426cd1b0 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b79400640 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b72000640 1 -- 172.21.15.27:0/2494704577 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4b5c010a90 con 0x7f4b7410b6c0 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.085+0000 7f4b78a00640 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242a9d0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009fd0 tx=0x7f4b640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b62f020ab7bc14a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.086+0000 7f4b69c00640 1 -- 172.21.15.27:0/1792710599 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64004030 con 0x5562426cd1b0 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.086+0000 7f4b69c00640 1 -- 172.21.15.27:0/1792710599 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64005020 con 0x5562426cd1b0 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.086+0000 7f4b69c00640 1 -- 172.21.15.27:0/1792710599 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64004c10 con 0x5562426cd1b0 2024-09-21T07:24:05.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 -- 172.21.15.27:0/1792710599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 msgr2=0x556242a9d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242a9d0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009fd0 tx=0x7f4b640118d0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 -- 172.21.15.27:0/1792710599 shutdown_connections 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242a9d0e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426d5390 0x556242a9d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1792710599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 -- 172.21.15.27:0/1792710599 >> 172.21.15.27:0/1792710599 conn(0x5562426ea460 msgr2=0x556242a9aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 -- 172.21.15.27:0/1792710599 shutdown_connections 2024-09-21T07:24:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 -- 172.21.15.27:0/1792710599 wait complete. 2024-09-21T07:24:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b8208a740 1 -- start start 2024-09-21T07:24:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.087+0000 7f4b72000640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4b580b6620 0x7f4b580b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 1 Processor -- start 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b72000640 1 -- 172.21.15.27:0/2494704577 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7f4b5c0e7cc0 con 0x7f4b7410b6c0 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b79400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4b580b6620 0x7f4b580b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242aa8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aa8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426f6070 0x556242ab4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556242864700 con 0x5562426d5390 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55624299e580 con 0x5562426cd1b0 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556242aab380 con 0x5562426f6070 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.088+0000 7f4b8208a740 10 client.0 _ll_register_callbacks cb 0x5562429ae470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242aa8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aa8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242aa8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44274/0 (socket says 172.21.15.27:44274) 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 -- 172.21.15.27:0/914325920 learned_addr learned my addr 172.21.15.27:0/914325920 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b79400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4b580b6620 0x7f4b580b8b30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0094e0 tx=0x7f4b6c005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b79400640 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426f6070 0x556242ab4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 -- 172.21.15.27:0/914325920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426f6070 msgr2=0x556242ab4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426f6070 0x556242ab4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 -- 172.21.15.27:0/914325920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 msgr2=0x556242aa8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aa8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.089+0000 7f4b78a00640 1 -- 172.21.15.27:0/914325920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556242761650 con 0x5562426cd1b0 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b78a00640 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242aa8790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009fa0 tx=0x7f4b640038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d11e747f091023 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b62000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b62000640 1 -- 172.21.15.27:0/914325920 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64028020 con 0x5562426cd1b0 2024-09-21T07:24:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562429a0b90 con 0x5562426cd1b0 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b62000640 1 -- 172.21.15.27:0/914325920 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7f4b64016040 con 0x5562426cd1b0 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b62000640 1 -- 172.21.15.27:0/914325920 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b64033050 con 0x5562426cd1b0 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b8208a740 10 client.28724 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.090+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562429bb950 con 0x5562426cd1b0 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b62000640 1 -- 172.21.15.27:0/914325920 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5562429a0b90 con 0x5562426cd1b0 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b62000640 1 -- 172.21.15.27:0/914325920 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b6400fbf0 con 0x5562426cd1b0 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b8208a740 10 client.28724 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b8208a740 10 client.28724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b8208a740 20 client.28724 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b8208a740 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5562426b49f0 0x556242aba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b8208a740 4 client.28724 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556242a15f00 con 0x5562426b49f0 2024-09-21T07:24:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.091+0000 7f4b6be00640 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5562426b49f0 0x556242aba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.092+0000 7f4b6be00640 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5562426b49f0 0x556242aba5f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c871423de6d141 server_cookie=8224065b0dacb857 in_seq=0 out_seq=0 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.092+0000 7f4b62000640 10 client.28724 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b62000640 1 -- 172.21.15.27:0/914325920 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556242a15f00 con 0x5562426b49f0 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b62000640 10 client.28724 handle_command_reply: tid=0 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 client.28724 shutdown 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 msgr2=0x556242aa8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242aa8790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009fa0 tx=0x7f4b640038a0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 shutdown_connections 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5562426b49f0 0x556242aba5f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426f6070 0x556242ab4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aa8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b62000640 0 client.28724 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:05.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 --2- 172.21.15.27:0/914325920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242aa8790 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 >> 172.21.15.27:0/914325920 conn(0x5562426ea460 msgr2=0x556242a9fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:05.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 shutdown_connections 2024-09-21T07:24:05.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 1 -- 172.21.15.27:0/914325920 wait complete. 2024-09-21T07:24:05.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.093+0000 7f4b8208a740 20 client.28724 trim_cache size 0 max 16384 2024-09-21T07:24:05.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.107+0000 7f4b8208a740 1 Processor -- start 2024-09-21T07:24:05.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b8208a740 1 -- start start 2024-09-21T07:24:05.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242a9e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426d5390 0x556242a9a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b8208a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741b5bc0 con 0x5562426f6070 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b8208a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741b3490 con 0x5562426cd1b0 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b8208a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741b1300 con 0x5562426d5390 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426d5390 0x556242a9a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39910/0 (socket says 172.21.15.27:39910) 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b79400640 1 -- 172.21.15.27:0/1705951708 learned_addr learned my addr 172.21.15.27:0/1705951708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.108+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426d5390 0x556242a9a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46676/0 (socket says 172.21.15.27:46676) 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b78a00640 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242a9e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b6be00640 1 -- 172.21.15.27:0/1705951708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 msgr2=0x556242a9e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b6be00640 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242a9e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b6be00640 1 -- 172.21.15.27:0/1705951708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 msgr2=0x556242a9a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b6be00640 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b6be00640 1 -- 172.21.15.27:0/1705951708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556242a03370 con 0x5562426d5390 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b6be00640 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426d5390 0x556242a9a040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0e5010 tx=0x7f4b5c009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bebfa8dd36f8287 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b69200640 1 -- 172.21.15.27:0/1705951708 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0b7130 con 0x5562426d5390 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b69200640 1 -- 172.21.15.27:0/1705951708 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5c0e5290 con 0x5562426d5390 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.109+0000 7f4b69200640 1 -- 172.21.15.27:0/1705951708 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0b3380 con 0x5562426d5390 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 -- 172.21.15.27:0/1705951708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426d5390 msgr2=0x556242a9a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426d5390 0x556242a9a040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0e5010 tx=0x7f4b5c009700 comp rx=0 tx=0).stop 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 -- 172.21.15.27:0/1705951708 shutdown_connections 2024-09-21T07:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426d5390 0x556242a9a040 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426f6070 0x556242a9a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 --2- 172.21.15.27:0/1705951708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426cd1b0 0x556242a9e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 -- 172.21.15.27:0/1705951708 >> 172.21.15.27:0/1705951708 conn(0x5562426ea460 msgr2=0x556242a9dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 -- 172.21.15.27:0/1705951708 shutdown_connections 2024-09-21T07:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.110+0000 7f4b8208a740 1 -- 172.21.15.27:0/1705951708 wait complete. 2024-09-21T07:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.111+0000 7f4b8208a740 1 -- start start 2024-09-21T07:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.111+0000 7f4b8208a740 1 Processor -- start 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242ab0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b8208a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426f6070 0x556242aac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b8208a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741b1950 con 0x5562426d5390 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b8208a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741b2dc0 con 0x5562426f6070 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b8208a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741b2020 con 0x5562426cd1b0 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b8208a740 10 client.0 _ll_register_callbacks cb 0x7f4b64004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242ab0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242ab0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46680/0 (socket says 172.21.15.27:46680) 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 -- 172.21.15.27:0/3041811850 learned_addr learned my addr 172.21.15.27:0/3041811850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b6be00640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b79400640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426f6070 0x556242aac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 -- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426f6070 msgr2=0x556242aac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426f6070 0x556242aac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 -- 172.21.15.27:0/3041811850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 msgr2=0x556242aac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b78a00640 1 -- 172.21.15.27:0/3041811850 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556242aad000 con 0x5562426cd1b0 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.112+0000 7f4b79400640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426f6070 0x556242aac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b78a00640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242ab0e20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4b640234a0 tx=0x7f4b64009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e6eb278d155df9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b62000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b62000640 1 -- 172.21.15.27:0/3041811850 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6403c060 con 0x5562426cd1b0 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b62000640 1 -- 172.21.15.27:0/3041811850 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7f4b64023b30 con 0x5562426cd1b0 2024-09-21T07:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b5c0e5290 con 0x5562426cd1b0 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b62000640 1 -- 172.21.15.27:0/3041811850 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b64043050 con 0x5562426cd1b0 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b8208a740 10 client.28705 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556242b24e80 con 0x5562426cd1b0 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.113+0000 7f4b62000640 1 -- 172.21.15.27:0/3041811850 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5c0e5290 con 0x5562426cd1b0 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b62000640 1 -- 172.21.15.27:0/3041811850 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b64023d30 con 0x5562426cd1b0 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b8208a740 10 client.28705 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b8208a740 10 client.28705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b8208a740 20 client.28705 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b8208a740 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556242689da0 0x556242b27610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b8208a740 4 client.28705 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556242a119b0 con 0x556242689da0 2024-09-21T07:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.114+0000 7f4b6be00640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556242689da0 0x556242b27610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b6be00640 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556242689da0 0x556242b27610 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f48a109d5686549 server_cookie=ee44f936ab7ab3cb in_seq=0 out_seq=0 2024-09-21T07:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b62000640 10 client.28705 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b62000640 1 -- 172.21.15.27:0/3041811850 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556242a119b0 con 0x556242689da0 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b62000640 10 client.28705 handle_command_reply: tid=0 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 client.28705 shutdown 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 msgr2=0x556242ab0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242ab0e20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4b640234a0 tx=0x7f4b64009eb0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 shutdown_connections 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556242689da0 0x556242b27610 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562426f6070 0x556242aac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562426d5390 0x556242aac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 --2- 172.21.15.27:0/3041811850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5562426cd1b0 0x556242ab0e20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b62000640 0 client.28705 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.115+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 >> 172.21.15.27:0/3041811850 conn(0x5562426ea460 msgr2=0x556242aa6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.116+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 shutdown_connections 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.116+0000 7f4b8208a740 1 -- 172.21.15.27:0/3041811850 wait complete. 2024-09-21T07:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.116+0000 7f4b8208a740 20 client.28705 trim_cache size 0 max 16384 2024-09-21T07:24:05.121 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:24:05.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.116+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4b580b6620 msgr2=0x7f4b580b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.116+0000 7f4b7b400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4b580b6620 0x7f4b580b8b30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0094e0 tx=0x7f4b6c005c50 comp rx=0 tx=0).stop 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.116+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7410b6c0 msgr2=0x7f4b7419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.116+0000 7f4b7b400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7410b6c0 0x7f4b7419cc80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b7406caa0 tx=0x7f4b5c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 shutdown_connections 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7410b6c0 0x7f4b7419cc80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4b580b6620 0x7f4b580b8b30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7410c010 0x7f4b741a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 --2- 172.21.15.27:0/2494704577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b74105d80 0x7f4b7419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 >> 172.21.15.27:0/2494704577 conn(0x7f4b74101810 msgr2=0x7f4b74108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 shutdown_connections 2024-09-21T07:24:05.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:05.117+0000 7f4b7b400640 1 -- 172.21.15.27:0/2494704577 wait complete. 2024-09-21T07:24:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:04 smithi119 ceph-mon[34519]: pgmap v719: 129 pgs: 129 active+clean; 3.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 85 KiB/s wr, 31 op/s 2024-09-21T07:24:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:04 smithi119 ceph-mon[34519]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T07:24:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:04 smithi027 ceph-mon[30779]: pgmap v719: 129 pgs: 129 active+clean; 3.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 85 KiB/s wr, 31 op/s 2024-09-21T07:24:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:04 smithi027 ceph-mon[30779]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T07:24:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:04 smithi153 ceph-mon[34747]: pgmap v719: 129 pgs: 129 active+clean; 3.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 85 KiB/s wr, 31 op/s 2024-09-21T07:24:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:04 smithi153 ceph-mon[34747]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T07:24:07.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:24:07.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:24:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:06 smithi119 ceph-mon[34519]: pgmap v721: 129 pgs: 129 active+clean; 3.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-21T07:24:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:06 smithi027 ceph-mon[30779]: pgmap v721: 129 pgs: 129 active+clean; 3.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-21T07:24:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:06 smithi153 ceph-mon[34747]: pgmap v721: 129 pgs: 129 active+clean; 3.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-21T07:24:07.458 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.065+0000 7fd65d400640 1 -- 172.21.15.27:0/464499210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6581039b0 msgr2=0x7fd658103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.065+0000 7fd65d400640 1 --2- 172.21.15.27:0/464499210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6581039b0 0x7fd658103dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd648009900 tx=0x7fd64802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.065+0000 7fd65d400640 1 -- 172.21.15.27:0/464499210 shutdown_connections 2024-09-21T07:24:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.065+0000 7fd65d400640 1 --2- 172.21.15.27:0/464499210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658109c40 0x7fd658110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.065+0000 7fd65d400640 1 --2- 172.21.15.27:0/464499210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581092f0 0x7fd658109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.065+0000 7fd65d400640 1 --2- 172.21.15.27:0/464499210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6581039b0 0x7fd658103dc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.065+0000 7fd65d400640 1 -- 172.21.15.27:0/464499210 >> 172.21.15.27:0/464499210 conn(0x7fd6580ff400 msgr2=0x7fd658101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:08.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.066+0000 7fd65d400640 1 -- 172.21.15.27:0/464499210 shutdown_connections 2024-09-21T07:24:08.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.066+0000 7fd65d400640 1 -- 172.21.15.27:0/464499210 wait complete. 2024-09-21T07:24:08.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 Processor -- start 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 -- start start 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581039b0 0x7fd65819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6581092f0 0x7fd65819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd658109c40 0x7fd6581a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd658113950 con 0x7fd6581092f0 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581137d0 con 0x7fd6581039b0 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.067+0000 7fd65d400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd658113ad0 con 0x7fd658109c40 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd657e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581039b0 0x7fd65819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd657400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6581092f0 0x7fd65819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd657e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581039b0 0x7fd65819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56210/0 (socket says 172.21.15.27:56210) 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd65ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd658109c40 0x7fd6581a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd657e00640 1 -- 172.21.15.27:0/925437692 learned_addr learned my addr 172.21.15.27:0/925437692 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:08.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd65ca00640 1 -- 172.21.15.27:0/925437692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581039b0 msgr2=0x7fd65819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd65ca00640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581039b0 0x7fd65819c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd65ca00640 1 -- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6581092f0 msgr2=0x7fd65819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd65ca00640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6581092f0 0x7fd65819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd65ca00640 1 -- 172.21.15.27:0/925437692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581a2470 con 0x7fd658109c40 2024-09-21T07:24:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd657e00640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581039b0 0x7fd65819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.068+0000 7fd65ca00640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd658109c40 0x7fd6581a1d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd650009fd0 tx=0x7fd65000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.069+0000 7fd654c00640 1 -- 172.21.15.27:0/925437692 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65000ce20 con 0x7fd658109c40 2024-09-21T07:24:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.069+0000 7fd654c00640 1 -- 172.21.15.27:0/925437692 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd650005020 con 0x7fd658109c40 2024-09-21T07:24:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.069+0000 7fd654c00640 1 -- 172.21.15.27:0/925437692 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650010690 con 0x7fd658109c40 2024-09-21T07:24:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.069+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6581a2760 con 0x7fd658109c40 2024-09-21T07:24:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.069+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6581a8ec0 con 0x7fd658109c40 2024-09-21T07:24:08.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.072+0000 7fd665650740 1 Processor -- start 2024-09-21T07:24:08.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd654c00640 1 -- 172.21.15.27:0/925437692 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd6500054d0 con 0x7fd658109c40 2024-09-21T07:24:08.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd665650740 1 -- start start 2024-09-21T07:24:08.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd665650740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e98c350 0x55937ed33f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd665650740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed34440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd665650740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e96b670 0x55937ed34980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd665650740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55937eca9e90 con 0x55937e96b670 2024-09-21T07:24:08.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd665650740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55937ec992c0 con 0x55937e963490 2024-09-21T07:24:08.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.073+0000 7fd665650740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55937ec9d0d0 con 0x55937e98c350 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed34440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed34440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56212/0 (socket says 172.21.15.27:56212) 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 -- 172.21.15.27:0/4036043683 learned_addr learned my addr 172.21.15.27:0/4036043683 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657400640 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e98c350 0x55937ed33f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657400640 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e96b670 0x55937ed34980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 -- 172.21.15.27:0/4036043683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e98c350 msgr2=0x55937ed33f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e98c350 0x55937ed33f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 -- 172.21.15.27:0/4036043683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e96b670 msgr2=0x55937ed34980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e96b670 0x55937ed34980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.074+0000 7fd657e00640 1 -- 172.21.15.27:0/4036043683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937e9f7930 con 0x55937e963490 2024-09-21T07:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.075+0000 7fd657e00640 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed34440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd648006010 tx=0x7fd6480044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43935a3defc6da30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.075+0000 7fd654c00640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd63c0b6700 0x7fd63c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.075+0000 7fd64d200640 1 -- 172.21.15.27:0/4036043683 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648040070 con 0x55937e963490 2024-09-21T07:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.075+0000 7fd654c00640 1 -- 172.21.15.27:0/925437692 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7fd6500e8200 con 0x7fd658109c40 2024-09-21T07:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.075+0000 7fd657400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd63c0b6700 0x7fd63c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.075+0000 7fd64d200640 1 -- 172.21.15.27:0/4036043683 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd648003de0 con 0x55937e963490 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.075+0000 7fd64d200640 1 -- 172.21.15.27:0/4036043683 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648035040 con 0x55937e963490 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 -- 172.21.15.27:0/4036043683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 msgr2=0x55937ed34440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed34440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd648006010 tx=0x7fd6480044b0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd657400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd63c0b6700 0x7fd63c0b8c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd658108c80 tx=0x7fd644013040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 -- 172.21.15.27:0/4036043683 shutdown_connections 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e96b670 0x55937ed34980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed34440 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 --2- 172.21.15.27:0/4036043683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e98c350 0x55937ed33f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 -- 172.21.15.27:0/4036043683 >> 172.21.15.27:0/4036043683 conn(0x55937e980740 msgr2=0x55937ed31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.076+0000 7fd665650740 1 -- 172.21.15.27:0/4036043683 shutdown_connections 2024-09-21T07:24:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.077+0000 7fd665650740 1 -- 172.21.15.27:0/4036043683 wait complete. 2024-09-21T07:24:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.077+0000 7fd665650740 1 -- start start 2024-09-21T07:24:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.077+0000 7fd665650740 1 Processor -- start 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd665650740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed3faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd665650740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed3ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd665650740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd665650740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55937eafa890 con 0x55937e98c350 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd665650740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55937ec34160 con 0x55937e963490 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd665650740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55937ed42680 con 0x55937e96b670 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd665650740 10 client.0 _ll_register_callbacks cb 0x55937ec50a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd657400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed3ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd65ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd657400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed3ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46320/0 (socket says 172.21.15.27:46320) 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd657400640 1 -- 172.21.15.27:0/161573000 learned_addr learned my addr 172.21.15.27:0/161573000 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.078+0000 7fd657e00640 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed3faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd657e00640 1 -- 172.21.15.27:0/161573000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 msgr2=0x55937ed3ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd657e00640 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed3ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd657e00640 1 -- 172.21.15.27:0/161573000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 msgr2=0x55937ed4b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd657e00640 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed4b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd657e00640 1 -- 172.21.15.27:0/161573000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937ed4bf60 con 0x55937e963490 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd657e00640 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed3faa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd648000c00 tx=0x7fd648031a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4416ebb7697b0f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd641600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.079+0000 7fd641600640 1 -- 172.21.15.27:0/161573000 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648035040 con 0x55937e963490 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd648003de0 con 0x55937e963490 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd641600640 1 -- 172.21.15.27:0/161573000 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7fd648003b40 con 0x55937e963490 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd641600640 1 -- 172.21.15.27:0/161573000 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd648040040 con 0x55937e963490 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd665650740 10 client.28742 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55937ed4e890 con 0x55937e963490 2024-09-21T07:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd641600640 1 -- 172.21.15.27:0/161573000 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd648003de0 con 0x55937e963490 2024-09-21T07:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd641600640 1 -- 172.21.15.27:0/161573000 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd64803c440 con 0x55937e963490 2024-09-21T07:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd665650740 10 client.28742 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.080+0000 7fd665650740 10 client.28742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.081+0000 7fd665650740 20 client.28742 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.081+0000 7fd665650740 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e94acd0 0x55937ed51bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.081+0000 7fd665650740 4 client.28742 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.081+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55937ecabde0 con 0x55937e94acd0 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.081+0000 7fd657400640 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e94acd0 0x55937ed51bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.082+0000 7fd657400640 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e94acd0 0x55937ed51bf0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919a889f2a973bdb server_cookie=edd9371fe803155c in_seq=0 out_seq=0 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.082+0000 7fd641600640 10 client.28742 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.082+0000 7fd641600640 1 -- 172.21.15.27:0/161573000 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55937ecabde0 con 0x55937e94acd0 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.082+0000 7fd641600640 10 client.28742 handle_command_reply: tid=0 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.082+0000 7fd665650740 1 client.28742 shutdown 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 msgr2=0x55937ed3faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed3faa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd648000c00 tx=0x7fd648031a10 comp rx=0 tx=0).stop 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 shutdown_connections 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e94acd0 0x55937ed51bf0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed3ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed4b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 --2- 172.21.15.27:0/161573000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e963490 0x55937ed3faa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 >> 172.21.15.27:0/161573000 conn(0x55937e980740 msgr2=0x55937ed37090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd641600640 0 client.28742 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 shutdown_connections 2024-09-21T07:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 1 -- 172.21.15.27:0/161573000 wait complete. 2024-09-21T07:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.083+0000 7fd665650740 20 client.28742 trim_cache size 0 max 16384 2024-09-21T07:24:08.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 Processor -- start 2024-09-21T07:24:08.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 -- start start 2024-09-21T07:24:08.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e963490 0x55937ed35640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e98c350 0x55937ed318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581b5880 con 0x55937e963490 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581b3150 con 0x55937e98c350 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd665650740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581b0fc0 con 0x55937e96b670 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd657e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e963490 0x55937ed35640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.102+0000 7fd657e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e963490 0x55937ed35640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58024/0 (socket says 172.21.15.27:58024) 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd657e00640 1 -- 172.21.15.27:0/2029353780 learned_addr learned my addr 172.21.15.27:0/2029353780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd65ca00640 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e98c350 0x55937ed318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd657400640 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd65ca00640 1 -- 172.21.15.27:0/2029353780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 msgr2=0x55937ed313a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd65ca00640 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed313a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd65ca00640 1 -- 172.21.15.27:0/2029353780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e963490 msgr2=0x55937ed35640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd65ca00640 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e963490 0x55937ed35640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd65ca00640 1 -- 172.21.15.27:0/2029353780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937ed361f0 con 0x55937e98c350 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd657e00640 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e963490 0x55937ed35640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd65ca00640 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e98c350 0x55937ed318e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd6500e54c0 tx=0x7fd650095050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4810d0cf6fed739 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.103+0000 7fd643e00640 1 -- 172.21.15.27:0/2029353780 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500b26d0 con 0x55937e98c350 2024-09-21T07:24:08.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd643e00640 1 -- 172.21.15.27:0/2029353780 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6500b7610 con 0x55937e98c350 2024-09-21T07:24:08.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd643e00640 1 -- 172.21.15.27:0/2029353780 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500a97a0 con 0x55937e98c350 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 -- 172.21.15.27:0/2029353780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e98c350 msgr2=0x55937ed318e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e98c350 0x55937ed318e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd6500e54c0 tx=0x7fd650095050 comp rx=0 tx=0).stop 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 -- 172.21.15.27:0/2029353780 shutdown_connections 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e96b670 0x55937ed313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e98c350 0x55937ed318e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 --2- 172.21.15.27:0/2029353780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e963490 0x55937ed35640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 -- 172.21.15.27:0/2029353780 >> 172.21.15.27:0/2029353780 conn(0x55937e980740 msgr2=0x55937ed34f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.104+0000 7fd665650740 1 -- 172.21.15.27:0/2029353780 shutdown_connections 2024-09-21T07:24:08.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.105+0000 7fd665650740 1 -- 172.21.15.27:0/2029353780 wait complete. 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.105+0000 7fd665650740 1 -- start start 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.105+0000 7fd665650740 1 Processor -- start 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd665650740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e963490 0x55937ed55640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd665650740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e96b670 0x55937ed52b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd665650740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed53280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd665650740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581b1610 con 0x55937e98c350 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd665650740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581b2a80 con 0x55937e96b670 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd665650740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581b1ce0 con 0x55937e963490 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd665650740 10 client.0 _ll_register_callbacks cb 0x7fd6440099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd65ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed53280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd657e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e963490 0x55937ed55640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd65ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed53280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58028/0 (socket says 172.21.15.27:58028) 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd65ca00640 1 -- 172.21.15.27:0/3259434818 learned_addr learned my addr 172.21.15.27:0/3259434818 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:08.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd657400640 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e96b670 0x55937ed52b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd65ca00640 1 -- 172.21.15.27:0/3259434818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e963490 msgr2=0x55937ed55640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.106+0000 7fd65ca00640 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e963490 0x55937ed55640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd65ca00640 1 -- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e96b670 msgr2=0x55937ed52b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:08.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd65ca00640 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e96b670 0x55937ed52b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd65ca00640 1 -- 172.21.15.27:0/3259434818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937edbb370 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd657e00640 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e963490 0x55937ed55640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd65ca00640 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed53280 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd6500e5520 tx=0x7fd650005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edd8e19ebc41b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd641600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd641600640 1 -- 172.21.15.27:0/3259434818 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500ace50 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd641600640 1 -- 172.21.15.27:0/3259434818 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7fd6500acff0 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6500b7610 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd641600640 1 -- 172.21.15.27:0/3259434818 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6500a9540 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd665650740 10 client.18729 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.107+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55937edbc530 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd641600640 1 -- 172.21.15.27:0/3259434818 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6500b7610 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd641600640 1 -- 172.21.15.27:0/3259434818 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6500a9040 con 0x55937e98c350 2024-09-21T07:24:08.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd665650740 10 client.18729 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd665650740 10 client.18729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd665650740 20 client.18729 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd665650740 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e920080 0x55937edbebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd665650740 4 client.18729 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55937eca7890 con 0x55937e920080 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.108+0000 7fd657e00640 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e920080 0x55937edbebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.109+0000 7fd657e00640 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e920080 0x55937edbebe0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc3375693bfe662 server_cookie=fdb3a7db44d7ae52 in_seq=0 out_seq=0 2024-09-21T07:24:08.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.109+0000 7fd641600640 10 client.18729 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd641600640 1 -- 172.21.15.27:0/3259434818 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55937eca7890 con 0x55937e920080 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd641600640 10 client.18729 handle_command_reply: tid=0 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 client.18729 shutdown 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 msgr2=0x55937ed53280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed53280 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd6500e5520 tx=0x7fd650005440 comp rx=0 tx=0).stop 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 shutdown_connections 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55937e920080 0x55937edbebe0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55937e98c350 0x55937ed53280 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55937e96b670 0x55937ed52b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 --2- 172.21.15.27:0/3259434818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55937e963490 0x55937ed55640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 >> 172.21.15.27:0/3259434818 conn(0x55937e980740 msgr2=0x55937ed40cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd641600640 0 client.18729 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.111+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 shutdown_connections 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.112+0000 7fd665650740 1 -- 172.21.15.27:0/3259434818 wait complete. 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.112+0000 7fd665650740 20 client.18729 trim_cache size 0 max 16384 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:24:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:24:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb", 2024-09-21T07:24:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:24:08.116 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:24:08.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.112+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd63c0b6700 msgr2=0x7fd63c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.112+0000 7fd65d400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd63c0b6700 0x7fd63c0b8c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd658108c80 tx=0x7fd644013040 comp rx=0 tx=0).stop 2024-09-21T07:24:08.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.112+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd658109c40 msgr2=0x7fd6581a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.112+0000 7fd65d400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd658109c40 0x7fd6581a1d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd650009fd0 tx=0x7fd65000ef10 comp rx=0 tx=0).stop 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 shutdown_connections 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd658109c40 0x7fd6581a1d40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd63c0b6700 0x7fd63c0b8c10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6581092f0 0x7fd65819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 --2- 172.21.15.27:0/925437692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6581039b0 0x7fd65819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 >> 172.21.15.27:0/925437692 conn(0x7fd6580ff400 msgr2=0x7fd65810e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 shutdown_connections 2024-09-21T07:24:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:08.113+0000 7fd65d400640 1 -- 172.21.15.27:0/925437692 wait complete. 2024-09-21T07:24:08.916 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:24:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:08 smithi119 ceph-mon[34519]: pgmap v722: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-21T07:24:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:08 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:24:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:08 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:24:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:08 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:24:09.309 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:08 smithi027 ceph-mon[30779]: pgmap v722: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-21T07:24:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:08 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:24:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:08 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:24:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:08 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:24:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:08 smithi153 ceph-mon[34747]: pgmap v722: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 35 op/s 2024-09-21T07:24:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:08 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:24:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:08 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:24:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:09.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:08 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:24:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.922+0000 7f3c4b200640 1 -- 172.21.15.27:0/3937238611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 msgr2=0x7f3c4c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.922+0000 7f3c4b200640 1 --2- 172.21.15.27:0/3937238611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c077d50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009900 tx=0x7f3c3c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.923+0000 7f3c4b200640 1 -- 172.21.15.27:0/3937238611 shutdown_connections 2024-09-21T07:24:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.923+0000 7f3c4b200640 1 --2- 172.21.15.27:0/3937238611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c114570 0x7f3c4c116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.923+0000 7f3c4b200640 1 --2- 172.21.15.27:0/3937238611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c4c078290 0x7f3c4c114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.923+0000 7f3c4b200640 1 --2- 172.21.15.27:0/3937238611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c077d50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.923+0000 7f3c4b200640 1 -- 172.21.15.27:0/3937238611 >> 172.21.15.27:0/3937238611 conn(0x7f3c4c101680 msgr2=0x7f3c4c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.923+0000 7f3c4b200640 1 -- 172.21.15.27:0/3937238611 shutdown_connections 2024-09-21T07:24:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.923+0000 7f3c4b200640 1 -- 172.21.15.27:0/3937238611 wait complete. 2024-09-21T07:24:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.924+0000 7f3c4b200640 1 Processor -- start 2024-09-21T07:24:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.924+0000 7f3c4b200640 1 -- start start 2024-09-21T07:24:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c4b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c4b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c4c078290 0x7f3c4c1a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c4b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c114570 0x7f3c4c1a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c4b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c119ef0 con 0x7f3c4c077940 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c4b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c119d70 con 0x7f3c4c078290 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c4b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c11a070 con 0x7f3c4c114570 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c1a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58040/0 (socket says 172.21.15.27:58040) 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 -- 172.21.15.27:0/4105390774 learned_addr learned my addr 172.21.15.27:0/4105390774 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49400640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c4c078290 0x7f3c4c1a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c4a800640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c114570 0x7f3c4c1a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 -- 172.21.15.27:0/4105390774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c114570 msgr2=0x7f3c4c1a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c114570 0x7f3c4c1a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 -- 172.21.15.27:0/4105390774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c4c078290 msgr2=0x7f3c4c1a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c4c078290 0x7f3c4c1a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49e00640 1 -- 172.21.15.27:0/4105390774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c1a6870 con 0x7f3c4c077940 2024-09-21T07:24:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.925+0000 7f3c49400640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c4c078290 0x7f3c4c1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:09.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.926+0000 7f3c49e00640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c1a0900 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005940 tx=0x7f3c3c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:09.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.926+0000 7f3c42a00640 1 -- 172.21.15.27:0/4105390774 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c03f030 con 0x7f3c4c077940 2024-09-21T07:24:09.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.926+0000 7f3c42a00640 1 -- 172.21.15.27:0/4105390774 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c3c0044c0 con 0x7f3c4c077940 2024-09-21T07:24:09.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.926+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c1a6b00 con 0x7f3c4c077940 2024-09-21T07:24:09.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.926+0000 7f3c42a00640 1 -- 172.21.15.27:0/4105390774 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c035070 con 0x7f3c4c077940 2024-09-21T07:24:09.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.926+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1ad2c0 con 0x7f3c4c077940 2024-09-21T07:24:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 Processor -- start 2024-09-21T07:24:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 -- start start 2024-09-21T07:24:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c42a00640 1 -- 172.21.15.27:0/4105390774 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3c3c042030 con 0x7f3c4c077940 2024-09-21T07:24:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d70230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a8390 0x563093d70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563093ce6f70 con 0x5630939c9070 2024-09-21T07:24:09.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563093cd63a0 con 0x5630939a8390 2024-09-21T07:24:09.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.930+0000 7f3c576b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563093cda1b0 con 0x5630939a01b0 2024-09-21T07:24:09.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d70230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c4a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d6fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d70230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46374/0 (socket says 172.21.15.27:46374) 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 -- 172.21.15.27:0/4028963281 learned_addr learned my addr 172.21.15.27:0/4028963281 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49400640 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a8390 0x563093d70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 -- 172.21.15.27:0/4028963281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a8390 msgr2=0x563093d70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a8390 0x563093d70770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 -- 172.21.15.27:0/4028963281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 msgr2=0x563093d6fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d6fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.931+0000 7f3c49e00640 1 -- 172.21.15.27:0/4028963281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563093ca0c90 con 0x5630939a01b0 2024-09-21T07:24:09.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.932+0000 7f3c49e00640 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d70230 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c1050e0 tx=0x7f3c3c10bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9f521a57131443 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.932+0000 7f3c37000640 1 -- 172.21.15.27:0/4028963281 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c110d70 con 0x5630939a01b0 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.932+0000 7f3c37000640 1 -- 172.21.15.27:0/4028963281 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c3c115070 con 0x5630939a01b0 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.932+0000 7f3c37000640 1 -- 172.21.15.27:0/4028963281 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c1186a0 con 0x5630939a01b0 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c42a00640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c300b6680 0x7f3c300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c42a00640 1 -- 172.21.15.27:0/4105390774 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7f3c3c107eb0 con 0x7f3c4c077940 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c4a800640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c300b6680 0x7f3c300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 -- 172.21.15.27:0/4028963281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 msgr2=0x563093d70230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d70230 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c1050e0 tx=0x7f3c3c10bf90 comp rx=0 tx=0).stop 2024-09-21T07:24:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 -- 172.21.15.27:0/4028963281 shutdown_connections 2024-09-21T07:24:09.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d70230 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a8390 0x563093d70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 --2- 172.21.15.27:0/4028963281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 -- 172.21.15.27:0/4028963281 >> 172.21.15.27:0/4028963281 conn(0x5630939bd460 msgr2=0x563093d6db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.933+0000 7f3c576b1740 1 -- 172.21.15.27:0/4028963281 shutdown_connections 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.934+0000 7f3c576b1740 1 -- 172.21.15.27:0/4028963281 wait complete. 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.934+0000 7f3c4a800640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c300b6680 0x7f3c300b8b90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005fd0 tx=0x7f3c44009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.934+0000 7f3c576b1740 1 -- start start 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.934+0000 7f3c576b1740 1 Processor -- start 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d7b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a8390 0x563093d7be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c576b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563093b37700 con 0x5630939c9070 2024-09-21T07:24:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c576b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563093c71540 con 0x5630939a01b0 2024-09-21T07:24:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c576b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563093d7e290 con 0x5630939a8390 2024-09-21T07:24:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c576b1740 10 client.0 _ll_register_callbacks cb 0x563093c81430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d7b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c4a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d7b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56264/0 (socket says 172.21.15.27:56264) 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c49e00640 1 -- 172.21.15.27:0/2162775067 learned_addr learned my addr 172.21.15.27:0/2162775067 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.935+0000 7f3c49400640 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a8390 0x563093d7be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c49e00640 1 -- 172.21.15.27:0/2162775067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a8390 msgr2=0x563093d7be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c49e00640 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a8390 0x563093d7be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c49e00640 1 -- 172.21.15.27:0/2162775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 msgr2=0x563093d874f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c49e00640 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d874f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c49e00640 1 -- 172.21.15.27:0/2162775067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563093a34650 con 0x5630939a01b0 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c4a800640 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c49e00640 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d7b900 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c032af0 tx=0x7f3c3c11abd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84946cc9c694e248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c2b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c2b400640 1 -- 172.21.15.27:0/2162775067 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c049070 con 0x5630939a01b0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563093c73b50 con 0x5630939a01b0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.936+0000 7f3c2b400640 1 -- 172.21.15.27:0/2162775067 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7f3c3c105420 con 0x5630939a01b0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c2b400640 1 -- 172.21.15.27:0/2162775067 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c0c9fa0 con 0x5630939a01b0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 10 client.28760 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563093c8e910 con 0x5630939a01b0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c2b400640 1 -- 172.21.15.27:0/2162775067 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563093c73b50 con 0x5630939a01b0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c2b400640 1 -- 172.21.15.27:0/2162775067 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c3c110b70 con 0x5630939a01b0 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 10 client.28760 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 10 client.28760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 20 client.28760 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5630939879f0 0x563093d8d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 4 client.28760 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.937+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563093ce8ec0 con 0x5630939879f0 2024-09-21T07:24:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.938+0000 7f3c49400640 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5630939879f0 0x563093d8d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.938+0000 7f3c49400640 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5630939879f0 0x563093d8d3a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36eeb4f407ccaca server_cookie=639e4511c91999fc in_seq=0 out_seq=0 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.938+0000 7f3c2b400640 10 client.28760 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.939+0000 7f3c2b400640 1 -- 172.21.15.27:0/2162775067 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563093ce8ec0 con 0x5630939879f0 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.939+0000 7f3c2b400640 10 client.28760 handle_command_reply: tid=0 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.939+0000 7f3c576b1740 1 client.28760 shutdown 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 msgr2=0x563093d7b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d7b900 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c032af0 tx=0x7f3c3c11abd0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 shutdown_connections 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5630939879f0 0x563093d8d3a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a8390 0x563093d7be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939c9070 0x563093d874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2162775067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d7b900 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 >> 172.21.15.27:0/2162775067 conn(0x5630939bd460 msgr2=0x563093d72ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c2b400640 0 client.28760 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 shutdown_connections 2024-09-21T07:24:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 1 -- 172.21.15.27:0/2162775067 wait complete. 2024-09-21T07:24:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.940+0000 7f3c576b1740 20 client.28760 trim_cache size 0 max 16384 2024-09-21T07:24:09.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.958+0000 7f3c576b1740 1 Processor -- start 2024-09-21T07:24:09.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c576b1740 1 -- start start 2024-09-21T07:24:09.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d6d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939c9070 0x563093d6d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c576b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1b9d50 con 0x5630939a8390 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c576b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1b7620 con 0x5630939c9070 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c576b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1b5490 con 0x5630939a01b0 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c49400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d6d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c49400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d6d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58060/0 (socket says 172.21.15.27:58060) 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c49400640 1 -- 172.21.15.27:0/2151434474 learned_addr learned my addr 172.21.15.27:0/2151434474 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c49e00640 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.959+0000 7f3c4a800640 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939c9070 0x563093d6d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c49400640 1 -- 172.21.15.27:0/2151434474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 msgr2=0x563093d768b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c49400640 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d768b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c49400640 1 -- 172.21.15.27:0/2151434474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939c9070 msgr2=0x563093d6d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c49400640 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939c9070 0x563093d6d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c49400640 1 -- 172.21.15.27:0/2151434474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563093d7c9c0 con 0x5630939a8390 2024-09-21T07:24:09.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c49400640 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d6d1d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3c38012620 tx=0x7f3c38005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d01c5784a93b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:09.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c36600640 1 -- 172.21.15.27:0/2151434474 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38005220 con 0x5630939a8390 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c36600640 1 -- 172.21.15.27:0/2151434474 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c38004c50 con 0x5630939a8390 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.960+0000 7f3c36600640 1 -- 172.21.15.27:0/2151434474 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380108c0 con 0x5630939a8390 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 -- 172.21.15.27:0/2151434474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 msgr2=0x563093d6d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d6d1d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3c38012620 tx=0x7f3c38005f90 comp rx=0 tx=0).stop 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 -- 172.21.15.27:0/2151434474 shutdown_connections 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939c9070 0x563093d6d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d6d1d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 --2- 172.21.15.27:0/2151434474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939a01b0 0x563093d768b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 -- 172.21.15.27:0/2151434474 >> 172.21.15.27:0/2151434474 conn(0x5630939bd460 msgr2=0x563093d79f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 -- 172.21.15.27:0/2151434474 shutdown_connections 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.961+0000 7f3c576b1740 1 -- 172.21.15.27:0/2151434474 wait complete. 2024-09-21T07:24:09.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.962+0000 7f3c576b1740 1 -- start start 2024-09-21T07:24:09.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.962+0000 7f3c576b1740 1 Processor -- start 2024-09-21T07:24:09.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.962+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d83a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c576b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939c9070 0x563093d80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c576b1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1b5ae0 con 0x5630939a8390 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c576b1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1b6f50 con 0x5630939a01b0 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c576b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1b61b0 con 0x5630939c9070 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c576b1740 10 client.0 _ll_register_callbacks cb 0x7f3c3c10be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d83a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d83a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56282/0 (socket says 172.21.15.27:56282) 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 -- 172.21.15.27:0/1281217292 learned_addr learned my addr 172.21.15.27:0/1281217292 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c4a800640 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939c9070 0x563093d80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 -- 172.21.15.27:0/1281217292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939c9070 msgr2=0x563093d80450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939c9070 0x563093d80450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 -- 172.21.15.27:0/1281217292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 msgr2=0x563093d7ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:09.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d7ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.963+0000 7f3c49e00640 1 -- 172.21.15.27:0/1281217292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563093d80a10 con 0x5630939a01b0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c49e00640 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d83a50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d2380 tx=0x7f3c3c108130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2308ae839209936 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c2b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c2b400640 1 -- 172.21.15.27:0/1281217292 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c049070 con 0x5630939a01b0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c38004c50 con 0x5630939a01b0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c2b400640 1 -- 172.21.15.27:0/1281217292 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9549+0+0 (secure 0 0 0) 0x7f3c3c105420 con 0x5630939a01b0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c2b400640 1 -- 172.21.15.27:0/1281217292 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c0d2180 con 0x5630939a01b0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c576b1740 10 client.28766 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.964+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563093d80a10 con 0x5630939a01b0 2024-09-21T07:24:09.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c2b400640 1 -- 172.21.15.27:0/1281217292 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c38004c50 con 0x5630939a01b0 2024-09-21T07:24:09.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c2b400640 1 -- 172.21.15.27:0/1281217292 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c3c110b70 con 0x5630939a01b0 2024-09-21T07:24:09.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c576b1740 10 client.28766 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c576b1740 10 client.28766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c576b1740 20 client.28766 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c576b1740 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56309395cda0 0x563093dfada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c576b1740 4 client.28766 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563093ce4970 con 0x56309395cda0 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.965+0000 7f3c49400640 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56309395cda0 0x563093dfada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.966+0000 7f3c49400640 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56309395cda0 0x563093dfada0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496e48cfa9248c66 server_cookie=f423a4884f833585 in_seq=0 out_seq=0 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.966+0000 7f3c2b400640 10 client.28766 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.966+0000 7f3c2b400640 1 -- 172.21.15.27:0/1281217292 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x563093ce4970 con 0x56309395cda0 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.966+0000 7f3c2b400640 10 client.28766 handle_command_reply: tid=0 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.966+0000 7f3c576b1740 1 client.28766 shutdown 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 msgr2=0x563093d83a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d83a50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d2380 tx=0x7f3c3c108130 comp rx=0 tx=0).stop 2024-09-21T07:24:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 shutdown_connections 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56309395cda0 0x563093dfada0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5630939c9070 0x563093d80450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630939a8390 0x563093d7ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 --2- 172.21.15.27:0/1281217292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5630939a01b0 0x563093d83a50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 >> 172.21.15.27:0/1281217292 conn(0x5630939bd460 msgr2=0x563093d74e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c2b400640 0 client.28766 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 shutdown_connections 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 1 -- 172.21.15.27:0/1281217292 wait complete. 2024-09-21T07:24:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.967+0000 7f3c576b1740 20 client.28766 trim_cache size 0 max 16384 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.972 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.977 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.978 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.979 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:09.980 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c300b6680 msgr2=0x7f3c300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c300b6680 0x7f3c300b8b90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005fd0 tx=0x7f3c44009500 comp rx=0 tx=0).stop 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 msgr2=0x7f3c4c1a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c1a0900 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005940 tx=0x7f3c3c0040c0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 shutdown_connections 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c114570 0x7f3c4c1a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c300b6680 0x7f3c300b8b90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c4c078290 0x7f3c4c1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 --2- 172.21.15.27:0/4105390774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c077940 0x7f3c4c1a0900 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 >> 172.21.15.27:0/4105390774 conn(0x7f3c4c101680 msgr2=0x7f3c4c1165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 shutdown_connections 2024-09-21T07:24:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:09.968+0000 7f3c4b200640 1 -- 172.21.15.27:0/4105390774 wait complete. 2024-09-21T07:24:11.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:24:11.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:24:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:10 smithi119 ceph-mon[34519]: pgmap v723: 129 pgs: 129 active+clean; 3.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:24:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:10 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:24:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:10 smithi027 ceph-mon[30779]: pgmap v723: 129 pgs: 129 active+clean; 3.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:24:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:10 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:24:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:10 smithi153 ceph-mon[34747]: pgmap v723: 129 pgs: 129 active+clean; 3.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-09-21T07:24:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:10 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:24:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 -- 172.21.15.27:0/795962897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410c020 msgr2=0x7f91b4112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 --2- 172.21.15.27:0/795962897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410c020 0x7f91b4112850 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009ca0 tx=0x7f91ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 -- 172.21.15.27:0/795962897 shutdown_connections 2024-09-21T07:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 --2- 172.21.15.27:0/795962897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410c020 0x7f91b4112850 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 --2- 172.21.15.27:0/795962897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91b410b6d0 0x7f91b410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 --2- 172.21.15.27:0/795962897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b4105d90 0x7f91b41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 -- 172.21.15.27:0/795962897 >> 172.21.15.27:0/795962897 conn(0x7f91b4101860 msgr2=0x7f91b4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:12.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 -- 172.21.15.27:0/795962897 shutdown_connections 2024-09-21T07:24:12.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.122+0000 7f91ba800640 1 -- 172.21.15.27:0/795962897 wait complete. 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.123+0000 7f91ba800640 1 Processor -- start 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.123+0000 7f91ba800640 1 -- start start 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.123+0000 7f91ba800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91b4105d90 0x7f91b419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.123+0000 7f91ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410b6d0 0x7f91b419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91ba800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b410c020 0x7f91b41a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91ba800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4115d30 con 0x7f91b410b6d0 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91ba800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4115bb0 con 0x7f91b4105d90 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91ba800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4115eb0 con 0x7f91b410c020 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b410c020 0x7f91b41a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410b6d0 0x7f91b419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b410c020 0x7f91b41a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46414/0 (socket says 172.21.15.27:46414) 2024-09-21T07:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 -- 172.21.15.27:0/3194106466 learned_addr learned my addr 172.21.15.27:0/3194106466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3400640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91b4105d90 0x7f91b419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 -- 172.21.15.27:0/3194106466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91b4105d90 msgr2=0x7f91b419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91b4105d90 0x7f91b419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 -- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410b6d0 msgr2=0x7f91b419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410b6d0 0x7f91b419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.124+0000 7f91b3e00640 1 -- 172.21.15.27:0/3194106466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b41a2610 con 0x7f91b410c020 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.125+0000 7f91b3e00640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b410c020 0x7f91b41a1ee0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f91ac02f540 tx=0x7f91ac0316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.125+0000 7f91abe00640 1 -- 172.21.15.27:0/3194106466 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac003b70 con 0x7f91b410c020 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.125+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b41a28a0 con 0x7f91b410c020 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.125+0000 7f91abe00640 1 -- 172.21.15.27:0/3194106466 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91ac003d10 con 0x7f91b410c020 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.125+0000 7f91abe00640 1 -- 172.21.15.27:0/3194106466 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac02f990 con 0x7f91b410c020 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.125+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b41a9060 con 0x7f91b410c020 2024-09-21T07:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.126+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b41061a0 con 0x7f91b410c020 2024-09-21T07:24:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.129+0000 7f91abe00640 1 -- 172.21.15.27:0/3194106466 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f91ac004b40 con 0x7f91b410c020 2024-09-21T07:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.130+0000 7f91abe00640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91980b6680 0x7f91980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.131+0000 7f91abe00640 1 -- 172.21.15.27:0/3194106466 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f91ac108e70 con 0x7f91b410c020 2024-09-21T07:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.131+0000 7f91b3400640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91980b6680 0x7f91980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.131+0000 7f91b3400640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91980b6680 0x7f91980b8b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009bd0 tx=0x7f91a4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:12.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.132+0000 7f91abe00640 1 -- 172.21.15.27:0/3194106466 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91ac0cefb0 con 0x7f91b410c020 2024-09-21T07:24:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:24:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:24:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:11 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:24:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:11 smithi119 ceph-mon[34519]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T07:24:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:24:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:24:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:11 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:24:12.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:11 smithi027 ceph-mon[30779]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T07:24:12.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.371+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91b419d7d0 con 0x7f91b410c020 2024-09-21T07:24:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.373+0000 7f91abe00640 1 -- 172.21.15.27:0/3194106466 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f91ac0d3390 con 0x7f91b410c020 2024-09-21T07:24:12.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:24:12.376 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:24:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91980b6680 msgr2=0x7f91980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91980b6680 0x7f91980b8b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009bd0 tx=0x7f91a4005f90 comp rx=0 tx=0).stop 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b410c020 msgr2=0x7f91b41a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b410c020 0x7f91b41a1ee0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f91ac02f540 tx=0x7f91ac0316f0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 shutdown_connections 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91b410c020 0x7f91b41a1ee0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91980b6680 0x7f91980b8b90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91b410b6d0 0x7f91b419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 --2- 172.21.15.27:0/3194106466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91b4105d90 0x7f91b419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 >> 172.21.15.27:0/3194106466 conn(0x7f91b4101860 msgr2=0x7f91b4108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 shutdown_connections 2024-09-21T07:24:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:12.376+0000 7f91ba800640 1 -- 172.21.15.27:0/3194106466 wait complete. 2024-09-21T07:24:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:24:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:24:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:11 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:24:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:11 smithi153 ceph-mon[34747]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T07:24:13.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:24:13.099 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:313/600: open fd = 3 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:314/600: open fd = 3 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:315/600: open fd = 3 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.100 INFO:tasks.workunit.client.0.smithi027.stdout:316/600: open fd = 3 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:317/600: open fd = 3 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:318/600: open fd = 3 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:319/600: open fd = 3 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.101 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:320/600: open fd = 3 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:321/600: open fd = 3 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:322/600: open fd = 3 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:323/600: open fd = 3 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.102 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:324/600: open fd = 3 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:325/600: open fd = 3 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:326/600: open fd = 3 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:327/600: open fd = 3 2024-09-21T07:24:13.103 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:328/600: open fd = 3 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:329/600: open fd = 3 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:330/600: open fd = 3 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.104 INFO:tasks.workunit.client.0.smithi027.stdout:331/600: open fd = 3 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:332/600: open fd = 3 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:333/600: open fd = 3 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:334/600: open fd = 3 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.105 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:335/600: open fd = 3 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:336/600: open fd = 3 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:337/600: open fd = 3 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:338/600: open fd = 3 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.106 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:339/600: open fd = 3 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:340/600: open fd = 3 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:341/600: open fd = 3 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:342/600: open fd = 3 2024-09-21T07:24:13.107 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:343/600: open fd = 3 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:344/600: open fd = 3 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:345/600: open fd = 3 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.108 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:346/600: open fd = 3 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:347/600: open fd = 3 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:348/600: open fd = 3 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.109 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:349/600: open fd = 3 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:350/600: open fd = 3 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:351/600: open fd = 3 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:352/600: open fd = 3 2024-09-21T07:24:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:353/600: open fd = 3 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:354/600: open fd = 3 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:355/600: open fd = 3 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.111 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:356/600: open fd = 3 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:357/600: open fd = 3 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:358/600: open fd = 3 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:359/600: open fd = 3 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:360/600: open fd = 3 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:361/600: open fd = 3 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:362/600: open fd = 3 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.113 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:363/600: open fd = 3 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:364/600: open fd = 3 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:365/600: open fd = 3 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.114 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:366/600: open fd = 3 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:367/600: open fd = 3 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:368/600: open fd = 3 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.115 INFO:tasks.workunit.client.0.smithi027.stdout:369/600: open fd = 3 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:370/600: open fd = 3 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:371/600: open fd = 3 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:372/600: open fd = 3 2024-09-21T07:24:13.116 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:373/600: open fd = 3 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:374/600: open fd = 3 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:24:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:24:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:12 smithi119 ceph-mon[34519]: pgmap v724: 129 pgs: 129 active+clean; 3.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-21T07:24:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:12 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3194106466' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:12 smithi027 ceph-mon[30779]: pgmap v724: 129 pgs: 129 active+clean; 3.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-21T07:24:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:12 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3194106466' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:12 smithi153 ceph-mon[34747]: pgmap v724: 129 pgs: 129 active+clean; 3.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-21T07:24:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:12 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3194106466' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:13.443 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:14.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 -- 172.21.15.27:0/1680377211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 msgr2=0x7fb29410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 --2- 172.21.15.27:0/1680377211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 0x7fb29410b8e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb284009900 tx=0x7fb28402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 -- 172.21.15.27:0/1680377211 shutdown_connections 2024-09-21T07:24:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 --2- 172.21.15.27:0/1680377211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb29410be20 0x7fb294112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 --2- 172.21.15.27:0/1680377211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 0x7fb29410b8e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 --2- 172.21.15.27:0/1680377211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 0x7fb294105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 -- 172.21.15.27:0/1680377211 >> 172.21.15.27:0/1680377211 conn(0x7fb2941015e0 msgr2=0x7fb294103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 -- 172.21.15.27:0/1680377211 shutdown_connections 2024-09-21T07:24:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.054+0000 7fb29b200640 1 -- 172.21.15.27:0/1680377211 wait complete. 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.055+0000 7fb29b200640 1 Processor -- start 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.055+0000 7fb29b200640 1 -- start start 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb29b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 0x7fb29419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb29b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 0x7fb29419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb29b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb29410be20 0x7fb2941a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb29b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb294115ae0 con 0x7fb29410b4d0 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb29b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb294115960 con 0x7fb29410be20 2024-09-21T07:24:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb29b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb294115c60 con 0x7fb294105b90 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb299e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 0x7fb29419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb29a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb29410be20 0x7fb2941a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb299e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 0x7fb29419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46434/0 (socket says 172.21.15.27:46434) 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb299e00640 1 -- 172.21.15.27:0/2991905151 learned_addr learned my addr 172.21.15.27:0/2991905151 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.056+0000 7fb299400640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 0x7fb29419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb29a800640 1 -- 172.21.15.27:0/2991905151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 msgr2=0x7fb29419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb29a800640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 0x7fb29419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb29a800640 1 -- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 msgr2=0x7fb29419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb29a800640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 0x7fb29419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb29a800640 1 -- 172.21.15.27:0/2991905151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2941a24a0 con 0x7fb29410be20 2024-09-21T07:24:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb299e00640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 0x7fb29419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb29a800640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb29410be20 0x7fb2941a1d70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb29000ec50 tx=0x7fb29000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb28ea00640 1 -- 172.21.15.27:0/2991905151 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290003b70 con 0x7fb29410be20 2024-09-21T07:24:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb28ea00640 1 -- 172.21.15.27:0/2991905151 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb290003d10 con 0x7fb29410be20 2024-09-21T07:24:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2941a2790 con 0x7fb29410be20 2024-09-21T07:24:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.057+0000 7fb28ea00640 1 -- 172.21.15.27:0/2991905151 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290010830 con 0x7fb29410be20 2024-09-21T07:24:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.058+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2941a8ef0 con 0x7fb29410be20 2024-09-21T07:24:14.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.061+0000 7fb2a34ca740 1 Processor -- start 2024-09-21T07:24:14.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.061+0000 7fb28ea00640 1 -- 172.21.15.27:0/2991905151 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb290003ec0 con 0x7fb29410be20 2024-09-21T07:24:14.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.061+0000 7fb2a34ca740 1 -- start start 2024-09-21T07:24:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x556766355d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766356270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x5567663567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5567662ccfb0 con 0x556765f8e390 2024-09-21T07:24:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5567662bc3e0 con 0x556765faf070 2024-09-21T07:24:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5567662c01f0 con 0x556765f861b0 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb299400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x556766355d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb299400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x556766355d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56318/0 (socket says 172.21.15.27:56318) 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb299400640 1 -- 172.21.15.27:0/1618017569 learned_addr learned my addr 172.21.15.27:0/1618017569 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb299e00640 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766356270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.062+0000 7fb299400640 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x5567663567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb299e00640 1 -- 172.21.15.27:0/1618017569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 msgr2=0x556766355d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb299e00640 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x556766355d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb299e00640 1 -- 172.21.15.27:0/1618017569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 msgr2=0x5567663567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb299e00640 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x5567663567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb299e00640 1 -- 172.21.15.27:0/1618017569 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556766286cd0 con 0x556765f861b0 2024-09-21T07:24:14.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb299e00640 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766356270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb288011510 tx=0x7fb2880119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ecaaaadbbf4002 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb28ea00640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb27c0b6700 0x7fb27c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb283000640 1 -- 172.21.15.27:0/1618017569 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288005020 con 0x556765f861b0 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb283000640 1 -- 172.21.15.27:0/1618017569 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2880047a0 con 0x556765f861b0 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.063+0000 7fb283000640 1 -- 172.21.15.27:0/1618017569 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800f8e0 con 0x556765f861b0 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.064+0000 7fb28ea00640 1 -- 172.21.15.27:0/2991905151 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fb2900e7ff0 con 0x7fb29410be20 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.064+0000 7fb299400640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb27c0b6700 0x7fb27c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.064+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1618017569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 msgr2=0x556766356270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.064+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766356270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb288011510 tx=0x7fb2880119d0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.064+0000 7fb299400640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb27c0b6700 0x7fb27c0b8c10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb2840096c0 tx=0x7fb284031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1618017569 shutdown_connections 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766356270 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x5567663567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1618017569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x556766355d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1618017569 >> 172.21.15.27:0/1618017569 conn(0x556765fa3460 msgr2=0x556766353bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1618017569 shutdown_connections 2024-09-21T07:24:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1618017569 wait complete. 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.065+0000 7fb2a34ca740 1 -- start start 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 1 Processor -- start 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f861b0 0x556766361870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f8e390 0x556766361db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55676611d700 con 0x556765f861b0 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556766257580 con 0x556765faf070 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556766364510 con 0x556765f8e390 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb2a34ca740 10 client.0 _ll_register_callbacks cb 0x556766267470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb299e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f861b0 0x556766361870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb299e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f861b0 0x556766361870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58120/0 (socket says 172.21.15.27:58120) 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb299400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f8e390 0x556766361db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.066+0000 7fb299e00640 1 -- 172.21.15.27:0/2694983362 learned_addr learned my addr 172.21.15.27:0/2694983362 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb29a800640 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb299e00640 1 -- 172.21.15.27:0/2694983362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f8e390 msgr2=0x556766361db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb299e00640 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f8e390 0x556766361db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb299e00640 1 -- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 msgr2=0x55676636d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb299e00640 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb299e00640 1 -- 172.21.15.27:0/2694983362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55676601a650 con 0x556765f861b0 2024-09-21T07:24:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb29a800640 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:14.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb299e00640 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f861b0 0x556766361870 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb28801b040 tx=0x7fb288005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a99eac5f3a3995c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.067+0000 7fb277400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:14.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.068+0000 7fb277400640 1 -- 172.21.15.27:0/2694983362 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288027070 con 0x556765f861b0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.068+0000 7fb277400640 1 -- 172.21.15.27:0/2694983362 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fb288005150 con 0x556765f861b0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.068+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556766259b90 con 0x556765f861b0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.068+0000 7fb277400640 1 -- 172.21.15.27:0/2694983362 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb288033050 con 0x556765f861b0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.068+0000 7fb2a34ca740 10 client.18765 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.068+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556766274950 con 0x556765f861b0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.068+0000 7fb277400640 1 -- 172.21.15.27:0/2694983362 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556766259b90 con 0x556765f861b0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb277400640 1 -- 172.21.15.27:0/2694983362 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb28800f900 con 0x556765f861b0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb2a34ca740 10 client.18765 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb2a34ca740 10 client.18765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb2a34ca740 20 client.18765 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f6d9f0 0x5567663736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb2a34ca740 4 client.18765 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567662cef00 con 0x556765f6d9f0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.069+0000 7fb299400640 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f6d9f0 0x5567663736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.070+0000 7fb299400640 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f6d9f0 0x5567663736c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=536f734a26523a server_cookie=174a55d8456af3a0 in_seq=0 out_seq=0 2024-09-21T07:24:14.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.070+0000 7fb277400640 10 client.18765 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:14.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb277400640 1 -- 172.21.15.27:0/2694983362 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567662cef00 con 0x556765f6d9f0 2024-09-21T07:24:14.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb277400640 10 client.18765 handle_command_reply: tid=0 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 client.18765 shutdown 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f861b0 msgr2=0x556766361870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f861b0 0x556766361870 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb28801b040 tx=0x7fb288005c50 comp rx=0 tx=0).stop 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 shutdown_connections 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f6d9f0 0x5567663736c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f8e390 0x556766361db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/2694983362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f861b0 0x556766361870 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb277400640 0 client.18765 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 >> 172.21.15.27:0/2694983362 conn(0x556765fa3460 msgr2=0x556766358ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 shutdown_connections 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.071+0000 7fb2a34ca740 1 -- 172.21.15.27:0/2694983362 wait complete. 2024-09-21T07:24:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.072+0000 7fb2a34ca740 20 client.18765 trim_cache size 0 max 16384 2024-09-21T07:24:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.085+0000 7fb2a34ca740 1 Processor -- start 2024-09-21T07:24:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb2a34ca740 1 -- start start 2024-09-21T07:24:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766357470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x55676636d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b5860 con 0x556765f8e390 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b3130 con 0x556765faf070 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b0fa0 con 0x556765f861b0 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb299400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x55676636d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb29a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb29a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56336/0 (socket says 172.21.15.27:56336) 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb29a800640 1 -- 172.21.15.27:0/3241457540 learned_addr learned my addr 172.21.15.27:0/3241457540 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.086+0000 7fb299e00640 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766357470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb29a800640 1 -- 172.21.15.27:0/3241457540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 msgr2=0x556766357470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb29a800640 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766357470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb29a800640 1 -- 172.21.15.27:0/3241457540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 msgr2=0x55676636d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb29a800640 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x55676636d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb29a800640 1 -- 172.21.15.27:0/3241457540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556766362930 con 0x556765faf070 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb299400640 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x55676636d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb29a800640 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636dc70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2900ad750 tx=0x7fb2900ad3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b503f24958226c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.087+0000 7fb282600640 1 -- 172.21.15.27:0/3241457540 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290009820 con 0x556765faf070 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb282600640 1 -- 172.21.15.27:0/3241457540 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2900b2070 con 0x556765faf070 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb282600640 1 -- 172.21.15.27:0/3241457540 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900aced0 con 0x556765faf070 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 -- 172.21.15.27:0/3241457540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 msgr2=0x55676636dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636dc70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2900ad750 tx=0x7fb2900ad3a0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 -- 172.21.15.27:0/3241457540 shutdown_connections 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x55676636dc70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x55676636d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/3241457540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x556766357470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 -- 172.21.15.27:0/3241457540 >> 172.21.15.27:0/3241457540 conn(0x556765fa3460 msgr2=0x556766356d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.088+0000 7fb2a34ca740 1 -- 172.21.15.27:0/3241457540 shutdown_connections 2024-09-21T07:24:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.089+0000 7fb2a34ca740 1 -- 172.21.15.27:0/3241457540 wait complete. 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.089+0000 7fb2a34ca740 1 -- start start 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.089+0000 7fb2a34ca740 1 Processor -- start 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x55676636a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x556766355b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb2a34ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x5567663561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b15f0 con 0x556765f8e390 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b2a60 con 0x556765faf070 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb2a34ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b1cc0 con 0x556765f861b0 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb2a34ca740 10 client.0 _ll_register_callbacks cb 0x7fb288009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb299e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x55676636a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb29a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x5567663561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb299e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x55676636a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46472/0 (socket says 172.21.15.27:46472) 2024-09-21T07:24:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb299e00640 1 -- 172.21.15.27:0/1241337005 learned_addr learned my addr 172.21.15.27:0/1241337005 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb299400640 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x556766355b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb29a800640 1 -- 172.21.15.27:0/1241337005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 msgr2=0x55676636a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb29a800640 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x55676636a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb29a800640 1 -- 172.21.15.27:0/1241337005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 msgr2=0x556766355b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb29a800640 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x556766355b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.090+0000 7fb29a800640 1 -- 172.21.15.27:0/1241337005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556766368ce0 con 0x556765faf070 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.091+0000 7fb29a800640 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x5567663561e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x556766361590 tx=0x7fb2900ad2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56deda1f260a8b50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.091+0000 7fb277400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.091+0000 7fb277400640 1 -- 172.21.15.27:0/1241337005 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900c8060 con 0x556765faf070 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.091+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2900b2070 con 0x556765faf070 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.091+0000 7fb277400640 1 -- 172.21.15.27:0/1241337005 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fb290004d40 con 0x556765faf070 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.091+0000 7fb277400640 1 -- 172.21.15.27:0/1241337005 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb290009820 con 0x556765faf070 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.091+0000 7fb2a34ca740 10 client.28790 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567663de0d0 con 0x556765faf070 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb277400640 1 -- 172.21.15.27:0/1241337005 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2900b2070 con 0x556765faf070 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb277400640 1 -- 172.21.15.27:0/1241337005 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb2900a93a0 con 0x556765faf070 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb2a34ca740 10 client.28790 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb2a34ca740 10 client.28790 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb2a34ca740 20 client.28790 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f42da0 0x5567663e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb2a34ca740 4 client.28790 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.092+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567662ca9b0 con 0x556765f42da0 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.093+0000 7fb299e00640 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f42da0 0x5567663e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.093+0000 7fb299e00640 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f42da0 0x5567663e07c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efad143e59679d8d server_cookie=bf68bd086d04cf7c in_seq=0 out_seq=0 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.093+0000 7fb277400640 10 client.28790 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.093+0000 7fb277400640 1 -- 172.21.15.27:0/1241337005 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567662ca9b0 con 0x556765f42da0 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.093+0000 7fb277400640 10 client.28790 handle_command_reply: tid=0 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.093+0000 7fb2a34ca740 1 client.28790 shutdown 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 msgr2=0x5567663561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x5567663561e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x556766361590 tx=0x7fb2900ad2c0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 shutdown_connections 2024-09-21T07:24:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556765f42da0 0x5567663e07c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556765faf070 0x5567663561e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb277400640 0 client.28790 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556765f8e390 0x556766355b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 --2- 172.21.15.27:0/1241337005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556765f861b0 0x55676636a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 >> 172.21.15.27:0/1241337005 conn(0x556765fa3460 msgr2=0x55676635fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 shutdown_connections 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 1 -- 172.21.15.27:0/1241337005 wait complete. 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.094+0000 7fb2a34ca740 20 client.28790 trim_cache size 0 max 16384 2024-09-21T07:24:14.098 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb27c0b6700 msgr2=0x7fb27c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb27c0b6700 0x7fb27c0b8c10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb2840096c0 tx=0x7fb284031400 comp rx=0 tx=0).stop 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb29410be20 msgr2=0x7fb2941a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb29410be20 0x7fb2941a1d70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb29000ec50 tx=0x7fb29000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 shutdown_connections 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb27c0b6700 0x7fb27c0b8c10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb29410be20 0x7fb2941a1d70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29410b4d0 0x7fb29419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 --2- 172.21.15.27:0/2991905151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb294105b90 0x7fb29419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 >> 172.21.15.27:0/2991905151 conn(0x7fb2941015e0 msgr2=0x7fb294103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 shutdown_connections 2024-09-21T07:24:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:14.095+0000 7fb29b200640 1 -- 172.21.15.27:0/2991905151 wait complete. 2024-09-21T07:24:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 -- 172.21.15.27:0/2762173211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 msgr2=0x7f62c81166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 --2- 172.21.15.27:0/2762173211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 0x7f62c81166a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009be0 tx=0x7f62c002a6f0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 -- 172.21.15.27:0/2762173211 shutdown_connections 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 --2- 172.21.15.27:0/2762173211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c8075dd0 0x7f62c80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 --2- 172.21.15.27:0/2762173211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 0x7f62c81166a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 --2- 172.21.15.27:0/2762173211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c8077390 0x7f62c80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 -- 172.21.15.27:0/2762173211 >> 172.21.15.27:0/2762173211 conn(0x7f62c80ff330 msgr2=0x7f62c8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 -- 172.21.15.27:0/2762173211 shutdown_connections 2024-09-21T07:24:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.599+0000 7f62cd400640 1 -- 172.21.15.27:0/2762173211 wait complete. 2024-09-21T07:24:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.600+0000 7f62cd400640 1 Processor -- start 2024-09-21T07:24:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cd400640 1 -- start start 2024-09-21T07:24:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c8075dd0 0x7f62c81a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c8077390 0x7f62c81a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 0x7f62c81a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cd400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8119e70 con 0x7f62c81141d0 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cd400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8119cf0 con 0x7f62c8077390 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cd400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8119ff0 con 0x7f62c8075dd0 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c8075dd0 0x7f62c81a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.601+0000 7f62cca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 0x7f62c81a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c8075dd0 0x7f62c81a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46492/0 (socket says 172.21.15.27:46492) 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62c7e00640 1 -- 172.21.15.27:0/760413205 learned_addr learned my addr 172.21.15.27:0/760413205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62bfe00640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c8077390 0x7f62c81a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62cca00640 1 -- 172.21.15.27:0/760413205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c8075dd0 msgr2=0x7f62c81a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62cca00640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c8075dd0 0x7f62c81a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62cca00640 1 -- 172.21.15.27:0/760413205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c8077390 msgr2=0x7f62c81a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62cca00640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c8077390 0x7f62c81a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62cca00640 1 -- 172.21.15.27:0/760413205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c81a6820 con 0x7f62c81141d0 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.602+0000 7f62cca00640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 0x7f62c81a6120 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f62c002ac10 tx=0x7f62c0003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.603+0000 7f62c5600640 1 -- 172.21.15.27:0/760413205 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0041020 con 0x7f62c81141d0 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.603+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c81a6ab0 con 0x7f62c81141d0 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.603+0000 7f62c5600640 1 -- 172.21.15.27:0/760413205 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c0030970 con 0x7f62c81141d0 2024-09-21T07:24:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.603+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c81ad2a0 con 0x7f62c81141d0 2024-09-21T07:24:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.603+0000 7f62c5600640 1 -- 172.21.15.27:0/760413205 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0036040 con 0x7f62c81141d0 2024-09-21T07:24:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.606+0000 7f62d5a5e740 1 Processor -- start 2024-09-21T07:24:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62c5600640 1 -- 172.21.15.27:0/760413205 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f62c0044020 con 0x7f62c81141d0 2024-09-21T07:24:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62d5a5e740 1 -- start start 2024-09-21T07:24:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be130ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be130d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559be12826b0 con 0x559be0f681b0 2024-09-21T07:24:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559be1271ae0 con 0x559be0f474d0 2024-09-21T07:24:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.607+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559be12758f0 con 0x559be0f3f2f0 2024-09-21T07:24:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be130d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be130ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be130d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46510/0 (socket says 172.21.15.27:46510) 2024-09-21T07:24:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62c7e00640 1 -- 172.21.15.27:0/1270095998 learned_addr learned my addr 172.21.15.27:0/1270095998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62bfe00640 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62bfe00640 1 -- 172.21.15.27:0/1270095998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 msgr2=0x559be130d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62bfe00640 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be130d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62bfe00640 1 -- 172.21.15.27:0/1270095998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 msgr2=0x559be130d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62bfe00640 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.608+0000 7f62bfe00640 1 -- 172.21.15.27:0/1270095998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be123e780 con 0x559be0f681b0 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.609+0000 7f62c5600640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62ac0b6680 0x7f62ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.609+0000 7f62bfe00640 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be130ce10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f62c8116160 tx=0x7f62b0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ab91057cd0d090 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.609+0000 7f62c5600640 1 -- 172.21.15.27:0/760413205 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f62c0108720 con 0x7f62c81141d0 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.609+0000 7f62c7e00640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62ac0b6680 0x7f62ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.609+0000 7f62bd200640 1 -- 172.21.15.27:0/1270095998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0013ce0 con 0x559be0f681b0 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.609+0000 7f62bd200640 1 -- 172.21.15.27:0/1270095998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b00054d0 con 0x559be0f681b0 2024-09-21T07:24:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.609+0000 7f62bd200640 1 -- 172.21.15.27:0/1270095998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0004ba0 con 0x559be0f681b0 2024-09-21T07:24:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 -- 172.21.15.27:0/1270095998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 msgr2=0x559be130ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62c7e00640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62ac0b6680 0x7f62ac0b8b90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559be130ba90 tx=0x7f62b8008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be130ce10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f62c8116160 tx=0x7f62b0009620 comp rx=0 tx=0).stop 2024-09-21T07:24:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 -- 172.21.15.27:0/1270095998 shutdown_connections 2024-09-21T07:24:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be130d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/1270095998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be130ce10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 -- 172.21.15.27:0/1270095998 >> 172.21.15.27:0/1270095998 conn(0x559be0f5c300 msgr2=0x559be130ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.610+0000 7f62d5a5e740 1 -- 172.21.15.27:0/1270095998 shutdown_connections 2024-09-21T07:24:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.611+0000 7f62d5a5e740 1 -- 172.21.15.27:0/1270095998 wait complete. 2024-09-21T07:24:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.611+0000 7f62d5a5e740 1 -- start start 2024-09-21T07:24:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.611+0000 7f62d5a5e740 1 Processor -- start 2024-09-21T07:24:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be13183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f474d0 0x559be13188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f681b0 0x559be1318e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559be11a9ab0 con 0x559be0f474d0 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559be1218570 con 0x559be0f681b0 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559be131b530 con 0x559be0f3f2f0 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62d5a5e740 10 client.0 _ll_register_callbacks cb 0x559be1227100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f474d0 0x559be13188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be13183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f474d0 0x559be13188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58174/0 (socket says 172.21.15.27:58174) 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62bfe00640 1 -- 172.21.15.27:0/544989153 learned_addr learned my addr 172.21.15.27:0/544989153 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62cca00640 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f681b0 0x559be1318e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62bfe00640 1 -- 172.21.15.27:0/544989153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 msgr2=0x559be13183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62bfe00640 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be13183a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62bfe00640 1 -- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f681b0 msgr2=0x559be1318e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.612+0000 7f62bfe00640 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f681b0 0x559be1318e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62bfe00640 1 -- 172.21.15.27:0/544989153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be0fd3790 con 0x559be0f474d0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62c7e00640 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be13183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62bfe00640 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f474d0 0x559be13188e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f62b0013400 tx=0x7f62b0013ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1149593389166d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62b5600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62b5600640 1 -- 172.21.15.27:0/544989153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0024070 con 0x559be0f474d0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b00054d0 con 0x559be0f474d0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62b5600640 1 -- 172.21.15.27:0/544989153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f62b0004d30 con 0x559be0f474d0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.613+0000 7f62b5600640 1 -- 172.21.15.27:0/544989153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62b0030050 con 0x559be0f474d0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 10 client.18783 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559be104c620 con 0x559be0f474d0 2024-09-21T07:24:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62b5600640 1 -- 172.21.15.27:0/544989153 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b00054d0 con 0x559be0f474d0 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62b5600640 1 -- 172.21.15.27:0/544989153 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f62b0020440 con 0x559be0f474d0 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 10 client.18783 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 10 client.18783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 20 client.18783 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0f26d80 0x559be132a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 4 client.18783 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.614+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559be1284600 con 0x559be0f26d80 2024-09-21T07:24:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.615+0000 7f62c7e00640 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0f26d80 0x559be132a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.615+0000 7f62c7e00640 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0f26d80 0x559be132a350 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b46987ca270792d server_cookie=edfad52925995f18 in_seq=0 out_seq=0 2024-09-21T07:24:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.615+0000 7f62b5600640 10 client.18783 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.616+0000 7f62b5600640 1 -- 172.21.15.27:0/544989153 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559be1284600 con 0x559be0f26d80 2024-09-21T07:24:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.616+0000 7f62b5600640 10 client.18783 handle_command_reply: tid=0 2024-09-21T07:24:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.616+0000 7f62d5a5e740 1 client.18783 shutdown 2024-09-21T07:24:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.616+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f474d0 msgr2=0x559be13188e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.616+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f474d0 0x559be13188e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f62b0013400 tx=0x7f62b0013ef0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.616+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 shutdown_connections 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.616+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0f26d80 0x559be132a350 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f681b0 0x559be1318e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f474d0 0x559be13188e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62b5600640 0 client.18783 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/544989153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f3f2f0 0x559be13183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 >> 172.21.15.27:0/544989153 conn(0x559be0f5c300 msgr2=0x559be130de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 shutdown_connections 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62d5a5e740 1 -- 172.21.15.27:0/544989153 wait complete. 2024-09-21T07:24:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.617+0000 7f62d5a5e740 20 client.18783 trim_cache size 0 max 16384 2024-09-21T07:24:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.632+0000 7f62d5a5e740 1 Processor -- start 2024-09-21T07:24:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.632+0000 7f62d5a5e740 1 -- start start 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f3f2f0 0x559be1309d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f681b0 0x559be132f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b9bc0 con 0x559be0f3f2f0 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b7490 con 0x559be0f474d0 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b5300 con 0x559be0f681b0 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f3f2f0 0x559be1309d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62c7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f3f2f0 0x559be1309d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58178/0 (socket says 172.21.15.27:58178) 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62c7e00640 1 -- 172.21.15.27:0/2076377790 learned_addr learned my addr 172.21.15.27:0/2076377790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62bfe00640 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62cca00640 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f681b0 0x559be132f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62c7e00640 1 -- 172.21.15.27:0/2076377790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f681b0 msgr2=0x559be132f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.633+0000 7f62c7e00640 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f681b0 0x559be132f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62c7e00640 1 -- 172.21.15.27:0/2076377790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 msgr2=0x559be130e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62c7e00640 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62c7e00640 1 -- 172.21.15.27:0/2076377790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be131aee0 con 0x559be0f3f2f0 2024-09-21T07:24:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62bfe00640 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62c7e00640 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f3f2f0 0x559be1309d20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x559be132a980 tx=0x7f62b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436cc3ac2592df7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62b7e00640 1 -- 172.21.15.27:0/2076377790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8015c70 con 0x559be0f3f2f0 2024-09-21T07:24:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62b7e00640 1 -- 172.21.15.27:0/2076377790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b802f070 con 0x559be0f3f2f0 2024-09-21T07:24:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.634+0000 7f62b7e00640 1 -- 172.21.15.27:0/2076377790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8014880 con 0x559be0f3f2f0 2024-09-21T07:24:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 -- 172.21.15.27:0/2076377790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f3f2f0 msgr2=0x559be1309d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f3f2f0 0x559be1309d20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x559be132a980 tx=0x7f62b8014000 comp rx=0 tx=0).stop 2024-09-21T07:24:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 -- 172.21.15.27:0/2076377790 shutdown_connections 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f681b0 0x559be132f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f474d0 0x559be130e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/2076377790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f3f2f0 0x559be1309d20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 -- 172.21.15.27:0/2076377790 >> 172.21.15.27:0/2076377790 conn(0x559be0f5c300 msgr2=0x559be130de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 -- 172.21.15.27:0/2076377790 shutdown_connections 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.635+0000 7f62d5a5e740 1 -- 172.21.15.27:0/2076377790 wait complete. 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 -- start start 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 Processor -- start 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f3f2f0 0x559be131e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f474d0 0x559be131ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be131f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b5950 con 0x559be0f681b0 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b6dc0 con 0x559be0f3f2f0 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81b6020 con 0x559be0f474d0 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.636+0000 7f62d5a5e740 10 client.0 _ll_register_callbacks cb 0x7f62b0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be131f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be131f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58188/0 (socket says 172.21.15.27:58188) 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 -- 172.21.15.27:0/3077529901 learned_addr learned my addr 172.21.15.27:0/3077529901 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62c7e00640 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f3f2f0 0x559be131e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62bfe00640 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f474d0 0x559be131ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 -- 172.21.15.27:0/3077529901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f474d0 msgr2=0x559be131ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f474d0 0x559be131ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 -- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f3f2f0 msgr2=0x559be131e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f3f2f0 0x559be131e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 -- 172.21.15.27:0/3077529901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be130fc20 con 0x559be0f681b0 2024-09-21T07:24:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62cca00640 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be131f3a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f62c00cde60 tx=0x7f62c00cdbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273d35c42a358234 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.637+0000 7f62b5600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62b5600640 1 -- 172.21.15.27:0/3077529901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00f1040 con 0x559be0f681b0 2024-09-21T07:24:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62b5600640 1 -- 172.21.15.27:0/3077529901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f62c0108430 con 0x559be0f681b0 2024-09-21T07:24:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b802f070 con 0x559be0f681b0 2024-09-21T07:24:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62b5600640 1 -- 172.21.15.27:0/3077529901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62c00ce860 con 0x559be0f681b0 2024-09-21T07:24:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 10 client.18789 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559be1394e40 con 0x559be0f681b0 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62b5600640 1 -- 172.21.15.27:0/3077529901 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b802f070 con 0x559be0f681b0 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62b5600640 1 -- 172.21.15.27:0/3077529901 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f62c00c9ab0 con 0x559be0f681b0 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 10 client.18789 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 10 client.18789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 20 client.18789 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0efc1c0 0x559be13975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 4 client.18789 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.638+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559be12800b0 con 0x559be0efc1c0 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.639+0000 7f62c7e00640 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0efc1c0 0x559be13975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.639+0000 7f62c7e00640 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0efc1c0 0x559be13975e0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7872731261f97ea4 server_cookie=88e91043d6b395fd in_seq=0 out_seq=0 2024-09-21T07:24:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.639+0000 7f62b5600640 10 client.18789 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62b5600640 1 -- 172.21.15.27:0/3077529901 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559be12800b0 con 0x559be0efc1c0 2024-09-21T07:24:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62b5600640 10 client.18789 handle_command_reply: tid=0 2024-09-21T07:24:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 client.18789 shutdown 2024-09-21T07:24:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 msgr2=0x559be131f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be131f3a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f62c00cde60 tx=0x7f62c00cdbe0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 shutdown_connections 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559be0efc1c0 0x559be13975e0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559be0f474d0 0x559be131ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559be0f681b0 0x559be131f3a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 --2- 172.21.15.27:0/3077529901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559be0f3f2f0 0x559be131e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62b5600640 0 client.18789 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 >> 172.21.15.27:0/3077529901 conn(0x559be0f5c300 msgr2=0x559be1309af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 shutdown_connections 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 1 -- 172.21.15.27:0/3077529901 wait complete. 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.653+0000 7f62d5a5e740 20 client.18789 trim_cache size 0 max 16384 2024-09-21T07:24:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:24:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:24:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:24:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:24:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62ac0b6680 msgr2=0x7f62ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62ac0b6680 0x7f62ac0b8b90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559be130ba90 tx=0x7f62b8008040 comp rx=0 tx=0).stop 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 msgr2=0x7f62c81a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 0x7f62c81a6120 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f62c002ac10 tx=0x7f62c0003c90 comp rx=0 tx=0).stop 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 shutdown_connections 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f62ac0b6680 0x7f62ac0b8b90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62c81141d0 0x7f62c81a6120 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c8077390 0x7f62c81a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 --2- 172.21.15.27:0/760413205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c8075dd0 0x7f62c81a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 >> 172.21.15.27:0/760413205 conn(0x7f62c80ff330 msgr2=0x7f62c81100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 shutdown_connections 2024-09-21T07:24:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:14.654+0000 7f62cd400640 1 -- 172.21.15.27:0/760413205 wait complete. 2024-09-21T07:24:14.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:24:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:14 smithi119 ceph-mon[34519]: pgmap v726: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-09-21T07:24:15.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:14 smithi027 ceph-mon[30779]: pgmap v726: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-09-21T07:24:15.393 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:14 smithi153 ceph-mon[34747]: pgmap v726: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 -- 172.21.15.27:0/1493374780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc10c040 msgr2=0x7f3ecc112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 --2- 172.21.15.27:0/1493374780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc10c040 0x7f3ecc112870 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009ca0 tx=0x7f3ec402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 -- 172.21.15.27:0/1493374780 shutdown_connections 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 --2- 172.21.15.27:0/1493374780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc10c040 0x7f3ecc112870 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 --2- 172.21.15.27:0/1493374780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 0x7f3ecc10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 --2- 172.21.15.27:0/1493374780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ecc105db0 0x7f3ecc1061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 -- 172.21.15.27:0/1493374780 >> 172.21.15.27:0/1493374780 conn(0x7f3ecc1017e0 msgr2=0x7f3ecc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 -- 172.21.15.27:0/1493374780 shutdown_connections 2024-09-21T07:24:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.075+0000 7f3ed4000640 1 -- 172.21.15.27:0/1493374780 wait complete. 2024-09-21T07:24:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.077+0000 7f3ed4000640 1 Processor -- start 2024-09-21T07:24:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.077+0000 7f3ed4000640 1 -- start start 2024-09-21T07:24:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.077+0000 7f3ed4000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc105db0 0x7f3ecc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed4000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 0x7f3ecc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed4000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ecc10c040 0x7f3ecc1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed4000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc115d00 con 0x7f3ecc105db0 2024-09-21T07:24:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed4000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc115b80 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed4000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc115e80 con 0x7f3ecc10c040 2024-09-21T07:24:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc105db0 0x7f3ecc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 0x7f3ecc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 0x7f3ecc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56422/0 (socket says 172.21.15.27:56422) 2024-09-21T07:24:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 -- 172.21.15.27:0/3227377274 learned_addr learned my addr 172.21.15.27:0/3227377274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc105db0 0x7f3ecc19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58194/0 (socket says 172.21.15.27:58194) 2024-09-21T07:24:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed2000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ecc10c040 0x7f3ecc1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 -- 172.21.15.27:0/3227377274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ecc10c040 msgr2=0x7f3ecc1a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ecc10c040 0x7f3ecc1a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 -- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc105db0 msgr2=0x7f3ecc19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc105db0 0x7f3ecc19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed0c00640 1 -- 172.21.15.27:0/3227377274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc1a2620 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.078+0000 7f3ed2000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ecc10c040 0x7f3ecc1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.079+0000 7f3ed0c00640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 0x7f3ecc19cc50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800e9f0 tx=0x7f3eb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.079+0000 7f3eca000640 1 -- 172.21.15.27:0/3227377274 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb800ccc0 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.079+0000 7f3eca000640 1 -- 172.21.15.27:0/3227377274 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb80108e0 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.079+0000 7f3eca000640 1 -- 172.21.15.27:0/3227377274 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8018860 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.079+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc1a2910 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.079+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc1a9070 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.082+0000 7f3edac2d740 1 Processor -- start 2024-09-21T07:24:16.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3edac2d740 1 -- start start 2024-09-21T07:24:16.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3eca000640 1 -- 172.21.15.27:0/3227377274 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3eb8010430 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3edac2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56018555dfb0 con 0x56018521f390 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3edac2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56018554d3e0 con 0x5601852171b0 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3edac2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5601855511f0 con 0x560185240070 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3ed0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46538/0 (socket says 172.21.15.27:46538) 2024-09-21T07:24:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.083+0000 7f3ed2000640 1 -- 172.21.15.27:0/419547794 learned_addr learned my addr 172.21.15.27:0/419547794 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.084+0000 7f3ed2000640 1 -- 172.21.15.27:0/419547794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 msgr2=0x5601855e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.084+0000 7f3ed2000640 1 --2- 172.21.15.27:0/419547794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.084+0000 7f3ed2000640 1 -- 172.21.15.27:0/419547794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 msgr2=0x5601855e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.084+0000 7f3ed2000640 1 --2- 172.21.15.27:0/419547794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855e7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.084+0000 7f3ed2000640 1 -- 172.21.15.27:0/419547794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560185517cd0 con 0x560185240070 2024-09-21T07:24:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.084+0000 7f3ed2000640 1 --2- 172.21.15.27:0/419547794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855e6da0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009880 tx=0x7f3ec40098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8483dd59c89e09d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.085+0000 7f3ebe600640 1 -- 172.21.15.27:0/419547794 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4030800 con 0x560185240070 2024-09-21T07:24:16.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.085+0000 7f3ebe600640 1 -- 172.21.15.27:0/419547794 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec4036040 con 0x560185240070 2024-09-21T07:24:16.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.085+0000 7f3ebe600640 1 -- 172.21.15.27:0/419547794 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4004440 con 0x560185240070 2024-09-21T07:24:16.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 -- 172.21.15.27:0/419547794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 msgr2=0x5601855e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 --2- 172.21.15.27:0/419547794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855e6da0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009880 tx=0x7f3ec40098b0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3eca000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3eb40b6680 0x7f3eb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 -- 172.21.15.27:0/419547794 shutdown_connections 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 --2- 172.21.15.27:0/419547794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 --2- 172.21.15.27:0/419547794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 --2- 172.21.15.27:0/419547794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855e6da0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 -- 172.21.15.27:0/419547794 >> 172.21.15.27:0/419547794 conn(0x560185234460 msgr2=0x5601855e4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3eca000640 1 -- 172.21.15.27:0/3227377274 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f3eb80e7e60 con 0x7f3ecc10b6f0 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3ed2000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3eb40b6680 0x7f3eb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 -- 172.21.15.27:0/419547794 shutdown_connections 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.086+0000 7f3edac2d740 1 -- 172.21.15.27:0/419547794 wait complete. 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.087+0000 7f3ed2000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3eb40b6680 0x7f3eb40b8b90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ec402add0 tx=0x7f3ec403e040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.087+0000 7f3edac2d740 1 -- start start 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.087+0000 7f3edac2d740 1 Processor -- start 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.087+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x5601855f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56018521f390 0x5601855f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560185240070 0x5601855fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3edac2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5601853ae700 con 0x5601852171b0 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3edac2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5601854e8580 con 0x560185240070 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3edac2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5601855f5520 con 0x56018521f390 2024-09-21T07:24:16.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3edac2d740 10 client.0 _ll_register_callbacks cb 0x5601854f8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x5601855f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560185240070 0x5601855fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x5601855f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58220/0 (socket says 172.21.15.27:58220) 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed1600640 1 -- 172.21.15.27:0/2708272230 learned_addr learned my addr 172.21.15.27:0/2708272230 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560185240070 0x5601855fe810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56454/0 (socket says 172.21.15.27:56454) 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed0c00640 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56018521f390 0x5601855f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed2000640 1 -- 172.21.15.27:0/2708272230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56018521f390 msgr2=0x5601855f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed2000640 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56018521f390 0x5601855f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed2000640 1 -- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 msgr2=0x5601855f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed2000640 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x5601855f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed2000640 1 -- 172.21.15.27:0/2708272230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601852ab650 con 0x560185240070 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.088+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x5601855f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3ed2000640 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560185240070 0x5601855fe810 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4004140 tx=0x7f3ec40098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571b0f0686840b53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:16.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2708272230 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec404a040 con 0x560185240070 2024-09-21T07:24:16.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5601854eab90 con 0x560185240070 2024-09-21T07:24:16.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2708272230 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f3ec4003970 con 0x560185240070 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2708272230 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ec4003b70 con 0x560185240070 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3edac2d740 10 client.28817 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.089+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560185505950 con 0x560185240070 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2708272230 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5601854eab90 con 0x560185240070 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2708272230 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec404d030 con 0x560185240070 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3edac2d740 10 client.28817 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3edac2d740 10 client.28817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3edac2d740 20 client.28817 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851fe9f0 0x5601856044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3edac2d740 4 client.28817 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56018555ff00 con 0x5601851fe9f0 2024-09-21T07:24:16.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.090+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851fe9f0 0x5601856044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.091+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851fe9f0 0x5601856044b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44c3cd4b92a84775 server_cookie=1a2cf96e94a401f1 in_seq=0 out_seq=0 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.091+0000 7f3eb2a00640 10 client.28817 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2708272230 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56018555ff00 con 0x5601851fe9f0 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3eb2a00640 10 client.28817 handle_command_reply: tid=0 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 client.28817 shutdown 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560185240070 msgr2=0x5601855fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560185240070 0x5601855fe810 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4004140 tx=0x7f3ec40098b0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 shutdown_connections 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851fe9f0 0x5601856044b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56018521f390 0x5601855f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560185240070 0x5601855fe810 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2708272230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x5601855f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 >> 172.21.15.27:0/2708272230 conn(0x560185234460 msgr2=0x5601855e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:16.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.092+0000 7f3eb2a00640 0 client.28817 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:16.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.093+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 shutdown_connections 2024-09-21T07:24:16.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.093+0000 7f3edac2d740 1 -- 172.21.15.27:0/2708272230 wait complete. 2024-09-21T07:24:16.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.093+0000 7f3edac2d740 20 client.28817 trim_cache size 0 max 16384 2024-09-21T07:24:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 Processor -- start 2024-09-21T07:24:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 -- start start 2024-09-21T07:24:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855fe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855ef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1b5a60 con 0x56018521f390 2024-09-21T07:24:16.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1b3330 con 0x5601852171b0 2024-09-21T07:24:16.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.107+0000 7f3edac2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1b11a0 con 0x560185240070 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855fe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855fe780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58226/0 (socket says 172.21.15.27:58226) 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855ea5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56466/0 (socket says 172.21.15.27:56466) 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed0c00640 1 -- 172.21.15.27:0/2741811256 learned_addr learned my addr 172.21.15.27:0/2741811256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 -- 172.21.15.27:0/2741811256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 msgr2=0x5601855ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855ef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 -- 172.21.15.27:0/2741811256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 msgr2=0x5601855fe780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855fe780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 -- 172.21.15.27:0/2741811256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601855e9c00 con 0x5601852171b0 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed2000640 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed0c00640 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.108+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855ea5b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5601853ae850 tx=0x7f3ec0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceabe72534fc242d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.109+0000 7f3ebdc00640 1 -- 172.21.15.27:0/2741811256 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0005220 con 0x5601852171b0 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.109+0000 7f3ebdc00640 1 -- 172.21.15.27:0/2741811256 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec0004c50 con 0x5601852171b0 2024-09-21T07:24:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.109+0000 7f3ebdc00640 1 -- 172.21.15.27:0/2741811256 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0025860 con 0x5601852171b0 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 -- 172.21.15.27:0/2741811256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 msgr2=0x5601855ea5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855ea5b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5601853ae850 tx=0x7f3ec0005f90 comp rx=0 tx=0).stop 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 -- 172.21.15.27:0/2741811256 shutdown_connections 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x5601855ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56018521f390 0x5601855fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2741811256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852171b0 0x5601855ea5b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 -- 172.21.15.27:0/2741811256 >> 172.21.15.27:0/2741811256 conn(0x560185234460 msgr2=0x5601855e9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 -- 172.21.15.27:0/2741811256 shutdown_connections 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 -- 172.21.15.27:0/2741811256 wait complete. 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.110+0000 7f3edac2d740 1 -- start start 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 1 Processor -- start 2024-09-21T07:24:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x560185605f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521f390 0x5601856064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x560185667c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1b17f0 con 0x5601852171b0 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1b2c60 con 0x56018521f390 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1b1ec0 con 0x560185240070 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.111+0000 7f3edac2d740 10 client.0 _ll_register_callbacks cb 0x7f3ec0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x560185605f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x560185667c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521f390 0x5601856064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x560185605f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58238/0 (socket says 172.21.15.27:58238) 2024-09-21T07:24:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed1600640 1 -- 172.21.15.27:0/2948927380 learned_addr learned my addr 172.21.15.27:0/2948927380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed2000640 1 -- 172.21.15.27:0/2948927380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521f390 msgr2=0x5601856064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed2000640 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521f390 0x5601856064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed2000640 1 -- 172.21.15.27:0/2948927380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 msgr2=0x560185605f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed2000640 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x560185605f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed2000640 1 -- 172.21.15.27:0/2948927380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601856682e0 con 0x560185240070 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed0c00640 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521f390 0x5601856064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3ed2000640 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x560185667c70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3ec404e040 tx=0x7f3ec4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df14aa548f2630e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.112+0000 7f3eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.113+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2948927380 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4054070 con 0x560185240070 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.113+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2948927380 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f3ec4032a60 con 0x560185240070 2024-09-21T07:24:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.113+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec0004c50 con 0x560185240070 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.113+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2948927380 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ec4074050 con 0x560185240070 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.113+0000 7f3edac2d740 10 client.28801 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.113+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56018566f6d0 con 0x560185240070 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.113+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2948927380 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec0004c50 con 0x560185240070 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2948927380 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec4069020 con 0x560185240070 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3edac2d740 10 client.28801 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3edac2d740 10 client.28801 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3edac2d740 20 client.28801 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851d3da0 0x560185671d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3edac2d740 4 client.28801 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56018555b9b0 con 0x5601851d3da0 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.114+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851d3da0 0x560185671d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3ed1600640 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851d3da0 0x560185671d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8f24aacd242544 server_cookie=29d64dcec1c3d767 in_seq=0 out_seq=0 2024-09-21T07:24:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3eb2a00640 10 client.28801 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3eb2a00640 1 -- 172.21.15.27:0/2948927380 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56018555b9b0 con 0x5601851d3da0 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3eb2a00640 10 client.28801 handle_command_reply: tid=0 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 client.28801 shutdown 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 msgr2=0x560185667c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x560185667c70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3ec404e040 tx=0x7f3ec4004a40 comp rx=0 tx=0).stop 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 shutdown_connections 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x5601851d3da0 0x560185671d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560185240070 0x560185667c70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521f390 0x5601856064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 --2- 172.21.15.27:0/2948927380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5601852171b0 0x560185605f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 >> 172.21.15.27:0/2948927380 conn(0x560185234460 msgr2=0x5601855e7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.115+0000 7f3eb2a00640 0 client.28801 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:24:16.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.116+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 shutdown_connections 2024-09-21T07:24:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.116+0000 7f3edac2d740 1 -- 172.21.15.27:0/2948927380 wait complete. 2024-09-21T07:24:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.116+0000 7f3edac2d740 20 client.28801 trim_cache size 0 max 16384 2024-09-21T07:24:16.121 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3eb40b6680 msgr2=0x7f3eb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3eb40b6680 0x7f3eb40b8b90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ec402add0 tx=0x7f3ec403e040 comp rx=0 tx=0).stop 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 msgr2=0x7f3ecc19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 0x7f3ecc19cc50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800e9f0 tx=0x7f3eb800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 shutdown_connections 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ecc10c040 0x7f3ecc1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3eb40b6680 0x7f3eb40b8b90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ecc10b6f0 0x7f3ecc19cc50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 --2- 172.21.15.27:0/3227377274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ecc105db0 0x7f3ecc19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 >> 172.21.15.27:0/3227377274 conn(0x7f3ecc1017e0 msgr2=0x7f3ecc108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:16.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 shutdown_connections 2024-09-21T07:24:16.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:16.117+0000 7f3ed4000640 1 -- 172.21.15.27:0/3227377274 wait complete. 2024-09-21T07:24:16.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:24:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:16 smithi027 ceph-mon[30779]: pgmap v727: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:24:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:16 smithi119 ceph-mon[34519]: pgmap v727: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:24:17.326 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:17.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:16 smithi153 ceph-mon[34747]: pgmap v727: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:24:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 -- 172.21.15.27:0/3354673568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 msgr2=0x7f5370112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 --2- 172.21.15.27:0/3354673568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 0x7f5370112500 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5368009ca0 tx=0x7f536802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 -- 172.21.15.27:0/3354673568 shutdown_connections 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 --2- 172.21.15.27:0/3354673568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 0x7f5370112500 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 --2- 172.21.15.27:0/3354673568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5370076100 0x7f537010fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 --2- 172.21.15.27:0/3354673568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 0x7f5370075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 -- 172.21.15.27:0/3354673568 >> 172.21.15.27:0/3354673568 conn(0x7f537007b850 msgr2=0x7f537007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 -- 172.21.15.27:0/3354673568 shutdown_connections 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.996+0000 7f5376400640 1 -- 172.21.15.27:0/3354673568 wait complete. 2024-09-21T07:24:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.997+0000 7f5376400640 1 Processor -- start 2024-09-21T07:24:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.997+0000 7f5376400640 1 -- start start 2024-09-21T07:24:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f5376400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 0x7f537019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f5376400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5370076100 0x7f537019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f5376400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 0x7f53701a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f5376400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5370115c80 con 0x7f53701100c0 2024-09-21T07:24:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f5376400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5370115b00 con 0x7f5370076100 2024-09-21T07:24:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f5376400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5370115e00 con 0x7f53700757b0 2024-09-21T07:24:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f536f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 0x7f537019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f536ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5370076100 0x7f537019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f536f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 0x7f537019c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52330/0 (socket says 172.21.15.27:52330) 2024-09-21T07:24:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.998+0000 7f536f400640 1 -- 172.21.15.27:0/3595252222 learned_addr learned my addr 172.21.15.27:0/3595252222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.999+0000 7f536fe00640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 0x7f53701a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.999+0000 7f536f400640 1 -- 172.21.15.27:0/3595252222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5370076100 msgr2=0x7f537019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.999+0000 7f536f400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5370076100 0x7f537019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.999+0000 7f536f400640 1 -- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 msgr2=0x7f53701a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.999+0000 7f536f400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 0x7f53701a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.999+0000 7f536f400640 1 -- 172.21.15.27:0/3595252222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53701a2560 con 0x7f53700757b0 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:17.999+0000 7f536f400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 0x7f537019c5f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f536000ea60 tx=0x7f536000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.000+0000 7f5367e00640 1 -- 172.21.15.27:0/3595252222 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536000cde0 con 0x7f53700757b0 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.000+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53701a2850 con 0x7f53700757b0 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.000+0000 7f5367e00640 1 -- 172.21.15.27:0/3595252222 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53600035f0 con 0x7f53700757b0 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.000+0000 7f5367e00640 1 -- 172.21.15.27:0/3595252222 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360010690 con 0x7f53700757b0 2024-09-21T07:24:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.000+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53701a8fb0 con 0x7f53700757b0 2024-09-21T07:24:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.003+0000 7f537d201740 1 Processor -- start 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.003+0000 7f537d201740 1 -- start start 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f5367e00640 1 -- 172.21.15.27:0/3595252222 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f53600037a0 con 0x7f53700757b0 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f537d201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211e98ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f537d201740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5bb1b0 0x56211e98b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f537d201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e98b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f537d201740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56211e901fb0 con 0x56211e5bb1b0 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f537d201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56211e8f13e0 con 0x56211e5e4070 2024-09-21T07:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f537d201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56211e8f51f0 con 0x56211e5c3390 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f536f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5bb1b0 0x56211e98b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f536f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5bb1b0 0x56211e98b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57590/0 (socket says 172.21.15.27:57590) 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f536f400640 1 -- 172.21.15.27:0/4282559074 learned_addr learned my addr 172.21.15.27:0/4282559074 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f536fe00640 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211e98ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.004+0000 7f536ea00640 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e98b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f536f400640 1 -- 172.21.15.27:0/4282559074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 msgr2=0x56211e98b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f536f400640 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e98b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f536f400640 1 -- 172.21.15.27:0/4282559074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 msgr2=0x56211e98ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f536f400640 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211e98ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f536f400640 1 -- 172.21.15.27:0/4282559074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211e8bbcd0 con 0x56211e5bb1b0 2024-09-21T07:24:18.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f536f400640 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5bb1b0 0x56211e98b2e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f53600e50c0 tx=0x7f53600f04b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e701a4c69dd82f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f5367e00640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53540b6680 0x7f53540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.005+0000 7f5364c00640 1 -- 172.21.15.27:0/4282559074 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600fb070 con 0x56211e5bb1b0 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.006+0000 7f5364c00640 1 -- 172.21.15.27:0/4282559074 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53600ad360 con 0x56211e5bb1b0 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.006+0000 7f5364c00640 1 -- 172.21.15.27:0/4282559074 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600acfc0 con 0x56211e5bb1b0 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.006+0000 7f536fe00640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53540b6680 0x7f53540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.006+0000 7f5367e00640 1 -- 172.21.15.27:0/3595252222 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f53600e7db0 con 0x7f53700757b0 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 -- 172.21.15.27:0/4282559074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5bb1b0 msgr2=0x56211e98b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5bb1b0 0x56211e98b2e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f53600e50c0 tx=0x7f53600f04b0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f536fe00640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53540b6680 0x7f53540b8b90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5368006010 tx=0x7f5368005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:18.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 -- 172.21.15.27:0/4282559074 shutdown_connections 2024-09-21T07:24:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e98b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5bb1b0 0x56211e98b2e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 --2- 172.21.15.27:0/4282559074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211e98ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 -- 172.21.15.27:0/4282559074 >> 172.21.15.27:0/4282559074 conn(0x56211e5d8460 msgr2=0x56211e988c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 -- 172.21.15.27:0/4282559074 shutdown_connections 2024-09-21T07:24:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.007+0000 7f537d201740 1 -- 172.21.15.27:0/4282559074 wait complete. 2024-09-21T07:24:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 -- start start 2024-09-21T07:24:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 Processor -- start 2024-09-21T07:24:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e996500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e996a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5e4070 0x56211e9a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56211e752700 con 0x56211e5e4070 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56211e88c580 con 0x56211e5bb1b0 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56211e999580 con 0x56211e5c3390 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.008+0000 7f537d201740 10 client.0 _ll_register_callbacks cb 0x56211e89c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e996500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e996500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41688/0 (socket says 172.21.15.27:41688) 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536f400640 1 -- 172.21.15.27:0/3998119084 learned_addr learned my addr 172.21.15.27:0/3998119084 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e996a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536fe00640 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5e4070 0x56211e9a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536ea00640 1 -- 172.21.15.27:0/3998119084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 msgr2=0x56211e996500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536ea00640 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e996500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536ea00640 1 -- 172.21.15.27:0/3998119084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5e4070 msgr2=0x56211e9a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536ea00640 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5e4070 0x56211e9a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.009+0000 7f536ea00640 1 -- 172.21.15.27:0/3998119084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211e64f650 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f536ea00640 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e996a40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f537006c760 tx=0x7f535c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3193f6928bf2f400 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f5359600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f5359600640 1 -- 172.21.15.27:0/3998119084 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c004050 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56211e88eb90 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f5359600640 1 -- 172.21.15.27:0/3998119084 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f535c0041f0 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f5359600640 1 -- 172.21.15.27:0/3998119084 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f535c025050 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f537d201740 10 client.28813 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.010+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56211e8a9950 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f5359600640 1 -- 172.21.15.27:0/3998119084 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56211e88eb90 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f5359600640 1 -- 172.21.15.27:0/3998119084 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f535c0039c0 con 0x56211e5c3390 2024-09-21T07:24:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f537d201740 10 client.28813 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f537d201740 10 client.28813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f537d201740 20 client.28813 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f537d201740 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e5a29f0 0x56211e9a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f537d201740 4 client.28813 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56211e903f00 con 0x56211e5a29f0 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.011+0000 7f536f400640 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e5a29f0 0x56211e9a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.012+0000 7f536f400640 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e5a29f0 0x56211e9a8720 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63776943db63a648 server_cookie=4eb2c2793931794c in_seq=0 out_seq=0 2024-09-21T07:24:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.012+0000 7f5359600640 10 client.28813 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f5359600640 1 -- 172.21.15.27:0/3998119084 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56211e903f00 con 0x56211e5a29f0 2024-09-21T07:24:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f5359600640 10 client.28813 handle_command_reply: tid=0 2024-09-21T07:24:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 client.28813 shutdown 2024-09-21T07:24:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 msgr2=0x56211e996a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e996a40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f537006c760 tx=0x7f535c012b60 comp rx=0 tx=0).stop 2024-09-21T07:24:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 shutdown_connections 2024-09-21T07:24:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e5a29f0 0x56211e9a8720 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5c3390 0x56211e996a40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5e4070 0x56211e9a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 --2- 172.21.15.27:0/3998119084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e996500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f5359600640 0 client.28813 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 >> 172.21.15.27:0/3998119084 conn(0x56211e5d8460 msgr2=0x56211e98df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 shutdown_connections 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.013+0000 7f537d201740 1 -- 172.21.15.27:0/3998119084 wait complete. 2024-09-21T07:24:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.014+0000 7f537d201740 20 client.28813 trim_cache size 0 max 16384 2024-09-21T07:24:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 Processor -- start 2024-09-21T07:24:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 -- start start 2024-09-21T07:24:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e9969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e9a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5e4070 0x56211e9a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b5900 con 0x56211e5c3390 2024-09-21T07:24:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b31d0 con 0x56211e5bb1b0 2024-09-21T07:24:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.028+0000 7f537d201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b1040 con 0x56211e5e4070 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e9a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e9969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e9a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57610/0 (socket says 172.21.15.27:57610) 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5e4070 0x56211e9a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536ea00640 1 -- 172.21.15.27:0/1499833415 learned_addr learned my addr 172.21.15.27:0/1499833415 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e9969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41692/0 (socket says 172.21.15.27:41692) 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 -- 172.21.15.27:0/1499833415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5e4070 msgr2=0x56211e9a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5e4070 0x56211e9a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 -- 172.21.15.27:0/1499833415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 msgr2=0x56211e9a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e9a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 -- 172.21.15.27:0/1499833415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211e994a00 con 0x56211e5bb1b0 2024-09-21T07:24:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.029+0000 7f536f400640 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e9969c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f53600ad550 tx=0x7f536000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a5b4d5056f5ee0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.030+0000 7f535be00640 1 -- 172.21.15.27:0/1499833415 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600d9560 con 0x56211e5bb1b0 2024-09-21T07:24:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.030+0000 7f535be00640 1 -- 172.21.15.27:0/1499833415 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53600ad360 con 0x56211e5bb1b0 2024-09-21T07:24:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.030+0000 7f535be00640 1 -- 172.21.15.27:0/1499833415 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600da1e0 con 0x56211e5bb1b0 2024-09-21T07:24:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.030+0000 7f537d201740 1 -- 172.21.15.27:0/1499833415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 msgr2=0x56211e9969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.030+0000 7f537d201740 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e9969c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f53600ad550 tx=0x7f536000ef90 comp rx=0 tx=0).stop 2024-09-21T07:24:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 -- 172.21.15.27:0/1499833415 shutdown_connections 2024-09-21T07:24:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5e4070 0x56211e9a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e9a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 --2- 172.21.15.27:0/1499833415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5bb1b0 0x56211e9969c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 -- 172.21.15.27:0/1499833415 >> 172.21.15.27:0/1499833415 conn(0x56211e5d8460 msgr2=0x56211e9962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 -- 172.21.15.27:0/1499833415 shutdown_connections 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 -- 172.21.15.27:0/1499833415 wait complete. 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 -- start start 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.031+0000 7f537d201740 1 Processor -- start 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f537d201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5bb1b0 0x56211e99ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f537d201740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e99b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f537d201740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211ea12120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f537d201740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b1690 con 0x56211e5c3390 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f537d201740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b2b00 con 0x56211e5e4070 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f537d201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b1d60 con 0x56211e5bb1b0 2024-09-21T07:24:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f537d201740 10 client.0 _ll_register_callbacks cb 0x7f535c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.032+0000 7f536f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5bb1b0 0x56211e99ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5bb1b0 0x56211e99ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52368/0 (socket says 172.21.15.27:52368) 2024-09-21T07:24:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536f400640 1 -- 172.21.15.27:0/2580153227 learned_addr learned my addr 172.21.15.27:0/2580153227 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536f400640 1 -- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 msgr2=0x56211ea12120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536fe00640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211ea12120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536f400640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211ea12120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536ea00640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e99b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536f400640 1 -- 172.21.15.27:0/2580153227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 msgr2=0x56211e99b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536f400640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e99b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536f400640 1 -- 172.21.15.27:0/2580153227 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211ea12660 con 0x56211e5bb1b0 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.033+0000 7f536ea00640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e99b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f536f400640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5bb1b0 0x56211e99ab20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f536000c760 tx=0x7f53600f0ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e479142c0184f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f5359600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f5359600640 1 -- 172.21.15.27:0/2580153227 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600d7320 con 0x56211e5bb1b0 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f5359600640 1 -- 172.21.15.27:0/2580153227 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f5360003ac0 con 0x56211e5bb1b0 2024-09-21T07:24:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53600ad360 con 0x56211e5bb1b0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f5359600640 1 -- 172.21.15.27:0/2580153227 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53600d64c0 con 0x56211e5bb1b0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f537d201740 10 client.28819 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56211ea13720 con 0x56211e5bb1b0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.034+0000 7f5359600640 1 -- 172.21.15.27:0/2580153227 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536006be50 con 0x56211e5bb1b0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f5359600640 1 -- 172.21.15.27:0/2580153227 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f53600e53d0 con 0x56211e5bb1b0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f537d201740 10 client.28819 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f537d201740 10 client.28819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f537d201740 20 client.28819 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f537d201740 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e577da0 0x56211ea15900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f537d201740 4 client.28819 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56211e8ff9b0 con 0x56211e577da0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.035+0000 7f536ea00640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e577da0 0x56211ea15900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f536ea00640 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e577da0 0x56211ea15900 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe243d2201fd8e6 server_cookie=b1957df1d4ac014d in_seq=0 out_seq=0 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f5359600640 10 client.28819 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f5359600640 1 -- 172.21.15.27:0/2580153227 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56211e8ff9b0 con 0x56211e577da0 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f5359600640 10 client.28819 handle_command_reply: tid=0 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 client.28819 shutdown 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5bb1b0 msgr2=0x56211e99ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5bb1b0 0x56211e99ab20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f536000c760 tx=0x7f53600f0ab0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 shutdown_connections 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56211e577da0 0x56211ea15900 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56211e5e4070 0x56211ea12120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56211e5c3390 0x56211e99b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f5359600640 0 client.28819 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 --2- 172.21.15.27:0/2580153227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211e5bb1b0 0x56211e99ab20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 >> 172.21.15.27:0/2580153227 conn(0x56211e5d8460 msgr2=0x56211e988590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.036+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 shutdown_connections 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.037+0000 7f537d201740 1 -- 172.21.15.27:0/2580153227 wait complete. 2024-09-21T07:24:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.037+0000 7f537d201740 20 client.28819 trim_cache size 0 max 16384 2024-09-21T07:24:18.042 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:24:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.037+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53540b6680 msgr2=0x7f53540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.037+0000 7f5376400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53540b6680 0x7f53540b8b90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5368006010 tx=0x7f5368005d80 comp rx=0 tx=0).stop 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.037+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 msgr2=0x7f537019c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.037+0000 7f5376400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 0x7f537019c5f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f536000ea60 tx=0x7f536000ef20 comp rx=0 tx=0).stop 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 shutdown_connections 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53540b6680 0x7f53540b8b90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53701100c0 0x7f53701a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5370076100 0x7f537019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 --2- 172.21.15.27:0/3595252222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53700757b0 0x7f537019c5f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 >> 172.21.15.27:0/3595252222 conn(0x7f537007b850 msgr2=0x7f5370108be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 shutdown_connections 2024-09-21T07:24:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:18.038+0000 7f5376400640 1 -- 172.21.15.27:0/3595252222 wait complete. 2024-09-21T07:24:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:18 smithi027 ceph-mon[30779]: pgmap v728: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:24:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:18 smithi119 ceph-mon[34519]: pgmap v728: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:24:19.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:18 smithi153 ceph-mon[34747]: pgmap v728: 129 pgs: 129 active+clean; 3.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-09-21T07:24:19.797 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:24:19.798 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:24:20.199 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 -- 172.21.15.27:0/2005798818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf81094a0 msgr2=0x7fecf81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 --2- 172.21.15.27:0/2005798818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf81094a0 0x7fecf81098b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fece4009900 tx=0x7fece402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 -- 172.21.15.27:0/2005798818 shutdown_connections 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 --2- 172.21.15.27:0/2005798818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf8109df0 0x7fecf8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 --2- 172.21.15.27:0/2005798818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf81094a0 0x7fecf81098b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 --2- 172.21.15.27:0/2005798818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf8103b60 0x7fecf8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 -- 172.21.15.27:0/2005798818 >> 172.21.15.27:0/2005798818 conn(0x7fecf807b850 msgr2=0x7fecf807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.827+0000 7fecfe200640 1 -- 172.21.15.27:0/2005798818 shutdown_connections 2024-09-21T07:24:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.828+0000 7fecfe200640 1 -- 172.21.15.27:0/2005798818 wait complete. 2024-09-21T07:24:20.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 Processor -- start 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 -- start start 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf8103b60 0x7fecf819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf81094a0 0x7fecf819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf8109df0 0x7fecf819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8113ab0 con 0x7fecf81094a0 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8113930 con 0x7fecf8109df0 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.829+0000 7fecfe200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8113c30 con 0x7fecf8103b60 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf8109df0 0x7fecf819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf8103b60 0x7fecf819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf81094a0 0x7fecf819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf8109df0 0x7fecf819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41724/0 (socket says 172.21.15.27:41724) 2024-09-21T07:24:20.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 -- 172.21.15.27:0/1764476226 learned_addr learned my addr 172.21.15.27:0/1764476226 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:20.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 -- 172.21.15.27:0/1764476226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf8103b60 msgr2=0x7fecf819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf8103b60 0x7fecf819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 -- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf81094a0 msgr2=0x7fecf819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf81094a0 0x7fecf819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 -- 172.21.15.27:0/1764476226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf81a0450 con 0x7fecf8109df0 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7400640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf8103b60 0x7fecf819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.830+0000 7fecf7e00640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf8109df0 0x7fecf819fd20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fecf000efd0 tx=0x7fecf000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.831+0000 7fecefe00640 1 -- 172.21.15.27:0/1764476226 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0003b70 con 0x7fecf8109df0 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.831+0000 7fecefe00640 1 -- 172.21.15.27:0/1764476226 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf0003d10 con 0x7fecf8109df0 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.831+0000 7fecefe00640 1 -- 172.21.15.27:0/1764476226 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf00108a0 con 0x7fecf8109df0 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.831+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf81a0740 con 0x7fecf8109df0 2024-09-21T07:24:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.831+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf81a6ea0 con 0x7fecf8109df0 2024-09-21T07:24:20.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.834+0000 7fed05001740 1 Processor -- start 2024-09-21T07:24:20.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fecefe00640 1 -- 172.21.15.27:0/1764476226 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fecf0004020 con 0x7fecf8109df0 2024-09-21T07:24:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fed05001740 1 -- start start 2024-09-21T07:24:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fed05001740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7feb6350 0x55ec8025dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fed05001740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec8025e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fed05001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec8025e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fed05001740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ec801d3e90 con 0x55ec7fe8d490 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fed05001740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ec801c32c0 con 0x55ec7feb6350 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.835+0000 7fed05001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ec801c70d0 con 0x55ec7fe95670 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7feb6350 0x55ec8025dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7feb6350 0x55ec8025dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41730/0 (socket says 172.21.15.27:41730) 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 -- 172.21.15.27:0/4041533107 learned_addr learned my addr 172.21.15.27:0/4041533107 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf7400640 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec8025e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec8025e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 -- 172.21.15.27:0/4041533107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 msgr2=0x55ec8025e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec8025e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 -- 172.21.15.27:0/4041533107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 msgr2=0x55ec8025e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec8025e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 -- 172.21.15.27:0/4041533107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7ff21930 con 0x55ec7feb6350 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.836+0000 7fecf6a00640 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7feb6350 0x55ec8025dd10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fece40057c0 tx=0x7fece4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ab5795095b5381 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.837+0000 7fece3e00640 1 -- 172.21.15.27:0/4041533107 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4035070 con 0x55ec7feb6350 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.837+0000 7fece3e00640 1 -- 172.21.15.27:0/4041533107 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece4005030 con 0x55ec7feb6350 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.837+0000 7fece3e00640 1 -- 172.21.15.27:0/4041533107 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40044e0 con 0x55ec7feb6350 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.837+0000 7fecefe00640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecdc0b6700 0x7fecdc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.837+0000 7fecefe00640 1 -- 172.21.15.27:0/1764476226 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fecf00e7ed0 con 0x7fecf8109df0 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.837+0000 7fecf7400640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecdc0b6700 0x7fecdc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 -- 172.21.15.27:0/4041533107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7feb6350 msgr2=0x55ec8025dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7feb6350 0x55ec8025dd10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fece40057c0 tx=0x7fece4000bc0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 -- 172.21.15.27:0/4041533107 shutdown_connections 2024-09-21T07:24:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec8025e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec8025e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 --2- 172.21.15.27:0/4041533107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7feb6350 0x55ec8025dd10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 -- 172.21.15.27:0/4041533107 >> 172.21.15.27:0/4041533107 conn(0x55ec7feaa740 msgr2=0x55ec8025bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 -- 172.21.15.27:0/4041533107 shutdown_connections 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fed05001740 1 -- 172.21.15.27:0/4041533107 wait complete. 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.838+0000 7fecf7400640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecdc0b6700 0x7fecdc0b8c10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55ec8025c990 tx=0x7fece8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.839+0000 7fed05001740 1 -- start start 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.839+0000 7fed05001740 1 Processor -- start 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fed05001740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec802698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fed05001740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec80269e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fed05001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7feb6350 0x55ec802757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fed05001740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ec80024890 con 0x55ec7fe8d490 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fed05001740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ec8015e160 con 0x55ec7fe95670 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fed05001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ec8026c490 con 0x55ec7feb6350 2024-09-21T07:24:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fed05001740 10 client.0 _ll_register_callbacks cb 0x55ec8017aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec802698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec80269e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec80269e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41740/0 (socket says 172.21.15.27:41740) 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 -- 172.21.15.27:0/2549903721 learned_addr learned my addr 172.21.15.27:0/2549903721 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf7e00640 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7feb6350 0x55ec802757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 -- 172.21.15.27:0/2549903721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7feb6350 msgr2=0x55ec802757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7feb6350 0x55ec802757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 -- 172.21.15.27:0/2549903721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 msgr2=0x55ec802698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec802698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.840+0000 7fecf6a00640 1 -- 172.21.15.27:0/2549903721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec80275d40 con 0x55ec7fe95670 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.841+0000 7fecf7e00640 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7feb6350 0x55ec802757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.841+0000 7fecf6a00640 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec80269e20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fece4006010 tx=0x7fece4032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f5412d1740f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.841+0000 7fece0c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.841+0000 7fece0c00640 1 -- 172.21.15.27:0/2549903721 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4035050 con 0x55ec7fe95670 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.841+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece4005030 con 0x55ec7fe95670 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.841+0000 7fece0c00640 1 -- 172.21.15.27:0/2549903721 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fece40053e0 con 0x55ec7fe95670 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fece0c00640 1 -- 172.21.15.27:0/2549903721 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fece4058050 con 0x55ec7fe95670 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 10 client.28850 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec80278650 con 0x55ec7fe95670 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fece0c00640 1 -- 172.21.15.27:0/2549903721 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece4005030 con 0x55ec7fe95670 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fece0c00640 1 -- 172.21.15.27:0/2549903721 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fece404a4f0 con 0x55ec7fe95670 2024-09-21T07:24:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 10 client.28850 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 10 client.28850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 20 client.28850 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe74cd0 0x55ec8027b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 4 client.28850 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.842+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec801d5de0 con 0x55ec7fe74cd0 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.843+0000 7fecf7400640 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe74cd0 0x55ec8027b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.843+0000 7fecf7400640 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe74cd0 0x55ec8027b9b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f139e6f05bed9c98 server_cookie=4961ff84dc163d51 in_seq=0 out_seq=0 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.843+0000 7fece0c00640 10 client.28850 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.844+0000 7fece0c00640 1 -- 172.21.15.27:0/2549903721 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec801d5de0 con 0x55ec7fe74cd0 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.844+0000 7fece0c00640 10 client.28850 handle_command_reply: tid=0 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.844+0000 7fed05001740 1 client.28850 shutdown 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.844+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 msgr2=0x55ec80269e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.844+0000 7fed05001740 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec80269e20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fece4006010 tx=0x7fece4032dc0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 shutdown_connections 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe74cd0 0x55ec8027b9b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7feb6350 0x55ec802757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec80269e20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 --2- 172.21.15.27:0/2549903721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7fe8d490 0x55ec802698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 >> 172.21.15.27:0/2549903721 conn(0x55ec7feaa740 msgr2=0x55ec80260ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fece0c00640 0 client.28850 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 shutdown_connections 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 1 -- 172.21.15.27:0/2549903721 wait complete. 2024-09-21T07:24:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.845+0000 7fed05001740 20 client.28850 trim_cache size 0 max 16384 2024-09-21T07:24:20.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.863+0000 7fed05001740 1 Processor -- start 2024-09-21T07:24:20.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.863+0000 7fed05001740 1 -- start start 2024-09-21T07:24:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fed05001740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe8d490 0x55ec8025bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fed05001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec80268e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fed05001740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8026b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fed05001740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81b3700 con 0x55ec7feb6350 2024-09-21T07:24:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fed05001740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81b0fd0 con 0x55ec7fe8d490 2024-09-21T07:24:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fed05001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81aee40 con 0x55ec7fe95670 2024-09-21T07:24:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8026b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fecf7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe8d490 0x55ec8025bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec80268e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8026b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57662/0 (socket says 172.21.15.27:57662) 2024-09-21T07:24:20.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.864+0000 7fecf7e00640 1 -- 172.21.15.27:0/201959249 learned_addr learned my addr 172.21.15.27:0/201959249 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fecf6a00640 1 -- 172.21.15.27:0/201959249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe8d490 msgr2=0x55ec8025bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fecf6a00640 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe8d490 0x55ec8025bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fecf6a00640 1 -- 172.21.15.27:0/201959249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 msgr2=0x55ec8026b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fecf6a00640 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8026b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fecf6a00640 1 -- 172.21.15.27:0/201959249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec80257400 con 0x55ec7fe95670 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fecf7e00640 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8026b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fecf6a00640 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec80268e40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fece402a430 tx=0x7fece4009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f8b36af3b54dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fece3400640 1 -- 172.21.15.27:0/201959249 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4035040 con 0x55ec7fe95670 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fece3400640 1 -- 172.21.15.27:0/201959249 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece404acb0 con 0x55ec7fe95670 2024-09-21T07:24:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.865+0000 7fece3400640 1 -- 172.21.15.27:0/201959249 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4005280 con 0x55ec7fe95670 2024-09-21T07:24:20.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 -- 172.21.15.27:0/201959249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 msgr2=0x55ec80268e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec80268e40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fece402a430 tx=0x7fece4009800 comp rx=0 tx=0).stop 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 -- 172.21.15.27:0/201959249 shutdown_connections 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe95670 0x55ec80268e40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8026b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 --2- 172.21.15.27:0/201959249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe8d490 0x55ec8025bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 -- 172.21.15.27:0/201959249 >> 172.21.15.27:0/201959249 conn(0x55ec7feaa740 msgr2=0x55ec8026ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 -- 172.21.15.27:0/201959249 shutdown_connections 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.866+0000 7fed05001740 1 -- 172.21.15.27:0/201959249 wait complete. 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.867+0000 7fed05001740 1 -- start start 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.867+0000 7fed05001740 1 Processor -- start 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fed05001740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe8d490 0x55ec8027f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fed05001740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec8027c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fed05001740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8027cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fed05001740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81af490 con 0x55ec7feb6350 2024-09-21T07:24:20.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fed05001740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81b0900 con 0x55ec7fe95670 2024-09-21T07:24:20.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fed05001740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81afb60 con 0x55ec7fe8d490 2024-09-21T07:24:20.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fed05001740 10 client.0 _ll_register_callbacks cb 0x7fece8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:20.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec8027c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe8d490 0x55ec8027f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8027cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec8027c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41754/0 (socket says 172.21.15.27:41754) 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe8d490 0x55ec8027f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52426/0 (socket says 172.21.15.27:52426) 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 -- 172.21.15.27:0/201932021 learned_addr learned my addr 172.21.15.27:0/201932021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 -- 172.21.15.27:0/201932021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe8d490 msgr2=0x55ec8027f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe8d490 0x55ec8027f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 -- 172.21.15.27:0/201932021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 msgr2=0x55ec8027cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8027cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.868+0000 7fecf6a00640 1 -- 172.21.15.27:0/201932021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8027d360 con 0x55ec7fe95670 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fecf6a00640 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec8027c6b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fece4005d00 tx=0x7fece4031550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc43c02b972684c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fece0c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fece0c00640 1 -- 172.21.15.27:0/201932021 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4061070 con 0x55ec7fe95670 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece404acb0 con 0x55ec7fe95670 2024-09-21T07:24:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fece0c00640 1 -- 172.21.15.27:0/201932021 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fece4031b80 con 0x55ec7fe95670 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fece0c00640 1 -- 172.21.15.27:0/201932021 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fece406a050 con 0x55ec7fe95670 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fed05001740 10 client.28856 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.869+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec802ea480 con 0x55ec7fe95670 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fece0c00640 1 -- 172.21.15.27:0/201932021 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece404acb0 con 0x55ec7fe95670 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fece0c00640 1 -- 172.21.15.27:0/201932021 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fece4064020 con 0x55ec7fe95670 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fed05001740 10 client.28856 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fed05001740 10 client.28856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fed05001740 20 client.28856 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fed05001740 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe4a080 0x55ec802ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fed05001740 4 client.28856 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.870+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ec801d1890 con 0x55ec7fe4a080 2024-09-21T07:24:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.871+0000 7fecf7400640 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe4a080 0x55ec802ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.871+0000 7fecf7400640 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe4a080 0x55ec802ecc70 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6edd08e32773bce server_cookie=8cb41a4355827de2 in_seq=0 out_seq=0 2024-09-21T07:24:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.871+0000 7fece0c00640 10 client.28856 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fece0c00640 1 -- 172.21.15.27:0/201932021 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ec801d1890 con 0x55ec7fe4a080 2024-09-21T07:24:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fece0c00640 10 client.28856 handle_command_reply: tid=0 2024-09-21T07:24:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 client.28856 shutdown 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 msgr2=0x55ec8027c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec8027c6b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fece4005d00 tx=0x7fece4031550 comp rx=0 tx=0).stop 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 shutdown_connections 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ec7fe4a080 0x55ec802ecc70 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec7feb6350 0x55ec8027cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec7fe95670 0x55ec8027c6b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 --2- 172.21.15.27:0/201932021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec7fe8d490 0x55ec8027f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fece0c00640 0 client.28856 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 >> 172.21.15.27:0/201932021 conn(0x55ec7feaa740 msgr2=0x55ec8025ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 shutdown_connections 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 1 -- 172.21.15.27:0/201932021 wait complete. 2024-09-21T07:24:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.873+0000 7fed05001740 20 client.28856 trim_cache size 0 max 16384 2024-09-21T07:24:20.877 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:24:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:24:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "9a0044a9-e2bc-476f-884e-4ed21d019901", 2024-09-21T07:24:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:24:20.877 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecdc0b6700 msgr2=0x7fecdc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecdc0b6700 0x7fecdc0b8c10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55ec8025c990 tx=0x7fece8005f90 comp rx=0 tx=0).stop 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf8109df0 msgr2=0x7fecf819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf8109df0 0x7fecf819fd20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fecf000efd0 tx=0x7fecf000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 shutdown_connections 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecdc0b6700 0x7fecdc0b8c10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf8109df0 0x7fecf819fd20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf81094a0 0x7fecf819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 --2- 172.21.15.27:0/1764476226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf8103b60 0x7fecf819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.874+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 >> 172.21.15.27:0/1764476226 conn(0x7fecf807b850 msgr2=0x7fecf8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.875+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 shutdown_connections 2024-09-21T07:24:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:20.875+0000 7fecfe200640 1 -- 172.21.15.27:0/1764476226 wait complete. 2024-09-21T07:24:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:20 smithi119 ceph-mon[34519]: pgmap v729: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:24:21.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:20 smithi027 ceph-mon[30779]: pgmap v729: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:24:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:20 smithi153 ceph-mon[34747]: pgmap v729: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:24:21.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:24:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:22 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:24:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:22 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:22 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:24:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:22 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:22.337 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:22 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:24:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:22 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:22 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:24:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:22 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:22 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:24:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:22 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:22 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:24:22.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:22 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:24:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:23 smithi027 ceph-mon[30779]: pgmap v730: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:24:23.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 -- 172.21.15.27:0/3426019422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410c040 msgr2=0x7f53a4112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 --2- 172.21.15.27:0/3426019422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410c040 0x7f53a4112870 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f539c009ca0 tx=0x7f539c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 -- 172.21.15.27:0/3426019422 shutdown_connections 2024-09-21T07:24:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 --2- 172.21.15.27:0/3426019422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a410b6f0 0x7f53a410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 --2- 172.21.15.27:0/3426019422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410c040 0x7f53a4112870 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 --2- 172.21.15.27:0/3426019422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4105db0 0x7f53a41061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 -- 172.21.15.27:0/3426019422 >> 172.21.15.27:0/3426019422 conn(0x7f53a41017e0 msgr2=0x7f53a4103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 -- 172.21.15.27:0/3426019422 shutdown_connections 2024-09-21T07:24:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.123+0000 7f53ab600640 1 -- 172.21.15.27:0/3426019422 wait complete. 2024-09-21T07:24:23.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.124+0000 7f53ab600640 1 Processor -- start 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.124+0000 7f53ab600640 1 -- start start 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53ab600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4105db0 0x7f53a419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53ab600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410b6f0 0x7f53a419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53ab600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a410c040 0x7f53a41a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53ab600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4115d00 con 0x7f53a410b6f0 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53ab600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4115b80 con 0x7f53a4105db0 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53ab600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4115e80 con 0x7f53a410c040 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410b6f0 0x7f53a419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a8c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4105db0 0x7f53a419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410b6f0 0x7f53a419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57690/0 (socket says 172.21.15.27:57690) 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a3e00640 1 -- 172.21.15.27:0/395846137 learned_addr learned my addr 172.21.15.27:0/395846137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a9600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a410c040 0x7f53a41a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a8c00640 1 -- 172.21.15.27:0/395846137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a410c040 msgr2=0x7f53a41a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a8c00640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a410c040 0x7f53a41a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a8c00640 1 -- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410b6f0 msgr2=0x7f53a419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a8c00640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410b6f0 0x7f53a419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a8c00640 1 -- 172.21.15.27:0/395846137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a41a2620 con 0x7f53a4105db0 2024-09-21T07:24:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.125+0000 7f53a9600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a410c040 0x7f53a41a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.126+0000 7f53a8c00640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4105db0 0x7f53a419c6b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9f0 tx=0x7f539400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.126+0000 7f53a1600640 1 -- 172.21.15.27:0/395846137 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539400ccc0 con 0x7f53a4105db0 2024-09-21T07:24:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.126+0000 7f53a1600640 1 -- 172.21.15.27:0/395846137 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940108e0 con 0x7f53a4105db0 2024-09-21T07:24:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.126+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a41a2910 con 0x7f53a4105db0 2024-09-21T07:24:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.126+0000 7f53a1600640 1 -- 172.21.15.27:0/395846137 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394018880 con 0x7f53a4105db0 2024-09-21T07:24:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.126+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a41a9070 con 0x7f53a4105db0 2024-09-21T07:24:23.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.129+0000 7f53b2266740 1 Processor -- start 2024-09-21T07:24:23.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53b2266740 1 -- start start 2024-09-21T07:24:23.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53b2266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84dd070 0x5567c8883b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53b2266740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84b41b0 0x5567c88840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53b2266740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c88845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53b2266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5567c87faf70 con 0x5567c84dd070 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53b2266740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5567c87ea3a0 con 0x5567c84bc390 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53b2266740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5567c87ee1b0 con 0x5567c84b41b0 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53a1600640 1 -- 172.21.15.27:0/395846137 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5394010430 con 0x7f53a4105db0 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53a9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84dd070 0x5567c8883b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53a3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c88845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53a9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84dd070 0x5567c8883b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57694/0 (socket says 172.21.15.27:57694) 2024-09-21T07:24:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.130+0000 7f53a9600640 1 -- 172.21.15.27:0/2359353675 learned_addr learned my addr 172.21.15.27:0/2359353675 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.131+0000 7f53a8c00640 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84b41b0 0x5567c88840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.131+0000 7f53a3e00640 1 -- 172.21.15.27:0/2359353675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84b41b0 msgr2=0x5567c88840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.131+0000 7f53a3e00640 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84b41b0 0x5567c88840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.131+0000 7f53a3e00640 1 -- 172.21.15.27:0/2359353675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84dd070 msgr2=0x5567c8883b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.131+0000 7f53a3e00640 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84dd070 0x5567c8883b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.131+0000 7f53a3e00640 1 -- 172.21.15.27:0/2359353675 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c87b4c90 con 0x5567c84bc390 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.131+0000 7f53a3e00640 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c88845e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f53a4108080 tx=0x7f5390011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7bb02dbce88313f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f5399c00640 1 -- 172.21.15.27:0/2359353675 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539001e070 con 0x5567c84bc390 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f5399c00640 1 -- 172.21.15.27:0/2359353675 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53900198e0 con 0x5567c84bc390 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f5399c00640 1 -- 172.21.15.27:0/2359353675 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539000e9b0 con 0x5567c84bc390 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f53a1600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53880b6700 0x7f53880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f53a1600640 1 -- 172.21.15.27:0/395846137 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f53940e7ea0 con 0x7f53a4105db0 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f53a9600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53880b6700 0x7f53880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f53b2266740 1 -- 172.21.15.27:0/2359353675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 msgr2=0x5567c88845e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.132+0000 7f53b2266740 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c88845e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f53a4108080 tx=0x7f5390011810 comp rx=0 tx=0).stop 2024-09-21T07:24:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53b2266740 1 -- 172.21.15.27:0/2359353675 shutdown_connections 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53b2266740 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84b41b0 0x5567c88840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53b2266740 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c88845e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53b2266740 1 --2- 172.21.15.27:0/2359353675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84dd070 0x5567c8883b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53b2266740 1 -- 172.21.15.27:0/2359353675 >> 172.21.15.27:0/2359353675 conn(0x5567c84d1460 msgr2=0x5567c8881a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53b2266740 1 -- 172.21.15.27:0/2359353675 shutdown_connections 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53b2266740 1 -- 172.21.15.27:0/2359353675 wait complete. 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.133+0000 7f53a9600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53880b6700 0x7f53880b8c10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f539c004af0 tx=0x7f539c005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 -- start start 2024-09-21T07:24:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 Processor -- start 2024-09-21T07:24:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c888f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c888fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c889b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5567c864b700 con 0x5567c84b41b0 2024-09-21T07:24:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5567c8785540 con 0x5567c84bc390 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5567c8892340 con 0x5567c84dd070 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.134+0000 7f53b2266740 10 client.0 _ll_register_callbacks cb 0x5567c8795430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c888f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c888f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57700/0 (socket says 172.21.15.27:57700) 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 -- 172.21.15.27:0/937013432 learned_addr learned my addr 172.21.15.27:0/937013432 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a3e00640 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c888fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a9600640 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c889b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 -- 172.21.15.27:0/937013432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 msgr2=0x5567c889b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c889b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 -- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 msgr2=0x5567c888fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c888fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.135+0000 7f53a8c00640 1 -- 172.21.15.27:0/937013432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c8548650 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f53a8c00640 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c888f770 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f53940e5240 tx=0x7f53940ad230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b27b03448a6063 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f538e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f538e000640 1 -- 172.21.15.27:0/937013432 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940e5670 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f538e000640 1 -- 172.21.15.27:0/937013432 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f53940a9430 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567c8787b50 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f538e000640 1 -- 172.21.15.27:0/937013432 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5394010720 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f53b2266740 10 client.18852 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.136+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567c87a2910 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f538e000640 1 -- 172.21.15.27:0/937013432 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5567c8787b50 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f538e000640 1 -- 172.21.15.27:0/937013432 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f53940b3550 con 0x5567c84b41b0 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f53b2266740 10 client.18852 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f53b2266740 10 client.18852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f53b2266740 20 client.18852 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f53b2266740 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c849b9f0 0x5567c88a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f53b2266740 4 client.18852 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c87fcec0 con 0x5567c849b9f0 2024-09-21T07:24:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.137+0000 7f53a3e00640 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c849b9f0 0x5567c88a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.138+0000 7f53a3e00640 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c849b9f0 0x5567c88a1660 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c407be68a0afdd46 server_cookie=e0cb6da793ee21a3 in_seq=0 out_seq=0 2024-09-21T07:24:23.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.138+0000 7f538e000640 10 client.18852 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:23.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f538e000640 1 -- 172.21.15.27:0/937013432 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567c87fcec0 con 0x5567c849b9f0 2024-09-21T07:24:23.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f538e000640 10 client.18852 handle_command_reply: tid=0 2024-09-21T07:24:23.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 client.18852 shutdown 2024-09-21T07:24:23.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 msgr2=0x5567c888f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c888f770 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f53940e5240 tx=0x7f53940ad230 comp rx=0 tx=0).stop 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 shutdown_connections 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c849b9f0 0x5567c88a1660 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c889b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84bc390 0x5567c888fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 --2- 172.21.15.27:0/937013432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c888f770 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 >> 172.21.15.27:0/937013432 conn(0x5567c84d1460 msgr2=0x5567c8886d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f538e000640 0 client.18852 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 shutdown_connections 2024-09-21T07:24:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.139+0000 7f53b2266740 1 -- 172.21.15.27:0/937013432 wait complete. 2024-09-21T07:24:23.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.140+0000 7f53b2266740 20 client.18852 trim_cache size 0 max 16384 2024-09-21T07:24:23.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.158+0000 7f53b2266740 1 Processor -- start 2024-09-21T07:24:23.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.158+0000 7f53b2266740 1 -- start start 2024-09-21T07:24:23.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53b2266740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84b41b0 0x5567c8888c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53b2266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84bc390 0x5567c88891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53b2266740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c8881000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53b2266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41b5a10 con 0x5567c84bc390 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53b2266740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41b32e0 con 0x5567c84b41b0 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53b2266740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41b1150 con 0x5567c84dd070 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84bc390 0x5567c88891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84bc390 0x5567c88891b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57706/0 (socket says 172.21.15.27:57706) 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a3e00640 1 -- 172.21.15.27:0/2980990892 learned_addr learned my addr 172.21.15.27:0/2980990892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a8c00640 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84b41b0 0x5567c8888c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a9600640 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c8881000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a8c00640 1 -- 172.21.15.27:0/2980990892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 msgr2=0x5567c8881000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a8c00640 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c8881000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a8c00640 1 -- 172.21.15.27:0/2980990892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84bc390 msgr2=0x5567c88891b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a8c00640 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84bc390 0x5567c88891b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a8c00640 1 -- 172.21.15.27:0/2980990892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c8885e50 con 0x5567c84b41b0 2024-09-21T07:24:23.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.159+0000 7f53a9600640 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c8881000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:23.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.160+0000 7f53a8c00640 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84b41b0 0x5567c8888c70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f53940b3a90 tx=0x7f53940a9050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a716937059fba6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.160+0000 7f5399200640 1 -- 172.21.15.27:0/2980990892 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940ad3f0 con 0x5567c84b41b0 2024-09-21T07:24:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.160+0000 7f5399200640 1 -- 172.21.15.27:0/2980990892 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940adfe0 con 0x5567c84b41b0 2024-09-21T07:24:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.160+0000 7f5399200640 1 -- 172.21.15.27:0/2980990892 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940b6840 con 0x5567c84b41b0 2024-09-21T07:24:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 -- 172.21.15.27:0/2980990892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84b41b0 msgr2=0x5567c8888c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84b41b0 0x5567c8888c70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f53940b3a90 tx=0x7f53940a9050 comp rx=0 tx=0).stop 2024-09-21T07:24:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 -- 172.21.15.27:0/2980990892 shutdown_connections 2024-09-21T07:24:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84dd070 0x5567c8881000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84bc390 0x5567c88891b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 --2- 172.21.15.27:0/2980990892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84b41b0 0x5567c8888c70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 -- 172.21.15.27:0/2980990892 >> 172.21.15.27:0/2980990892 conn(0x5567c84d1460 msgr2=0x5567c888ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 -- 172.21.15.27:0/2980990892 shutdown_connections 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.161+0000 7f53b2266740 1 -- 172.21.15.27:0/2980990892 wait complete. 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.162+0000 7f53b2266740 1 -- start start 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.162+0000 7f53b2266740 1 Processor -- start 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.162+0000 7f53b2266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c8897c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.162+0000 7f53b2266740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84bc390 0x5567c88930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.162+0000 7f53b2266740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84dd070 0x5567c88935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53b2266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41b17a0 con 0x5567c84b41b0 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53b2266740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41b2c10 con 0x5567c84dd070 2024-09-21T07:24:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53b2266740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41b1e70 con 0x5567c84bc390 2024-09-21T07:24:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53b2266740 10 client.0 _ll_register_callbacks cb 0x7f53940a9870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c8897c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c8897c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57716/0 (socket says 172.21.15.27:57716) 2024-09-21T07:24:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 -- 172.21.15.27:0/307741991 learned_addr learned my addr 172.21.15.27:0/307741991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a3e00640 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84bc390 0x5567c88930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a9600640 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84dd070 0x5567c88935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 -- 172.21.15.27:0/307741991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84bc390 msgr2=0x5567c88930b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84bc390 0x5567c88930b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 -- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84dd070 msgr2=0x5567c88935f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84dd070 0x5567c88935f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.163+0000 7f53a8c00640 1 -- 172.21.15.27:0/307741991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c8893be0 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f53a8c00640 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c8897c00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f53940a9840 tx=0x7f53940ad360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c81dd8391e6ae2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f538e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f538e000640 1 -- 172.21.15.27:0/307741991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940ad3f0 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f538e000640 1 -- 172.21.15.27:0/307741991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f53940b6840 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53940adfe0 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f538e000640 1 -- 172.21.15.27:0/307741991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53940b23c0 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f53b2266740 10 client.18858 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567c890bdd0 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.164+0000 7f538e000640 1 -- 172.21.15.27:0/307741991 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940adfe0 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f538e000640 1 -- 172.21.15.27:0/307741991 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f53940ea020 con 0x5567c84b41b0 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f53b2266740 10 client.18858 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f53b2266740 10 client.18858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f53b2266740 20 client.18858 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f53b2266740 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c8470da0 0x5567c890e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f53b2266740 4 client.18858 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567c87f8970 con 0x5567c8470da0 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.165+0000 7f53a3e00640 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c8470da0 0x5567c890e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53a3e00640 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c8470da0 0x5567c890e130 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca171429ba007047 server_cookie=6aa65849f3451908 in_seq=0 out_seq=0 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f538e000640 10 client.18858 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f538e000640 1 -- 172.21.15.27:0/307741991 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x5567c87f8970 con 0x5567c8470da0 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f538e000640 10 client.18858 handle_command_reply: tid=0 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 client.18858 shutdown 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 msgr2=0x5567c8897c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c8897c00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f53940a9840 tx=0x7f53940ad360 comp rx=0 tx=0).stop 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 shutdown_connections 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5567c8470da0 0x5567c890e130 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567c84bc390 0x5567c88930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567c84dd070 0x5567c88935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 --2- 172.21.15.27:0/307741991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567c84b41b0 0x5567c8897c00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.166+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 >> 172.21.15.27:0/307741991 conn(0x5567c84d1460 msgr2=0x5567c8888c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.167+0000 7f538e000640 0 client.18858 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.167+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 shutdown_connections 2024-09-21T07:24:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.167+0000 7f53b2266740 1 -- 172.21.15.27:0/307741991 wait complete. 2024-09-21T07:24:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.167+0000 7f53b2266740 20 client.18858 trim_cache size 0 max 16384 2024-09-21T07:24:23.172 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "9a0044a9-e2bc-476f-884e-4ed21d019901", 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.173 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.174 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:24:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.179 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:24:23.180 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:24:23.181 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:23.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.167+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53880b6700 msgr2=0x7f53880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.167+0000 7f53ab600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53880b6700 0x7f53880b8c10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f539c004af0 tx=0x7f539c005c30 comp rx=0 tx=0).stop 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4105db0 msgr2=0x7f53a419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4105db0 0x7f53a419c6b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9f0 tx=0x7f539400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 shutdown_connections 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53a410c040 0x7f53a41a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53880b6700 0x7f53880b8c10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a410b6f0 0x7f53a419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 --2- 172.21.15.27:0/395846137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4105db0 0x7f53a419c6b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:23.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 >> 172.21.15.27:0/395846137 conn(0x7f53a41017e0 msgr2=0x7f53a410ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:23.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 shutdown_connections 2024-09-21T07:24:23.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:23.168+0000 7f53ab600640 1 -- 172.21.15.27:0/395846137 wait complete. 2024-09-21T07:24:23.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:23 smithi119 ceph-mon[34519]: pgmap v730: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:24:23.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:23 smithi153 ceph-mon[34747]: pgmap v730: 129 pgs: 129 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:24:24.031 INFO:tasks.cephfs.filesystem:scrub status for tag:9a0044a9-e2bc-476f-884e-4ed21d019901 - {'path': '/', 'tag': '9a0044a9-e2bc-476f-884e-4ed21d019901', 'options': 'recursive,force'} 2024-09-21T07:24:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:24 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:24:24.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:24 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:24:24.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:24 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:24:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:25 smithi119 ceph-mon[34519]: pgmap v731: 129 pgs: 129 active+clean; 4.1 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-09-21T07:24:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:25 smithi027 ceph-mon[30779]: pgmap v731: 129 pgs: 129 active+clean; 4.1 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-09-21T07:24:25.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:25 smithi153 ceph-mon[34747]: pgmap v731: 129 pgs: 129 active+clean; 4.1 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-09-21T07:24:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:27 smithi119 ceph-mon[34519]: pgmap v732: 129 pgs: 129 active+clean; 4.1 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:24:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:27 smithi027 ceph-mon[30779]: pgmap v732: 129 pgs: 129 active+clean; 4.1 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:24:27.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:27 smithi153 ceph-mon[34747]: pgmap v732: 129 pgs: 129 active+clean; 4.1 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.884+0000 7fe5ad800640 1 -- 172.21.15.27:0/3617345758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a810b610 msgr2=0x7fe5a810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.884+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3617345758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a810b610 0x7fe5a810ba20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe594009860 tx=0x7fe59402a340 comp rx=0 tx=0).stop 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.885+0000 7fe5ad800640 1 -- 172.21.15.27:0/3617345758 shutdown_connections 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.885+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3617345758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a810bf60 0x7fe5a8112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.885+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3617345758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a810b610 0x7fe5a810ba20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.885+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3617345758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a8105ce0 0x7fe5a81060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.885+0000 7fe5ad800640 1 -- 172.21.15.27:0/3617345758 >> 172.21.15.27:0/3617345758 conn(0x7fe5a81016e0 msgr2=0x7fe5a8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.885+0000 7fe5ad800640 1 -- 172.21.15.27:0/3617345758 shutdown_connections 2024-09-21T07:24:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.885+0000 7fe5ad800640 1 -- 172.21.15.27:0/3617345758 wait complete. 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 Processor -- start 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 -- start start 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a8105ce0 0x7fe5a819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a810b610 0x7fe5a819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a810bf60 0x7fe5a81a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8115c20 con 0x7fe5a810b610 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8115aa0 con 0x7fe5a8105ce0 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5ad800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8115da0 con 0x7fe5a810bf60 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.886+0000 7fe5a6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a8105ce0 0x7fe5a819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a810b610 0x7fe5a819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a8105ce0 0x7fe5a819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41840/0 (socket says 172.21.15.27:41840) 2024-09-21T07:24:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 -- 172.21.15.27:0/3549292578 learned_addr learned my addr 172.21.15.27:0/3549292578 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a7400640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a810bf60 0x7fe5a81a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 -- 172.21.15.27:0/3549292578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a810bf60 msgr2=0x7fe5a81a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a810bf60 0x7fe5a81a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 -- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a810b610 msgr2=0x7fe5a819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a810b610 0x7fe5a819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 -- 172.21.15.27:0/3549292578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81a25d0 con 0x7fe5a8105ce0 2024-09-21T07:24:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.887+0000 7fe5a6a00640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a8105ce0 0x7fe5a819c6a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.888+0000 7fe59f400640 1 -- 172.21.15.27:0/3549292578 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59800cd80 con 0x7fe5a8105ce0 2024-09-21T07:24:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.888+0000 7fe59f400640 1 -- 172.21.15.27:0/3549292578 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5980054d0 con 0x7fe5a8105ce0 2024-09-21T07:24:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.888+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81a28c0 con 0x7fe5a8105ce0 2024-09-21T07:24:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.888+0000 7fe59f400640 1 -- 172.21.15.27:0/3549292578 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598010690 con 0x7fe5a8105ce0 2024-09-21T07:24:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.888+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81a9050 con 0x7fe5a8105ce0 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 Processor -- start 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 -- start start 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846166c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 0x55a846167170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a8461676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a8460dc6b0 con 0x55a845dc21b0 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a8460cbae0 con 0x55a845d992f0 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8460cf8f0 con 0x55a845da14d0 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846166c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.891+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846166c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57740/0 (socket says 172.21.15.27:57740) 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a7400640 1 -- 172.21.15.27:0/1142006784 learned_addr learned my addr 172.21.15.27:0/1142006784 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a6000640 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a8461676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe59f400640 1 -- 172.21.15.27:0/3549292578 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe598004e90 con 0x7fe5a8105ce0 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a7400640 1 -- 172.21.15.27:0/1142006784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 msgr2=0x55a8461676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a8461676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a7400640 1 -- 172.21.15.27:0/1142006784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 msgr2=0x55a846167170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 0x55a846167170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a7400640 1 -- 172.21.15.27:0/1142006784 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a846098780 con 0x55a845dc21b0 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.892+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846166c30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000e450 tx=0x7fe5a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18df600bfeb842a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe593e00640 1 -- 172.21.15.27:0/1142006784 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000d910 con 0x55a845dc21b0 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe593e00640 1 -- 172.21.15.27:0/1142006784 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a001c070 con 0x55a845dc21b0 2024-09-21T07:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe593e00640 1 -- 172.21.15.27:0/1142006784 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0017de0 con 0x55a845dc21b0 2024-09-21T07:24:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe59f400640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe58c0b6700 0x7fe58c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1142006784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 msgr2=0x55a846166c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846166c30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000e450 tx=0x7fe5a00095c0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1142006784 shutdown_connections 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a8461676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 0x55a846167170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1142006784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846166c30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.893+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1142006784 >> 172.21.15.27:0/1142006784 conn(0x55a845db6300 msgr2=0x55a846164b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.894+0000 7fe59f400640 1 -- 172.21.15.27:0/3549292578 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fe5980e7e60 con 0x7fe5a8105ce0 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.894+0000 7fe5a6a00640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe58c0b6700 0x7fe58c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.894+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1142006784 shutdown_connections 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.894+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1142006784 wait complete. 2024-09-21T07:24:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.894+0000 7fe5b49c8740 1 -- start start 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.894+0000 7fe5a6a00640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe58c0b6700 0x7fe58c0b8c10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55a8461658b0 tx=0x7fe598004990 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 1 Processor -- start 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 0x55a8461721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a846172730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846172c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a846003ab0 con 0x55a845dc21b0 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a846072570 con 0x55a845d992f0 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8461753b0 con 0x55a845da14d0 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5b49c8740 10 client.0 _ll_register_callbacks cb 0x55a846081100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846172c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.895+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846172c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57750/0 (socket says 172.21.15.27:57750) 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 -- 172.21.15.27:0/3520056163 learned_addr learned my addr 172.21.15.27:0/3520056163 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a6000640 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a846172730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 0x55a8461721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 -- 172.21.15.27:0/3520056163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 msgr2=0x55a846172730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a846172730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 -- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 msgr2=0x55a8461721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 0x55a8461721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 -- 172.21.15.27:0/3520056163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a845e2d790 con 0x55a845dc21b0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe5a7400640 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846172c70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0017b20 tx=0x7fe5a0017a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c55a357e38deb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.896+0000 7fe590c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe590c00640 1 -- 172.21.15.27:0/3520056163 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003970 con 0x55a845dc21b0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe590c00640 1 -- 172.21.15.27:0/3520056163 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fe5a0029030 con 0x55a845dc21b0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a001c070 con 0x55a845dc21b0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe590c00640 1 -- 172.21.15.27:0/3520056163 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5a0036050 con 0x55a845dc21b0 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 10 client.18870 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a845ea6620 con 0x55a845dc21b0 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe590c00640 1 -- 172.21.15.27:0/3520056163 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a001c070 con 0x55a845dc21b0 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe590c00640 1 -- 172.21.15.27:0/3520056163 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe5a000b040 con 0x55a845dc21b0 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 10 client.18870 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 10 client.18870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 20 client.18870 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d80d80 0x55a846184310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 4 client.18870 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.897+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8460de600 con 0x55a845d80d80 2024-09-21T07:24:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.898+0000 7fe5a6000640 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d80d80 0x55a846184310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.898+0000 7fe5a6000640 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d80d80 0x55a846184310 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4b6bab31e9527b server_cookie=ad7dd3318975684 in_seq=0 out_seq=0 2024-09-21T07:24:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.898+0000 7fe590c00640 10 client.18870 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe590c00640 1 -- 172.21.15.27:0/3520056163 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8460de600 con 0x55a845d80d80 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe590c00640 10 client.18870 handle_command_reply: tid=0 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 client.18870 shutdown 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 msgr2=0x55a846172c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846172c70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0017b20 tx=0x7fe5a0017a20 comp rx=0 tx=0).stop 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 shutdown_connections 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d80d80 0x55a846184310 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845da14d0 0x55a846172730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845dc21b0 0x55a846172c70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/3520056163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845d992f0 0x55a8461721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe590c00640 0 client.18870 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.899+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 >> 172.21.15.27:0/3520056163 conn(0x55a845db6300 msgr2=0x55a846167c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.900+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 shutdown_connections 2024-09-21T07:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.900+0000 7fe5b49c8740 1 -- 172.21.15.27:0/3520056163 wait complete. 2024-09-21T07:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.900+0000 7fe5b49c8740 20 client.18870 trim_cache size 0 max 16384 2024-09-21T07:24:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.915+0000 7fe5b49c8740 1 Processor -- start 2024-09-21T07:24:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.915+0000 7fe5b49c8740 1 -- start start 2024-09-21T07:24:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845d992f0 0x55a846168370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845da14d0 0x55a84616f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845dc21b0 0x55a84616f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b59a0 con 0x55a845d992f0 2024-09-21T07:24:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b3270 con 0x55a845da14d0 2024-09-21T07:24:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b10e0 con 0x55a845dc21b0 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845d992f0 0x55a846168370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845d992f0 0x55a846168370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57766/0 (socket says 172.21.15.27:57766) 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 -- 172.21.15.27:0/4126199647 learned_addr learned my addr 172.21.15.27:0/4126199647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a6000640 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845da14d0 0x55a84616f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845dc21b0 0x55a84616f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 -- 172.21.15.27:0/4126199647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845dc21b0 msgr2=0x55a84616f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845dc21b0 0x55a84616f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 -- 172.21.15.27:0/4126199647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845da14d0 msgr2=0x55a84616f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845da14d0 0x55a84616f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a7400640 1 -- 172.21.15.27:0/4126199647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a846173070 con 0x55a845d992f0 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.916+0000 7fe5a6000640 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845da14d0 0x55a84616f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.917+0000 7fe5a7400640 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845d992f0 0x55a846168370 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009fd0 tx=0x7fe5a0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c166d32989c5f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.917+0000 7fe593400640 1 -- 172.21.15.27:0/4126199647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000eda0 con 0x55a845d992f0 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.917+0000 7fe593400640 1 -- 172.21.15.27:0/4126199647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a001c040 con 0x55a845d992f0 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.917+0000 7fe593400640 1 -- 172.21.15.27:0/4126199647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003e80 con 0x55a845d992f0 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 -- 172.21.15.27:0/4126199647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845d992f0 msgr2=0x55a846168370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845d992f0 0x55a846168370 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009fd0 tx=0x7fe5a0000f40 comp rx=0 tx=0).stop 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 -- 172.21.15.27:0/4126199647 shutdown_connections 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845dc21b0 0x55a84616f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845da14d0 0x55a84616f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/4126199647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845d992f0 0x55a846168370 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 -- 172.21.15.27:0/4126199647 >> 172.21.15.27:0/4126199647 conn(0x55a845db6300 msgr2=0x55a846167c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 -- 172.21.15.27:0/4126199647 shutdown_connections 2024-09-21T07:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 -- 172.21.15.27:0/4126199647 wait complete. 2024-09-21T07:24:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.918+0000 7fe5b49c8740 1 -- start start 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 1 Processor -- start 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845d992f0 0x55a84617acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845da14d0 0x55a846178170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845dc21b0 0x55a8461786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b1730 con 0x55a845da14d0 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b2ba0 con 0x55a845dc21b0 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b1e00 con 0x55a845d992f0 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5b49c8740 10 client.0 _ll_register_callbacks cb 0x7fe5a0009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845da14d0 0x55a846178170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845da14d0 0x55a846178170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57772/0 (socket says 172.21.15.27:57772) 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5a6000640 1 -- 172.21.15.27:0/1337414625 learned_addr learned my addr 172.21.15.27:0/1337414625 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.919+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845d992f0 0x55a84617acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845dc21b0 0x55a8461786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a6000640 1 -- 172.21.15.27:0/1337414625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845d992f0 msgr2=0x55a84617acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a6000640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845d992f0 0x55a84617acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a6000640 1 -- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845dc21b0 msgr2=0x55a8461786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a6000640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845dc21b0 0x55a8461786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a6000640 1 -- 172.21.15.27:0/1337414625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a846178cf0 con 0x55a845da14d0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845dc21b0 0x55a8461786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5a6000640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845da14d0 0x55a846178170 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe5a806c9e0 tx=0x7fe594030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21b2532ffe53156 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe590c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe590c00640 1 -- 172.21.15.27:0/1337414625 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594037040 con 0x55a845da14d0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a001c040 con 0x55a845da14d0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.920+0000 7fe590c00640 1 -- 172.21.15.27:0/1337414625 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fe594004300 con 0x55a845da14d0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe590c00640 1 -- 172.21.15.27:0/1337414625 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe594056050 con 0x55a845da14d0 2024-09-21T07:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 10 client.18876 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a8461ee9c0 con 0x55a845da14d0 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe590c00640 1 -- 172.21.15.27:0/1337414625 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a001c040 con 0x55a845da14d0 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe590c00640 1 -- 172.21.15.27:0/1337414625 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe594051030 con 0x55a845da14d0 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 10 client.18876 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 10 client.18876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 20 client.18876 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d561c0 0x55a8461f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 4 client.18876 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.921+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8460da0b0 con 0x55a845d561c0 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.922+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d561c0 0x55a8461f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.922+0000 7fe5a7400640 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d561c0 0x55a8461f1200 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec72d1f51aa4d47e server_cookie=aa594b556f5f4324 in_seq=0 out_seq=0 2024-09-21T07:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.922+0000 7fe590c00640 10 client.18876 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe590c00640 1 -- 172.21.15.27:0/1337414625 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8460da0b0 con 0x55a845d561c0 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe590c00640 10 client.18876 handle_command_reply: tid=0 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 client.18876 shutdown 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845da14d0 msgr2=0x55a846178170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845da14d0 0x55a846178170 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe5a806c9e0 tx=0x7fe594030ec0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 shutdown_connections 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a845d561c0 0x55a8461f1200 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a845dc21b0 0x55a8461786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a845da14d0 0x55a846178170 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 --2- 172.21.15.27:0/1337414625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a845d992f0 0x55a84617acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe590c00640 0 client.18876 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.937+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 >> 172.21.15.27:0/1337414625 conn(0x55a845db6300 msgr2=0x55a846164ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.938+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 shutdown_connections 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.938+0000 7fe5b49c8740 1 -- 172.21.15.27:0/1337414625 wait complete. 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.938+0000 7fe5b49c8740 20 client.18876 trim_cache size 0 max 16384 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:24:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:24:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.938+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe58c0b6700 msgr2=0x7fe58c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.938+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe58c0b6700 0x7fe58c0b8c10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55a8461658b0 tx=0x7fe598004990 comp rx=0 tx=0).stop 2024-09-21T07:24:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.938+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a8105ce0 msgr2=0x7fe5a819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.938+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a8105ce0 0x7fe5a819c6a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 shutdown_connections 2024-09-21T07:24:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a810bf60 0x7fe5a81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe58c0b6700 0x7fe58c0b8c10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a810b610 0x7fe5a819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 --2- 172.21.15.27:0/3549292578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a8105ce0 0x7fe5a819c6a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 >> 172.21.15.27:0/3549292578 conn(0x7fe5a81016e0 msgr2=0x7fe5a8108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 shutdown_connections 2024-09-21T07:24:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:27.939+0000 7fe5ad800640 1 -- 172.21.15.27:0/3549292578 wait complete. 2024-09-21T07:24:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:29 smithi119 ceph-mon[34519]: pgmap v733: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:24:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:29 smithi027 ceph-mon[30779]: pgmap v733: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:24:29.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:29 smithi153 ceph-mon[34747]: pgmap v733: 129 pgs: 129 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:24:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:31 smithi119 ceph-mon[34519]: pgmap v734: 129 pgs: 129 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 17 op/s 2024-09-21T07:24:31.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:31 smithi027 ceph-mon[30779]: pgmap v734: 129 pgs: 129 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 17 op/s 2024-09-21T07:24:31.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:31 smithi153 ceph-mon[34747]: pgmap v734: 129 pgs: 129 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 17 op/s 2024-09-21T07:24:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:33 smithi119 ceph-mon[34519]: pgmap v735: 129 pgs: 129 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:24:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:33 smithi027 ceph-mon[30779]: pgmap v735: 129 pgs: 129 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:24:33.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:33 smithi153 ceph-mon[34747]: pgmap v735: 129 pgs: 129 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:24:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:35 smithi027 ceph-mon[30779]: pgmap v736: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:24:35.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:35 smithi153 ceph-mon[34747]: pgmap v736: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:24:35.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:35 smithi119 ceph-mon[34519]: pgmap v736: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-09-21T07:24:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:37 smithi027 ceph-mon[30779]: pgmap v737: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:24:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:37 smithi153 ceph-mon[34747]: pgmap v737: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:24:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:37 smithi119 ceph-mon[34519]: pgmap v737: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-09-21T07:24:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:39 smithi027 ceph-mon[30779]: pgmap v738: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-09-21T07:24:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:39 smithi153 ceph-mon[34747]: pgmap v738: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-09-21T07:24:39.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:39 smithi119 ceph-mon[34519]: pgmap v738: 129 pgs: 129 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 12 op/s 2024-09-21T07:24:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 -- 172.21.15.27:0/1074085003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109d80 msgr2=0x7fcff01105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 --2- 172.21.15.27:0/1074085003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109d80 0x7fcff01105b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009900 tx=0x7fcfe802a410 comp rx=0 tx=0).stop 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 -- 172.21.15.27:0/1074085003 shutdown_connections 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 --2- 172.21.15.27:0/1074085003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff0109430 0x7fcff0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 --2- 172.21.15.27:0/1074085003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109d80 0x7fcff01105b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 --2- 172.21.15.27:0/1074085003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0103b00 0x7fcff0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 -- 172.21.15.27:0/1074085003 >> 172.21.15.27:0/1074085003 conn(0x7fcff00ff500 msgr2=0x7fcff0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 -- 172.21.15.27:0/1074085003 shutdown_connections 2024-09-21T07:24:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.143+0000 7fcff6000640 1 -- 172.21.15.27:0/1074085003 wait complete. 2024-09-21T07:24:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.144+0000 7fcff6000640 1 Processor -- start 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcff6000640 1 -- start start 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcff6000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff0103b00 0x7fcff019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcff6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109430 0x7fcff019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcff6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0109d80 0x7fcff019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcff6000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0113a90 con 0x7fcff0109430 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcff6000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0113910 con 0x7fcff0109d80 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcff6000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0113c10 con 0x7fcff0103b00 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109430 0x7fcff019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109430 0x7fcff019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52530/0 (socket says 172.21.15.27:52530) 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcfe6a00640 1 -- 172.21.15.27:0/2578301100 learned_addr learned my addr 172.21.15.27:0/2578301100 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.145+0000 7fcfef400640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff0103b00 0x7fcff019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfefe00640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0109d80 0x7fcff019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfe6a00640 1 -- 172.21.15.27:0/2578301100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff0103b00 msgr2=0x7fcff019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff0103b00 0x7fcff019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfe6a00640 1 -- 172.21.15.27:0/2578301100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0109d80 msgr2=0x7fcff019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0109d80 0x7fcff019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfe6a00640 1 -- 172.21.15.27:0/2578301100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff01a03a0 con 0x7fcff0109430 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109430 0x7fcff019aa10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800d950 tx=0x7fcfd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.146+0000 7fcfecc00640 1 -- 172.21.15.27:0/2578301100 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8017070 con 0x7fcff0109430 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.147+0000 7fcfecc00640 1 -- 172.21.15.27:0/2578301100 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd8009c30 con 0x7fcff0109430 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.147+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff01a0690 con 0x7fcff0109430 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.147+0000 7fcfecc00640 1 -- 172.21.15.27:0/2578301100 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80047a0 con 0x7fcff0109430 2024-09-21T07:24:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.147+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff01a6e20 con 0x7fcff0109430 2024-09-21T07:24:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 Processor -- start 2024-09-21T07:24:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 -- start start 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf32c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb652f0 0x55d13bf33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf336b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d13bea86b0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d13be97ae0 con 0x55d13bb8e1b0 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcffd070740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d13be9b8f0 con 0x55d13bb652f0 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.150+0000 7fcfecc00640 1 -- 172.21.15.27:0/2578301100 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcfd8004020 con 0x7fcff0109430 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf32c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfef400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb652f0 0x55d13bf33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf32c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45770/0 (socket says 172.21.15.27:45770) 2024-09-21T07:24:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 -- 172.21.15.27:0/2292592456 learned_addr learned my addr 172.21.15.27:0/2292592456 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 -- 172.21.15.27:0/2292592456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb652f0 msgr2=0x55d13bf33170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb652f0 0x55d13bf33170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 -- 172.21.15.27:0/2292592456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 msgr2=0x55d13bf336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 -- 172.21.15.27:0/2292592456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13be64780 con 0x55d13bb8e1b0 2024-09-21T07:24:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.151+0000 7fcfefe00640 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf32c30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8033000 tx=0x7fcfe8005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9926fab33790bb07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.152+0000 7fcfdfe00640 1 -- 172.21.15.27:0/2292592456 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8035040 con 0x55d13bb8e1b0 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.152+0000 7fcfdfe00640 1 -- 172.21.15.27:0/2292592456 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe80054d0 con 0x55d13bb8e1b0 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.152+0000 7fcfdfe00640 1 -- 172.21.15.27:0/2292592456 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8003810 con 0x55d13bb8e1b0 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.152+0000 7fcfecc00640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcfd40b6700 0x7fcfd40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.152+0000 7fcfecc00640 1 -- 172.21.15.27:0/2578301100 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fcfd80e7570 con 0x7fcff0109430 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.152+0000 7fcfef400640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcfd40b6700 0x7fcfd40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 -- 172.21.15.27:0/2292592456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 msgr2=0x55d13bf32c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf32c30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8033000 tx=0x7fcfe8005f30 comp rx=0 tx=0).stop 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 -- 172.21.15.27:0/2292592456 shutdown_connections 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb652f0 0x55d13bf33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 --2- 172.21.15.27:0/2292592456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf32c30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 -- 172.21.15.27:0/2292592456 >> 172.21.15.27:0/2292592456 conn(0x55d13bb82300 msgr2=0x55d13bf30b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 -- 172.21.15.27:0/2292592456 shutdown_connections 2024-09-21T07:24:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcffd070740 1 -- 172.21.15.27:0/2292592456 wait complete. 2024-09-21T07:24:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.153+0000 7fcfef400640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcfd40b6700 0x7fcfd40b8c10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55d13bf318b0 tx=0x7fcfe0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 -- start start 2024-09-21T07:24:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 Processor -- start 2024-09-21T07:24:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bf3e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb8e1b0 0x55d13bf3ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d13bdcfab0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d13be3e570 con 0x55d13bb652f0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d13bf41350 con 0x55d13bb8e1b0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.154+0000 7fcffd070740 10 client.0 _ll_register_callbacks cb 0x55d13be4d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52544/0 (socket says 172.21.15.27:52544) 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 -- 172.21.15.27:0/3960283273 learned_addr learned my addr 172.21.15.27:0/3960283273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfef400640 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bf3e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfefe00640 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb8e1b0 0x55d13bf3ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 -- 172.21.15.27:0/3960283273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb8e1b0 msgr2=0x55d13bf3ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb8e1b0 0x55d13bf3ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 -- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 msgr2=0x55d13bf3e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bf3e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 -- 172.21.15.27:0/3960283273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d13bbf9790 con 0x55d13bb6d4d0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3e7f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80e4970 tx=0x7fcfd80eacb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5e8e9dea52e422 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.155+0000 7fcfdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcfdcc00640 1 -- 172.21.15.27:0/3960283273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80a8eb0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcfdcc00640 1 -- 172.21.15.27:0/3960283273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fcfd80e72d0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfe80054d0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcfdcc00640 1 -- 172.21.15.27:0/3960283273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfd80ac780 con 0x55d13bb6d4d0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcffd070740 10 client.18888 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d13bc72620 con 0x55d13bb6d4d0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcfdcc00640 1 -- 172.21.15.27:0/3960283273 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe80054d0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.156+0000 7fcfdcc00640 1 -- 172.21.15.27:0/3960283273 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcfd8098190 con 0x55d13bb6d4d0 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.157+0000 7fcffd070740 10 client.18888 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.157+0000 7fcffd070740 10 client.18888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.157+0000 7fcffd070740 20 client.18888 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.157+0000 7fcffd070740 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb4cd80 0x55d13bf50300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.157+0000 7fcffd070740 4 client.18888 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.157+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d13beaa600 con 0x55d13bb4cd80 2024-09-21T07:24:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.157+0000 7fcfef400640 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb4cd80 0x55d13bf50300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.158+0000 7fcfef400640 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb4cd80 0x55d13bf50300 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47627f5489a25bbf server_cookie=1202eb13b3234243 in_seq=0 out_seq=0 2024-09-21T07:24:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.158+0000 7fcfdcc00640 10 client.18888 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.158+0000 7fcfdcc00640 1 -- 172.21.15.27:0/3960283273 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d13beaa600 con 0x55d13bb4cd80 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.158+0000 7fcfdcc00640 10 client.18888 handle_command_reply: tid=0 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 client.18888 shutdown 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 msgr2=0x55d13bf3e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3e7f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80e4970 tx=0x7fcfd80eacb0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 shutdown_connections 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb4cd80 0x55d13bf50300 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb8e1b0 0x55d13bf3ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3e7f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcfdcc00640 0 client.18888 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 --2- 172.21.15.27:0/3960283273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bf3e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 >> 172.21.15.27:0/3960283273 conn(0x55d13bb82300 msgr2=0x55d13bf33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 shutdown_connections 2024-09-21T07:24:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 1 -- 172.21.15.27:0/3960283273 wait complete. 2024-09-21T07:24:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.159+0000 7fcffd070740 20 client.18888 trim_cache size 0 max 16384 2024-09-21T07:24:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.174+0000 7fcffd070740 1 Processor -- start 2024-09-21T07:24:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcffd070740 1 -- start start 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcffd070740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb652f0 0x55d13bf34620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcffd070740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcffd070740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcffd070740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01b37b0 con 0x55d13bb652f0 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcffd070740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01b1080 con 0x55d13bb8e1b0 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcffd070740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01aeef0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcfef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb652f0 0x55d13bf34620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcfef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb652f0 0x55d13bf34620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52548/0 (socket says 172.21.15.27:52548) 2024-09-21T07:24:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcfef400640 1 -- 172.21.15.27:0/3079326536 learned_addr learned my addr 172.21.15.27:0/3079326536 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.175+0000 7fcfefe00640 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfe6a00640 1 -- 172.21.15.27:0/3079326536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 msgr2=0x55d13bf469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfe6a00640 1 -- 172.21.15.27:0/3079326536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb652f0 msgr2=0x55d13bf34620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb652f0 0x55d13bf34620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfe6a00640 1 -- 172.21.15.27:0/3079326536 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13bf50670 con 0x55d13bb6d4d0 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfef400640 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb652f0 0x55d13bf34620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3fe70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80e4f10 tx=0x7fcfd8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fb62919c1f05bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfdf400640 1 -- 172.21.15.27:0/3079326536 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80ac960 con 0x55d13bb6d4d0 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfdf400640 1 -- 172.21.15.27:0/3079326536 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd80acb20 con 0x55d13bb6d4d0 2024-09-21T07:24:41.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.176+0000 7fcfdf400640 1 -- 172.21.15.27:0/3079326536 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80acde0 con 0x55d13bb6d4d0 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 -- 172.21.15.27:0/3079326536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 msgr2=0x55d13bf3fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3fe70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80e4f10 tx=0x7fcfd8000bc0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 -- 172.21.15.27:0/3079326536 shutdown_connections 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bf3fe70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb8e1b0 0x55d13bf469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 --2- 172.21.15.27:0/3079326536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb652f0 0x55d13bf34620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 -- 172.21.15.27:0/3079326536 >> 172.21.15.27:0/3079326536 conn(0x55d13bb82300 msgr2=0x55d13bf3f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 -- 172.21.15.27:0/3079326536 shutdown_connections 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 -- 172.21.15.27:0/3079326536 wait complete. 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.177+0000 7fcffd070740 1 -- start start 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 1 Processor -- start 2024-09-21T07:24:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bfb2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bfb3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb8e1b0 0x55d13bfbdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01af540 con 0x55d13bb8e1b0 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01b09b0 con 0x55d13bb652f0 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01afc10 con 0x55d13bb6d4d0 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcffd070740 10 client.0 _ll_register_callbacks cb 0x7fcfd8004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb8e1b0 0x55d13bfbdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb8e1b0 0x55d13bfbdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52550/0 (socket says 172.21.15.27:52550) 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.178+0000 7fcfefe00640 1 -- 172.21.15.27:0/885980353 learned_addr learned my addr 172.21.15.27:0/885980353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfe6a00640 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bfb3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfef400640 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bfb2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfefe00640 1 -- 172.21.15.27:0/885980353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 msgr2=0x55d13bfb3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfefe00640 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bfb3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfefe00640 1 -- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 msgr2=0x55d13bfb2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfefe00640 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bfb2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfefe00640 1 -- 172.21.15.27:0/885980353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d13bfb3920 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfefe00640 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb8e1b0 0x55d13bfbdc70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004130 tx=0x7fcfe8004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17779e2cfebdc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.179+0000 7fcfdcc00640 1 -- 172.21.15.27:0/885980353 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8035040 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd80acb20 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcfdcc00640 1 -- 172.21.15.27:0/885980353 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fcfe80049b0 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcfdcc00640 1 -- 172.21.15.27:0/885980353 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfe8009520 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 10 client.18894 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d13bfbf1a0 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcfdcc00640 1 -- 172.21.15.27:0/885980353 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd80acb20 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcfdcc00640 1 -- 172.21.15.27:0/885980353 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcfe8004fe0 con 0x55d13bb8e1b0 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 10 client.18894 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 10 client.18894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 20 client.18894 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb221c0 0x55d13bfc1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 4 client.18894 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:24:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.180+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d13bea60b0 con 0x55d13bb221c0 2024-09-21T07:24:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.181+0000 7fcfef400640 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb221c0 0x55d13bfc1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.181+0000 7fcfef400640 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb221c0 0x55d13bfc1530 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c18204117b5be3 server_cookie=9352bcbcec81a84b in_seq=0 out_seq=0 2024-09-21T07:24:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.181+0000 7fcfdcc00640 10 client.18894 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.195+0000 7fcfdcc00640 1 -- 172.21.15.27:0/885980353 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d13bea60b0 con 0x55d13bb221c0 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.195+0000 7fcfdcc00640 10 client.18894 handle_command_reply: tid=0 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.195+0000 7fcffd070740 1 client.18894 shutdown 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb8e1b0 msgr2=0x55d13bfbdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb8e1b0 0x55d13bfbdc70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004130 tx=0x7fcfe8004bc0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 shutdown_connections 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d13bb221c0 0x55d13bfc1530 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d13bb6d4d0 0x55d13bfb3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d13bb8e1b0 0x55d13bfbdc70 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 --2- 172.21.15.27:0/885980353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d13bb652f0 0x55d13bfb2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 >> 172.21.15.27:0/885980353 conn(0x55d13bb82300 msgr2=0x55d13bf49cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcfdcc00640 0 client.18894 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 shutdown_connections 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 1 -- 172.21.15.27:0/885980353 wait complete. 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.196+0000 7fcffd070740 20 client.18894 trim_cache size 0 max 16384 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:24:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:24:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcfd40b6700 msgr2=0x7fcfd40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcfd40b6700 0x7fcfd40b8c10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55d13bf318b0 tx=0x7fcfe0009500 comp rx=0 tx=0).stop 2024-09-21T07:24:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109430 msgr2=0x7fcff019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109430 0x7fcff019aa10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800d950 tx=0x7fcfd800de10 comp rx=0 tx=0).stop 2024-09-21T07:24:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 shutdown_connections 2024-09-21T07:24:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcfd40b6700 0x7fcfd40b8c10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcff0109d80 0x7fcff019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0109430 0x7fcff019aa10 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 --2- 172.21.15.27:0/2578301100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff0103b00 0x7fcff019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 >> 172.21.15.27:0/2578301100 conn(0x7fcff00ff500 msgr2=0x7fcff0106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 shutdown_connections 2024-09-21T07:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:41.197+0000 7fcff6000640 1 -- 172.21.15.27:0/2578301100 wait complete. 2024-09-21T07:24:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:41 smithi027 ceph-mon[30779]: pgmap v739: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-09-21T07:24:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:41 smithi153 ceph-mon[34747]: pgmap v739: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-09-21T07:24:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:41 smithi119 ceph-mon[34519]: pgmap v739: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-09-21T07:24:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:43 smithi027 ceph-mon[30779]: pgmap v740: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-21T07:24:43.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:43 smithi153 ceph-mon[34747]: pgmap v740: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-21T07:24:43.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:43 smithi119 ceph-mon[34519]: pgmap v740: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-21T07:24:45.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:45 smithi119 ceph-mon[34519]: pgmap v741: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:24:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:45 smithi027 ceph-mon[30779]: pgmap v741: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:24:45.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:45 smithi153 ceph-mon[34747]: pgmap v741: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:24:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:47 smithi027 ceph-mon[30779]: pgmap v742: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:24:47.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:47 smithi153 ceph-mon[34747]: pgmap v742: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:24:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:47 smithi119 ceph-mon[34519]: pgmap v742: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:24:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:49 smithi027 ceph-mon[30779]: pgmap v743: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:24:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:49 smithi153 ceph-mon[34747]: pgmap v743: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:24:49.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:49 smithi119 ceph-mon[34519]: pgmap v743: 129 pgs: 129 active+clean; 5.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:24:51.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:51 smithi027 ceph-mon[30779]: pgmap v744: 129 pgs: 129 active+clean; 5.2 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:24:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:51 smithi153 ceph-mon[34747]: pgmap v744: 129 pgs: 129 active+clean; 5.2 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:24:51.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:51 smithi119 ceph-mon[34519]: pgmap v744: 129 pgs: 129 active+clean; 5.2 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:24:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:53 smithi027 ceph-mon[30779]: pgmap v745: 129 pgs: 129 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:24:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:53 smithi153 ceph-mon[34747]: pgmap v745: 129 pgs: 129 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:24:53.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:53 smithi119 ceph-mon[34519]: pgmap v745: 129 pgs: 129 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:24:54.032 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 -- 172.21.15.27:0/422680632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 msgr2=0x7f677c10d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 --2- 172.21.15.27:0/422680632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c10d790 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f676802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 -- 172.21.15.27:0/422680632 shutdown_connections 2024-09-21T07:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 --2- 172.21.15.27:0/422680632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f677c10dd60 0x7f677c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 --2- 172.21.15.27:0/422680632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c10d790 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 --2- 172.21.15.27:0/422680632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f677c104900 0x7f677c104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 -- 172.21.15.27:0/422680632 >> 172.21.15.27:0/422680632 conn(0x7f677c07a740 msgr2=0x7f677c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 -- 172.21.15.27:0/422680632 shutdown_connections 2024-09-21T07:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.398+0000 7f6780c00640 1 -- 172.21.15.27:0/422680632 wait complete. 2024-09-21T07:24:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 Processor -- start 2024-09-21T07:24:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 -- start start 2024-09-21T07:24:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f677c104900 0x7f677c102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f677c10dd60 0x7f677c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1139c0 con 0x7f677c105250 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f677c113840 con 0x7f677c10dd60 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.399+0000 7f6780c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f677c113b40 con 0x7f677c104900 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38864/0 (socket says 172.21.15.27:38864) 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 -- 172.21.15.27:0/3471943389 learned_addr learned my addr 172.21.15.27:0/3471943389 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779e00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f677c104900 0x7f677c102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f677a800640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f677c10dd60 0x7f677c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 -- 172.21.15.27:0/3471943389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f677c104900 msgr2=0x7f677c102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f677c104900 0x7f677c102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 -- 172.21.15.27:0/3471943389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f677c10dd60 msgr2=0x7f677c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f677c10dd60 0x7f677c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.400+0000 7f6779400640 1 -- 172.21.15.27:0/3471943389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c1a86d0 con 0x7f677c105250 2024-09-21T07:24:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.401+0000 7f6779400640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c102b70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f676802a910 tx=0x7f6768031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.401+0000 7f6772a00640 1 -- 172.21.15.27:0/3471943389 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676803f030 con 0x7f677c105250 2024-09-21T07:24:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.401+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c1a8960 con 0x7f677c105250 2024-09-21T07:24:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.401+0000 7f6772a00640 1 -- 172.21.15.27:0/3471943389 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67680043d0 con 0x7f677c105250 2024-09-21T07:24:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.401+0000 7f6772a00640 1 -- 172.21.15.27:0/3471943389 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768035070 con 0x7f677c105250 2024-09-21T07:24:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.401+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a8e50 con 0x7f677c105250 2024-09-21T07:24:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.405+0000 7f6787c00740 1 Processor -- start 2024-09-21T07:24:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6787c00740 1 -- start start 2024-09-21T07:24:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6787c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397d6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6787c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394092f0 0x55e9397d7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6787c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394114d0 0x55e9397d76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6787c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e93974c6b0 con 0x55e9394092f0 2024-09-21T07:24:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6787c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e93973bae0 con 0x55e9394114d0 2024-09-21T07:24:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6787c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e93973f8f0 con 0x55e9394321b0 2024-09-21T07:24:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6772a00640 1 -- 172.21.15.27:0/3471943389 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6768042030 con 0x7f677c105250 2024-09-21T07:24:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.406+0000 7f6779e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394092f0 0x55e9397d7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f6779e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394092f0 0x55e9397d7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38868/0 (socket says 172.21.15.27:38868) 2024-09-21T07:24:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f6779e00640 1 -- 172.21.15.27:0/2509548257 learned_addr learned my addr 172.21.15.27:0/2509548257 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f677a800640 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397d6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f6779400640 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394114d0 0x55e9397d76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f677a800640 1 -- 172.21.15.27:0/2509548257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394114d0 msgr2=0x55e9397d76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f677a800640 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394114d0 0x55e9397d76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f677a800640 1 -- 172.21.15.27:0/2509548257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394092f0 msgr2=0x55e9397d7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f677a800640 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394092f0 0x55e9397d7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f677a800640 1 -- 172.21.15.27:0/2509548257 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e939708780 con 0x55e9394321b0 2024-09-21T07:24:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f6779400640 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394114d0 0x55e9397d76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:24:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f6779e00640 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394092f0 0x55e9397d7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.407+0000 7f677a800640 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397d6c30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6774009f40 tx=0x7f67740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccbacfa8c669c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.408+0000 7f6767000640 1 -- 172.21.15.27:0/2509548257 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774004d20 con 0x55e9394321b0 2024-09-21T07:24:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.408+0000 7f6767000640 1 -- 172.21.15.27:0/2509548257 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6774004050 con 0x55e9394321b0 2024-09-21T07:24:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.408+0000 7f6767000640 1 -- 172.21.15.27:0/2509548257 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774005240 con 0x55e9394321b0 2024-09-21T07:24:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6772a00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67600b66b0 0x7f67600b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 -- 172.21.15.27:0/2509548257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 msgr2=0x55e9397d6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397d6c30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6774009f40 tx=0x7f67740118d0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 -- 172.21.15.27:0/2509548257 shutdown_connections 2024-09-21T07:24:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6772a00640 1 -- 172.21.15.27:0/3471943389 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f6768107d80 con 0x7f677c105250 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394114d0 0x55e9397d76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394092f0 0x55e9397d7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 --2- 172.21.15.27:0/2509548257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397d6c30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 -- 172.21.15.27:0/2509548257 >> 172.21.15.27:0/2509548257 conn(0x55e939426300 msgr2=0x55e9397d4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6779e00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67600b66b0 0x7f67600b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 -- 172.21.15.27:0/2509548257 shutdown_connections 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.409+0000 7f6787c00740 1 -- 172.21.15.27:0/2509548257 wait complete. 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.410+0000 7f6787c00740 1 -- start start 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.410+0000 7f6779e00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67600b66b0 0x7f67600b8bc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f676c009530 tx=0x7f676c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.410+0000 7f6787c00740 1 Processor -- start 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6787c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394092f0 0x55e9397e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6787c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397e2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6787c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394321b0 0x55e9397e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6787c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e939673ab0 con 0x55e9394114d0 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6787c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e9396e2570 con 0x55e9394321b0 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6787c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9397e53b0 con 0x55e9394092f0 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6787c00740 10 client.0 _ll_register_callbacks cb 0x55e9396f1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6779400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6779400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397e2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38884/0 (socket says 172.21.15.27:38884) 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6779400640 1 -- 172.21.15.27:0/1633885272 learned_addr learned my addr 172.21.15.27:0/1633885272 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f677a800640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394321b0 0x55e9397e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.411+0000 7f6779e00640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394092f0 0x55e9397e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f6779400640 1 -- 172.21.15.27:0/1633885272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394092f0 msgr2=0x55e9397e2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f6779400640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394092f0 0x55e9397e2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f6779400640 1 -- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394321b0 msgr2=0x55e9397e2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f6779400640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394321b0 0x55e9397e2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f6779400640 1 -- 172.21.15.27:0/1633885272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93949d790 con 0x55e9394114d0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f677a800640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394321b0 0x55e9397e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f6779400640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397e2780 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6768006010 tx=0x7f6768003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e057639b913f15b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f675b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f675b400640 1 -- 172.21.15.27:0/1633885272 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768049070 con 0x55e9394114d0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f675b400640 1 -- 172.21.15.27:0/1633885272 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f67680cca80 con 0x55e9394114d0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6774004050 con 0x55e9394114d0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.412+0000 7f675b400640 1 -- 172.21.15.27:0/1633885272 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67680ccf90 con 0x55e9394114d0 2024-09-21T07:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 10 client.18906 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e939516620 con 0x55e9394114d0 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f675b400640 1 -- 172.21.15.27:0/1633885272 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6774004050 con 0x55e9394114d0 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f675b400640 1 -- 172.21.15.27:0/1633885272 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f67680cc4f0 con 0x55e9394114d0 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 10 client.18906 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 10 client.18906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 20 client.18906 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393f0d80 0x55e9397f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 4 client.18906 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.413+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e93974e600 con 0x55e9393f0d80 2024-09-21T07:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.414+0000 7f6779e00640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393f0d80 0x55e9397f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.414+0000 7f675b400640 10 client.18906 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.414+0000 7f6779e00640 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393f0d80 0x55e9397f4340 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdd521a8f00190d server_cookie=44e049f62eaf905b in_seq=0 out_seq=0 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.415+0000 7f675b400640 1 -- 172.21.15.27:0/1633885272 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e93974e600 con 0x55e9393f0d80 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.415+0000 7f675b400640 10 client.18906 handle_command_reply: tid=0 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.415+0000 7f6787c00740 1 client.18906 shutdown 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 msgr2=0x55e9397e2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397e2780 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6768006010 tx=0x7f6768003f30 comp rx=0 tx=0).stop 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 shutdown_connections 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393f0d80 0x55e9397f4340 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394321b0 0x55e9397e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397e2780 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 --2- 172.21.15.27:0/1633885272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394092f0 0x55e9397e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 >> 172.21.15.27:0/1633885272 conn(0x55e939426300 msgr2=0x55e9397d7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f675b400640 0 client.18906 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 shutdown_connections 2024-09-21T07:24:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 1 -- 172.21.15.27:0/1633885272 wait complete. 2024-09-21T07:24:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.416+0000 7f6787c00740 20 client.18906 trim_cache size 0 max 16384 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 Processor -- start 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 -- start start 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397e3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397df950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b5900 con 0x55e9394114d0 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b31d0 con 0x55e9394092f0 2024-09-21T07:24:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.433+0000 7f6787c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b1040 con 0x55e9394321b0 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397e3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397e3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37126/0 (socket says 172.21.15.27:37126) 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779e00640 1 -- 172.21.15.27:0/2340847905 learned_addr learned my addr 172.21.15.27:0/2340847905 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f677a800640 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397df950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779400640 1 -- 172.21.15.27:0/2340847905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 msgr2=0x55e9397df950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779400640 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397df950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779400640 1 -- 172.21.15.27:0/2340847905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 msgr2=0x55e9397e3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779400640 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397e3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779400640 1 -- 172.21.15.27:0/2340847905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9397d7a10 con 0x55e9394114d0 2024-09-21T07:24:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779e00640 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.434+0000 7f6779400640 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397df410 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6768031440 tx=0x7f6768009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ac318ecd22d62b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.435+0000 7f6766600640 1 -- 172.21.15.27:0/2340847905 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768049030 con 0x55e9394114d0 2024-09-21T07:24:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.435+0000 7f6766600640 1 -- 172.21.15.27:0/2340847905 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67680cd330 con 0x55e9394114d0 2024-09-21T07:24:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.435+0000 7f6766600640 1 -- 172.21.15.27:0/2340847905 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67681055b0 con 0x55e9394114d0 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 -- 172.21.15.27:0/2340847905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 msgr2=0x55e9397df410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397df410 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6768031440 tx=0x7f6768009650 comp rx=0 tx=0).stop 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 -- 172.21.15.27:0/2340847905 shutdown_connections 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397df950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397df410 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 --2- 172.21.15.27:0/2340847905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 -- 172.21.15.27:0/2340847905 >> 172.21.15.27:0/2340847905 conn(0x55e939426300 msgr2=0x55e9397e3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 -- 172.21.15.27:0/2340847905 shutdown_connections 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 -- 172.21.15.27:0/2340847905 wait complete. 2024-09-21T07:24:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 -- start start 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.436+0000 7f6787c00740 1 Processor -- start 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6787c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397eaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6787c00740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6787c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6787c00740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b1690 con 0x55e9394114d0 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6787c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b2b00 con 0x55e9394092f0 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6787c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b1d60 con 0x55e9394321b0 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6787c00740 10 client.0 _ll_register_callbacks cb 0x7f6768005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6779400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6779e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397eaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6779e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397eaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37136/0 (socket says 172.21.15.27:37136) 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f6779e00640 1 -- 172.21.15.27:0/2529427098 learned_addr learned my addr 172.21.15.27:0/2529427098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.437+0000 7f677a800640 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f6779e00640 1 -- 172.21.15.27:0/2529427098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 msgr2=0x55e9397e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f6779e00640 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397e60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f6779e00640 1 -- 172.21.15.27:0/2529427098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 msgr2=0x55e9397eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f6779e00640 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397eb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f6779e00640 1 -- 172.21.15.27:0/2529427098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9397e6840 con 0x55e9394092f0 2024-09-21T07:24:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f6779e00640 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397eaa60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f676c0045a0 tx=0x7f676c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be91362ae4b47a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f675b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.438+0000 7f675b400640 1 -- 172.21.15.27:0/2529427098 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c036060 con 0x55e9394092f0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.439+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67680cd330 con 0x55e9394092f0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.439+0000 7f675b400640 1 -- 172.21.15.27:0/2529427098 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f676c02b590 con 0x55e9394092f0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.439+0000 7f675b400640 1 -- 172.21.15.27:0/2529427098 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f676c03d050 con 0x55e9394092f0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.439+0000 7f6787c00740 10 client.28922 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.439+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e93985e9a0 con 0x55e9394092f0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.439+0000 7f675b400640 1 -- 172.21.15.27:0/2529427098 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67680cd330 con 0x55e9394092f0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f675b400640 1 -- 172.21.15.27:0/2529427098 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f676c031960 con 0x55e9394092f0 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f6787c00740 10 client.28922 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f6787c00740 10 client.28922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f6787c00740 20 client.28922 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f6787c00740 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393c61c0 0x55e939861170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f6787c00740 4 client.28922 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:24:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e93974a0b0 con 0x55e9393c61c0 2024-09-21T07:24:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.440+0000 7f6779400640 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393c61c0 0x55e939861170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.441+0000 7f675b400640 10 client.28922 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.441+0000 7f6779400640 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393c61c0 0x55e939861170 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4f9cbff83d51fb server_cookie=2824605d37d41e5b in_seq=0 out_seq=0 2024-09-21T07:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f675b400640 1 -- 172.21.15.27:0/2529427098 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e93974a0b0 con 0x55e9393c61c0 2024-09-21T07:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f675b400640 10 client.28922 handle_command_reply: tid=0 2024-09-21T07:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 client.28922 shutdown 2024-09-21T07:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 msgr2=0x55e9397eaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397eaa60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f676c0045a0 tx=0x7f676c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 shutdown_connections 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e9393c61c0 0x55e939861170 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9394321b0 0x55e9397e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e9394114d0 0x55e9397eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 --2- 172.21.15.27:0/2529427098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9394092f0 0x55e9397eaa60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f675b400640 0 client.28922 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 >> 172.21.15.27:0/2529427098 conn(0x55e939426300 msgr2=0x55e9397d4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 shutdown_connections 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.455+0000 7f6787c00740 1 -- 172.21.15.27:0/2529427098 wait complete. 2024-09-21T07:24:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.456+0000 7f6787c00740 20 client.28922 trim_cache size 0 max 16384 2024-09-21T07:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.456+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67600b66b0 msgr2=0x7f67600b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.456+0000 7f6780c00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67600b66b0 0x7f67600b8bc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f676c009530 tx=0x7f676c005f90 comp rx=0 tx=0).stop 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.456+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 msgr2=0x7f677c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.456+0000 7f6780c00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c102b70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f676802a910 tx=0x7f6768031a20 comp rx=0 tx=0).stop 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 shutdown_connections 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f67600b66b0 0x7f67600b8bc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f677c10dd60 0x7f677c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677c105250 0x7f677c102b70 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 --2- 172.21.15.27:0/3471943389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f677c104900 0x7f677c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 >> 172.21.15.27:0/3471943389 conn(0x7f677c07a740 msgr2=0x7f677c100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 shutdown_connections 2024-09-21T07:24:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:24:54.457+0000 7f6780c00640 1 -- 172.21.15.27:0/3471943389 wait complete. 2024-09-21T07:24:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 -- 172.21.15.27:0/1957555537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25241092f0 msgr2=0x7f2524109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 --2- 172.21.15.27:0/1957555537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25241092f0 0x7f2524109700 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2510009900 tx=0x7f251002a410 comp rx=0 tx=0).stop 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 -- 172.21.15.27:0/1957555537 shutdown_connections 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 --2- 172.21.15.27:0/1957555537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2524109c40 0x7f2524110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 --2- 172.21.15.27:0/1957555537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25241092f0 0x7f2524109700 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 --2- 172.21.15.27:0/1957555537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 0x7f2524103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 -- 172.21.15.27:0/1957555537 >> 172.21.15.27:0/1957555537 conn(0x7f25240ff440 msgr2=0x7f2524101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 -- 172.21.15.27:0/1957555537 shutdown_connections 2024-09-21T07:24:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.077+0000 7f252b000640 1 -- 172.21.15.27:0/1957555537 wait complete. 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.078+0000 7f252b000640 1 Processor -- start 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.078+0000 7f252b000640 1 -- start start 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f252b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 0x7f252419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f252b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25241092f0 0x7f252419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f252b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2524109c40 0x7f25241a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f252b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2524113900 con 0x7f2524109c40 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f252b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2524113780 con 0x7f25241039b0 2024-09-21T07:24:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f252b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2524113a80 con 0x7f25241092f0 2024-09-21T07:24:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f2529c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 0x7f252419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f2529c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 0x7f252419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37168/0 (socket says 172.21.15.27:37168) 2024-09-21T07:24:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f2529c00640 1 -- 172.21.15.27:0/4161603726 learned_addr learned my addr 172.21.15.27:0/4161603726 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.079+0000 7f2529200640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25241092f0 0x7f252419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f252a600640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2524109c40 0x7f25241a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2529c00640 1 -- 172.21.15.27:0/4161603726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25241092f0 msgr2=0x7f252419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2529c00640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25241092f0 0x7f252419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2529c00640 1 -- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2524109c40 msgr2=0x7f25241a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2529c00640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2524109c40 0x7f25241a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2529c00640 1 -- 172.21.15.27:0/4161603726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25241a24a0 con 0x7f25241039b0 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f252a600640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2524109c40 0x7f25241a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2529c00640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 0x7f252419c530 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f251800e9f0 tx=0x7f251800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2522000640 1 -- 172.21.15.27:0/4161603726 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251800cd80 con 0x7f25241039b0 2024-09-21T07:24:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.080+0000 7f2522000640 1 -- 172.21.15.27:0/4161603726 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25180108e0 con 0x7f25241039b0 2024-09-21T07:24:55.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.081+0000 7f2522000640 1 -- 172.21.15.27:0/4161603726 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518018980 con 0x7f25241039b0 2024-09-21T07:24:55.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.081+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25241a2790 con 0x7f25241039b0 2024-09-21T07:24:55.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.081+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25241a8ef0 con 0x7f25241039b0 2024-09-21T07:24:55.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.084+0000 7f25333cf740 1 Processor -- start 2024-09-21T07:24:55.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.084+0000 7f25333cf740 1 -- start start 2024-09-21T07:24:55.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f25333cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5f0070 0x55c73c996f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f2522000640 1 -- 172.21.15.27:0/4161603726 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2518010430 con 0x7f25241039b0 2024-09-21T07:24:55.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f25333cf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c997470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f25333cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f25333cf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c73c90df70 con 0x55c73c5c71b0 2024-09-21T07:24:55.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f25333cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c73c8fd3a0 con 0x55c73c5f0070 2024-09-21T07:24:55.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f25333cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c73c9011b0 con 0x55c73c5cf390 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f2529c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c997470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f252a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5f0070 0x55c73c996f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f252a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5f0070 0x55c73c996f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37170/0 (socket says 172.21.15.27:37170) 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f252a600640 1 -- 172.21.15.27:0/2897796902 learned_addr learned my addr 172.21.15.27:0/2897796902 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.085+0000 7f2529200640 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.086+0000 7f252a600640 1 -- 172.21.15.27:0/2897796902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 msgr2=0x55c73c9979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.086+0000 7f252a600640 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.086+0000 7f252a600640 1 -- 172.21.15.27:0/2897796902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 msgr2=0x55c73c997470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.086+0000 7f252a600640 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c997470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.086+0000 7f252a600640 1 -- 172.21.15.27:0/2897796902 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73c8c7c90 con 0x55c73c5f0070 2024-09-21T07:24:55.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.086+0000 7f252a600640 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5f0070 0x55c73c996f30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f251c009fd0 tx=0x7f251c0108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80cedb3495f98a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.086+0000 7f2516a00640 1 -- 172.21.15.27:0/2897796902 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c011760 con 0x55c73c5f0070 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.087+0000 7f2516a00640 1 -- 172.21.15.27:0/2897796902 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251c011db0 con 0x55c73c5f0070 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.087+0000 7f2516a00640 1 -- 172.21.15.27:0/2897796902 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c00e3c0 con 0x55c73c5f0070 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.087+0000 7f2522000640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f250c0b6700 0x7f250c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.087+0000 7f2522000640 1 -- 172.21.15.27:0/4161603726 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f25180e7e30 con 0x7f25241039b0 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.087+0000 7f2529c00640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f250c0b6700 0x7f250c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 -- 172.21.15.27:0/2897796902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5f0070 msgr2=0x55c73c996f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5f0070 0x55c73c996f30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f251c009fd0 tx=0x7f251c0108e0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 -- 172.21.15.27:0/2897796902 shutdown_connections 2024-09-21T07:24:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c997470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 --2- 172.21.15.27:0/2897796902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5f0070 0x55c73c996f30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 -- 172.21.15.27:0/2897796902 >> 172.21.15.27:0/2897796902 conn(0x55c73c5e4460 msgr2=0x55c73c994e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 -- 172.21.15.27:0/2897796902 shutdown_connections 2024-09-21T07:24:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f25333cf740 1 -- 172.21.15.27:0/2897796902 wait complete. 2024-09-21T07:24:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.088+0000 7f2529c00640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f250c0b6700 0x7f250c0b8c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f25180e5280 tx=0x7f25180eb980 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.089+0000 7f25333cf740 1 -- start start 2024-09-21T07:24:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.089+0000 7f25333cf740 1 Processor -- start 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.089+0000 7f25333cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5c71b0 0x55c73c9a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f25333cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f25333cf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f25333cf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c73c75e700 con 0x55c73c5f0070 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f25333cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c73c898540 con 0x55c73c5c71b0 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f25333cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c73c9a5710 con 0x55c73c5cf390 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f25333cf740 10 client.0 _ll_register_callbacks cb 0x55c73c8a8430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f252a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5c71b0 0x55c73c9a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f252a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5c71b0 0x55c73c9a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37178/0 (socket says 172.21.15.27:37178) 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f252a600640 1 -- 172.21.15.27:0/3654574732 learned_addr learned my addr 172.21.15.27:0/3654574732 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f2529200640 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f252a600640 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f2529200640 1 -- 172.21.15.27:0/3654574732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5c71b0 msgr2=0x55c73c9a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f2529200640 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5c71b0 0x55c73c9a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f2529200640 1 -- 172.21.15.27:0/3654574732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 msgr2=0x55c73c9aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f2529200640 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.090+0000 7f2529200640 1 -- 172.21.15.27:0/3654574732 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73c65b650 con 0x55c73c5cf390 2024-09-21T07:24:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.091+0000 7f2529200640 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9a2c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f251002a930 tx=0x7f2510032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8955d97ea11ef27b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.091+0000 7f250b400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:24:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.091+0000 7f250b400640 1 -- 172.21.15.27:0/3654574732 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510035040 con 0x55c73c5cf390 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.091+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c73c89ab50 con 0x55c73c5cf390 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.091+0000 7f250b400640 1 -- 172.21.15.27:0/3654574732 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f2510031760 con 0x55c73c5cf390 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f250b400640 1 -- 172.21.15.27:0/3654574732 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f251004f050 con 0x55c73c5cf390 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 10 client.28918 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c73c8b5910 con 0x55c73c5cf390 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f250b400640 1 -- 172.21.15.27:0/3654574732 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c73c89ab50 con 0x55c73c5cf390 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f250b400640 1 -- 172.21.15.27:0/3654574732 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2510004750 con 0x55c73c5cf390 2024-09-21T07:24:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 10 client.28918 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 10 client.28918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 20 client.28918 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c5ae9f0 0x55c73c9b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 4 client.28918 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.092+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73c90fec0 con 0x55c73c5ae9f0 2024-09-21T07:24:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.093+0000 7f252a600640 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c5ae9f0 0x55c73c9b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.093+0000 7f252a600640 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c5ae9f0 0x55c73c9b4860 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0a5b329baaed37 server_cookie=22bd0c66c7feb00f in_seq=0 out_seq=0 2024-09-21T07:24:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.093+0000 7f250b400640 10 client.28918 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f250b400640 1 -- 172.21.15.27:0/3654574732 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c73c90fec0 con 0x55c73c5ae9f0 2024-09-21T07:24:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f250b400640 10 client.28918 handle_command_reply: tid=0 2024-09-21T07:24:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 client.28918 shutdown 2024-09-21T07:24:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 msgr2=0x55c73c9a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9a2c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f251002a930 tx=0x7f2510032e90 comp rx=0 tx=0).stop 2024-09-21T07:24:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 shutdown_connections 2024-09-21T07:24:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c5ae9f0 0x55c73c9b4860 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5cf390 0x55c73c9a2c30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 --2- 172.21.15.27:0/3654574732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5c71b0 0x55c73c9a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.094+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 >> 172.21.15.27:0/3654574732 conn(0x55c73c5e4460 msgr2=0x55c73c99a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.095+0000 7f250b400640 0 client.28918 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.095+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 shutdown_connections 2024-09-21T07:24:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.095+0000 7f25333cf740 1 -- 172.21.15.27:0/3654574732 wait complete. 2024-09-21T07:24:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.095+0000 7f25333cf740 20 client.28918 trim_cache size 0 max 16384 2024-09-21T07:24:55.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.115+0000 7f25333cf740 1 Processor -- start 2024-09-21T07:24:55.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.115+0000 7f25333cf740 1 -- start start 2024-09-21T07:24:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f25333cf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c9a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f25333cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c998670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f25333cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5f0070 0x55c73c9ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f25333cf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25241b5860 con 0x55c73c5c71b0 2024-09-21T07:24:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f25333cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25241b3130 con 0x55c73c5cf390 2024-09-21T07:24:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f25333cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f25241b0fa0 con 0x55c73c5f0070 2024-09-21T07:24:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c9a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c9a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38940/0 (socket says 172.21.15.27:38940) 2024-09-21T07:24:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 -- 172.21.15.27:0/649901546 learned_addr learned my addr 172.21.15.27:0/649901546 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f2529200640 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c998670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 -- 172.21.15.27:0/649901546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5f0070 msgr2=0x55c73c9ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5f0070 0x55c73c9ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 -- 172.21.15.27:0/649901546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 msgr2=0x55c73c998670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c998670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.116+0000 7f252a600640 1 -- 172.21.15.27:0/649901546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73c9a3790 con 0x55c73c5c71b0 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.117+0000 7f252a600640 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c9a15e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55c73c9a4f40 tx=0x7f251c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11188bed00b30cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.117+0000 7f2516000640 1 -- 172.21.15.27:0/649901546 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c024040 con 0x55c73c5c71b0 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.117+0000 7f2516000640 1 -- 172.21.15.27:0/649901546 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251c011560 con 0x55c73c5c71b0 2024-09-21T07:24:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.117+0000 7f2516000640 1 -- 172.21.15.27:0/649901546 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0052a0 con 0x55c73c5c71b0 2024-09-21T07:24:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 -- 172.21.15.27:0/649901546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 msgr2=0x55c73c9a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c9a15e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55c73c9a4f40 tx=0x7f251c009500 comp rx=0 tx=0).stop 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 -- 172.21.15.27:0/649901546 shutdown_connections 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5f0070 0x55c73c9ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c998670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 --2- 172.21.15.27:0/649901546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5c71b0 0x55c73c9a15e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 -- 172.21.15.27:0/649901546 >> 172.21.15.27:0/649901546 conn(0x55c73c5e4460 msgr2=0x55c73c997f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 -- 172.21.15.27:0/649901546 shutdown_connections 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.118+0000 7f25333cf740 1 -- 172.21.15.27:0/649901546 wait complete. 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 -- start start 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 Processor -- start 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5c71b0 0x55c73c9aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c9aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9a5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25241b15f0 con 0x55c73c5f0070 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25241b2a60 con 0x55c73c5cf390 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f25241b1cc0 con 0x55c73c5c71b0 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.119+0000 7f25333cf740 10 client.0 _ll_register_callbacks cb 0x7f2510032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f252a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f252a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38946/0 (socket says 172.21.15.27:38946) 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f252a600640 1 -- 172.21.15.27:0/2874031141 learned_addr learned my addr 172.21.15.27:0/2874031141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f2529200640 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c9aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f252a600640 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5c71b0 0x55c73c9aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f2529200640 1 -- 172.21.15.27:0/2874031141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5c71b0 msgr2=0x55c73c9aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f2529200640 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5c71b0 0x55c73c9aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f2529200640 1 -- 172.21.15.27:0/2874031141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 msgr2=0x55c73c9a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f2529200640 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f2529200640 1 -- 172.21.15.27:0/2874031141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73c9a5e90 con 0x55c73c5cf390 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f2529200640 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c9aaee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2510001eb0 tx=0x7f25100048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b41764d5075eea7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.120+0000 7f250b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.121+0000 7f250b400640 1 -- 172.21.15.27:0/2874031141 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510035070 con 0x55c73c5cf390 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.121+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f251c011560 con 0x55c73c5cf390 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.121+0000 7f250b400640 1 -- 172.21.15.27:0/2874031141 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f251004c040 con 0x55c73c5cf390 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.121+0000 7f250b400640 1 -- 172.21.15.27:0/2874031141 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2510061050 con 0x55c73c5cf390 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.121+0000 7f25333cf740 10 client.28940 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.121+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c73ca1f8b0 con 0x55c73c5cf390 2024-09-21T07:24:55.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.121+0000 7f250b400640 1 -- 172.21.15.27:0/2874031141 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251c011560 con 0x55c73c5cf390 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f250b400640 1 -- 172.21.15.27:0/2874031141 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2510005230 con 0x55c73c5cf390 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f25333cf740 10 client.28940 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f25333cf740 10 client.28940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f25333cf740 20 client.28940 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f25333cf740 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c583da0 0x55c73ca220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f25333cf740 4 client.28940 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c73c90b970 con 0x55c73c583da0 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.122+0000 7f252a600640 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c583da0 0x55c73ca220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.123+0000 7f252a600640 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c583da0 0x55c73ca220f0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6aff400b8d3198 server_cookie=bed1303434259d99 in_seq=0 out_seq=0 2024-09-21T07:24:55.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.123+0000 7f250b400640 10 client.28940 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:55.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.123+0000 7f250b400640 1 -- 172.21.15.27:0/2874031141 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x55c73c90b970 con 0x55c73c583da0 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.123+0000 7f250b400640 10 client.28940 handle_command_reply: tid=0 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.123+0000 7f25333cf740 1 client.28940 shutdown 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.123+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 msgr2=0x55c73c9aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.123+0000 7f25333cf740 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c9aaee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2510001eb0 tx=0x7f25100048f0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 shutdown_connections 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c73c583da0 0x55c73ca220f0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c73c5f0070 0x55c73c9a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c73c5cf390 0x55c73c9aaee0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 --2- 172.21.15.27:0/2874031141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c73c5c71b0 0x55c73c9aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f250b400640 0 client.28940 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 >> 172.21.15.27:0/2874031141 conn(0x55c73c5e4460 msgr2=0x55c73c9a3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 shutdown_connections 2024-09-21T07:24:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 1 -- 172.21.15.27:0/2874031141 wait complete. 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.124+0000 7f25333cf740 20 client.28940 trim_cache size 0 max 16384 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:24:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:24:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:24:55.137 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.138 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:24:55.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f250c0b6700 msgr2=0x7f250c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f250c0b6700 0x7f250c0b8c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f25180e5280 tx=0x7f25180eb980 comp rx=0 tx=0).stop 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 msgr2=0x7f252419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 0x7f252419c530 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f251800e9f0 tx=0x7f251800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 shutdown_connections 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25241092f0 0x7f252419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f250c0b6700 0x7f250c0b8c10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2524109c40 0x7f25241a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 --2- 172.21.15.27:0/4161603726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25241039b0 0x7f252419c530 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 >> 172.21.15.27:0/4161603726 conn(0x7f25240ff440 msgr2=0x7f2524108a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 shutdown_connections 2024-09-21T07:24:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:55.125+0000 7f252b000640 1 -- 172.21.15.27:0/4161603726 wait complete. 2024-09-21T07:24:55.148 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:55 smithi027 ceph-mon[30779]: pgmap v746: 129 pgs: 129 active+clean; 5.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-09-21T07:24:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:55 smithi153 ceph-mon[34747]: pgmap v746: 129 pgs: 129 active+clean; 5.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-09-21T07:24:55.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:55 smithi119 ceph-mon[34519]: pgmap v746: 129 pgs: 129 active+clean; 5.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-09-21T07:24:56.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:24:56.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:24:56.467 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 -- 172.21.15.27:0/2145276864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4109c20 msgr2=0x7f57c4110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2145276864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4109c20 0x7f57c4110450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009be0 tx=0x7f57bc02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 -- 172.21.15.27:0/2145276864 shutdown_connections 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2145276864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4109c20 0x7f57c4110450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2145276864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 0x7f57c41096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2145276864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57c4103990 0x7f57c4103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 -- 172.21.15.27:0/2145276864 >> 172.21.15.27:0/2145276864 conn(0x7f57c40ff440 msgr2=0x7f57c4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 -- 172.21.15.27:0/2145276864 shutdown_connections 2024-09-21T07:24:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.085+0000 7f57c3e00640 1 -- 172.21.15.27:0/2145276864 wait complete. 2024-09-21T07:24:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 Processor -- start 2024-09-21T07:24:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 -- start start 2024-09-21T07:24:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4103990 0x7f57c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 0x7f57c4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57c4109c20 0x7f57c4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f57c41138e0 con 0x7f57c41092d0 2024-09-21T07:24:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f57c4113760 con 0x7f57c4103990 2024-09-21T07:24:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.087+0000 7f57c3e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57c4113a60 con 0x7f57c4109c20 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 0x7f57c4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4103990 0x7f57c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 0x7f57c4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38964/0 (socket says 172.21.15.27:38964) 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57bbe00640 1 -- 172.21.15.27:0/2847591883 learned_addr learned my addr 172.21.15.27:0/2847591883 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4103990 0x7f57c4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37214/0 (socket says 172.21.15.27:37214) 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c3400640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57c4109c20 0x7f57c4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 -- 172.21.15.27:0/2847591883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57c4109c20 msgr2=0x7f57c4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57c4109c20 0x7f57c4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 -- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 msgr2=0x7f57c4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 0x7f57c4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 -- 172.21.15.27:0/2847591883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c41a8510 con 0x7f57c4103990 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57bbe00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 0x7f57c4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.088+0000 7f57c2a00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4103990 0x7f57c4075710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f57b400e9f0 tx=0x7f57b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.089+0000 7f57c8a00640 1 -- 172.21.15.27:0/2847591883 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b400ccc0 con 0x7f57c4103990 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.089+0000 7f57c8a00640 1 -- 172.21.15.27:0/2847591883 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57b40108e0 con 0x7f57c4103990 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.089+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c41a8800 con 0x7f57c4103990 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.089+0000 7f57c8a00640 1 -- 172.21.15.27:0/2847591883 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4018880 con 0x7f57c4103990 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.089+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c41a8d40 con 0x7f57c4103990 2024-09-21T07:24:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.090+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c4103da0 con 0x7f57c4103990 2024-09-21T07:24:57.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.092+0000 7f57c8a00640 1 -- 172.21.15.27:0/2847591883 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f57b4010430 con 0x7f57c4103990 2024-09-21T07:24:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.094+0000 7f57c8a00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57a80b6680 0x7f57a80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.094+0000 7f57c8a00640 1 -- 172.21.15.27:0/2847591883 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f57b40e7e60 con 0x7f57c4103990 2024-09-21T07:24:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.094+0000 7f57bbe00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57a80b6680 0x7f57a80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.095+0000 7f57bbe00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57a80b6680 0x7f57a80b8b90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f57c4102c20 tx=0x7f57ac009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:57.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.098+0000 7f57c8a00640 1 -- 172.21.15.27:0/2847591883 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f57b40b2380 con 0x7f57c4103990 2024-09-21T07:24:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.337+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57c402e640 con 0x7f57c4103990 2024-09-21T07:24:57.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:57 smithi027 ceph-mon[30779]: pgmap v747: 129 pgs: 129 active+clean; 5.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:24:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.338+0000 7f57c8a00640 1 -- 172.21.15.27:0/2847591883 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f57b40b72c0 con 0x7f57c4103990 2024-09-21T07:24:57.341 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:24:57.341 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:24:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.341+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57a80b6680 msgr2=0x7f57a80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.341+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57a80b6680 0x7f57a80b8b90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f57c4102c20 tx=0x7f57ac009260 comp rx=0 tx=0).stop 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4103990 msgr2=0x7f57c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4103990 0x7f57c4075710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f57b400e9f0 tx=0x7f57b400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 shutdown_connections 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57c4109c20 0x7f57c4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57a80b6680 0x7f57a80b8b90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57c41092d0 0x7f57c4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 --2- 172.21.15.27:0/2847591883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57c4103990 0x7f57c4075710 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 >> 172.21.15.27:0/2847591883 conn(0x7f57c40ff440 msgr2=0x7f57c4106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 shutdown_connections 2024-09-21T07:24:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:57.342+0000 7f57c3e00640 1 -- 172.21.15.27:0/2847591883 wait complete. 2024-09-21T07:24:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:57 smithi153 ceph-mon[34747]: pgmap v747: 129 pgs: 129 active+clean; 5.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:24:57.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:57 smithi119 ceph-mon[34519]: pgmap v747: 129 pgs: 129 active+clean; 5.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:24:58.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:24:58.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:58 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2847591883' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:58 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2847591883' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:58.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:58 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2847591883' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:24:58.559 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.168+0000 7fd42b200640 1 -- 172.21.15.27:0/2018342542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c109c40 msgr2=0x7fd42c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.168+0000 7fd42b200640 1 --2- 172.21.15.27:0/2018342542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c109c40 0x7fd42c110470 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd424009ca0 tx=0x7fd42402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.168+0000 7fd42b200640 1 -- 172.21.15.27:0/2018342542 shutdown_connections 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.168+0000 7fd42b200640 1 --2- 172.21.15.27:0/2018342542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd42c1092f0 0x7fd42c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.168+0000 7fd42b200640 1 --2- 172.21.15.27:0/2018342542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c109c40 0x7fd42c110470 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.168+0000 7fd42b200640 1 --2- 172.21.15.27:0/2018342542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd42c1039b0 0x7fd42c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.168+0000 7fd42b200640 1 -- 172.21.15.27:0/2018342542 >> 172.21.15.27:0/2018342542 conn(0x7fd42c0ff400 msgr2=0x7fd42c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.169+0000 7fd42b200640 1 -- 172.21.15.27:0/2018342542 shutdown_connections 2024-09-21T07:24:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.169+0000 7fd42b200640 1 -- 172.21.15.27:0/2018342542 wait complete. 2024-09-21T07:24:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 Processor -- start 2024-09-21T07:24:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 -- start start 2024-09-21T07:24:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd42c1039b0 0x7fd42c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c1092f0 0x7fd42c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd42c109c40 0x7fd42c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c113900 con 0x7fd42c1092f0 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c113780 con 0x7fd42c1039b0 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.170+0000 7fd42b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c113a80 con 0x7fd42c109c40 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c1092f0 0x7fd42c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd42c1039b0 0x7fd42c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c1092f0 0x7fd42c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44672/0 (socket says 172.21.15.27:44672) 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 -- 172.21.15.27:0/402139189 learned_addr learned my addr 172.21.15.27:0/402139189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd42a800640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd42c109c40 0x7fd42c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 -- 172.21.15.27:0/402139189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd42c109c40 msgr2=0x7fd42c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd42c109c40 0x7fd42c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 -- 172.21.15.27:0/402139189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd42c1039b0 msgr2=0x7fd42c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd42c1039b0 0x7fd42c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.171+0000 7fd429400640 1 -- 172.21.15.27:0/402139189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c1a8530 con 0x7fd42c1092f0 2024-09-21T07:24:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.172+0000 7fd429400640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c1092f0 0x7fd42c075c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd41800e9c0 tx=0x7fd41800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.172+0000 7fd422a00640 1 -- 172.21.15.27:0/402139189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41800ccc0 con 0x7fd42c1092f0 2024-09-21T07:24:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.172+0000 7fd422a00640 1 -- 172.21.15.27:0/402139189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4180054d0 con 0x7fd42c1092f0 2024-09-21T07:24:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.172+0000 7fd422a00640 1 -- 172.21.15.27:0/402139189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418010430 con 0x7fd42c1092f0 2024-09-21T07:24:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.172+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42c1a8820 con 0x7fd42c1092f0 2024-09-21T07:24:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.172+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42c1a8d10 con 0x7fd42c1092f0 2024-09-21T07:24:59.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.175+0000 7fd437ebf740 1 Processor -- start 2024-09-21T07:24:59.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd437ebf740 1 -- start start 2024-09-21T07:24:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd422a00640 1 -- 172.21.15.27:0/402139189 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd418005020 con 0x7fd42c1092f0 2024-09-21T07:24:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f31891f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f318924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318929f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd437ebf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556f31808fb0 con 0x556f314eb070 2024-09-21T07:24:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd437ebf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556f317f83e0 con 0x556f314ca390 2024-09-21T07:24:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd437ebf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556f317fc1f0 con 0x556f314c21b0 2024-09-21T07:24:59.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd42a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f31891f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd42a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f31891f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44676/0 (socket says 172.21.15.27:44676) 2024-09-21T07:24:59.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd42a800640 1 -- 172.21.15.27:0/795759145 learned_addr learned my addr 172.21.15.27:0/795759145 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:59.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd429e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f318924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.176+0000 7fd429400640 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318929f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.177+0000 7fd42a800640 1 -- 172.21.15.27:0/795759145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 msgr2=0x556f318924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.177+0000 7fd42a800640 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f318924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.177+0000 7fd42a800640 1 -- 172.21.15.27:0/795759145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 msgr2=0x556f318929f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.177+0000 7fd42a800640 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318929f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.177+0000 7fd42a800640 1 -- 172.21.15.27:0/795759145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f317c2cd0 con 0x556f314eb070 2024-09-21T07:24:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.177+0000 7fd429e00640 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f318924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:24:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.177+0000 7fd42a800640 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f31891f70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd42402f410 tx=0x7fd42402f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4c1d85211c37c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.178+0000 7fd417400640 1 -- 172.21.15.27:0/795759145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424031cf0 con 0x556f314eb070 2024-09-21T07:24:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.178+0000 7fd417400640 1 -- 172.21.15.27:0/795759145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd424036030 con 0x556f314eb070 2024-09-21T07:24:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.178+0000 7fd417400640 1 -- 172.21.15.27:0/795759145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4240042a0 con 0x556f314eb070 2024-09-21T07:24:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.178+0000 7fd422a00640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4100b6680 0x7fd4100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.178+0000 7fd422a00640 1 -- 172.21.15.27:0/402139189 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fd4180e7e50 con 0x7fd42c1092f0 2024-09-21T07:24:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.178+0000 7fd429e00640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4100b6680 0x7fd4100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 -- 172.21.15.27:0/795759145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 msgr2=0x556f31891f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f31891f70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd42402f410 tx=0x7fd42402f440 comp rx=0 tx=0).stop 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd429e00640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4100b6680 0x7fd4100b8b90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x556f31890bf0 tx=0x7fd41c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 -- 172.21.15.27:0/795759145 shutdown_connections 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f318924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318929f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 --2- 172.21.15.27:0/795759145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f31891f70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 -- 172.21.15.27:0/795759145 >> 172.21.15.27:0/795759145 conn(0x556f314df460 msgr2=0x556f3188fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 -- 172.21.15.27:0/795759145 shutdown_connections 2024-09-21T07:24:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.179+0000 7fd437ebf740 1 -- 172.21.15.27:0/795759145 wait complete. 2024-09-21T07:24:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.180+0000 7fd437ebf740 1 -- start start 2024-09-21T07:24:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.180+0000 7fd437ebf740 1 Processor -- start 2024-09-21T07:24:59.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f3189d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314ca390 0x556f3189dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314eb070 0x556f318a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd437ebf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556f31659700 con 0x556f314c21b0 2024-09-21T07:24:59.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd437ebf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556f31793580 con 0x556f314eb070 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd437ebf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556f318a0570 con 0x556f314ca390 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd437ebf740 10 client.0 _ll_register_callbacks cb 0x556f317a3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd429e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f3189d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd429400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314ca390 0x556f3189dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd429400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314ca390 0x556f3189dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56590/0 (socket says 172.21.15.27:56590) 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd429400640 1 -- 172.21.15.27:0/3465474326 learned_addr learned my addr 172.21.15.27:0/3465474326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.181+0000 7fd42a800640 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314eb070 0x556f318a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd429400640 1 -- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314eb070 msgr2=0x556f318a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd429400640 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314eb070 0x556f318a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd429400640 1 -- 172.21.15.27:0/3465474326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 msgr2=0x556f3189d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd429400640 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f3189d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd429400640 1 -- 172.21.15.27:0/3465474326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f31556650 con 0x556f314ca390 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd429400640 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314ca390 0x556f3189dcf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd4180e5250 tx=0x7fd4180eb590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3129c7449739c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:59.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd40be00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:24:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd40be00640 1 -- 172.21.15.27:0/3465474326 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180ad6c0 con 0x556f314ca390 2024-09-21T07:24:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd40be00640 1 -- 172.21.15.27:0/3465474326 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fd4180acb50 con 0x556f314ca390 2024-09-21T07:24:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f31795b90 con 0x556f314ca390 2024-09-21T07:24:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.182+0000 7fd40be00640 1 -- 172.21.15.27:0/3465474326 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4180a9060 con 0x556f314ca390 2024-09-21T07:24:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 10 client.28939 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f317b0950 con 0x556f314ca390 2024-09-21T07:24:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd40be00640 1 -- 172.21.15.27:0/3465474326 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556f31795b90 con 0x556f314ca390 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd40be00640 1 -- 172.21.15.27:0/3465474326 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd4180a9240 con 0x556f314ca390 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 10 client.28939 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 10 client.28939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 20 client.28939 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f314a99f0 0x556f318af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 4 client.28939 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.183+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3180af00 con 0x556f314a99f0 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.184+0000 7fd429e00640 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f314a99f0 0x556f318af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.184+0000 7fd429e00640 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f314a99f0 0x556f318af620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=554b33bd233809b8 server_cookie=6d6f06c0728e2da4 in_seq=0 out_seq=0 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.184+0000 7fd40be00640 10 client.28939 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd40be00640 1 -- 172.21.15.27:0/3465474326 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f3180af00 con 0x556f314a99f0 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd40be00640 10 client.28939 handle_command_reply: tid=0 2024-09-21T07:24:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 client.28939 shutdown 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314ca390 msgr2=0x556f3189dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314ca390 0x556f3189dcf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd4180e5250 tx=0x7fd4180eb590 comp rx=0 tx=0).stop 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 shutdown_connections 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f314a99f0 0x556f318af620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314ca390 0x556f3189dcf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314eb070 0x556f318a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 --2- 172.21.15.27:0/3465474326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f3189d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 >> 172.21.15.27:0/3465474326 conn(0x556f314df460 msgr2=0x556f31895150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.185+0000 7fd40be00640 0 client.28939 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.186+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 shutdown_connections 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.186+0000 7fd437ebf740 1 -- 172.21.15.27:0/3465474326 wait complete. 2024-09-21T07:24:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.186+0000 7fd437ebf740 20 client.28939 trim_cache size 0 max 16384 2024-09-21T07:24:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 Processor -- start 2024-09-21T07:24:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 -- start start 2024-09-21T07:24:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f3189f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f318a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c1b5660 con 0x556f314eb070 2024-09-21T07:24:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c1b2f30 con 0x556f314ca390 2024-09-21T07:24:59.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.200+0000 7fd437ebf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c1b0da0 con 0x556f314c21b0 2024-09-21T07:24:59.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd42a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f318a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318936b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48460/0 (socket says 172.21.15.27:48460) 2024-09-21T07:24:59.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429400640 1 -- 172.21.15.27:0/1263969679 learned_addr learned my addr 172.21.15.27:0/1263969679 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:59.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd42a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f318a97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44702/0 (socket says 172.21.15.27:44702) 2024-09-21T07:24:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429e00640 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f3189f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429e00640 1 -- 172.21.15.27:0/1263969679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 msgr2=0x556f318936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429e00640 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429e00640 1 -- 172.21.15.27:0/1263969679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 msgr2=0x556f318a97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429e00640 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f318a97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.201+0000 7fd429e00640 1 -- 172.21.15.27:0/1263969679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f318903a0 con 0x556f314c21b0 2024-09-21T07:24:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.202+0000 7fd429e00640 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f3189f690 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x556f318a0000 tx=0x7fd41c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f817d4f5eb4c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.202+0000 7fd416a00640 1 -- 172.21.15.27:0/1263969679 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c031440 con 0x556f314c21b0 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.202+0000 7fd416a00640 1 -- 172.21.15.27:0/1263969679 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c02a8c0 con 0x556f314c21b0 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.202+0000 7fd416a00640 1 -- 172.21.15.27:0/1263969679 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00f740 con 0x556f314c21b0 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 -- 172.21.15.27:0/1263969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 msgr2=0x556f3189f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f3189f690 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x556f318a0000 tx=0x7fd41c01b000 comp rx=0 tx=0).stop 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 -- 172.21.15.27:0/1263969679 shutdown_connections 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314eb070 0x556f318a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 --2- 172.21.15.27:0/1263969679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314c21b0 0x556f3189f690 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 -- 172.21.15.27:0/1263969679 >> 172.21.15.27:0/1263969679 conn(0x556f314df460 msgr2=0x556f31892fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 -- 172.21.15.27:0/1263969679 shutdown_connections 2024-09-21T07:24:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 -- 172.21.15.27:0/1263969679 wait complete. 2024-09-21T07:24:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 -- start start 2024-09-21T07:24:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.203+0000 7fd437ebf740 1 Processor -- start 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f318a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd437ebf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314eb070 0x556f31912f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd437ebf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c1b13f0 con 0x556f314c21b0 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd437ebf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c1b2860 con 0x556f314ca390 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd437ebf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c1b1ac0 con 0x556f314eb070 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd437ebf740 10 client.0 _ll_register_callbacks cb 0x7fd418004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd429e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f318a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd429400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd429e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f318a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44710/0 (socket says 172.21.15.27:44710) 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd429e00640 1 -- 172.21.15.27:0/73738318 learned_addr learned my addr 172.21.15.27:0/73738318 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.204+0000 7fd42a800640 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314eb070 0x556f31912f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd429400640 1 -- 172.21.15.27:0/73738318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314eb070 msgr2=0x556f31912f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd429400640 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314eb070 0x556f31912f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd429400640 1 -- 172.21.15.27:0/73738318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 msgr2=0x556f318a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd429400640 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f318a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd429400640 1 -- 172.21.15.27:0/73738318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f31894530 con 0x556f314ca390 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd429400640 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318a26c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd4180049d0 tx=0x7fd4180acd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677c301cf68671c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd40be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.205+0000 7fd40be00640 1 -- 172.21.15.27:0/73738318 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41800ce60 con 0x556f314ca390 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd40be00640 1 -- 172.21.15.27:0/73738318 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fd4180ac5c0 con 0x556f314ca390 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c02a8c0 con 0x556f314ca390 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd40be00640 1 -- 172.21.15.27:0/73738318 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4180ad060 con 0x556f314ca390 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 10 client.28964 fetch_fsmap learned FSMap version 27 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f3191a750 con 0x556f314ca390 2024-09-21T07:24:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd40be00640 1 -- 172.21.15.27:0/73738318 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c02a8c0 con 0x556f314ca390 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd40be00640 1 -- 172.21.15.27:0/73738318 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd4180ac7c0 con 0x556f314ca390 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 10 client.28964 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 10 client.28964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 20 client.28964 populate_metadata read hostname 'smithi027' 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f3147eda0 0x556f3191cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 4 client.28964 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.206+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f318069b0 con 0x556f3147eda0 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.207+0000 7fd429e00640 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f3147eda0 0x556f3191cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.207+0000 7fd429e00640 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f3147eda0 0x556f3191cf60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecea82207c64040c server_cookie=63bd0cdd123c7d6f in_seq=0 out_seq=0 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.207+0000 7fd40be00640 10 client.28964 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd40be00640 1 -- 172.21.15.27:0/73738318 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f318069b0 con 0x556f3147eda0 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd40be00640 10 client.28964 handle_command_reply: tid=0 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 client.28964 shutdown 2024-09-21T07:24:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 msgr2=0x556f318a26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318a26c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd4180049d0 tx=0x7fd4180acd80 comp rx=0 tx=0).stop 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 shutdown_connections 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556f3147eda0 0x556f3191cf60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f314eb070 0x556f31912f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f314ca390 0x556f318a26c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 --2- 172.21.15.27:0/73738318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f314c21b0 0x556f318a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd40be00640 0 client.28964 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 >> 172.21.15.27:0/73738318 conn(0x556f314df460 msgr2=0x556f31893810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 shutdown_connections 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 1 -- 172.21.15.27:0/73738318 wait complete. 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.208+0000 7fd437ebf740 20 client.28964 trim_cache size 0 max 16384 2024-09-21T07:24:59.212 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:24:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4100b6680 msgr2=0x7fd4100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4100b6680 0x7fd4100b8b90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x556f31890bf0 tx=0x7fd41c005f90 comp rx=0 tx=0).stop 2024-09-21T07:24:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c1092f0 msgr2=0x7fd42c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:24:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c1092f0 0x7fd42c075c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd41800e9c0 tx=0x7fd41800ee80 comp rx=0 tx=0).stop 2024-09-21T07:24:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 shutdown_connections 2024-09-21T07:24:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd42c109c40 0x7fd42c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4100b6680 0x7fd4100b8b90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd42c1092f0 0x7fd42c075c50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 --2- 172.21.15.27:0/402139189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd42c1039b0 0x7fd42c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:24:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 >> 172.21.15.27:0/402139189 conn(0x7fd42c0ff400 msgr2=0x7fd42c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:24:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 shutdown_connections 2024-09-21T07:24:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:24:59.209+0000 7fd42b200640 1 -- 172.21.15.27:0/402139189 wait complete. 2024-09-21T07:24:59.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:24:59 smithi027 ceph-mon[30779]: pgmap v748: 129 pgs: 129 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-09-21T07:24:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:24:59 smithi153 ceph-mon[34747]: pgmap v748: 129 pgs: 129 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-09-21T07:24:59.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:24:59 smithi119 ceph-mon[34519]: pgmap v748: 129 pgs: 129 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-09-21T07:25:00.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:25:00.634 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 -- 172.21.15.27:0/3932538701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea410b6d0 msgr2=0x7faea410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 --2- 172.21.15.27:0/3932538701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea410b6d0 0x7faea410bae0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009900 tx=0x7fae8c02a400 comp rx=0 tx=0).stop 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 -- 172.21.15.27:0/3932538701 shutdown_connections 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 --2- 172.21.15.27:0/3932538701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 0x7faea4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 --2- 172.21.15.27:0/3932538701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea410b6d0 0x7faea410bae0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 --2- 172.21.15.27:0/3932538701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea4105d90 0x7faea41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 -- 172.21.15.27:0/3932538701 >> 172.21.15.27:0/3932538701 conn(0x7faea4101860 msgr2=0x7faea4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.254+0000 7faea9c00640 1 -- 172.21.15.27:0/3932538701 shutdown_connections 2024-09-21T07:25:01.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.255+0000 7faea9c00640 1 -- 172.21.15.27:0/3932538701 wait complete. 2024-09-21T07:25:01.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 Processor -- start 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 -- start start 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4105d90 0x7faea419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea410b6d0 0x7faea419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 0x7faea41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faea4115ce0 con 0x7faea4105d90 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faea4115b60 con 0x7faea410b6d0 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.256+0000 7faea9c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faea4115e60 con 0x7faea410c020 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4105d90 0x7faea419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 0x7faea41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7fae9a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea410b6d0 0x7faea419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4105d90 0x7faea419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44738/0 (socket says 172.21.15.27:44738) 2024-09-21T07:25:01.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7fae9a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea410b6d0 0x7faea419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48486/0 (socket says 172.21.15.27:48486) 2024-09-21T07:25:01.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 0x7faea41a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56612/0 (socket says 172.21.15.27:56612) 2024-09-21T07:25:01.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea2a00640 1 -- 172.21.15.27:0/3655729694 learned_addr learned my addr 172.21.15.27:0/3655729694 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:01.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea3400640 1 -- 172.21.15.27:0/3655729694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea410b6d0 msgr2=0x7faea419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea3400640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea410b6d0 0x7faea419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea3400640 1 -- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4105d90 msgr2=0x7faea419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea3400640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4105d90 0x7faea419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.257+0000 7faea3400640 1 -- 172.21.15.27:0/3655729694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea41a2640 con 0x7faea410c020 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.258+0000 7faea3400640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 0x7faea41a1f10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00e970 tx=0x7fae9c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.258+0000 7fae9be00640 1 -- 172.21.15.27:0/3655729694 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00ccc0 con 0x7faea410c020 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.258+0000 7fae9be00640 1 -- 172.21.15.27:0/3655729694 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9c005020 con 0x7faea410c020 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.258+0000 7fae9be00640 1 -- 172.21.15.27:0/3655729694 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004a00 con 0x7faea410c020 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.258+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea41a2930 con 0x7faea410c020 2024-09-21T07:25:01.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.258+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea41a9090 con 0x7faea410c020 2024-09-21T07:25:01.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.261+0000 7faeb08bd740 1 Processor -- start 2024-09-21T07:25:01.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.261+0000 7fae9be00640 1 -- 172.21.15.27:0/3655729694 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fae9c004ba0 con 0x7faea410c020 2024-09-21T07:25:01.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faeb08bd740 1 -- start start 2024-09-21T07:25:01.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cfac070 0x559d7d352ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d3530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d353620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faeb08bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559d7d2c9fb0 con 0x559d7cfac070 2024-09-21T07:25:01.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faeb08bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559d7d2b93e0 con 0x559d7cf8b390 2024-09-21T07:25:01.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faeb08bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559d7d2bd1f0 con 0x559d7cf831b0 2024-09-21T07:25:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7fae9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cfac070 0x559d7d352ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faea2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d3530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7fae9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cfac070 0x559d7d352ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44750/0 (socket says 172.21.15.27:44750) 2024-09-21T07:25:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7faea2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d3530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56628/0 (socket says 172.21.15.27:56628) 2024-09-21T07:25:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.262+0000 7fae9a000640 1 -- 172.21.15.27:0/1622050499 learned_addr learned my addr 172.21.15.27:0/1622050499 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.263+0000 7fae9a000640 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d353620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.263+0000 7faea2a00640 1 -- 172.21.15.27:0/1622050499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 msgr2=0x559d7d353620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.263+0000 7faea2a00640 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d353620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.263+0000 7faea2a00640 1 -- 172.21.15.27:0/1622050499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cfac070 msgr2=0x559d7d352ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.263+0000 7faea2a00640 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cfac070 0x559d7d352ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.263+0000 7faea2a00640 1 -- 172.21.15.27:0/1622050499 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7d283cd0 con 0x559d7cf831b0 2024-09-21T07:25:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.263+0000 7faea2a00640 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d3530e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae940114f0 tx=0x7fae94009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc702acdde6b1bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.264+0000 7fae93e00640 1 -- 172.21.15.27:0/1622050499 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9401e070 con 0x559d7cf831b0 2024-09-21T07:25:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.264+0000 7fae93e00640 1 -- 172.21.15.27:0/1622050499 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae94011310 con 0x559d7cf831b0 2024-09-21T07:25:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.264+0000 7fae93e00640 1 -- 172.21.15.27:0/1622050499 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940198d0 con 0x559d7cf831b0 2024-09-21T07:25:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.264+0000 7fae9be00640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fae880b6700 0x7fae880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.264+0000 7fae9be00640 1 -- 172.21.15.27:0/3655729694 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fae9c0e79b0 con 0x7faea410c020 2024-09-21T07:25:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.264+0000 7fae9a000640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fae880b6700 0x7fae880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 -- 172.21.15.27:0/1622050499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 msgr2=0x559d7d3530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d3530e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae940114f0 tx=0x7fae94009f90 comp rx=0 tx=0).stop 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 -- 172.21.15.27:0/1622050499 shutdown_connections 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d3530e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d353620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 --2- 172.21.15.27:0/1622050499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cfac070 0x559d7d352ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 -- 172.21.15.27:0/1622050499 >> 172.21.15.27:0/1622050499 conn(0x559d7cfa0460 msgr2=0x559d7d350a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 -- 172.21.15.27:0/1622050499 shutdown_connections 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7fae9a000640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fae880b6700 0x7fae880b8c10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0057c0 tx=0x7fae8c000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.265+0000 7faeb08bd740 1 -- 172.21.15.27:0/1622050499 wait complete. 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 -- start start 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 Processor -- start 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d35e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d35eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cfac070 0x559d7d36a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559d7d11a700 con 0x559d7cf8b390 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559d7d254580 con 0x559d7cfac070 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559d7d361380 con 0x559d7cf831b0 2024-09-21T07:25:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.266+0000 7faeb08bd740 10 client.0 _ll_register_callbacks cb 0x559d7d264470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cfac070 0x559d7d36a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d35e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d35e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56630/0 (socket says 172.21.15.27:56630) 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cfac070 0x559d7d36a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48502/0 (socket says 172.21.15.27:48502) 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea2a00640 1 -- 172.21.15.27:0/2915115005 learned_addr learned my addr 172.21.15.27:0/2915115005 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7fae9a000640 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d35eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 -- 172.21.15.27:0/2915115005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 msgr2=0x559d7d35e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d35e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 -- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 msgr2=0x559d7d35eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d35eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 -- 172.21.15.27:0/2915115005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7d017650 con 0x559d7cfac070 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7faea3400640 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cfac070 0x559d7d36a670 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0e4cf0 tx=0x7fae9c0acb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8abf9be4a82b576a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.267+0000 7fae90c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7fae90c00640 1 -- 172.21.15.27:0/2915115005 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c01c040 con 0x559d7cfac070 2024-09-21T07:25:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7fae90c00640 1 -- 172.21.15.27:0/2915115005 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fae9c0ac850 con 0x559d7cfac070 2024-09-21T07:25:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d7d256b90 con 0x559d7cfac070 2024-09-21T07:25:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7fae90c00640 1 -- 172.21.15.27:0/2915115005 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae9c0b1ed0 con 0x559d7cfac070 2024-09-21T07:25:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7faeb08bd740 10 client.28976 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559d7d271950 con 0x559d7cfac070 2024-09-21T07:25:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7fae90c00640 1 -- 172.21.15.27:0/2915115005 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559d7d256b90 con 0x559d7cfac070 2024-09-21T07:25:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.268+0000 7fae90c00640 1 -- 172.21.15.27:0/2915115005 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fae9c0a8dd0 con 0x559d7cfac070 2024-09-21T07:25:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.269+0000 7faeb08bd740 10 client.28976 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.269+0000 7faeb08bd740 10 client.28976 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.269+0000 7faeb08bd740 20 client.28976 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.269+0000 7faeb08bd740 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf6a9f0 0x559d7d370660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.269+0000 7faeb08bd740 4 client.28976 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.269+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d7d2cbf00 con 0x559d7cf6a9f0 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.269+0000 7faea2a00640 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf6a9f0 0x559d7d370660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.270+0000 7faea2a00640 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf6a9f0 0x559d7d370660 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dcb292e12cf2058 server_cookie=9747139d1997278 in_seq=0 out_seq=0 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.270+0000 7fae90c00640 10 client.28976 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.270+0000 7fae90c00640 1 -- 172.21.15.27:0/2915115005 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d7d2cbf00 con 0x559d7cf6a9f0 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.270+0000 7fae90c00640 10 client.28976 handle_command_reply: tid=0 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.270+0000 7faeb08bd740 1 client.28976 shutdown 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cfac070 msgr2=0x559d7d36a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cfac070 0x559d7d36a670 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0e4cf0 tx=0x7fae9c0acb60 comp rx=0 tx=0).stop 2024-09-21T07:25:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 shutdown_connections 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf6a9f0 0x559d7d370660 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cfac070 0x559d7d36a670 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7fae90c00640 0 client.28976 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d35eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 --2- 172.21.15.27:0/2915115005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cf831b0 0x559d7d35e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 >> 172.21.15.27:0/2915115005 conn(0x559d7cfa0460 msgr2=0x559d7d355d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 shutdown_connections 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 1 -- 172.21.15.27:0/2915115005 wait complete. 2024-09-21T07:25:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.271+0000 7faeb08bd740 20 client.28976 trim_cache size 0 max 16384 2024-09-21T07:25:01.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.285+0000 7faeb08bd740 1 Processor -- start 2024-09-21T07:25:01.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.285+0000 7faeb08bd740 1 -- start start 2024-09-21T07:25:01.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf831b0 0x559d7d36a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d35b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d35bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faeb08bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b5880 con 0x559d7cf831b0 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faeb08bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b3150 con 0x559d7cf8b390 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faeb08bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b0fc0 con 0x559d7cfac070 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faea2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf831b0 0x559d7d36a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faea2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf831b0 0x559d7d36a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44780/0 (socket says 172.21.15.27:44780) 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faea2a00640 1 -- 172.21.15.27:0/546579836 learned_addr learned my addr 172.21.15.27:0/546579836 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7faea3400640 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d35bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.286+0000 7fae9a000640 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d35b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7faea2a00640 1 -- 172.21.15.27:0/546579836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 msgr2=0x559d7d35bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7faea2a00640 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d35bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7faea2a00640 1 -- 172.21.15.27:0/546579836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 msgr2=0x559d7d35b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7faea2a00640 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d35b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7faea2a00640 1 -- 172.21.15.27:0/546579836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7d35a0d0 con 0x559d7cf831b0 2024-09-21T07:25:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7faea3400640 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d35bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:01.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7fae9a000640 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d35b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7faea2a00640 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf831b0 0x559d7d36a570 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fae94011e40 tx=0x7fae94004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44de7b11414a9327 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7fae93400640 1 -- 172.21.15.27:0/546579836 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9401e040 con 0x559d7cf831b0 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7fae93400640 1 -- 172.21.15.27:0/546579836 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9400fa70 con 0x559d7cf831b0 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.287+0000 7fae93400640 1 -- 172.21.15.27:0/546579836 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940197c0 con 0x559d7cf831b0 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 -- 172.21.15.27:0/546579836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf831b0 msgr2=0x559d7d36a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf831b0 0x559d7d36a570 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fae94011e40 tx=0x7fae94004c10 comp rx=0 tx=0).stop 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 -- 172.21.15.27:0/546579836 shutdown_connections 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d35bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf8b390 0x559d7d35b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 --2- 172.21.15.27:0/546579836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf831b0 0x559d7d36a570 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 -- 172.21.15.27:0/546579836 >> 172.21.15.27:0/546579836 conn(0x559d7cfa0460 msgr2=0x559d7d357cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:01.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 -- 172.21.15.27:0/546579836 shutdown_connections 2024-09-21T07:25:01.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.288+0000 7faeb08bd740 1 -- 172.21.15.27:0/546579836 wait complete. 2024-09-21T07:25:01.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.289+0000 7faeb08bd740 1 -- start start 2024-09-21T07:25:01.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.289+0000 7faeb08bd740 1 Processor -- start 2024-09-21T07:25:01.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.289+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf831b0 0x559d7d362f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.289+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d363570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.289+0000 7faeb08bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d3d3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7faeb08bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b1610 con 0x559d7cf8b390 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7faeb08bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b2a80 con 0x559d7cf831b0 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7faeb08bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faea41b1ce0 con 0x559d7cfac070 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7faeb08bd740 10 client.0 _ll_register_callbacks cb 0x7fae9c0a9340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d363570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d363570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44782/0 (socket says 172.21.15.27:44782) 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 -- 172.21.15.27:0/697808559 learned_addr learned my addr 172.21.15.27:0/697808559 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7faea2a00640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf831b0 0x559d7d362f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7faea3400640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d3d3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 -- 172.21.15.27:0/697808559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 msgr2=0x559d7d3d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d3d3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 -- 172.21.15.27:0/697808559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf831b0 msgr2=0x559d7d362f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf831b0 0x559d7d362f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7fae9a000640 1 -- 172.21.15.27:0/697808559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7d3d4660 con 0x559d7cf8b390 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.290+0000 7faea2a00640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf831b0 0x559d7d362f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.291+0000 7fae9a000640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d363570 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009900 tx=0x7fae8c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01f3d77c57e9c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.291+0000 7fae90c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.291+0000 7fae90c00640 1 -- 172.21.15.27:0/697808559 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c050070 con 0x559d7cf8b390 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.291+0000 7fae90c00640 1 -- 172.21.15.27:0/697808559 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7fae8c005240 con 0x559d7cf8b390 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.291+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9400fa70 con 0x559d7cf8b390 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7fae90c00640 1 -- 172.21.15.27:0/697808559 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae8c035040 con 0x559d7cf8b390 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 10 client.18972 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559d7d3dba60 con 0x559d7cf8b390 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7fae90c00640 1 -- 172.21.15.27:0/697808559 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9400fa70 con 0x559d7cf8b390 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7fae90c00640 1 -- 172.21.15.27:0/697808559 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fae8c0035f0 con 0x559d7cf8b390 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 10 client.18972 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 10 client.18972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 20 client.18972 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf3fda0 0x559d7d3de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 4 client.18972 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.292+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d7d2c79b0 con 0x559d7cf3fda0 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.293+0000 7faea2a00640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf3fda0 0x559d7d3de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.293+0000 7faea2a00640 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf3fda0 0x559d7d3de250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=824afe8209edb11 server_cookie=7761d85093fb7b78 in_seq=0 out_seq=0 2024-09-21T07:25:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.293+0000 7fae90c00640 10 client.18972 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7fae90c00640 1 -- 172.21.15.27:0/697808559 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d7d2c79b0 con 0x559d7cf3fda0 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7fae90c00640 10 client.18972 handle_command_reply: tid=0 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 client.18972 shutdown 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 msgr2=0x559d7d363570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d363570 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009900 tx=0x7fae8c031400 comp rx=0 tx=0).stop 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 shutdown_connections 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559d7cf3fda0 0x559d7d3de250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d7cfac070 0x559d7d3d3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d7cf8b390 0x559d7d363570 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 --2- 172.21.15.27:0/697808559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d7cf831b0 0x559d7d362f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7fae90c00640 0 client.18972 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 >> 172.21.15.27:0/697808559 conn(0x559d7cfa0460 msgr2=0x559d7d35cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 shutdown_connections 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 1 -- 172.21.15.27:0/697808559 wait complete. 2024-09-21T07:25:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.294+0000 7faeb08bd740 20 client.18972 trim_cache size 0 max 16384 2024-09-21T07:25:01.299 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:25:01.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fae880b6700 msgr2=0x7fae880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fae880b6700 0x7fae880b8c10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0057c0 tx=0x7fae8c000bc0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 msgr2=0x7faea41a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 0x7faea41a1f10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00e970 tx=0x7fae9c00ee30 comp rx=0 tx=0).stop 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 shutdown_connections 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea410c020 0x7faea41a1f10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fae880b6700 0x7fae880b8c10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea410b6d0 0x7faea419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 --2- 172.21.15.27:0/3655729694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4105d90 0x7faea419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 >> 172.21.15.27:0/3655729694 conn(0x7faea4101860 msgr2=0x7faea4108d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 shutdown_connections 2024-09-21T07:25:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:01.295+0000 7faea9c00640 1 -- 172.21.15.27:0/3655729694 wait complete. 2024-09-21T07:25:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:01 smithi027 ceph-mon[30779]: pgmap v749: 129 pgs: 129 active+clean; 1.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-09-21T07:25:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:01 smithi153 ceph-mon[34747]: pgmap v749: 129 pgs: 129 active+clean; 1.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-09-21T07:25:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:01 smithi119 ceph-mon[34519]: pgmap v749: 129 pgs: 129 active+clean; 1.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-09-21T07:25:02.143 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:25:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 -- 172.21.15.27:0/1671146594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab0105bc0 msgr2=0x7f9ab0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/1671146594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab0105bc0 0x7f9ab0105fd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009900 tx=0x7f9aa002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 -- 172.21.15.27:0/1671146594 shutdown_connections 2024-09-21T07:25:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/1671146594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab010be50 0x7f9ab0112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/1671146594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ab010b500 0x7f9ab010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/1671146594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab0105bc0 0x7f9ab0105fd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 -- 172.21.15.27:0/1671146594 >> 172.21.15.27:0/1671146594 conn(0x7f9ab0101670 msgr2=0x7f9ab0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 -- 172.21.15.27:0/1671146594 shutdown_connections 2024-09-21T07:25:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.159+0000 7f9ab4e00640 1 -- 172.21.15.27:0/1671146594 wait complete. 2024-09-21T07:25:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.160+0000 7f9ab4e00640 1 Processor -- start 2024-09-21T07:25:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9ab4e00640 1 -- start start 2024-09-21T07:25:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9ab4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab0105bc0 0x7f9ab019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9ab4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab010b500 0x7f9ab019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9ab4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ab010be50 0x7f9ab01a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9ab4e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0115b10 con 0x7f9ab010b500 2024-09-21T07:25:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9ab4e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0115990 con 0x7f9ab010be50 2024-09-21T07:25:03.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9ab4e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0115c90 con 0x7f9ab0105bc0 2024-09-21T07:25:03.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab010b500 0x7f9ab019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab010b500 0x7f9ab019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44808/0 (socket says 172.21.15.27:44808) 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9aa7e00640 1 -- 172.21.15.27:0/34419592 learned_addr learned my addr 172.21.15.27:0/34419592 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.161+0000 7f9aaf400640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab0105bc0 0x7f9ab019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aafe00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ab010be50 0x7f9ab01a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aa7e00640 1 -- 172.21.15.27:0/34419592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab0105bc0 msgr2=0x7f9ab019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab0105bc0 0x7f9ab019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aa7e00640 1 -- 172.21.15.27:0/34419592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ab010be50 msgr2=0x7f9ab01a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ab010be50 0x7f9ab01a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aa7e00640 1 -- 172.21.15.27:0/34419592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab01a2470 con 0x7f9ab010b500 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aafe00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ab010be50 0x7f9ab01a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab010b500 0x7f9ab019caa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800d950 tx=0x7f9a9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.162+0000 7f9aacc00640 1 -- 172.21.15.27:0/34419592 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98017070 con 0x7f9ab010b500 2024-09-21T07:25:03.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.163+0000 7f9aacc00640 1 -- 172.21.15.27:0/34419592 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a98009c30 con 0x7f9ab010b500 2024-09-21T07:25:03.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.163+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab01a2760 con 0x7f9ab010b500 2024-09-21T07:25:03.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.163+0000 7f9aacc00640 1 -- 172.21.15.27:0/34419592 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a980047a0 con 0x7f9ab010b500 2024-09-21T07:25:03.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.163+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01a8ec0 con 0x7f9ab010b500 2024-09-21T07:25:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.166+0000 7f9abd077740 1 Processor -- start 2024-09-21T07:25:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.166+0000 7f9abd077740 1 -- start start 2024-09-21T07:25:03.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.166+0000 7f9aacc00640 1 -- 172.21.15.27:0/34419592 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9a98004020 con 0x7f9ab010b500 2024-09-21T07:25:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.166+0000 7f9abd077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096086070 0x56209642cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9abd077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x56209642d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9abd077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096065390 0x56209642d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9abd077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5620963a3fb0 con 0x56209605d1b0 2024-09-21T07:25:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9abd077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5620963933e0 con 0x562096065390 2024-09-21T07:25:03.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9abd077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5620963971f0 con 0x562096086070 2024-09-21T07:25:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x56209642d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096086070 0x56209642cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096086070 0x56209642cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56688/0 (socket says 172.21.15.27:56688) 2024-09-21T07:25:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096065390 0x56209642d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 -- 172.21.15.27:0/369803065 learned_addr learned my addr 172.21.15.27:0/369803065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 -- 172.21.15.27:0/369803065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096065390 msgr2=0x56209642d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 --2- 172.21.15.27:0/369803065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096065390 0x56209642d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 -- 172.21.15.27:0/369803065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 msgr2=0x56209642d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 --2- 172.21.15.27:0/369803065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x56209642d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.167+0000 7f9aafe00640 1 -- 172.21.15.27:0/369803065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209635dcd0 con 0x562096086070 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.168+0000 7f9aafe00640 1 --2- 172.21.15.27:0/369803065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096086070 0x56209642cd30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009f30 tx=0x7f9aa8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf2be9dfdb629fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.168+0000 7f9aa4c00640 1 -- 172.21.15.27:0/369803065 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8016070 con 0x562096086070 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.168+0000 7f9aa4c00640 1 -- 172.21.15.27:0/369803065 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa801bb10 con 0x562096086070 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.168+0000 7f9aa4c00640 1 -- 172.21.15.27:0/369803065 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8012c10 con 0x562096086070 2024-09-21T07:25:03.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.168+0000 7f9aacc00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a940b6700 0x7f9a940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9aacc00640 1 -- 172.21.15.27:0/34419592 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f9a980e77e0 con 0x7f9ab010b500 2024-09-21T07:25:03.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9aaf400640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a940b6700 0x7f9a940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 -- 172.21.15.27:0/369803065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096086070 msgr2=0x56209642cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 --2- 172.21.15.27:0/369803065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096086070 0x56209642cd30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009f30 tx=0x7f9aa8010800 comp rx=0 tx=0).stop 2024-09-21T07:25:03.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 -- 172.21.15.27:0/369803065 shutdown_connections 2024-09-21T07:25:03.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 --2- 172.21.15.27:0/369803065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096065390 0x56209642d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 --2- 172.21.15.27:0/369803065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x56209642d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 --2- 172.21.15.27:0/369803065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096086070 0x56209642cd30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 -- 172.21.15.27:0/369803065 >> 172.21.15.27:0/369803065 conn(0x56209607a460 msgr2=0x56209642abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.169+0000 7f9abd077740 1 -- 172.21.15.27:0/369803065 shutdown_connections 2024-09-21T07:25:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.170+0000 7f9abd077740 1 -- 172.21.15.27:0/369803065 wait complete. 2024-09-21T07:25:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.170+0000 7f9aaf400640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a940b6700 0x7f9a940b8c10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0031440 tx=0x7f9aa0009610 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.170+0000 7f9abd077740 1 -- start start 2024-09-21T07:25:03.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.170+0000 7f9abd077740 1 Processor -- start 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9abd077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56209605d1b0 0x5620964388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9abd077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096438e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9abd077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096086070 0x562096444830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9abd077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5620961f4700 con 0x562096086070 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9abd077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56209632e580 con 0x56209605d1b0 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9abd077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56209643b510 con 0x562096065390 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9abd077740 10 client.0 _ll_register_callbacks cb 0x56209633e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56209605d1b0 0x5620964388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096086070 0x562096444830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56209605d1b0 0x5620964388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48560/0 (socket says 172.21.15.27:48560) 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9aaf400640 1 -- 172.21.15.27:0/2791748521 learned_addr learned my addr 172.21.15.27:0/2791748521 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.171+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096438e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9aaf400640 1 -- 172.21.15.27:0/2791748521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 msgr2=0x562096438e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9aaf400640 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096438e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9aaf400640 1 -- 172.21.15.27:0/2791748521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096086070 msgr2=0x562096444830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9aaf400640 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096086070 0x562096444830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9aaf400640 1 -- 172.21.15.27:0/2791748521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620960f1650 con 0x56209605d1b0 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9aaf400640 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56209605d1b0 0x5620964388d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0033000 tx=0x7f9aa0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82456eb22fb9669f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:03.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9a9d600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:03.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.172+0000 7f9a9d600640 1 -- 172.21.15.27:0/2791748521 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa004d070 con 0x56209605d1b0 2024-09-21T07:25:03.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9a9d600640 1 -- 172.21.15.27:0/2791748521 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f9aa0005440 con 0x56209605d1b0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562096330b90 con 0x56209605d1b0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9a9d600640 1 -- 172.21.15.27:0/2791748521 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9aa0059050 con 0x56209605d1b0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9abd077740 10 client.28994 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56209634b950 con 0x56209605d1b0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9a9d600640 1 -- 172.21.15.27:0/2791748521 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562096330b90 con 0x56209605d1b0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9a9d600640 1 -- 172.21.15.27:0/2791748521 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9aa0003930 con 0x56209605d1b0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9abd077740 10 client.28994 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9abd077740 10 client.28994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.173+0000 7f9abd077740 20 client.28994 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.174+0000 7f9abd077740 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5620960449f0 0x56209644a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.174+0000 7f9abd077740 4 client.28994 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.174+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620963a5f00 con 0x5620960449f0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.174+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5620960449f0 0x56209644a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.174+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5620960449f0 0x56209644a890 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2d00af8c5d55f7 server_cookie=522c9f72967f097c in_seq=0 out_seq=0 2024-09-21T07:25:03.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.174+0000 7f9a9d600640 10 client.28994 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:03.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.175+0000 7f9a9d600640 1 -- 172.21.15.27:0/2791748521 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620963a5f00 con 0x5620960449f0 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.175+0000 7f9a9d600640 10 client.28994 handle_command_reply: tid=0 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.175+0000 7f9abd077740 1 client.28994 shutdown 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56209605d1b0 msgr2=0x5620964388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56209605d1b0 0x5620964388d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0033000 tx=0x7f9aa0009260 comp rx=0 tx=0).stop 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 shutdown_connections 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5620960449f0 0x56209644a890 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096438e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096086070 0x562096444830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9a9d600640 0 client.28994 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 --2- 172.21.15.27:0/2791748521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56209605d1b0 0x5620964388d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 >> 172.21.15.27:0/2791748521 conn(0x56209607a460 msgr2=0x56209642fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 shutdown_connections 2024-09-21T07:25:03.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 1 -- 172.21.15.27:0/2791748521 wait complete. 2024-09-21T07:25:03.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.176+0000 7f9abd077740 20 client.28994 trim_cache size 0 max 16384 2024-09-21T07:25:03.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.190+0000 7f9abd077740 1 Processor -- start 2024-09-21T07:25:03.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.190+0000 7f9abd077740 1 -- start start 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.190+0000 7f9abd077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x562096444730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.190+0000 7f9abd077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096444c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9abd077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209642a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9abd077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01b57b0 con 0x56209605d1b0 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9abd077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01b3080 con 0x562096086070 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9abd077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01b0ef0 con 0x562096065390 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x562096444730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x562096444730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44832/0 (socket says 172.21.15.27:44832) 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 -- 172.21.15.27:0/2403545259 learned_addr learned my addr 172.21.15.27:0/2403545259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:03.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096444c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aafe00640 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209642a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 -- 172.21.15.27:0/2403545259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 msgr2=0x562096444c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096444c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 -- 172.21.15.27:0/2403545259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 msgr2=0x56209642a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209642a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.191+0000 7f9aaf400640 1 -- 172.21.15.27:0/2403545259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562096439990 con 0x56209605d1b0 2024-09-21T07:25:03.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.192+0000 7f9aaf400640 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x562096444730 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0003f70 tx=0x7f9aa00314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b937a56d03dfb37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:03.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.192+0000 7f9a9fe00640 1 -- 172.21.15.27:0/2403545259 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa004d070 con 0x56209605d1b0 2024-09-21T07:25:03.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.192+0000 7f9a9fe00640 1 -- 172.21.15.27:0/2403545259 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa00035f0 con 0x56209605d1b0 2024-09-21T07:25:03.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.192+0000 7f9a9fe00640 1 -- 172.21.15.27:0/2403545259 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0035040 con 0x56209605d1b0 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 -- 172.21.15.27:0/2403545259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 msgr2=0x562096444730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x562096444730 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0003f70 tx=0x7f9aa00314c0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 -- 172.21.15.27:0/2403545259 shutdown_connections 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562096065390 0x562096444c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209642a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 --2- 172.21.15.27:0/2403545259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56209605d1b0 0x562096444730 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 -- 172.21.15.27:0/2403545259 >> 172.21.15.27:0/2403545259 conn(0x56209607a460 msgr2=0x562096433880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 -- 172.21.15.27:0/2403545259 shutdown_connections 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.193+0000 7f9abd077740 1 -- 172.21.15.27:0/2403545259 wait complete. 2024-09-21T07:25:03.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 -- start start 2024-09-21T07:25:03.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 Processor -- start 2024-09-21T07:25:03.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56209605d1b0 0x5620964409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096065390 0x562096440ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209643b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01b1540 con 0x562096065390 2024-09-21T07:25:03.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01b29b0 con 0x562096086070 2024-09-21T07:25:03.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01b1c10 con 0x56209605d1b0 2024-09-21T07:25:03.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.194+0000 7f9abd077740 10 client.0 _ll_register_callbacks cb 0x7f9aa0031550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:03.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096065390 0x562096440ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56209605d1b0 0x5620964409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56209605d1b0 0x5620964409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56702/0 (socket says 172.21.15.27:56702) 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 -- 172.21.15.27:0/3805571123 learned_addr learned my addr 172.21.15.27:0/3805571123 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aafe00640 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209643b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 -- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 msgr2=0x56209643b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209643b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 -- 172.21.15.27:0/3805571123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096065390 msgr2=0x562096440ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096065390 0x562096440ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 -- 172.21.15.27:0/3805571123 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209643bce0 con 0x56209605d1b0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9aaf400640 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56209605d1b0 0x5620964409a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005530 tx=0x7f9aa0005560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f613f68758f06838 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.195+0000 7f9a9d600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.196+0000 7f9a9d600640 1 -- 172.21.15.27:0/3805571123 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa004d040 con 0x56209605d1b0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.196+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa00035f0 con 0x56209605d1b0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.196+0000 7f9a9d600640 1 -- 172.21.15.27:0/3805571123 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9461+0+0 (secure 0 0 0) 0x7f9aa00039b0 con 0x56209605d1b0 2024-09-21T07:25:03.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.196+0000 7f9a9d600640 1 -- 172.21.15.27:0/3805571123 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9aa0072050 con 0x56209605d1b0 2024-09-21T07:25:03.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.196+0000 7f9abd077740 10 client.28981 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:03.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.196+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620964b5a50 con 0x56209605d1b0 2024-09-21T07:25:03.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.196+0000 7f9a9d600640 1 -- 172.21.15.27:0/3805571123 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa00035f0 con 0x56209605d1b0 2024-09-21T07:25:03.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9a9d600640 1 -- 172.21.15.27:0/3805571123 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9aa006d030 con 0x56209605d1b0 2024-09-21T07:25:03.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9abd077740 10 client.28981 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:03.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9abd077740 10 client.28981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:03.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9abd077740 20 client.28981 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:03.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9abd077740 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562096019da0 0x5620964b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:03.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9abd077740 4 client.28981 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620963a19b0 con 0x562096019da0 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.197+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562096019da0 0x5620964b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9aa7e00640 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562096019da0 0x5620964b8110 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0233af6715df09e server_cookie=4d5452190063dc25 in_seq=0 out_seq=0 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9a9d600640 10 client.28981 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9a9d600640 1 -- 172.21.15.27:0/3805571123 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620963a19b0 con 0x562096019da0 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9a9d600640 10 client.28981 handle_command_reply: tid=0 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 client.28981 shutdown 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56209605d1b0 msgr2=0x5620964409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56209605d1b0 0x5620964409a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005530 tx=0x7f9aa0005560 comp rx=0 tx=0).stop 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 shutdown_connections 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562096019da0 0x5620964b8110 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562096086070 0x56209643b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562096065390 0x562096440ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 --2- 172.21.15.27:0/3805571123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56209605d1b0 0x5620964409a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 >> 172.21.15.27:0/3805571123 conn(0x56209607a460 msgr2=0x562096433880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.198+0000 7f9a9d600640 0 client.28981 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 shutdown_connections 2024-09-21T07:25:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9abd077740 1 -- 172.21.15.27:0/3805571123 wait complete. 2024-09-21T07:25:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9abd077740 20 client.28981 trim_cache size 0 max 16384 2024-09-21T07:25:03.203 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:25:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a940b6700 msgr2=0x7f9a940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a940b6700 0x7f9a940b8c10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0031440 tx=0x7f9aa0009610 comp rx=0 tx=0).stop 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab010b500 msgr2=0x7f9ab019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab010b500 0x7f9ab019caa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800d950 tx=0x7f9a9800de10 comp rx=0 tx=0).stop 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 shutdown_connections 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9a940b6700 0x7f9a940b8c10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ab010be50 0x7f9ab01a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ab010b500 0x7f9ab019caa0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 --2- 172.21.15.27:0/34419592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ab0105bc0 0x7f9ab019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.199+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 >> 172.21.15.27:0/34419592 conn(0x7f9ab0101670 msgr2=0x7f9ab0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.200+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 shutdown_connections 2024-09-21T07:25:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:03.200+0000 7f9ab4e00640 1 -- 172.21.15.27:0/34419592 wait complete. 2024-09-21T07:25:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:03 smithi027 ceph-mon[30779]: pgmap v750: 129 pgs: 129 active+clean; 1.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 69 KiB/s wr, 17 op/s 2024-09-21T07:25:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:03 smithi153 ceph-mon[34747]: pgmap v750: 129 pgs: 129 active+clean; 1.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 69 KiB/s wr, 17 op/s 2024-09-21T07:25:03.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:03 smithi119 ceph-mon[34519]: pgmap v750: 129 pgs: 129 active+clean; 1.7 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 69 KiB/s wr, 17 op/s 2024-09-21T07:25:05.171 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:25:05.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:25:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:05 smithi153 ceph-mon[34747]: pgmap v751: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 111 KiB/s wr, 33 op/s 2024-09-21T07:25:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:05 smithi153 ceph-mon[34747]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T07:25:05.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:05 smithi119 ceph-mon[34519]: pgmap v751: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 111 KiB/s wr, 33 op/s 2024-09-21T07:25:05.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:05 smithi119 ceph-mon[34519]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T07:25:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:05.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:05 smithi027 ceph-mon[30779]: pgmap v751: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 111 KiB/s wr, 33 op/s 2024-09-21T07:25:05.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:05 smithi027 ceph-mon[30779]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T07:25:06.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 -- 172.21.15.27:0/2633764732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58103990 msgr2=0x7fbf58103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 --2- 172.21.15.27:0/2633764732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58103990 0x7fbf58103da0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009900 tx=0x7fbf4802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 -- 172.21.15.27:0/2633764732 shutdown_connections 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 --2- 172.21.15.27:0/2633764732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 0x7fbf58110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 --2- 172.21.15.27:0/2633764732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf581092d0 0x7fbf581096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 --2- 172.21.15.27:0/2633764732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58103990 0x7fbf58103da0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 -- 172.21.15.27:0/2633764732 >> 172.21.15.27:0/2633764732 conn(0x7fbf580ff440 msgr2=0x7fbf58101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.289+0000 7fbf5da00640 1 -- 172.21.15.27:0/2633764732 shutdown_connections 2024-09-21T07:25:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.290+0000 7fbf5da00640 1 -- 172.21.15.27:0/2633764732 wait complete. 2024-09-21T07:25:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 Processor -- start 2024-09-21T07:25:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 -- start start 2024-09-21T07:25:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf58103990 0x7fbf5819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf581092d0 0x7fbf5819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 0x7fbf581a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581138e0 con 0x7fbf581092d0 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58113760 con 0x7fbf58103990 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.291+0000 7fbf5da00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58113a60 con 0x7fbf58109c20 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf4f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf581092d0 0x7fbf5819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 0x7fbf581a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf58103990 0x7fbf5819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 0x7fbf581a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56716/0 (socket says 172.21.15.27:56716) 2024-09-21T07:25:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 -- 172.21.15.27:0/3381632860 learned_addr learned my addr 172.21.15.27:0/3381632860 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 -- 172.21.15.27:0/3381632860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf58103990 msgr2=0x7fbf5819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf58103990 0x7fbf5819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 -- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf581092d0 msgr2=0x7fbf5819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf581092d0 0x7fbf5819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 -- 172.21.15.27:0/3381632860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf581a2480 con 0x7fbf58109c20 2024-09-21T07:25:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.292+0000 7fbf5d000640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 0x7fbf581a1d50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000eb10 tx=0x7fbf5000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.293+0000 7fbf55600640 1 -- 172.21.15.27:0/3381632860 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50004b20 con 0x7fbf58109c20 2024-09-21T07:25:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.293+0000 7fbf55600640 1 -- 172.21.15.27:0/3381632860 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50004cc0 con 0x7fbf58109c20 2024-09-21T07:25:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.293+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf581a2770 con 0x7fbf58109c20 2024-09-21T07:25:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.293+0000 7fbf55600640 1 -- 172.21.15.27:0/3381632860 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50010670 con 0x7fbf58109c20 2024-09-21T07:25:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.293+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a8ed0 con 0x7fbf58109c20 2024-09-21T07:25:06.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.296+0000 7fbf65c40740 1 Processor -- start 2024-09-21T07:25:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf55600640 1 -- 172.21.15.27:0/3381632860 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbf50005020 con 0x7fbf58109c20 2024-09-21T07:25:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf65c40740 1 -- start start 2024-09-21T07:25:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc0df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc0e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf65c40740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cbedb83e90 con 0x55cbed845670 2024-09-21T07:25:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf65c40740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cbedb732c0 con 0x55cbed83d490 2024-09-21T07:25:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.297+0000 7fbf65c40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cbedb770d0 con 0x55cbed866350 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc0e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48612/0 (socket says 172.21.15.27:48612) 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 -- 172.21.15.27:0/3992786464 learned_addr learned my addr 172.21.15.27:0/3992786464 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf4f400640 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc0df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf4f400640 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc0e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 -- 172.21.15.27:0/3992786464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 msgr2=0x55cbedc0df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc0df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 -- 172.21.15.27:0/3992786464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 msgr2=0x55cbedc0e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc0e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.298+0000 7fbf57e00640 1 -- 172.21.15.27:0/3992786464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbed8d1930 con 0x55cbed83d490 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.299+0000 7fbf55600640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbf3c0b6700 0x7fbf3c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.299+0000 7fbf57e00640 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc0e440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009900 tx=0x7fbf48032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c071b85dd243732 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.299+0000 7fbf55600640 1 -- 172.21.15.27:0/3381632860 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7fbf500e8080 con 0x7fbf58109c20 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.299+0000 7fbf4f400640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbf3c0b6700 0x7fbf3c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.299+0000 7fbf4d600640 1 -- 172.21.15.27:0/3992786464 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48040070 con 0x55cbed83d490 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.299+0000 7fbf4d600640 1 -- 172.21.15.27:0/3992786464 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48004030 con 0x55cbed83d490 2024-09-21T07:25:06.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.299+0000 7fbf4d600640 1 -- 172.21.15.27:0/3992786464 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48035040 con 0x55cbed83d490 2024-09-21T07:25:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 -- 172.21.15.27:0/3992786464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 msgr2=0x55cbedc0e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf4f400640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbf3c0b6700 0x7fbf3c0b8c10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbf400097e0 tx=0x7fbf40005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc0e440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009900 tx=0x7fbf48032b90 comp rx=0 tx=0).stop 2024-09-21T07:25:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 -- 172.21.15.27:0/3992786464 shutdown_connections 2024-09-21T07:25:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc0e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc0e440 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 --2- 172.21.15.27:0/3992786464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc0df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 -- 172.21.15.27:0/3992786464 >> 172.21.15.27:0/3992786464 conn(0x55cbed85a740 msgr2=0x55cbedc0bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 -- 172.21.15.27:0/3992786464 shutdown_connections 2024-09-21T07:25:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.300+0000 7fbf65c40740 1 -- 172.21.15.27:0/3992786464 wait complete. 2024-09-21T07:25:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.301+0000 7fbf65c40740 1 -- start start 2024-09-21T07:25:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.301+0000 7fbf65c40740 1 Processor -- start 2024-09-21T07:25:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc19f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf65c40740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cbed9d4890 con 0x55cbed83d490 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf65c40740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cbedb0e160 con 0x55cbed866350 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf65c40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cbedc1c680 con 0x55cbed845670 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf65c40740 10 client.0 _ll_register_callbacks cb 0x55cbedb2aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf4f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc19f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf4f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc19f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56730/0 (socket says 172.21.15.27:56730) 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf4f400640 1 -- 172.21.15.27:0/2732915597 learned_addr learned my addr 172.21.15.27:0/2732915597 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf5d000640 1 -- 172.21.15.27:0/2732915597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 msgr2=0x55cbedc19f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf5d000640 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc19f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf5d000640 1 -- 172.21.15.27:0/2732915597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 msgr2=0x55cbedc199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf5d000640 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.302+0000 7fbf5d000640 1 -- 172.21.15.27:0/2732915597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbedc25eb0 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf5d000640 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc25970 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbf500e5370 tx=0x7fbf500ad970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c2926d512f22c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf46000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf46000640 1 -- 172.21.15.27:0/2732915597 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500b2070 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf48004030 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf46000640 1 -- 172.21.15.27:0/2732915597 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7fbf500b2210 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf46000640 1 -- 172.21.15.27:0/2732915597 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf500a9050 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf65c40740 10 client.29012 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbedc287e0 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.303+0000 7fbf46000640 1 -- 172.21.15.27:0/2732915597 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48004030 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf46000640 1 -- 172.21.15.27:0/2732915597 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf500a93c0 con 0x55cbed866350 2024-09-21T07:25:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf65c40740 10 client.29012 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:06.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf65c40740 10 client.29012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:06.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf65c40740 20 client.29012 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:06.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf65c40740 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed824cd0 0x55cbedc2bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf65c40740 4 client.29012 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:06.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbedb85de0 con 0x55cbed824cd0 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.304+0000 7fbf57e00640 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed824cd0 0x55cbedc2bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.305+0000 7fbf57e00640 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed824cd0 0x55cbedc2bd00 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5cc610c82c130f7 server_cookie=3f5ef16fe6fedead in_seq=0 out_seq=0 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.305+0000 7fbf46000640 10 client.29012 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.305+0000 7fbf46000640 1 -- 172.21.15.27:0/2732915597 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cbedb85de0 con 0x55cbed824cd0 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.305+0000 7fbf46000640 10 client.29012 handle_command_reply: tid=0 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 client.29012 shutdown 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 msgr2=0x55cbedc25970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc25970 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbf500e5370 tx=0x7fbf500ad970 comp rx=0 tx=0).stop 2024-09-21T07:25:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 shutdown_connections 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed824cd0 0x55cbedc2bd00 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc19f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc25970 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 --2- 172.21.15.27:0/2732915597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 >> 172.21.15.27:0/2732915597 conn(0x55cbed85a740 msgr2=0x55cbedc11090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf46000640 0 client.29012 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 shutdown_connections 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 1 -- 172.21.15.27:0/2732915597 wait complete. 2024-09-21T07:25:06.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.306+0000 7fbf65c40740 20 client.29012 trim_cache size 0 max 16384 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 Processor -- start 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 -- start start 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc25870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc16f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b5950 con 0x55cbed845670 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b3220 con 0x55cbed83d490 2024-09-21T07:25:06.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.325+0000 7fbf65c40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b1090 con 0x55cbed866350 2024-09-21T07:25:06.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44886/0 (socket says 172.21.15.27:44886) 2024-09-21T07:25:06.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 -- 172.21.15.27:0/1389343252 learned_addr learned my addr 172.21.15.27:0/1389343252 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf5d000640 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc16f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf57e00640 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc25870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 -- 172.21.15.27:0/1389343252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 msgr2=0x55cbedc16f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc16f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 -- 172.21.15.27:0/1389343252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 msgr2=0x55cbedc25870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc25870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf4f400640 1 -- 172.21.15.27:0/1389343252 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbedc10d60 con 0x55cbed845670 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf5d000640 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.326+0000 7fbf57e00640 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc25870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:06.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.327+0000 7fbf4f400640 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc169e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbf400044f0 tx=0x7fbf40019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558df23c5dfe5958 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:06.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.327+0000 7fbf4cc00640 1 -- 172.21.15.27:0/1389343252 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40020640 con 0x55cbed845670 2024-09-21T07:25:06.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.327+0000 7fbf4cc00640 1 -- 172.21.15.27:0/1389343252 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf40020c90 con 0x55cbed845670 2024-09-21T07:25:06.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.327+0000 7fbf4cc00640 1 -- 172.21.15.27:0/1389343252 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4002bd80 con 0x55cbed845670 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 -- 172.21.15.27:0/1389343252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 msgr2=0x55cbedc169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc169e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbf400044f0 tx=0x7fbf40019000 comp rx=0 tx=0).stop 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 -- 172.21.15.27:0/1389343252 shutdown_connections 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed866350 0x55cbedc16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed845670 0x55cbedc169e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 --2- 172.21.15.27:0/1389343252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed83d490 0x55cbedc25870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 -- 172.21.15.27:0/1389343252 >> 172.21.15.27:0/1389343252 conn(0x55cbed85a740 msgr2=0x55cbedc0b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 -- 172.21.15.27:0/1389343252 shutdown_connections 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 -- 172.21.15.27:0/1389343252 wait complete. 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.328+0000 7fbf65c40740 1 -- start start 2024-09-21T07:25:06.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 1 Processor -- start 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc21e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc1f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc1fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b16e0 con 0x55cbed83d490 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b2b50 con 0x55cbed866350 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b1db0 con 0x55cbed845670 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf65c40740 10 client.0 _ll_register_callbacks cb 0x7fbf500a9980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc21e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc21e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44902/0 (socket says 172.21.15.27:44902) 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.329+0000 7fbf57e00640 1 -- 172.21.15.27:0/500096143 learned_addr learned my addr 172.21.15.27:0/500096143 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf4f400640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc1f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf5d000640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc1fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf57e00640 1 -- 172.21.15.27:0/500096143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 msgr2=0x55cbedc1f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf57e00640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc1f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf57e00640 1 -- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 msgr2=0x55cbedc1fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf57e00640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc1fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf57e00640 1 -- 172.21.15.27:0/500096143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbedc20360 con 0x55cbed83d490 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf5d000640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc1fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf57e00640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc21e70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x55cbedc0cb80 tx=0x7fbf4802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67f9dc18ea16d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.330+0000 7fbf46000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.331+0000 7fbf46000640 1 -- 172.21.15.27:0/500096143 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48040040 con 0x55cbed83d490 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.331+0000 7fbf46000640 1 -- 172.21.15.27:0/500096143 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7fbf48031d10 con 0x55cbed83d490 2024-09-21T07:25:06.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.331+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf40020c90 con 0x55cbed83d490 2024-09-21T07:25:06.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.331+0000 7fbf46000640 1 -- 172.21.15.27:0/500096143 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf4805e050 con 0x55cbed83d490 2024-09-21T07:25:06.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.331+0000 7fbf65c40740 10 client.19008 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:06.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.331+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbedc973f0 con 0x55cbed83d490 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.331+0000 7fbf46000640 1 -- 172.21.15.27:0/500096143 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf40020c90 con 0x55cbed83d490 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf46000640 1 -- 172.21.15.27:0/500096143 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf48004750 con 0x55cbed83d490 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf65c40740 10 client.19008 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf65c40740 10 client.19008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf65c40740 20 client.19008 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf65c40740 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed7fa080 0x55cbedc99740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf65c40740 4 client.19008 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cbedb81890 con 0x55cbed7fa080 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.332+0000 7fbf4f400640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed7fa080 0x55cbedc99740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.333+0000 7fbf4f400640 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed7fa080 0x55cbedc99740 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d611bb20f3f6d6c server_cookie=695bb92ae4463bbf in_seq=0 out_seq=0 2024-09-21T07:25:06.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.333+0000 7fbf46000640 10 client.19008 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:06.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.334+0000 7fbf46000640 1 -- 172.21.15.27:0/500096143 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cbedb81890 con 0x55cbed7fa080 2024-09-21T07:25:06.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.334+0000 7fbf46000640 10 client.19008 handle_command_reply: tid=0 2024-09-21T07:25:06.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.334+0000 7fbf65c40740 1 client.19008 shutdown 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 msgr2=0x55cbedc21e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc21e70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x55cbedc0cb80 tx=0x7fbf4802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 shutdown_connections 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cbed7fa080 0x55cbedc99740 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbed845670 0x55cbedc1f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbed866350 0x55cbedc1fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 --2- 172.21.15.27:0/500096143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cbed83d490 0x55cbedc21e70 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 >> 172.21.15.27:0/500096143 conn(0x55cbed85a740 msgr2=0x55cbedc0eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf46000640 0 client.19008 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 shutdown_connections 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 1 -- 172.21.15.27:0/500096143 wait complete. 2024-09-21T07:25:06.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.335+0000 7fbf65c40740 20 client.19008 trim_cache size 0 max 16384 2024-09-21T07:25:06.339 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:25:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "06f8f5c7-be29-4731-b3a2-a4e12b5538d6", 2024-09-21T07:25:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:25:06.339 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:06.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbf3c0b6700 msgr2=0x7fbf3c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbf3c0b6700 0x7fbf3c0b8c10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbf400097e0 tx=0x7fbf40005f90 comp rx=0 tx=0).stop 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 msgr2=0x7fbf581a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 0x7fbf581a1d50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000eb10 tx=0x7fbf5000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 shutdown_connections 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf58109c20 0x7fbf581a1d50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbf3c0b6700 0x7fbf3c0b8c10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf581092d0 0x7fbf5819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 --2- 172.21.15.27:0/3381632860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf58103990 0x7fbf5819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 >> 172.21.15.27:0/3381632860 conn(0x7fbf580ff440 msgr2=0x7fbf58108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:06.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 shutdown_connections 2024-09-21T07:25:06.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:06.336+0000 7fbf5da00640 1 -- 172.21.15.27:0/3381632860 wait complete. 2024-09-21T07:25:07.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:25:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:07 smithi153 ceph-mon[34747]: pgmap v753: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-09-21T07:25:07.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:07 smithi027 ceph-mon[30779]: pgmap v753: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-09-21T07:25:07.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:07 smithi119 ceph-mon[34519]: pgmap v753: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-09-21T07:25:07.564 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 -- 172.21.15.27:0/2422295946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 msgr2=0x7fdb0010ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 --2- 172.21.15.27:0/2422295946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0010ba20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02a410 comp rx=0 tx=0).stop 2024-09-21T07:25:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 -- 172.21.15.27:0/2422295946 shutdown_connections 2024-09-21T07:25:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 --2- 172.21.15.27:0/2422295946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0010bf60 0x7fdb00112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 --2- 172.21.15.27:0/2422295946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0010ba20 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 --2- 172.21.15.27:0/2422295946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb00105ce0 0x7fdb001060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 -- 172.21.15.27:0/2422295946 >> 172.21.15.27:0/2422295946 conn(0x7fdb001016e0 msgr2=0x7fdb00103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 -- 172.21.15.27:0/2422295946 shutdown_connections 2024-09-21T07:25:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.664+0000 7fdb08600640 1 -- 172.21.15.27:0/2422295946 wait complete. 2024-09-21T07:25:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.665+0000 7fdb08600640 1 Processor -- start 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb08600640 1 -- start start 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb08600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb00105ce0 0x7fdb0019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb08600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb08600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0010bf60 0x7fdb001a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb08600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00115c20 con 0x7fdb0010b610 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb08600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00115aa0 con 0x7fdb0010bf60 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb08600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00115da0 con 0x7fdb00105ce0 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44930/0 (socket says 172.21.15.27:44930) 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb05200640 1 -- 172.21.15.27:0/2470848084 learned_addr learned my addr 172.21.15.27:0/2470848084 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb05c00640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb00105ce0 0x7fdb0019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.666+0000 7fdb06600640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0010bf60 0x7fdb001a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdb05200640 1 -- 172.21.15.27:0/2470848084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb00105ce0 msgr2=0x7fdb0019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdb05200640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb00105ce0 0x7fdb0019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdb05200640 1 -- 172.21.15.27:0/2470848084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0010bf60 msgr2=0x7fdb001a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdb05200640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0010bf60 0x7fdb001a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdb05200640 1 -- 172.21.15.27:0/2470848084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb001a2590 con 0x7fdb0010b610 2024-09-21T07:25:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdb05200640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0019cc00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdaec006010 tx=0x7fdaec004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdafe000640 1 -- 172.21.15.27:0/2470848084 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec03f030 con 0x7fdb0010b610 2024-09-21T07:25:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdafe000640 1 -- 172.21.15.27:0/2470848084 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec003aa0 con 0x7fdb0010b610 2024-09-21T07:25:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.667+0000 7fdafe000640 1 -- 172.21.15.27:0/2470848084 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec035070 con 0x7fdb0010b610 2024-09-21T07:25:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.668+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb001a2820 con 0x7fdb0010b610 2024-09-21T07:25:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.668+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001a9010 con 0x7fdb0010b610 2024-09-21T07:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.671+0000 7fdb0f812740 1 Processor -- start 2024-09-21T07:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.671+0000 7fdb0f812740 1 -- start start 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.671+0000 7fdafe000640 1 -- 172.21.15.27:0/2470848084 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdaec042030 con 0x7fdb0010b610 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3adc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb0f812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55941e3236b0 con 0x55941dfe02f0 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb0f812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55941e312ae0 con 0x55941dfe84d0 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb0f812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55941e3168f0 con 0x55941e0091b0 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56778/0 (socket says 172.21.15.27:56778) 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44938/0 (socket says 172.21.15.27:44938) 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 -- 172.21.15.27:0/672175183 learned_addr learned my addr 172.21.15.27:0/672175183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb05200640 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 -- 172.21.15.27:0/672175183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 msgr2=0x55941e3ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 -- 172.21.15.27:0/672175183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 msgr2=0x55941e3ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.672+0000 7fdb06600640 1 -- 172.21.15.27:0/672175183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941e2df780 con 0x55941e0091b0 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.673+0000 7fdb05c00640 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.673+0000 7fdb06600640 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3adc30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009f80 tx=0x7fdaf80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=518bdeb02563c91e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.673+0000 7fdaf2a00640 1 -- 172.21.15.27:0/672175183 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8004d20 con 0x55941e0091b0 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.673+0000 7fdaf2a00640 1 -- 172.21.15.27:0/672175183 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf8004050 con 0x55941e0091b0 2024-09-21T07:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.673+0000 7fdaf2a00640 1 -- 172.21.15.27:0/672175183 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8005220 con 0x55941e0091b0 2024-09-21T07:25:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.673+0000 7fdafe000640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdae80b6680 0x7fdae80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb05c00640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdae80b6680 0x7fdae80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdafe000640 1 -- 172.21.15.27:0/2470848084 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7fdaec107fc0 con 0x7fdb0010b610 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb0f812740 1 -- 172.21.15.27:0/672175183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 msgr2=0x55941e3adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb0f812740 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3adc30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009f80 tx=0x7fdaf80118d0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb05c00640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdae80b6680 0x7fdae80b8b90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55941e3ac8b0 tx=0x7fdaf400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb0f812740 1 -- 172.21.15.27:0/672175183 shutdown_connections 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb0f812740 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb0f812740 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb0f812740 1 --2- 172.21.15.27:0/672175183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3adc30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.674+0000 7fdb0f812740 1 -- 172.21.15.27:0/672175183 >> 172.21.15.27:0/672175183 conn(0x55941dffd300 msgr2=0x55941e3abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.675+0000 7fdb0f812740 1 -- 172.21.15.27:0/672175183 shutdown_connections 2024-09-21T07:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.675+0000 7fdb0f812740 1 -- 172.21.15.27:0/672175183 wait complete. 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.675+0000 7fdb0f812740 1 -- start start 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.675+0000 7fdb0f812740 1 Processor -- start 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb0f812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55941e24aab0 con 0x55941dfe02f0 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb0f812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55941e2b9570 con 0x55941dfe84d0 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb0f812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55941e3bc3b0 con 0x55941e0091b0 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb0f812740 10 client.0 _ll_register_callbacks cb 0x55941e2c8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44954/0 (socket says 172.21.15.27:44954) 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb05c00640 1 -- 172.21.15.27:0/1931342072 learned_addr learned my addr 172.21.15.27:0/1931342072 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb05200640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.676+0000 7fdb06600640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb05200640 1 -- 172.21.15.27:0/1931342072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 msgr2=0x55941e3b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb05200640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb05200640 1 -- 172.21.15.27:0/1931342072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 msgr2=0x55941e3b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb05200640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb05200640 1 -- 172.21.15.27:0/1931342072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941e074790 con 0x55941dfe84d0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb05c00640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb05200640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3b9770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdaec105310 tx=0x7fdaec000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6844c72655cbb62e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdae7400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdae7400640 1 -- 172.21.15.27:0/1931342072 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec049070 con 0x55941dfe84d0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf8004050 con 0x55941dfe84d0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdae7400640 1 -- 172.21.15.27:0/1931342072 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7fdaec10a0d0 con 0x55941dfe84d0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdae7400640 1 -- 172.21.15.27:0/1931342072 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdaec0ca110 con 0x55941dfe84d0 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb0f812740 10 client.29030 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.677+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55941e0ed620 con 0x55941dfe84d0 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdae7400640 1 -- 172.21.15.27:0/1931342072 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf8004050 con 0x55941dfe84d0 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdae7400640 1 -- 172.21.15.27:0/1931342072 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdaec0d6fa0 con 0x55941dfe84d0 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdb0f812740 10 client.29030 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdb0f812740 10 client.29030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdb0f812740 20 client.29030 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdb0f812740 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941dfc7d80 0x55941e3cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdb0f812740 4 client.29030 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55941e325600 con 0x55941dfc7d80 2024-09-21T07:25:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.678+0000 7fdb05c00640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941dfc7d80 0x55941e3cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.679+0000 7fdb05c00640 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941dfc7d80 0x55941e3cb350 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78745a125be3c92 server_cookie=ee4d03d54b3d22ca in_seq=0 out_seq=0 2024-09-21T07:25:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.679+0000 7fdae7400640 10 client.29030 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdae7400640 1 -- 172.21.15.27:0/1931342072 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55941e325600 con 0x55941dfc7d80 2024-09-21T07:25:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdae7400640 10 client.29030 handle_command_reply: tid=0 2024-09-21T07:25:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 client.29030 shutdown 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 msgr2=0x55941e3b9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3b9770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdaec105310 tx=0x7fdaec000bc0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 shutdown_connections 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941dfc7d80 0x55941e3cb350 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3b9770 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 --2- 172.21.15.27:0/1931342072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 >> 172.21.15.27:0/1931342072 conn(0x55941dffd300 msgr2=0x55941e3aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdae7400640 0 client.29030 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.680+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 shutdown_connections 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.681+0000 7fdb0f812740 1 -- 172.21.15.27:0/1931342072 wait complete. 2024-09-21T07:25:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.681+0000 7fdb0f812740 20 client.29030 trim_cache size 0 max 16384 2024-09-21T07:25:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 Processor -- start 2024-09-21T07:25:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 -- start start 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3bae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001b5a50 con 0x55941dfe02f0 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001b3320 con 0x55941dfe84d0 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.696+0000 7fdb0f812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001b1190 con 0x55941e0091b0 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44962/0 (socket says 172.21.15.27:44962) 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 -- 172.21.15.27:0/2777009486 learned_addr learned my addr 172.21.15.27:0/2777009486 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05200640 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3bae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb06600640 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 -- 172.21.15.27:0/2777009486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 msgr2=0x55941e3b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 -- 172.21.15.27:0/2777009486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 msgr2=0x55941e3bae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3bae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 -- 172.21.15.27:0/2777009486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941e3cb6c0 con 0x55941dfe02f0 2024-09-21T07:25:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.697+0000 7fdb05c00640 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3aaaa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x55941e3cba30 tx=0x7fdaf4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee10f44c7ded89f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.698+0000 7fdaf2000640 1 -- 172.21.15.27:0/2777009486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf402a420 con 0x55941dfe02f0 2024-09-21T07:25:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.698+0000 7fdaf2000640 1 -- 172.21.15.27:0/2777009486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf402aa70 con 0x55941dfe02f0 2024-09-21T07:25:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.698+0000 7fdaf2000640 1 -- 172.21.15.27:0/2777009486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40319f0 con 0x55941dfe02f0 2024-09-21T07:25:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.698+0000 7fdb0f812740 1 -- 172.21.15.27:0/2777009486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 msgr2=0x55941e3aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.698+0000 7fdb0f812740 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3aaaa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x55941e3cba30 tx=0x7fdaf4019000 comp rx=0 tx=0).stop 2024-09-21T07:25:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 -- 172.21.15.27:0/2777009486 shutdown_connections 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe84d0 0x55941e3bae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 --2- 172.21.15.27:0/2777009486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe02f0 0x55941e3aaaa0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 -- 172.21.15.27:0/2777009486 >> 172.21.15.27:0/2777009486 conn(0x55941dffd300 msgr2=0x55941e3ba750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 -- 172.21.15.27:0/2777009486 shutdown_connections 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 -- 172.21.15.27:0/2777009486 wait complete. 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 -- start start 2024-09-21T07:25:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.699+0000 7fdb0f812740 1 Processor -- start 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe02f0 0x55941e3c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe84d0 0x55941e3bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb0f812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb0f812740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001b17e0 con 0x55941dfe84d0 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb0f812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001b2c50 con 0x55941dfe02f0 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb0f812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001b1eb0 con 0x55941e0091b0 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb0f812740 10 client.0 _ll_register_callbacks cb 0x7fdaec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe84d0 0x55941e3bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe84d0 0x55941e3bf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44970/0 (socket says 172.21.15.27:44970) 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 -- 172.21.15.27:0/283681428 learned_addr learned my addr 172.21.15.27:0/283681428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05c00640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe02f0 0x55941e3c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb06600640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 -- 172.21.15.27:0/283681428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 msgr2=0x55941e3bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 -- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe02f0 msgr2=0x55941e3c1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe02f0 0x55941e3c1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb05200640 1 -- 172.21.15.27:0/283681428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941e3bfd70 con 0x55941dfe84d0 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.700+0000 7fdb06600640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdb05c00640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe02f0 0x55941e3c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdb05200640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe84d0 0x55941e3bf1c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0cd7c0 tx=0x7fdaec10bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab415f31f4bb8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdae7400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdae7400640 1 -- 172.21.15.27:0/283681428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec049040 con 0x55941dfe84d0 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdae7400640 1 -- 172.21.15.27:0/283681428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7fdaec0ccd20 con 0x55941dfe84d0 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf402aa70 con 0x55941dfe84d0 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdae7400640 1 -- 172.21.15.27:0/283681428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdaec0c9230 con 0x55941dfe84d0 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdb0f812740 10 client.19026 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.701+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55941e435ae0 con 0x55941dfe84d0 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdae7400640 1 -- 172.21.15.27:0/283681428 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf402aa70 con 0x55941dfe84d0 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdae7400640 1 -- 172.21.15.27:0/283681428 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdaec03bad0 con 0x55941dfe84d0 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdb0f812740 10 client.19026 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdb0f812740 10 client.19026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdb0f812740 20 client.19026 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdb0f812740 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941df9d1c0 0x55941e4380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdb0f812740 4 client.19026 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55941e3210b0 con 0x55941df9d1c0 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.702+0000 7fdb05c00640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941df9d1c0 0x55941e4380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.703+0000 7fdb05c00640 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941df9d1c0 0x55941e4380f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb06867206b293b2 server_cookie=537c1f6d25f738bd in_seq=0 out_seq=0 2024-09-21T07:25:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.703+0000 7fdae7400640 10 client.19026 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdae7400640 1 -- 172.21.15.27:0/283681428 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55941e3210b0 con 0x55941df9d1c0 2024-09-21T07:25:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdae7400640 10 client.19026 handle_command_reply: tid=0 2024-09-21T07:25:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 client.19026 shutdown 2024-09-21T07:25:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe84d0 msgr2=0x55941e3bf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe84d0 0x55941e3bf1c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0cd7c0 tx=0x7fdaec10bed0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 shutdown_connections 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55941df9d1c0 0x55941e4380f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55941e0091b0 0x55941e3bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55941dfe84d0 0x55941e3bf1c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdae7400640 0 client.19026 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 --2- 172.21.15.27:0/283681428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55941dfe02f0 0x55941e3c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 >> 172.21.15.27:0/283681428 conn(0x55941dffd300 msgr2=0x55941e3aa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 shutdown_connections 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.716+0000 7fdb0f812740 1 -- 172.21.15.27:0/283681428 wait complete. 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb0f812740 20 client.19026 trim_cache size 0 max 16384 2024-09-21T07:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdae80b6680 msgr2=0x7fdae80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdae80b6680 0x7fdae80b8b90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55941e3ac8b0 tx=0x7fdaf400b410 comp rx=0 tx=0).stop 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 msgr2=0x7fdb0019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0019cc00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdaec006010 tx=0x7fdaec004060 comp rx=0 tx=0).stop 2024-09-21T07:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 shutdown_connections 2024-09-21T07:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdae80b6680 0x7fdae80b8b90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0010bf60 0x7fdb001a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb0010b610 0x7fdb0019cc00 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 --2- 172.21.15.27:0/2470848084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb00105ce0 0x7fdb0019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 >> 172.21.15.27:0/2470848084 conn(0x7fdb001016e0 msgr2=0x7fdb0010acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.717+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 shutdown_connections 2024-09-21T07:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:07.718+0000 7fdb08600640 1 -- 172.21.15.27:0/2470848084 wait complete. 2024-09-21T07:25:08.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 -- 172.21.15.27:0/2143877576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8077940 msgr2=0x7f09a8077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 --2- 172.21.15.27:0/2143877576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8077940 0x7f09a8077d50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0998009900 tx=0x7f099802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 -- 172.21.15.27:0/2143877576 shutdown_connections 2024-09-21T07:25:08.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 --2- 172.21.15.27:0/2143877576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a8114570 0x7f09a8116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 --2- 172.21.15.27:0/2143877576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8078290 0x7f09a8114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 --2- 172.21.15.27:0/2143877576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8077940 0x7f09a8077d50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 -- 172.21.15.27:0/2143877576 >> 172.21.15.27:0/2143877576 conn(0x7f09a8101680 msgr2=0x7f09a8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 -- 172.21.15.27:0/2143877576 shutdown_connections 2024-09-21T07:25:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.183+0000 7f09a7200640 1 -- 172.21.15.27:0/2143877576 wait complete. 2024-09-21T07:25:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.184+0000 7f09a7200640 1 Processor -- start 2024-09-21T07:25:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.184+0000 7f09a7200640 1 -- start start 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8077940 0x7f09a81a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8078290 0x7f09a81a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a7200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a8114570 0x7f09a81a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8119ef0 con 0x7f09a8078290 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a7200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8119d70 con 0x7f09a8077940 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a7200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09a811a070 con 0x7f09a8114570 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8077940 0x7f09a81a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f099fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8078290 0x7f09a81a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8077940 0x7f09a81a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44922/0 (socket says 172.21.15.27:44922) 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a5e00640 1 -- 172.21.15.27:0/2156360519 learned_addr learned my addr 172.21.15.27:0/2156360519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.185+0000 7f09a6800640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a8114570 0x7f09a81a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f09a5e00640 1 -- 172.21.15.27:0/2156360519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a8114570 msgr2=0x7f09a81a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f09a5e00640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a8114570 0x7f09a81a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f09a5e00640 1 -- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8078290 msgr2=0x7f09a81a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f09a5e00640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8078290 0x7f09a81a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f09a5e00640 1 -- 172.21.15.27:0/2156360519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a81a6870 con 0x7f09a8077940 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f099fe00640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8078290 0x7f09a81a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f09a5e00640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8077940 0x7f09a81a0900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0998005940 tx=0x7f0998004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.186+0000 7f099ea00640 1 -- 172.21.15.27:0/2156360519 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099803f030 con 0x7f09a8077940 2024-09-21T07:25:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.187+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a81a6b00 con 0x7f09a8077940 2024-09-21T07:25:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.187+0000 7f099ea00640 1 -- 172.21.15.27:0/2156360519 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0998003aa0 con 0x7f09a8077940 2024-09-21T07:25:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.187+0000 7f099ea00640 1 -- 172.21.15.27:0/2156360519 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998035070 con 0x7f09a8077940 2024-09-21T07:25:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.187+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81ad2c0 con 0x7f09a8077940 2024-09-21T07:25:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.190+0000 7f09b3bc3740 1 Processor -- start 2024-09-21T07:25:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.190+0000 7f09b3bc3740 1 -- start start 2024-09-21T07:25:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.190+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.190+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37e4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f099ea00640 1 -- 172.21.15.27:0/2156360519 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0998042030 con 0x7f09a8077940 2024-09-21T07:25:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b7f375af70 con 0x55b7f343d070 2024-09-21T07:25:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7f374a3a0 con 0x55b7f34141b0 2024-09-21T07:25:08.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b7f374e1b0 con 0x55b7f341c390 2024-09-21T07:25:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37e3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40768/0 (socket says 172.21.15.27:40768) 2024-09-21T07:25:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 -- 172.21.15.27:0/3538987946 learned_addr learned my addr 172.21.15.27:0/3538987946 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a5e00640 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f099fe00640 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 -- 172.21.15.27:0/3538987946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 msgr2=0x55b7f37e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 -- 172.21.15.27:0/3538987946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 msgr2=0x55b7f37e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.191+0000 7f09a6800640 1 -- 172.21.15.27:0/3538987946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f3714c90 con 0x55b7f343d070 2024-09-21T07:25:08.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.192+0000 7f09a6800640 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37e3f30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f09a0010370 tx=0x7f09a0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc0d2a8d98ac8ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:08.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.192+0000 7f0997400640 1 -- 172.21.15.27:0/3538987946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0016070 con 0x55b7f343d070 2024-09-21T07:25:08.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.192+0000 7f0997400640 1 -- 172.21.15.27:0/3538987946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a0004d20 con 0x55b7f343d070 2024-09-21T07:25:08.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.192+0000 7f0997400640 1 -- 172.21.15.27:0/3538987946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00053e0 con 0x55b7f343d070 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f099ea00640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f098c0b6680 0x7f098c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f099ea00640 1 -- 172.21.15.27:0/2156360519 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7f09981081e0 con 0x7f09a8077940 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3538987946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 msgr2=0x55b7f37e3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37e3f30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f09a0010370 tx=0x7f09a0010830 comp rx=0 tx=0).stop 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09a5e00640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f098c0b6680 0x7f098c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3538987946 shutdown_connections 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3538987946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37e3f30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.193+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3538987946 >> 172.21.15.27:0/3538987946 conn(0x55b7f3431460 msgr2=0x55b7f37e1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.194+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3538987946 shutdown_connections 2024-09-21T07:25:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.194+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3538987946 wait complete. 2024-09-21T07:25:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.194+0000 7f09a5e00640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f098c0b6680 0x7f098c0b8b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b7f37e2bb0 tx=0x7f099810bc90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:08.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.194+0000 7f09b3bc3740 1 -- start start 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 1 Processor -- start 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f34141b0 0x55b7f37ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f341c390 0x55b7f37efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b7f35ab700 con 0x55b7f343d070 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7f36e5540 con 0x55b7f341c390 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b7f37f2710 con 0x55b7f34141b0 2024-09-21T07:25:08.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09b3bc3740 10 client.0 _ll_register_callbacks cb 0x55b7f36f5430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.195+0000 7f09a6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f34141b0 0x55b7f37ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f09a6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f34141b0 0x55b7f37ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49182/0 (socket says 172.21.15.27:49182) 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f09a6800640 1 -- 172.21.15.27:0/3967667220 learned_addr learned my addr 172.21.15.27:0/3967667220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f099fe00640 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f341c390 0x55b7f37efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f09a6800640 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f099fe00640 1 -- 172.21.15.27:0/3967667220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f34141b0 msgr2=0x55b7f37ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f099fe00640 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f34141b0 0x55b7f37ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f099fe00640 1 -- 172.21.15.27:0/3967667220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 msgr2=0x55b7f37fba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f099fe00640 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37fba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f099fe00640 1 -- 172.21.15.27:0/3967667220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f34a8650 con 0x55b7f341c390 2024-09-21T07:25:08.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f099fe00640 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f341c390 0x55b7f37efc50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f09a806c630 tx=0x7f099000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d66c7f5180ada77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.196+0000 7f098be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.197+0000 7f098be00640 1 -- 172.21.15.27:0/3967667220 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990012110 con 0x55b7f341c390 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.197+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7f36e7b50 con 0x55b7f341c390 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.197+0000 7f098be00640 1 -- 172.21.15.27:0/3967667220 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7f0990004050 con 0x55b7f341c390 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.197+0000 7f098be00640 1 -- 172.21.15.27:0/3967667220 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0990025050 con 0x55b7f341c390 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.197+0000 7f09b3bc3740 10 client.29048 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.197+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7f3702910 con 0x55b7f341c390 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.197+0000 7f098be00640 1 -- 172.21.15.27:0/3967667220 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b7f36e7b50 con 0x55b7f341c390 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f098be00640 1 -- 172.21.15.27:0/3967667220 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0990010430 con 0x55b7f341c390 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f09b3bc3740 10 client.29048 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f09b3bc3740 10 client.29048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f09b3bc3740 20 client.29048 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33fb9f0 0x55b7f38018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f09b3bc3740 4 client.29048 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7f375cec0 con 0x55b7f33fb9f0 2024-09-21T07:25:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.198+0000 7f09a6800640 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33fb9f0 0x55b7f38018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.199+0000 7f09a6800640 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33fb9f0 0x55b7f38018f0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18100b032a7fc52 server_cookie=c39ad450c56b323 in_seq=0 out_seq=0 2024-09-21T07:25:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.199+0000 7f098be00640 10 client.29048 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.199+0000 7f098be00640 1 -- 172.21.15.27:0/3967667220 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7f375cec0 con 0x55b7f33fb9f0 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.199+0000 7f098be00640 10 client.29048 handle_command_reply: tid=0 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 client.29048 shutdown 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f341c390 msgr2=0x55b7f37efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f341c390 0x55b7f37efc50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f09a806c630 tx=0x7f099000f790 comp rx=0 tx=0).stop 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 shutdown_connections 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33fb9f0 0x55b7f38018f0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f341c390 0x55b7f37efc50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3967667220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f34141b0 0x55b7f37ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f098be00640 0 client.29048 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:08.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 >> 172.21.15.27:0/3967667220 conn(0x55b7f3431460 msgr2=0x55b7f37e7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 shutdown_connections 2024-09-21T07:25:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3967667220 wait complete. 2024-09-21T07:25:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.200+0000 7f09b3bc3740 20 client.29048 trim_cache size 0 max 16384 2024-09-21T07:25:08.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.219+0000 7f09b3bc3740 1 Processor -- start 2024-09-21T07:25:08.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.219+0000 7f09b3bc3740 1 -- start start 2024-09-21T07:25:08.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37eab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37ece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b9c40 con 0x55b7f343d070 2024-09-21T07:25:08.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b7510 con 0x55b7f34141b0 2024-09-21T07:25:08.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b5380 con 0x55b7f341c390 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09a6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37ece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09a6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37ece40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40776/0 (socket says 172.21.15.27:40776) 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09a6800640 1 -- 172.21.15.27:0/3858366583 learned_addr learned my addr 172.21.15.27:0/3858366583 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09a6800640 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f099fe00640 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.220+0000 7f09a6800640 1 -- 172.21.15.27:0/3858366583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 msgr2=0x55b7f37ec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f09a6800640 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37ec900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f09a6800640 1 -- 172.21.15.27:0/3858366583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 msgr2=0x55b7f37eab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f09a6800640 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37eab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f09a6800640 1 -- 172.21.15.27:0/3858366583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f37fe400 con 0x55b7f343d070 2024-09-21T07:25:08.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f09a6800640 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37ece40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009c40 tx=0x7f09a0010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7469653f957290e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:08.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f0996a00640 1 -- 172.21.15.27:0/3858366583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0016040 con 0x55b7f343d070 2024-09-21T07:25:08.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f0996a00640 1 -- 172.21.15.27:0/3858366583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a0003b70 con 0x55b7f343d070 2024-09-21T07:25:08.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.221+0000 7f0996a00640 1 -- 172.21.15.27:0/3858366583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0023040 con 0x55b7f343d070 2024-09-21T07:25:08.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3858366583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 msgr2=0x55b7f37ece40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37ece40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009c40 tx=0x7f09a0010fa0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3858366583 shutdown_connections 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37ec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37ece40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3858366583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37eab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3858366583 >> 172.21.15.27:0/3858366583 conn(0x55b7f3431460 msgr2=0x55b7f37e1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3858366583 shutdown_connections 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.222+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3858366583 wait complete. 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 -- start start 2024-09-21T07:25:08.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 Processor -- start 2024-09-21T07:25:08.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b59d0 con 0x55b7f343d070 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b6e40 con 0x55b7f34141b0 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.223+0000 7f09b3bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b60a0 con 0x55b7f341c390 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f09b3bc3740 10 client.0 _ll_register_callbacks cb 0x7f0990012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f09a6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f09a6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44962/0 (socket says 172.21.15.27:44962) 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f09a6800640 1 -- 172.21.15.27:0/3758181013 learned_addr learned my addr 172.21.15.27:0/3758181013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f099fe00640 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f09a6800640 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f099fe00640 1 -- 172.21.15.27:0/3758181013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 msgr2=0x55b7f37f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f099fe00640 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37f7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f099fe00640 1 -- 172.21.15.27:0/3758181013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 msgr2=0x55b7f37f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f099fe00640 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.224+0000 7f099fe00640 1 -- 172.21.15.27:0/3758181013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f37f63f0 con 0x55b7f341c390 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.225+0000 7f099fe00640 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37f5970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f099000ffd0 tx=0x7f0990012340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9085b5b0b2a2364e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:08.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.225+0000 7f098be00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.225+0000 7f098be00640 1 -- 172.21.15.27:0/3758181013 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990022040 con 0x55b7f341c390 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.225+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a0003b70 con 0x55b7f341c390 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.225+0000 7f098be00640 1 -- 172.21.15.27:0/3758181013 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7f09900109f0 con 0x55b7f341c390 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.226+0000 7f098be00640 1 -- 172.21.15.27:0/3758181013 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0990038050 con 0x55b7f341c390 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.226+0000 7f09b3bc3740 10 client.29032 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.226+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7f386cf50 con 0x55b7f341c390 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.226+0000 7f098be00640 1 -- 172.21.15.27:0/3758181013 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a0003b70 con 0x55b7f341c390 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.226+0000 7f098be00640 1 -- 172.21.15.27:0/3758181013 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0990003820 con 0x55b7f341c390 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.227+0000 7f09b3bc3740 10 client.29032 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.227+0000 7f09b3bc3740 10 client.29032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:08.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.227+0000 7f09b3bc3740 20 client.29032 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:08.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.227+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33d0da0 0x55b7f386f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:08.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.227+0000 7f09b3bc3740 4 client.29032 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:25:08.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.227+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7f3758970 con 0x55b7f33d0da0 2024-09-21T07:25:08.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.227+0000 7f09a6800640 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33d0da0 0x55b7f386f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09a6800640 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33d0da0 0x55b7f386f700 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca247e219b1afa05 server_cookie=1339ded52bc1bdf1 in_seq=0 out_seq=0 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f098be00640 10 client.29032 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f098be00640 1 -- 172.21.15.27:0/3758181013 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x55b7f3758970 con 0x55b7f33d0da0 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f098be00640 10 client.29032 handle_command_reply: tid=0 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 client.29032 shutdown 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 msgr2=0x55b7f37f5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37f5970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f099000ffd0 tx=0x7f0990012340 comp rx=0 tx=0).stop 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 shutdown_connections 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b7f33d0da0 0x55b7f386f700 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7f341c390 0x55b7f37f5970 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b7f343d070 0x55b7f37f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 --2- 172.21.15.27:0/3758181013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7f34141b0 0x55b7f37f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f098be00640 0 client.29032 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:08.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.228+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 >> 172.21.15.27:0/3758181013 conn(0x55b7f3431460 msgr2=0x55b7f37eab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:08.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.229+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 shutdown_connections 2024-09-21T07:25:08.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.229+0000 7f09b3bc3740 1 -- 172.21.15.27:0/3758181013 wait complete. 2024-09-21T07:25:08.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.229+0000 7f09b3bc3740 20 client.29032 trim_cache size 0 max 16384 2024-09-21T07:25:08.232 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:25:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:25:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:25:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.229+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f098c0b6680 msgr2=0x7f098c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.229+0000 7f09a7200640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f098c0b6680 0x7f098c0b8b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b7f37e2bb0 tx=0x7f099810bc90 comp rx=0 tx=0).stop 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.229+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8077940 msgr2=0x7f09a81a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.229+0000 7f09a7200640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8077940 0x7f09a81a0900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0998005940 tx=0x7f0998004270 comp rx=0 tx=0).stop 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 shutdown_connections 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a8114570 0x7f09a81a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f098c0b6680 0x7f098c0b8b90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a8078290 0x7f09a81a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 --2- 172.21.15.27:0/2156360519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8077940 0x7f09a81a0900 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 >> 172.21.15.27:0/2156360519 conn(0x7f09a8101680 msgr2=0x7f09a81165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 shutdown_connections 2024-09-21T07:25:08.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:08.230+0000 7f09a7200640 1 -- 172.21.15.27:0/2156360519 wait complete. 2024-09-21T07:25:08.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:08 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:25:08.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:08.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:08 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:25:08.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:08.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:08 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:25:08.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:08.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:08 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:25:08.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:08.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:08 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:25:08.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:08.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:08 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:25:08.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:09.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:25:09.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:25:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:09 smithi027 ceph-mon[30779]: pgmap v754: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 34 op/s 2024-09-21T07:25:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:09 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:25:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:09 smithi153 ceph-mon[34747]: pgmap v754: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 34 op/s 2024-09-21T07:25:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:09 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:25:09.512 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:09 smithi119 ceph-mon[34519]: pgmap v754: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 34 op/s 2024-09-21T07:25:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:09 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:25:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 -- 172.21.15.27:0/1554949208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 msgr2=0x7f8bf010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1554949208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 0x7f8bf010b900 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009900 tx=0x7f8bd802a410 comp rx=0 tx=0).stop 2024-09-21T07:25:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 -- 172.21.15.27:0/1554949208 shutdown_connections 2024-09-21T07:25:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1554949208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf010be40 0x7f8bf0112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1554949208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 0x7f8bf010b900 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1554949208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf0105bb0 0x7f8bf0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 -- 172.21.15.27:0/1554949208 >> 172.21.15.27:0/1554949208 conn(0x7f8bf0101680 msgr2=0x7f8bf0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 -- 172.21.15.27:0/1554949208 shutdown_connections 2024-09-21T07:25:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.144+0000 7f8bf6600640 1 -- 172.21.15.27:0/1554949208 wait complete. 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.145+0000 7f8bf6600640 1 Processor -- start 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.145+0000 7f8bf6600640 1 -- start start 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0105bb0 0x7f8bf019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf6600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 0x7f8bf019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf6600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf010be40 0x7f8bf01a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf6600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf0115b00 con 0x7f8bf010b4f0 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf6600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf0115980 con 0x7f8bf0105bb0 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf6600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf0115c80 con 0x7f8bf010be40 2024-09-21T07:25:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8be7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 0x7f8bf019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0105bb0 0x7f8bf019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf5c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf010be40 0x7f8bf01a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0105bb0 0x7f8bf019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44988/0 (socket says 172.21.15.27:44988) 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.146+0000 7f8bf5200640 1 -- 172.21.15.27:0/1710558980 learned_addr learned my addr 172.21.15.27:0/1710558980 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.147+0000 7f8bf5c00640 1 -- 172.21.15.27:0/1710558980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0105bb0 msgr2=0x7f8bf019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.147+0000 7f8bf5c00640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0105bb0 0x7f8bf019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.147+0000 7f8bf5c00640 1 -- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 msgr2=0x7f8bf019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.147+0000 7f8bf5c00640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 0x7f8bf019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.147+0000 7f8bf5c00640 1 -- 172.21.15.27:0/1710558980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf01a2480 con 0x7f8bf010be40 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.147+0000 7f8be7e00640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 0x7f8bf019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.147+0000 7f8bf5c00640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf010be40 0x7f8bf01a1d50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009fd0 tx=0x7f8be800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.148+0000 7f8bee000640 1 -- 172.21.15.27:0/1710558980 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be800ce20 con 0x7f8bf010be40 2024-09-21T07:25:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.148+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf01a2770 con 0x7f8bf010be40 2024-09-21T07:25:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.148+0000 7f8bee000640 1 -- 172.21.15.27:0/1710558980 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be8010430 con 0x7f8bf010be40 2024-09-21T07:25:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.148+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf01a8ed0 con 0x7f8bf010be40 2024-09-21T07:25:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.148+0000 7f8bee000640 1 -- 172.21.15.27:0/1710558980 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be800f620 con 0x7f8bf010be40 2024-09-21T07:25:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.149+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc0052c0 con 0x7f8bf010be40 2024-09-21T07:25:10.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.151+0000 7f8bee000640 1 -- 172.21.15.27:0/1710558980 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8be8004a40 con 0x7f8bf010be40 2024-09-21T07:25:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.153+0000 7f8bee000640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bd40b6700 0x7f8bd40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.153+0000 7f8bee000640 1 -- 172.21.15.27:0/1710558980 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9549+0+0 (secure 0 0 0) 0x7f8be80e8090 con 0x7f8bf010be40 2024-09-21T07:25:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.153+0000 7f8bf5200640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bd40b6700 0x7f8bd40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.154+0000 7f8bf5200640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bd40b6700 0x7f8bd40b8c10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8be0005fd0 tx=0x7f8be000b480 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:10.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.155+0000 7f8bee000640 1 -- 172.21.15.27:0/1710558980 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8be80ae1e0 con 0x7f8bf010be40 2024-09-21T07:25:10.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.390+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bcc0055b0 con 0x7f8bf010be40 2024-09-21T07:25:10.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.392+0000 7f8bee000640 1 -- 172.21.15.27:0/1710558980 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8be80b25c0 con 0x7f8bf010be40 2024-09-21T07:25:10.394 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:25:10.394 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:25:10.394 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:25:10.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bd40b6700 msgr2=0x7f8bd40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:10.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bd40b6700 0x7f8bd40b8c10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8be0005fd0 tx=0x7f8be000b480 comp rx=0 tx=0).stop 2024-09-21T07:25:10.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf010be40 msgr2=0x7f8bf01a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:10.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf010be40 0x7f8bf01a1d50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009fd0 tx=0x7f8be800ef10 comp rx=0 tx=0).stop 2024-09-21T07:25:10.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 shutdown_connections 2024-09-21T07:25:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf010be40 0x7f8bf01a1d50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bd40b6700 0x7f8bd40b8c10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf010b4f0 0x7f8bf019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 --2- 172.21.15.27:0/1710558980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0105bb0 0x7f8bf019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 >> 172.21.15.27:0/1710558980 conn(0x7f8bf0101680 msgr2=0x7f8bf0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 shutdown_connections 2024-09-21T07:25:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:10.394+0000 7f8bf6600640 1 -- 172.21.15.27:0/1710558980 wait complete. 2024-09-21T07:25:11.194 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:25:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:11 smithi153 ceph-mon[34747]: pgmap v755: 129 pgs: 129 active+clean; 2.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-09-21T07:25:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:11 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1710558980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:11.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:11 smithi119 ceph-mon[34519]: pgmap v755: 129 pgs: 129 active+clean; 2.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-09-21T07:25:11.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:11 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1710558980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:11 smithi027 ceph-mon[30779]: pgmap v755: 129 pgs: 129 active+clean; 2.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 35 op/s 2024-09-21T07:25:11.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:11 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1710558980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:12.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:12 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:25:12.324 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:12 smithi027 ceph-mon[30779]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T07:25:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 -- 172.21.15.27:0/1582218727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 msgr2=0x7f55b8112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 --2- 172.21.15.27:0/1582218727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 0x7f55b8112840 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009be0 tx=0x7f55b002a6c0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 -- 172.21.15.27:0/1582218727 shutdown_connections 2024-09-21T07:25:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 --2- 172.21.15.27:0/1582218727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55b810b6c0 0x7f55b810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 --2- 172.21.15.27:0/1582218727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 0x7f55b8112840 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 --2- 172.21.15.27:0/1582218727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55b8105d80 0x7f55b8106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 -- 172.21.15.27:0/1582218727 >> 172.21.15.27:0/1582218727 conn(0x7f55b8101810 msgr2=0x7f55b8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 -- 172.21.15.27:0/1582218727 shutdown_connections 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.255+0000 7f55bd400640 1 -- 172.21.15.27:0/1582218727 wait complete. 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.256+0000 7f55bd400640 1 Processor -- start 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55bd400640 1 -- start start 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55bd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55b8105d80 0x7f55b8074e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55bd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55b810b6c0 0x7f55b806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55bd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 0x7f55b806e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55bd400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8115cd0 con 0x7f55b810b6c0 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55bd400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8115b50 con 0x7f55b810c010 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55bd400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8115e50 con 0x7f55b8105d80 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55b5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55b810b6c0 0x7f55b806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.257+0000 7f55b6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55b8105d80 0x7f55b8074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55b8105d80 0x7f55b8074e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49238/0 (socket says 172.21.15.27:49238) 2024-09-21T07:25:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 0x7f55b806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b6800640 1 -- 172.21.15.27:0/3572919916 learned_addr learned my addr 172.21.15.27:0/3572919916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b7200640 1 -- 172.21.15.27:0/3572919916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55b8105d80 msgr2=0x7f55b8074e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b7200640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55b8105d80 0x7f55b8074e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b7200640 1 -- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55b810b6c0 msgr2=0x7f55b806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b7200640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55b810b6c0 0x7f55b806e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b7200640 1 -- 172.21.15.27:0/3572919916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b806ee50 con 0x7f55b810c010 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.258+0000 7f55b7200640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 0x7f55b806e630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009b30 tx=0x7f55b0003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.259+0000 7f55af400640 1 -- 172.21.15.27:0/3572919916 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0041020 con 0x7f55b810c010 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.259+0000 7f55af400640 1 -- 172.21.15.27:0/3572919916 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b00318f0 con 0x7f55b810c010 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.259+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b8073a90 con 0x7f55b810c010 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.259+0000 7f55af400640 1 -- 172.21.15.27:0/3572919916 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0036040 con 0x7f55b810c010 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.259+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b8073f90 con 0x7f55b810c010 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.262+0000 7f55c40c1740 1 Processor -- start 2024-09-21T07:25:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.262+0000 7f55af400640 1 -- 172.21.15.27:0/3572919916 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f55b0044020 con 0x7f55b810c010 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.263+0000 7f55c40c1740 1 -- start start 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.263+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd54070 0x555e2e0fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.263+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.263+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e0fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.263+0000 7f55c40c1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555e2e071fb0 con 0x555e2dd2b1b0 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.263+0000 7f55c40c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555e2e0613e0 con 0x555e2dd54070 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.263+0000 7f55c40c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555e2e0651f0 con 0x555e2dd33390 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e0fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e0fb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49240/0 (socket says 172.21.15.27:49240) 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 -- 172.21.15.27:0/2084715315 learned_addr learned my addr 172.21.15.27:0/2084715315 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b6800640 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd54070 0x555e2e0fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 -- 172.21.15.27:0/2084715315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd54070 msgr2=0x555e2e0fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd54070 0x555e2e0fada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 -- 172.21.15.27:0/2084715315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd2b1b0 msgr2=0x555e2e0fb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 -- 172.21.15.27:0/2084715315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e2e02bcd0 con 0x555e2dd33390 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.264+0000 7f55b5e00640 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e0fb820 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f55a4011710 tx=0x7f55a4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6866ed9ed985105 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.265+0000 7f55a3400640 1 -- 172.21.15.27:0/2084715315 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4005020 con 0x555e2dd33390 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.265+0000 7f55af400640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f559c0b6700 0x7f559c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.265+0000 7f55a3400640 1 -- 172.21.15.27:0/2084715315 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a400f840 con 0x555e2dd33390 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.265+0000 7f55a3400640 1 -- 172.21.15.27:0/2084715315 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4004a00 con 0x555e2dd33390 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.265+0000 7f55af400640 1 -- 172.21.15.27:0/3572919916 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f55b0108510 con 0x7f55b810c010 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.265+0000 7f55b6800640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f559c0b6700 0x7f559c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 -- 172.21.15.27:0/2084715315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 msgr2=0x555e2e0fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e0fb820 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f55a4011710 tx=0x7f55a4005850 comp rx=0 tx=0).stop 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 -- 172.21.15.27:0/2084715315 shutdown_connections 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e0fb820 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 --2- 172.21.15.27:0/2084715315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd54070 0x555e2e0fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 -- 172.21.15.27:0/2084715315 >> 172.21.15.27:0/2084715315 conn(0x555e2dd48460 msgr2=0x555e2e0f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55b6800640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f559c0b6700 0x7f559c0b8c10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x555e2e0f9a20 tx=0x7f55a8009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 -- 172.21.15.27:0/2084715315 shutdown_connections 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.266+0000 7f55c40c1740 1 -- 172.21.15.27:0/2084715315 wait complete. 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.267+0000 7f55c40c1740 1 -- start start 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.267+0000 7f55c40c1740 1 Processor -- start 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e1065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e1128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55c40c1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555e2dec2700 con 0x555e2dd54070 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55c40c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555e2dffc580 con 0x555e2dd33390 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55c40c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555e2e109580 con 0x555e2dd2b1b0 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55c40c1740 10 client.0 _ll_register_callbacks cb 0x555e2e00c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55b5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55b6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e1065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55b5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45022/0 (socket says 172.21.15.27:45022) 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55b5e00640 1 -- 172.21.15.27:0/932049749 learned_addr learned my addr 172.21.15.27:0/932049749 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.268+0000 7f55b7200640 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e1128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f55b5e00640 1 -- 172.21.15.27:0/932049749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 msgr2=0x555e2e1065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f55b5e00640 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e1065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f55b5e00640 1 -- 172.21.15.27:0/932049749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 msgr2=0x555e2e1128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f55b5e00640 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e1128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f55b5e00640 1 -- 172.21.15.27:0/932049749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e2ddbf650 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f55b5e00640 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e106ae0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55a40058c0 tx=0x7f55a4004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d919fc988dc87cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f5597e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.269+0000 7f5597e00640 1 -- 172.21.15.27:0/932049749 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a402b060 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.270+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e2dffeb90 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.270+0000 7f5597e00640 1 -- 172.21.15.27:0/932049749 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f55a4005230 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.270+0000 7f5597e00640 1 -- 172.21.15.27:0/932049749 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f55a4032050 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.270+0000 7f55c40c1740 10 client.29072 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.270+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e2e019950 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.270+0000 7f5597e00640 1 -- 172.21.15.27:0/932049749 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555e2dffeb90 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.270+0000 7f5597e00640 1 -- 172.21.15.27:0/932049749 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f55a401d030 con 0x555e2dd33390 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.271+0000 7f55c40c1740 10 client.29072 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.271+0000 7f55c40c1740 10 client.29072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:25:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.271+0000 7f55c40c1740 20 client.29072 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.271+0000 7f55c40c1740 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dd129f0 0x555e2e118540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.271+0000 7f55c40c1740 4 client.29072 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.271+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e2e073f00 con 0x555e2dd129f0 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.271+0000 7f55b6800640 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dd129f0 0x555e2e118540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.272+0000 7f55b6800640 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dd129f0 0x555e2e118540 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d852bd4cdee6328 server_cookie=1f077c124ca7b654 in_seq=0 out_seq=0 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.272+0000 7f5597e00640 10 client.29072 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.272+0000 7f5597e00640 1 -- 172.21.15.27:0/932049749 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555e2e073f00 con 0x555e2dd129f0 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.272+0000 7f5597e00640 10 client.29072 handle_command_reply: tid=0 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.272+0000 7f55c40c1740 1 client.29072 shutdown 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 msgr2=0x555e2e106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e106ae0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55a40058c0 tx=0x7f55a4004500 comp rx=0 tx=0).stop 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 shutdown_connections 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dd129f0 0x555e2e118540 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e1128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e106ae0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 --2- 172.21.15.27:0/932049749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e1065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f5597e00640 0 client.29072 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 >> 172.21.15.27:0/932049749 conn(0x555e2dd48460 msgr2=0x555e2e0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 shutdown_connections 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 1 -- 172.21.15.27:0/932049749 wait complete. 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.273+0000 7f55c40c1740 20 client.29072 trim_cache size 0 max 16384 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.288+0000 7f55c40c1740 1 Processor -- start 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.288+0000 7f55c40c1740 1 -- start start 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e103790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e103cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55c40c1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81be0a0 con 0x555e2dd54070 2024-09-21T07:25:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55c40c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81bb970 con 0x555e2dd33390 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55c40c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81b97e0 con 0x555e2dd2b1b0 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e103790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45034/0 (socket says 172.21.15.27:45034) 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 -- 172.21.15.27:0/679614878 learned_addr learned my addr 172.21.15.27:0/679614878 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b7200640 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e103cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 -- 172.21.15.27:0/679614878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 msgr2=0x555e2e0fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 -- 172.21.15.27:0/679614878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 msgr2=0x555e2e103cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e103cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.289+0000 7f55b5e00640 1 -- 172.21.15.27:0/679614878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e2e107860 con 0x555e2dd33390 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.290+0000 7f55b7200640 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.290+0000 7f55b5e00640 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e103790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f55a40057c0 tx=0x7f55a40114a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef9cf01e927624 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.290+0000 7f55a2a00640 1 -- 172.21.15.27:0/679614878 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a402b030 con 0x555e2dd33390 2024-09-21T07:25:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.290+0000 7f55a2a00640 1 -- 172.21.15.27:0/679614878 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a4004cd0 con 0x555e2dd33390 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.290+0000 7f55a2a00640 1 -- 172.21.15.27:0/679614878 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4004570 con 0x555e2dd33390 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.291+0000 7f55c40c1740 1 -- 172.21.15.27:0/679614878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 msgr2=0x555e2e103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.291+0000 7f55c40c1740 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e103790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f55a40057c0 tx=0x7f55a40114a0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.291+0000 7f55c40c1740 1 -- 172.21.15.27:0/679614878 shutdown_connections 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.291+0000 7f55c40c1740 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.291+0000 7f55c40c1740 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd33390 0x555e2e103790 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.291+0000 7f55c40c1740 1 --2- 172.21.15.27:0/679614878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd2b1b0 0x555e2e0fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.291+0000 7f55c40c1740 1 -- 172.21.15.27:0/679614878 >> 172.21.15.27:0/679614878 conn(0x555e2dd48460 msgr2=0x555e2e0fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.292+0000 7f55c40c1740 1 -- 172.21.15.27:0/679614878 shutdown_connections 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.292+0000 7f55c40c1740 1 -- 172.21.15.27:0/679614878 wait complete. 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.292+0000 7f55c40c1740 1 -- start start 2024-09-21T07:25:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.292+0000 7f55c40c1740 1 Processor -- start 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd2b1b0 0x555e2e10a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e10adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55c40c1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e10b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55c40c1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81b9e30 con 0x555e2dd54070 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55c40c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81bb2a0 con 0x555e2dd2b1b0 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55c40c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81ba500 con 0x555e2dd33390 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55c40c1740 10 client.0 _ll_register_callbacks cb 0x7f55a4011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55b6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd2b1b0 0x555e2e10a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55b6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd2b1b0 0x555e2e10a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45036/0 (socket says 172.21.15.27:45036) 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55b6800640 1 -- 172.21.15.27:0/3420752978 learned_addr learned my addr 172.21.15.27:0/3420752978 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55b5e00640 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e10adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.293+0000 7f55b7200640 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e10b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f55b6800640 1 -- 172.21.15.27:0/3420752978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 msgr2=0x555e2e10adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f55b6800640 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e10adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f55b6800640 1 -- 172.21.15.27:0/3420752978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 msgr2=0x555e2e10b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f55b6800640 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e10b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f55b6800640 1 -- 172.21.15.27:0/3420752978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e2e181d90 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f55b6800640 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd2b1b0 0x555e2e10a870 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f55a8013010 tx=0x7f55a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71f4c2a1308dee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f5597e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.294+0000 7f5597e00640 1 -- 172.21.15.27:0/3420752978 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8036060 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a4004cd0 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f5597e00640 1 -- 172.21.15.27:0/3420752978 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f55a8031470 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f5597e00640 1 -- 172.21.15.27:0/3420752978 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f55a803d050 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f55c40c1740 10 client.29078 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e2e183070 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f5597e00640 1 -- 172.21.15.27:0/3420752978 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a4004cd0 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f5597e00640 1 -- 172.21.15.27:0/3420752978 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f55a802a6f0 con 0x555e2dd2b1b0 2024-09-21T07:25:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f55c40c1740 10 client.29078 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f55c40c1740 10 client.29078 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f55c40c1740 20 client.29078 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.295+0000 7f55c40c1740 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dce7da0 0x555e2e185810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.296+0000 7f55c40c1740 4 client.29078 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.296+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e2e06f9b0 con 0x555e2dce7da0 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.296+0000 7f55b5e00640 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dce7da0 0x555e2e185810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.296+0000 7f55b5e00640 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dce7da0 0x555e2e185810 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70939d77944166c3 server_cookie=7dbc13bf967abeb8 in_seq=0 out_seq=0 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.296+0000 7f5597e00640 10 client.29078 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f5597e00640 1 -- 172.21.15.27:0/3420752978 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e2e06f9b0 con 0x555e2dce7da0 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f5597e00640 10 client.29078 handle_command_reply: tid=0 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 client.29078 shutdown 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd2b1b0 msgr2=0x555e2e10a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd2b1b0 0x555e2e10a870 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f55a8013010 tx=0x7f55a80046c0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 shutdown_connections 2024-09-21T07:25:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x555e2dce7da0 0x555e2e185810 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555e2dd33390 0x555e2e10adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555e2dd54070 0x555e2e10b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f5597e00640 0 client.29078 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 --2- 172.21.15.27:0/3420752978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e2dd2b1b0 0x555e2e10a870 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 >> 172.21.15.27:0/3420752978 conn(0x555e2dd48460 msgr2=0x555e2e0fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 shutdown_connections 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 1 -- 172.21.15.27:0/3420752978 wait complete. 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.297+0000 7f55c40c1740 20 client.29078 trim_cache size 0 max 16384 2024-09-21T07:25:12.342 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f559c0b6700 msgr2=0x7f559c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f559c0b6700 0x7f559c0b8c10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x555e2e0f9a20 tx=0x7f55a8009260 comp rx=0 tx=0).stop 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 msgr2=0x7f55b806e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 0x7f55b806e630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009b30 tx=0x7f55b0003b80 comp rx=0 tx=0).stop 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 shutdown_connections 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f559c0b6700 0x7f559c0b8c10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b810c010 0x7f55b806e630 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55b810b6c0 0x7f55b806e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 --2- 172.21.15.27:0/3572919916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55b8105d80 0x7f55b8074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 >> 172.21.15.27:0/3572919916 conn(0x7f55b8101810 msgr2=0x7f55b810ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 shutdown_connections 2024-09-21T07:25:12.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:12.298+0000 7f55bd400640 1 -- 172.21.15.27:0/3572919916 wait complete. 2024-09-21T07:25:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:12 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:25:12.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:12 smithi153 ceph-mon[34747]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T07:25:12.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:12 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:25:12.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:12 smithi119 ceph-mon[34519]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T07:25:13.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:25:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:13 smithi027 ceph-mon[30779]: pgmap v756: 129 pgs: 129 active+clean; 2.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-21T07:25:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:25:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:25:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:25:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:13 smithi153 ceph-mon[34747]: pgmap v756: 129 pgs: 129 active+clean; 2.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-21T07:25:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:25:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:25:13.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:25:13.438 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:13 smithi119 ceph-mon[34519]: pgmap v756: 129 pgs: 129 active+clean; 2.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-21T07:25:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:25:13.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:25:13.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 -- 172.21.15.27:0/1089888390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c78105d80 msgr2=0x7f9c78106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 --2- 172.21.15.27:0/1089888390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c78105d80 0x7f9c78106190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009900 tx=0x7f9c6802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 -- 172.21.15.27:0/1089888390 shutdown_connections 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 --2- 172.21.15.27:0/1089888390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7810c010 0x7f9c78112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 --2- 172.21.15.27:0/1089888390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c7810b6c0 0x7f9c7810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 --2- 172.21.15.27:0/1089888390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c78105d80 0x7f9c78106190 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 -- 172.21.15.27:0/1089888390 >> 172.21.15.27:0/1089888390 conn(0x7f9c781017d0 msgr2=0x7f9c78103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.127+0000 7f9c7e600640 1 -- 172.21.15.27:0/1089888390 shutdown_connections 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.128+0000 7f9c7e600640 1 -- 172.21.15.27:0/1089888390 wait complete. 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.128+0000 7f9c7e600640 1 Processor -- start 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c7e600640 1 -- start start 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c7e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c78105d80 0x7f9c7819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c7e600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c7810b6c0 0x7f9c7819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c7e600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7810c010 0x7f9c781a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c7e600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78115d20 con 0x7f9c7810b6c0 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c7e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78115ba0 con 0x7f9c78105d80 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c7e600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78115ea0 con 0x7f9c7810c010 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c76a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c7810b6c0 0x7f9c7819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c76a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c7810b6c0 0x7f9c7819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40882/0 (socket says 172.21.15.27:40882) 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.129+0000 7f9c76a00640 1 -- 172.21.15.27:0/3374481481 learned_addr learned my addr 172.21.15.27:0/3374481481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.130+0000 7f9c77400640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c78105d80 0x7f9c7819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.130+0000 7f9c77e00640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7810c010 0x7f9c781a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.130+0000 7f9c76a00640 1 -- 172.21.15.27:0/3374481481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7810c010 msgr2=0x7f9c781a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.130+0000 7f9c76a00640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7810c010 0x7f9c781a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.130+0000 7f9c76a00640 1 -- 172.21.15.27:0/3374481481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c78105d80 msgr2=0x7f9c7819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.130+0000 7f9c76a00640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c78105d80 0x7f9c7819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.130+0000 7f9c76a00640 1 -- 172.21.15.27:0/3374481481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c781a2620 con 0x7f9c7810b6c0 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.131+0000 7f9c76a00640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c7810b6c0 0x7f9c7819cc50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400e9f0 tx=0x7f9c6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.131+0000 7f9c6fe00640 1 -- 172.21.15.27:0/3374481481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400ccc0 con 0x7f9c7810b6c0 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.131+0000 7f9c6fe00640 1 -- 172.21.15.27:0/3374481481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c640108e0 con 0x7f9c7810b6c0 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.131+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c781a2910 con 0x7f9c7810b6c0 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.131+0000 7f9c6fe00640 1 -- 172.21.15.27:0/3374481481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64018860 con 0x7f9c7810b6c0 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.131+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c781a9070 con 0x7f9c7810b6c0 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 Processor -- start 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 -- start start 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ac820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a9a6322fb0 con 0x55a9a5fe4390 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a9a63123e0 con 0x55a9a6005070 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c85208740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a9a63161f0 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.135+0000 7f9c77400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45062/0 (socket says 172.21.15.27:45062) 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77e00640 1 -- 172.21.15.27:0/1361801476 learned_addr learned my addr 172.21.15.27:0/1361801476 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c6fe00640 1 -- 172.21.15.27:0/3374481481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9c64010430 con 0x7f9c7810b6c0 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77400640 1 -- 172.21.15.27:0/1361801476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 msgr2=0x55a9a63abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77400640 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77400640 1 -- 172.21.15.27:0/1361801476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 msgr2=0x55a9a63ac820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77400640 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77400640 1 -- 172.21.15.27:0/1361801476 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a62dccd0 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.136+0000 7f9c77e00640 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.137+0000 7f9c77400640 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ac2e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802a9e0 tx=0x7f9c68004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b1b28ba7bb2fbee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.137+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1361801476 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68035040 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.137+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1361801476 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c680054f0 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.137+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1361801476 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680035f0 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 -- 172.21.15.27:0/1361801476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 msgr2=0x55a9a63ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ac2e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802a9e0 tx=0x7f9c68004040 comp rx=0 tx=0).stop 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c6fe00640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 -- 172.21.15.27:0/1361801476 shutdown_connections 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ac2e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 --2- 172.21.15.27:0/1361801476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 -- 172.21.15.27:0/1361801476 >> 172.21.15.27:0/1361801476 conn(0x55a9a5ff9460 msgr2=0x55a9a63a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c6fe00640 1 -- 172.21.15.27:0/3374481481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f9c640e7d50 con 0x7f9c7810b6c0 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c77e00640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 -- 172.21.15.27:0/1361801476 shutdown_connections 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.138+0000 7f9c85208740 1 -- 172.21.15.27:0/1361801476 wait complete. 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.139+0000 7f9c77e00640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9c70005fd0 tx=0x7f9c70009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.139+0000 7f9c85208740 1 -- start start 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.139+0000 7f9c85208740 1 Processor -- start 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c85208740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c85208740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c85208740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c85208740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a9a6173700 con 0x55a9a5fe4390 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c85208740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a9a62ad580 con 0x55a9a6005070 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c85208740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a9a63ba520 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c85208740 10 client.0 _ll_register_callbacks cb 0x55a9a62bd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c77400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c76a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c77400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49314/0 (socket says 172.21.15.27:49314) 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c77400640 1 -- 172.21.15.27:0/2755321355 learned_addr learned my addr 172.21.15.27:0/2755321355 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.140+0000 7f9c77e00640 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c77400640 1 -- 172.21.15.27:0/2755321355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 msgr2=0x55a9a63c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c77400640 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c77400640 1 -- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 msgr2=0x55a9a63c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c77400640 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c77400640 1 -- 172.21.15.27:0/2755321355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a6070650 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c76a00640 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c77400640 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c0660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802a910 tx=0x7f9c68009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc049811f3289d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c61600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c61600640 1 -- 172.21.15.27:0/2755321355 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68052060 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9a62afb90 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.141+0000 7f9c61600640 1 -- 172.21.15.27:0/2755321355 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f9c680037f0 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c61600640 1 -- 172.21.15.27:0/2755321355 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c68035040 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 10 client.29068 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9a62ca950 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c61600640 1 -- 172.21.15.27:0/2755321355 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a9a62afb90 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c61600640 1 -- 172.21.15.27:0/2755321355 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9c6804c070 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 10 client.29068 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 10 client.29068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:25:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 20 client.29068 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5fc39f0 0x55a9a63d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 4 client.29068 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.142+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9a6324f00 con 0x55a9a5fc39f0 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.143+0000 7f9c76a00640 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5fc39f0 0x55a9a63d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.143+0000 7f9c76a00640 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5fc39f0 0x55a9a63d1d40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30fa5622c27184a8 server_cookie=8cd1d3ae93fbd7ee in_seq=0 out_seq=0 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.143+0000 7f9c61600640 10 client.29068 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c61600640 1 -- 172.21.15.27:0/2755321355 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9a6324f00 con 0x55a9a5fc39f0 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c61600640 10 client.29068 handle_command_reply: tid=0 2024-09-21T07:25:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 client.29068 shutdown 2024-09-21T07:25:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 msgr2=0x55a9a63c0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c0660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802a910 tx=0x7f9c68009260 comp rx=0 tx=0).stop 2024-09-21T07:25:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 shutdown_connections 2024-09-21T07:25:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5fc39f0 0x55a9a63d1d40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 --2- 172.21.15.27:0/2755321355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c0660 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c61600640 0 client.29068 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.144+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 >> 172.21.15.27:0/2755321355 conn(0x55a9a5ff9460 msgr2=0x55a9a63aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.145+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 shutdown_connections 2024-09-21T07:25:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.145+0000 7f9c85208740 1 -- 172.21.15.27:0/2755321355 wait complete. 2024-09-21T07:25:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.145+0000 7f9c85208740 20 client.29068 trim_cache size 0 max 16384 2024-09-21T07:25:14.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 Processor -- start 2024-09-21T07:25:14.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 -- start start 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781b1540 con 0x55a9a5fe4390 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781aee10 con 0x55a9a6005070 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.159+0000 7f9c85208740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781acc80 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c76a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49328/0 (socket says 172.21.15.27:49328) 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 -- 172.21.15.27:0/1946464732 learned_addr learned my addr 172.21.15.27:0/1946464732 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77e00640 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 -- 172.21.15.27:0/1946464732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 msgr2=0x55a9a63b98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:14.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63b98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 -- 172.21.15.27:0/1946464732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 msgr2=0x55a9a63ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:14.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 -- 172.21.15.27:0/1946464732 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a63ae090 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c76a00640 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.160+0000 7f9c77400640 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c6360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9c68031440 tx=0x7f9c680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d1317db2ceee7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.161+0000 7f9c63e00640 1 -- 172.21.15.27:0/1946464732 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68062070 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.161+0000 7f9c63e00640 1 -- 172.21.15.27:0/1946464732 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c68004e30 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.161+0000 7f9c63e00640 1 -- 172.21.15.27:0/1946464732 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68052070 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 -- 172.21.15.27:0/1946464732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 msgr2=0x55a9a63c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c6360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9c68031440 tx=0x7f9c680046c0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 -- 172.21.15.27:0/1946464732 shutdown_connections 2024-09-21T07:25:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a6005070 0x55a9a63b98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fe4390 0x55a9a63ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 --2- 172.21.15.27:0/1946464732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63c6360 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 -- 172.21.15.27:0/1946464732 >> 172.21.15.27:0/1946464732 conn(0x55a9a5ff9460 msgr2=0x55a9a63acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 -- 172.21.15.27:0/1946464732 shutdown_connections 2024-09-21T07:25:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 -- 172.21.15.27:0/1946464732 wait complete. 2024-09-21T07:25:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.162+0000 7f9c85208740 1 -- start start 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 1 Processor -- start 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a5fe4390 0x55a9a63ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a6005070 0x55a9a643b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781ad2d0 con 0x55a9a5fdc1b0 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781ae740 con 0x55a9a5fe4390 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781ad9a0 con 0x55a9a6005070 2024-09-21T07:25:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c85208740 10 client.0 _ll_register_callbacks cb 0x7f9c68005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c76a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a5fe4390 0x55a9a63ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40930/0 (socket says 172.21.15.27:40930) 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.163+0000 7f9c77400640 1 -- 172.21.15.27:0/924485427 learned_addr learned my addr 172.21.15.27:0/924485427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c77e00640 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a6005070 0x55a9a643b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c76a00640 1 -- 172.21.15.27:0/924485427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a6005070 msgr2=0x55a9a643b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c76a00640 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a6005070 0x55a9a643b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c76a00640 1 -- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fdc1b0 msgr2=0x55a9a63ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c76a00640 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c76a00640 1 -- 172.21.15.27:0/924485427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a643ba00 con 0x55a9a5fe4390 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c77400640 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c76a00640 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a5fe4390 0x55a9a63ca990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9c640e50f0 tx=0x7f9c640096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7bf74bb5622256 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.164+0000 7f9c61600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:14.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c61600640 1 -- 172.21.15.27:0/924485427 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640d2030 con 0x55a9a5fe4390 2024-09-21T07:25:14.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c61600640 1 -- 172.21.15.27:0/924485427 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f9c640aeb20 con 0x55a9a5fe4390 2024-09-21T07:25:14.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c68004e30 con 0x55a9a5fe4390 2024-09-21T07:25:14.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c61600640 1 -- 172.21.15.27:0/924485427 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c64010720 con 0x55a9a5fe4390 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 10 client.29096 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9a643cb30 con 0x55a9a5fe4390 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c61600640 1 -- 172.21.15.27:0/924485427 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c68004e30 con 0x55a9a5fe4390 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c61600640 1 -- 172.21.15.27:0/924485427 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9c640ac570 con 0x55a9a5fe4390 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 10 client.29096 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 10 client.29096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 20 client.29096 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5f98da0 0x55a9a643f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 4 client.29096 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.165+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9a63209b0 con 0x55a9a5f98da0 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.166+0000 7f9c77400640 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5f98da0 0x55a9a643f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.166+0000 7f9c77400640 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5f98da0 0x55a9a643f1c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec97ce301267eeb2 server_cookie=16bd7acd089cbf3c in_seq=0 out_seq=0 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.166+0000 7f9c61600640 10 client.29096 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c61600640 1 -- 172.21.15.27:0/924485427 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9a63209b0 con 0x55a9a5f98da0 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c61600640 10 client.29096 handle_command_reply: tid=0 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 client.29096 shutdown 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a5fe4390 msgr2=0x55a9a63ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a5fe4390 0x55a9a63ca990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9c640e50f0 tx=0x7f9c640096a0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 shutdown_connections 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a9a5f98da0 0x55a9a643f1c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9a6005070 0x55a9a643b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a5fe4390 0x55a9a63ca990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 --2- 172.21.15.27:0/924485427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a9a5fdc1b0 0x55a9a63ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c61600640 0 client.29096 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 >> 172.21.15.27:0/924485427 conn(0x55a9a5ff9460 msgr2=0x55a9a63c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 shutdown_connections 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 1 -- 172.21.15.27:0/924485427 wait complete. 2024-09-21T07:25:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.167+0000 7f9c85208740 20 client.29096 trim_cache size 0 max 16384 2024-09-21T07:25:14.171 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:25:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9c5c0b6680 msgr2=0x7f9c5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9c70005fd0 tx=0x7f9c70009500 comp rx=0 tx=0).stop 2024-09-21T07:25:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c7810b6c0 msgr2=0x7f9c7819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c7810b6c0 0x7f9c7819cc50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400e9f0 tx=0x7f9c6400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 shutdown_connections 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c7810c010 0x7f9c781a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9c5c0b6680 0x7f9c5c0b8b90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c7810b6c0 0x7f9c7819cc50 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 --2- 172.21.15.27:0/3374481481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c78105d80 0x7f9c7819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 >> 172.21.15.27:0/3374481481 conn(0x7f9c781017d0 msgr2=0x7f9c78103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 shutdown_connections 2024-09-21T07:25:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:14.168+0000 7f9c7e600640 1 -- 172.21.15.27:0/3374481481 wait complete. 2024-09-21T07:25:14.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:25:15.300 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:15.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:15 smithi027 ceph-mon[30779]: pgmap v758: 129 pgs: 129 active+clean; 2.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-21T07:25:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:15 smithi153 ceph-mon[34747]: pgmap v758: 129 pgs: 129 active+clean; 2.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-21T07:25:15.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:15 smithi119 ceph-mon[34519]: pgmap v758: 129 pgs: 129 active+clean; 2.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-09-21T07:25:15.650 INFO:tasks.workunit.client.0.smithi027.stdout:375/600: open fd = 3 2024-09-21T07:25:15.650 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.650 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.650 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.650 INFO:tasks.workunit.client.0.smithi027.stdout:376/600: open fd = 3 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:377/600: open fd = 3 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:378/600: open fd = 3 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:379/600: open fd = 3 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:380/600: open fd = 3 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:381/600: open fd = 3 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:382/600: open fd = 3 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:383/600: open fd = 3 2024-09-21T07:25:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:384/600: open fd = 3 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:385/600: open fd = 3 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:386/600: open fd = 3 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:387/600: open fd = 3 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:388/600: open fd = 3 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:389/600: open fd = 3 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:390/600: open fd = 3 2024-09-21T07:25:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:391/600: open fd = 3 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:392/600: open fd = 3 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:393/600: open fd = 3 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:394/600: open fd = 3 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:395/600: open fd = 3 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:396/600: open fd = 3 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:397/600: open fd = 3 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:398/600: open fd = 3 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:399/600: open fd = 3 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:400/600: open fd = 3 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:401/600: open fd = 3 2024-09-21T07:25:15.657 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:402/600: open fd = 3 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:403/600: open fd = 3 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:404/600: open fd = 3 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:405/600: open fd = 3 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:406/600: open fd = 3 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:407/600: open fd = 3 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:408/600: open fd = 3 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:409/600: open fd = 3 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:410/600: open fd = 3 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:411/600: open fd = 3 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:412/600: open fd = 3 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.660 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:413/600: open fd = 3 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:414/600: open fd = 3 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:415/600: open fd = 3 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:416/600: open fd = 3 2024-09-21T07:25:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:417/600: open fd = 3 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:418/600: open fd = 3 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:419/600: open fd = 3 2024-09-21T07:25:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:420/600: open fd = 3 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:421/600: open fd = 3 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:422/600: open fd = 3 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.663 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:423/600: open fd = 3 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:424/600: open fd = 3 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:425/600: open fd = 3 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:426/600: open fd = 3 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:427/600: open fd = 3 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:428/600: open fd = 3 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:429/600: open fd = 3 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:430/600: open fd = 3 2024-09-21T07:25:15.665 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:431/600: open fd = 3 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:432/600: open fd = 3 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:433/600: open fd = 3 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.666 INFO:tasks.workunit.client.0.smithi027.stdout:434/600: open fd = 3 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:435/600: open fd = 3 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:436/600: open fd = 3 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:25:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:25:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 -- 172.21.15.27:0/2579543857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0077940 msgr2=0x7f58f0077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 --2- 172.21.15.27:0/2579543857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0077940 0x7f58f0077d50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009900 tx=0x7f58e002a410 comp rx=0 tx=0).stop 2024-09-21T07:25:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 -- 172.21.15.27:0/2579543857 shutdown_connections 2024-09-21T07:25:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 --2- 172.21.15.27:0/2579543857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58f0078290 0x7f58f0114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 --2- 172.21.15.27:0/2579543857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58f0114740 0x7f58f0116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 --2- 172.21.15.27:0/2579543857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0077940 0x7f58f0077d50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 -- 172.21.15.27:0/2579543857 >> 172.21.15.27:0/2579543857 conn(0x7f58f0101810 msgr2=0x7f58f0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 -- 172.21.15.27:0/2579543857 shutdown_connections 2024-09-21T07:25:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.935+0000 7f58f5000640 1 -- 172.21.15.27:0/2579543857 wait complete. 2024-09-21T07:25:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.936+0000 7f58f5000640 1 Processor -- start 2024-09-21T07:25:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.936+0000 7f58f5000640 1 -- start start 2024-09-21T07:25:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58f5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58f0077940 0x7f58f01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58f5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58f0078290 0x7f58f01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58f5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0114740 0x7f58f01a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58f5000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58f011a0d0 con 0x7f58f0114740 2024-09-21T07:25:15.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58f5000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0119f50 con 0x7f58f0078290 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58f5000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58f011a250 con 0x7f58f0077940 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58ee800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0114740 0x7f58f01a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58ee800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0114740 0x7f58f01a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40952/0 (socket says 172.21.15.27:40952) 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58ede00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58f0077940 0x7f58f01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58ee800640 1 -- 172.21.15.27:0/3032004716 learned_addr learned my addr 172.21.15.27:0/3032004716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.937+0000 7f58e5400640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58f0078290 0x7f58f01a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58ee800640 1 -- 172.21.15.27:0/3032004716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58f0077940 msgr2=0x7f58f01a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58ee800640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58f0077940 0x7f58f01a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58ee800640 1 -- 172.21.15.27:0/3032004716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58f0078290 msgr2=0x7f58f01a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58ee800640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58f0078290 0x7f58f01a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58ee800640 1 -- 172.21.15.27:0/3032004716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01a6a50 con 0x7f58f0114740 2024-09-21T07:25:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58ee800640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0114740 0x7f58f01a6320 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f58e800eb10 tx=0x7f58e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58e7400640 1 -- 172.21.15.27:0/3032004716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8004b20 con 0x7f58f0114740 2024-09-21T07:25:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58e7400640 1 -- 172.21.15.27:0/3032004716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e8004cc0 con 0x7f58f0114740 2024-09-21T07:25:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.938+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01a6d40 con 0x7f58f0114740 2024-09-21T07:25:15.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.939+0000 7f58e7400640 1 -- 172.21.15.27:0/3032004716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8010670 con 0x7f58f0114740 2024-09-21T07:25:15.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.939+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01ad4a0 con 0x7f58f0114740 2024-09-21T07:25:15.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.942+0000 7f58fbc3b740 1 Processor -- start 2024-09-21T07:25:15.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58e7400640 1 -- 172.21.15.27:0/3032004716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f58e8005020 con 0x7f58f0114740 2024-09-21T07:25:15.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58fbc3b740 1 -- start start 2024-09-21T07:25:15.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e21faba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e21fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e21fb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2171fb0 con 0x55d1e1e33390 2024-09-21T07:25:15.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d1e21613e0 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.943+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d1e21651f0 con 0x55d1e1e54070 2024-09-21T07:25:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e21fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58e5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e21faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e21fb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:45132/0 (socket says 172.21.15.27:45132) 2024-09-21T07:25:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 -- 172.21.15.27:0/2889054497 learned_addr learned my addr 172.21.15.27:0/2889054497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58e5400640 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e21fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 -- 172.21.15.27:0/2889054497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 msgr2=0x55d1e21faba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e21faba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 -- 172.21.15.27:0/2889054497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 msgr2=0x55d1e21fb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e21fb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 -- 172.21.15.27:0/2889054497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e212bcd0 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58e5400640 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e21faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.944+0000 7f58ede00640 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e21fb0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009900 tx=0x7f58e0032c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac43afb06eade08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.945+0000 7f58df400640 1 -- 172.21.15.27:0/2889054497 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0040070 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.945+0000 7f58e7400640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58d40b6700 0x7f58d40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.945+0000 7f58df400640 1 -- 172.21.15.27:0/2889054497 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e0004030 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.945+0000 7f58df400640 1 -- 172.21.15.27:0/2889054497 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0035040 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.945+0000 7f58e7400640 1 -- 172.21.15.27:0/3032004716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f58e80e8080 con 0x7f58f0114740 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.945+0000 7f58e5400640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58d40b6700 0x7f58d40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 -- 172.21.15.27:0/2889054497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 msgr2=0x55d1e21fb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e21fb0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009900 tx=0x7f58e0032c40 comp rx=0 tx=0).stop 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58e5400640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58d40b6700 0x7f58d40b8c10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f58d8005d00 tx=0x7f58d800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 -- 172.21.15.27:0/2889054497 shutdown_connections 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e21fb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e21fb0e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/2889054497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e21faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 -- 172.21.15.27:0/2889054497 >> 172.21.15.27:0/2889054497 conn(0x55d1e1e48460 msgr2=0x55d1e21f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 -- 172.21.15.27:0/2889054497 shutdown_connections 2024-09-21T07:25:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.946+0000 7f58fbc3b740 1 -- 172.21.15.27:0/2889054497 wait complete. 2024-09-21T07:25:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.947+0000 7f58fbc3b740 1 -- start start 2024-09-21T07:25:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.947+0000 7f58fbc3b740 1 Processor -- start 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2206760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2206ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2212670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d1e1fc2700 con 0x55d1e1e33390 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d1e20fc580 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2209320 con 0x55d1e1e54070 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58fbc3b740 10 client.0 _ll_register_callbacks cb 0x55d1e210c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58e5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2206ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58e5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2206ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40966/0 (socket says 172.21.15.27:40966) 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58e5400640 1 -- 172.21.15.27:0/1145648205 learned_addr learned my addr 172.21.15.27:0/1145648205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58ede00640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2206760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.948+0000 7f58ee800640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2212670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58e5400640 1 -- 172.21.15.27:0/1145648205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 msgr2=0x55d1e2212670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58e5400640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2212670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58e5400640 1 -- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 msgr2=0x55d1e2206760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58e5400640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2206760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58e5400640 1 -- 172.21.15.27:0/1145648205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e1ebf650 con 0x55d1e1e33390 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58ee800640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2212670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58e5400640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2206ca0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f58d8010750 tx=0x7f58d8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c58204349ff509 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58d3e00640 1 -- 172.21.15.27:0/1145648205 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8025070 con 0x55d1e1e33390 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1e20feb90 con 0x55d1e1e33390 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.949+0000 7f58d3e00640 1 -- 172.21.15.27:0/1145648205 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f58d800f9e0 con 0x55d1e1e33390 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58d3e00640 1 -- 172.21.15.27:0/1145648205 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58d8031050 con 0x55d1e1e33390 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 10 client.19095 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1e2119950 con 0x55d1e1e33390 2024-09-21T07:25:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58d3e00640 1 -- 172.21.15.27:0/1145648205 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d1e20feb90 con 0x55d1e1e33390 2024-09-21T07:25:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58d3e00640 1 -- 172.21.15.27:0/1145648205 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f58d80293b0 con 0x55d1e1e33390 2024-09-21T07:25:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 10 client.19095 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 10 client.19095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 20 client.19095 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1e129f0 0x55d1e22187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 4 client.19095 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.950+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e2173f00 con 0x55d1e1e129f0 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.951+0000 7f58ede00640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1e129f0 0x55d1e22187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.951+0000 7f58ede00640 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1e129f0 0x55d1e22187c0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56c3485b02a35f6 server_cookie=56ead2c9c9bfac25 in_seq=0 out_seq=0 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.951+0000 7f58d3e00640 10 client.19095 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.952+0000 7f58d3e00640 1 -- 172.21.15.27:0/1145648205 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1e2173f00 con 0x55d1e1e129f0 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.952+0000 7f58d3e00640 10 client.19095 handle_command_reply: tid=0 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.952+0000 7f58fbc3b740 1 client.19095 shutdown 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 msgr2=0x55d1e2206ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2206ca0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f58d8010750 tx=0x7f58d8005c90 comp rx=0 tx=0).stop 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 shutdown_connections 2024-09-21T07:25:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1e129f0 0x55d1e22187c0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2212670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2206ca0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1145648205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2206760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58d3e00640 0 client.19095 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 >> 172.21.15.27:0/1145648205 conn(0x55d1e1e48460 msgr2=0x55d1e21fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 shutdown_connections 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1145648205 wait complete. 2024-09-21T07:25:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.953+0000 7f58fbc3b740 20 client.19095 trim_cache size 0 max 16384 2024-09-21T07:25:15.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.967+0000 7f58fbc3b740 1 Processor -- start 2024-09-21T07:25:15.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.967+0000 7f58fbc3b740 1 -- start start 2024-09-21T07:25:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2212570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2204de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2205320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01b9d70 con 0x55d1e1e33390 2024-09-21T07:25:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01b7640 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01b54b0 con 0x55d1e1e54070 2024-09-21T07:25:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58e5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2204de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58e5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2204de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40978/0 (socket says 172.21.15.27:40978) 2024-09-21T07:25:15.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58e5400640 1 -- 172.21.15.27:0/1036616055 learned_addr learned my addr 172.21.15.27:0/1036616055 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:15.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58ede00640 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2212570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58ee800640 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2205320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58ede00640 1 -- 172.21.15.27:0/1036616055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 msgr2=0x55d1e2205320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58ede00640 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2205320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58ede00640 1 -- 172.21.15.27:0/1036616055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 msgr2=0x55d1e2204de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.968+0000 7f58ede00640 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2204de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.969+0000 7f58ede00640 1 -- 172.21.15.27:0/1036616055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e21fce90 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.969+0000 7f58e5400640 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2204de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:15.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.969+0000 7f58ede00640 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2212570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006010 tx=0x7f58e0005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7a9bb14f20db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.969+0000 7f58dea00640 1 -- 172.21.15.27:0/1036616055 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0035040 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.969+0000 7f58dea00640 1 -- 172.21.15.27:0/1036616055 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e0005370 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.969+0000 7f58dea00640 1 -- 172.21.15.27:0/1036616055 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0040040 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1036616055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 msgr2=0x55d1e2212570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2212570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006010 tx=0x7f58e0005260 comp rx=0 tx=0).stop 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1036616055 shutdown_connections 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e54070 0x55d1e2205320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e2204de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1036616055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e2b1b0 0x55d1e2212570 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1036616055 >> 172.21.15.27:0/1036616055 conn(0x55d1e1e48460 msgr2=0x55d1e2201710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1036616055 shutdown_connections 2024-09-21T07:25:15.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.970+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1036616055 wait complete. 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 -- start start 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 Processor -- start 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e2b1b0 0x55d1e220ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e220b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e54070 0x55d1e2282180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01b5b00 con 0x55d1e1e33390 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01b6f70 con 0x55d1e1e54070 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.971+0000 7f58fbc3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01b61d0 con 0x55d1e1e2b1b0 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58fbc3b740 10 client.0 _ll_register_callbacks cb 0x7f58d80103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58e5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e220b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58ede00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e2b1b0 0x55d1e220ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58ede00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e2b1b0 0x55d1e220ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:49368/0 (socket says 172.21.15.27:49368) 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58ede00640 1 -- 172.21.15.27:0/1545347789 learned_addr learned my addr 172.21.15.27:0/1545347789 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58ee800640 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e54070 0x55d1e2282180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58e5400640 1 -- 172.21.15.27:0/1545347789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e2b1b0 msgr2=0x55d1e220ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58e5400640 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e2b1b0 0x55d1e220ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58e5400640 1 -- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e54070 msgr2=0x55d1e2282180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58e5400640 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e54070 0x55d1e2282180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58e5400640 1 -- 172.21.15.27:0/1545347789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e22827f0 con 0x55d1e1e33390 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.972+0000 7f58ede00640 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e2b1b0 0x55d1e220ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58e5400640 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e220b410 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x55d1e21f9170 tx=0x7f58d800ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b9d8eb23e2a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58d3e00640 1 -- 172.21.15.27:0/1545347789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8025040 con 0x55d1e1e33390 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58d3e00640 1 -- 172.21.15.27:0/1545347789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f58d800f7e0 con 0x55d1e1e33390 2024-09-21T07:25:15.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e0005370 con 0x55d1e1e33390 2024-09-21T07:25:15.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58d3e00640 1 -- 172.21.15.27:0/1545347789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58d8018070 con 0x55d1e1e33390 2024-09-21T07:25:15.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58fbc3b740 10 client.19101 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:15.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.973+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1e2283a70 con 0x55d1e1e33390 2024-09-21T07:25:15.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58d3e00640 1 -- 172.21.15.27:0/1545347789 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e0005370 con 0x55d1e1e33390 2024-09-21T07:25:15.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58d3e00640 1 -- 172.21.15.27:0/1545347789 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f58d802a560 con 0x55d1e1e33390 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58fbc3b740 10 client.19101 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58fbc3b740 10 client.19101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58fbc3b740 20 client.19101 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1de7da0 0x55d1e2286200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58fbc3b740 4 client.19101 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1e216f9b0 con 0x55d1e1de7da0 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.974+0000 7f58ede00640 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1de7da0 0x55d1e2286200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58ede00640 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1de7da0 0x55d1e2286200 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4058a2f7f7e6e server_cookie=242e961ccc48a226 in_seq=0 out_seq=0 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58d3e00640 10 client.19101 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58d3e00640 1 -- 172.21.15.27:0/1545347789 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1e216f9b0 con 0x55d1e1de7da0 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58d3e00640 10 client.19101 handle_command_reply: tid=0 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 client.19101 shutdown 2024-09-21T07:25:15.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 msgr2=0x55d1e220b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e220b410 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x55d1e21f9170 tx=0x7f58d800ba80 comp rx=0 tx=0).stop 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 shutdown_connections 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d1e1de7da0 0x55d1e2286200 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1e1e54070 0x55d1e2282180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58d3e00640 0 client.19101 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d1e1e33390 0x55d1e220b410 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 --2- 172.21.15.27:0/1545347789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d1e1e2b1b0 0x55d1e220ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.975+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 >> 172.21.15.27:0/1545347789 conn(0x55d1e1e48460 msgr2=0x55d1e2201710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 shutdown_connections 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58fbc3b740 1 -- 172.21.15.27:0/1545347789 wait complete. 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58fbc3b740 20 client.19101 trim_cache size 0 max 16384 2024-09-21T07:25:15.980 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:25:15.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58d40b6700 msgr2=0x7f58d40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58f5000640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58d40b6700 0x7f58d40b8c10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f58d8005d00 tx=0x7f58d800b390 comp rx=0 tx=0).stop 2024-09-21T07:25:15.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0114740 msgr2=0x7f58f01a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:15.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58f5000640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0114740 0x7f58f01a6320 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f58e800eb10 tx=0x7f58e800c2d0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 shutdown_connections 2024-09-21T07:25:15.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58f5000640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58d40b6700 0x7f58d40b8c10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.976+0000 7f58f5000640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0114740 0x7f58f01a6320 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.977+0000 7f58f5000640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58f0078290 0x7f58f01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.977+0000 7f58f5000640 1 --2- 172.21.15.27:0/3032004716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58f0077940 0x7f58f01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:15.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.977+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 >> 172.21.15.27:0/3032004716 conn(0x7f58f0101810 msgr2=0x7f58f0116800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:15.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.977+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 shutdown_connections 2024-09-21T07:25:15.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:15.977+0000 7f58f5000640 1 -- 172.21.15.27:0/3032004716 wait complete. 2024-09-21T07:25:17.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:17 smithi153 ceph-mon[34747]: pgmap v759: 129 pgs: 129 active+clean; 2.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:25:17.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:17 smithi119 ceph-mon[34519]: pgmap v759: 129 pgs: 129 active+clean; 2.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:25:17.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:17 smithi027 ceph-mon[30779]: pgmap v759: 129 pgs: 129 active+clean; 2.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:25:17.913 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:25:17.913 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-09-21T07:25:17.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:25:17.926 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:25:17.926 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:25:18.349 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:18.377 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:19.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:19 smithi027 ceph-mon[30779]: pgmap v760: 129 pgs: 129 active+clean; 2.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.306+0000 7f4fcdc00640 1 -- 172.21.15.27:0/313144359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8076100 msgr2=0x7f4fc810f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.306+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/313144359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8076100 0x7f4fc810f9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009900 tx=0x7f4fb402a410 comp rx=0 tx=0).stop 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.307+0000 7f4fcdc00640 1 -- 172.21.15.27:0/313144359 shutdown_connections 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.307+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/313144359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc810ff10 0x7f4fc8112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.307+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/313144359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8076100 0x7f4fc810f9d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.307+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/313144359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 0x7f4fc8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.307+0000 7f4fcdc00640 1 -- 172.21.15.27:0/313144359 >> 172.21.15.27:0/313144359 conn(0x7f4fc80ff440 msgr2=0x7f4fc8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.307+0000 7f4fcdc00640 1 -- 172.21.15.27:0/313144359 shutdown_connections 2024-09-21T07:25:19.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.307+0000 7f4fcdc00640 1 -- 172.21.15.27:0/313144359 wait complete. 2024-09-21T07:25:19.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.308+0000 7f4fcdc00640 1 Processor -- start 2024-09-21T07:25:19.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.309+0000 7f4fcdc00640 1 -- start start 2024-09-21T07:25:19.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.309+0000 7f4fcdc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 0x7f4fc819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.309+0000 7f4fcdc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc8076100 0x7f4fc819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.309+0000 7f4fcdc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc810ff10 0x7f4fc81a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.309+0000 7f4fcdc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8115b10 con 0x7f4fc8076100 2024-09-21T07:25:19.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.309+0000 7f4fcdc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8115990 con 0x7f4fc810ff10 2024-09-21T07:25:19.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.309+0000 7f4fcdc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8115c90 con 0x7f4fc80757b0 2024-09-21T07:25:19.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 0x7f4fc819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc8076100 0x7f4fc819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 0x7f4fc819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35750/0 (socket says 172.21.15.27:35750) 2024-09-21T07:25:19.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 -- 172.21.15.27:0/1117006815 learned_addr learned my addr 172.21.15.27:0/1117006815 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:19.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fcd200640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc810ff10 0x7f4fc81a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 -- 172.21.15.27:0/1117006815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc810ff10 msgr2=0x7f4fc81a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc810ff10 0x7f4fc81a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 -- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc8076100 msgr2=0x7f4fc819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc8076100 0x7f4fc819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.310+0000 7f4fc7e00640 1 -- 172.21.15.27:0/1117006815 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc81a2410 con 0x7f4fc80757b0 2024-09-21T07:25:19.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.311+0000 7f4fc7e00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 0x7f4fc819c4a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.311+0000 7f4fc4c00640 1 -- 172.21.15.27:0/1117006815 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800cd80 con 0x7f4fc80757b0 2024-09-21T07:25:19.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.311+0000 7f4fc4c00640 1 -- 172.21.15.27:0/1117006815 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb80054d0 con 0x7f4fc80757b0 2024-09-21T07:25:19.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.311+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc81a2700 con 0x7f4fc80757b0 2024-09-21T07:25:19.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.311+0000 7f4fc4c00640 1 -- 172.21.15.27:0/1117006815 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8010430 con 0x7f4fc80757b0 2024-09-21T07:25:19.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.312+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc81a8e60 con 0x7f4fc80757b0 2024-09-21T07:25:19.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.315+0000 7f4fd5e84740 1 Processor -- start 2024-09-21T07:25:19.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.315+0000 7f4fd5e84740 1 -- start start 2024-09-21T07:25:19.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.315+0000 7f4fc4c00640 1 -- 172.21.15.27:0/1117006815 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4fb8005020 con 0x7f4fc80757b0 2024-09-21T07:25:19.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.315+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2c02350 0x55d9b2fa9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.315+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2bd9490 0x55d9b2faa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.315+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2be1670 0x55d9b2faa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2f1fe90 con 0x55d9b2bd9490 2024-09-21T07:25:19.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2f0f2c0 con 0x55d9b2c02350 2024-09-21T07:25:19.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2f130d0 con 0x55d9b2be1670 2024-09-21T07:25:19.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2c02350 0x55d9b2fa9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2c02350 0x55d9b2fa9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52712/0 (socket says 172.21.15.27:52712) 2024-09-21T07:25:19.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 -- 172.21.15.27:0/739867826 learned_addr learned my addr 172.21.15.27:0/739867826 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:19.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fc7400640 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2be1670 0x55d9b2faa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fc7e00640 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2bd9490 0x55d9b2faa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 -- 172.21.15.27:0/739867826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2be1670 msgr2=0x55d9b2faa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2be1670 0x55d9b2faa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 -- 172.21.15.27:0/739867826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2bd9490 msgr2=0x55d9b2faa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2bd9490 0x55d9b2faa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fcd200640 1 -- 172.21.15.27:0/739867826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b2c6d930 con 0x55d9b2c02350 2024-09-21T07:25:19.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.316+0000 7f4fc7e00640 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2bd9490 0x55d9b2faa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:25:19.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.317+0000 7f4fcd200640 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2c02350 0x55d9b2fa9f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009fd0 tx=0x7f4fc00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e9a7cc886a88da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.317+0000 7f4fbd600640 1 -- 172.21.15.27:0/739867826 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0004d20 con 0x55d9b2c02350 2024-09-21T07:25:19.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.317+0000 7f4fbd600640 1 -- 172.21.15.27:0/739867826 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fc0004050 con 0x55d9b2c02350 2024-09-21T07:25:19.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.317+0000 7f4fbd600640 1 -- 172.21.15.27:0/739867826 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0005240 con 0x55d9b2c02350 2024-09-21T07:25:19.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.318+0000 7f4fc4c00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6700 0x7f4fac0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.318+0000 7f4fc4c00640 1 -- 172.21.15.27:0/1117006815 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f4fb80e7f70 con 0x7f4fc80757b0 2024-09-21T07:25:19.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.318+0000 7f4fc7e00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6700 0x7f4fac0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.318+0000 7f4fd5e84740 1 -- 172.21.15.27:0/739867826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2c02350 msgr2=0x55d9b2fa9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.318+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2c02350 0x55d9b2fa9f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009fd0 tx=0x7f4fc00118d0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 -- 172.21.15.27:0/739867826 shutdown_connections 2024-09-21T07:25:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2be1670 0x55d9b2faa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2bd9490 0x55d9b2faa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/739867826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2c02350 0x55d9b2fa9f00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 -- 172.21.15.27:0/739867826 >> 172.21.15.27:0/739867826 conn(0x55d9b2bf6740 msgr2=0x55d9b2fa7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 -- 172.21.15.27:0/739867826 shutdown_connections 2024-09-21T07:25:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fc7e00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6700 0x7f4fac0b8c10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55d9b2fa8b80 tx=0x7f4fb8004a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 -- 172.21.15.27:0/739867826 wait complete. 2024-09-21T07:25:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.319+0000 7f4fd5e84740 1 -- start start 2024-09-21T07:25:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 1 Processor -- start 2024-09-21T07:25:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fb5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2d70890 con 0x55d9b2c02350 2024-09-21T07:25:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2eaa160 con 0x55d9b2be1670 2024-09-21T07:25:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2fb8730 con 0x55d9b2bd9490 2024-09-21T07:25:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.320+0000 7f4fd5e84740 10 client.0 _ll_register_callbacks cb 0x55d9b2ec6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fcd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fb5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52720/0 (socket says 172.21.15.27:52720) 2024-09-21T07:25:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 -- 172.21.15.27:0/528325822 learned_addr learned my addr 172.21.15.27:0/528325822 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fcd200640 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 -- 172.21.15.27:0/528325822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 msgr2=0x55d9b2fb5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fb5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 -- 172.21.15.27:0/528325822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 msgr2=0x55d9b2fc1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fc7400640 1 -- 172.21.15.27:0/528325822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b2fc1f60 con 0x55d9b2be1670 2024-09-21T07:25:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.321+0000 7f4fcd200640 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fb5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:25:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.322+0000 7f4fc7400640 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb6000 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa6d771867c0c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.322+0000 7f4fb2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.322+0000 7f4fb2000640 1 -- 172.21.15.27:0/528325822 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4035040 con 0x55d9b2be1670 2024-09-21T07:25:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.322+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc0004050 con 0x55d9b2be1670 2024-09-21T07:25:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.322+0000 7f4fb2000640 1 -- 172.21.15.27:0/528325822 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f4fb40048c0 con 0x55d9b2be1670 2024-09-21T07:25:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fb2000640 1 -- 172.21.15.27:0/528325822 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fb404e050 con 0x55d9b2be1670 2024-09-21T07:25:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 10 client.29132 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9b2fc4800 con 0x55d9b2be1670 2024-09-21T07:25:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fb2000640 1 -- 172.21.15.27:0/528325822 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fc0004050 con 0x55d9b2be1670 2024-09-21T07:25:19.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fb2000640 1 -- 172.21.15.27:0/528325822 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4fb4003c60 con 0x55d9b2be1670 2024-09-21T07:25:19.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 10 client.29132 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:19.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 10 client.29132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:19.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 20 client.29132 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:19.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2bc0cd0 0x55d9b2fc7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 4 client.29132 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.323+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9b2f21de0 con 0x55d9b2bc0cd0 2024-09-21T07:25:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.324+0000 7f4fcd200640 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2bc0cd0 0x55d9b2fc7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.325+0000 7f4fb2000640 10 client.29132 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.325+0000 7f4fcd200640 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2bc0cd0 0x55d9b2fc7ca0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e7d3542c273c3d4 server_cookie=11e39eef26209216 in_seq=0 out_seq=0 2024-09-21T07:25:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.325+0000 7f4fb2000640 1 -- 172.21.15.27:0/528325822 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9b2f21de0 con 0x55d9b2bc0cd0 2024-09-21T07:25:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.325+0000 7f4fb2000640 10 client.29132 handle_command_reply: tid=0 2024-09-21T07:25:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 client.29132 shutdown 2024-09-21T07:25:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 msgr2=0x55d9b2fb6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb6000 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4005270 comp rx=0 tx=0).stop 2024-09-21T07:25:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 -- 172.21.15.27:0/3915792250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c840757b0 msgr2=0x7f8c84075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 --2- 172.21.15.27:0/3915792250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c840757b0 0x7f8c84075bc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009900 tx=0x7f8c7802a410 comp rx=0 tx=0).stop 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 -- 172.21.15.27:0/3915792250 shutdown_connections 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 --2- 172.21.15.27:0/3915792250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c8410ff10 0x7f8c84112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 --2- 172.21.15.27:0/3915792250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c84076100 0x7f8c8410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 --2- 172.21.15.27:0/3915792250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c840757b0 0x7f8c84075bc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 -- 172.21.15.27:0/3915792250 >> 172.21.15.27:0/3915792250 conn(0x7f8c840ff440 msgr2=0x7f8c84101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 -- 172.21.15.27:0/3915792250 shutdown_connections 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.199+0000 7f8c8a400640 1 -- 172.21.15.27:0/3915792250 wait complete. 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.200+0000 7f8c8a400640 1 Processor -- start 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.201+0000 7f8c8a400640 1 -- start start 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.201+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c840757b0 0x7f8c84106a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.201+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c84076100 0x7f8c84106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.201+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c8410ff10 0x7f8c84107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.201+0000 7f8c8a400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84115b10 con 0x7f8c84076100 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.201+0000 7f8c8a400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84115990 con 0x7f8c8410ff10 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.201+0000 7f8c8a400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84115c90 con 0x7f8c840757b0 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c840757b0 0x7f8c84106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c840757b0 0x7f8c84106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35724/0 (socket says 172.21.15.27:35724) 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 -- 172.21.15.27:0/903718780 learned_addr learned my addr 172.21.15.27:0/903718780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c83e00640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c84076100 0x7f8c84106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89a00640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c8410ff10 0x7f8c84107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 -- 172.21.15.27:0/903718780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c8410ff10 msgr2=0x7f8c84107490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c8410ff10 0x7f8c84107490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 -- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c84076100 msgr2=0x7f8c84106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c84076100 0x7f8c84106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.202+0000 7f8c89000640 1 -- 172.21.15.27:0/903718780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c841ac9a0 con 0x7f8c840757b0 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.203+0000 7f8c89000640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c840757b0 0x7f8c84106a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c78005ab0 tx=0x7f8c78003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.203+0000 7f8c81600640 1 -- 172.21.15.27:0/903718780 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7803f070 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.203+0000 7f8c81600640 1 -- 172.21.15.27:0/903718780 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c78031b60 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.203+0000 7f8c81600640 1 -- 172.21.15.27:0/903718780 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78035030 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.203+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c841acc30 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.203+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841ad160 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.205+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c84075bc0 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.209+0000 7f8c81600640 1 -- 172.21.15.27:0/903718780 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8c78042070 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.211+0000 7f8c81600640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8c6c0b6700 0x7f8c6c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.211+0000 7f8c81600640 1 -- 172.21.15.27:0/903718780 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f8c78108140 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.211+0000 7f8c83e00640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8c6c0b6700 0x7f8c6c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.211+0000 7f8c81600640 1 -- 172.21.15.27:0/903718780 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8c780c9f00 con 0x7f8c840757b0 2024-09-21T07:25:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.212+0000 7f8c83e00640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8c6c0b6700 0x7f8c6c0b8c10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8c84101b70 tx=0x7f8c70005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 shutdown_connections 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2bc0cd0 0x55d9b2fc7ca0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb6000 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/528325822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fb5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 >> 172.21.15.27:0/528325822 conn(0x55d9b2bf6740 msgr2=0x55d9b2faaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.326+0000 7f4fb2000640 0 client.29132 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.327+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 shutdown_connections 2024-09-21T07:25:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.327+0000 7f4fd5e84740 1 -- 172.21.15.27:0/528325822 wait complete. 2024-09-21T07:25:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.327+0000 7f4fd5e84740 20 client.29132 trim_cache size 0 max 16384 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 Processor -- start 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 -- start start 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b5760 con 0x55d9b2c02350 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b3030 con 0x55d9b2be1670 2024-09-21T07:25:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.346+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b0ea0 con 0x55d9b2bd9490 2024-09-21T07:25:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52724/0 (socket says 172.21.15.27:52724) 2024-09-21T07:25:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 -- 172.21.15.27:0/2578398693 learned_addr learned my addr 172.21.15.27:0/2578398693 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fcd200640 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fcd200640 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 -- 172.21.15.27:0/2578398693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 msgr2=0x55d9b2fab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 -- 172.21.15.27:0/2578398693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 msgr2=0x55d9b2fc1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.347+0000 7f4fc7400640 1 -- 172.21.15.27:0/2578398693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b2fac1f0 con 0x55d9b2be1670 2024-09-21T07:25:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.348+0000 7f4fc7400640 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb1070 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006010 tx=0x7f4fb4031590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97886e9d1e0cb75c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.348+0000 7f4fbcc00640 1 -- 172.21.15.27:0/2578398693 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4035070 con 0x55d9b2be1670 2024-09-21T07:25:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.348+0000 7f4fbcc00640 1 -- 172.21.15.27:0/2578398693 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb4040070 con 0x55d9b2be1670 2024-09-21T07:25:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.348+0000 7f4fbcc00640 1 -- 172.21.15.27:0/2578398693 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4004a20 con 0x55d9b2be1670 2024-09-21T07:25:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 -- 172.21.15.27:0/2578398693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 msgr2=0x55d9b2fb1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb1070 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006010 tx=0x7f4fb4031590 comp rx=0 tx=0).stop 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 -- 172.21.15.27:0/2578398693 shutdown_connections 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fc1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb1070 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/2578398693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 -- 172.21.15.27:0/2578398693 >> 172.21.15.27:0/2578398693 conn(0x55d9b2bf6740 msgr2=0x55d9b2fb6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 -- 172.21.15.27:0/2578398693 shutdown_connections 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.349+0000 7f4fd5e84740 1 -- 172.21.15.27:0/2578398693 wait complete. 2024-09-21T07:25:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.350+0000 7f4fd5e84740 1 -- start start 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.350+0000 7f4fd5e84740 1 Processor -- start 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fbdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fd5e84740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fb9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b14f0 con 0x55d9b2c02350 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b2960 con 0x55d9b2be1670 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fd5e84740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81b1bc0 con 0x55d9b2bd9490 2024-09-21T07:25:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fd5e84740 10 client.0 _ll_register_callbacks cb 0x7f4fb8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fcd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fbdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52736/0 (socket says 172.21.15.27:52736) 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fcd200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fbdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35792/0 (socket says 172.21.15.27:35792) 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fc7400640 1 -- 172.21.15.27:0/4042166197 learned_addr learned my addr 172.21.15.27:0/4042166197 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.351+0000 7f4fcd200640 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fb9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fc7400640 1 -- 172.21.15.27:0/4042166197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 msgr2=0x55d9b2fbdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fc7400640 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fbdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fc7400640 1 -- 172.21.15.27:0/4042166197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 msgr2=0x55d9b2fb9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fc7400640 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fb9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fc7400640 1 -- 172.21.15.27:0/4042166197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b2fb9f00 con 0x55d9b2be1670 2024-09-21T07:25:19.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fcd200640 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fb9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:19.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fc7400640 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb9450 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005a80 tx=0x7f4fb4032b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bc0ff9188b7029 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:19.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fb2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fb2000640 1 -- 172.21.15.27:0/4042166197 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4058040 con 0x55d9b2be1670 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fb2000640 1 -- 172.21.15.27:0/4042166197 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f4fb4032d00 con 0x55d9b2be1670 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.352+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb4040070 con 0x55d9b2be1670 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.353+0000 7f4fb2000640 1 -- 172.21.15.27:0/4042166197 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fb4035040 con 0x55d9b2be1670 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.353+0000 7f4fd5e84740 10 client.29138 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.353+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9b30323c0 con 0x55d9b2be1670 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.353+0000 7f4fb2000640 1 -- 172.21.15.27:0/4042166197 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb4040070 con 0x55d9b2be1670 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fb2000640 1 -- 172.21.15.27:0/4042166197 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4fb405b070 con 0x55d9b2be1670 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fd5e84740 10 client.29138 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fd5e84740 10 client.29138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fd5e84740 20 client.29138 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2b96080 0x55d9b3034ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fd5e84740 4 client.29138 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:25:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d9b2f1d890 con 0x55d9b2b96080 2024-09-21T07:25:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.354+0000 7f4fcd200640 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2b96080 0x55d9b3034ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.355+0000 7f4fcd200640 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2b96080 0x55d9b3034ad0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe342e162ed18d2 server_cookie=1f934955ff1a96e6 in_seq=0 out_seq=0 2024-09-21T07:25:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.355+0000 7f4fb2000640 10 client.29138 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fb2000640 1 -- 172.21.15.27:0/4042166197 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d9b2f1d890 con 0x55d9b2b96080 2024-09-21T07:25:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fb2000640 10 client.29138 handle_command_reply: tid=0 2024-09-21T07:25:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 client.29138 shutdown 2024-09-21T07:25:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 msgr2=0x55d9b2fb9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb9450 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005a80 tx=0x7f4fb4032b10 comp rx=0 tx=0).stop 2024-09-21T07:25:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 shutdown_connections 2024-09-21T07:25:19.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d9b2b96080 0x55d9b3034ad0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9b2c02350 0x55d9b2fb9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9b2be1670 0x55d9b2fb9450 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 --2- 172.21.15.27:0/4042166197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9b2bd9490 0x55d9b2fbdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fb2000640 0 client.29138 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.357+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 >> 172.21.15.27:0/4042166197 conn(0x55d9b2bf6740 msgr2=0x55d9b2fab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.358+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 shutdown_connections 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.358+0000 7f4fd5e84740 1 -- 172.21.15.27:0/4042166197 wait complete. 2024-09-21T07:25:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.358+0000 7f4fd5e84740 20 client.29138 trim_cache size 0 max 16384 2024-09-21T07:25:19.362 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:19.362 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:25:19.362 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "842dd810-32aa-4200-bb13-a6fd281291ee", 2024-09-21T07:25:19.363 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:25:19.363 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6700 msgr2=0x7f4fac0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6700 0x7f4fac0b8c10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55d9b2fa8b80 tx=0x7f4fb8004a20 comp rx=0 tx=0).stop 2024-09-21T07:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 msgr2=0x7f4fc819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 0x7f4fc819c4a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800edf0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 shutdown_connections 2024-09-21T07:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4fac0b6700 0x7f4fac0b8c10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc810ff10 0x7f4fc81a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc8076100 0x7f4fc819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 --2- 172.21.15.27:0/1117006815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fc80757b0 0x7f4fc819c4a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 >> 172.21.15.27:0/1117006815 conn(0x7f4fc80ff440 msgr2=0x7f4fc8107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 shutdown_connections 2024-09-21T07:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.359+0000 7f4fcdc00640 1 -- 172.21.15.27:0/1117006815 wait complete. 2024-09-21T07:25:19.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:19 smithi153 ceph-mon[34747]: pgmap v760: 129 pgs: 129 active+clean; 2.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:25:19.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.467+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c84107d10 con 0x7f8c840757b0 2024-09-21T07:25:19.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.468+0000 7f8c81600640 1 -- 172.21.15.27:0/903718780 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8c780d26c0 con 0x7f8c840757b0 2024-09-21T07:25:19.470 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:25:19.471 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:25:19.471 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:25:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8c6c0b6700 msgr2=0x7f8c6c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8c6c0b6700 0x7f8c6c0b8c10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8c84101b70 tx=0x7f8c70005f90 comp rx=0 tx=0).stop 2024-09-21T07:25:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c840757b0 msgr2=0x7f8c84106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c840757b0 0x7f8c84106a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c78005ab0 tx=0x7f8c78003c10 comp rx=0 tx=0).stop 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 shutdown_connections 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8c6c0b6700 0x7f8c6c0b8c10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c8410ff10 0x7f8c84107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c84076100 0x7f8c84106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 --2- 172.21.15.27:0/903718780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c840757b0 0x7f8c84106a10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.471+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 >> 172.21.15.27:0/903718780 conn(0x7f8c840ff440 msgr2=0x7f8c84109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.472+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 shutdown_connections 2024-09-21T07:25:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:19.472+0000 7f8c8a400640 1 -- 172.21.15.27:0/903718780 wait complete. 2024-09-21T07:25:19.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:19 smithi119 ceph-mon[34519]: pgmap v760: 129 pgs: 129 active+clean; 2.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-09-21T07:25:20.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:20 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/903718780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:20.504 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:20 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/903718780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:20.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:20 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/903718780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:20.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-09-21T07:25:20.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-09-21T07:25:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 -- 172.21.15.27:0/1922411338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 msgr2=0x7f1cc4110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 --2- 172.21.15.27:0/1922411338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 0x7f1cc4110590 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009ca0 tx=0x7f1cbc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 -- 172.21.15.27:0/1922411338 shutdown_connections 2024-09-21T07:25:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 --2- 172.21.15.27:0/1922411338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cc4109410 0x7f1cc4109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 --2- 172.21.15.27:0/1922411338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 0x7f1cc4110590 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 --2- 172.21.15.27:0/1922411338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 0x7f1cc4103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 -- 172.21.15.27:0/1922411338 >> 172.21.15.27:0/1922411338 conn(0x7f1cc40ff520 msgr2=0x7f1cc4101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 -- 172.21.15.27:0/1922411338 shutdown_connections 2024-09-21T07:25:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.923+0000 7f1ccba00640 1 -- 172.21.15.27:0/1922411338 wait complete. 2024-09-21T07:25:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 Processor -- start 2024-09-21T07:25:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 -- start start 2024-09-21T07:25:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 0x7f1cc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cc4109410 0x7f1cc4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 0x7f1cc4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4113a20 con 0x7f1cc4109d60 2024-09-21T07:25:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41138a0 con 0x7f1cc4103ae0 2024-09-21T07:25:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.925+0000 7f1ccba00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4113ba0 con 0x7f1cc4109410 2024-09-21T07:25:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 0x7f1cc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 0x7f1cc4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52758/0 (socket says 172.21.15.27:52758) 2024-09-21T07:25:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 -- 172.21.15.27:0/3544545065 learned_addr learned my addr 172.21.15.27:0/3544545065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cc4109410 0x7f1cc4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 0x7f1cc4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 -- 172.21.15.27:0/3544545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cc4109410 msgr2=0x7f1cc4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cc4109410 0x7f1cc4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 -- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 msgr2=0x7f1cc4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 0x7f1cc4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9000640 1 -- 172.21.15.27:0/3544545065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc41a86d0 con 0x7f1cc4103ae0 2024-09-21T07:25:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.926+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 0x7f1cc4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.927+0000 7f1cc9000640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 0x7f1cc4075710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800e9f0 tx=0x7f1cb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.927+0000 7f1cc1600640 1 -- 172.21.15.27:0/3544545065 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800cd80 con 0x7f1cc4103ae0 2024-09-21T07:25:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.927+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc41a89c0 con 0x7f1cc4103ae0 2024-09-21T07:25:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.927+0000 7f1cc1600640 1 -- 172.21.15.27:0/3544545065 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb80108e0 con 0x7f1cc4103ae0 2024-09-21T07:25:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.927+0000 7f1cc1600640 1 -- 172.21.15.27:0/3544545065 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8018980 con 0x7f1cc4103ae0 2024-09-21T07:25:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.927+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a8f60 con 0x7f1cc4103ae0 2024-09-21T07:25:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.930+0000 7f1cd2a94740 1 Processor -- start 2024-09-21T07:25:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cd2a94740 1 -- start start 2024-09-21T07:25:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2be1b0 0x55727e662c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 0x55727e663170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e6636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55727e5d86b0 con 0x55727e2be1b0 2024-09-21T07:25:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55727e5c7ae0 con 0x55727e29d4d0 2024-09-21T07:25:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55727e5cb8f0 con 0x55727e2952f0 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cc9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2be1b0 0x55727e662c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cc9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2be1b0 0x55727e662c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58256/0 (socket says 172.21.15.27:58256) 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.931+0000 7f1cc9a00640 1 -- 172.21.15.27:0/1183272773 learned_addr learned my addr 172.21.15.27:0/1183272773 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e6636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc1600640 1 -- 172.21.15.27:0/3544545065 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1cb8018b20 con 0x7f1cc4103ae0 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc9a00640 1 -- 172.21.15.27:0/1183272773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 msgr2=0x55727e663170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 0x55727e663170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc9a00640 1 -- 172.21.15.27:0/1183272773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 msgr2=0x55727e6636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e6636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc9a00640 1 -- 172.21.15.27:0/1183272773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727e594780 con 0x55727e2be1b0 2024-09-21T07:25:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.932+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e6636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.933+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2be1b0 0x55727e662c30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009470 tx=0x7f1cbc0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12e9356ccdf8335 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.933+0000 7f1cb6000640 1 -- 172.21.15.27:0/1183272773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc031cf0 con 0x55727e2be1b0 2024-09-21T07:25:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.933+0000 7f1cb6000640 1 -- 172.21.15.27:0/1183272773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc036040 con 0x55727e2be1b0 2024-09-21T07:25:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.933+0000 7f1cb6000640 1 -- 172.21.15.27:0/1183272773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0042a0 con 0x55727e2be1b0 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.934+0000 7f1cc1600640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1cac0b66b0 0x7f1cac0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.934+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1183272773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2be1b0 msgr2=0x55727e662c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.934+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2be1b0 0x55727e662c30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009470 tx=0x7f1cbc0094a0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.934+0000 7f1cc1600640 1 -- 172.21.15.27:0/3544545065 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f1cb80e7ec0 con 0x7f1cc4103ae0 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cc9000640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1cac0b66b0 0x7f1cac0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1183272773 shutdown_connections 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 0x55727e663170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e6636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1183272773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2be1b0 0x55727e662c30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1183272773 >> 172.21.15.27:0/1183272773 conn(0x55727e2b2300 msgr2=0x55727e660b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1183272773 shutdown_connections 2024-09-21T07:25:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1183272773 wait complete. 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cd2a94740 1 -- start start 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.935+0000 7f1cc9000640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1cac0b66b0 0x7f1cac0b8bc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55727e6618b0 tx=0x7f1cb80ad800 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 1 Processor -- start 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 0x55727e66e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e29d4d0 0x55727e66e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e67a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55727e4ffab0 con 0x55727e29d4d0 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55727e56e570 con 0x55727e2be1b0 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55727e671350 con 0x55727e2952f0 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.936+0000 7f1cd2a94740 10 client.0 _ll_register_callbacks cb 0x55727e57d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e67a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e67a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52784/0 (socket says 172.21.15.27:52784) 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 -- 172.21.15.27:0/663144891 learned_addr learned my addr 172.21.15.27:0/663144891 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 0x55727e66e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e29d4d0 0x55727e66e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 -- 172.21.15.27:0/663144891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 msgr2=0x55727e66e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 0x55727e66e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 -- 172.21.15.27:0/663144891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e29d4d0 msgr2=0x55727e66e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e29d4d0 0x55727e66e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.937+0000 7f1cc9a00640 1 -- 172.21.15.27:0/663144891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727e329790 con 0x55727e2be1b0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e67a640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc02f470 tx=0x7f1cbc03e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7146e067835390db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1caaa00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1caaa00640 1 -- 172.21.15.27:0/663144891 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc059060 con 0x55727e2be1b0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc036040 con 0x55727e2be1b0 2024-09-21T07:25:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1caaa00640 1 -- 172.21.15.27:0/663144891 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f1cbc031cf0 con 0x55727e2be1b0 2024-09-21T07:25:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1caaa00640 1 -- 172.21.15.27:0/663144891 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc004b40 con 0x55727e2be1b0 2024-09-21T07:25:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1cd2a94740 10 client.29150 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.938+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55727e3a2620 con 0x55727e2be1b0 2024-09-21T07:25:20.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1caaa00640 1 -- 172.21.15.27:0/663144891 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc036040 con 0x55727e2be1b0 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1caaa00640 1 -- 172.21.15.27:0/663144891 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1cbc031720 con 0x55727e2be1b0 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1cd2a94740 10 client.29150 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1cd2a94740 10 client.29150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1cd2a94740 20 client.29150 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e27cd80 0x55727e680190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1cd2a94740 4 client.29150 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.939+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55727e5da600 con 0x55727e27cd80 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.940+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e27cd80 0x55727e680190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.940+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e27cd80 0x55727e680190 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4acf33a92755bd2f server_cookie=6dd08e867739bf44 in_seq=0 out_seq=0 2024-09-21T07:25:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.940+0000 7f1caaa00640 10 client.29150 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1caaa00640 1 -- 172.21.15.27:0/663144891 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55727e5da600 con 0x55727e27cd80 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1caaa00640 10 client.29150 handle_command_reply: tid=0 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 client.29150 shutdown 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 msgr2=0x55727e67a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e67a640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc02f470 tx=0x7f1cbc03e040 comp rx=0 tx=0).stop 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 shutdown_connections 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e27cd80 0x55727e680190 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e67a640 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e29d4d0 0x55727e66e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/663144891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2952f0 0x55727e66e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1caaa00640 0 client.29150 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.941+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 >> 172.21.15.27:0/663144891 conn(0x55727e2b2300 msgr2=0x55727e663c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.942+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 shutdown_connections 2024-09-21T07:25:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.942+0000 7f1cd2a94740 1 -- 172.21.15.27:0/663144891 wait complete. 2024-09-21T07:25:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.942+0000 7f1cd2a94740 20 client.29150 trim_cache size 0 max 16384 2024-09-21T07:25:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.957+0000 7f1cd2a94740 1 Processor -- start 2024-09-21T07:25:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cd2a94740 1 -- start start 2024-09-21T07:25:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e670160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e67a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2be1b0 0x55727e66b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b1680 con 0x55727e2952f0 2024-09-21T07:25:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41aef50 con 0x55727e29d4d0 2024-09-21T07:25:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41acdc0 con 0x55727e2be1b0 2024-09-21T07:25:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cc9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e670160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cc9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e670160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58272/0 (socket says 172.21.15.27:58272) 2024-09-21T07:25:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cc9a00640 1 -- 172.21.15.27:0/1105598779 learned_addr learned my addr 172.21.15.27:0/1105598779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.958+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e67a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2be1b0 0x55727e66b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc3e00640 1 -- 172.21.15.27:0/1105598779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2be1b0 msgr2=0x55727e66b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2be1b0 0x55727e66b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc3e00640 1 -- 172.21.15.27:0/1105598779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 msgr2=0x55727e670160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e670160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc3e00640 1 -- 172.21.15.27:0/1105598779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727e665a40 con 0x55727e29d4d0 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e670160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.959+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e67a5b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55727e5c7b90 tx=0x7f1cb0009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e435076b1a38eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.960+0000 7f1cb5600640 1 -- 172.21.15.27:0/1105598779 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0030020 con 0x55727e29d4d0 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.960+0000 7f1cb5600640 1 -- 172.21.15.27:0/1105598779 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00054d0 con 0x55727e29d4d0 2024-09-21T07:25:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.960+0000 7f1cb5600640 1 -- 172.21.15.27:0/1105598779 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0027410 con 0x55727e29d4d0 2024-09-21T07:25:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1105598779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 msgr2=0x55727e67a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e67a5b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55727e5c7b90 tx=0x7f1cb0009ef0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1105598779 shutdown_connections 2024-09-21T07:25:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e2be1b0 0x55727e66b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e29d4d0 0x55727e67a5b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/1105598779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e670160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1105598779 >> 172.21.15.27:0/1105598779 conn(0x55727e2b2300 msgr2=0x55727e6642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1105598779 shutdown_connections 2024-09-21T07:25:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.961+0000 7f1cd2a94740 1 -- 172.21.15.27:0/1105598779 wait complete. 2024-09-21T07:25:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 -- start start 2024-09-21T07:25:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 Processor -- start 2024-09-21T07:25:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e682c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e29d4d0 0x55727e683250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e6e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ad410 con 0x55727e2952f0 2024-09-21T07:25:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ae880 con 0x55727e2be1b0 2024-09-21T07:25:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41adae0 con 0x55727e29d4d0 2024-09-21T07:25:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.962+0000 7f1cd2a94740 10 client.0 _ll_register_callbacks cb 0x7f1cb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e29d4d0 0x55727e683250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e682c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e29d4d0 0x55727e683250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35842/0 (socket says 172.21.15.27:35842) 2024-09-21T07:25:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 -- 172.21.15.27:0/2819800214 learned_addr learned my addr 172.21.15.27:0/2819800214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e6e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 -- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 msgr2=0x55727e6e18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e6e18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 -- 172.21.15.27:0/2819800214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 msgr2=0x55727e682c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e682c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc3e00640 1 -- 172.21.15.27:0/2819800214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727e6e1f70 con 0x55727e29d4d0 2024-09-21T07:25:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.963+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e682c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1cc3e00640 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e29d4d0 0x55727e683250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0005df0 tx=0x7f1cb0005680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8fee9052b3307 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1caaa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1caaa00640 1 -- 172.21.15.27:0/2819800214 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0030030 con 0x55727e29d4d0 2024-09-21T07:25:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb00054d0 con 0x55727e29d4d0 2024-09-21T07:25:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1caaa00640 1 -- 172.21.15.27:0/2819800214 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f1cb0004700 con 0x55727e29d4d0 2024-09-21T07:25:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1caaa00640 1 -- 172.21.15.27:0/2819800214 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cb0043050 con 0x55727e29d4d0 2024-09-21T07:25:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1cd2a94740 10 client.29125 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55727e6ea450 con 0x55727e29d4d0 2024-09-21T07:25:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.964+0000 7f1caaa00640 1 -- 172.21.15.27:0/2819800214 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00054d0 con 0x55727e29d4d0 2024-09-21T07:25:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1caaa00640 1 -- 172.21.15.27:0/2819800214 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb002c7f0 con 0x55727e29d4d0 2024-09-21T07:25:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1cd2a94740 10 client.29125 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1cd2a94740 10 client.29125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1cd2a94740 20 client.29125 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e2521c0 0x55727e6ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1cd2a94740 4 client.29125 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:25:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55727e5d60b0 con 0x55727e2521c0 2024-09-21T07:25:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.965+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e2521c0 0x55727e6ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.966+0000 7f1cc9a00640 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e2521c0 0x55727e6ecbb0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e1500d9f1a71af server_cookie=cc75dd77ea69cee7 in_seq=0 out_seq=0 2024-09-21T07:25:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.966+0000 7f1caaa00640 10 client.29125 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1caaa00640 1 -- 172.21.15.27:0/2819800214 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55727e5d60b0 con 0x55727e2521c0 2024-09-21T07:25:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1caaa00640 10 client.29125 handle_command_reply: tid=0 2024-09-21T07:25:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 client.29125 shutdown 2024-09-21T07:25:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e29d4d0 msgr2=0x55727e683250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e29d4d0 0x55727e683250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0005df0 tx=0x7f1cb0005680 comp rx=0 tx=0).stop 2024-09-21T07:25:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 shutdown_connections 2024-09-21T07:25:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55727e2521c0 0x55727e6ecbb0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55727e29d4d0 0x55727e683250 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55727e2be1b0 0x55727e6e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 --2- 172.21.15.27:0/2819800214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55727e2952f0 0x55727e682c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1caaa00640 0 client.29125 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.979+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 >> 172.21.15.27:0/2819800214 conn(0x55727e2b2300 msgr2=0x55727e66ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.980+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 shutdown_connections 2024-09-21T07:25:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.980+0000 7f1cd2a94740 1 -- 172.21.15.27:0/2819800214 wait complete. 2024-09-21T07:25:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.980+0000 7f1cd2a94740 20 client.29125 trim_cache size 0 max 16384 2024-09-21T07:25:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:25:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:25:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:25:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:25:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1cac0b66b0 msgr2=0x7f1cac0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1cac0b66b0 0x7f1cac0b8bc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55727e6618b0 tx=0x7f1cb80ad800 comp rx=0 tx=0).stop 2024-09-21T07:25:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 msgr2=0x7f1cc4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 0x7f1cc4075710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800e9f0 tx=0x7f1cb800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 shutdown_connections 2024-09-21T07:25:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cc4109410 0x7f1cc4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1cac0b66b0 0x7f1cac0b8bc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc4109d60 0x7f1cc4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 --2- 172.21.15.27:0/3544545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4103ae0 0x7f1cc4075710 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 >> 172.21.15.27:0/3544545065 conn(0x7f1cc40ff520 msgr2=0x7f1cc4101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 shutdown_connections 2024-09-21T07:25:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:20.981+0000 7f1ccba00640 1 -- 172.21.15.27:0/3544545065 wait complete. 2024-09-21T07:25:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:21 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:25:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:21 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:21 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:25:21.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:21 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:21.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:21 smithi153 ceph-mon[34747]: pgmap v761: 129 pgs: 129 active+clean; 2.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:25:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:21 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:25:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:21 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:21.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:21 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:25:21.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:21 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:21.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:21 smithi119 ceph-mon[34519]: pgmap v761: 129 pgs: 129 active+clean; 2.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:25:21.535 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:21 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:25:21.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:21 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:21.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:21 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:25:21.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:21 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:25:21.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:21 smithi027 ceph-mon[30779]: pgmap v761: 129 pgs: 129 active+clean; 2.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:25:21.714 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 -- 172.21.15.27:0/3266220355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 msgr2=0x7f579c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 --2- 172.21.15.27:0/3266220355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 0x7f579c110640 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5794009ca0 tx=0x7f579402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 -- 172.21.15.27:0/3266220355 shutdown_connections 2024-09-21T07:25:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 --2- 172.21.15.27:0/3266220355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f579c1094c0 0x7f579c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 --2- 172.21.15.27:0/3266220355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 0x7f579c110640 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 --2- 172.21.15.27:0/3266220355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c103b80 0x7f579c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 -- 172.21.15.27:0/3266220355 >> 172.21.15.27:0/3266220355 conn(0x7f579c0ff5d0 msgr2=0x7f579c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 -- 172.21.15.27:0/3266220355 shutdown_connections 2024-09-21T07:25:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.750+0000 7f57a3000640 1 -- 172.21.15.27:0/3266220355 wait complete. 2024-09-21T07:25:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.751+0000 7f57a3000640 1 Processor -- start 2024-09-21T07:25:21.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.751+0000 7f57a3000640 1 -- start start 2024-09-21T07:25:21.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f57a3000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f579c103b80 0x7f579c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f57a3000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c1094c0 0x7f579c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f57a3000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 0x7f579c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f57a3000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f579c113b20 con 0x7f579c109e10 2024-09-21T07:25:21.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f57a3000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1139a0 con 0x7f579c1094c0 2024-09-21T07:25:21.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f57a3000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f579c113ca0 con 0x7f579c103b80 2024-09-21T07:25:21.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f5793e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c1094c0 0x7f579c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f57a1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 0x7f579c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f5793e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c1094c0 0x7f579c19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52820/0 (socket says 172.21.15.27:52820) 2024-09-21T07:25:21.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f5793e00640 1 -- 172.21.15.27:0/3029054793 learned_addr learned my addr 172.21.15.27:0/3029054793 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:21.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.752+0000 7f579be00640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f579c103b80 0x7f579c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5793e00640 1 -- 172.21.15.27:0/3029054793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f579c103b80 msgr2=0x7f579c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5793e00640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f579c103b80 0x7f579c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5793e00640 1 -- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 msgr2=0x7f579c1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5793e00640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 0x7f579c1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5793e00640 1 -- 172.21.15.27:0/3029054793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c1a25c0 con 0x7f579c1094c0 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5793e00640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c1094c0 0x7f579c19cbf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f578400e9f0 tx=0x7f578400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5799600640 1 -- 172.21.15.27:0/3029054793 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578400ccc0 con 0x7f579c1094c0 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c1a28b0 con 0x7f579c1094c0 2024-09-21T07:25:21.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5799600640 1 -- 172.21.15.27:0/3029054793 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57840108e0 con 0x7f579c1094c0 2024-09-21T07:25:21.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f5799600640 1 -- 172.21.15.27:0/3029054793 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784018860 con 0x7f579c1094c0 2024-09-21T07:25:21.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.753+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c1a9010 con 0x7f579c1094c0 2024-09-21T07:25:21.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.757+0000 7f57a9cc8740 1 Processor -- start 2024-09-21T07:25:21.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.757+0000 7f5799600640 1 -- 172.21.15.27:0/3029054793 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5784018a00 con 0x7f579c1094c0 2024-09-21T07:25:21.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.757+0000 7f57a9cc8740 1 -- start start 2024-09-21T07:25:21.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c95e840 0x560d4cd06350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c935980 0x560d4cd06890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd06dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560d4cc7bca0 con 0x560d4c935980 2024-09-21T07:25:21.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560d4cc6b0d0 con 0x560d4c93db60 2024-09-21T07:25:21.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d4cc6eee0 con 0x560d4c95e840 2024-09-21T07:25:21.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f5793e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd06dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f579be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c935980 0x560d4cd06890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f5793e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd06dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52834/0 (socket says 172.21.15.27:52834) 2024-09-21T07:25:21.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.758+0000 7f5793e00640 1 -- 172.21.15.27:0/2914692160 learned_addr learned my addr 172.21.15.27:0/2914692160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:21.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f57a1000640 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c95e840 0x560d4cd06350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f5793e00640 1 -- 172.21.15.27:0/2914692160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c95e840 msgr2=0x560d4cd06350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f5793e00640 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c95e840 0x560d4cd06350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f5793e00640 1 -- 172.21.15.27:0/2914692160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c935980 msgr2=0x560d4cd06890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f5793e00640 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c935980 0x560d4cd06890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f5793e00640 1 -- 172.21.15.27:0/2914692160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4cc1d400 con 0x560d4c93db60 2024-09-21T07:25:21.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f57a1000640 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c95e840 0x560d4cd06350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:25:21.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f579be00640 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c935980 0x560d4cd06890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:21.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.759+0000 7f5793e00640 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd06dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f57840e5060 tx=0x7f57840ebf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d6930333446243 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.760+0000 7f5791600640 1 -- 172.21.15.27:0/2914692160 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57840f9430 con 0x560d4c93db60 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.760+0000 7f5791600640 1 -- 172.21.15.27:0/2914692160 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57840ec8e0 con 0x560d4c93db60 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.760+0000 7f5791600640 1 -- 172.21.15.27:0/2914692160 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57840f85f0 con 0x560d4c93db60 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.760+0000 7f5799600640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57800b6700 0x7f57800b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f5799600640 1 -- 172.21.15.27:0/3029054793 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f57840e7de0 con 0x7f579c1094c0 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2914692160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 msgr2=0x560d4cd06dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd06dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f57840e5060 tx=0x7f57840ebf90 comp rx=0 tx=0).stop 2024-09-21T07:25:21.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a1000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57800b6700 0x7f57800b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2914692160 shutdown_connections 2024-09-21T07:25:21.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd06dd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c935980 0x560d4cd06890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2914692160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c95e840 0x560d4cd06350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2914692160 >> 172.21.15.27:0/2914692160 conn(0x560d4c952990 msgr2=0x560d4cd04240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:21.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2914692160 shutdown_connections 2024-09-21T07:25:21.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.761+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2914692160 wait complete. 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.762+0000 7f57a9cc8740 1 -- start start 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.762+0000 7f57a1000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57800b6700 0x7f57800b8c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5794006010 tx=0x7f5794005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.762+0000 7f57a9cc8740 1 Processor -- start 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c935980 0x560d4cd1a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c93db60 0x560d4cd1a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560d4cacc7a0 con 0x560d4c95e840 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560d4cc10a10 con 0x560d4c935980 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d4cd1d260 con 0x560d4c93db60 2024-09-21T07:25:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a9cc8740 10 client.0 _ll_register_callbacks cb 0x560d4cc20be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:21.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f579be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c935980 0x560d4cd1a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f579be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c935980 0x560d4cd1a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52838/0 (socket says 172.21.15.27:52838) 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f579be00640 1 -- 172.21.15.27:0/2410562008 learned_addr learned my addr 172.21.15.27:0/2410562008 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f57a1000640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.763+0000 7f5793e00640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c93db60 0x560d4cd1a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f579be00640 1 -- 172.21.15.27:0/2410562008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c93db60 msgr2=0x560d4cd1a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f579be00640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c93db60 0x560d4cd1a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f579be00640 1 -- 172.21.15.27:0/2410562008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 msgr2=0x560d4cd265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f579be00640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f579be00640 1 -- 172.21.15.27:0/2410562008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4c9d8b90 con 0x560d4c935980 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f57a1000640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f5793e00640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c93db60 0x560d4cd1a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f579be00640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c935980 0x560d4cd1a1f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f578c012810 tx=0x7f578c012cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26c418a38d87b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:21.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f578a000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:21.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f578a000640 1 -- 172.21.15.27:0/2410562008 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c004020 con 0x560d4c935980 2024-09-21T07:25:21.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f578a000640 1 -- 172.21.15.27:0/2410562008 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f578c0041c0 con 0x560d4c935980 2024-09-21T07:25:21.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.764+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57840ec8e0 con 0x560d4c935980 2024-09-21T07:25:21.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f578a000640 1 -- 172.21.15.27:0/2410562008 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f578c025050 con 0x560d4c935980 2024-09-21T07:25:21.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 10 client.29168 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:21.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d4cd28f80 con 0x560d4c935980 2024-09-21T07:25:21.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f578a000640 1 -- 172.21.15.27:0/2410562008 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57840ec8e0 con 0x560d4c935980 2024-09-21T07:25:21.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f578a000640 1 -- 172.21.15.27:0/2410562008 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f578c0038c0 con 0x560d4c935980 2024-09-21T07:25:21.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 10 client.29168 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:21.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 10 client.29168 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:25:21.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 20 client.29168 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:21.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c91d1c0 0x560d4cd2c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 4 client.29168 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:21.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.765+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d4cc7dbf0 con 0x560d4c91d1c0 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.766+0000 7f5793e00640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c91d1c0 0x560d4cd2c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.766+0000 7f5793e00640 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c91d1c0 0x560d4cd2c4c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41f649c2192b5a3 server_cookie=f9185aef7798335 in_seq=0 out_seq=0 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.766+0000 7f578a000640 10 client.29168 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.767+0000 7f578a000640 1 -- 172.21.15.27:0/2410562008 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d4cc7dbf0 con 0x560d4c91d1c0 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.767+0000 7f578a000640 10 client.29168 handle_command_reply: tid=0 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.767+0000 7f57a9cc8740 1 client.29168 shutdown 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.767+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c935980 msgr2=0x560d4cd1a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.767+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c935980 0x560d4cd1a1f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f578c012810 tx=0x7f578c012cd0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 shutdown_connections 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c91d1c0 0x560d4cd2c4c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c93db60 0x560d4cd1a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2410562008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c935980 0x560d4cd1a1f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f578a000640 0 client.29168 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:21.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 >> 172.21.15.27:0/2410562008 conn(0x560d4c952990 msgr2=0x560d4cd029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:21.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 shutdown_connections 2024-09-21T07:25:21.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2410562008 wait complete. 2024-09-21T07:25:21.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.768+0000 7f57a9cc8740 20 client.29168 trim_cache size 0 max 16384 2024-09-21T07:25:21.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.787+0000 7f57a9cc8740 1 Processor -- start 2024-09-21T07:25:21.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.787+0000 7f57a9cc8740 1 -- start start 2024-09-21T07:25:21.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c93db60 0x560d4cd264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c95e840 0x560d4cd173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1b1650 con 0x560d4c93db60 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1aef20 con 0x560d4c95e840 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1acd90 con 0x560d4c935980 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c95e840 0x560d4cd173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f579be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c95e840 0x560d4cd173e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52842/0 (socket says 172.21.15.27:52842) 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f57a1000640 1 -- 172.21.15.27:0/3085309105 learned_addr learned my addr 172.21.15.27:0/3085309105 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.788+0000 7f5793e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c93db60 0x560d4cd264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f57a1000640 1 -- 172.21.15.27:0/3085309105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 msgr2=0x560d4cd030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f57a1000640 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f57a1000640 1 -- 172.21.15.27:0/3085309105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c93db60 msgr2=0x560d4cd264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f57a1000640 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c93db60 0x560d4cd264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f57a1000640 1 -- 172.21.15.27:0/3085309105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4cd18ed0 con 0x560d4c95e840 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f5793e00640 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c93db60 0x560d4cd264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f57a1000640 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c95e840 0x560d4cd173e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5794009ca0 tx=0x7f5794040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41cbb6344320911e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.789+0000 7f5790c00640 1 -- 172.21.15.27:0/3085309105 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794003970 con 0x560d4c95e840 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.790+0000 7f5790c00640 1 -- 172.21.15.27:0/3085309105 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5794036040 con 0x560d4c95e840 2024-09-21T07:25:21.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.790+0000 7f5790c00640 1 -- 172.21.15.27:0/3085309105 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579402f610 con 0x560d4c95e840 2024-09-21T07:25:21.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.790+0000 7f57a9cc8740 1 -- 172.21.15.27:0/3085309105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c95e840 msgr2=0x560d4cd173e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.790+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c95e840 0x560d4cd173e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5794009ca0 tx=0x7f5794040000 comp rx=0 tx=0).stop 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 -- 172.21.15.27:0/3085309105 shutdown_connections 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c95e840 0x560d4cd173e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c93db60 0x560d4cd264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/3085309105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 -- 172.21.15.27:0/3085309105 >> 172.21.15.27:0/3085309105 conn(0x560d4c952990 msgr2=0x560d4cd029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 -- 172.21.15.27:0/3085309105 shutdown_connections 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 -- 172.21.15.27:0/3085309105 wait complete. 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 -- start start 2024-09-21T07:25:21.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.791+0000 7f57a9cc8740 1 Processor -- start 2024-09-21T07:25:21.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd22a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd1ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a9cc8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd1f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1ad3e0 con 0x560d4c95e840 2024-09-21T07:25:21.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1ae850 con 0x560d4c93db60 2024-09-21T07:25:21.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a9cc8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1adab0 con 0x560d4c935980 2024-09-21T07:25:21.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a9cc8740 10 client.0 _ll_register_callbacks cb 0x7f578c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f5793e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd1ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f5793e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd1ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52850/0 (socket says 172.21.15.27:52850) 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f5793e00640 1 -- 172.21.15.27:0/2861249903 learned_addr learned my addr 172.21.15.27:0/2861249903 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f57a1000640 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd1f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.792+0000 7f579be00640 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd22a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f5793e00640 1 -- 172.21.15.27:0/2861249903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 msgr2=0x560d4cd22a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f5793e00640 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd22a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f5793e00640 1 -- 172.21.15.27:0/2861249903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 msgr2=0x560d4cd1f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f5793e00640 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd1f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f5793e00640 1 -- 172.21.15.27:0/2861249903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4cd1f9c0 con 0x560d4c93db60 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f5793e00640 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd1ef40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f578400c6a0 tx=0x7f578400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4363c0d48683c6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f578a000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f578a000640 1 -- 172.21.15.27:0/2861249903 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57840a9770 con 0x560d4c93db60 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5794036040 con 0x560d4c93db60 2024-09-21T07:25:21.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.793+0000 7f578a000640 1 -- 172.21.15.27:0/2861249903 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f57840ebb40 con 0x560d4c93db60 2024-09-21T07:25:21.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f578a000640 1 -- 172.21.15.27:0/2861249903 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57840a9040 con 0x560d4c93db60 2024-09-21T07:25:21.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 10 client.29174 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:21.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d4cd97b60 con 0x560d4c93db60 2024-09-21T07:25:21.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f578a000640 1 -- 172.21.15.27:0/2861249903 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5794036040 con 0x560d4c93db60 2024-09-21T07:25:21.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f578a000640 1 -- 172.21.15.27:0/2861249903 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f57840e53a0 con 0x560d4c93db60 2024-09-21T07:25:21.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 10 client.29174 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 10 client.29174 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 20 client.29174 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c8f2570 0x560d4cd99df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 4 client.29174 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.794+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x560d4cc796a0 con 0x560d4c8f2570 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.795+0000 7f579be00640 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c8f2570 0x560d4cd99df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.795+0000 7f579be00640 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c8f2570 0x560d4cd99df0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21cff1be2ed0c0e7 server_cookie=a46437154d2a60e2 in_seq=0 out_seq=0 2024-09-21T07:25:21.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.795+0000 7f578a000640 10 client.29174 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:21.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f578a000640 1 -- 172.21.15.27:0/2861249903 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d4cc796a0 con 0x560d4c8f2570 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f578a000640 10 client.29174 handle_command_reply: tid=0 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 client.29174 shutdown 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 msgr2=0x560d4cd1ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd1ef40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f578400c6a0 tx=0x7f578400cef0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 shutdown_connections 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560d4c8f2570 0x560d4cd99df0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d4c95e840 0x560d4cd1f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d4c93db60 0x560d4cd1ef40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 --2- 172.21.15.27:0/2861249903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d4c935980 0x560d4cd22a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f578a000640 0 client.29174 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.806+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 >> 172.21.15.27:0/2861249903 conn(0x560d4c952990 msgr2=0x560d4cd07380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.807+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 shutdown_connections 2024-09-21T07:25:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.807+0000 7f57a9cc8740 1 -- 172.21.15.27:0/2861249903 wait complete. 2024-09-21T07:25:21.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.807+0000 7f57a9cc8740 20 client.29174 trim_cache size 0 max 16384 2024-09-21T07:25:21.810 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:21.810 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:25:21.810 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:25:21.810 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 46, 2024-09-21T07:25:21.810 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "d8798047": { 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 46, 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.8, 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:25:21.811 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:21.812 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.807+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57800b6700 msgr2=0x7f57800b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57800b6700 0x7f57800b8c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5794006010 tx=0x7f5794005d80 comp rx=0 tx=0).stop 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c1094c0 msgr2=0x7f579c19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c1094c0 0x7f579c19cbf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f578400e9f0 tx=0x7f578400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 shutdown_connections 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f57800b6700 0x7f57800b8c10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c109e10 0x7f579c1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f579c1094c0 0x7f579c19cbf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 --2- 172.21.15.27:0/3029054793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f579c103b80 0x7f579c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:21.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 >> 172.21.15.27:0/3029054793 conn(0x7f579c0ff5d0 msgr2=0x7f579c106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:21.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 shutdown_connections 2024-09-21T07:25:21.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:21.808+0000 7f57a3000640 1 -- 172.21.15.27:0/3029054793 wait complete. 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.659+0000 7f3795400640 1 -- 172.21.15.27:0/3536872427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 msgr2=0x7f3790077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.659+0000 7f3795400640 1 --2- 172.21.15.27:0/3536872427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f3790077d50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3780009900 tx=0x7f378002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.659+0000 7f3795400640 1 -- 172.21.15.27:0/3536872427 shutdown_connections 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.659+0000 7f3795400640 1 --2- 172.21.15.27:0/3536872427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3790078290 0x7f3790114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.659+0000 7f3795400640 1 --2- 172.21.15.27:0/3536872427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3790114740 0x7f3790116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.659+0000 7f3795400640 1 --2- 172.21.15.27:0/3536872427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f3790077d50 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.659+0000 7f3795400640 1 -- 172.21.15.27:0/3536872427 >> 172.21.15.27:0/3536872427 conn(0x7f3790101810 msgr2=0x7f3790103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.660+0000 7f3795400640 1 -- 172.21.15.27:0/3536872427 shutdown_connections 2024-09-21T07:25:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.660+0000 7f3795400640 1 -- 172.21.15.27:0/3536872427 wait complete. 2024-09-21T07:25:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.661+0000 7f3795400640 1 Processor -- start 2024-09-21T07:25:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.661+0000 7f3795400640 1 -- start start 2024-09-21T07:25:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f3795400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f37901a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f3795400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3790078290 0x7f37901a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f3795400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3790114740 0x7f37901a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f3795400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f379011a0b0 con 0x7f3790077940 2024-09-21T07:25:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f3795400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3790119f30 con 0x7f3790078290 2024-09-21T07:25:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f3795400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f379011a230 con 0x7f3790114740 2024-09-21T07:25:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f378e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f37901a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f378e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f37901a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58382/0 (socket says 172.21.15.27:58382) 2024-09-21T07:25:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f378e800640 1 -- 172.21.15.27:0/927679236 learned_addr learned my addr 172.21.15.27:0/927679236 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f3785e00640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3790078290 0x7f37901a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f378f200640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3790114740 0x7f37901a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f378e800640 1 -- 172.21.15.27:0/927679236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3790114740 msgr2=0x7f37901a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.662+0000 7f378e800640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3790114740 0x7f37901a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f378e800640 1 -- 172.21.15.27:0/927679236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3790078290 msgr2=0x7f37901a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f378e800640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3790078290 0x7f37901a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f378e800640 1 -- 172.21.15.27:0/927679236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37901a6a50 con 0x7f3790077940 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f378e800640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f37901a0ae0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3780005940 tx=0x7f37800040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f3787e00640 1 -- 172.21.15.27:0/927679236 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378003f030 con 0x7f3790077940 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f3787e00640 1 -- 172.21.15.27:0/927679236 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37800044c0 con 0x7f3790077940 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f3787e00640 1 -- 172.21.15.27:0/927679236 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780035070 con 0x7f3790077940 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37901a6ce0 con 0x7f3790077940 2024-09-21T07:25:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.663+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37901ad4a0 con 0x7f3790077940 2024-09-21T07:25:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 Processor -- start 2024-09-21T07:25:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 -- start start 2024-09-21T07:25:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f3787e00640 1 -- 172.21.15.27:0/927679236 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3780042030 con 0x7f3790077940 2024-09-21T07:25:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d097cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05ad1b0 0x5578d097d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05b5390 0x5578d097d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578d08f3f70 con 0x5578d05b5390 2024-09-21T07:25:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5578d08e33a0 con 0x5578d05ad1b0 2024-09-21T07:25:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.667+0000 7f379c04c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578d08e71b0 con 0x5578d05d6070 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d097cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05ad1b0 0x5578d097d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d097cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35918/0 (socket says 172.21.15.27:35918) 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 -- 172.21.15.27:0/1413170573 learned_addr learned my addr 172.21.15.27:0/1413170573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f3785e00640 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05b5390 0x5578d097d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 -- 172.21.15.27:0/1413170573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05ad1b0 msgr2=0x5578d097d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05ad1b0 0x5578d097d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 -- 172.21.15.27:0/1413170573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05b5390 msgr2=0x5578d097d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05b5390 0x5578d097d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f378f200640 1 -- 172.21.15.27:0/1413170573 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d08adc90 con 0x5578d05d6070 2024-09-21T07:25:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.668+0000 7f3785e00640 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05b5390 0x5578d097d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.669+0000 7f378f200640 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d097cb60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3788009fc0 tx=0x7f37880118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851d58957ed26fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.669+0000 7f377f400640 1 -- 172.21.15.27:0/1413170573 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378801e070 con 0x5578d05d6070 2024-09-21T07:25:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.669+0000 7f377f400640 1 -- 172.21.15.27:0/1413170573 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f378800fa50 con 0x5578d05d6070 2024-09-21T07:25:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.669+0000 7f377f400640 1 -- 172.21.15.27:0/1413170573 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788019b60 con 0x5578d05d6070 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f3787e00640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37740b6700 0x7f37740b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f3787e00640 1 -- 172.21.15.27:0/927679236 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f3780107da0 con 0x7f3790077940 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 -- 172.21.15.27:0/1413170573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 msgr2=0x5578d097cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d097cb60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3788009fc0 tx=0x7f37880118d0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f378e800640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37740b6700 0x7f37740b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 -- 172.21.15.27:0/1413170573 shutdown_connections 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05b5390 0x5578d097d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05ad1b0 0x5578d097d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 --2- 172.21.15.27:0/1413170573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d097cb60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 -- 172.21.15.27:0/1413170573 >> 172.21.15.27:0/1413170573 conn(0x5578d05ca460 msgr2=0x5578d097aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:22.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.670+0000 7f379c04c740 1 -- 172.21.15.27:0/1413170573 shutdown_connections 2024-09-21T07:25:22.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.671+0000 7f379c04c740 1 -- 172.21.15.27:0/1413170573 wait complete. 2024-09-21T07:25:22.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.671+0000 7f378e800640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37740b6700 0x7f37740b8c10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f37801051f0 tx=0x7f378010b8f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:22.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.671+0000 7f379c04c740 1 -- start start 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.671+0000 7f379c04c740 1 Processor -- start 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f379c04c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0988730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f379c04c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05b5390 0x5578d0988c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f379c04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05d6070 0x5578d0994660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f379c04c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578d0744700 con 0x5578d05ad1b0 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f379c04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5578d087e540 con 0x5578d05d6070 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f379c04c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578d098b340 con 0x5578d05b5390 2024-09-21T07:25:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f379c04c740 10 client.0 _ll_register_callbacks cb 0x5578d088e430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f378f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0988730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f378f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0988730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58392/0 (socket says 172.21.15.27:58392) 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.672+0000 7f3785e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05b5390 0x5578d0988c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f378f200640 1 -- 172.21.15.27:0/3472468350 learned_addr learned my addr 172.21.15.27:0/3472468350 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f378f200640 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05d6070 0x5578d0994660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f3785e00640 1 -- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05d6070 msgr2=0x5578d0994660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f3785e00640 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05d6070 0x5578d0994660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f3785e00640 1 -- 172.21.15.27:0/3472468350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 msgr2=0x5578d0988730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f3785e00640 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0988730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f3785e00640 1 -- 172.21.15.27:0/3472468350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d0641650 con 0x5578d05b5390 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f378f200640 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0988730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f3785e00640 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05b5390 0x5578d0988c70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f379006c810 tx=0x7f3778012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b441f9cd5de923fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.673+0000 7f3773e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.674+0000 7f3773e00640 1 -- 172.21.15.27:0/3472468350 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778004050 con 0x5578d05b5390 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.674+0000 7f3773e00640 1 -- 172.21.15.27:0/3472468350 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f37780041f0 con 0x5578d05b5390 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.674+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5578d0880b50 con 0x5578d05b5390 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.674+0000 7f3773e00640 1 -- 172.21.15.27:0/3472468350 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37780043f0 con 0x5578d05b5390 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.674+0000 7f379c04c740 10 client.29152 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.674+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578d089b910 con 0x5578d05b5390 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.674+0000 7f3773e00640 1 -- 172.21.15.27:0/3472468350 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5578d0880b50 con 0x5578d05b5390 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f3773e00640 1 -- 172.21.15.27:0/3472468350 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3778003b00 con 0x5578d05b5390 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f379c04c740 10 client.29152 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f379c04c740 10 client.29152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f379c04c740 20 client.29152 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f379c04c740 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d05949f0 0x5578d099a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f379c04c740 4 client.29152 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:22.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d08f5ec0 con 0x5578d05949f0 2024-09-21T07:25:22.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.675+0000 7f378f200640 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d05949f0 0x5578d099a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.676+0000 7f3773e00640 10 client.29152 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:22.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.676+0000 7f378f200640 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d05949f0 0x5578d099a7d0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75aebd2e6f8e371 server_cookie=ed0d8ec175fc0dba in_seq=0 out_seq=0 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f3773e00640 1 -- 172.21.15.27:0/3472468350 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578d08f5ec0 con 0x5578d05949f0 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f3773e00640 10 client.29152 handle_command_reply: tid=0 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 client.29152 shutdown 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05b5390 msgr2=0x5578d0988c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05b5390 0x5578d0988c70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f379006c810 tx=0x7f3778012b60 comp rx=0 tx=0).stop 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 shutdown_connections 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d05949f0 0x5578d099a7d0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05b5390 0x5578d0988c70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05d6070 0x5578d0994660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f3773e00640 0 client.29152 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 --2- 172.21.15.27:0/3472468350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0988730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 >> 172.21.15.27:0/3472468350 conn(0x5578d05ca460 msgr2=0x5578d097fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 shutdown_connections 2024-09-21T07:25:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.677+0000 7f379c04c740 1 -- 172.21.15.27:0/3472468350 wait complete. 2024-09-21T07:25:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.678+0000 7f379c04c740 20 client.29152 trim_cache size 0 max 16384 2024-09-21T07:25:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.696+0000 7f379c04c740 1 Processor -- start 2024-09-21T07:25:22.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f379c04c740 1 -- start start 2024-09-21T07:25:22.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f379c04c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0982380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f379c04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d0985720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f379c04c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d0985c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f379c04c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b9d70 con 0x5578d05ad1b0 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f379c04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b7640 con 0x5578d05b5390 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f379c04c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b54b0 con 0x5578d05d6070 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f378f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0982380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f378f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0982380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58396/0 (socket says 172.21.15.27:58396) 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.697+0000 7f378f200640 1 -- 172.21.15.27:0/2939825303 learned_addr learned my addr 172.21.15.27:0/2939825303 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f3785e00640 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d0985720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f378f200640 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d0985c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f3785e00640 1 -- 172.21.15.27:0/2939825303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 msgr2=0x5578d0985c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f3785e00640 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d0985c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f3785e00640 1 -- 172.21.15.27:0/2939825303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 msgr2=0x5578d0982380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f3785e00640 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0982380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f3785e00640 1 -- 172.21.15.27:0/2939825303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d097f9c0 con 0x5578d05b5390 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f378f200640 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0982380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.698+0000 7f3785e00640 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d0985720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3778003ef0 tx=0x7f3778009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea8a8824afc3062 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.699+0000 7f377ea00640 1 -- 172.21.15.27:0/2939825303 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377802d020 con 0x5578d05b5390 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.699+0000 7f377ea00640 1 -- 172.21.15.27:0/2939825303 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778010e50 con 0x5578d05b5390 2024-09-21T07:25:22.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.699+0000 7f377ea00640 1 -- 172.21.15.27:0/2939825303 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778003850 con 0x5578d05b5390 2024-09-21T07:25:22.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 -- 172.21.15.27:0/2939825303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 msgr2=0x5578d0985720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d0985720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3778003ef0 tx=0x7f3778009300 comp rx=0 tx=0).stop 2024-09-21T07:25:22.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 -- 172.21.15.27:0/2939825303 shutdown_connections 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d0985c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d0985720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 --2- 172.21.15.27:0/2939825303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0982380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 -- 172.21.15.27:0/2939825303 >> 172.21.15.27:0/2939825303 conn(0x5578d05ca460 msgr2=0x5578d0981c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 -- 172.21.15.27:0/2939825303 shutdown_connections 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 -- 172.21.15.27:0/2939825303 wait complete. 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.700+0000 7f379c04c740 1 -- start start 2024-09-21T07:25:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 1 Processor -- start 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0990b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d098f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d098f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b5b00 con 0x5578d05ad1b0 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b6f70 con 0x5578d05b5390 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b61d0 con 0x5578d05d6070 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.701+0000 7f379c04c740 10 client.0 _ll_register_callbacks cb 0x7f3778009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0990b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f3785e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d098f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0990b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58408/0 (socket says 172.21.15.27:58408) 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 -- 172.21.15.27:0/3681587909 learned_addr learned my addr 172.21.15.27:0/3681587909 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f3785e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d098f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:52882/0 (socket says 172.21.15.27:52882) 2024-09-21T07:25:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 -- 172.21.15.27:0/3681587909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 msgr2=0x5578d098f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d098f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 -- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 msgr2=0x5578d098f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d098f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f378f200640 1 -- 172.21.15.27:0/3681587909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d098fb10 con 0x5578d05ad1b0 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.702+0000 7f3785e00640 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d098f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f378f200640 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0990b80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3788005c40 tx=0x7f3788005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d93a41ad9c0e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f3773e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f3773e00640 1 -- 172.21.15.27:0/3681587909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788034070 con 0x5578d05ad1b0 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f3773e00640 1 -- 172.21.15.27:0/3681587909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f37880197f0 con 0x5578d05ad1b0 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3778010e50 con 0x5578d05ad1b0 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f3773e00640 1 -- 172.21.15.27:0/3681587909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f378801e030 con 0x5578d05ad1b0 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f379c04c740 10 client.19173 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.703+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578d0a05f20 con 0x5578d05ad1b0 2024-09-21T07:25:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f3773e00640 1 -- 172.21.15.27:0/3681587909 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778010e50 con 0x5578d05ad1b0 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f3773e00640 1 -- 172.21.15.27:0/3681587909 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f378800fc50 con 0x5578d05ad1b0 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f379c04c740 10 client.19173 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f379c04c740 10 client.19173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f379c04c740 20 client.19173 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f379c04c740 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d0569da0 0x5578d0a08520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f379c04c740 4 client.19173 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.704+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578d08f1970 con 0x5578d0569da0 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.705+0000 7f3785e00640 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d0569da0 0x5578d0a08520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.705+0000 7f3785e00640 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d0569da0 0x5578d0a08520 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439220914ee8e8d2 server_cookie=afcb0f3eb9822513 in_seq=0 out_seq=0 2024-09-21T07:25:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.705+0000 7f3773e00640 10 client.19173 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f3773e00640 1 -- 172.21.15.27:0/3681587909 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5966 (crc 0 0 0) 0x5578d08f1970 con 0x5578d0569da0 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f3773e00640 10 client.19173 handle_command_reply: tid=0 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 client.19173 shutdown 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 msgr2=0x5578d0990b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0990b80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3788005c40 tx=0x7f3788005bd0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 shutdown_connections 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5578d0569da0 0x5578d0a08520 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578d05d6070 0x5578d098f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578d05b5390 0x5578d098f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 --2- 172.21.15.27:0/3681587909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578d05ad1b0 0x5578d0990b80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f3773e00640 0 client.19173 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 >> 172.21.15.27:0/3681587909 conn(0x5578d05ca460 msgr2=0x5578d097db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 shutdown_connections 2024-09-21T07:25:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 1 -- 172.21.15.27:0/3681587909 wait complete. 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.706+0000 7f379c04c740 20 client.19173 trim_cache size 0 max 16384 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "842dd810-32aa-4200-bb13-a6fd281291ee", 2024-09-21T07:25:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:25:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.719 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:22.721 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37740b6700 msgr2=0x7f37740b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37740b6700 0x7f37740b8c10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f37801051f0 tx=0x7f378010b8f0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 msgr2=0x7f37901a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f37901a0ae0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3780005940 tx=0x7f37800040c0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 shutdown_connections 2024-09-21T07:25:22.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3790114740 0x7f37901a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37740b6700 0x7f37740b8c10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3790078290 0x7f37901a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 --2- 172.21.15.27:0/927679236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3790077940 0x7f37901a0ae0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 >> 172.21.15.27:0/927679236 conn(0x7f3790101810 msgr2=0x7f3790116870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 shutdown_connections 2024-09-21T07:25:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:22.707+0000 7f3795400640 1 -- 172.21.15.27:0/927679236 wait complete. 2024-09-21T07:25:23.253 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd8798047', quiesce took 5.3 seconds. Will release after: 12.5 2024-09-21T07:25:23.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:23 smithi027 ceph-mon[30779]: pgmap v762: 129 pgs: 129 active+clean; 2.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:25:23.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:23 smithi119 ceph-mon[34519]: pgmap v762: 129 pgs: 129 active+clean; 2.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:25:23.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:23 smithi153 ceph-mon[34747]: pgmap v762: 129 pgs: 129 active+clean; 2.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-21T07:25:23.785 INFO:tasks.cephfs.filesystem:scrub status for tag:842dd810-32aa-4200-bb13-a6fd281291ee - {'path': '/', 'tag': '842dd810-32aa-4200-bb13-a6fd281291ee', 'options': 'recursive,force'} 2024-09-21T07:25:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:24 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:25:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:24 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:25:24.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:24 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:25:25.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:25 smithi119 ceph-mon[34519]: pgmap v763: 129 pgs: 129 active+clean; 2.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-21T07:25:25.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:25 smithi027 ceph-mon[30779]: pgmap v763: 129 pgs: 129 active+clean; 2.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-21T07:25:25.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:25 smithi153 ceph-mon[34747]: pgmap v763: 129 pgs: 129 active+clean; 2.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-21T07:25:27.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:27 smithi119 ceph-mon[34519]: pgmap v764: 129 pgs: 129 active+clean; 2.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-21T07:25:27.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:27 smithi027 ceph-mon[30779]: pgmap v764: 129 pgs: 129 active+clean; 2.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-21T07:25:27.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:27 smithi153 ceph-mon[34747]: pgmap v764: 129 pgs: 129 active+clean; 2.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-09-21T07:25:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:29 smithi027 ceph-mon[30779]: pgmap v765: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:25:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:29 smithi119 ceph-mon[34519]: pgmap v765: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:25:29.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:29 smithi153 ceph-mon[34747]: pgmap v765: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:25:31.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:31 smithi119 ceph-mon[34519]: pgmap v766: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:25:31.651 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:31 smithi027 ceph-mon[30779]: pgmap v766: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:25:31.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:31 smithi153 ceph-mon[34747]: pgmap v766: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-09-21T07:25:33.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:33 smithi119 ceph-mon[34519]: pgmap v767: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-21T07:25:33.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:33 smithi027 ceph-mon[30779]: pgmap v767: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-21T07:25:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:33 smithi153 ceph-mon[34747]: pgmap v767: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.187+0000 7fc2a3000640 1 -- 172.21.15.27:0/1868580785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109240 msgr2=0x7fc29c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.187+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1868580785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109240 0x7fc29c109650 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc288009900 tx=0x7fc28802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.188+0000 7fc2a3000640 1 -- 172.21.15.27:0/1868580785 shutdown_connections 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.188+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1868580785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc29c109b90 0x7fc29c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.188+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1868580785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109240 0x7fc29c109650 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.188+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1868580785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c103910 0x7fc29c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.188+0000 7fc2a3000640 1 -- 172.21.15.27:0/1868580785 >> 172.21.15.27:0/1868580785 conn(0x7fc29c0ff330 msgr2=0x7fc29c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.188+0000 7fc2a3000640 1 -- 172.21.15.27:0/1868580785 shutdown_connections 2024-09-21T07:25:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.188+0000 7fc2a3000640 1 -- 172.21.15.27:0/1868580785 wait complete. 2024-09-21T07:25:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.189+0000 7fc2a3000640 1 Processor -- start 2024-09-21T07:25:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.189+0000 7fc2a3000640 1 -- start start 2024-09-21T07:25:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a3000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc29c103910 0x7fc29c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a3000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c109240 0x7fc29c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a3000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109b90 0x7fc29c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a3000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c113850 con 0x7fc29c109b90 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a3000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1136d0 con 0x7fc29c109240 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a3000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1139d0 con 0x7fc29c103910 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109b90 0x7fc29c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109b90 0x7fc29c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41902/0 (socket says 172.21.15.27:41902) 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a1200640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c109240 0x7fc29c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 -- 172.21.15.27:0/1258774320 learned_addr learned my addr 172.21.15.27:0/1258774320 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc29c103910 0x7fc29c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 -- 172.21.15.27:0/1258774320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc29c103910 msgr2=0x7fc29c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc29c103910 0x7fc29c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 -- 172.21.15.27:0/1258774320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c109240 msgr2=0x7fc29c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c109240 0x7fc29c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a2600640 1 -- 172.21.15.27:0/1258774320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c1a8560 con 0x7fc29c109b90 2024-09-21T07:25:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.190+0000 7fc2a1200640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c109240 0x7fc29c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.191+0000 7fc2a2600640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109b90 0x7fc29c076190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc29400e970 tx=0x7fc29400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.191+0000 7fc29a000640 1 -- 172.21.15.27:0/1258774320 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29400ccc0 con 0x7fc29c109b90 2024-09-21T07:25:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.191+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c1a8850 con 0x7fc29c109b90 2024-09-21T07:25:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.191+0000 7fc29a000640 1 -- 172.21.15.27:0/1258774320 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc29400ce60 con 0x7fc29c109b90 2024-09-21T07:25:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.191+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c1a8dc0 con 0x7fc29c109b90 2024-09-21T07:25:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.191+0000 7fc29a000640 1 -- 172.21.15.27:0/1258774320 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294018430 con 0x7fc29c109b90 2024-09-21T07:25:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.194+0000 7fc2ab846740 1 Processor -- start 2024-09-21T07:25:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc29a000640 1 -- 172.21.15.27:0/1258774320 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc294004ef0 con 0x7fc29c109b90 2024-09-21T07:25:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2ab846740 1 -- start start 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f591b0 0x5564692fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x5564692fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f384d0 0x5564692fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2ab846740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564692736b0 con 0x556468f302f0 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2ab846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556469262ae0 con 0x556468f384d0 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2ab846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5564692668f0 con 0x556468f591b0 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.195+0000 7fc2a1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x5564692fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x5564692fe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41908/0 (socket says 172.21.15.27:41908) 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 -- 172.21.15.27:0/240131731 learned_addr learned my addr 172.21.15.27:0/240131731 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1200640 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f591b0 0x5564692fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1200640 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f384d0 0x5564692fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 -- 172.21.15.27:0/240131731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f591b0 msgr2=0x5564692fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f591b0 0x5564692fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 -- 172.21.15.27:0/240131731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f384d0 msgr2=0x5564692fe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f384d0 0x5564692fe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 -- 172.21.15.27:0/240131731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646922f780 con 0x556468f302f0 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1200640 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f591b0 0x5564692fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x5564692fe350 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc290011470 tx=0x7fc290011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6255d0f05a870c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.196+0000 7fc29a000640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2840b66b0 0x7fc2840b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc28ea00640 1 -- 172.21.15.27:0/240131731 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29001e070 con 0x556468f302f0 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc28ea00640 1 -- 172.21.15.27:0/240131731 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc290004030 con 0x556468f302f0 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc28ea00640 1 -- 172.21.15.27:0/240131731 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000f7a0 con 0x556468f302f0 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc29a000640 1 -- 172.21.15.27:0/1258774320 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fc2940e7ff0 con 0x7fc29c109b90 2024-09-21T07:25:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc2a1200640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2840b66b0 0x7fc2840b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc2ab846740 1 -- 172.21.15.27:0/240131731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 msgr2=0x5564692fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc2ab846740 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x5564692fe350 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc290011470 tx=0x7fc290011930 comp rx=0 tx=0).stop 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.197+0000 7fc2a1200640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2840b66b0 0x7fc2840b8bc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc288009230 tx=0x7fc288031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 -- 172.21.15.27:0/240131731 shutdown_connections 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f384d0 0x5564692fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x5564692fe350 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 --2- 172.21.15.27:0/240131731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f591b0 0x5564692fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 -- 172.21.15.27:0/240131731 >> 172.21.15.27:0/240131731 conn(0x556468f4d300 msgr2=0x5564692fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 -- 172.21.15.27:0/240131731 shutdown_connections 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 -- 172.21.15.27:0/240131731 wait complete. 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.198+0000 7fc2ab846740 1 -- start start 2024-09-21T07:25:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 1 Processor -- start 2024-09-21T07:25:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f302f0 0x5564693093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x556469309910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f591b0 0x556469309e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55646919aab0 con 0x556468f591b0 2024-09-21T07:25:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556469209570 con 0x556468f302f0 2024-09-21T07:25:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55646930c590 con 0x556468f384d0 2024-09-21T07:25:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2ab846740 10 client.0 _ll_register_callbacks cb 0x556469218100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2a1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x556469309910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2a2600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f591b0 0x556469309e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2a1200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x556469309910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58386/0 (socket says 172.21.15.27:58386) 2024-09-21T07:25:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.199+0000 7fc2a1200640 1 -- 172.21.15.27:0/367344591 learned_addr learned my addr 172.21.15.27:0/367344591 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f302f0 0x5564693093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a2600640 1 -- 172.21.15.27:0/367344591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 msgr2=0x556469309910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a2600640 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x556469309910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a2600640 1 -- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f302f0 msgr2=0x5564693093d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a2600640 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f302f0 0x5564693093d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a2600640 1 -- 172.21.15.27:0/367344591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556468fc4790 con 0x556468f591b0 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f302f0 0x5564693093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc2a2600640 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f591b0 0x556469309e50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc2940e5330 tx=0x7fc2940054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b79cbd1e4831731 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc283400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.200+0000 7fc283400640 1 -- 172.21.15.27:0/367344591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2940a9580 con 0x556468f591b0 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc283400640 1 -- 172.21.15.27:0/367344591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fc2940ac700 con 0x556468f591b0 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc290004030 con 0x556468f591b0 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc283400640 1 -- 172.21.15.27:0/367344591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2940ad540 con 0x556468f591b0 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 10 client.19185 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55646903d620 con 0x556468f591b0 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc283400640 1 -- 172.21.15.27:0/367344591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc290004030 con 0x556468f591b0 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc283400640 1 -- 172.21.15.27:0/367344591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2940ac900 con 0x556468f591b0 2024-09-21T07:25:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 10 client.19185 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 10 client.19185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 20 client.19185 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468f17d80 0x55646931b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 4 client.19185 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.201+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556469275600 con 0x556468f17d80 2024-09-21T07:25:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.202+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468f17d80 0x55646931b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.203+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468f17d80 0x55646931b590 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759d0b725c42a77c server_cookie=ebeaa83545ee54a in_seq=0 out_seq=0 2024-09-21T07:25:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.203+0000 7fc283400640 10 client.19185 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.203+0000 7fc283400640 1 -- 172.21.15.27:0/367344591 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556469275600 con 0x556468f17d80 2024-09-21T07:25:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.203+0000 7fc283400640 10 client.19185 handle_command_reply: tid=0 2024-09-21T07:25:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.203+0000 7fc2ab846740 1 client.19185 shutdown 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f591b0 msgr2=0x556469309e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f591b0 0x556469309e50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc2940e5330 tx=0x7fc2940054e0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 shutdown_connections 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468f17d80 0x55646931b590 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x556469309910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f591b0 0x556469309e50 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 --2- 172.21.15.27:0/367344591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f302f0 0x5564693093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc283400640 0 client.19185 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 >> 172.21.15.27:0/367344591 conn(0x556468f4d300 msgr2=0x5564692fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 shutdown_connections 2024-09-21T07:25:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 1 -- 172.21.15.27:0/367344591 wait complete. 2024-09-21T07:25:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.204+0000 7fc2ab846740 20 client.19185 trim_cache size 0 max 16384 2024-09-21T07:25:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 Processor -- start 2024-09-21T07:25:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 -- start start 2024-09-21T07:25:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x5564693065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x556469306ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1b57b0 con 0x556468f302f0 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1b3080 con 0x556468f591b0 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2ab846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1b0ef0 con 0x556468f384d0 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.220+0000 7fc2a1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41922/0 (socket says 172.21.15.27:41922) 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 -- 172.21.15.27:0/3136793284 learned_addr learned my addr 172.21.15.27:0/3136793284 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a2600640 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x556469306ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1200640 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x5564693065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 -- 172.21.15.27:0/3136793284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 msgr2=0x5564693065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x5564693065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 -- 172.21.15.27:0/3136793284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 msgr2=0x556469306ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x556469306ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 -- 172.21.15.27:0/3136793284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564692febf0 con 0x556468f302f0 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930b000 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc290009a00 tx=0x7fc290004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f13c4d07972015 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.221+0000 7fc28e000640 1 -- 172.21.15.27:0/3136793284 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290031070 con 0x556468f302f0 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc28e000640 1 -- 172.21.15.27:0/3136793284 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc290005950 con 0x556468f302f0 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc28e000640 1 -- 172.21.15.27:0/3136793284 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29001e040 con 0x556468f302f0 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc2ab846740 1 -- 172.21.15.27:0/3136793284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 msgr2=0x55646930b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc2ab846740 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930b000 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc290009a00 tx=0x7fc290004140 comp rx=0 tx=0).stop 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc2ab846740 1 -- 172.21.15.27:0/3136793284 shutdown_connections 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc2ab846740 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x5564693065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc2ab846740 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x556469306ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc2ab846740 1 --2- 172.21.15.27:0/3136793284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930b000 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.222+0000 7fc2ab846740 1 -- 172.21.15.27:0/3136793284 >> 172.21.15.27:0/3136793284 conn(0x556468f4d300 msgr2=0x55646930a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.223+0000 7fc2ab846740 1 -- 172.21.15.27:0/3136793284 shutdown_connections 2024-09-21T07:25:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.223+0000 7fc2ab846740 1 -- 172.21.15.27:0/3136793284 wait complete. 2024-09-21T07:25:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.223+0000 7fc2ab846740 1 -- start start 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.223+0000 7fc2ab846740 1 Processor -- start 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x55646930f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2ab846740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x55646937e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2ab846740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1b1540 con 0x556468f302f0 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2ab846740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1b29b0 con 0x556468f591b0 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2ab846740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1b1c10 con 0x556468f384d0 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2ab846740 10 client.0 _ll_register_callbacks cb 0x7fc2940aa080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41926/0 (socket says 172.21.15.27:41926) 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 -- 172.21.15.27:0/1464490504 learned_addr learned my addr 172.21.15.27:0/1464490504 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1200640 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x55646930f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a2600640 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x55646937e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 -- 172.21.15.27:0/1464490504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 msgr2=0x55646930f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x55646930f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 -- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 msgr2=0x55646937e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x55646937e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.224+0000 7fc2a1c00640 1 -- 172.21.15.27:0/1464490504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646937e7d0 con 0x556468f302f0 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc2a1c00640 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930f390 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc290009da0 tx=0x7fc290011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f975b4ff57cb3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc283400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc283400640 1 -- 172.21.15.27:0/1464490504 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29003a060 con 0x556468f302f0 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc290005950 con 0x556468f302f0 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc283400640 1 -- 172.21.15.27:0/1464490504 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fc290003b30 con 0x556468f302f0 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc283400640 1 -- 172.21.15.27:0/1464490504 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc290041050 con 0x556468f302f0 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc2ab846740 10 client.19191 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556469385bc0 con 0x556468f302f0 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.225+0000 7fc283400640 1 -- 172.21.15.27:0/1464490504 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc290005950 con 0x556468f302f0 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc283400640 1 -- 172.21.15.27:0/1464490504 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc29000f770 con 0x556468f302f0 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc2ab846740 10 client.19191 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc2ab846740 10 client.19191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc2ab846740 20 client.19191 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc2ab846740 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468eed1c0 0x556469388100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc2ab846740 4 client.19191 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:25:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564692710b0 con 0x556468eed1c0 2024-09-21T07:25:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.226+0000 7fc2a1200640 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468eed1c0 0x556469388100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.227+0000 7fc2a1200640 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468eed1c0 0x556469388100 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228ee9989ed93654 server_cookie=89af62438bfd4ac9 in_seq=0 out_seq=0 2024-09-21T07:25:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.227+0000 7fc283400640 10 client.19191 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc283400640 1 -- 172.21.15.27:0/1464490504 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564692710b0 con 0x556468eed1c0 2024-09-21T07:25:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc283400640 10 client.19191 handle_command_reply: tid=0 2024-09-21T07:25:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 client.19191 shutdown 2024-09-21T07:25:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 msgr2=0x55646930f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930f390 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc290009da0 tx=0x7fc290011930 comp rx=0 tx=0).stop 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 shutdown_connections 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556468eed1c0 0x556469388100 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556468f384d0 0x55646930f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556468f591b0 0x55646937e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 --2- 172.21.15.27:0/1464490504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556468f302f0 0x55646930f390 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 >> 172.21.15.27:0/1464490504 conn(0x556468f4d300 msgr2=0x5564692fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc283400640 0 client.19191 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 shutdown_connections 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 1 -- 172.21.15.27:0/1464490504 wait complete. 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.241+0000 7fc2ab846740 20 client.19191 trim_cache size 0 max 16384 2024-09-21T07:25:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:25:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:25:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:25:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:25:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2840b66b0 msgr2=0x7fc2840b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2840b66b0 0x7fc2840b8bc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc288009230 tx=0x7fc288031400 comp rx=0 tx=0).stop 2024-09-21T07:25:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109b90 msgr2=0x7fc29c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109b90 0x7fc29c076190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc29400e970 tx=0x7fc29400ed70 comp rx=0 tx=0).stop 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 shutdown_connections 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2840b66b0 0x7fc2840b8bc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc29c109b90 0x7fc29c076190 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c109240 0x7fc29c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 --2- 172.21.15.27:0/1258774320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc29c103910 0x7fc29c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 >> 172.21.15.27:0/1258774320 conn(0x7fc29c0ff330 msgr2=0x7fc29c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 shutdown_connections 2024-09-21T07:25:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:34.242+0000 7fc2a3000640 1 -- 172.21.15.27:0/1258774320 wait complete. 2024-09-21T07:25:35.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:35 smithi119 ceph-mon[34519]: pgmap v768: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T07:25:35.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:35 smithi027 ceph-mon[30779]: pgmap v768: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T07:25:35.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:35 smithi153 ceph-mon[34747]: pgmap v768: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T07:25:35.754 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:25:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:36.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 -- 172.21.15.27:0/1180906431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 msgr2=0x7f0b78103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:36.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1180906431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 0x7f0b78103dc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009900 tx=0x7f0b6c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 -- 172.21.15.27:0/1180906431 shutdown_connections 2024-09-21T07:25:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1180906431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 0x7f0b78110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1180906431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b781092f0 0x7f0b78109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1180906431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 0x7f0b78103dc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 -- 172.21.15.27:0/1180906431 >> 172.21.15.27:0/1180906431 conn(0x7f0b780ff440 msgr2=0x7f0b78101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 -- 172.21.15.27:0/1180906431 shutdown_connections 2024-09-21T07:25:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.788+0000 7f0b7e800640 1 -- 172.21.15.27:0/1180906431 wait complete. 2024-09-21T07:25:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.789+0000 7f0b7e800640 1 Processor -- start 2024-09-21T07:25:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.790+0000 7f0b7e800640 1 -- start start 2024-09-21T07:25:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.790+0000 7f0b7e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 0x7f0b7819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.790+0000 7f0b7e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b781092f0 0x7f0b7819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.790+0000 7f0b7e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 0x7f0b781a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.790+0000 7f0b7e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b78113900 con 0x7f0b781039b0 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.790+0000 7f0b7e800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b78113780 con 0x7f0b781092f0 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.790+0000 7f0b7e800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b78113a80 con 0x7f0b78109c40 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b781092f0 0x7f0b7819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 0x7f0b781a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 0x7f0b7819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b781092f0 0x7f0b7819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:56068/0 (socket says 172.21.15.27:56068) 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 0x7f0b781a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:58410/0 (socket says 172.21.15.27:58410) 2024-09-21T07:25:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7ca00640 1 -- 172.21.15.27:0/1121708992 learned_addr learned my addr 172.21.15.27:0/1121708992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7de00640 1 -- 172.21.15.27:0/1121708992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b781092f0 msgr2=0x7f0b7819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7de00640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b781092f0 0x7f0b7819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7de00640 1 -- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 msgr2=0x7f0b7819c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7de00640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 0x7f0b7819c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.791+0000 7f0b7de00640 1 -- 172.21.15.27:0/1121708992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b781a24a0 con 0x7f0b78109c40 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.792+0000 7f0b7de00640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 0x7f0b781a1d70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009fd0 tx=0x7f0b7000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.792+0000 7f0b76000640 1 -- 172.21.15.27:0/1121708992 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000ce20 con 0x7f0b78109c40 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.792+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b781a2790 con 0x7f0b78109c40 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.792+0000 7f0b76000640 1 -- 172.21.15.27:0/1121708992 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b70005020 con 0x7f0b78109c40 2024-09-21T07:25:36.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.792+0000 7f0b76000640 1 -- 172.21.15.27:0/1121708992 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70010690 con 0x7f0b78109c40 2024-09-21T07:25:36.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.792+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b781a8ef0 con 0x7f0b78109c40 2024-09-21T07:25:36.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.793+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b78103dc0 con 0x7f0b78109c40 2024-09-21T07:25:36.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.796+0000 7f0b76000640 1 -- 172.21.15.27:0/1121708992 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0b700054d0 con 0x7f0b78109c40 2024-09-21T07:25:36.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.797+0000 7f0b76000640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b600b6680 0x7f0b600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:36.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.798+0000 7f0b76000640 1 -- 172.21.15.27:0/1121708992 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f0b700e81c0 con 0x7f0b78109c40 2024-09-21T07:25:36.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.798+0000 7f0b7d400640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b600b6680 0x7f0b600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:36.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.798+0000 7f0b7d400640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b600b6680 0x7f0b600b8b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c031440 tx=0x7f0b6c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:36.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:36.800+0000 7f0b76000640 1 -- 172.21.15.27:0/1121708992 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b700ae300 con 0x7f0b78109c40 2024-09-21T07:25:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.036+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b7819d660 con 0x7f0b78109c40 2024-09-21T07:25:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.037+0000 7f0b76000640 1 -- 172.21.15.27:0/1121708992 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f0b7819d660 con 0x7f0b78109c40 2024-09-21T07:25:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:25:37.040 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:25:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:25:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b600b6680 msgr2=0x7f0b600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b600b6680 0x7f0b600b8b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c031440 tx=0x7f0b6c009560 comp rx=0 tx=0).stop 2024-09-21T07:25:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 msgr2=0x7f0b781a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 0x7f0b781a1d70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009fd0 tx=0x7f0b7000ef10 comp rx=0 tx=0).stop 2024-09-21T07:25:37.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 shutdown_connections 2024-09-21T07:25:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b78109c40 0x7f0b781a1d70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0b600b6680 0x7f0b600b8b90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b781092f0 0x7f0b7819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 --2- 172.21.15.27:0/1121708992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b781039b0 0x7f0b7819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.039+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 >> 172.21.15.27:0/1121708992 conn(0x7f0b780ff440 msgr2=0x7f0b7810e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.040+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 shutdown_connections 2024-09-21T07:25:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:37.040+0000 7f0b7e800640 1 -- 172.21.15.27:0/1121708992 wait complete. 2024-09-21T07:25:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:37 smithi027 ceph-mon[30779]: pgmap v769: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-09-21T07:25:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:37 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1121708992' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:37 smithi119 ceph-mon[34519]: pgmap v769: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-09-21T07:25:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:37 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1121708992' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:37.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:37 smithi153 ceph-mon[34747]: pgmap v769: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-09-21T07:25:37.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:37 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1121708992' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:37.712 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id d8798047 --release --await' 2024-09-21T07:25:37.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id d8798047 --release --await 2024-09-21T07:25:38.099 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.706+0000 7f9406800640 1 -- 172.21.15.27:0/92741499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94001047c0 msgr2=0x7f9400104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.706+0000 7f9406800640 1 --2- 172.21.15.27:0/92741499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94001047c0 0x7f9400104bd0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009900 tx=0x7f93f402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.707+0000 7f9406800640 1 -- 172.21.15.27:0/92741499 shutdown_connections 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.707+0000 7f9406800640 1 --2- 172.21.15.27:0/92741499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f940010dc20 0x7f9400110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.707+0000 7f9406800640 1 --2- 172.21.15.27:0/92741499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400105110 0x7f940010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.707+0000 7f9406800640 1 --2- 172.21.15.27:0/92741499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94001047c0 0x7f9400104bd0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.707+0000 7f9406800640 1 -- 172.21.15.27:0/92741499 >> 172.21.15.27:0/92741499 conn(0x7f94000fe2c0 msgr2=0x7f9400100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.707+0000 7f9406800640 1 -- 172.21.15.27:0/92741499 shutdown_connections 2024-09-21T07:25:38.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.707+0000 7f9406800640 1 -- 172.21.15.27:0/92741499 wait complete. 2024-09-21T07:25:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.708+0000 7f9406800640 1 Processor -- start 2024-09-21T07:25:38.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.708+0000 7f9406800640 1 -- start start 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9406800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94001047c0 0x7f940019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9406800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400105110 0x7f940019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9406800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f940010dc20 0x7f94001a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9406800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94001138d0 con 0x7f940010dc20 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9406800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113750 con 0x7f9400105110 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9406800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113a50 con 0x7f94001047c0 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94001047c0 0x7f940019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94001047c0 0x7f940019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56624/0 (socket says 172.21.15.27:56624) 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 -- 172.21.15.27:0/40995533 learned_addr learned my addr 172.21.15.27:0/40995533 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405e00640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f940010dc20 0x7f94001a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9404a00640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400105110 0x7f940019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 -- 172.21.15.27:0/40995533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400105110 msgr2=0x7f940019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400105110 0x7f940019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 -- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f940010dc20 msgr2=0x7f94001a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f940010dc20 0x7f94001a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.709+0000 7f9405400640 1 -- 172.21.15.27:0/40995533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94001a2430 con 0x7f94001047c0 2024-09-21T07:25:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.710+0000 7f9405400640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94001047c0 0x7f940019c4c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005ab0 tx=0x7f93f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.710+0000 7f93fe000640 1 -- 172.21.15.27:0/40995533 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f403f070 con 0x7f94001047c0 2024-09-21T07:25:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.710+0000 7f93fe000640 1 -- 172.21.15.27:0/40995533 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f4031b60 con 0x7f94001047c0 2024-09-21T07:25:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.710+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94001a26c0 con 0x7f94001047c0 2024-09-21T07:25:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.710+0000 7f93fe000640 1 -- 172.21.15.27:0/40995533 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4035030 con 0x7f94001047c0 2024-09-21T07:25:38.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.710+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94001a8e80 con 0x7f94001047c0 2024-09-21T07:25:38.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.713+0000 7f940eac6740 1 Processor -- start 2024-09-21T07:25:38.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.713+0000 7f940eac6740 1 -- start start 2024-09-21T07:25:38.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f940eac6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da29340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f940eac6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d659660 0x55980da29880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f940eac6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da29dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f940eac6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55980d9a0290 con 0x55980d661840 2024-09-21T07:25:38.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f940eac6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55980d98f6c0 con 0x55980d659660 2024-09-21T07:25:38.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f940eac6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55980d9934d0 con 0x55980d682520 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f93fe000640 1 -- 172.21.15.27:0/40995533 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f93f4042070 con 0x7f94001047c0 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f9405e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da29340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f9404a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da29dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f9405e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da29340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56630/0 (socket says 172.21.15.27:56630) 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.714+0000 7f9405e00640 1 -- 172.21.15.27:0/3759866497 learned_addr learned my addr 172.21.15.27:0/3759866497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9405400640 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d659660 0x55980da29880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9405e00640 1 -- 172.21.15.27:0/3759866497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d659660 msgr2=0x55980da29880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9405e00640 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d659660 0x55980da29880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9405e00640 1 -- 172.21.15.27:0/3759866497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 msgr2=0x55980da29dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9405e00640 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da29dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9405e00640 1 -- 172.21.15.27:0/3759866497 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980d91c460 con 0x55980d682520 2024-09-21T07:25:38.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9404a00640 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da29dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:38.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f9405e00640 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da29340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f93f8011440 tx=0x7f93f8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ee4499b7fbb140 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.715+0000 7f93f2600640 1 -- 172.21.15.27:0/3759866497 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8003c30 con 0x55980d682520 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.716+0000 7f93f2600640 1 -- 172.21.15.27:0/3759866497 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f8003dd0 con 0x55980d682520 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.716+0000 7f93f2600640 1 -- 172.21.15.27:0/3759866497 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f800fae0 con 0x55980d682520 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.716+0000 7f93fe000640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93e80b6680 0x7f93e80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.716+0000 7f940eac6740 1 -- 172.21.15.27:0/3759866497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 msgr2=0x55980da29340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.716+0000 7f940eac6740 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da29340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f93f8011440 tx=0x7f93f8011900 comp rx=0 tx=0).stop 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.716+0000 7f93fe000640 1 -- 172.21.15.27:0/40995533 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f93f4108190 con 0x7f94001047c0 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.716+0000 7f9405400640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93e80b6680 0x7f93e80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 -- 172.21.15.27:0/3759866497 shutdown_connections 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da29dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d659660 0x55980da29880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 --2- 172.21.15.27:0/3759866497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da29340 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 -- 172.21.15.27:0/3759866497 >> 172.21.15.27:0/3759866497 conn(0x55980d676910 msgr2=0x55980da27230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:38.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 -- 172.21.15.27:0/3759866497 shutdown_connections 2024-09-21T07:25:38.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 -- 172.21.15.27:0/3759866497 wait complete. 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f9405400640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93e80b6680 0x7f93e80b8b90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f93f41055e0 tx=0x7f93f4009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.717+0000 7f940eac6740 1 -- start start 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 1 Processor -- start 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da34ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da35410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da40e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55980d7f0bb0 con 0x55980d661840 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55980d92a6c0 con 0x55980d682520 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55980da37b20 con 0x55980d659660 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.718+0000 7f940eac6740 10 client.0 _ll_register_callbacks cb 0x55980d945490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9405e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da40e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9405e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da40e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38818/0 (socket says 172.21.15.27:38818) 2024-09-21T07:25:38.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9405e00640 1 -- 172.21.15.27:0/3455687669 learned_addr learned my addr 172.21.15.27:0/3455687669 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:38.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9404a00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da35410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9405e00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da34ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9404a00640 1 -- 172.21.15.27:0/3455687669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 msgr2=0x55980da34ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9404a00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da34ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9404a00640 1 -- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 msgr2=0x55980da40e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9404a00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da40e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9404a00640 1 -- 172.21.15.27:0/3455687669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980d6edb00 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.719+0000 7f9405e00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da40e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f9404a00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da35410 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f940006a800 tx=0x7f93ec012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a8cefa02b723ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f93e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f93e6a00640 1 -- 172.21.15.27:0/3455687669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec004030 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f8003dd0 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f93e6a00640 1 -- 172.21.15.27:0/3455687669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f93ec0041d0 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f93e6a00640 1 -- 172.21.15.27:0/3455687669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93ec025050 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f940eac6740 10 client.19209 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.720+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55980d764960 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f93e6a00640 1 -- 172.21.15.27:0/3455687669 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f8003dd0 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f93e6a00640 1 -- 172.21.15.27:0/3455687669 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f93ec003930 con 0x55980d661840 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f940eac6740 10 client.19209 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f940eac6740 10 client.19209 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f940eac6740 20 client.19209 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f940eac6740 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d640ea0 0x55980da46ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f940eac6740 4 client.19209 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:38.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55980d9a21e0 con 0x55980d640ea0 2024-09-21T07:25:38.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.721+0000 7f9405e00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d640ea0 0x55980da46ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.722+0000 7f9405e00640 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d640ea0 0x55980da46ea0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6bc02fb87e7fc2 server_cookie=147c1e8fef84dc4a in_seq=0 out_seq=0 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.722+0000 7f93e6a00640 10 client.19209 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f93e6a00640 1 -- 172.21.15.27:0/3455687669 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55980d9a21e0 con 0x55980d640ea0 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f93e6a00640 10 client.19209 handle_command_reply: tid=0 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 client.19209 shutdown 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 msgr2=0x55980da35410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da35410 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f940006a800 tx=0x7f93ec012b60 comp rx=0 tx=0).stop 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 shutdown_connections 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d640ea0 0x55980da46ea0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da40e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da35410 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 --2- 172.21.15.27:0/3455687669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da34ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 >> 172.21.15.27:0/3455687669 conn(0x55980d676910 msgr2=0x55980da2c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.723+0000 7f93e6a00640 0 client.19209 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.724+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 shutdown_connections 2024-09-21T07:25:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.724+0000 7f940eac6740 1 -- 172.21.15.27:0/3455687669 wait complete. 2024-09-21T07:25:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.724+0000 7f940eac6740 20 client.19209 trim_cache size 0 max 16384 2024-09-21T07:25:38.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.742+0000 7f940eac6740 1 Processor -- start 2024-09-21T07:25:38.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.742+0000 7f940eac6740 1 -- start start 2024-09-21T07:25:38.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f940eac6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d659660 0x55980da40d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f940eac6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d661840 0x55980da36bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f940eac6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da2e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f940eac6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b57d0 con 0x55980d659660 2024-09-21T07:25:38.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f940eac6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b30a0 con 0x55980d661840 2024-09-21T07:25:38.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f940eac6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b0f10 con 0x55980d682520 2024-09-21T07:25:38.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f9405e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d659660 0x55980da40d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f9404a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d661840 0x55980da36bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f9405e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d659660 0x55980da40d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35726/0 (socket says 172.21.15.27:35726) 2024-09-21T07:25:38.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f9405e00640 1 -- 172.21.15.27:0/490678106 learned_addr learned my addr 172.21.15.27:0/490678106 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:38.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.743+0000 7f9405e00640 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da2e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f9404a00640 1 -- 172.21.15.27:0/490678106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 msgr2=0x55980da2e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f9404a00640 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da2e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f9404a00640 1 -- 172.21.15.27:0/490678106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d659660 msgr2=0x55980da40d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f9404a00640 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d659660 0x55980da40d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f9404a00640 1 -- 172.21.15.27:0/490678106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980da43a40 con 0x55980d661840 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f9404a00640 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d661840 0x55980da36bb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f93ec003d80 tx=0x7f93ec010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c175f994cad4795b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f93f1c00640 1 -- 172.21.15.27:0/490678106 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec02e070 con 0x55980d661840 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f93f1c00640 1 -- 172.21.15.27:0/490678106 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec00fda0 con 0x55980d661840 2024-09-21T07:25:38.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.744+0000 7f93f1c00640 1 -- 172.21.15.27:0/490678106 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec022040 con 0x55980d661840 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 -- 172.21.15.27:0/490678106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d661840 msgr2=0x55980da36bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d661840 0x55980da36bb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f93ec003d80 tx=0x7f93ec010430 comp rx=0 tx=0).stop 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 -- 172.21.15.27:0/490678106 shutdown_connections 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d682520 0x55980da2e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d661840 0x55980da36bb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 --2- 172.21.15.27:0/490678106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d659660 0x55980da40d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 -- 172.21.15.27:0/490678106 >> 172.21.15.27:0/490678106 conn(0x55980d676910 msgr2=0x55980da271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 -- 172.21.15.27:0/490678106 shutdown_connections 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.745+0000 7f940eac6740 1 -- 172.21.15.27:0/490678106 wait complete. 2024-09-21T07:25:38.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 -- start start 2024-09-21T07:25:38.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 Processor -- start 2024-09-21T07:25:38.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da37ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da38420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da38960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b1560 con 0x55980d661840 2024-09-21T07:25:38.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b29d0 con 0x55980d682520 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.746+0000 7f940eac6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94001b1c30 con 0x55980d659660 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f940eac6740 10 client.0 _ll_register_callbacks cb 0x7f93ec009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da38420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9405e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da37ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da38420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35738/0 (socket says 172.21.15.27:35738) 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 -- 172.21.15.27:0/3864835247 learned_addr learned my addr 172.21.15.27:0/3864835247 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9405e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da37ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56660/0 (socket says 172.21.15.27:56660) 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9405e00640 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da38960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 -- 172.21.15.27:0/3864835247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 msgr2=0x55980da37ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da37ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 -- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 msgr2=0x55980da38960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da38960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9404a00640 1 -- 172.21.15.27:0/3864835247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980da26960 con 0x55980d661840 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.747+0000 7f9405e00640 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da38960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:38.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f9404a00640 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da38420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f93ec003b30 tx=0x7f93ec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ace1b7b97ad290 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f93e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f93e6a00640 1 -- 172.21.15.27:0/3864835247 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec02e040 con 0x55980d661840 2024-09-21T07:25:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f93e6a00640 1 -- 172.21.15.27:0/3864835247 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f93ec00f8f0 con 0x55980d661840 2024-09-21T07:25:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec00fda0 con 0x55980d661840 2024-09-21T07:25:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f93e6a00640 1 -- 172.21.15.27:0/3864835247 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93ec03b050 con 0x55980d661840 2024-09-21T07:25:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f940eac6740 10 client.19215 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.748+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55980dab2b00 con 0x55980d661840 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f93e6a00640 1 -- 172.21.15.27:0/3864835247 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec00fda0 con 0x55980d661840 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f93e6a00640 1 -- 172.21.15.27:0/3864835247 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f93ec031030 con 0x55980d661840 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f940eac6740 10 client.19215 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f940eac6740 10 client.19215 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f940eac6740 20 client.19215 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f940eac6740 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d616250 0x55980dab5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f940eac6740 4 client.19215 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d8798047", "release": true, "await": true}] 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d8798047", "release": true, "await": true}) -- 0x55980d99dc90 con 0x55980d616250 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.749+0000 7f9405e00640 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d616250 0x55980dab5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.750+0000 7f9405e00640 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d616250 0x55980dab5160 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2153f35f97d2b66 server_cookie=677335fcdbef82d8 in_seq=0 out_seq=0 2024-09-21T07:25:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.750+0000 7f93e6a00640 10 client.19215 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:38.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.753+0000 7f93e6a00640 1 -- 172.21.15.27:0/3864835247 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55980d99dc90 con 0x55980d616250 2024-09-21T07:25:38.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.753+0000 7f93e6a00640 10 client.19215 handle_command_reply: tid=0 2024-09-21T07:25:38.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.753+0000 7f940eac6740 1 client.19215 shutdown 2024-09-21T07:25:38.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.753+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 msgr2=0x55980da38420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.753+0000 7f940eac6740 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da38420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f93ec003b30 tx=0x7f93ec0038a0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 shutdown_connections 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55980d616250 0x55980dab5160 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55980d682520 0x55980da38960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55980d661840 0x55980da38420 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 --2- 172.21.15.27:0/3864835247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55980d659660 0x55980da37ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 >> 172.21.15.27:0/3864835247 conn(0x55980d676910 msgr2=0x55980da271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f93e6a00640 0 client.19215 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 shutdown_connections 2024-09-21T07:25:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 1 -- 172.21.15.27:0/3864835247 wait complete. 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.754+0000 7f940eac6740 20 client.19215 trim_cache size 0 max 16384 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 48, 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "d8798047": { 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 48, 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:25:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.8, 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:38.759 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93e80b6680 msgr2=0x7f93e80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93e80b6680 0x7f93e80b8b90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f93f41055e0 tx=0x7f93f4009260 comp rx=0 tx=0).stop 2024-09-21T07:25:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94001047c0 msgr2=0x7f940019c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94001047c0 0x7f940019c4c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005ab0 tx=0x7f93f4003c10 comp rx=0 tx=0).stop 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 shutdown_connections 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f93e80b6680 0x7f93e80b8b90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f940010dc20 0x7f94001a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400105110 0x7f940019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 --2- 172.21.15.27:0/40995533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94001047c0 0x7f940019c4c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 >> 172.21.15.27:0/40995533 conn(0x7f94000fe2c0 msgr2=0x7f94001023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 shutdown_connections 2024-09-21T07:25:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:38.755+0000 7f9406800640 1 -- 172.21.15.27:0/40995533 wait complete. 2024-09-21T07:25:39.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:39 smithi119 ceph-mon[34519]: pgmap v770: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-21T07:25:39.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:39 smithi027 ceph-mon[30779]: pgmap v770: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-21T07:25:39.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:39 smithi153 ceph-mon[34747]: pgmap v770: 129 pgs: 129 active+clean; 2.7 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-21T07:25:39.739 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd8798047', total seconds elapsed: 21.8 2024-09-21T07:25:39.739 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-09-21T07:25:39.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:25:40.150 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 -- 172.21.15.27:0/4237240012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640103b80 msgr2=0x7f9640103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 --2- 172.21.15.27:0/4237240012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640103b80 0x7f9640103f90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9634009860 tx=0x7f963402a340 comp rx=0 tx=0).stop 2024-09-21T07:25:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 -- 172.21.15.27:0/4237240012 shutdown_connections 2024-09-21T07:25:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 --2- 172.21.15.27:0/4237240012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640109e10 0x7f9640110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 --2- 172.21.15.27:0/4237240012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96401094c0 0x7f96401098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 --2- 172.21.15.27:0/4237240012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640103b80 0x7f9640103f90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 -- 172.21.15.27:0/4237240012 >> 172.21.15.27:0/4237240012 conn(0x7f96400ff5d0 msgr2=0x7f9640101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:40.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 -- 172.21.15.27:0/4237240012 shutdown_connections 2024-09-21T07:25:40.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.771+0000 7f9647600640 1 -- 172.21.15.27:0/4237240012 wait complete. 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.772+0000 7f9647600640 1 Processor -- start 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.772+0000 7f9647600640 1 -- start start 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9647600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640103b80 0x7f964019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9647600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96401094c0 0x7f964019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9647600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109e10 0x7f96401a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9647600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9640113ad0 con 0x7f9640109e10 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9647600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9640113950 con 0x7f96401094c0 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9647600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9640113c50 con 0x7f9640103b80 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9645600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109e10 0x7f96401a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9644c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640103b80 0x7f964019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.773+0000 7f9644c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640103b80 0x7f964019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56680/0 (socket says 172.21.15.27:56680) 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f9645600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109e10 0x7f96401a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35762/0 (socket says 172.21.15.27:35762) 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f9644c00640 1 -- 172.21.15.27:0/468431677 learned_addr learned my addr 172.21.15.27:0/468431677 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:40.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f963fe00640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96401094c0 0x7f964019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:40.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f963fe00640 1 -- 172.21.15.27:0/468431677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640103b80 msgr2=0x7f964019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:40.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f963fe00640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640103b80 0x7f964019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:40.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f963fe00640 1 -- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109e10 msgr2=0x7f96401a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f963fe00640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109e10 0x7f96401a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f963fe00640 1 -- 172.21.15.27:0/468431677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96401a2610 con 0x7f96401094c0 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f9644c00640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640103b80 0x7f964019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f9645600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109e10 0x7f96401a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.774+0000 7f963fe00640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96401094c0 0x7f964019cc40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f964006c780 tx=0x7f962c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.775+0000 7f963d600640 1 -- 172.21.15.27:0/468431677 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00ccd0 con 0x7f96401094c0 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.775+0000 7f963d600640 1 -- 172.21.15.27:0/468431677 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c0035f0 con 0x7f96401094c0 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.775+0000 7f963d600640 1 -- 172.21.15.27:0/468431677 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c010690 con 0x7f96401094c0 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.775+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96401a2900 con 0x7f96401094c0 2024-09-21T07:25:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.775+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96401a9060 con 0x7f96401094c0 2024-09-21T07:25:40.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.776+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9640103f90 con 0x7f96401094c0 2024-09-21T07:25:40.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.782+0000 7f963d600640 1 -- 172.21.15.27:0/468431677 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f962c003a80 con 0x7f96401094c0 2024-09-21T07:25:40.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.784+0000 7f963d600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96280b6680 0x7f96280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:40.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.784+0000 7f963d600640 1 -- 172.21.15.27:0/468431677 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f962c0e7db0 con 0x7f96401094c0 2024-09-21T07:25:40.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.785+0000 7f963d600640 1 -- 172.21.15.27:0/468431677 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f962c0e8210 con 0x7f96401094c0 2024-09-21T07:25:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.785+0000 7f9644c00640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96280b6680 0x7f96280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:40.786+0000 7f9644c00640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96280b6680 0x7f96280b8b90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f96340095f0 tx=0x7f9634030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:41.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.021+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f964019d7d0 con 0x7f96401094c0 2024-09-21T07:25:41.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.022+0000 7f963d600640 1 -- 172.21.15.27:0/468431677 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f962c0adf80 con 0x7f96401094c0 2024-09-21T07:25:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:25:41.025 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:25:41.025 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:25:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96280b6680 msgr2=0x7f96280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96280b6680 0x7f96280b8b90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f96340095f0 tx=0x7f9634030340 comp rx=0 tx=0).stop 2024-09-21T07:25:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96401094c0 msgr2=0x7f964019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96401094c0 0x7f964019cc40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f964006c780 tx=0x7f962c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:25:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 shutdown_connections 2024-09-21T07:25:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96280b6680 0x7f96280b8b90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109e10 0x7f96401a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:41.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96401094c0 0x7f964019cc40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:41.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 --2- 172.21.15.27:0/468431677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640103b80 0x7f964019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:41.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 >> 172.21.15.27:0/468431677 conn(0x7f96400ff5d0 msgr2=0x7f9640108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:41.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.025+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 shutdown_connections 2024-09-21T07:25:41.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:41.026+0000 7f9647600640 1 -- 172.21.15.27:0/468431677 wait complete. 2024-09-21T07:25:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:41 smithi027 ceph-mon[30779]: pgmap v771: 129 pgs: 129 active+clean; 2.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:25:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/468431677' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:41.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:41 smithi153 ceph-mon[34747]: pgmap v771: 129 pgs: 129 active+clean; 2.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:25:41.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/468431677' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:41 smithi119 ceph-mon[34519]: pgmap v771: 129 pgs: 129 active+clean; 2.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:25:41.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/468431677' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:41.865 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-09-21T07:25:41.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-09-21T07:25:42.257 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.875+0000 7f7ea0e00640 1 -- 172.21.15.27:0/684972227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c10c010 msgr2=0x7f7e9c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.875+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/684972227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c10c010 0x7f7e9c112840 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009ca0 tx=0x7f7e9402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.876+0000 7f7ea0e00640 1 -- 172.21.15.27:0/684972227 shutdown_connections 2024-09-21T07:25:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.876+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/684972227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c10c010 0x7f7e9c112840 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.876+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/684972227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 0x7f7e9c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.876+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/684972227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e9c105d80 0x7f7e9c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.876+0000 7f7ea0e00640 1 -- 172.21.15.27:0/684972227 >> 172.21.15.27:0/684972227 conn(0x7f7e9c1017d0 msgr2=0x7f7e9c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:42.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.876+0000 7f7ea0e00640 1 -- 172.21.15.27:0/684972227 shutdown_connections 2024-09-21T07:25:42.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.876+0000 7f7ea0e00640 1 -- 172.21.15.27:0/684972227 wait complete. 2024-09-21T07:25:42.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.877+0000 7f7ea0e00640 1 Processor -- start 2024-09-21T07:25:42.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.877+0000 7f7ea0e00640 1 -- start start 2024-09-21T07:25:42.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.877+0000 7f7ea0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c105d80 0x7f7e9c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.877+0000 7f7ea0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 0x7f7e9c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.877+0000 7f7ea0e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e9c10c010 0x7f7e9c1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.877+0000 7f7ea0e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c115d20 con 0x7f7e9c105d80 2024-09-21T07:25:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.877+0000 7f7ea0e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c115ba0 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7ea0e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c115ea0 con 0x7f7e9c10c010 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c105d80 0x7f7e9c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 0x7f7e9c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 0x7f7e9c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38864/0 (socket says 172.21.15.27:38864) 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 -- 172.21.15.27:0/1040562248 learned_addr learned my addr 172.21.15.27:0/1040562248 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e9a800640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e9c10c010 0x7f7e9c1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 -- 172.21.15.27:0/1040562248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e9c10c010 msgr2=0x7f7e9c1a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e9c10c010 0x7f7e9c1a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 -- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c105d80 msgr2=0x7f7e9c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c105d80 0x7f7e9c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.878+0000 7f7e93e00640 1 -- 172.21.15.27:0/1040562248 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c1a2660 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.879+0000 7f7e93e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 0x7f7e9c19cc90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e9f0 tx=0x7f7e8400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:42.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.879+0000 7f7e92a00640 1 -- 172.21.15.27:0/1040562248 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400ccc0 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.879+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c1a2950 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.879+0000 7f7e92a00640 1 -- 172.21.15.27:0/1040562248 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e840108e0 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.879+0000 7f7e92a00640 1 -- 172.21.15.27:0/1040562248 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84018860 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.879+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1a90b0 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.882+0000 7f7ea7a47740 1 Processor -- start 2024-09-21T07:25:42.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.882+0000 7f7ea7a47740 1 -- start start 2024-09-21T07:25:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdfa4840 0x560fbe34c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf7b980 0x560fbe34c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe34cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560fbe2c1ca0 con 0x560fbdfa4840 2024-09-21T07:25:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560fbe2b10d0 con 0x560fbdf7b980 2024-09-21T07:25:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560fbe2b4ee0 con 0x560fbdf83b60 2024-09-21T07:25:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7e9a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdfa4840 0x560fbe34c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf7b980 0x560fbe34c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf7b980 0x560fbe34c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38880/0 (socket says 172.21.15.27:38880) 2024-09-21T07:25:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7e99e00640 1 -- 172.21.15.27:0/3957998488 learned_addr learned my addr 172.21.15.27:0/3957998488 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.883+0000 7f7e92a00640 1 -- 172.21.15.27:0/1040562248 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7e84010430 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e99e00640 1 -- 172.21.15.27:0/3957998488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 msgr2=0x560fbe34cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e93e00640 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe34cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe34cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e99e00640 1 -- 172.21.15.27:0/3957998488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdfa4840 msgr2=0x560fbe34c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdfa4840 0x560fbe34c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e99e00640 1 -- 172.21.15.27:0/3957998488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbe263400 con 0x560fbdf7b980 2024-09-21T07:25:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e9a800640 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdfa4840 0x560fbe34c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e93e00640 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe34cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:25:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf7b980 0x560fbe34c890 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009d20 tx=0x7f7e8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72fa5e87dac0a97f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.884+0000 7f7e8b400640 1 -- 172.21.15.27:0/3957998488 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c01e070 con 0x560fbdf7b980 2024-09-21T07:25:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.885+0000 7f7e8b400640 1 -- 172.21.15.27:0/3957998488 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c019ae0 con 0x560fbdf7b980 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.885+0000 7f7e8b400640 1 -- 172.21.15.27:0/3957998488 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00ea80 con 0x560fbdf7b980 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.885+0000 7f7e92a00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e800b6700 0x7f7e800b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.885+0000 7f7e92a00640 1 -- 172.21.15.27:0/1040562248 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f7e840e7cc0 con 0x7f7e9c10b6c0 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.885+0000 7f7e9a800640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e800b6700 0x7f7e800b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.885+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3957998488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf7b980 msgr2=0x560fbe34c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.885+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf7b980 0x560fbe34c890 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009d20 tx=0x7f7e8c011810 comp rx=0 tx=0).stop 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3957998488 shutdown_connections 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe34cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf7b980 0x560fbe34c890 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3957998488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdfa4840 0x560fbe34c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3957998488 >> 172.21.15.27:0/3957998488 conn(0x560fbdf98990 msgr2=0x560fbe34a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3957998488 shutdown_connections 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3957998488 wait complete. 2024-09-21T07:25:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7e9a800640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e800b6700 0x7f7e800b8c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7e94006010 tx=0x7f7e94005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.886+0000 7f7ea7a47740 1 -- start start 2024-09-21T07:25:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 1 Processor -- start 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf83b60 0x560fbe357f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdfa4840 0x560fbe358460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560fbe1127a0 con 0x560fbdf7b980 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560fbe256a10 con 0x560fbdf83b60 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560fbe35ab30 con 0x560fbdfa4840 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.887+0000 7f7ea7a47740 10 client.0 _ll_register_callbacks cb 0x560fbe266be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf83b60 0x560fbe357f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf83b60 0x560fbe357f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38888/0 (socket says 172.21.15.27:38888) 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 -- 172.21.15.27:0/3233849215 learned_addr learned my addr 172.21.15.27:0/3233849215 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e9a800640 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdfa4840 0x560fbe358460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 -- 172.21.15.27:0/3233849215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdfa4840 msgr2=0x560fbe358460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdfa4840 0x560fbe358460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 -- 172.21.15.27:0/3233849215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 msgr2=0x560fbe3579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e93e00640 1 -- 172.21.15.27:0/3233849215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbe01eb90 con 0x560fbdf83b60 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.888+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7e93e00640 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf83b60 0x560fbe357f00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e840aaa20 tx=0x7f7e840a9f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e282ce17e94953c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7e7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3233849215 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e840acf20 con 0x560fbdf83b60 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3233849215 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f7e840ac480 con 0x560fbdf83b60 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c019ae0 con 0x560fbdf83b60 2024-09-21T07:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3233849215 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e84010720 con 0x560fbdf83b60 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7ea7a47740 10 client.29252 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560fbe3667b0 con 0x560fbdf83b60 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.889+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3233849215 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c019ae0 con 0x560fbdf83b60 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3233849215 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e840b6040 con 0x560fbdf83b60 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7ea7a47740 10 client.29252 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7ea7a47740 10 client.29252 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7ea7a47740 20 client.29252 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf631c0 0x560fbe369d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7ea7a47740 4 client.29252 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fbe2c3bf0 con 0x560fbdf631c0 2024-09-21T07:25:42.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.890+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf631c0 0x560fbe369d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.891+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf631c0 0x560fbe369d20 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af2057202c28b5 server_cookie=a063f40b02c8dde4 in_seq=0 out_seq=0 2024-09-21T07:25:42.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.891+0000 7f7e7fe00640 10 client.29252 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7e7fe00640 1 -- 172.21.15.27:0/3233849215 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560fbe2c3bf0 con 0x560fbdf631c0 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7e7fe00640 10 client.29252 handle_command_reply: tid=0 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 client.29252 shutdown 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf83b60 msgr2=0x560fbe357f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf83b60 0x560fbe357f00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e840aaa20 tx=0x7f7e840a9f30 comp rx=0 tx=0).stop 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 shutdown_connections 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf631c0 0x560fbe369d20 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdfa4840 0x560fbe358460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdf83b60 0x560fbe357f00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3233849215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7e7fe00640 0 client.29252 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.892+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 >> 172.21.15.27:0/3233849215 conn(0x560fbdf98990 msgr2=0x560fbe3489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.893+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 shutdown_connections 2024-09-21T07:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.893+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3233849215 wait complete. 2024-09-21T07:25:42.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.893+0000 7f7ea7a47740 20 client.29252 trim_cache size 0 max 16384 2024-09-21T07:25:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 Processor -- start 2024-09-21T07:25:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 -- start start 2024-09-21T07:25:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe36f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe36fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b5a50 con 0x560fbdf7b980 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b3320 con 0x560fbdfa4840 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.911+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b1190 con 0x560fbdf83b60 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e9a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe36fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35794/0 (socket says 172.21.15.27:35794) 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 -- 172.21.15.27:0/3786903405 learned_addr learned my addr 172.21.15.27:0/3786903405 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e93e00640 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe36f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 -- 172.21.15.27:0/3786903405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 msgr2=0x560fbe36f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe36f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 -- 172.21.15.27:0/3786903405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 msgr2=0x560fbe36fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe36fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 -- 172.21.15.27:0/3786903405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbe345c50 con 0x560fbdf7b980 2024-09-21T07:25:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e93e00640 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe36f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e9a800640 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe36fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.912+0000 7f7e99e00640 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3595f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c01afd0 tx=0x7f7e8c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8d7c4c659bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.913+0000 7f7e8aa00640 1 -- 172.21.15.27:0/3786903405 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c032070 con 0x560fbdf7b980 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.913+0000 7f7e8aa00640 1 -- 172.21.15.27:0/3786903405 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c01f040 con 0x560fbdf7b980 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.913+0000 7f7e8aa00640 1 -- 172.21.15.27:0/3786903405 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c01e040 con 0x560fbdf7b980 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3786903405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 msgr2=0x560fbe3595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3595f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c01afd0 tx=0x7f7e8c0097a0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3786903405 shutdown_connections 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe36f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe36fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/3786903405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe3595f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3786903405 >> 172.21.15.27:0/3786903405 conn(0x560fbdf98990 msgr2=0x560fbe358ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3786903405 shutdown_connections 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 -- 172.21.15.27:0/3786903405 wait complete. 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.914+0000 7f7ea7a47740 1 -- start start 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 1 Processor -- start 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe360580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe35ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe35cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b17e0 con 0x560fbdf7b980 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b2c50 con 0x560fbdfa4840 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b1eb0 con 0x560fbdf83b60 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7ea7a47740 10 client.0 _ll_register_callbacks cb 0x7f7e8400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe360580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7e99e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe360580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35798/0 (socket says 172.21.15.27:35798) 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe35ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.915+0000 7f7e99e00640 1 -- 172.21.15.27:0/359179845 learned_addr learned my addr 172.21.15.27:0/359179845 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:42.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e9a800640 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe35cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e93e00640 1 -- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 msgr2=0x560fbe35cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e93e00640 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe35cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e93e00640 1 -- 172.21.15.27:0/359179845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 msgr2=0x560fbe360580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:42.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e93e00640 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe360580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e93e00640 1 -- 172.21.15.27:0/359179845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbe363d50 con 0x560fbdf83b60 2024-09-21T07:25:42.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e93e00640 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe35ca40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c880 tx=0x7f7e840b6a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49787f06e5d69efc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:42.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:42.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7e7fe00640 1 -- 172.21.15.27:0/359179845 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e840a9650 con 0x560fbdf83b60 2024-09-21T07:25:42.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.916+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c01f040 con 0x560fbdf83b60 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7e7fe00640 1 -- 172.21.15.27:0/359179845 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f7e840b3370 con 0x560fbdf83b60 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7e7fe00640 1 -- 172.21.15.27:0/359179845 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e840b6660 con 0x560fbdf83b60 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 10 client.29218 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560fbe3d5420 con 0x560fbdf83b60 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7e7fe00640 1 -- 172.21.15.27:0/359179845 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c01f040 con 0x560fbdf83b60 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7e7fe00640 1 -- 172.21.15.27:0/359179845 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e840b3570 con 0x560fbdf83b60 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 10 client.29218 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 10 client.29218 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 20 client.29218 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf38570 0x560fbe3d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 4 client.29218 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.917+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x560fbe2bf6a0 con 0x560fbdf38570 2024-09-21T07:25:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.918+0000 7f7e99e00640 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf38570 0x560fbe3d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:42.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.918+0000 7f7e99e00640 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf38570 0x560fbe3d7ca0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9ef972759d766c server_cookie=ec9e1339498b7e52 in_seq=0 out_seq=0 2024-09-21T07:25:42.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.918+0000 7f7e7fe00640 10 client.29218 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:42.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7e7fe00640 1 -- 172.21.15.27:0/359179845 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560fbe2bf6a0 con 0x560fbdf38570 2024-09-21T07:25:42.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7e7fe00640 10 client.29218 handle_command_reply: tid=0 2024-09-21T07:25:42.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 client.29218 shutdown 2024-09-21T07:25:42.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 msgr2=0x560fbe35ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe35ca40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c880 tx=0x7f7e840b6a30 comp rx=0 tx=0).stop 2024-09-21T07:25:42.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 shutdown_connections 2024-09-21T07:25:42.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x560fbdf38570 0x560fbe3d7ca0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fbdf83b60 0x560fbe35ca40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560fbdfa4840 0x560fbe35cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.931+0000 7f7ea7a47740 1 --2- 172.21.15.27:0/359179845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbdf7b980 0x560fbe360580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.932+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 >> 172.21.15.27:0/359179845 conn(0x560fbdf98990 msgr2=0x560fbe3489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.932+0000 7f7e7fe00640 0 client.29218 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.932+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 shutdown_connections 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.932+0000 7f7ea7a47740 1 -- 172.21.15.27:0/359179845 wait complete. 2024-09-21T07:25:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.932+0000 7f7ea7a47740 20 client.29218 trim_cache size 0 max 16384 2024-09-21T07:25:42.936 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:42.936 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 50, 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "4b6fbeca": { 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 50, 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:25:42.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.8, 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:42.939 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:42.939 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e800b6700 msgr2=0x7f7e800b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e800b6700 0x7f7e800b8c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7e94006010 tx=0x7f7e94005d80 comp rx=0 tx=0).stop 2024-09-21T07:25:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 msgr2=0x7f7e9c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 0x7f7e9c19cc90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e9f0 tx=0x7f7e8400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 shutdown_connections 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e9c10c010 0x7f7e9c1a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e800b6700 0x7f7e800b8c10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c10b6c0 0x7f7e9c19cc90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 --2- 172.21.15.27:0/1040562248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e9c105d80 0x7f7e9c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 >> 172.21.15.27:0/1040562248 conn(0x7f7e9c1017d0 msgr2=0x7f7e9c10acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 shutdown_connections 2024-09-21T07:25:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:42.933+0000 7f7ea0e00640 1 -- 172.21.15.27:0/1040562248 wait complete. 2024-09-21T07:25:43.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:43 smithi119 ceph-mon[34519]: pgmap v772: 129 pgs: 129 active+clean; 2.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-09-21T07:25:43.541 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:43 smithi027 ceph-mon[30779]: pgmap v772: 129 pgs: 129 active+clean; 2.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-09-21T07:25:43.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:43 smithi153 ceph-mon[34747]: pgmap v772: 129 pgs: 129 active+clean; 2.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-09-21T07:25:43.741 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4b6fbeca', quiesce took 4.0 seconds. Will release after: 13.8 2024-09-21T07:25:44.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:44 smithi119 ceph-mon[34519]: pgmap v773: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:44.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:44 smithi027 ceph-mon[30779]: pgmap v773: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:44 smithi153 ceph-mon[34747]: pgmap v773: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:46 smithi119 ceph-mon[34519]: pgmap v774: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:47.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:46 smithi027 ceph-mon[30779]: pgmap v774: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:46 smithi153 ceph-mon[34747]: pgmap v774: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2031748048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0109240 msgr2=0x7fdaf0109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2031748048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0109240 0x7fdaf0109650 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009900 tx=0x7fdadc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2031748048 shutdown_connections 2024-09-21T07:25:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2031748048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf0109b90 0x7fdaf01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2031748048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0109240 0x7fdaf0109650 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2031748048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf0103910 0x7fdaf0103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2031748048 >> 172.21.15.27:0/2031748048 conn(0x7fdaf00ff330 msgr2=0x7fdaf0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2031748048 shutdown_connections 2024-09-21T07:25:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.454+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2031748048 wait complete. 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.455+0000 7fdaf5a00640 1 Processor -- start 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.455+0000 7fdaf5a00640 1 -- start start 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.455+0000 7fdaf5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0103910 0x7fdaf019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.455+0000 7fdaf5a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf0109240 0x7fdaf019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.455+0000 7fdaf5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf0109b90 0x7fdaf01a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaf5a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01138a0 con 0x7fdaf0103910 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaf5a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0113720 con 0x7fdaf0109b90 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaf5a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0113a20 con 0x7fdaf0109240 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0103910 0x7fdaf019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0103910 0x7fdaf019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35826/0 (socket says 172.21.15.27:35826) 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaef400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf0109240 0x7fdaf019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaefe00640 1 -- 172.21.15.27:0/2493988454 learned_addr learned my addr 172.21.15.27:0/2493988454 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaf5000640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf0109b90 0x7fdaf01a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaef400640 1 -- 172.21.15.27:0/2493988454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf0109b90 msgr2=0x7fdaf01a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaef400640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf0109b90 0x7fdaf01a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaef400640 1 -- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0103910 msgr2=0x7fdaf019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaef400640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0103910 0x7fdaf019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaef400640 1 -- 172.21.15.27:0/2493988454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf01a23f0 con 0x7fdaf0109240 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.456+0000 7fdaefe00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0103910 0x7fdaf019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.457+0000 7fdaef400640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf0109240 0x7fdaf019ca60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009230 tx=0x7fdadc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.457+0000 7fdaecc00640 1 -- 172.21.15.27:0/2493988454 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc03f070 con 0x7fdaf0109240 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.457+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf01a2680 con 0x7fdaf0109240 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.457+0000 7fdaecc00640 1 -- 172.21.15.27:0/2493988454 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdadc031b50 con 0x7fdaf0109240 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.457+0000 7fdaecc00640 1 -- 172.21.15.27:0/2493988454 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc035030 con 0x7fdaf0109240 2024-09-21T07:25:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.457+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf01a8e70 con 0x7fdaf0109240 2024-09-21T07:25:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 Processor -- start 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 -- start start 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26182e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26183350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe26183890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fe260f86b0 con 0x55fe25dbd4d0 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fe260e7ae0 con 0x55fe25dde1b0 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.460+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fe260eb8f0 con 0x55fe25db52f0 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26182e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26182e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38916/0 (socket says 172.21.15.27:38916) 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 -- 172.21.15.27:0/587480424 learned_addr learned my addr 172.21.15.27:0/587480424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaefe00640 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26183350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 -- 172.21.15.27:0/587480424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 msgr2=0x55fe26183350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26183350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 -- 172.21.15.27:0/587480424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 msgr2=0x55fe26183890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe26183890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 -- 172.21.15.27:0/587480424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe260b4780 con 0x55fe25dde1b0 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaefe00640 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26183350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaecc00640 1 -- 172.21.15.27:0/2493988454 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdadc042070 con 0x7fdaf0109240 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.461+0000 7fdaf5000640 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26182e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdae8011470 tx=0x7fdae8011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490eae00a149df2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.462+0000 7fdae5600640 1 -- 172.21.15.27:0/587480424 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae801e070 con 0x55fe25dde1b0 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.462+0000 7fdae5600640 1 -- 172.21.15.27:0/587480424 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae8003c30 con 0x55fe25dde1b0 2024-09-21T07:25:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.462+0000 7fdae5600640 1 -- 172.21.15.27:0/587480424 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800f900 con 0x55fe25dde1b0 2024-09-21T07:25:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 -- 172.21.15.27:0/587480424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 msgr2=0x55fe26182e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26182e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdae8011470 tx=0x7fdae8011930 comp rx=0 tx=0).stop 2024-09-21T07:25:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 -- 172.21.15.27:0/587480424 shutdown_connections 2024-09-21T07:25:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26183350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe26183890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/587480424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26182e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 -- 172.21.15.27:0/587480424 >> 172.21.15.27:0/587480424 conn(0x55fe25dd2300 msgr2=0x55fe26180d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 -- 172.21.15.27:0/587480424 shutdown_connections 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 -- 172.21.15.27:0/587480424 wait complete. 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdaecc00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdad40b6590 0x7fdad40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdaecc00640 1 -- 172.21.15.27:0/2493988454 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fdadc108060 con 0x7fdaf0109240 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdaefe00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdad40b6590 0x7fdad40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.463+0000 7fdafe1ee740 1 -- start start 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 1 Processor -- start 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdaefe00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdad40b6590 0x7fdad40b8aa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55fe26181a90 tx=0x7fdae0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2618e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe2618ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dde1b0 0x55fe2619a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fe2601fab0 con 0x55fe25dbd4d0 2024-09-21T07:25:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fe2608e570 con 0x55fe25db52f0 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fe26191590 con 0x55fe25dde1b0 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdafe1ee740 10 client.0 _ll_register_callbacks cb 0x55fe2609d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2618e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.464+0000 7fdaef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe2618ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaf5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dde1b0 0x55fe2619a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2618e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38926/0 (socket says 172.21.15.27:38926) 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaefe00640 1 -- 172.21.15.27:0/1654759581 learned_addr learned my addr 172.21.15.27:0/1654759581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaf5000640 1 -- 172.21.15.27:0/1654759581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 msgr2=0x55fe2618e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaf5000640 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2618e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaf5000640 1 -- 172.21.15.27:0/1654759581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 msgr2=0x55fe2618ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaf5000640 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe2618ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaf5000640 1 -- 172.21.15.27:0/1654759581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe25e49790 con 0x55fe25dde1b0 2024-09-21T07:25:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdaf5000640 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dde1b0 0x55fe2619a880 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdae80049d0 tx=0x7fdae8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3183d3dac5850465 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdada000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.465+0000 7fdada000640 1 -- 172.21.15.27:0/1654759581 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae801e040 con 0x55fe25dde1b0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.466+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae8003c30 con 0x55fe25dde1b0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.466+0000 7fdada000640 1 -- 172.21.15.27:0/1654759581 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fdae8009bf0 con 0x55fe25dde1b0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.466+0000 7fdada000640 1 -- 172.21.15.27:0/1654759581 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdae802f050 con 0x55fe25dde1b0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.466+0000 7fdafe1ee740 10 client.29230 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.466+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe25ec2620 con 0x55fe25dde1b0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.466+0000 7fdada000640 1 -- 172.21.15.27:0/1654759581 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae8003c30 con 0x55fe25dde1b0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdada000640 1 -- 172.21.15.27:0/1654759581 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdae8021070 con 0x55fe25dde1b0 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdafe1ee740 10 client.29230 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdafe1ee740 10 client.29230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdafe1ee740 20 client.29230 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d9cd80 0x55fe261a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdafe1ee740 4 client.29230 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe260fa600 con 0x55fe25d9cd80 2024-09-21T07:25:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.467+0000 7fdaefe00640 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d9cd80 0x55fe261a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.468+0000 7fdaefe00640 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d9cd80 0x55fe261a04f0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4f8b094d25c59e server_cookie=eb63e5e8a98a324b in_seq=0 out_seq=0 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.468+0000 7fdada000640 10 client.29230 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdada000640 1 -- 172.21.15.27:0/1654759581 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe260fa600 con 0x55fe25d9cd80 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdada000640 10 client.29230 handle_command_reply: tid=0 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 client.29230 shutdown 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dde1b0 msgr2=0x55fe2619a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dde1b0 0x55fe2619a880 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdae80049d0 tx=0x7fdae8004770 comp rx=0 tx=0).stop 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 shutdown_connections 2024-09-21T07:25:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d9cd80 0x55fe261a04f0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dde1b0 0x55fe2619a880 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe2618ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1654759581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2618e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdada000640 0 client.29230 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 >> 172.21.15.27:0/1654759581 conn(0x55fe25dd2300 msgr2=0x55fe26183e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 shutdown_connections 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1654759581 wait complete. 2024-09-21T07:25:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.469+0000 7fdafe1ee740 20 client.29230 trim_cache size 0 max 16384 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.484+0000 7fdafe1ee740 1 Processor -- start 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdafe1ee740 1 -- start start 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2617fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dbd4d0 0x55fe2619a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dde1b0 0x55fe2618b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b5710 con 0x55fe25dde1b0 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b2fe0 con 0x55fe25db52f0 2024-09-21T07:25:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b0e50 con 0x55fe25dbd4d0 2024-09-21T07:25:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdaf5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dde1b0 0x55fe2618b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdaef400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dbd4d0 0x55fe2619a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2617fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdaef400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dbd4d0 0x55fe2619a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56756/0 (socket says 172.21.15.27:56756) 2024-09-21T07:25:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.485+0000 7fdaef400640 1 -- 172.21.15.27:0/193289028 learned_addr learned my addr 172.21.15.27:0/193289028 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdaef400640 1 -- 172.21.15.27:0/193289028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 msgr2=0x55fe2617fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdaef400640 1 --2- 172.21.15.27:0/193289028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2617fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdaef400640 1 -- 172.21.15.27:0/193289028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dde1b0 msgr2=0x55fe2618b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdaef400640 1 --2- 172.21.15.27:0/193289028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dde1b0 0x55fe2618b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdaef400640 1 -- 172.21.15.27:0/193289028 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe26181600 con 0x55fe25dbd4d0 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdaef400640 1 --2- 172.21.15.27:0/193289028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dbd4d0 0x55fe2619a780 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02a9b0 tx=0x7fdadc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be6e792193ee3a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdae4c00640 1 -- 172.21.15.27:0/193289028 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0d2070 con 0x55fe25dbd4d0 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdae4c00640 1 -- 172.21.15.27:0/193289028 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdadc031d00 con 0x55fe25dbd4d0 2024-09-21T07:25:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.486+0000 7fdae4c00640 1 -- 172.21.15.27:0/193289028 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc049040 con 0x55fe25dbd4d0 2024-09-21T07:25:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 -- 172.21.15.27:0/193289028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dbd4d0 msgr2=0x55fe2619a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/193289028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dbd4d0 0x55fe2619a780 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02a9b0 tx=0x7fdadc009260 comp rx=0 tx=0).stop 2024-09-21T07:25:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 -- 172.21.15.27:0/193289028 shutdown_connections 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/193289028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25dbd4d0 0x55fe2619a780 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/193289028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dde1b0 0x55fe2618b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/193289028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25db52f0 0x55fe2617fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 -- 172.21.15.27:0/193289028 >> 172.21.15.27:0/193289028 conn(0x55fe25dd2300 msgr2=0x55fe26180c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 -- 172.21.15.27:0/193289028 shutdown_connections 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.487+0000 7fdafe1ee740 1 -- 172.21.15.27:0/193289028 wait complete. 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 -- start start 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 Processor -- start 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26193ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe26194530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26208cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b14a0 con 0x55fe25dbd4d0 2024-09-21T07:25:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b2910 con 0x55fe25dde1b0 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b1b70 con 0x55fe25db52f0 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.488+0000 7fdafe1ee740 10 client.0 _ll_register_callbacks cb 0x7fdae801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26193ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26193ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:56760/0 (socket says 172.21.15.27:56760) 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 -- 172.21.15.27:0/1648181160 learned_addr learned my addr 172.21.15.27:0/1648181160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaef400640 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe26194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaf5000640 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26208cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 -- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 msgr2=0x55fe26208cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26208cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 -- 172.21.15.27:0/1648181160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 msgr2=0x55fe26194530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe26194530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 -- 172.21.15.27:0/1648181160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe26209230 con 0x55fe25db52f0 2024-09-21T07:25:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdaefe00640 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26193ff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55fe261a0a30 tx=0x7fdae0012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cabd6e508a387538 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.489+0000 7fdada000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdada000640 1 -- 172.21.15.27:0/1648181160 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae001da00 con 0x55fe25db52f0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe260f60b0 con 0x55fe25db52f0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdada000640 1 -- 172.21.15.27:0/1648181160 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fdae001dba0 con 0x55fe25db52f0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdada000640 1 -- 172.21.15.27:0/1648181160 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdae003e050 con 0x55fe25db52f0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdafe1ee740 10 client.29236 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe2620a480 con 0x55fe25db52f0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdada000640 1 -- 172.21.15.27:0/1648181160 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fe260f60b0 con 0x55fe25db52f0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.490+0000 7fdada000640 1 -- 172.21.15.27:0/1648181160 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdae0030420 con 0x55fe25db52f0 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdafe1ee740 10 client.29236 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdafe1ee740 10 client.29236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdafe1ee740 20 client.29236 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d721c0 0x55fe2620cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdafe1ee740 4 client.29236 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:25:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe260f34e0 con 0x55fe25d721c0 2024-09-21T07:25:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdaef400640 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d721c0 0x55fe2620cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdaef400640 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d721c0 0x55fe2620cc10 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820f5ed7f5fc01be server_cookie=f52881da982c2644 in_seq=0 out_seq=0 2024-09-21T07:25:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.491+0000 7fdada000640 10 client.29236 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdada000640 1 -- 172.21.15.27:0/1648181160 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe260f34e0 con 0x55fe25d721c0 2024-09-21T07:25:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdada000640 10 client.29236 handle_command_reply: tid=0 2024-09-21T07:25:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 client.29236 shutdown 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 msgr2=0x55fe26193ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26193ff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55fe261a0a30 tx=0x7fdae0012f90 comp rx=0 tx=0).stop 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 shutdown_connections 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fe25d721c0 0x55fe2620cc10 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe25dde1b0 0x55fe26208cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdada000640 0 client.29236 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fe25dbd4d0 0x55fe26194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 --2- 172.21.15.27:0/1648181160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fe25db52f0 0x55fe26193ff0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 >> 172.21.15.27:0/1648181160 conn(0x55fe25dd2300 msgr2=0x55fe26180c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 shutdown_connections 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 1 -- 172.21.15.27:0/1648181160 wait complete. 2024-09-21T07:25:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.504+0000 7fdafe1ee740 20 client.29236 trim_cache size 0 max 16384 2024-09-21T07:25:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:25:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:25:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:25:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdad40b6590 msgr2=0x7fdad40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdad40b6590 0x7fdad40b8aa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55fe26181a90 tx=0x7fdae0005f90 comp rx=0 tx=0).stop 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf0109240 msgr2=0x7fdaf019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf0109240 0x7fdaf019ca60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009230 tx=0x7fdadc003b90 comp rx=0 tx=0).stop 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 shutdown_connections 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf0109240 0x7fdaf019ca60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fdad40b6590 0x7fdad40b8aa0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf0109b90 0x7fdaf01a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 --2- 172.21.15.27:0/2493988454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf0103910 0x7fdaf019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 >> 172.21.15.27:0/2493988454 conn(0x7fdaf00ff330 msgr2=0x7fdaf0108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 shutdown_connections 2024-09-21T07:25:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:25:47.505+0000 7fdaf5a00640 1 -- 172.21.15.27:0/2493988454 wait complete. 2024-09-21T07:25:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:48 smithi119 ceph-mon[34519]: pgmap v775: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:48 smithi027 ceph-mon[30779]: pgmap v775: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:48 smithi153 ceph-mon[34747]: pgmap v775: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:50 smithi119 ceph-mon[34519]: pgmap v776: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T07:25:51.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:50 smithi027 ceph-mon[30779]: pgmap v776: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T07:25:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:50 smithi153 ceph-mon[34747]: pgmap v776: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T07:25:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:52 smithi119 ceph-mon[34519]: pgmap v777: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:52 smithi027 ceph-mon[30779]: pgmap v777: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:52 smithi153 ceph-mon[34747]: pgmap v777: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-21T07:25:53.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:25:54.194 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 -- 172.21.15.27:0/791702157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb634109c20 msgr2=0x7fb634110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 --2- 172.21.15.27:0/791702157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb634109c20 0x7fb634110450 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009ca0 tx=0x7fb62c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 -- 172.21.15.27:0/791702157 shutdown_connections 2024-09-21T07:25:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 --2- 172.21.15.27:0/791702157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6341092d0 0x7fb6341096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 --2- 172.21.15.27:0/791702157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb634109c20 0x7fb634110450 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 --2- 172.21.15.27:0/791702157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb634103990 0x7fb634103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 -- 172.21.15.27:0/791702157 >> 172.21.15.27:0/791702157 conn(0x7fb6340ff3e0 msgr2=0x7fb634101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 -- 172.21.15.27:0/791702157 shutdown_connections 2024-09-21T07:25:54.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.815+0000 7fb639a00640 1 -- 172.21.15.27:0/791702157 wait complete. 2024-09-21T07:25:54.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.816+0000 7fb639a00640 1 Processor -- start 2024-09-21T07:25:54.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb639a00640 1 -- start start 2024-09-21T07:25:54.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb639a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb634103990 0x7fb634075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb639a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6341092d0 0x7fb634075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb639a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb634109c20 0x7fb634076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb639a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341138e0 con 0x7fb6341092d0 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb639a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb634113760 con 0x7fb634109c20 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb639a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb634113a60 con 0x7fb634103990 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6341092d0 0x7fb634075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6341092d0 0x7fb634075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45980/0 (socket says 172.21.15.27:45980) 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.817+0000 7fb633400640 1 -- 172.21.15.27:0/1825314174 learned_addr learned my addr 172.21.15.27:0/1825314174 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb633e00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb634103990 0x7fb634075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb639000640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb634109c20 0x7fb634076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb633400640 1 -- 172.21.15.27:0/1825314174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb634103990 msgr2=0x7fb634075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb633400640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb634103990 0x7fb634075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb633400640 1 -- 172.21.15.27:0/1825314174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb634109c20 msgr2=0x7fb634076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb633400640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb634109c20 0x7fb634076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb633400640 1 -- 172.21.15.27:0/1825314174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6341a8570 con 0x7fb6341092d0 2024-09-21T07:25:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb639000640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb634109c20 0x7fb634076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:54.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.818+0000 7fb633400640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6341092d0 0x7fb634075c50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9c0 tx=0x7fb62000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:54.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.819+0000 7fb630c00640 1 -- 172.21.15.27:0/1825314174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000ccc0 con 0x7fb6341092d0 2024-09-21T07:25:54.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.819+0000 7fb630c00640 1 -- 172.21.15.27:0/1825314174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6200054d0 con 0x7fb6341092d0 2024-09-21T07:25:54.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.819+0000 7fb630c00640 1 -- 172.21.15.27:0/1825314174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620010430 con 0x7fb6341092d0 2024-09-21T07:25:54.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.819+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6341a8860 con 0x7fb6341092d0 2024-09-21T07:25:54.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.819+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6341a8d50 con 0x7fb6341092d0 2024-09-21T07:25:54.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.822+0000 7fb641c7c740 1 Processor -- start 2024-09-21T07:25:54.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb641c7c740 1 -- start start 2024-09-21T07:25:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb630c00640 1 -- 172.21.15.27:0/1825314174 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb620005020 con 0x7fb6341092d0 2024-09-21T07:25:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb641c7c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56116662cf70 con 0x5611662e61b0 2024-09-21T07:25:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb641c7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56116661c3a0 con 0x56116630f070 2024-09-21T07:25:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb641c7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5611666201b0 con 0x5611662ee390 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb633e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb633e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45988/0 (socket says 172.21.15.27:45988) 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb633e00640 1 -- 172.21.15.27:0/2129039821 learned_addr learned my addr 172.21.15.27:0/2129039821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb639000640 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.823+0000 7fb633400640 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.824+0000 7fb633e00640 1 -- 172.21.15.27:0/2129039821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 msgr2=0x5611666b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.824+0000 7fb633e00640 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.824+0000 7fb633e00640 1 -- 172.21.15.27:0/2129039821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 msgr2=0x5611666b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.824+0000 7fb633e00640 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.824+0000 7fb633e00640 1 -- 172.21.15.27:0/2129039821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611665e6c90 con 0x5611662e61b0 2024-09-21T07:25:54.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.824+0000 7fb639000640 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:54.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.824+0000 7fb633e00640 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666b6230 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb624009c90 tx=0x7fb624011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29db25333d9bf94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:54.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.825+0000 7fb629600640 1 -- 172.21.15.27:0/2129039821 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624005220 con 0x5611662e61b0 2024-09-21T07:25:54.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.825+0000 7fb629600640 1 -- 172.21.15.27:0/2129039821 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb624004c50 con 0x5611662e61b0 2024-09-21T07:25:54.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.825+0000 7fb629600640 1 -- 172.21.15.27:0/2129039821 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240197a0 con 0x5611662e61b0 2024-09-21T07:25:54.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.825+0000 7fb630c00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6180b6680 0x7fb6180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.825+0000 7fb630c00640 1 -- 172.21.15.27:0/1825314174 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fb6200e7eb0 con 0x7fb6341092d0 2024-09-21T07:25:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.825+0000 7fb639000640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6180b6680 0x7fb6180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 -- 172.21.15.27:0/2129039821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 msgr2=0x5611666b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666b6230 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb624009c90 tx=0x7fb624011810 comp rx=0 tx=0).stop 2024-09-21T07:25:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 -- 172.21.15.27:0/2129039821 shutdown_connections 2024-09-21T07:25:54.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666b6230 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 --2- 172.21.15.27:0/2129039821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 -- 172.21.15.27:0/2129039821 >> 172.21.15.27:0/2129039821 conn(0x561166303460 msgr2=0x5611666b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb639000640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6180b6680 0x7fb6180b8b90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb62c02f500 tx=0x7fb62c02f350 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 -- 172.21.15.27:0/2129039821 shutdown_connections 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.826+0000 7fb641c7c740 1 -- 172.21.15.27:0/2129039821 wait complete. 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.827+0000 7fb641c7c740 1 -- start start 2024-09-21T07:25:54.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.827+0000 7fb641c7c740 1 Processor -- start 2024-09-21T07:25:54.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662ee390 0x5611666c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116630f070 0x5611666cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb641c7c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56116647d700 con 0x5611662e61b0 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb641c7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5611665b7540 con 0x5611662ee390 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb641c7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5611666c44d0 con 0x56116630f070 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb641c7c740 10 client.0 _ll_register_callbacks cb 0x5611665c7430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb633e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb633e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45990/0 (socket says 172.21.15.27:45990) 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb633e00640 1 -- 172.21.15.27:0/3009806204 learned_addr learned my addr 172.21.15.27:0/3009806204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb633400640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662ee390 0x5611666c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.828+0000 7fb639000640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116630f070 0x5611666cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb633e00640 1 -- 172.21.15.27:0/3009806204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116630f070 msgr2=0x5611666cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb633e00640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116630f070 0x5611666cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb633e00640 1 -- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662ee390 msgr2=0x5611666c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb633e00640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662ee390 0x5611666c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb633e00640 1 -- 172.21.15.27:0/3009806204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116637a650 con 0x5611662e61b0 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb633400640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662ee390 0x5611666c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb633e00640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666c1950 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb624009560 tx=0x7fb6240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c453db83ad4ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb61e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb61e000640 1 -- 172.21.15.27:0/3009806204 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624029070 con 0x5611662e61b0 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb61e000640 1 -- 172.21.15.27:0/3009806204 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fb6240194a0 con 0x5611662e61b0 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.829+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5611665b9b50 con 0x5611662e61b0 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.830+0000 7fb61e000640 1 -- 172.21.15.27:0/3009806204 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb624035050 con 0x5611662e61b0 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.830+0000 7fb641c7c740 10 client.19266 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:54.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.830+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611665d4910 con 0x5611662e61b0 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.830+0000 7fb61e000640 1 -- 172.21.15.27:0/3009806204 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5611665b9b50 con 0x5611662e61b0 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.830+0000 7fb61e000640 1 -- 172.21.15.27:0/3009806204 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb624023430 con 0x5611662e61b0 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.830+0000 7fb641c7c740 10 client.19266 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.830+0000 7fb641c7c740 10 client.19266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.831+0000 7fb641c7c740 20 client.19266 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.831+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662cd9f0 0x5611666d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.831+0000 7fb641c7c740 4 client.19266 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:25:54.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.831+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56116662eec0 con 0x5611662cd9f0 2024-09-21T07:25:54.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.831+0000 7fb633400640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662cd9f0 0x5611666d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.831+0000 7fb633400640 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662cd9f0 0x5611666d3720 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29204edd6bdfb28 server_cookie=d6f6c33a693ecb66 in_seq=0 out_seq=0 2024-09-21T07:25:54.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.831+0000 7fb61e000640 10 client.19266 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:54.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.832+0000 7fb61e000640 1 -- 172.21.15.27:0/3009806204 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56116662eec0 con 0x5611662cd9f0 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.832+0000 7fb61e000640 10 client.19266 handle_command_reply: tid=0 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.832+0000 7fb641c7c740 1 client.19266 shutdown 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 msgr2=0x5611666c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666c1950 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb624009560 tx=0x7fb6240048f0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 shutdown_connections 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662cd9f0 0x5611666d3720 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116630f070 0x5611666cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662ee390 0x5611666c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3009806204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662e61b0 0x5611666c1950 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 >> 172.21.15.27:0/3009806204 conn(0x561166303460 msgr2=0x5611666b8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb61e000640 0 client.19266 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 shutdown_connections 2024-09-21T07:25:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 1 -- 172.21.15.27:0/3009806204 wait complete. 2024-09-21T07:25:54.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.833+0000 7fb641c7c740 20 client.19266 trim_cache size 0 max 16384 2024-09-21T07:25:54.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 Processor -- start 2024-09-21T07:25:54.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 -- start start 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662e61b0 0x5611666bb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56116630f070 0x5611666b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b5780 con 0x56116630f070 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b3050 con 0x5611662e61b0 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.852+0000 7fb641c7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b0ec0 con 0x5611662ee390 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb639000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56116630f070 0x5611666b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb639000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56116630f070 0x5611666b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46006/0 (socket says 172.21.15.27:46006) 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb639000640 1 -- 172.21.15.27:0/3067346997 learned_addr learned my addr 172.21.15.27:0/3067346997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53956/0 (socket says 172.21.15.27:53956) 2024-09-21T07:25:54.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633e00640 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662e61b0 0x5611666bb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633400640 1 -- 172.21.15.27:0/3067346997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662e61b0 msgr2=0x5611666bb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633400640 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662e61b0 0x5611666bb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633400640 1 -- 172.21.15.27:0/3067346997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56116630f070 msgr2=0x5611666b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633400640 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56116630f070 0x5611666b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633400640 1 -- 172.21.15.27:0/3067346997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611666c2a10 con 0x5611662ee390 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.853+0000 7fb633e00640 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662e61b0 0x5611666bb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.854+0000 7fb633400640 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b3410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb6200e51f0 tx=0x7fb6200eb960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70f722e91e83a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.854+0000 7fb628c00640 1 -- 172.21.15.27:0/3067346997 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200a9790 con 0x5611662ee390 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.854+0000 7fb628c00640 1 -- 172.21.15.27:0/3067346997 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb620005310 con 0x5611662ee390 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.854+0000 7fb628c00640 1 -- 172.21.15.27:0/3067346997 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200a9060 con 0x5611662ee390 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 -- 172.21.15.27:0/3067346997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 msgr2=0x5611666b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b3410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb6200e51f0 tx=0x7fb6200eb960 comp rx=0 tx=0).stop 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 -- 172.21.15.27:0/3067346997 shutdown_connections 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662ee390 0x5611666b3410 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56116630f070 0x5611666b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 --2- 172.21.15.27:0/3067346997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611662e61b0 0x5611666bb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 -- 172.21.15.27:0/3067346997 >> 172.21.15.27:0/3067346997 conn(0x561166303460 msgr2=0x5611666bae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 -- 172.21.15.27:0/3067346997 shutdown_connections 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 -- 172.21.15.27:0/3067346997 wait complete. 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.855+0000 7fb641c7c740 1 -- start start 2024-09-21T07:25:54.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 1 Processor -- start 2024-09-21T07:25:54.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662e61b0 0x5611666c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662ee390 0x5611666ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b1510 con 0x5611662ee390 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b2980 con 0x56116630f070 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341b1be0 con 0x5611662e61b0 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb641c7c740 10 client.0 _ll_register_callbacks cb 0x7fb624009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662ee390 0x5611666ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb633e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662e61b0 0x5611666c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb633400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662ee390 0x5611666ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46010/0 (socket says 172.21.15.27:46010) 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.856+0000 7fb633400640 1 -- 172.21.15.27:0/4144066487 learned_addr learned my addr 172.21.15.27:0/4144066487 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb639000640 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb633400640 1 -- 172.21.15.27:0/4144066487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662e61b0 msgr2=0x5611666c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb633400640 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662e61b0 0x5611666c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb633400640 1 -- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 msgr2=0x5611666b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb633400640 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb633400640 1 -- 172.21.15.27:0/4144066487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611666b5ba0 con 0x5611662ee390 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb633e00640 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662e61b0 0x5611666c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb633400640 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662ee390 0x5611666ca1b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb6200e5250 tx=0x7fb6200b2300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3367d1bf7c5857e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb61e000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.857+0000 7fb61e000640 1 -- 172.21.15.27:0/4144066487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000ce60 con 0x5611662ee390 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb61e000640 1 -- 172.21.15.27:0/4144066487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fb6200acb50 con 0x5611662ee390 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56116662a970 con 0x5611662ee390 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb61e000640 1 -- 172.21.15.27:0/4144066487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6200ad060 con 0x5611662ee390 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 10 client.19272 fetch_fsmap learned FSMap version 27 2024-09-21T07:25:54.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611666c83a0 con 0x5611662ee390 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb61e000640 1 -- 172.21.15.27:0/4144066487 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56116662a970 con 0x5611662ee390 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb61e000640 1 -- 172.21.15.27:0/4144066487 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb6200ac5c0 con 0x5611662ee390 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 10 client.19272 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 10 client.19272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 20 client.19272 populate_metadata read hostname 'smithi027' 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662a2da0 0x5611667403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.858+0000 7fb641c7c740 4 client.19272 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.859+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561166627da0 con 0x5611662a2da0 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.859+0000 7fb633e00640 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662a2da0 0x5611667403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.859+0000 7fb633e00640 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662a2da0 0x5611667403e0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b365ffac0d53b0ed server_cookie=472c3455318fef2c in_seq=0 out_seq=0 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.859+0000 7fb61e000640 10 client.19272 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb61e000640 1 -- 172.21.15.27:0/4144066487 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x561166627da0 con 0x5611662a2da0 2024-09-21T07:25:54.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb61e000640 10 client.19272 handle_command_reply: tid=0 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 client.19272 shutdown 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662ee390 msgr2=0x5611666ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662ee390 0x5611666ca1b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb6200e5250 tx=0x7fb6200b2300 comp rx=0 tx=0).stop 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 shutdown_connections 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5611662a2da0 0x5611667403e0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56116630f070 0x5611666b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5611662ee390 0x5611666ca1b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 --2- 172.21.15.27:0/4144066487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611662e61b0 0x5611666c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb61e000640 0 client.19272 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 >> 172.21.15.27:0/4144066487 conn(0x561166303460 msgr2=0x5611666b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 shutdown_connections 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.860+0000 7fb641c7c740 1 -- 172.21.15.27:0/4144066487 wait complete. 2024-09-21T07:25:54.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb641c7c740 20 client.19272 trim_cache size 0 max 16384 2024-09-21T07:25:54.868 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:25:54.868 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:25:54.868 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:25:54.868 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:25:54.868 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:25:54.869 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:25:54.869 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.869 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.869 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.869 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.869 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.870 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:25:54.870 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:25:54.870 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.870 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.870 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.871 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.871 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.871 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:25:54.871 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:25:54.871 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.871 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.872 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.872 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.872 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.872 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:25:54.872 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:25:54.872 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:25:54.873 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.874 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.874 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.874 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.874 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.874 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:25:54.874 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:25:54.875 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.876 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.876 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.876 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.876 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.876 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:25:54.876 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:25:54.881 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.883 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:25:54.884 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6180b6680 msgr2=0x7fb6180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6180b6680 0x7fb6180b8b90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb62c02f500 tx=0x7fb62c02f350 comp rx=0 tx=0).stop 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6341092d0 msgr2=0x7fb634075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6341092d0 0x7fb634075c50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9c0 tx=0x7fb62000ee80 comp rx=0 tx=0).stop 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 shutdown_connections 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb6180b6680 0x7fb6180b8b90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb634109c20 0x7fb634076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6341092d0 0x7fb634075c50 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 --2- 172.21.15.27:0/1825314174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb634103990 0x7fb634075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 >> 172.21.15.27:0/1825314174 conn(0x7fb6340ff3e0 msgr2=0x7fb634101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.861+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 shutdown_connections 2024-09-21T07:25:54.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:54.862+0000 7fb639a00640 1 -- 172.21.15.27:0/1825314174 wait complete. 2024-09-21T07:25:54.886 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:54 smithi027 ceph-mon[30779]: pgmap v778: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:25:55.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:54 smithi119 ceph-mon[34519]: pgmap v778: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:25:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:54 smithi153 ceph-mon[34747]: pgmap v778: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-09-21T07:25:55.859 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:25:55.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:25:56.244 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:56.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.878+0000 7f68e7a00640 1 -- 172.21.15.27:0/182611605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e010c010 msgr2=0x7f68e0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:56.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.878+0000 7f68e7a00640 1 --2- 172.21.15.27:0/182611605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e010c010 0x7f68e0112840 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009ca0 tx=0x7f68d802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:25:56.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.879+0000 7f68e7a00640 1 -- 172.21.15.27:0/182611605 shutdown_connections 2024-09-21T07:25:56.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.879+0000 7f68e7a00640 1 --2- 172.21.15.27:0/182611605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e010c010 0x7f68e0112840 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.879+0000 7f68e7a00640 1 --2- 172.21.15.27:0/182611605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68e010b6c0 0x7f68e010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.879+0000 7f68e7a00640 1 --2- 172.21.15.27:0/182611605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68e0105d80 0x7f68e0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.879+0000 7f68e7a00640 1 -- 172.21.15.27:0/182611605 >> 172.21.15.27:0/182611605 conn(0x7f68e0101810 msgr2=0x7f68e0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.879+0000 7f68e7a00640 1 -- 172.21.15.27:0/182611605 shutdown_connections 2024-09-21T07:25:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.879+0000 7f68e7a00640 1 -- 172.21.15.27:0/182611605 wait complete. 2024-09-21T07:25:56.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.880+0000 7f68e7a00640 1 Processor -- start 2024-09-21T07:25:56.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.880+0000 7f68e7a00640 1 -- start start 2024-09-21T07:25:56.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.880+0000 7f68e7a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e0105d80 0x7f68e019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:56.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.880+0000 7f68e7a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68e010b6c0 0x7f68e019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:56.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e7a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68e010c010 0x7f68e01a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:56.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e7a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0115d20 con 0x7f68e0105d80 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e7a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0115ba0 con 0x7f68e010b6c0 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e7a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0115ea0 con 0x7f68e010c010 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e0105d80 0x7f68e019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e0105d80 0x7f68e019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46026/0 (socket says 172.21.15.27:46026) 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 -- 172.21.15.27:0/2311817522 learned_addr learned my addr 172.21.15.27:0/2311817522 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68d7e00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68e010b6c0 0x7f68e019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5a00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68e010c010 0x7f68e01a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 -- 172.21.15.27:0/2311817522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68e010c010 msgr2=0x7f68e01a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68e010c010 0x7f68e01a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 -- 172.21.15.27:0/2311817522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68e010b6c0 msgr2=0x7f68e019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68e010b6c0 0x7f68e019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.881+0000 7f68e5000640 1 -- 172.21.15.27:0/2311817522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e01a25e0 con 0x7f68e0105d80 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.882+0000 7f68e5000640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e0105d80 0x7f68e019c670 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e9f0 tx=0x7f68d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.882+0000 7f68de000640 1 -- 172.21.15.27:0/2311817522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d000cd80 con 0x7f68e0105d80 2024-09-21T07:25:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.882+0000 7f68de000640 1 -- 172.21.15.27:0/2311817522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d00108e0 con 0x7f68e0105d80 2024-09-21T07:25:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.882+0000 7f68de000640 1 -- 172.21.15.27:0/2311817522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0018980 con 0x7f68e0105d80 2024-09-21T07:25:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.882+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e01a28d0 con 0x7f68e0105d80 2024-09-21T07:25:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.882+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01a9030 con 0x7f68e0105d80 2024-09-21T07:25:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.883+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e0106190 con 0x7f68e0105d80 2024-09-21T07:25:56.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.886+0000 7f68de000640 1 -- 172.21.15.27:0/2311817522 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f68d0010430 con 0x7f68e0105d80 2024-09-21T07:25:56.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.887+0000 7f68de000640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68c40b6700 0x7f68c40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:56.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.888+0000 7f68de000640 1 -- 172.21.15.27:0/2311817522 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f68d00e7dd0 con 0x7f68e0105d80 2024-09-21T07:25:56.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.888+0000 7f68d7e00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68c40b6700 0x7f68c40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:56.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.889+0000 7f68d7e00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68c40b6700 0x7f68c40b8c10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f68e01050c0 tx=0x7f68c8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:56.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:56.892+0000 7f68de000640 1 -- 172.21.15.27:0/2311817522 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68d00b22f0 con 0x7f68e0105d80 2024-09-21T07:25:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:56 smithi119 ceph-mon[34519]: pgmap v779: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-21T07:25:57.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:56 smithi027 ceph-mon[30779]: pgmap v779: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-21T07:25:57.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.129+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68e019d7a0 con 0x7f68e0105d80 2024-09-21T07:25:57.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.130+0000 7f68de000640 1 -- 172.21.15.27:0/2311817522 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f68d00b7230 con 0x7f68e0105d80 2024-09-21T07:25:57.132 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:25:57.133 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:25:57.133 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:25:57.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68c40b6700 msgr2=0x7f68c40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:57.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68c40b6700 0x7f68c40b8c10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f68e01050c0 tx=0x7f68c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:25:57.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e0105d80 msgr2=0x7f68e019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e0105d80 0x7f68e019c670 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e9f0 tx=0x7f68d000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 shutdown_connections 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68e010c010 0x7f68e01a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f68c40b6700 0x7f68c40b8c10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68e010b6c0 0x7f68e019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 --2- 172.21.15.27:0/2311817522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68e0105d80 0x7f68e019c670 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.133+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 >> 172.21.15.27:0/2311817522 conn(0x7f68e0101810 msgr2=0x7f68e0108c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.134+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 shutdown_connections 2024-09-21T07:25:57.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:57.134+0000 7f68e7a00640 1 -- 172.21.15.27:0/2311817522 wait complete. 2024-09-21T07:25:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:56 smithi153 ceph-mon[34747]: pgmap v779: 129 pgs: 129 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-21T07:25:57.542 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:25:57.818 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:57 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2311817522' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:57.860 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:25:57.953 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:58.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:57 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2311817522' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:58.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:57 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2311817522' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.750+0000 7fecfa800640 1 -- 172.21.15.27:0/854905054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 msgr2=0x7fecf4112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.750+0000 7fecfa800640 1 --2- 172.21.15.27:0/854905054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf4112850 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fecec009ca0 tx=0x7fecec02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.751+0000 7fecfa800640 1 -- 172.21.15.27:0/854905054 shutdown_connections 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.751+0000 7fecfa800640 1 --2- 172.21.15.27:0/854905054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf4112850 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.751+0000 7fecfa800640 1 --2- 172.21.15.27:0/854905054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf410b6d0 0x7fecf410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.751+0000 7fecfa800640 1 --2- 172.21.15.27:0/854905054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf4105d90 0x7fecf41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.751+0000 7fecfa800640 1 -- 172.21.15.27:0/854905054 >> 172.21.15.27:0/854905054 conn(0x7fecf4101860 msgr2=0x7fecf4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.751+0000 7fecfa800640 1 -- 172.21.15.27:0/854905054 shutdown_connections 2024-09-21T07:25:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.751+0000 7fecfa800640 1 -- 172.21.15.27:0/854905054 wait complete. 2024-09-21T07:25:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.752+0000 7fecfa800640 1 Processor -- start 2024-09-21T07:25:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.752+0000 7fecfa800640 1 -- start start 2024-09-21T07:25:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.752+0000 7fecfa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf4105d90 0x7fecf419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.752+0000 7fecfa800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf410b6d0 0x7fecf419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.752+0000 7fecfa800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf41a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecfa800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4115ce0 con 0x7fecf410c020 2024-09-21T07:25:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecfa800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4115b60 con 0x7fecf4105d90 2024-09-21T07:25:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecfa800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4115e60 con 0x7fecf410b6d0 2024-09-21T07:25:58.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecf3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf41a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:58.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecf3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf41a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53216/0 (socket says 172.21.15.27:53216) 2024-09-21T07:25:58.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf410b6d0 0x7fecf419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:58.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecf3e00640 1 -- 172.21.15.27:0/3592048490 learned_addr learned my addr 172.21.15.27:0/3592048490 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:25:58.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.753+0000 7fecf3400640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf4105d90 0x7fecf419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:58.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf3400640 1 -- 172.21.15.27:0/3592048490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf410b6d0 msgr2=0x7fecf419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf3400640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf410b6d0 0x7fecf419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf3400640 1 -- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 msgr2=0x7fecf41a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf3400640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf41a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf3400640 1 -- 172.21.15.27:0/3592048490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf41a26e0 con 0x7fecf4105d90 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf3e00640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf41a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf2a00640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf410b6d0 0x7fecf419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecf3400640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf4105d90 0x7fecf419c770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fece400e9f0 tx=0x7fece400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecebe00640 1 -- 172.21.15.27:0/3592048490 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece400ccc0 con 0x7fecf4105d90 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf41a29d0 con 0x7fecf4105d90 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecebe00640 1 -- 172.21.15.27:0/3592048490 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece4005020 con 0x7fecf4105d90 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecebe00640 1 -- 172.21.15.27:0/3592048490 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4004a00 con 0x7fecf4105d90 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.754+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41a9130 con 0x7fecf4105d90 2024-09-21T07:25:58.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.756+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf41061a0 con 0x7fecf4105d90 2024-09-21T07:25:58.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.759+0000 7fecebe00640 1 -- 172.21.15.27:0/3592048490 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fece4004ba0 con 0x7fecf4105d90 2024-09-21T07:25:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.760+0000 7fecebe00640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecd80b6680 0x7fecd80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.760+0000 7fecebe00640 1 -- 172.21.15.27:0/3592048490 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fece40e7b60 con 0x7fecf4105d90 2024-09-21T07:25:58.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.760+0000 7fecf2a00640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecd80b6680 0x7fecd80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.761+0000 7fecf2a00640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecd80b6680 0x7fecd80b8b90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fecf4104f60 tx=0x7fece0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:58.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:58.761+0000 7fecebe00640 1 -- 172.21.15.27:0/3592048490 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fece40add30 con 0x7fecf4105d90 2024-09-21T07:25:58.812 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:25:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.017+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecf419d8a0 con 0x7fecf4105d90 2024-09-21T07:25:59.019 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:58 smithi027 ceph-mon[30779]: pgmap v780: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 KiB/s wr, 8 op/s 2024-09-21T07:25:59.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.018+0000 7fecebe00640 1 -- 172.21.15.27:0/3592048490 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fecf419d8a0 con 0x7fecf4105d90 2024-09-21T07:25:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:25:59.020 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:25:59.021 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.020+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecd80b6680 msgr2=0x7fecd80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.020+0000 7fecfa800640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecd80b6680 0x7fecd80b8b90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fecf4104f60 tx=0x7fece0005f90 comp rx=0 tx=0).stop 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.020+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf4105d90 msgr2=0x7fecf419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.020+0000 7fecfa800640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf4105d90 0x7fecf419c770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fece400e9f0 tx=0x7fece400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 shutdown_connections 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecf410b6d0 0x7fecf419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fecd80b6680 0x7fecd80b8b90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf410c020 0x7fecf41a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 --2- 172.21.15.27:0/3592048490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf4105d90 0x7fecf419c770 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 >> 172.21.15.27:0/3592048490 conn(0x7fecf4101860 msgr2=0x7fecf4108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 shutdown_connections 2024-09-21T07:25:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:25:59.021+0000 7fecfa800640 1 -- 172.21.15.27:0/3592048490 wait complete. 2024-09-21T07:25:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:58 smithi119 ceph-mon[34519]: pgmap v780: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 KiB/s wr, 8 op/s 2024-09-21T07:25:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:58 smithi153 ceph-mon[34747]: pgmap v780: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 KiB/s wr, 8 op/s 2024-09-21T07:25:59.888 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 4b6fbeca --release --await' 2024-09-21T07:25:59.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 4b6fbeca --release --await 2024-09-21T07:26:00.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:25:59 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3592048490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:00.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:25:59 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3592048490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:00.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:25:59 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3592048490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:00.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 -- 172.21.15.27:0/1698635627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c109c40 msgr2=0x7f7c4c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 --2- 172.21.15.27:0/1698635627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c109c40 0x7f7c4c110470 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009be0 tx=0x7f7c4402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 -- 172.21.15.27:0/1698635627 shutdown_connections 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 --2- 172.21.15.27:0/1698635627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c109c40 0x7f7c4c110470 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 --2- 172.21.15.27:0/1698635627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c4c1092f0 0x7f7c4c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 --2- 172.21.15.27:0/1698635627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c1039b0 0x7f7c4c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 -- 172.21.15.27:0/1698635627 >> 172.21.15.27:0/1698635627 conn(0x7f7c4c0ff440 msgr2=0x7f7c4c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 -- 172.21.15.27:0/1698635627 shutdown_connections 2024-09-21T07:26:00.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.241+0000 7f7c52c00640 1 -- 172.21.15.27:0/1698635627 wait complete. 2024-09-21T07:26:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.242+0000 7f7c52c00640 1 Processor -- start 2024-09-21T07:26:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c52c00640 1 -- start start 2024-09-21T07:26:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c52c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c4c1039b0 0x7f7c4c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c52c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c1092f0 0x7f7c4c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c52c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c109c40 0x7f7c4c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c52c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c113900 con 0x7f7c4c1039b0 2024-09-21T07:26:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c52c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c113780 con 0x7f7c4c109c40 2024-09-21T07:26:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c52c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c113a80 con 0x7f7c4c1092f0 2024-09-21T07:26:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c50e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c1092f0 0x7f7c4c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.243+0000 7f7c51800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c4c1039b0 0x7f7c4c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c50e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c1092f0 0x7f7c4c19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40702/0 (socket says 172.21.15.27:40702) 2024-09-21T07:26:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c50e00640 1 -- 172.21.15.27:0/4122392810 learned_addr learned my addr 172.21.15.27:0/4122392810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c52200640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c109c40 0x7f7c4c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c52200640 1 -- 172.21.15.27:0/4122392810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c1092f0 msgr2=0x7f7c4c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c52200640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c1092f0 0x7f7c4c19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c52200640 1 -- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c4c1039b0 msgr2=0x7f7c4c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c52200640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c4c1039b0 0x7f7c4c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.244+0000 7f7c52200640 1 -- 172.21.15.27:0/4122392810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c1a0260 con 0x7f7c4c109c40 2024-09-21T07:26:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.245+0000 7f7c52200640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c109c40 0x7f7c4c19fb30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009b30 tx=0x7f7c44003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.245+0000 7f7c4a000640 1 -- 172.21.15.27:0/4122392810 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44041020 con 0x7f7c4c109c40 2024-09-21T07:26:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.246+0000 7f7c4a000640 1 -- 172.21.15.27:0/4122392810 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c440318f0 con 0x7f7c4c109c40 2024-09-21T07:26:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.246+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c1a04f0 con 0x7f7c4c109c40 2024-09-21T07:26:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.246+0000 7f7c4a000640 1 -- 172.21.15.27:0/4122392810 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44036040 con 0x7f7c4c109c40 2024-09-21T07:26:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.246+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1a6cb0 con 0x7f7c4c109c40 2024-09-21T07:26:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.251+0000 7f7c4a000640 1 -- 172.21.15.27:0/4122392810 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7c44044020 con 0x7f7c4c109c40 2024-09-21T07:26:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.252+0000 7f7c5ae59740 1 Processor -- start 2024-09-21T07:26:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.252+0000 7f7c5ae59740 1 -- start start 2024-09-21T07:26:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.252+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e83054140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c801b0 0x556e83054680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e83054bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556e82fc6fb0 con 0x556e82c88390 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556e82fb63e0 con 0x556e82ca9070 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e82fba1f0 con 0x556e82c801b0 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c51800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c801b0 0x556e83054680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c51800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c801b0 0x556e83054680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40704/0 (socket says 172.21.15.27:40704) 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c51800640 1 -- 172.21.15.27:0/3594490273 learned_addr learned my addr 172.21.15.27:0/3594490273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c50e00640 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e83054bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c4a000640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c340b6700 0x7f7c340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c4a000640 1 -- 172.21.15.27:0/4122392810 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f7c44108620 con 0x7f7c4c109c40 2024-09-21T07:26:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c51800640 1 -- 172.21.15.27:0/3594490273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 msgr2=0x556e83054140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:26:00.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c51800640 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e83054140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.253+0000 7f7c51800640 1 -- 172.21.15.27:0/3594490273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 msgr2=0x556e83054bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.254+0000 7f7c51800640 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e83054bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.254+0000 7f7c51800640 1 -- 172.21.15.27:0/3594490273 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e82f80cd0 con 0x556e82c801b0 2024-09-21T07:26:00.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.254+0000 7f7c50e00640 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e83054bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.254+0000 7f7c52200640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c340b6700 0x7f7c340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.254+0000 7f7c51800640 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c801b0 0x556e83054680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009cd0 tx=0x7f7c40011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb7fb8ec6eaae90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.255+0000 7f7c3ea00640 1 -- 172.21.15.27:0/3594490273 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4001e070 con 0x556e82c801b0 2024-09-21T07:26:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.255+0000 7f7c3ea00640 1 -- 172.21.15.27:0/3594490273 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c40019ae0 con 0x556e82c801b0 2024-09-21T07:26:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.255+0000 7f7c52200640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c340b6700 0x7f7c340b8c10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c441059c0 tx=0x7f7c44009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.255+0000 7f7c3ea00640 1 -- 172.21.15.27:0/3594490273 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000ea20 con 0x556e82c801b0 2024-09-21T07:26:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.256+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3594490273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c801b0 msgr2=0x556e83054680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.256+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c801b0 0x556e83054680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009cd0 tx=0x7f7c40011810 comp rx=0 tx=0).stop 2024-09-21T07:26:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3594490273 shutdown_connections 2024-09-21T07:26:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c801b0 0x556e83054680 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e83054bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3594490273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e83054140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3594490273 >> 172.21.15.27:0/3594490273 conn(0x556e82c9d460 msgr2=0x556e83051fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3594490273 shutdown_connections 2024-09-21T07:26:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3594490273 wait complete. 2024-09-21T07:26:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.257+0000 7f7c5ae59740 1 -- start start 2024-09-21T07:26:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 1 Processor -- start 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82c801b0 0x556e8305fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e830602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82ca9070 0x556e8306bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556e82e17700 con 0x556e82c88390 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556e82f51580 con 0x556e82c801b0 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e83062920 con 0x556e82ca9070 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c5ae59740 10 client.0 _ll_register_callbacks cb 0x556e82f61470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.258+0000 7f7c50e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82ca9070 0x556e8306bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82ca9070 0x556e8306bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40710/0 (socket says 172.21.15.27:40710) 2024-09-21T07:26:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 -- 172.21.15.27:0/1744262169 learned_addr learned my addr 172.21.15.27:0/1744262169 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82c801b0 0x556e8305fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c51800640 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e830602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 -- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82c801b0 msgr2=0x556e8305fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82c801b0 0x556e8305fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 -- 172.21.15.27:0/1744262169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 msgr2=0x556e830602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e830602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 -- 172.21.15.27:0/1744262169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e82d14650 con 0x556e82ca9070 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c50e00640 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82ca9070 0x556e8306bc70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7c38011d60 tx=0x7f7c3800a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61860a95c861998 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.259+0000 7f7c33400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.260+0000 7f7c33400640 1 -- 172.21.15.27:0/1744262169 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800ee50 con 0x556e82ca9070 2024-09-21T07:26:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.260+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e82f53b90 con 0x556e82ca9070 2024-09-21T07:26:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.260+0000 7f7c33400640 1 -- 172.21.15.27:0/1744262169 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f7c38011110 con 0x556e82ca9070 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.260+0000 7f7c33400640 1 -- 172.21.15.27:0/1744262169 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c38027050 con 0x556e82ca9070 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.260+0000 7f7c5ae59740 10 client.29278 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.260+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e82f6e950 con 0x556e82ca9070 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.260+0000 7f7c33400640 1 -- 172.21.15.27:0/1744262169 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556e82f53b90 con 0x556e82ca9070 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c33400640 1 -- 172.21.15.27:0/1744262169 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7c3801f3f0 con 0x556e82ca9070 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c5ae59740 10 client.29278 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c5ae59740 10 client.29278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c5ae59740 20 client.29278 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c679f0 0x556e83071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c5ae59740 4 client.29278 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e82fc8f00 con 0x556e82c679f0 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.261+0000 7f7c51800640 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c679f0 0x556e83071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.262+0000 7f7c51800640 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c679f0 0x556e83071d80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbbc5324c2ff1785 server_cookie=30d6694355d8ad68 in_seq=0 out_seq=0 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.262+0000 7f7c33400640 10 client.29278 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c33400640 1 -- 172.21.15.27:0/1744262169 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e82fc8f00 con 0x556e82c679f0 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c33400640 10 client.29278 handle_command_reply: tid=0 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 client.29278 shutdown 2024-09-21T07:26:00.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82ca9070 msgr2=0x556e8306bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82ca9070 0x556e8306bc70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7c38011d60 tx=0x7f7c3800a640 comp rx=0 tx=0).stop 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 shutdown_connections 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c679f0 0x556e83071d80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82ca9070 0x556e8306bc70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c88390 0x556e830602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/1744262169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82c801b0 0x556e8305fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 >> 172.21.15.27:0/1744262169 conn(0x556e82c9d460 msgr2=0x556e830572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c33400640 0 client.29278 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 shutdown_connections 2024-09-21T07:26:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.263+0000 7f7c5ae59740 1 -- 172.21.15.27:0/1744262169 wait complete. 2024-09-21T07:26:00.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.264+0000 7f7c5ae59740 20 client.29278 trim_cache size 0 max 16384 2024-09-21T07:26:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.281+0000 7f7c5ae59740 1 Processor -- start 2024-09-21T07:26:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.281+0000 7f7c5ae59740 1 -- start start 2024-09-21T07:26:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83057240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83057780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e8306bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1b13d0 con 0x556e82c801b0 2024-09-21T07:26:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1aeca0 con 0x556e82ca9070 2024-09-21T07:26:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1acb10 con 0x556e82c88390 2024-09-21T07:26:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c50e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83057240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c51800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83057780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c50e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83057240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53266/0 (socket says 172.21.15.27:53266) 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c50e00640 1 -- 172.21.15.27:0/3626330777 learned_addr learned my addr 172.21.15.27:0/3626330777 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.282+0000 7f7c50e00640 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e8306bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.283+0000 7f7c51800640 1 -- 172.21.15.27:0/3626330777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 msgr2=0x556e8306bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.283+0000 7f7c51800640 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e8306bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.283+0000 7f7c51800640 1 -- 172.21.15.27:0/3626330777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 msgr2=0x556e83057240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.283+0000 7f7c51800640 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83057240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.283+0000 7f7c51800640 1 -- 172.21.15.27:0/3626330777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e83072350 con 0x556e82c88390 2024-09-21T07:26:00.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.283+0000 7f7c51800640 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83057780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3876aa530bb0ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.284+0000 7f7c3e000640 1 -- 172.21.15.27:0/3626330777 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4001e040 con 0x556e82c88390 2024-09-21T07:26:00.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.284+0000 7f7c3e000640 1 -- 172.21.15.27:0/3626330777 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c400054f0 con 0x556e82c88390 2024-09-21T07:26:00.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.284+0000 7f7c3e000640 1 -- 172.21.15.27:0/3626330777 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4001e040 con 0x556e82c88390 2024-09-21T07:26:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3626330777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 msgr2=0x556e83057780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83057780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40009890 comp rx=0 tx=0).stop 2024-09-21T07:26:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3626330777 shutdown_connections 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83057780 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e8306bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3626330777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83057240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3626330777 >> 172.21.15.27:0/3626330777 conn(0x556e82c9d460 msgr2=0x556e8305e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.285+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3626330777 shutdown_connections 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.286+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3626330777 wait complete. 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.286+0000 7f7c5ae59740 1 -- start start 2024-09-21T07:26:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.286+0000 7f7c5ae59740 1 Processor -- start 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83052110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83067bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c5ae59740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e830682b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1ad160 con 0x556e82c801b0 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1ae5d0 con 0x556e82ca9070 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c5ae59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1ad830 con 0x556e82c88390 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c5ae59740 10 client.0 _ll_register_callbacks cb 0x7f7c380119a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c51800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83067bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.287+0000 7f7c51800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83067bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40718/0 (socket says 172.21.15.27:40718) 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.288+0000 7f7c51800640 1 -- 172.21.15.27:0/3928159151 learned_addr learned my addr 172.21.15.27:0/3928159151 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.288+0000 7f7c50e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e830682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.288+0000 7f7c50e00640 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83052110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.288+0000 7f7c50e00640 1 -- 172.21.15.27:0/3928159151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 msgr2=0x556e83067bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.288+0000 7f7c50e00640 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83067bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.288+0000 7f7c50e00640 1 -- 172.21.15.27:0/3928159151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 msgr2=0x556e830682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.289+0000 7f7c50e00640 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e830682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.289+0000 7f7c50e00640 1 -- 172.21.15.27:0/3928159151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8304d610 con 0x556e82c801b0 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.289+0000 7f7c50e00640 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83052110 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7c380118e0 tx=0x7f7c3801ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880c260a8afcb5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.289+0000 7f7c33400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.289+0000 7f7c33400640 1 -- 172.21.15.27:0/3928159151 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3801fbf0 con 0x556e82c801b0 2024-09-21T07:26:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.289+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c400054f0 con 0x556e82c801b0 2024-09-21T07:26:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.290+0000 7f7c33400640 1 -- 172.21.15.27:0/3928159151 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f7c3801a070 con 0x556e82c801b0 2024-09-21T07:26:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.290+0000 7f7c33400640 1 -- 172.21.15.27:0/3928159151 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c3800f420 con 0x556e82c801b0 2024-09-21T07:26:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.290+0000 7f7c5ae59740 10 client.19299 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.290+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e830e1110 con 0x556e82c801b0 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.290+0000 7f7c33400640 1 -- 172.21.15.27:0/3928159151 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c400054f0 con 0x556e82c801b0 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c33400640 1 -- 172.21.15.27:0/3928159151 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7c3803a470 con 0x556e82c801b0 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c5ae59740 10 client.19299 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c5ae59740 10 client.19299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c5ae59740 20 client.19299 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c3cda0 0x556e830e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c5ae59740 4 client.19299 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:26:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e83059da0 con 0x556e82c3cda0 2024-09-21T07:26:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.291+0000 7f7c51800640 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c3cda0 0x556e830e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.292+0000 7f7c33400640 10 client.19299 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.292+0000 7f7c51800640 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c3cda0 0x556e830e3780 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edf2301ab6a33d5e server_cookie=8957a47d0c83db8e in_seq=0 out_seq=0 2024-09-21T07:26:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.292+0000 7f7c33400640 1 -- 172.21.15.27:0/3928159151 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e83059da0 con 0x556e82c3cda0 2024-09-21T07:26:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.292+0000 7f7c33400640 10 client.19299 handle_command_reply: tid=0 2024-09-21T07:26:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.292+0000 7f7c5ae59740 1 client.19299 shutdown 2024-09-21T07:26:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 msgr2=0x556e83052110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83052110 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7c380118e0 tx=0x7f7c3801ff00 comp rx=0 tx=0).stop 2024-09-21T07:26:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 shutdown_connections 2024-09-21T07:26:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x556e82c3cda0 0x556e830e3780 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e82c88390 0x556e83067bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e82ca9070 0x556e830682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c33400640 0 client.19299 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 --2- 172.21.15.27:0/3928159151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e82c801b0 0x556e83052110 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.293+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 >> 172.21.15.27:0/3928159151 conn(0x556e82c9d460 msgr2=0x556e8305e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.294+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 shutdown_connections 2024-09-21T07:26:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.294+0000 7f7c5ae59740 1 -- 172.21.15.27:0/3928159151 wait complete. 2024-09-21T07:26:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.294+0000 7f7c5ae59740 20 client.19299 trim_cache size 0 max 16384 2024-09-21T07:26:00.297 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:26:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c340b6700 msgr2=0x7f7c340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c340b6700 0x7f7c340b8c10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c441059c0 tx=0x7f7c44009500 comp rx=0 tx=0).stop 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c109c40 msgr2=0x7f7c4c19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c109c40 0x7f7c4c19fb30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009b30 tx=0x7f7c44003c30 comp rx=0 tx=0).stop 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 shutdown_connections 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c1092f0 0x7f7c4c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7c340b6700 0x7f7c340b8c10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c109c40 0x7f7c4c19fb30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 --2- 172.21.15.27:0/4122392810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c4c1039b0 0x7f7c4c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 >> 172.21.15.27:0/4122392810 conn(0x7f7c4c0ff440 msgr2=0x7f7c4c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 shutdown_connections 2024-09-21T07:26:00.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:00.295+0000 7f7c52c00640 1 -- 172.21.15.27:0/4122392810 wait complete. 2024-09-21T07:26:00.314 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.705+0000 7fec5b200640 1 -- 172.21.15.27:0/965618031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c103910 msgr2=0x7fec5c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.705+0000 7fec5b200640 1 --2- 172.21.15.27:0/965618031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c103910 0x7fec5c103d20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009900 tx=0x7fec4c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.705+0000 7fec5b200640 1 -- 172.21.15.27:0/965618031 shutdown_connections 2024-09-21T07:26:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.706+0000 7fec5b200640 1 --2- 172.21.15.27:0/965618031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec5c109b90 0x7fec5c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.706+0000 7fec5b200640 1 --2- 172.21.15.27:0/965618031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c109240 0x7fec5c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.706+0000 7fec5b200640 1 --2- 172.21.15.27:0/965618031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c103910 0x7fec5c103d20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.706+0000 7fec5b200640 1 -- 172.21.15.27:0/965618031 >> 172.21.15.27:0/965618031 conn(0x7fec5c0ff330 msgr2=0x7fec5c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.706+0000 7fec5b200640 1 -- 172.21.15.27:0/965618031 shutdown_connections 2024-09-21T07:26:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.706+0000 7fec5b200640 1 -- 172.21.15.27:0/965618031 wait complete. 2024-09-21T07:26:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 Processor -- start 2024-09-21T07:26:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 -- start start 2024-09-21T07:26:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c103910 0x7fec5c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec5c109240 0x7fec5c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c109b90 0x7fec5c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c113850 con 0x7fec5c109240 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1136d0 con 0x7fec5c103910 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec5b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1139d0 con 0x7fec5c109b90 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec59400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec5c109240 0x7fec5c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.707+0000 7fec59e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c103910 0x7fec5c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c103910 0x7fec5c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38964/0 (socket says 172.21.15.27:38964) 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 -- 172.21.15.27:0/4026607790 learned_addr learned my addr 172.21.15.27:0/4026607790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec5a800640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c109b90 0x7fec5c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 -- 172.21.15.27:0/4026607790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c109b90 msgr2=0x7fec5c1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c109b90 0x7fec5c1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 -- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec5c109240 msgr2=0x7fec5c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec5c109240 0x7fec5c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 -- 172.21.15.27:0/4026607790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c1a2460 con 0x7fec5c103910 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59400640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec5c109240 0x7fec5c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.708+0000 7fec59e00640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c103910 0x7fec5c19c530 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005940 tx=0x7fec4c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.709+0000 7fec52a00640 1 -- 172.21.15.27:0/4026607790 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c03f030 con 0x7fec5c103910 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.709+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c1a26f0 con 0x7fec5c103910 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.709+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c1a8ee0 con 0x7fec5c103910 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.709+0000 7fec52a00640 1 -- 172.21.15.27:0/4026607790 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4c0044c0 con 0x7fec5c103910 2024-09-21T07:26:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.709+0000 7fec52a00640 1 -- 172.21.15.27:0/4026607790 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c049060 con 0x7fec5c103910 2024-09-21T07:26:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 Processor -- start 2024-09-21T07:26:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 -- start start 2024-09-21T07:26:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a086e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a087350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cc14d0 0x56359a087890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563599ffc6b0 con 0x563599cc14d0 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563599febae0 con 0x563599ce21b0 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.712+0000 7fec67bfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563599fef8f0 con 0x563599cb92f0 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a086e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a086e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38970/0 (socket says 172.21.15.27:38970) 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 -- 172.21.15.27:0/3669126272 learned_addr learned my addr 172.21.15.27:0/3669126272 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec59400640 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cc14d0 0x56359a087890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 -- 172.21.15.27:0/3669126272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 msgr2=0x56359a087350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:26:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a087350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 -- 172.21.15.27:0/3669126272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cc14d0 msgr2=0x56359a087890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cc14d0 0x56359a087890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec52a00640 1 -- 172.21.15.27:0/4026607790 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fec4c042030 con 0x7fec5c103910 2024-09-21T07:26:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 -- 172.21.15.27:0/3669126272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563599fb8780 con 0x563599ce21b0 2024-09-21T07:26:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.713+0000 7fec5a800640 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a086e10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fec54009f90 tx=0x7fec540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1363a5e918b485e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.714+0000 7fec47000640 1 -- 172.21.15.27:0/3669126272 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5401e070 con 0x563599ce21b0 2024-09-21T07:26:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.714+0000 7fec47000640 1 -- 172.21.15.27:0/3669126272 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec54003c30 con 0x563599ce21b0 2024-09-21T07:26:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.714+0000 7fec47000640 1 -- 172.21.15.27:0/3669126272 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5400f860 con 0x563599ce21b0 2024-09-21T07:26:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.714+0000 7fec67bfd740 1 -- 172.21.15.27:0/3669126272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 msgr2=0x56359a086e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.714+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a086e10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fec54009f90 tx=0x7fec540118d0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 -- 172.21.15.27:0/3669126272 shutdown_connections 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a087350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cc14d0 0x56359a087890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3669126272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a086e10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 -- 172.21.15.27:0/3669126272 >> 172.21.15.27:0/3669126272 conn(0x563599cd6300 msgr2=0x56359a084d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 -- 172.21.15.27:0/3669126272 shutdown_connections 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 -- 172.21.15.27:0/3669126272 wait complete. 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec52a00640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec400b6590 0x7fec400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec52a00640 1 -- 172.21.15.27:0/4026607790 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fec4c107b50 con 0x7fec5c103910 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec59400640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec400b6590 0x7fec400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.715+0000 7fec67bfd740 1 -- start start 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 1 Processor -- start 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec59400640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec400b6590 0x7fec400b8aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fec5c06aa40 tx=0x7fec48005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a092490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a0929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599ce21b0 0x56359a09e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563599f23ab0 con 0x563599ce21b0 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563599f92570 con 0x563599cc14d0 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56359a095590 con 0x563599cb92f0 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec67bfd740 10 client.0 _ll_register_callbacks cb 0x563599fa1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec5a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599ce21b0 0x56359a09e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec5a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599ce21b0 0x56359a09e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53304/0 (socket says 172.21.15.27:53304) 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec5a800640 1 -- 172.21.15.27:0/3997663768 learned_addr learned my addr 172.21.15.27:0/3997663768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.716+0000 7fec59400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a0929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec59e00640 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a092490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec5a800640 1 -- 172.21.15.27:0/3997663768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 msgr2=0x56359a092490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec5a800640 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a092490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec5a800640 1 -- 172.21.15.27:0/3997663768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 msgr2=0x56359a0929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec5a800640 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a0929d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec5a800640 1 -- 172.21.15.27:0/3997663768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563599d4d790 con 0x563599ce21b0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec59e00640 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a092490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec5a800640 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599ce21b0 0x56359a09e8b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec54009aa0 tx=0x7fec540048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da4b8167bfdc95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec3b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.717+0000 7fec3b400640 1 -- 172.21.15.27:0/3997663768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5401e040 con 0x563599ce21b0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec54003c30 con 0x563599ce21b0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec3b400640 1 -- 172.21.15.27:0/3997663768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fec54004050 con 0x563599ce21b0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec3b400640 1 -- 172.21.15.27:0/3997663768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec54030050 con 0x563599ce21b0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec67bfd740 10 client.19311 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563599dc6620 con 0x563599ce21b0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec3b400640 1 -- 172.21.15.27:0/3997663768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec54003c30 con 0x563599ce21b0 2024-09-21T07:26:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec3b400640 1 -- 172.21.15.27:0/3997663768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fec54021070 con 0x563599ce21b0 2024-09-21T07:26:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec67bfd740 10 client.19311 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec67bfd740 10 client.19311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.718+0000 7fec67bfd740 20 client.19311 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.719+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599ca0d80 0x56359a0a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.719+0000 7fec67bfd740 4 client.19311 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.719+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563599ffe600 con 0x563599ca0d80 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.719+0000 7fec59e00640 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599ca0d80 0x56359a0a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.719+0000 7fec59e00640 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599ca0d80 0x56359a0a44e0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a234072702b5b9d8 server_cookie=446ee5ef87b95286 in_seq=0 out_seq=0 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.719+0000 7fec3b400640 10 client.19311 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec3b400640 1 -- 172.21.15.27:0/3997663768 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563599ffe600 con 0x563599ca0d80 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec3b400640 10 client.19311 handle_command_reply: tid=0 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 client.19311 shutdown 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599ce21b0 msgr2=0x56359a09e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599ce21b0 0x56359a09e8b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec54009aa0 tx=0x7fec540048d0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 shutdown_connections 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599ca0d80 0x56359a0a44e0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599ce21b0 0x56359a09e8b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a0929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 --2- 172.21.15.27:0/3997663768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cb92f0 0x56359a092490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec3b400640 0 client.19311 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.720+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 >> 172.21.15.27:0/3997663768 conn(0x563599cd6300 msgr2=0x56359a087e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.721+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 shutdown_connections 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.721+0000 7fec67bfd740 1 -- 172.21.15.27:0/3997663768 wait complete. 2024-09-21T07:26:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.721+0000 7fec67bfd740 20 client.19311 trim_cache size 0 max 16384 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 Processor -- start 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 -- start start 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a092970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cc14d0 0x56359a0940c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a09e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1b57b0 con 0x563599cb92f0 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1b3080 con 0x563599ce21b0 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.736+0000 7fec67bfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1b0ef0 con 0x563599cc14d0 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a092970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a092970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53314/0 (socket says 172.21.15.27:53314) 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59e00640 1 -- 172.21.15.27:0/911884434 learned_addr learned my addr 172.21.15.27:0/911884434 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59400640 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cc14d0 0x56359a0940c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec5a800640 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a09e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59400640 1 -- 172.21.15.27:0/911884434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 msgr2=0x56359a09e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59400640 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a09e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59400640 1 -- 172.21.15.27:0/911884434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 msgr2=0x56359a092970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59400640 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a092970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.737+0000 7fec59400640 1 -- 172.21.15.27:0/911884434 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56359a085600 con 0x563599cc14d0 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.738+0000 7fec59400640 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cc14d0 0x56359a0940c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fec5c06c400 tx=0x7fec4801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca88209434134c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.738+0000 7fec46600640 1 -- 172.21.15.27:0/911884434 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4802a480 con 0x563599cc14d0 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.738+0000 7fec46600640 1 -- 172.21.15.27:0/911884434 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4802aad0 con 0x563599cc14d0 2024-09-21T07:26:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.738+0000 7fec46600640 1 -- 172.21.15.27:0/911884434 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4801fbe0 con 0x563599cc14d0 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 -- 172.21.15.27:0/911884434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cc14d0 msgr2=0x56359a0940c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cc14d0 0x56359a0940c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fec5c06c400 tx=0x7fec4801a000 comp rx=0 tx=0).stop 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 -- 172.21.15.27:0/911884434 shutdown_connections 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599cc14d0 0x56359a0940c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599ce21b0 0x56359a09e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 --2- 172.21.15.27:0/911884434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a092970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 -- 172.21.15.27:0/911884434 >> 172.21.15.27:0/911884434 conn(0x563599cd6300 msgr2=0x56359a0939b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 -- 172.21.15.27:0/911884434 shutdown_connections 2024-09-21T07:26:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 -- 172.21.15.27:0/911884434 wait complete. 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.739+0000 7fec67bfd740 1 -- start start 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 1 Processor -- start 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a097520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a097a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599ce21b0 0x56359a10cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1b1540 con 0x563599cb92f0 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1b29b0 con 0x563599cc14d0 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1b1c10 con 0x563599ce21b0 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec67bfd740 10 client.0 _ll_register_callbacks cb 0x7fec5401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a097520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec59e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a097520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53318/0 (socket says 172.21.15.27:53318) 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.740+0000 7fec59e00640 1 -- 172.21.15.27:0/2101808826 learned_addr learned my addr 172.21.15.27:0/2101808826 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec5a800640 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599ce21b0 0x56359a10cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec59400640 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec59e00640 1 -- 172.21.15.27:0/2101808826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599ce21b0 msgr2=0x56359a10cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec59e00640 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599ce21b0 0x56359a10cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec59e00640 1 -- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 msgr2=0x56359a097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec59e00640 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a097a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec59e00640 1 -- 172.21.15.27:0/2101808826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56359a08ca10 con 0x563599cb92f0 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec59e00640 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a097520 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fec4c032af0 tx=0x7fec4c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c569637e01551 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec3b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec3b400640 1 -- 172.21.15.27:0/2101808826 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c035040 con 0x563599cb92f0 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.741+0000 7fec3b400640 1 -- 172.21.15.27:0/2101808826 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7fec4c0d6240 con 0x563599cb92f0 2024-09-21T07:26:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec4802aad0 con 0x563599cb92f0 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec3b400640 1 -- 172.21.15.27:0/2101808826 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec4c0d2070 con 0x563599cb92f0 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 10 client.19317 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56359a10e3e0 con 0x563599cb92f0 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec3b400640 1 -- 172.21.15.27:0/2101808826 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4802aad0 con 0x563599cb92f0 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec3b400640 1 -- 172.21.15.27:0/2101808826 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fec4c04c060 con 0x563599cb92f0 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 10 client.19317 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 10 client.19317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 20 client.19317 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599c761c0 0x56359a110bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 4 client.19317 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.742+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563599ffa0b0 con 0x563599c761c0 2024-09-21T07:26:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.743+0000 7fec59400640 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599c761c0 0x56359a110bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.743+0000 7fec59400640 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599c761c0 0x56359a110bf0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e9a5375d169393 server_cookie=a7d70b1b8edb01cd in_seq=0 out_seq=0 2024-09-21T07:26:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.743+0000 7fec3b400640 10 client.19317 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec3b400640 1 -- 172.21.15.27:0/2101808826 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563599ffa0b0 con 0x563599c761c0 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec3b400640 10 client.19317 handle_command_reply: tid=0 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 client.19317 shutdown 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 msgr2=0x56359a097520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a097520 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fec4c032af0 tx=0x7fec4c009560 comp rx=0 tx=0).stop 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 shutdown_connections 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563599c761c0 0x56359a110bf0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563599ce21b0 0x56359a10cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563599cc14d0 0x56359a097a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec3b400640 0 client.19317 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 --2- 172.21.15.27:0/2101808826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563599cb92f0 0x56359a097520 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 >> 172.21.15.27:0/2101808826 conn(0x563599cd6300 msgr2=0x56359a087cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 shutdown_connections 2024-09-21T07:26:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.754+0000 7fec67bfd740 1 -- 172.21.15.27:0/2101808826 wait complete. 2024-09-21T07:26:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec67bfd740 20 client.19317 trim_cache size 0 max 16384 2024-09-21T07:26:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:26:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:26:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:26:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:26:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec400b6590 msgr2=0x7fec400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec400b6590 0x7fec400b8aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fec5c06aa40 tx=0x7fec48005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c103910 msgr2=0x7fec5c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c103910 0x7fec5c19c530 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005940 tx=0x7fec4c0040c0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 shutdown_connections 2024-09-21T07:26:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec5c109b90 0x7fec5c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec400b6590 0x7fec400b8aa0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec5c109240 0x7fec5c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 --2- 172.21.15.27:0/4026607790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec5c103910 0x7fec5c19c530 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.755+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 >> 172.21.15.27:0/4026607790 conn(0x7fec5c0ff330 msgr2=0x7fec5c108950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.756+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 shutdown_connections 2024-09-21T07:26:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:00.756+0000 7fec5b200640 1 -- 172.21.15.27:0/4026607790 wait complete. 2024-09-21T07:26:00.906 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:00 smithi027 ceph-mon[30779]: pgmap v781: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-21T07:26:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:00 smithi119 ceph-mon[34519]: pgmap v781: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-21T07:26:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:00 smithi153 ceph-mon[34747]: pgmap v781: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-21T07:26:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 -- 172.21.15.27:0/1124137847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c105d80 msgr2=0x7f3e4c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 --2- 172.21.15.27:0/1124137847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c105d80 0x7f3e4c106190 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009900 tx=0x7f3e3c02a400 comp rx=0 tx=0).stop 2024-09-21T07:26:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 -- 172.21.15.27:0/1124137847 shutdown_connections 2024-09-21T07:26:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 --2- 172.21.15.27:0/1124137847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e4c10b6c0 0x7f3e4c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 --2- 172.21.15.27:0/1124137847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c10c010 0x7f3e4c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 --2- 172.21.15.27:0/1124137847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c105d80 0x7f3e4c106190 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 -- 172.21.15.27:0/1124137847 >> 172.21.15.27:0/1124137847 conn(0x7f3e4c101810 msgr2=0x7f3e4c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 -- 172.21.15.27:0/1124137847 shutdown_connections 2024-09-21T07:26:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.179+0000 7f3e52600640 1 -- 172.21.15.27:0/1124137847 wait complete. 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.180+0000 7f3e52600640 1 Processor -- start 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.180+0000 7f3e52600640 1 -- start start 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e52600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e4c105d80 0x7f3e4c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e52600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c10b6c0 0x7f3e4c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e52600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c10c010 0x7f3e4c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e52600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c115d20 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e52600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c115ba0 con 0x7f3e4c10c010 2024-09-21T07:26:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e52600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c115ea0 con 0x7f3e4c105d80 2024-09-21T07:26:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e4aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c10b6c0 0x7f3e4c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e4aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c10b6c0 0x7f3e4c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53326/0 (socket says 172.21.15.27:53326) 2024-09-21T07:26:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e4aa00640 1 -- 172.21.15.27:0/2783223328 learned_addr learned my addr 172.21.15.27:0/2783223328 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e4b400640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e4c105d80 0x7f3e4c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.181+0000 7f3e4be00640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c10c010 0x7f3e4c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e4aa00640 1 -- 172.21.15.27:0/2783223328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e4c105d80 msgr2=0x7f3e4c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e4c105d80 0x7f3e4c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e4aa00640 1 -- 172.21.15.27:0/2783223328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c10c010 msgr2=0x7f3e4c1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c10c010 0x7f3e4c1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e4aa00640 1 -- 172.21.15.27:0/2783223328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c1a2640 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c10b6c0 0x7f3e4c19cc70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800d950 tx=0x7f3e3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e43e00640 1 -- 172.21.15.27:0/2783223328 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38017070 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c1a2930 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e43e00640 1 -- 172.21.15.27:0/2783223328 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e38009c30 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.182+0000 7f3e43e00640 1 -- 172.21.15.27:0/2783223328 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380047a0 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.183+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c1a9090 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 Processor -- start 2024-09-21T07:26:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 -- start start 2024-09-21T07:26:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8f1520 0x55a4abc981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abc98700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e43e00640 1 -- 172.21.15.27:0/2783223328 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3e38004020 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8d0840 0x55a4abc98c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a4abc0f290 con 0x55a4ab8f1520 2024-09-21T07:26:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a4abbfe6c0 con 0x55a4ab8d0840 2024-09-21T07:26:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.186+0000 7f3e5927e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4abc024d0 con 0x55a4ab8c8660 2024-09-21T07:26:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8f1520 0x55a4abc981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abc98700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abc98700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40784/0 (socket says 172.21.15.27:40784) 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4b400640 1 -- 172.21.15.27:0/1464905982 learned_addr learned my addr 172.21.15.27:0/1464905982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4be00640 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8f1520 0x55a4abc981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53330/0 (socket says 172.21.15.27:53330) 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8d0840 0x55a4abc98c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4be00640 1 -- 172.21.15.27:0/1464905982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 msgr2=0x55a4abc98700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4be00640 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abc98700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4be00640 1 -- 172.21.15.27:0/1464905982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8d0840 msgr2=0x55a4abc98c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4be00640 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8d0840 0x55a4abc98c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4be00640 1 -- 172.21.15.27:0/1464905982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4abb8b460 con 0x55a4ab8f1520 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.187+0000 7f3e4b400640 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abc98700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.188+0000 7f3e4be00640 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8f1520 0x55a4abc981c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009eb0 tx=0x7f3e44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a50a7d8213613cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.188+0000 7f3e40c00640 1 -- 172.21.15.27:0/1464905982 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44016070 con 0x55a4ab8f1520 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.188+0000 7f3e40c00640 1 -- 172.21.15.27:0/1464905982 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44004d20 con 0x55a4ab8f1520 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.188+0000 7f3e40c00640 1 -- 172.21.15.27:0/1464905982 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44012780 con 0x55a4ab8f1520 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.188+0000 7f3e43e00640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e300b6700 0x7f3e300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e43e00640 1 -- 172.21.15.27:0/2783223328 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f3e380e7710 con 0x7f3e4c10b6c0 2024-09-21T07:26:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e4b400640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e300b6700 0x7f3e300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e5927e740 1 -- 172.21.15.27:0/1464905982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8f1520 msgr2=0x55a4abc981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8f1520 0x55a4abc981c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009eb0 tx=0x7f3e44010740 comp rx=0 tx=0).stop 2024-09-21T07:26:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e5927e740 1 -- 172.21.15.27:0/1464905982 shutdown_connections 2024-09-21T07:26:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abc98700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8d0840 0x55a4abc98c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1464905982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8f1520 0x55a4abc981c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.189+0000 7f3e5927e740 1 -- 172.21.15.27:0/1464905982 >> 172.21.15.27:0/1464905982 conn(0x55a4ab8e5910 msgr2=0x55a4abc960b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.190+0000 7f3e5927e740 1 -- 172.21.15.27:0/1464905982 shutdown_connections 2024-09-21T07:26:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.190+0000 7f3e5927e740 1 -- 172.21.15.27:0/1464905982 wait complete. 2024-09-21T07:26:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.190+0000 7f3e4b400640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e300b6700 0x7f3e300b8c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55a4abc96e40 tx=0x7f3e3c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.190+0000 7f3e5927e740 1 -- start start 2024-09-21T07:26:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 1 Processor -- start 2024-09-21T07:26:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abca3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abca4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abcafcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a4aba5fbb0 con 0x55a4ab8c8660 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a4abb996c0 con 0x55a4ab8f1520 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4abca69a0 con 0x55a4ab8d0840 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.191+0000 7f3e5927e740 10 client.0 _ll_register_callbacks cb 0x55a4abbb4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abca3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abcafcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abca4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abcafcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39058/0 (socket says 172.21.15.27:39058) 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abca3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53336/0 (socket says 172.21.15.27:53336) 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4be00640 1 -- 172.21.15.27:0/2218707997 learned_addr learned my addr 172.21.15.27:0/2218707997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4aa00640 1 -- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 msgr2=0x55a4abcafcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abcafcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4aa00640 1 -- 172.21.15.27:0/2218707997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 msgr2=0x55a4abca3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abca3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4aa00640 1 -- 172.21.15.27:0/2218707997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ab95cb00 con 0x55a4ab8d0840 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4be00640 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abcafcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abca4320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3e380e4a50 tx=0x7f3e380a9670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a7d5c14ce83797 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.192+0000 7f3e35600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.193+0000 7f3e35600640 1 -- 172.21.15.27:0/2218707997 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380a8ff0 con 0x55a4ab8d0840 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.193+0000 7f3e35600640 1 -- 172.21.15.27:0/2218707997 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f3e380e7490 con 0x55a4ab8d0840 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.193+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e44004d20 con 0x55a4ab8d0840 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.193+0000 7f3e35600640 1 -- 172.21.15.27:0/2218707997 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e380e4ee0 con 0x55a4ab8d0840 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.193+0000 7f3e5927e740 10 client.29311 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.193+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4ab9d3960 con 0x55a4ab8d0840 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.193+0000 7f3e35600640 1 -- 172.21.15.27:0/2218707997 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44004d20 con 0x55a4ab8d0840 2024-09-21T07:26:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e35600640 1 -- 172.21.15.27:0/2218707997 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3e380e7b00 con 0x55a4ab8d0840 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e5927e740 10 client.29311 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e5927e740 10 client.29311 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e5927e740 20 client.29311 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e5927e740 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab8afea0 0x55a4abcb5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e5927e740 4 client.29311 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4abc111e0 con 0x55a4ab8afea0 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.194+0000 7f3e4b400640 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab8afea0 0x55a4abcb5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.195+0000 7f3e4b400640 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab8afea0 0x55a4abcb5cb0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7307d2744122d04 server_cookie=4318382149ba67f5 in_seq=0 out_seq=0 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.195+0000 7f3e35600640 10 client.29311 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e35600640 1 -- 172.21.15.27:0/2218707997 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a4abc111e0 con 0x55a4ab8afea0 2024-09-21T07:26:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e35600640 10 client.29311 handle_command_reply: tid=0 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 client.29311 shutdown 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 msgr2=0x55a4abca4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abca4320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3e380e4a50 tx=0x7f3e380a9670 comp rx=0 tx=0).stop 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 shutdown_connections 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab8afea0 0x55a4abcb5cb0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abca4320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abcafcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e35600640 0 client.29311 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 --2- 172.21.15.27:0/2218707997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abca3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 >> 172.21.15.27:0/2218707997 conn(0x55a4ab8e5910 msgr2=0x55a4abc9b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 shutdown_connections 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 1 -- 172.21.15.27:0/2218707997 wait complete. 2024-09-21T07:26:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.196+0000 7f3e5927e740 20 client.29311 trim_cache size 0 max 16384 2024-09-21T07:26:01.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.214+0000 7f3e5927e740 1 Processor -- start 2024-09-21T07:26:01.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e5927e740 1 -- start start 2024-09-21T07:26:01.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abcafbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abc99bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abc9d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e5927e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1b59f0 con 0x55a4ab8c8660 2024-09-21T07:26:01.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e5927e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1b32c0 con 0x55a4ab8f1520 2024-09-21T07:26:01.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e5927e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1b1130 con 0x55a4ab8d0840 2024-09-21T07:26:01.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abcafbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abcafbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53342/0 (socket says 172.21.15.27:53342) 2024-09-21T07:26:01.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e4b400640 1 -- 172.21.15.27:0/1185951312 learned_addr learned my addr 172.21.15.27:0/1185951312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:01.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e4be00640 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abc9d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.215+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abc99bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e4b400640 1 -- 172.21.15.27:0/1185951312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 msgr2=0x55a4abc99bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e4b400640 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abc99bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e4b400640 1 -- 172.21.15.27:0/1185951312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 msgr2=0x55a4abc9d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e4b400640 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abc9d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e4b400640 1 -- 172.21.15.27:0/1185951312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4abc943b0 con 0x55a4ab8c8660 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abc99bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e4b400640 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abcafbf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c02a9f0 tx=0x7f3e3c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419a3fbe6535f0cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.216+0000 7f3e37e00640 1 -- 172.21.15.27:0/1185951312 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c059070 con 0x55a4ab8c8660 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e37e00640 1 -- 172.21.15.27:0/1185951312 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3c0035f0 con 0x55a4ab8c8660 2024-09-21T07:26:01.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e37e00640 1 -- 172.21.15.27:0/1185951312 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c035040 con 0x55a4ab8c8660 2024-09-21T07:26:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e5927e740 1 -- 172.21.15.27:0/1185951312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 msgr2=0x55a4abcafbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abcafbf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c02a9f0 tx=0x7f3e3c03f000 comp rx=0 tx=0).stop 2024-09-21T07:26:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e5927e740 1 -- 172.21.15.27:0/1185951312 shutdown_connections 2024-09-21T07:26:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8d0840 0x55a4abc99bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abc9d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e5927e740 1 --2- 172.21.15.27:0/1185951312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8c8660 0x55a4abcafbf0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.217+0000 7f3e5927e740 1 -- 172.21.15.27:0/1185951312 >> 172.21.15.27:0/1185951312 conn(0x55a4ab8e5910 msgr2=0x55a4abc9b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.218+0000 7f3e5927e740 1 -- 172.21.15.27:0/1185951312 shutdown_connections 2024-09-21T07:26:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.218+0000 7f3e5927e740 1 -- 172.21.15.27:0/1185951312 wait complete. 2024-09-21T07:26:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.218+0000 7f3e5927e740 1 -- start start 2024-09-21T07:26:01.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.218+0000 7f3e5927e740 1 Processor -- start 2024-09-21T07:26:01.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abcac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8d0840 0x55a4abca6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e5927e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abca7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e5927e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1b1780 con 0x55a4ab8d0840 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e5927e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1b2bf0 con 0x55a4ab8f1520 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e5927e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c1b1e50 con 0x55a4ab8c8660 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e5927e740 10 client.0 _ll_register_callbacks cb 0x7f3e380046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e4aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8d0840 0x55a4abca6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e4aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8d0840 0x55a4abca6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53352/0 (socket says 172.21.15.27:53352) 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e4aa00640 1 -- 172.21.15.27:0/4031476527 learned_addr learned my addr 172.21.15.27:0/4031476527 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e4b400640 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abcac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.219+0000 7f3e4be00640 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abca7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e4aa00640 1 -- 172.21.15.27:0/4031476527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 msgr2=0x55a4abcac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abcac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e4aa00640 1 -- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 msgr2=0x55a4abca7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abca7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e4aa00640 1 -- 172.21.15.27:0/4031476527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4abca76e0 con 0x55a4ab8d0840 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8d0840 0x55a4abca6c10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005fd0 tx=0x7f3e380a95d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78bce85b91b6043 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e35600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e35600640 1 -- 172.21.15.27:0/4031476527 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380ac8c0 con 0x55a4ab8d0840 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e35600640 1 -- 172.21.15.27:0/4031476527 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7f3e380e7490 con 0x55a4ab8d0840 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.220+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3c0035f0 con 0x55a4ab8d0840 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.221+0000 7f3e35600640 1 -- 172.21.15.27:0/4031476527 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e380acaa0 con 0x55a4ab8d0840 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.221+0000 7f3e5927e740 10 client.19335 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.221+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4abd21440 con 0x55a4ab8d0840 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.221+0000 7f3e35600640 1 -- 172.21.15.27:0/4031476527 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3c0035f0 con 0x55a4ab8d0840 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e35600640 1 -- 172.21.15.27:0/4031476527 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3e380981a0 con 0x55a4ab8d0840 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e5927e740 10 client.19335 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e5927e740 10 client.19335 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e5927e740 20 client.19335 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e5927e740 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab885250 0x55a4abd23990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e5927e740 4 client.19335 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4b6fbeca", "release": true, "await": true}] 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4b6fbeca", "release": true, "await": true}) -- 0x55a4abc0cc90 con 0x55a4ab885250 2024-09-21T07:26:01.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.222+0000 7f3e4b400640 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab885250 0x55a4abd23990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:01.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.223+0000 7f3e4b400640 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab885250 0x55a4abd23990 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a88f47380e7f48 server_cookie=18f68aff6065479d in_seq=0 out_seq=0 2024-09-21T07:26:01.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.223+0000 7f3e35600640 10 client.19335 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e35600640 1 -- 172.21.15.27:0/4031476527 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a4abc0cc90 con 0x55a4ab885250 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e35600640 10 client.19335 handle_command_reply: tid=0 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 client.19335 shutdown 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8d0840 msgr2=0x55a4abca6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8d0840 0x55a4abca6c10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005fd0 tx=0x7f3e380a95d0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 shutdown_connections 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a4ab885250 0x55a4abd23990 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4ab8f1520 0x55a4abca7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a4ab8d0840 0x55a4abca6c10 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e35600640 0 client.19335 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:01.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 --2- 172.21.15.27:0/4031476527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4ab8c8660 0x55a4abcac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 >> 172.21.15.27:0/4031476527 conn(0x55a4ab8e5910 msgr2=0x55a4abc9b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 shutdown_connections 2024-09-21T07:26:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 1 -- 172.21.15.27:0/4031476527 wait complete. 2024-09-21T07:26:01.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.226+0000 7f3e5927e740 20 client.19335 trim_cache size 0 max 16384 2024-09-21T07:26:01.229 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 52, 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "4b6fbeca": { 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 52, 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:01.230 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.8, 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:01.231 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:26:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e300b6700 msgr2=0x7f3e300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e300b6700 0x7f3e300b8c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55a4abc96e40 tx=0x7f3e3c005a20 comp rx=0 tx=0).stop 2024-09-21T07:26:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c10b6c0 msgr2=0x7f3e4c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c10b6c0 0x7f3e4c19cc70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800d950 tx=0x7f3e3800de10 comp rx=0 tx=0).stop 2024-09-21T07:26:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 shutdown_connections 2024-09-21T07:26:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3e300b6700 0x7f3e300b8c10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c10c010 0x7f3e4c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e4c10b6c0 0x7f3e4c19cc70 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 --2- 172.21.15.27:0/2783223328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e4c105d80 0x7f3e4c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 >> 172.21.15.27:0/2783223328 conn(0x7f3e4c101810 msgr2=0x7f3e4c10ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.227+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 shutdown_connections 2024-09-21T07:26:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:01.228+0000 7f3e52600640 1 -- 172.21.15.27:0/2783223328 wait complete. 2024-09-21T07:26:01.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:26:02.336 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4b6fbeca', total seconds elapsed: 22.6 2024-09-21T07:26:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:02.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:02 smithi027 ceph-mon[30779]: pgmap v782: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-21T07:26:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.012+0000 7efc31600640 1 -- 172.21.15.27:0/3015339568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 msgr2=0x7efc2c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.012+0000 7efc31600640 1 --2- 172.21.15.27:0/3015339568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c105fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009900 tx=0x7efc1c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.013+0000 7efc31600640 1 -- 172.21.15.27:0/3015339568 shutdown_connections 2024-09-21T07:26:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.013+0000 7efc31600640 1 --2- 172.21.15.27:0/3015339568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc2c10be40 0x7efc2c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.013+0000 7efc31600640 1 --2- 172.21.15.27:0/3015339568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc2c10b4f0 0x7efc2c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.013+0000 7efc31600640 1 --2- 172.21.15.27:0/3015339568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c105fc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.013+0000 7efc31600640 1 -- 172.21.15.27:0/3015339568 >> 172.21.15.27:0/3015339568 conn(0x7efc2c101680 msgr2=0x7efc2c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:03.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.013+0000 7efc31600640 1 -- 172.21.15.27:0/3015339568 shutdown_connections 2024-09-21T07:26:03.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.013+0000 7efc31600640 1 -- 172.21.15.27:0/3015339568 wait complete. 2024-09-21T07:26:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 Processor -- start 2024-09-21T07:26:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 -- start start 2024-09-21T07:26:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc2c10b4f0 0x7efc2c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc2c10be40 0x7efc2c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c115b50 con 0x7efc2c10b4f0 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1159d0 con 0x7efc2c10be40 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.014+0000 7efc31600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c115cd0 con 0x7efc2c105bb0 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40828/0 (socket says 172.21.15.27:40828) 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 -- 172.21.15.27:0/1538429696 learned_addr learned my addr 172.21.15.27:0/1538429696 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2b400640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc2c10b4f0 0x7efc2c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc30c00640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc2c10be40 0x7efc2c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 -- 172.21.15.27:0/1538429696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc2c10be40 msgr2=0x7efc2c1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc2c10be40 0x7efc2c1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 -- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc2c10b4f0 msgr2=0x7efc2c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc2c10b4f0 0x7efc2c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.015+0000 7efc2be00640 1 -- 172.21.15.27:0/1538429696 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2c1a2590 con 0x7efc2c105bb0 2024-09-21T07:26:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.016+0000 7efc2be00640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c19c620 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005ab0 tx=0x7efc1c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.016+0000 7efc28c00640 1 -- 172.21.15.27:0/1538429696 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c03f070 con 0x7efc2c105bb0 2024-09-21T07:26:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.016+0000 7efc28c00640 1 -- 172.21.15.27:0/1538429696 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1c031b60 con 0x7efc2c105bb0 2024-09-21T07:26:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.016+0000 7efc28c00640 1 -- 172.21.15.27:0/1538429696 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c035030 con 0x7efc2c105bb0 2024-09-21T07:26:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.016+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2c1a2820 con 0x7efc2c105bb0 2024-09-21T07:26:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.016+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2c1a8fe0 con 0x7efc2c105bb0 2024-09-21T07:26:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.019+0000 7efc3985e740 1 Processor -- start 2024-09-21T07:26:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc3985e740 1 -- start start 2024-09-21T07:26:03.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc3985e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533add070 0x55d533e83f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc3985e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533ab41b0 0x55d533e844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc3985e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc3985e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d533dfafb0 con 0x55d533abc390 2024-09-21T07:26:03.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc3985e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d533dea3e0 con 0x55d533ab41b0 2024-09-21T07:26:03.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc28c00640 1 -- 172.21.15.27:0/1538429696 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efc1c042070 con 0x7efc2c105bb0 2024-09-21T07:26:03.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc3985e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d533dee1f0 con 0x55d533add070 2024-09-21T07:26:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc30c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533add070 0x55d533e83f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.020+0000 7efc2b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533add070 0x55d533e83f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:40830/0 (socket says 172.21.15.27:40830) 2024-09-21T07:26:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 -- 172.21.15.27:0/3173078670 learned_addr learned my addr 172.21.15.27:0/3173078670 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc2be00640 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533ab41b0 0x55d533e844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 -- 172.21.15.27:0/3173078670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533ab41b0 msgr2=0x55d533e844b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533ab41b0 0x55d533e844b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 -- 172.21.15.27:0/3173078670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 msgr2=0x55d533e849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 -- 172.21.15.27:0/3173078670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d533db4cd0 con 0x55d533add070 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc2b400640 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.021+0000 7efc30c00640 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533add070 0x55d533e83f70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efc24009f50 tx=0x7efc240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e1f71948c36d718 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.022+0000 7efc21200640 1 -- 172.21.15.27:0/3173078670 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24004d20 con 0x55d533add070 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.022+0000 7efc21200640 1 -- 172.21.15.27:0/3173078670 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc24004050 con 0x55d533add070 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.022+0000 7efc21200640 1 -- 172.21.15.27:0/3173078670 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24005280 con 0x55d533add070 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.022+0000 7efc28c00640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efc100b6700 0x7efc100b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc28c00640 1 -- 172.21.15.27:0/1538429696 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7efc1c108170 con 0x7efc2c105bb0 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc2be00640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efc100b6700 0x7efc100b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 -- 172.21.15.27:0/3173078670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533add070 msgr2=0x55d533e83f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533add070 0x55d533e83f70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efc24009f50 tx=0x7efc240118d0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 -- 172.21.15.27:0/3173078670 shutdown_connections 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533ab41b0 0x55d533e844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 --2- 172.21.15.27:0/3173078670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533add070 0x55d533e83f70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 -- 172.21.15.27:0/3173078670 >> 172.21.15.27:0/3173078670 conn(0x55d533ad1460 msgr2=0x55d533e81e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 -- 172.21.15.27:0/3173078670 shutdown_connections 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc3985e740 1 -- 172.21.15.27:0/3173078670 wait complete. 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.023+0000 7efc2be00640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efc100b6700 0x7efc100b8c10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55d533e82bf0 tx=0x7efc1c10bc20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.024+0000 7efc3985e740 1 -- start start 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.024+0000 7efc3985e740 1 Processor -- start 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc3985e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e8f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc3985e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e8fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc3985e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc3985e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d533c4b700 con 0x55d533abc390 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc3985e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d533d85580 con 0x55d533add070 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc3985e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d533e92750 con 0x55d533ab41b0 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc3985e740 10 client.0 _ll_register_callbacks cb 0x55d533d95470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc2b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e8fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc2b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e8fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53384/0 (socket says 172.21.15.27:53384) 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc2b400640 1 -- 172.21.15.27:0/1612105112 learned_addr learned my addr 172.21.15.27:0/1612105112 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc30c00640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e8f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc30c00640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.025+0000 7efc2b400640 1 -- 172.21.15.27:0/1612105112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 msgr2=0x55d533e8f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc2b400640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e8f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc2b400640 1 -- 172.21.15.27:0/1612105112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 msgr2=0x55d533e9ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc2b400640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc2b400640 1 -- 172.21.15.27:0/1612105112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d533b48650 con 0x55d533abc390 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc30c00640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e8f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc2b400640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e8fbe0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efc18012700 tx=0x7efc18012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4ef0d83214401a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc15600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc15600640 1 -- 172.21.15.27:0/1612105112 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18004050 con 0x55d533abc390 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc15600640 1 -- 172.21.15.27:0/1612105112 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7efc180041f0 con 0x55d533abc390 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d533d87b90 con 0x55d533abc390 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.026+0000 7efc15600640 1 -- 172.21.15.27:0/1612105112 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc18025050 con 0x55d533abc390 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 10 client.19347 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d533da2950 con 0x55d533abc390 2024-09-21T07:26:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc15600640 1 -- 172.21.15.27:0/1612105112 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d533d87b90 con 0x55d533abc390 2024-09-21T07:26:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc15600640 1 -- 172.21.15.27:0/1612105112 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efc180039e0 con 0x55d533abc390 2024-09-21T07:26:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 10 client.19347 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 10 client.19347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:26:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 20 client.19347 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a9b9f0 0x55d533ea18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 4 client.19347 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.027+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d533dfcf00 con 0x55d533a9b9f0 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.028+0000 7efc30c00640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a9b9f0 0x55d533ea18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.028+0000 7efc30c00640 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a9b9f0 0x55d533ea18c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7cf8bd99e8ee64f3 server_cookie=7f3467e29650fb58 in_seq=0 out_seq=0 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.028+0000 7efc15600640 10 client.19347 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc15600640 1 -- 172.21.15.27:0/1612105112 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d533dfcf00 con 0x55d533a9b9f0 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc15600640 10 client.19347 handle_command_reply: tid=0 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 client.19347 shutdown 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 msgr2=0x55d533e8fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e8fbe0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efc18012700 tx=0x7efc18012bc0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 shutdown_connections 2024-09-21T07:26:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a9b9f0 0x55d533ea18c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e8fbe0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 --2- 172.21.15.27:0/1612105112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e8f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc15600640 0 client.19347 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.029+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 >> 172.21.15.27:0/1612105112 conn(0x55d533ad1460 msgr2=0x55d533e87100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.030+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 shutdown_connections 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.030+0000 7efc3985e740 1 -- 172.21.15.27:0/1612105112 wait complete. 2024-09-21T07:26:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.030+0000 7efc3985e740 20 client.19347 trim_cache size 0 max 16384 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 Processor -- start 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 -- start start 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533ab41b0 0x55d533e912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533abc390 0x55d533e9b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b5860 con 0x55d533ab41b0 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b3130 con 0x55d533add070 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc3985e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b0fa0 con 0x55d533abc390 2024-09-21T07:26:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc30c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533ab41b0 0x55d533e912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc30c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533ab41b0 0x55d533e912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53400/0 (socket says 172.21.15.27:53400) 2024-09-21T07:26:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.044+0000 7efc30c00640 1 -- 172.21.15.27:0/2118590588 learned_addr learned my addr 172.21.15.27:0/2118590588 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc2b400640 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533abc390 0x55d533e9b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc30c00640 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc30c00640 1 -- 172.21.15.27:0/2118590588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533abc390 msgr2=0x55d533e9b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc30c00640 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533abc390 0x55d533e9b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc30c00640 1 -- 172.21.15.27:0/2118590588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 msgr2=0x55d533e9be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc30c00640 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc30c00640 1 -- 172.21.15.27:0/2118590588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d533e86dd0 con 0x55d533ab41b0 2024-09-21T07:26:03.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.045+0000 7efc30c00640 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533ab41b0 0x55d533e912d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efc24009d60 tx=0x7efc24000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed381fd24aaba3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc17e00640 1 -- 172.21.15.27:0/2118590588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24004050 con 0x55d533ab41b0 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc17e00640 1 -- 172.21.15.27:0/2118590588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc24033070 con 0x55d533ab41b0 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc17e00640 1 -- 172.21.15.27:0/2118590588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2400fb00 con 0x55d533ab41b0 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc3985e740 1 -- 172.21.15.27:0/2118590588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533ab41b0 msgr2=0x55d533e912d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc3985e740 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533ab41b0 0x55d533e912d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efc24009d60 tx=0x7efc24000f40 comp rx=0 tx=0).stop 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc3985e740 1 -- 172.21.15.27:0/2118590588 shutdown_connections 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc3985e740 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533abc390 0x55d533e9b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc3985e740 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e9be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc3985e740 1 --2- 172.21.15.27:0/2118590588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533ab41b0 0x55d533e912d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.046+0000 7efc3985e740 1 -- 172.21.15.27:0/2118590588 >> 172.21.15.27:0/2118590588 conn(0x55d533ad1460 msgr2=0x55d533e90bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.047+0000 7efc3985e740 1 -- 172.21.15.27:0/2118590588 shutdown_connections 2024-09-21T07:26:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.047+0000 7efc3985e740 1 -- 172.21.15.27:0/2118590588 wait complete. 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.047+0000 7efc3985e740 1 -- start start 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.047+0000 7efc3985e740 1 Processor -- start 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc3985e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e97f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc3985e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc3985e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e96f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc3985e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b15f0 con 0x55d533abc390 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc3985e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b2a60 con 0x55d533add070 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc3985e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b1cc0 con 0x55d533ab41b0 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc3985e740 10 client.0 _ll_register_callbacks cb 0x7efc18009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc2b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc2b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53402/0 (socket says 172.21.15.27:53402) 2024-09-21T07:26:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc2b400640 1 -- 172.21.15.27:0/1145567326 learned_addr learned my addr 172.21.15.27:0/1145567326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc30c00640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e97f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.048+0000 7efc30c00640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e96f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc2b400640 1 -- 172.21.15.27:0/1145567326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 msgr2=0x55d533e97f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc2b400640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e97f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc2b400640 1 -- 172.21.15.27:0/1145567326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 msgr2=0x55d533e96f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc2b400640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e96f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc2b400640 1 -- 172.21.15.27:0/1145567326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d533e97440 con 0x55d533abc390 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc30c00640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e96f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc2b400640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e969c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efc18003e30 tx=0x7efc180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7409e40f1a91589b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc15600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc15600640 1 -- 172.21.15.27:0/1145567326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18004370 con 0x55d533abc390 2024-09-21T07:26:03.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc15600640 1 -- 172.21.15.27:0/1145567326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9461+0+0 (secure 0 0 0) 0x7efc18018040 con 0x55d533abc390 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.049+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc24033070 con 0x55d533abc390 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc15600640 1 -- 172.21.15.27:0/1145567326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc1800f920 con 0x55d533abc390 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 10 client.19353 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d533f0cef0 con 0x55d533abc390 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc15600640 1 -- 172.21.15.27:0/1145567326 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc24033070 con 0x55d533abc390 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc15600640 1 -- 172.21.15.27:0/1145567326 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efc180035f0 con 0x55d533abc390 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 10 client.19353 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 10 client.19353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 20 client.19353 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a70da0 0x55d533f0f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 4 client.19353 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.050+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d533df89b0 con 0x55d533a70da0 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.051+0000 7efc30c00640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a70da0 0x55d533f0f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.051+0000 7efc30c00640 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a70da0 0x55d533f0f6e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7cbf309765235a7f server_cookie=be669a230a8b8942 in_seq=0 out_seq=0 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.051+0000 7efc15600640 10 client.19353 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc15600640 1 -- 172.21.15.27:0/1145567326 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d533df89b0 con 0x55d533a70da0 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc15600640 10 client.19353 handle_command_reply: tid=0 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 client.19353 shutdown 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 msgr2=0x55d533e969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e969c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efc18003e30 tx=0x7efc180107a0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 shutdown_connections 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55d533a70da0 0x55d533f0f6e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d533add070 0x55d533e96f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d533abc390 0x55d533e969c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 --2- 172.21.15.27:0/1145567326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d533ab41b0 0x55d533e97f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 >> 172.21.15.27:0/1145567326 conn(0x55d533ad1460 msgr2=0x55d533e84fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:03.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc15600640 0 client.19353 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:03.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 shutdown_connections 2024-09-21T07:26:03.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 1 -- 172.21.15.27:0/1145567326 wait complete. 2024-09-21T07:26:03.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.052+0000 7efc3985e740 20 client.19353 trim_cache size 0 max 16384 2024-09-21T07:26:03.060 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:26:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efc100b6700 msgr2=0x7efc100b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efc100b6700 0x7efc100b8c10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55d533e82bf0 tx=0x7efc1c10bc20 comp rx=0 tx=0).stop 2024-09-21T07:26:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 msgr2=0x7efc2c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c19c620 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005ab0 tx=0x7efc1c003c10 comp rx=0 tx=0).stop 2024-09-21T07:26:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 shutdown_connections 2024-09-21T07:26:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7efc100b6700 0x7efc100b8c10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc2c10be40 0x7efc2c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc2c10b4f0 0x7efc2c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 --2- 172.21.15.27:0/1538429696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc2c105bb0 0x7efc2c19c620 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 >> 172.21.15.27:0/1538429696 conn(0x7efc2c101680 msgr2=0x7efc2c10ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 shutdown_connections 2024-09-21T07:26:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:03.053+0000 7efc31600640 1 -- 172.21.15.27:0/1538429696 wait complete. 2024-09-21T07:26:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:02 smithi153 ceph-mon[34747]: pgmap v782: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-21T07:26:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:02 smithi119 ceph-mon[34519]: pgmap v782: 129 pgs: 129 active+clean; 3.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-21T07:26:03.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:26:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:04 smithi027 ceph-mon[30779]: pgmap v783: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-09-21T07:26:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:04 smithi027 ceph-mon[30779]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T07:26:05.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 -- 172.21.15.27:0/2811510716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01039b0 msgr2=0x7f32c0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 --2- 172.21.15.27:0/2811510716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01039b0 0x7f32c0103dc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009900 tx=0x7f32b402a410 comp rx=0 tx=0).stop 2024-09-21T07:26:05.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 -- 172.21.15.27:0/2811510716 shutdown_connections 2024-09-21T07:26:05.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 --2- 172.21.15.27:0/2811510716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c01092f0 0x7f32c0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 --2- 172.21.15.27:0/2811510716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0109c40 0x7f32c0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 --2- 172.21.15.27:0/2811510716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01039b0 0x7f32c0103dc0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 -- 172.21.15.27:0/2811510716 >> 172.21.15.27:0/2811510716 conn(0x7f32c00ff440 msgr2=0x7f32c0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:05.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 -- 172.21.15.27:0/2811510716 shutdown_connections 2024-09-21T07:26:05.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.049+0000 7f32c6200640 1 -- 172.21.15.27:0/2811510716 wait complete. 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.050+0000 7f32c6200640 1 Processor -- start 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.050+0000 7f32c6200640 1 -- start start 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c6200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c01039b0 0x7f32c019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01092f0 0x7f32c019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0109c40 0x7f32c019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c6200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0113900 con 0x7f32c01092f0 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c6200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0113780 con 0x7f32c0109c40 2024-09-21T07:26:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c6200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0113a80 con 0x7f32c01039b0 2024-09-21T07:26:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01092f0 0x7f32c019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c01039b0 0x7f32c019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01092f0 0x7f32c019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53424/0 (socket says 172.21.15.27:53424) 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32bfe00640 1 -- 172.21.15.27:0/2852466902 learned_addr learned my addr 172.21.15.27:0/2852466902 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.051+0000 7f32c5800640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0109c40 0x7f32c019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bfe00640 1 -- 172.21.15.27:0/2852466902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c01039b0 msgr2=0x7f32c019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bfe00640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c01039b0 0x7f32c019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bfe00640 1 -- 172.21.15.27:0/2852466902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0109c40 msgr2=0x7f32c019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bfe00640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0109c40 0x7f32c019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bfe00640 1 -- 172.21.15.27:0/2852466902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01a02a0 con 0x7f32c01092f0 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32c5800640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0109c40 0x7f32c019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bfe00640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01092f0 0x7f32c019a8d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00e9f0 tx=0x7f32ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bd600640 1 -- 172.21.15.27:0/2852466902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00ccc0 con 0x7f32c01092f0 2024-09-21T07:26:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bd600640 1 -- 172.21.15.27:0/2852466902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ac0108e0 con 0x7f32c01092f0 2024-09-21T07:26:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32bd600640 1 -- 172.21.15.27:0/2852466902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac018880 con 0x7f32c01092f0 2024-09-21T07:26:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.052+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01a0590 con 0x7f32c01092f0 2024-09-21T07:26:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.053+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a6cf0 con 0x7f32c01092f0 2024-09-21T07:26:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.056+0000 7f32ce4c2740 1 Processor -- start 2024-09-21T07:26:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.056+0000 7f32ce4c2740 1 -- start start 2024-09-21T07:26:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.056+0000 7f32bd600640 1 -- 172.21.15.27:0/2852466902 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f32ac010430 con 0x7f32c01092f0 2024-09-21T07:26:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3143070 0x561de34e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de311a1b0 0x561de34ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34ea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561de3460fb0 con 0x561de3122390 2024-09-21T07:26:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561de34503e0 con 0x561de3143070 2024-09-21T07:26:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561de34541f0 con 0x561de311a1b0 2024-09-21T07:26:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32c5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3143070 0x561de34e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32c5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3143070 0x561de34e9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39158/0 (socket says 172.21.15.27:39158) 2024-09-21T07:26:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34ea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32c5800640 1 -- 172.21.15.27:0/1731153014 learned_addr learned my addr 172.21.15.27:0/1731153014 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.057+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de311a1b0 0x561de34ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32c5800640 1 -- 172.21.15.27:0/1731153014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de311a1b0 msgr2=0x561de34ea4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32c5800640 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de311a1b0 0x561de34ea4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32c5800640 1 -- 172.21.15.27:0/1731153014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 msgr2=0x561de34ea9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32c5800640 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34ea9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32c5800640 1 -- 172.21.15.27:0/1731153014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de341acd0 con 0x561de3143070 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32bfe00640 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34ea9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32c5800640 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3143070 0x561de34e9f70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f32b8011810 tx=0x7f32b8011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d2e1f8949793c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32b2000640 1 -- 172.21.15.27:0/1731153014 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8011310 con 0x561de3143070 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32b2000640 1 -- 172.21.15.27:0/1731153014 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b800f9a0 con 0x561de3143070 2024-09-21T07:26:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.058+0000 7f32b2000640 1 -- 172.21.15.27:0/1731153014 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b80055f0 con 0x561de3143070 2024-09-21T07:26:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.059+0000 7f32bd600640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f32a80b6700 0x7f32a80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.059+0000 7f32bd600640 1 -- 172.21.15.27:0/2852466902 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f32ac0e7eb0 con 0x7f32c01092f0 2024-09-21T07:26:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.059+0000 7f32c4e00640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f32a80b6700 0x7f32a80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.059+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1731153014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3143070 msgr2=0x561de34e9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.059+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3143070 0x561de34e9f70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f32b8011810 tx=0x7f32b8011cd0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.059+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1731153014 shutdown_connections 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de311a1b0 0x561de34ea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34ea9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1731153014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3143070 0x561de34e9f70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1731153014 >> 172.21.15.27:0/1731153014 conn(0x561de3137460 msgr2=0x561de34e7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1731153014 shutdown_connections 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1731153014 wait complete. 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32c4e00640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f32a80b6700 0x7f32a80b8c10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f32b4031440 tx=0x7f32b4009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 -- start start 2024-09-21T07:26:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.060+0000 7f32ce4c2740 1 Processor -- start 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de3501a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561de32b1700 con 0x561de311a1b0 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561de33eb580 con 0x561de3122390 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561de34f8750 con 0x561de3143070 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32ce4c2740 10 client.0 _ll_register_callbacks cb 0x561de33fb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53440/0 (socket says 172.21.15.27:53440) 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32c4e00640 1 -- 172.21.15.27:0/1386322901 learned_addr learned my addr 172.21.15.27:0/1386322901 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.061+0000 7f32bfe00640 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32c5800640 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de3501a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32bfe00640 1 -- 172.21.15.27:0/1386322901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 msgr2=0x561de3501a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32bfe00640 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de3501a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32bfe00640 1 -- 172.21.15.27:0/1386322901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 msgr2=0x561de34f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32bfe00640 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32bfe00640 1 -- 172.21.15.27:0/1386322901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de31ae650 con 0x561de3122390 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32bfe00640 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34f5bf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0e51a0 tx=0x7f32ac0ad730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd03be400d8024f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.062+0000 7f32a6a00640 1 -- 172.21.15.27:0/1386322901 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0e5630 con 0x561de3122390 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561de33edb90 con 0x561de3122390 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32a6a00640 1 -- 172.21.15.27:0/1386322901 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f32ac0eb250 con 0x561de3122390 2024-09-21T07:26:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32a6a00640 1 -- 172.21.15.27:0/1386322901 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32ac010720 con 0x561de3122390 2024-09-21T07:26:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32ce4c2740 10 client.29387 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561de3408950 con 0x561de3122390 2024-09-21T07:26:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32a6a00640 1 -- 172.21.15.27:0/1386322901 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561de33edb90 con 0x561de3122390 2024-09-21T07:26:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32a6a00640 1 -- 172.21.15.27:0/1386322901 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f32ac0ac570 con 0x561de3122390 2024-09-21T07:26:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32ce4c2740 10 client.29387 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32ce4c2740 10 client.29387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:05.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32ce4c2740 20 client.29387 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.063+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de31019f0 0x561de35077c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.064+0000 7f32ce4c2740 4 client.29387 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.064+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de3462f00 con 0x561de31019f0 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.064+0000 7f32c4e00640 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de31019f0 0x561de35077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.064+0000 7f32c4e00640 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de31019f0 0x561de35077c0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9c32b6b31bb37e server_cookie=7e3c8d6cf9e1723d in_seq=0 out_seq=0 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.064+0000 7f32a6a00640 10 client.29387 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.065+0000 7f32a6a00640 1 -- 172.21.15.27:0/1386322901 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561de3462f00 con 0x561de31019f0 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.065+0000 7f32a6a00640 10 client.29387 handle_command_reply: tid=0 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.065+0000 7f32ce4c2740 1 client.29387 shutdown 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 msgr2=0x561de34f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34f5bf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0e51a0 tx=0x7f32ac0ad730 comp rx=0 tx=0).stop 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 shutdown_connections 2024-09-21T07:26:05.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de31019f0 0x561de35077c0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de3501a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34f5bf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1386322901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32a6a00640 0 client.29387 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 >> 172.21.15.27:0/1386322901 conn(0x561de3137460 msgr2=0x561de34e6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 shutdown_connections 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1386322901 wait complete. 2024-09-21T07:26:05.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.066+0000 7f32ce4c2740 20 client.29387 trim_cache size 0 max 16384 2024-09-21T07:26:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.080+0000 7f32ce4c2740 1 Processor -- start 2024-09-21T07:26:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32ce4c2740 1 -- start start 2024-09-21T07:26:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b36a0 con 0x561de311a1b0 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b0f70 con 0x561de3122390 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01aede0 con 0x561de3143070 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53448/0 (socket says 172.21.15.27:53448) 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32c4e00640 1 -- 172.21.15.27:0/1498908464 learned_addr learned my addr 172.21.15.27:0/1498908464 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32c5800640 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.081+0000 7f32bfe00640 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.082+0000 7f32c4e00640 1 -- 172.21.15.27:0/1498908464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 msgr2=0x561de34fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.082+0000 7f32c4e00640 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.082+0000 7f32c4e00640 1 -- 172.21.15.27:0/1498908464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 msgr2=0x561de34fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.082+0000 7f32c4e00640 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34fdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.082+0000 7f32c4e00640 1 -- 172.21.15.27:0/1498908464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de34ec260 con 0x561de311a1b0 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.082+0000 7f32c4e00640 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f5b70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f32b4006010 tx=0x7f32b403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51e7d4b54f04f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.082+0000 7f32b1600640 1 -- 172.21.15.27:0/1498908464 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4035040 con 0x561de311a1b0 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32b1600640 1 -- 172.21.15.27:0/1498908464 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b4004c00 con 0x561de311a1b0 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32b1600640 1 -- 172.21.15.27:0/1498908464 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b403d890 con 0x561de311a1b0 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1498908464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 msgr2=0x561de34f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f5b70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f32b4006010 tx=0x7f32b403f000 comp rx=0 tx=0).stop 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1498908464 shutdown_connections 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de3122390 0x561de34fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/1498908464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de311a1b0 0x561de34f5b70 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.083+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1498908464 >> 172.21.15.27:0/1498908464 conn(0x561de3137460 msgr2=0x561de34f5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.084+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1498908464 shutdown_connections 2024-09-21T07:26:05.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.084+0000 7f32ce4c2740 1 -- 172.21.15.27:0/1498908464 wait complete. 2024-09-21T07:26:05.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.084+0000 7f32ce4c2740 1 -- start start 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.084+0000 7f32ce4c2740 1 Processor -- start 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de311a1b0 0x561de34f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32ce4c2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01af430 con 0x561de3122390 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b08a0 con 0x561de311a1b0 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32ce4c2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01afb00 con 0x561de3143070 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32ce4c2740 10 client.0 _ll_register_callbacks cb 0x7f32ac0a9740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de311a1b0 0x561de34f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32c4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de311a1b0 0x561de34f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:39182/0 (socket says 172.21.15.27:39182) 2024-09-21T07:26:05.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32c4e00640 1 -- 172.21.15.27:0/699068512 learned_addr learned my addr 172.21.15.27:0/699068512 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.085+0000 7f32c5800640 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32c4e00640 1 -- 172.21.15.27:0/699068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 msgr2=0x561de34f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32c4e00640 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32c4e00640 1 -- 172.21.15.27:0/699068512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 msgr2=0x561de34f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32c4e00640 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34f1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32c4e00640 1 -- 172.21.15.27:0/699068512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de34f26c0 con 0x561de311a1b0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32c4e00640 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de311a1b0 0x561de34f32a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f32b4031770 tx=0x7f32b4009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786b0d00969ac768 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32a6a00640 1 -- 172.21.15.27:0/699068512 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4004960 con 0x561de311a1b0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b4004c00 con 0x561de311a1b0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.086+0000 7f32a6a00640 1 -- 172.21.15.27:0/699068512 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f32b40051c0 con 0x561de311a1b0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32a6a00640 1 -- 172.21.15.27:0/699068512 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32b4005400 con 0x561de311a1b0 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 10 client.29393 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:05.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561de3577270 con 0x561de311a1b0 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32a6a00640 1 -- 172.21.15.27:0/699068512 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b4004c00 con 0x561de311a1b0 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32a6a00640 1 -- 172.21.15.27:0/699068512 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f32b4059020 con 0x561de311a1b0 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 10 client.29393 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 10 client.29393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 20 client.29393 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de30d6da0 0x561de35799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 4 client.29393 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.087+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561de345e9b0 con 0x561de30d6da0 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.088+0000 7f32bfe00640 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de30d6da0 0x561de35799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.088+0000 7f32bfe00640 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de30d6da0 0x561de35799e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487d3ca48a3d0601 server_cookie=7a71dc9da8d06584 in_seq=0 out_seq=0 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.088+0000 7f32a6a00640 10 client.29393 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.088+0000 7f32a6a00640 1 -- 172.21.15.27:0/699068512 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561de345e9b0 con 0x561de30d6da0 2024-09-21T07:26:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.088+0000 7f32a6a00640 10 client.29393 handle_command_reply: tid=0 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.088+0000 7f32ce4c2740 1 client.29393 shutdown 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de311a1b0 msgr2=0x561de34f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de311a1b0 0x561de34f32a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f32b4031770 tx=0x7f32b4009650 comp rx=0 tx=0).stop 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 shutdown_connections 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561de30d6da0 0x561de35799e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de3143070 0x561de34f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32a6a00640 0 client.29393 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561de3122390 0x561de34f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 --2- 172.21.15.27:0/699068512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561de311a1b0 0x561de34f32a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 >> 172.21.15.27:0/699068512 conn(0x561de3137460 msgr2=0x561de34ed070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 shutdown_connections 2024-09-21T07:26:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 1 -- 172.21.15.27:0/699068512 wait complete. 2024-09-21T07:26:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.089+0000 7f32ce4c2740 20 client.29393 trim_cache size 0 max 16384 2024-09-21T07:26:05.095 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f32a80b6700 msgr2=0x7f32a80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f32a80b6700 0x7f32a80b8c10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f32b4031440 tx=0x7f32b4009560 comp rx=0 tx=0).stop 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01092f0 msgr2=0x7f32c019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01092f0 0x7f32c019a8d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00e9f0 tx=0x7f32ac00edf0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 shutdown_connections 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f32a80b6700 0x7f32a80b8c10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0109c40 0x7f32c019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c01092f0 0x7f32c019a8d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 --2- 172.21.15.27:0/2852466902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c01039b0 0x7f32c019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 >> 172.21.15.27:0/2852466902 conn(0x7f32c00ff440 msgr2=0x7f32c0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 shutdown_connections 2024-09-21T07:26:05.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:05.090+0000 7f32c6200640 1 -- 172.21.15.27:0/2852466902 wait complete. 2024-09-21T07:26:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:04 smithi153 ceph-mon[34747]: pgmap v783: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-09-21T07:26:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:04 smithi153 ceph-mon[34747]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T07:26:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:04 smithi119 ceph-mon[34519]: pgmap v783: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-09-21T07:26:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:04 smithi119 ceph-mon[34519]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T07:26:06.970 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:26:06.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:26:07.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:06 smithi027 ceph-mon[30779]: pgmap v785: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-09-21T07:26:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:06 smithi153 ceph-mon[34747]: pgmap v785: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-09-21T07:26:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:06 smithi119 ceph-mon[34519]: pgmap v785: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-09-21T07:26:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:08.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 -- 172.21.15.27:0/858256047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 msgr2=0x7f5390075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 --2- 172.21.15.27:0/858256047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f5390075bc0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5380009900 tx=0x7f538002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 -- 172.21.15.27:0/858256047 shutdown_connections 2024-09-21T07:26:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 --2- 172.21.15.27:0/858256047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53901100c0 0x7f5390112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 --2- 172.21.15.27:0/858256047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5390076100 0x7f539010fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 --2- 172.21.15.27:0/858256047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f5390075bc0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 -- 172.21.15.27:0/858256047 >> 172.21.15.27:0/858256047 conn(0x7f539007b850 msgr2=0x7f539007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 -- 172.21.15.27:0/858256047 shutdown_connections 2024-09-21T07:26:08.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.010+0000 7f5396200640 1 -- 172.21.15.27:0/858256047 wait complete. 2024-09-21T07:26:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.011+0000 7f5396200640 1 Processor -- start 2024-09-21T07:26:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.011+0000 7f5396200640 1 -- start start 2024-09-21T07:26:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f5396200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f539019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f5396200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5390076100 0x7f539019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f5396200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53901100c0 0x7f53901a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f5396200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5390115c70 con 0x7f53900757b0 2024-09-21T07:26:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f5396200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5390115af0 con 0x7f53901100c0 2024-09-21T07:26:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f5396200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5390115df0 con 0x7f5390076100 2024-09-21T07:26:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f538f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f539019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f538f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f539019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38550/0 (socket says 172.21.15.27:38550) 2024-09-21T07:26:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f538f400640 1 -- 172.21.15.27:0/4200210535 learned_addr learned my addr 172.21.15.27:0/4200210535 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f538ea00640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5390076100 0x7f539019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.012+0000 7f538fe00640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53901100c0 0x7f53901a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.013+0000 7f538f400640 1 -- 172.21.15.27:0/4200210535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5390076100 msgr2=0x7f539019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.013+0000 7f538f400640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5390076100 0x7f539019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.013+0000 7f538f400640 1 -- 172.21.15.27:0/4200210535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53901100c0 msgr2=0x7f53901a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.013+0000 7f538f400640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53901100c0 0x7f53901a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.013+0000 7f538f400640 1 -- 172.21.15.27:0/4200210535 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53901a2610 con 0x7f53900757b0 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.013+0000 7f538f400640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f539019c6a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5380005940 tx=0x7f5380004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.013+0000 7f5387e00640 1 -- 172.21.15.27:0/4200210535 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538003f030 con 0x7f53900757b0 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.014+0000 7f5387e00640 1 -- 172.21.15.27:0/4200210535 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5380003aa0 con 0x7f53900757b0 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.014+0000 7f5387e00640 1 -- 172.21.15.27:0/4200210535 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380035070 con 0x7f53900757b0 2024-09-21T07:26:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.014+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901a28a0 con 0x7f53900757b0 2024-09-21T07:26:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.014+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a9060 con 0x7f53900757b0 2024-09-21T07:26:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.017+0000 7f539cff1740 1 Processor -- start 2024-09-21T07:26:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.017+0000 7f539cff1740 1 -- start start 2024-09-21T07:26:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.017+0000 7f539cff1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774abb5350 0x55774af5cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.017+0000 7f539cff1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af5d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.017+0000 7f539cff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af5d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f539cff1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55774aed2e90 con 0x55774ab8c490 2024-09-21T07:26:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f539cff1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55774aec22c0 con 0x55774abb5350 2024-09-21T07:26:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f539cff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55774aec60d0 con 0x55774ab94670 2024-09-21T07:26:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f5387e00640 1 -- 172.21.15.27:0/4200210535 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5380042030 con 0x7f53900757b0 2024-09-21T07:26:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774abb5350 0x55774af5cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774abb5350 0x55774af5cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55504/0 (socket says 172.21.15.27:55504) 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 -- 172.21.15.27:0/587090121 learned_addr learned my addr 172.21.15.27:0/587090121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538f400640 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af5d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538ea00640 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af5d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 -- 172.21.15.27:0/587090121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 msgr2=0x55774af5d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af5d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 -- 172.21.15.27:0/587090121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 msgr2=0x55774af5d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af5d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.018+0000 7f538fe00640 1 -- 172.21.15.27:0/587090121 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774ac20930 con 0x55774abb5350 2024-09-21T07:26:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.019+0000 7f538fe00640 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774abb5350 0x55774af5cd10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5388009f50 tx=0x7f53880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ebffb6041217d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.019+0000 7f537be00640 1 -- 172.21.15.27:0/587090121 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388004d20 con 0x55774abb5350 2024-09-21T07:26:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.020+0000 7f537be00640 1 -- 172.21.15.27:0/587090121 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5388004050 con 0x55774abb5350 2024-09-21T07:26:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.020+0000 7f537be00640 1 -- 172.21.15.27:0/587090121 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388005280 con 0x55774abb5350 2024-09-21T07:26:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.020+0000 7f5387e00640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53740b6680 0x7f53740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.020+0000 7f5387e00640 1 -- 172.21.15.27:0/4200210535 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f5380107ff0 con 0x7f53900757b0 2024-09-21T07:26:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.020+0000 7f538f400640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53740b6680 0x7f53740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.020+0000 7f539cff1740 1 -- 172.21.15.27:0/587090121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774abb5350 msgr2=0x55774af5cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.020+0000 7f539cff1740 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774abb5350 0x55774af5cd10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5388009f50 tx=0x7f53880118d0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 -- 172.21.15.27:0/587090121 shutdown_connections 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af5d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af5d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 --2- 172.21.15.27:0/587090121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774abb5350 0x55774af5cd10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 -- 172.21.15.27:0/587090121 >> 172.21.15.27:0/587090121 conn(0x55774aba9740 msgr2=0x55774af5ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f538f400640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53740b6680 0x7f53740b8b90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55774af5b990 tx=0x7f538010bcb0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:08.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 -- 172.21.15.27:0/587090121 shutdown_connections 2024-09-21T07:26:08.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 -- 172.21.15.27:0/587090121 wait complete. 2024-09-21T07:26:08.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.021+0000 7f539cff1740 1 -- start start 2024-09-21T07:26:08.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 1 Processor -- start 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af68920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774abb5350 0x55774af74810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55774ad23890 con 0x55774abb5350 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55774ae5d160 con 0x55774ab8c490 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55774af6b4f0 con 0x55774ab94670 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.022+0000 7f539cff1740 10 client.0 _ll_register_callbacks cb 0x55774ae79a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774abb5350 0x55774af74810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774abb5350 0x55774af74810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38576/0 (socket says 172.21.15.27:38576) 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538fe00640 1 -- 172.21.15.27:0/3225722552 learned_addr learned my addr 172.21.15.27:0/3225722552 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af68e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54748/0 (socket says 172.21.15.27:54748) 2024-09-21T07:26:08.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538fe00640 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af68920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 -- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 msgr2=0x55774af68920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af68920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 -- 172.21.15.27:0/3225722552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774abb5350 msgr2=0x55774af74810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774abb5350 0x55774af74810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 -- 172.21.15.27:0/3225722552 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55774af74dd0 con 0x55774ab94670 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f538ea00640 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af68e60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f539006c760 tx=0x7f537c00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cebf6449b3ceb74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.023+0000 7f5378c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.024+0000 7f5378c00640 1 -- 172.21.15.27:0/3225722552 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c003e20 con 0x55774ab94670 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.024+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5388004050 con 0x55774ab94670 2024-09-21T07:26:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.024+0000 7f5378c00640 1 -- 172.21.15.27:0/3225722552 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f537c005020 con 0x55774ab94670 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.024+0000 7f5378c00640 1 -- 172.21.15.27:0/3225722552 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f537c024050 con 0x55774ab94670 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.024+0000 7f539cff1740 10 client.29365 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.024+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55774af777b0 con 0x55774ab94670 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.024+0000 7f5378c00640 1 -- 172.21.15.27:0/3225722552 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5388004050 con 0x55774ab94670 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f5378c00640 1 -- 172.21.15.27:0/3225722552 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f537c01b470 con 0x55774ab94670 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f539cff1740 10 client.29365 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f539cff1740 10 client.29365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f539cff1740 20 client.29365 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f539cff1740 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab73cd0 0x55774af7ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f539cff1740 4 client.29365 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55774aed4de0 con 0x55774ab73cd0 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.025+0000 7f538fe00640 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab73cd0 0x55774af7ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.026+0000 7f538fe00640 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab73cd0 0x55774af7ac20 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c04575d9223bae server_cookie=4e0073a54750334d in_seq=0 out_seq=0 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.026+0000 7f5378c00640 10 client.29365 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f5378c00640 1 -- 172.21.15.27:0/3225722552 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55774aed4de0 con 0x55774ab73cd0 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f5378c00640 10 client.29365 handle_command_reply: tid=0 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 client.29365 shutdown 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 msgr2=0x55774af68e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af68e60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f539006c760 tx=0x7f537c00be60 comp rx=0 tx=0).stop 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 shutdown_connections 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab73cd0 0x55774af7ac20 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774ab94670 0x55774af68e60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774abb5350 0x55774af74810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 --2- 172.21.15.27:0/3225722552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af68920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 >> 172.21.15.27:0/3225722552 conn(0x55774aba9740 msgr2=0x55774af59280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:08.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f5378c00640 0 client.29365 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:08.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 shutdown_connections 2024-09-21T07:26:08.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.027+0000 7f539cff1740 1 -- 172.21.15.27:0/3225722552 wait complete. 2024-09-21T07:26:08.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.028+0000 7f539cff1740 20 client.29365 trim_cache size 0 max 16384 2024-09-21T07:26:08.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.046+0000 7f539cff1740 1 Processor -- start 2024-09-21T07:26:08.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.046+0000 7f539cff1740 1 -- start start 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f539cff1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af61f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f539cff1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab94670 0x55774af624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f539cff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af59280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f539cff1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b5ab0 con 0x55774ab94670 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f539cff1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b3380 con 0x55774ab8c490 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f539cff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b11f0 con 0x55774abb5350 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f538ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab94670 0x55774af624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f538ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab94670 0x55774af624a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38580/0 (socket says 172.21.15.27:38580) 2024-09-21T07:26:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f538ea00640 1 -- 172.21.15.27:0/4040844803 learned_addr learned my addr 172.21.15.27:0/4040844803 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:08.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f538fe00640 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af61f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.047+0000 7f538fe00640 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af59280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f538ea00640 1 -- 172.21.15.27:0/4040844803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 msgr2=0x55774af59280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f538ea00640 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af59280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f538ea00640 1 -- 172.21.15.27:0/4040844803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 msgr2=0x55774af61f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f538ea00640 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af61f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f538ea00640 1 -- 172.21.15.27:0/4040844803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774af5b4f0 con 0x55774ab94670 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f538fe00640 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af59280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f538ea00640 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab94670 0x55774af624a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f537c01ba30 tx=0x7f537c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e212ac7ae7c1cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f537b400640 1 -- 172.21.15.27:0/4040844803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c00b480 con 0x55774ab94670 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f537b400640 1 -- 172.21.15.27:0/4040844803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537c003e20 con 0x55774ab94670 2024-09-21T07:26:08.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.048+0000 7f537b400640 1 -- 172.21.15.27:0/4040844803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c004650 con 0x55774ab94670 2024-09-21T07:26:08.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 -- 172.21.15.27:0/4040844803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab94670 msgr2=0x55774af624a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab94670 0x55774af624a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f537c01ba30 tx=0x7f537c003a80 comp rx=0 tx=0).stop 2024-09-21T07:26:08.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 -- 172.21.15.27:0/4040844803 shutdown_connections 2024-09-21T07:26:08.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af59280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab94670 0x55774af624a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 --2- 172.21.15.27:0/4040844803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab8c490 0x55774af61f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 -- 172.21.15.27:0/4040844803 >> 172.21.15.27:0/4040844803 conn(0x55774aba9740 msgr2=0x55774af66fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:08.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 -- 172.21.15.27:0/4040844803 shutdown_connections 2024-09-21T07:26:08.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.049+0000 7f539cff1740 1 -- 172.21.15.27:0/4040844803 wait complete. 2024-09-21T07:26:08.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 -- start start 2024-09-21T07:26:08.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 Processor -- start 2024-09-21T07:26:08.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af70780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab94670 0x55774af70cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b1840 con 0x55774ab8c490 2024-09-21T07:26:08.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b2cb0 con 0x55774ab94670 2024-09-21T07:26:08.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.050+0000 7f539cff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b1f10 con 0x55774abb5350 2024-09-21T07:26:08.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f539cff1740 10 client.0 _ll_register_callbacks cb 0x7f537c0041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af70780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab94670 0x55774af70cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af70780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38590/0 (socket says 172.21.15.27:38590) 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538fe00640 1 -- 172.21.15.27:0/46607515 learned_addr learned my addr 172.21.15.27:0/46607515 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538fe00640 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538ea00640 1 -- 172.21.15.27:0/46607515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 msgr2=0x55774af6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538ea00640 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538ea00640 1 -- 172.21.15.27:0/46607515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 msgr2=0x55774af70780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538ea00640 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af70780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538ea00640 1 -- 172.21.15.27:0/46607515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55774af71280 con 0x55774ab94670 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.051+0000 7f538fe00640 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af70780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f538ea00640 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab94670 0x55774af70cc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f537c00bea0 tx=0x7f537c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b1bd9a79689f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f5378c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:08.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f5378c00640 1 -- 172.21.15.27:0/46607515 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c035020 con 0x55774ab94670 2024-09-21T07:26:08.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537c003e20 con 0x55774ab94670 2024-09-21T07:26:08.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f5378c00640 1 -- 172.21.15.27:0/46607515 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f537c021040 con 0x55774ab94670 2024-09-21T07:26:08.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f5378c00640 1 -- 172.21.15.27:0/46607515 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f537c031460 con 0x55774ab94670 2024-09-21T07:26:08.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f539cff1740 10 client.29411 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:08.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.052+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55774afe5950 con 0x55774ab94670 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f5378c00640 1 -- 172.21.15.27:0/46607515 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537c003e20 con 0x55774ab94670 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f5378c00640 1 -- 172.21.15.27:0/46607515 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f537c011480 con 0x55774ab94670 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f539cff1740 10 client.29411 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f539cff1740 10 client.29411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f539cff1740 20 client.29411 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f539cff1740 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab49080 0x55774afe7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f539cff1740 4 client.29411 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.053+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55774aed0890 con 0x55774ab49080 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.054+0000 7f538fe00640 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab49080 0x55774afe7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.054+0000 7f538fe00640 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab49080 0x55774afe7fc0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8db94dbaca91af0 server_cookie=f367813f5312a6aa in_seq=0 out_seq=0 2024-09-21T07:26:08.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.054+0000 7f5378c00640 10 client.29411 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:08.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f5378c00640 1 -- 172.21.15.27:0/46607515 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55774aed0890 con 0x55774ab49080 2024-09-21T07:26:08.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f5378c00640 10 client.29411 handle_command_reply: tid=0 2024-09-21T07:26:08.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 client.29411 shutdown 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab94670 msgr2=0x55774af70cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab94670 0x55774af70cc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f537c00bea0 tx=0x7f537c005750 comp rx=0 tx=0).stop 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 shutdown_connections 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55774ab49080 0x55774afe7fc0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55774abb5350 0x55774af6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55774ab94670 0x55774af70cc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.056+0000 7f539cff1740 1 --2- 172.21.15.27:0/46607515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55774ab8c490 0x55774af70780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.057+0000 7f5378c00640 0 client.29411 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.057+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 >> 172.21.15.27:0/46607515 conn(0x55774aba9740 msgr2=0x55774af66f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.057+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 shutdown_connections 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.057+0000 7f539cff1740 1 -- 172.21.15.27:0/46607515 wait complete. 2024-09-21T07:26:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.057+0000 7f539cff1740 20 client.29411 trim_cache size 0 max 16384 2024-09-21T07:26:08.060 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:26:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:26:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "ddc913bc-1450-443e-8516-d628dd0da731", 2024-09-21T07:26:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:26:08.060 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.057+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53740b6680 msgr2=0x7f53740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.057+0000 7f5396200640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53740b6680 0x7f53740b8b90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55774af5b990 tx=0x7f538010bcb0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 msgr2=0x7f539019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f539019c6a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5380005940 tx=0x7f5380004270 comp rx=0 tx=0).stop 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 shutdown_connections 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5390076100 0x7f539019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f53740b6680 0x7f53740b8b90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53901100c0 0x7f53901a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 --2- 172.21.15.27:0/4200210535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53900757b0 0x7f539019c6a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 >> 172.21.15.27:0/4200210535 conn(0x7f539007b850 msgr2=0x7f5390109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 shutdown_connections 2024-09-21T07:26:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:08.058+0000 7f5396200640 1 -- 172.21.15.27:0/4200210535 wait complete. 2024-09-21T07:26:08.824 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:26:09.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:08 smithi027 ceph-mon[30779]: pgmap v786: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-09-21T07:26:09.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:08 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:26:09.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:09.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:08 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:26:09.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:08 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:09.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:08 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:26:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:08 smithi153 ceph-mon[34747]: pgmap v786: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-09-21T07:26:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:08 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:26:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:08 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:26:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:08 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:09.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:08 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:26:09.209 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:08 smithi119 ceph-mon[34519]: pgmap v786: 129 pgs: 129 active+clean; 3.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-09-21T07:26:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:08 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:26:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:08 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:26:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:08 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:08 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:26:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 -- 172.21.15.27:0/3036739172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 msgr2=0x7f4178103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 --2- 172.21.15.27:0/3036739172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f4178103f70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4168009900 tx=0x7f416802a410 comp rx=0 tx=0).stop 2024-09-21T07:26:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 -- 172.21.15.27:0/3036739172 shutdown_connections 2024-09-21T07:26:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 --2- 172.21.15.27:0/3036739172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41781094a0 0x7f41781098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 --2- 172.21.15.27:0/3036739172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178109df0 0x7f4178110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 --2- 172.21.15.27:0/3036739172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f4178103f70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 -- 172.21.15.27:0/3036739172 >> 172.21.15.27:0/3036739172 conn(0x7f417807b850 msgr2=0x7f417807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 -- 172.21.15.27:0/3036739172 shutdown_connections 2024-09-21T07:26:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.837+0000 7f417d600640 1 -- 172.21.15.27:0/3036739172 wait complete. 2024-09-21T07:26:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.838+0000 7f417d600640 1 Processor -- start 2024-09-21T07:26:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.838+0000 7f417d600640 1 -- start start 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f417d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f417819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f417d600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41781094a0 0x7f417819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f417d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178109df0 0x7f41781a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f417d600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4178113ab0 con 0x7f4178103b60 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f417d600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4178113930 con 0x7f4178109df0 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f417d600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4178113c30 con 0x7f41781094a0 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4176a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f417819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4176a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f417819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38604/0 (socket says 172.21.15.27:38604) 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4176a00640 1 -- 172.21.15.27:0/2083333547 learned_addr learned my addr 172.21.15.27:0/2083333547 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f416fe00640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41781094a0 0x7f417819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4177400640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178109df0 0x7f41781a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4176a00640 1 -- 172.21.15.27:0/2083333547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41781094a0 msgr2=0x7f417819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4176a00640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41781094a0 0x7f417819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4176a00640 1 -- 172.21.15.27:0/2083333547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178109df0 msgr2=0x7f41781a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.839+0000 7f4176a00640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178109df0 0x7f41781a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.840+0000 7f4176a00640 1 -- 172.21.15.27:0/2083333547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41781a25f0 con 0x7f4178103b60 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.840+0000 7f4176a00640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f417819c680 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4168005940 tx=0x7f41680040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.840+0000 7f416f400640 1 -- 172.21.15.27:0/2083333547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416803f030 con 0x7f4178103b60 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.840+0000 7f416f400640 1 -- 172.21.15.27:0/2083333547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41680044c0 con 0x7f4178103b60 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.840+0000 7f416f400640 1 -- 172.21.15.27:0/2083333547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168035070 con 0x7f4178103b60 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.840+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41781a2880 con 0x7f4178103b60 2024-09-21T07:26:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.840+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41781a9040 con 0x7f4178103b60 2024-09-21T07:26:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 Processor -- start 2024-09-21T07:26:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 -- start start 2024-09-21T07:26:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f416f400640 1 -- 172.21.15.27:0/2083333547 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4168042030 con 0x7f4178103b60 2024-09-21T07:26:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a8fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562e54a06f70 con 0x562e546c8390 2024-09-21T07:26:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562e549f63a0 con 0x562e546c01b0 2024-09-21T07:26:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.844+0000 7f41843c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e549fa1b0 con 0x562e546e9070 2024-09-21T07:26:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a8fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f416fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a8fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54800/0 (socket says 172.21.15.27:54800) 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4176a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 -- 172.21.15.27:0/3788980103 learned_addr learned my addr 172.21.15.27:0/3788980103 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 -- 172.21.15.27:0/3788980103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 msgr2=0x562e54a902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 --2- 172.21.15.27:0/3788980103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 -- 172.21.15.27:0/3788980103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 msgr2=0x562e54a907e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 --2- 172.21.15.27:0/3788980103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a907e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.845+0000 7f4177400640 1 -- 172.21.15.27:0/3788980103 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e549c0c90 con 0x562e546e9070 2024-09-21T07:26:09.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.846+0000 7f4177400640 1 --2- 172.21.15.27:0/3788980103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a8fd60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4170009fd0 tx=0x7f41700118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab3b1af73de4ed3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.846+0000 7f4167400640 1 -- 172.21.15.27:0/3788980103 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170004d20 con 0x562e546e9070 2024-09-21T07:26:09.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.846+0000 7f4167400640 1 -- 172.21.15.27:0/3788980103 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4170004050 con 0x562e546e9070 2024-09-21T07:26:09.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.846+0000 7f4167400640 1 -- 172.21.15.27:0/3788980103 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170005220 con 0x562e546e9070 2024-09-21T07:26:09.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.846+0000 7f416f400640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f415c0b6700 0x7f415c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f4176a00640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f415c0b6700 0x7f415c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f416f400640 1 -- 172.21.15.27:0/2083333547 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f4168107e20 con 0x7f4178103b60 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 -- 172.21.15.27:0/3788980103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 msgr2=0x562e54a8fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 --2- 172.21.15.27:0/3788980103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a8fd60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4170009fd0 tx=0x7f41700118d0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 -- 172.21.15.27:0/3788980103 shutdown_connections 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 --2- 172.21.15.27:0/3788980103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a907e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 --2- 172.21.15.27:0/3788980103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 --2- 172.21.15.27:0/3788980103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a8fd60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 -- 172.21.15.27:0/3788980103 >> 172.21.15.27:0/3788980103 conn(0x562e546dd460 msgr2=0x562e54a8dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 -- 172.21.15.27:0/3788980103 shutdown_connections 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.847+0000 7f41843c6740 1 -- 172.21.15.27:0/3788980103 wait complete. 2024-09-21T07:26:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.848+0000 7f4176a00640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f415c0b6700 0x7f415c0b8c10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4168105220 tx=0x7f416810b970 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.848+0000 7f41843c6740 1 -- start start 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.848+0000 7f41843c6740 1 Processor -- start 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f41843c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a9b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f41843c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f41843c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54aa7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f41843c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562e54857700 con 0x562e546c8390 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f41843c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562e54991540 con 0x562e546c01b0 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f41843c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e54a9e540 con 0x562e546e9070 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f41843c6740 10 client.0 _ll_register_callbacks cb 0x562e549a1430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f416fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f416fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a9b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38622/0 (socket says 172.21.15.27:38622) 2024-09-21T07:26:09.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f416fe00640 1 -- 172.21.15.27:0/2542202646 learned_addr learned my addr 172.21.15.27:0/2542202646 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.849+0000 7f4177400640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a9b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f4177400640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54aa7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f416fe00640 1 -- 172.21.15.27:0/2542202646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 msgr2=0x562e54aa7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f416fe00640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54aa7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f416fe00640 1 -- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 msgr2=0x562e54a9b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f416fe00640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a9b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f416fe00640 1 -- 172.21.15.27:0/2542202646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e54754650 con 0x562e546c8390 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f4177400640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54aa7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f416fe00640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a9b9e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4160012700 tx=0x7f4160012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28ae9b691c5ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f415be00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:09.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f415be00640 1 -- 172.21.15.27:0/2542202646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416000f920 con 0x562e546c8390 2024-09-21T07:26:09.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e54993b50 con 0x562e546c8390 2024-09-21T07:26:09.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.850+0000 7f415be00640 1 -- 172.21.15.27:0/2542202646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f416000fac0 con 0x562e546c8390 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f415be00640 1 -- 172.21.15.27:0/2542202646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f416000fcc0 con 0x562e546c8390 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 10 client.19401 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e549ae910 con 0x562e546c8390 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f415be00640 1 -- 172.21.15.27:0/2542202646 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e54993b50 con 0x562e546c8390 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f415be00640 1 -- 172.21.15.27:0/2542202646 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f416001c420 con 0x562e546c8390 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 10 client.19401 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 10 client.19401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 20 client.19401 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e546a79f0 0x562e54aad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 4 client.19401 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.851+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e54a08ec0 con 0x562e546a79f0 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.852+0000 7f4177400640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e546a79f0 0x562e54aad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.852+0000 7f4177400640 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e546a79f0 0x562e54aad780 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6098cb421c623a8 server_cookie=5dd4d7198999e25d in_seq=0 out_seq=0 2024-09-21T07:26:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.852+0000 7f415be00640 10 client.19401 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.853+0000 7f415be00640 1 -- 172.21.15.27:0/2542202646 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e54a08ec0 con 0x562e546a79f0 2024-09-21T07:26:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.853+0000 7f415be00640 10 client.19401 handle_command_reply: tid=0 2024-09-21T07:26:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.853+0000 7f41843c6740 1 client.19401 shutdown 2024-09-21T07:26:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.853+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 msgr2=0x562e54a9b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.853+0000 7f41843c6740 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a9b9e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4160012700 tx=0x7f4160012bc0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 shutdown_connections 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e546a79f0 0x562e54aad780 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54aa7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c8390 0x562e54a9b9e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 --2- 172.21.15.27:0/2542202646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c01b0 0x562e54a9b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 >> 172.21.15.27:0/2542202646 conn(0x562e546dd460 msgr2=0x562e54a92ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f415be00640 0 client.19401 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 shutdown_connections 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 1 -- 172.21.15.27:0/2542202646 wait complete. 2024-09-21T07:26:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.854+0000 7f41843c6740 20 client.19401 trim_cache size 0 max 16384 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 Processor -- start 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 -- start start 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c01b0 0x562e54a9a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54a91750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a9d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f41781b59f0 con 0x562e546c01b0 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41781b32c0 con 0x562e546c8390 2024-09-21T07:26:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.873+0000 7f41843c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41781b1130 con 0x562e546e9070 2024-09-21T07:26:09.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f4177400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c01b0 0x562e54a9a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f4177400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c01b0 0x562e54a9a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38634/0 (socket says 172.21.15.27:38634) 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f416fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54a91750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f4177400640 1 -- 172.21.15.27:0/175698883 learned_addr learned my addr 172.21.15.27:0/175698883 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f4177400640 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a9d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f416fe00640 1 -- 172.21.15.27:0/175698883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 msgr2=0x562e54a9d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f416fe00640 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a9d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f416fe00640 1 -- 172.21.15.27:0/175698883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c01b0 msgr2=0x562e54a9a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f416fe00640 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c01b0 0x562e54a9a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f416fe00640 1 -- 172.21.15.27:0/175698883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e54a92570 con 0x562e546c8390 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f4177400640 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a9d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.874+0000 7f416fe00640 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54a91750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f416001c890 tx=0x7f416000f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3aea26acdd19be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.875+0000 7f4166a00640 1 -- 172.21.15.27:0/175698883 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416000f9a0 con 0x562e546c8390 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.875+0000 7f4166a00640 1 -- 172.21.15.27:0/175698883 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416001c420 con 0x562e546c8390 2024-09-21T07:26:09.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.875+0000 7f4166a00640 1 -- 172.21.15.27:0/175698883 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41600035f0 con 0x562e546c8390 2024-09-21T07:26:09.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.875+0000 7f41843c6740 1 -- 172.21.15.27:0/175698883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 msgr2=0x562e54a91750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.875+0000 7f41843c6740 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54a91750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f416001c890 tx=0x7f416000f850 comp rx=0 tx=0).stop 2024-09-21T07:26:09.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 -- 172.21.15.27:0/175698883 shutdown_connections 2024-09-21T07:26:09.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546e9070 0x562e54a9d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54a91750 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 --2- 172.21.15.27:0/175698883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546c01b0 0x562e54a9a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 -- 172.21.15.27:0/175698883 >> 172.21.15.27:0/175698883 conn(0x562e546dd460 msgr2=0x562e54a9c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 -- 172.21.15.27:0/175698883 shutdown_connections 2024-09-21T07:26:09.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 -- 172.21.15.27:0/175698883 wait complete. 2024-09-21T07:26:09.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 -- start start 2024-09-21T07:26:09.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.876+0000 7f41843c6740 1 Processor -- start 2024-09-21T07:26:09.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f41843c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546c01b0 0x562e54aa0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f41843c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54aa0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f41843c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546e9070 0x562e54b11210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f41843c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f41781b1780 con 0x562e546e9070 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f41843c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41781b2bf0 con 0x562e546c8390 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f41843c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41781b1e50 con 0x562e546c01b0 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f41843c6740 10 client.0 _ll_register_callbacks cb 0x7f4160009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f416fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54aa0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f4177400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546e9070 0x562e54b11210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f416fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54aa0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55604/0 (socket says 172.21.15.27:55604) 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.877+0000 7f416fe00640 1 -- 172.21.15.27:0/3360576800 learned_addr learned my addr 172.21.15.27:0/3360576800 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f4177400640 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546c01b0 0x562e54aa0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f416fe00640 1 -- 172.21.15.27:0/3360576800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546c01b0 msgr2=0x562e54aa0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f416fe00640 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546c01b0 0x562e54aa0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f416fe00640 1 -- 172.21.15.27:0/3360576800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546e9070 msgr2=0x562e54b11210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f416fe00640 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546e9070 0x562e54b11210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f416fe00640 1 -- 172.21.15.27:0/3360576800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e54a9acc0 con 0x562e546c8390 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f4177400640 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546e9070 0x562e54b11210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f416fe00640 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54aa0800 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4160005510 tx=0x7f4160005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5941e39c87db70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f415be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.878+0000 7f415be00640 1 -- 172.21.15.27:0/3360576800 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416001ca30 con 0x562e546c8390 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416001c420 con 0x562e546c8390 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f415be00640 1 -- 172.21.15.27:0/3360576800 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f416001cbd0 con 0x562e546c8390 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f415be00640 1 -- 172.21.15.27:0/3360576800 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4160039050 con 0x562e546c8390 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 10 client.29429 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e54b18a30 con 0x562e546c8390 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f415be00640 1 -- 172.21.15.27:0/3360576800 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416001c420 con 0x562e546c8390 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f415be00640 1 -- 172.21.15.27:0/3360576800 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4160031020 con 0x562e546c8390 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 10 client.29429 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 10 client.29429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 20 client.29429 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e5467cda0 0x562e54b1b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 4 client.29429 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.879+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e54a04970 con 0x562e5467cda0 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.880+0000 7f4177400640 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e5467cda0 0x562e54b1b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.880+0000 7f4177400640 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e5467cda0 0x562e54b1b100 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfe2c3c521e1b28 server_cookie=d5db6d0b9a3dd3ad in_seq=0 out_seq=0 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.880+0000 7f415be00640 10 client.29429 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f415be00640 1 -- 172.21.15.27:0/3360576800 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x562e54a04970 con 0x562e5467cda0 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f415be00640 10 client.29429 handle_command_reply: tid=0 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 client.29429 shutdown 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 msgr2=0x562e54aa0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54aa0800 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4160005510 tx=0x7f4160005880 comp rx=0 tx=0).stop 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 shutdown_connections 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562e5467cda0 0x562e54b1b100 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562e546e9070 0x562e54b11210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e546c8390 0x562e54aa0800 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 --2- 172.21.15.27:0/3360576800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e546c01b0 0x562e54aa0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f415be00640 0 client.29429 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 >> 172.21.15.27:0/3360576800 conn(0x562e546dd460 msgr2=0x562e54a91520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 shutdown_connections 2024-09-21T07:26:09.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 1 -- 172.21.15.27:0/3360576800 wait complete. 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.881+0000 7f41843c6740 20 client.29429 trim_cache size 0 max 16384 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:26:09.886 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:26:09.892 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:26:09.893 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.895 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.896 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:26:09.897 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f415c0b6700 msgr2=0x7f415c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f415c0b6700 0x7f415c0b8c10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4168105220 tx=0x7f416810b970 comp rx=0 tx=0).stop 2024-09-21T07:26:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 msgr2=0x7f417819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f417819c680 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4168005940 tx=0x7f41680040c0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 shutdown_connections 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41781094a0 0x7f417819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f415c0b6700 0x7f415c0b8c10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4178109df0 0x7f41781a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 --2- 172.21.15.27:0/2083333547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178103b60 0x7f417819c680 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 >> 172.21.15.27:0/2083333547 conn(0x7f417807b850 msgr2=0x7f41781073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 shutdown_connections 2024-09-21T07:26:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:09.882+0000 7f417d600640 1 -- 172.21.15.27:0/2083333547 wait complete. 2024-09-21T07:26:10.734 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:26:10.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:26:11.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:10 smithi027 ceph-mon[30779]: pgmap v787: 129 pgs: 129 active+clean; 3.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-09-21T07:26:11.126 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:10 smithi153 ceph-mon[34747]: pgmap v787: 129 pgs: 129 active+clean; 3.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-09-21T07:26:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:10 smithi119 ceph-mon[34519]: pgmap v787: 129 pgs: 129 active+clean; 3.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 -- 172.21.15.27:0/3132155723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 msgr2=0x7f10b0112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 --2- 172.21.15.27:0/3132155723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 0x7f10b0112560 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009ca0 tx=0x7f10a802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 -- 172.21.15.27:0/3132155723 shutdown_connections 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 --2- 172.21.15.27:0/3132155723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 0x7f10b0112560 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 --2- 172.21.15.27:0/3132155723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b0076080 0x7f10b010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 --2- 172.21.15.27:0/3132155723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b0075730 0x7f10b0075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 -- 172.21.15.27:0/3132155723 >> 172.21.15.27:0/3132155723 conn(0x7f10b007b850 msgr2=0x7f10b007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 -- 172.21.15.27:0/3132155723 shutdown_connections 2024-09-21T07:26:11.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.788+0000 7f10b6c00640 1 -- 172.21.15.27:0/3132155723 wait complete. 2024-09-21T07:26:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.789+0000 7f10b6c00640 1 Processor -- start 2024-09-21T07:26:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b6c00640 1 -- start start 2024-09-21T07:26:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b6c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b0075730 0x7f10b019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b6c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b0076080 0x7f10b019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b6c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 0x7f10b01a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b6c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0115cf0 con 0x7f10b0110140 2024-09-21T07:26:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b6c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0115b70 con 0x7f10b0075730 2024-09-21T07:26:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b6c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0115e70 con 0x7f10b0076080 2024-09-21T07:26:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.790+0000 7f10b4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 0x7f10b01a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b0075730 0x7f10b019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10b4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 0x7f10b01a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38654/0 (socket says 172.21.15.27:38654) 2024-09-21T07:26:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10b4c00640 1 -- 172.21.15.27:0/424320589 learned_addr learned my addr 172.21.15.27:0/424320589 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10a7400640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b0076080 0x7f10b019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10a7400640 1 -- 172.21.15.27:0/424320589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b0075730 msgr2=0x7f10b019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10a7400640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b0075730 0x7f10b019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10a7400640 1 -- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 msgr2=0x7f10b01a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10a7400640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 0x7f10b01a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.791+0000 7f10a7400640 1 -- 172.21.15.27:0/424320589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b01a25b0 con 0x7f10b0076080 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.792+0000 7f10a7400640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b0076080 0x7f10b019cbe0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f10b006c9f0 tx=0x7f109800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.792+0000 7f10ad600640 1 -- 172.21.15.27:0/424320589 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109800cc40 con 0x7f10b0076080 2024-09-21T07:26:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.792+0000 7f10ad600640 1 -- 172.21.15.27:0/424320589 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109800cde0 con 0x7f10b0076080 2024-09-21T07:26:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.792+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b01a28a0 con 0x7f10b0076080 2024-09-21T07:26:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.792+0000 7f10ad600640 1 -- 172.21.15.27:0/424320589 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098018780 con 0x7f10b0076080 2024-09-21T07:26:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.792+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b01a9000 con 0x7f10b0076080 2024-09-21T07:26:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.793+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b0075b40 con 0x7f10b0076080 2024-09-21T07:26:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.797+0000 7f10ad600640 1 -- 172.21.15.27:0/424320589 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1098010a90 con 0x7f10b0076080 2024-09-21T07:26:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.801+0000 7f10ad600640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10940b6780 0x7f10940b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.801+0000 7f10ad600640 1 -- 172.21.15.27:0/424320589 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9549+0+0 (secure 0 0 0) 0x7f10980e8250 con 0x7f10b0076080 2024-09-21T07:26:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.801+0000 7f10afe00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10940b6780 0x7f10940b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.802+0000 7f10afe00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10940b6780 0x7f10940b8c90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009bd0 tx=0x7f10a0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:11.804+0000 7f10ad600640 1 -- 172.21.15.27:0/424320589 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10980b2720 con 0x7f10b0076080 2024-09-21T07:26:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.053+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10b019d770 con 0x7f10b0076080 2024-09-21T07:26:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.054+0000 7f10ad600640 1 -- 172.21.15.27:0/424320589 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f10b019d770 con 0x7f10b0076080 2024-09-21T07:26:12.056 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:26:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:26:12.056 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:26:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.056+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10940b6780 msgr2=0x7f10940b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.056+0000 7f10b6c00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10940b6780 0x7f10940b8c90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009bd0 tx=0x7f10a0005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.056+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b0076080 msgr2=0x7f10b019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.056+0000 7f10b6c00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b0076080 0x7f10b019cbe0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f10b006c9f0 tx=0x7f109800ee50 comp rx=0 tx=0).stop 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 shutdown_connections 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b0076080 0x7f10b019cbe0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10940b6780 0x7f10940b8c90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10b0110140 0x7f10b01a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 --2- 172.21.15.27:0/424320589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10b0075730 0x7f10b019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 >> 172.21.15.27:0/424320589 conn(0x7f10b007b850 msgr2=0x7f10b0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 shutdown_connections 2024-09-21T07:26:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:12.057+0000 7f10b6c00640 1 -- 172.21.15.27:0/424320589 wait complete. 2024-09-21T07:26:12.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:26:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:12 smithi027 ceph-mon[30779]: pgmap v788: 129 pgs: 129 active+clean; 3.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-09-21T07:26:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:12 smithi027 ceph-mon[30779]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T07:26:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:12 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/424320589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:12 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:26:13.168 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:12 smithi153 ceph-mon[34747]: pgmap v788: 129 pgs: 129 active+clean; 3.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-09-21T07:26:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:12 smithi153 ceph-mon[34747]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T07:26:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:12 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/424320589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:12 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:26:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:12 smithi119 ceph-mon[34519]: pgmap v788: 129 pgs: 129 active+clean; 3.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-09-21T07:26:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:12 smithi119 ceph-mon[34519]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T07:26:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:12 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/424320589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:12 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.783+0000 7feca9000640 1 -- 172.21.15.27:0/1271653574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 msgr2=0x7feca4075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.783+0000 7feca9000640 1 --2- 172.21.15.27:0/1271653574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 0x7feca4075b60 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fec94009900 tx=0x7fec9402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.783+0000 7feca9000640 1 -- 172.21.15.27:0/1271653574 shutdown_connections 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.783+0000 7feca9000640 1 --2- 172.21.15.27:0/1271653574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feca410ff70 0x7feca4112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.783+0000 7feca9000640 1 --2- 172.21.15.27:0/1271653574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 0x7feca410fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.783+0000 7feca9000640 1 --2- 172.21.15.27:0/1271653574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 0x7feca4075b60 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.783+0000 7feca9000640 1 -- 172.21.15.27:0/1271653574 >> 172.21.15.27:0/1271653574 conn(0x7feca40ff440 msgr2=0x7feca4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.784+0000 7feca9000640 1 -- 172.21.15.27:0/1271653574 shutdown_connections 2024-09-21T07:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.784+0000 7feca9000640 1 -- 172.21.15.27:0/1271653574 wait complete. 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 Processor -- start 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 -- start start 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 0x7feca419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 0x7feca419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feca410ff70 0x7feca41a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feca4115b30 con 0x7feca4075750 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feca41159b0 con 0x7feca40760a0 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.785+0000 7feca9000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feca4115cb0 con 0x7feca410ff70 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 0x7feca419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 0x7feca419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55640/0 (socket says 172.21.15.27:55640) 2024-09-21T07:26:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 -- 172.21.15.27:0/2081134126 learned_addr learned my addr 172.21.15.27:0/2081134126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca3e00640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feca410ff70 0x7feca41a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca3400640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 0x7feca419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 -- 172.21.15.27:0/2081134126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feca410ff70 msgr2=0x7feca41a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feca410ff70 0x7feca41a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 -- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 msgr2=0x7feca419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 0x7feca419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 -- 172.21.15.27:0/2081134126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca41a2490 con 0x7feca40760a0 2024-09-21T07:26:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.786+0000 7feca2a00640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 0x7feca419cac0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fec9000e9f0 tx=0x7fec9000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.787+0000 7fec9be00640 1 -- 172.21.15.27:0/2081134126 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9000ccc0 con 0x7feca40760a0 2024-09-21T07:26:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.787+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca41a2780 con 0x7feca40760a0 2024-09-21T07:26:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.787+0000 7fec9be00640 1 -- 172.21.15.27:0/2081134126 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec900108e0 con 0x7feca40760a0 2024-09-21T07:26:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.787+0000 7fec9be00640 1 -- 172.21.15.27:0/2081134126 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90018880 con 0x7feca40760a0 2024-09-21T07:26:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.787+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca41a8ee0 con 0x7feca40760a0 2024-09-21T07:26:13.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.790+0000 7fecb13ca740 1 Processor -- start 2024-09-21T07:26:13.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.790+0000 7fecb13ca740 1 -- start start 2024-09-21T07:26:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55784574a070 0x557845af0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845af1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845af17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7fecb13ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557845a67fb0 con 0x55784574a070 2024-09-21T07:26:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7fecb13ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557845a573e0 con 0x557845729390 2024-09-21T07:26:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7fecb13ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557845a5b1f0 con 0x5578457211b0 2024-09-21T07:26:13.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7fec9be00640 1 -- 172.21.15.27:0/2081134126 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fec90010430 con 0x7feca40760a0 2024-09-21T07:26:13.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7feca3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55784574a070 0x557845af0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7feca3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55784574a070 0x557845af0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38676/0 (socket says 172.21.15.27:38676) 2024-09-21T07:26:13.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7feca3e00640 1 -- 172.21.15.27:0/412324293 learned_addr learned my addr 172.21.15.27:0/412324293 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7feca3400640 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845af1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.791+0000 7feca2a00640 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845af17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.792+0000 7feca3e00640 1 -- 172.21.15.27:0/412324293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 msgr2=0x557845af1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.792+0000 7feca3e00640 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845af1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.792+0000 7feca3e00640 1 -- 172.21.15.27:0/412324293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 msgr2=0x557845af17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.792+0000 7feca3e00640 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845af17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.792+0000 7feca3e00640 1 -- 172.21.15.27:0/412324293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557845a21cd0 con 0x55784574a070 2024-09-21T07:26:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.792+0000 7feca2a00640 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845af17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.792+0000 7feca3e00640 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55784574a070 0x557845af0d30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fec9c011440 tx=0x7fec9c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221e81d67942f1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.793+0000 7fec98c00640 1 -- 172.21.15.27:0/412324293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c003c30 con 0x55784574a070 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.793+0000 7fec98c00640 1 -- 172.21.15.27:0/412324293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9c003dd0 con 0x55784574a070 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.793+0000 7fec98c00640 1 -- 172.21.15.27:0/412324293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00fae0 con 0x55784574a070 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.793+0000 7fecb13ca740 1 -- 172.21.15.27:0/412324293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55784574a070 msgr2=0x557845af0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.793+0000 7fecb13ca740 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55784574a070 0x557845af0d30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fec9c011440 tx=0x7fec9c011900 comp rx=0 tx=0).stop 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fecb13ca740 1 -- 172.21.15.27:0/412324293 shutdown_connections 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fecb13ca740 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845af1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fecb13ca740 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845af17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fecb13ca740 1 --2- 172.21.15.27:0/412324293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55784574a070 0x557845af0d30 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fecb13ca740 1 -- 172.21.15.27:0/412324293 >> 172.21.15.27:0/412324293 conn(0x55784573e460 msgr2=0x557845aeebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fecb13ca740 1 -- 172.21.15.27:0/412324293 shutdown_connections 2024-09-21T07:26:13.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fecb13ca740 1 -- 172.21.15.27:0/412324293 wait complete. 2024-09-21T07:26:13.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fec9be00640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec880b6590 0x7fec880b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7feca3400640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec880b6590 0x7fec880b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.794+0000 7fec9be00640 1 -- 172.21.15.27:0/2081134126 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fec900e7f40 con 0x7feca40760a0 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.795+0000 7fecb13ca740 1 -- start start 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.795+0000 7fecb13ca740 1 Processor -- start 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.795+0000 7feca3400640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec880b6590 0x7fec880b8aa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fec94000c00 tx=0x7fec94005a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845afc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845b08800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7fecb13ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5578458b8700 con 0x5578457211b0 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7fecb13ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5578459f2580 con 0x557845729390 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7fecb13ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557845aff510 con 0x55784574a070 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7fecb13ca740 10 client.0 _ll_register_callbacks cb 0x557845a02470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7feca2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7feca2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55666/0 (socket says 172.21.15.27:55666) 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7feca2a00640 1 -- 172.21.15.27:0/3942134953 learned_addr learned my addr 172.21.15.27:0/3942134953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7feca3e00640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845b08800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7feca2a00640 1 -- 172.21.15.27:0/3942134953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 msgr2=0x557845b08800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.796+0000 7feca2a00640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845b08800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.797+0000 7feca2a00640 1 -- 172.21.15.27:0/3942134953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 msgr2=0x557845afc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.797+0000 7feca3400640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845afc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.797+0000 7feca2a00640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845afc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.797+0000 7feca2a00640 1 -- 172.21.15.27:0/3942134953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578457b5650 con 0x557845729390 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.797+0000 7feca3400640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845afc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.797+0000 7feca2a00640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afcea0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec900ad680 tx=0x7fec900a9910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4172466fe514b475 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.797+0000 7fec8d600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:13.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.802+0000 7fec8d600640 1 -- 172.21.15.27:0/3942134953 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900ad0f0 con 0x557845729390 2024-09-21T07:26:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.802+0000 7fec8d600640 1 -- 172.21.15.27:0/3942134953 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fec900eb230 con 0x557845729390 2024-09-21T07:26:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.802+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5578459f4b90 con 0x557845729390 2024-09-21T07:26:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.803+0000 7fec8d600640 1 -- 172.21.15.27:0/3942134953 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec900e5710 con 0x557845729390 2024-09-21T07:26:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.803+0000 7fecb13ca740 10 client.29447 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.803+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557845a0f950 con 0x557845729390 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.803+0000 7fec8d600640 1 -- 172.21.15.27:0/3942134953 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5578459f4b90 con 0x557845729390 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7fec8d600640 1 -- 172.21.15.27:0/3942134953 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fec900ac650 con 0x557845729390 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7fecb13ca740 10 client.29447 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7fecb13ca740 10 client.29447 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7fecb13ca740 20 client.29447 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578457089f0 0x557845b0e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7fecb13ca740 4 client.29447 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557845a69f00 con 0x5578457089f0 2024-09-21T07:26:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.804+0000 7feca3400640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578457089f0 0x557845b0e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.805+0000 7feca3400640 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578457089f0 0x557845b0e840 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24953ff0f2dd777d server_cookie=3d73170fa8129cf5 in_seq=0 out_seq=0 2024-09-21T07:26:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.805+0000 7fec8d600640 10 client.29447 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fec8d600640 1 -- 172.21.15.27:0/3942134953 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557845a69f00 con 0x5578457089f0 2024-09-21T07:26:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fec8d600640 10 client.29447 handle_command_reply: tid=0 2024-09-21T07:26:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 client.29447 shutdown 2024-09-21T07:26:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 msgr2=0x557845afcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afcea0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec900ad680 tx=0x7fec900a9910 comp rx=0 tx=0).stop 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 shutdown_connections 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578457089f0 0x557845b0e840 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845b08800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afcea0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3942134953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845afc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fec8d600640 0 client.29447 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 >> 172.21.15.27:0/3942134953 conn(0x55784573e460 msgr2=0x557845af3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 shutdown_connections 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.806+0000 7fecb13ca740 1 -- 172.21.15.27:0/3942134953 wait complete. 2024-09-21T07:26:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.807+0000 7fecb13ca740 20 client.29447 trim_cache size 0 max 16384 2024-09-21T07:26:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.820+0000 7fecb13ca740 1 Processor -- start 2024-09-21T07:26:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7fecb13ca740 1 -- start start 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845af7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845aee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7fecb13ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feca41b5870 con 0x5578457211b0 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7fecb13ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feca41b3140 con 0x557845729390 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7fecb13ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feca41b0fb0 con 0x55784574a070 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7feca3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845af7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7feca3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845af7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38686/0 (socket says 172.21.15.27:38686) 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7feca3400640 1 -- 172.21.15.27:0/3902965292 learned_addr learned my addr 172.21.15.27:0/3902965292 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.821+0000 7feca2a00640 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca3e00640 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845aee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca3400640 1 -- 172.21.15.27:0/3902965292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 msgr2=0x557845aee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca3400640 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845aee210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca3400640 1 -- 172.21.15.27:0/3902965292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 msgr2=0x557845afe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca3400640 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca3400640 1 -- 172.21.15.27:0/3902965292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557845af3b90 con 0x5578457211b0 2024-09-21T07:26:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca2a00640 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.822+0000 7feca3400640 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845af7ed0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fec94005ab0 tx=0x7fec9403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbca48df6bef96c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.823+0000 7fec8fe00640 1 -- 172.21.15.27:0/3902965292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94056070 con 0x5578457211b0 2024-09-21T07:26:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.823+0000 7fec8fe00640 1 -- 172.21.15.27:0/3902965292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec940054e0 con 0x5578457211b0 2024-09-21T07:26:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.823+0000 7fec8fe00640 1 -- 172.21.15.27:0/3902965292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94035040 con 0x5578457211b0 2024-09-21T07:26:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 -- 172.21.15.27:0/3902965292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 msgr2=0x557845af7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845af7ed0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fec94005ab0 tx=0x7fec9403e000 comp rx=0 tx=0).stop 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 -- 172.21.15.27:0/3902965292 shutdown_connections 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55784574a070 0x557845aee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557845729390 0x557845afe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 --2- 172.21.15.27:0/3902965292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5578457211b0 0x557845af7ed0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 -- 172.21.15.27:0/3902965292 >> 172.21.15.27:0/3902965292 conn(0x55784573e460 msgr2=0x557845afdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 -- 172.21.15.27:0/3902965292 shutdown_connections 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.824+0000 7fecb13ca740 1 -- 172.21.15.27:0/3902965292 wait complete. 2024-09-21T07:26:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 -- start start 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 Processor -- start 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845b00e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557845729390 0x557845b01350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55784574a070 0x557845b78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feca41b1600 con 0x557845729390 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feca41b2a70 con 0x55784574a070 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feca41b1cd0 con 0x5578457211b0 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.825+0000 7fecb13ca740 10 client.0 _ll_register_callbacks cb 0x7fec900aa0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845b00e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557845729390 0x557845b01350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845b00e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54902/0 (socket says 172.21.15.27:54902) 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 -- 172.21.15.27:0/1478445191 learned_addr learned my addr 172.21.15.27:0/1478445191 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3e00640 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55784574a070 0x557845b78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 -- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55784574a070 msgr2=0x557845b78200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55784574a070 0x557845b78200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 -- 172.21.15.27:0/1478445191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557845729390 msgr2=0x557845b01350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557845729390 0x557845b01350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.826+0000 7feca3400640 1 -- 172.21.15.27:0/1478445191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557845b78740 con 0x5578457211b0 2024-09-21T07:26:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7feca3400640 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845b00e10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec9402a9e0 tx=0x7fec94004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cd24065abbc97c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7fec8d600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7fec8d600640 1 -- 172.21.15.27:0/1478445191 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94056070 con 0x5578457211b0 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec940054e0 con 0x5578457211b0 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7fec8d600640 1 -- 172.21.15.27:0/1478445191 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fec940035f0 con 0x5578457211b0 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7fec8d600640 1 -- 172.21.15.27:0/1478445191 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec94035070 con 0x5578457211b0 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7fecb13ca740 10 client.29407 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.827+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557845b79980 con 0x5578457211b0 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fec8d600640 1 -- 172.21.15.27:0/1478445191 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec940054e0 con 0x5578457211b0 2024-09-21T07:26:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fec8d600640 1 -- 172.21.15.27:0/1478445191 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fec940324b0 con 0x5578457211b0 2024-09-21T07:26:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fecb13ca740 10 client.29407 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fecb13ca740 10 client.29407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:26:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fecb13ca740 20 client.29407 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fecb13ca740 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578456ddda0 0x557845b7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fecb13ca740 4 client.29407 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:26:13.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557845afd230 con 0x5578456ddda0 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.828+0000 7feca2a00640 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578456ddda0 0x557845b7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7feca2a00640 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578456ddda0 0x557845b7c120 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=708e4a495928deb server_cookie=75a37838a7c76abe in_seq=0 out_seq=0 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fec8d600640 10 client.29407 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fec8d600640 1 -- 172.21.15.27:0/1478445191 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557845afd230 con 0x5578456ddda0 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fec8d600640 10 client.29407 handle_command_reply: tid=0 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 client.29407 shutdown 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 msgr2=0x557845b00e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845b00e10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec9402a9e0 tx=0x7fec94004610 comp rx=0 tx=0).stop 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 shutdown_connections 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5578456ddda0 0x557845b7c120 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55784574a070 0x557845b78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557845729390 0x557845b01350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 --2- 172.21.15.27:0/1478445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578457211b0 0x557845b00e10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 >> 172.21.15.27:0/1478445191 conn(0x55784573e460 msgr2=0x557845af2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.829+0000 7fec8d600640 0 client.29407 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.830+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 shutdown_connections 2024-09-21T07:26:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.830+0000 7fecb13ca740 1 -- 172.21.15.27:0/1478445191 wait complete. 2024-09-21T07:26:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.830+0000 7fecb13ca740 20 client.29407 trim_cache size 0 max 16384 2024-09-21T07:26:13.838 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:26:13.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec880b6590 msgr2=0x7fec880b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec880b6590 0x7fec880b8aa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fec94000c00 tx=0x7fec94005a40 comp rx=0 tx=0).stop 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 msgr2=0x7feca419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 0x7feca419cac0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fec9000e9f0 tx=0x7fec9000edf0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 shutdown_connections 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feca410ff70 0x7feca41a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fec880b6590 0x7fec880b8aa0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca40760a0 0x7feca419cac0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 --2- 172.21.15.27:0/2081134126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feca4075750 0x7feca419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 >> 172.21.15.27:0/2081134126 conn(0x7feca40ff440 msgr2=0x7feca4109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 shutdown_connections 2024-09-21T07:26:13.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:13.831+0000 7feca9000640 1 -- 172.21.15.27:0/2081134126 wait complete. 2024-09-21T07:26:13.855 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:26:13.855 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:26:13.855 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:26:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 -- 172.21.15.27:0/66041704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a00760a0 msgr2=0x7fe5a010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 --2- 172.21.15.27:0/66041704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a00760a0 0x7fe5a010fb70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009860 tx=0x7fe58c02a340 comp rx=0 tx=0).stop 2024-09-21T07:26:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 -- 172.21.15.27:0/66041704 shutdown_connections 2024-09-21T07:26:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 --2- 172.21.15.27:0/66041704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a00760a0 0x7fe5a010fb70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 --2- 172.21.15.27:0/66041704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a01100b0 0x7fe5a01124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 --2- 172.21.15.27:0/66041704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 0x7fe5a0075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 -- 172.21.15.27:0/66041704 >> 172.21.15.27:0/66041704 conn(0x7fe5a00ff500 msgr2=0x7fe5a0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 -- 172.21.15.27:0/66041704 shutdown_connections 2024-09-21T07:26:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.963+0000 7fe5a7600640 1 -- 172.21.15.27:0/66041704 wait complete. 2024-09-21T07:26:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.964+0000 7fe5a7600640 1 Processor -- start 2024-09-21T07:26:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.964+0000 7fe5a7600640 1 -- start start 2024-09-21T07:26:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a7600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 0x7fe5a019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a7600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a00760a0 0x7fe5a019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a7600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a01100b0 0x7fe5a01a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a7600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0115c20 con 0x7fe5a0075750 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a7600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0115aa0 con 0x7fe5a00760a0 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a7600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0115da0 con 0x7fe5a01100b0 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 0x7fe5a019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a4c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 0x7fe5a019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38710/0 (socket says 172.21.15.27:38710) 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a4c00640 1 -- 172.21.15.27:0/2638787353 learned_addr learned my addr 172.21.15.27:0/2638787353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe59fe00640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a00760a0 0x7fe5a019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a5600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a01100b0 0x7fe5a01a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a4c00640 1 -- 172.21.15.27:0/2638787353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a01100b0 msgr2=0x7fe5a01a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a01100b0 0x7fe5a01a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.965+0000 7fe5a4c00640 1 -- 172.21.15.27:0/2638787353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a00760a0 msgr2=0x7fe5a019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a00760a0 0x7fe5a019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe5a4c00640 1 -- 172.21.15.27:0/2638787353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a01a2550 con 0x7fe5a0075750 2024-09-21T07:26:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 0x7fe5a019c620 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe59400e9c0 tx=0x7fe59400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe59d600640 1 -- 172.21.15.27:0/2638787353 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59400cde0 con 0x7fe5a0075750 2024-09-21T07:26:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe59d600640 1 -- 172.21.15.27:0/2638787353 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940035f0 con 0x7fe5a0075750 2024-09-21T07:26:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a01a2840 con 0x7fe5a0075750 2024-09-21T07:26:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe59d600640 1 -- 172.21.15.27:0/2638787353 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594010690 con 0x7fe5a0075750 2024-09-21T07:26:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.966+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a01a8fd0 con 0x7fe5a0075750 2024-09-21T07:26:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.969+0000 7fe5ae68f740 1 Processor -- start 2024-09-21T07:26:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5ae68f740 1 -- start start 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe59d600640 1 -- 172.21.15.27:0/2638787353 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe594004000 con 0x7fe5a0075750 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c8808286b0 con 0x55c88050e1b0 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c880817ae0 con 0x55c8804ed4d0 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c88081b8f0 con 0x55c8804e52f0 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5a4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38712/0 (socket says 172.21.15.27:38712) 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe5a5600640 1 -- 172.21.15.27:0/3579689602 learned_addr learned my addr 172.21.15.27:0/3579689602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.970+0000 7fe59fe00640 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.971+0000 7fe5a4c00640 1 -- 172.21.15.27:0/3579689602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 msgr2=0x55c8808b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.971+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.971+0000 7fe5a4c00640 1 -- 172.21.15.27:0/3579689602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 msgr2=0x55c8808b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.971+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.971+0000 7fe5a4c00640 1 -- 172.21.15.27:0/3579689602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8807e4780 con 0x55c8804e52f0 2024-09-21T07:26:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.971+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808b3170 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe5940e5160 tx=0x7fe5940f04b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551c81ae309b25d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.972+0000 7fe591c00640 1 -- 172.21.15.27:0/3579689602 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940fb070 con 0x55c8804e52f0 2024-09-21T07:26:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.972+0000 7fe591c00640 1 -- 172.21.15.27:0/3579689602 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940f68b0 con 0x55c8804e52f0 2024-09-21T07:26:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.972+0000 7fe59d600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe5880b6680 0x7fe5880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.972+0000 7fe591c00640 1 -- 172.21.15.27:0/3579689602 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940f5800 con 0x55c8804e52f0 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.972+0000 7fe59d600640 1 -- 172.21.15.27:0/2638787353 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fe5940e7e50 con 0x7fe5a0075750 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.972+0000 7fe5a5600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe5880b6680 0x7fe5880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3579689602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 msgr2=0x55c8808b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808b3170 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe5940e5160 tx=0x7fe5940f04b0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5a5600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe5880b6680 0x7fe5880b8b90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe5980116b0 tx=0x7fe598009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3579689602 shutdown_connections 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808b3170 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3579689602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3579689602 >> 172.21.15.27:0/3579689602 conn(0x55c880502300 msgr2=0x55c8808b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3579689602 shutdown_connections 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.973+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3579689602 wait complete. 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 -- start start 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 Processor -- start 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8804ed4d0 0x55c8808be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c88050e1b0 0x55c8808ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c88074fab0 con 0x55c8804ed4d0 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c8807be570 con 0x55c88050e1b0 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c8808c13b0 con 0x55c8804e52f0 2024-09-21T07:26:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.974+0000 7fe5ae68f740 10 client.0 _ll_register_callbacks cb 0x55c8807cd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe59fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8804ed4d0 0x55c8808be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c88050e1b0 0x55c8808ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808be270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54952/0 (socket says 172.21.15.27:54952) 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe59fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8804ed4d0 0x55c8808be7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38720/0 (socket says 172.21.15.27:38720) 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 -- 172.21.15.27:0/1119627198 learned_addr learned my addr 172.21.15.27:0/1119627198 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 -- 172.21.15.27:0/1119627198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c88050e1b0 msgr2=0x55c8808ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c88050e1b0 0x55c8808ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 -- 172.21.15.27:0/1119627198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8804ed4d0 msgr2=0x55c8808be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8804ed4d0 0x55c8808be7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.975+0000 7fe5a4c00640 1 -- 172.21.15.27:0/1119627198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c880579790 con 0x55c8804e52f0 2024-09-21T07:26:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808be270 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe594009230 tx=0x7fe5940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a81d4635938152b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe586000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe586000640 1 -- 172.21.15.27:0/1119627198 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940a9ee0 con 0x55c8804e52f0 2024-09-21T07:26:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5940f68b0 con 0x55c8804e52f0 2024-09-21T07:26:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe586000640 1 -- 172.21.15.27:0/1119627198 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fe5940037a0 con 0x55c8804e52f0 2024-09-21T07:26:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe586000640 1 -- 172.21.15.27:0/1119627198 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5940042f0 con 0x55c8804e52f0 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe5ae68f740 10 client.29419 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8805f2620 con 0x55c8804e52f0 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.976+0000 7fe586000640 1 -- 172.21.15.27:0/1119627198 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940f68b0 con 0x55c8804e52f0 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe586000640 1 -- 172.21.15.27:0/1119627198 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe5940e5410 con 0x55c8804e52f0 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe5ae68f740 10 client.29419 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe5ae68f740 10 client.29419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe5ae68f740 20 client.29419 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804ccd80 0x55c8808d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe5ae68f740 4 client.29419 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c88082a600 con 0x55c8804ccd80 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.977+0000 7fe59fe00640 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804ccd80 0x55c8808d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.978+0000 7fe59fe00640 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804ccd80 0x55c8808d0220 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a56423240d05d5c server_cookie=bca97b43c3143c5e in_seq=0 out_seq=0 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.978+0000 7fe586000640 10 client.29419 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.978+0000 7fe586000640 1 -- 172.21.15.27:0/1119627198 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c88082a600 con 0x55c8804ccd80 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.978+0000 7fe586000640 10 client.29419 handle_command_reply: tid=0 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.978+0000 7fe5ae68f740 1 client.29419 shutdown 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 msgr2=0x55c8808be270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808be270 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe594009230 tx=0x7fe5940046c0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 shutdown_connections 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804ccd80 0x55c8808d0220 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c88050e1b0 0x55c8808ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8804ed4d0 0x55c8808be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe586000640 0 client.29419 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/1119627198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808be270 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 >> 172.21.15.27:0/1119627198 conn(0x55c880502300 msgr2=0x55c8808b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 shutdown_connections 2024-09-21T07:26:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 1 -- 172.21.15.27:0/1119627198 wait complete. 2024-09-21T07:26:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.979+0000 7fe5ae68f740 20 client.29419 trim_cache size 0 max 16384 2024-09-21T07:26:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 Processor -- start 2024-09-21T07:26:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 -- start start 2024-09-21T07:26:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804e52f0 0x55c8808b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804ed4d0 0x55c8808ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b5940 con 0x55c88050e1b0 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b3210 con 0x55c8804e52f0 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b1080 con 0x55c8804ed4d0 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808bb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38724/0 (socket says 172.21.15.27:38724) 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5a5600640 1 -- 172.21.15.27:0/2945174445 learned_addr learned my addr 172.21.15.27:0/2945174445 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe59fe00640 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804ed4d0 0x55c8808ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.995+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804e52f0 0x55c8808b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe59fe00640 1 -- 172.21.15.27:0/2945174445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804e52f0 msgr2=0x55c8808b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe59fe00640 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804e52f0 0x55c8808b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe59fe00640 1 -- 172.21.15.27:0/2945174445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 msgr2=0x55c8808bb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe59fe00640 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808bb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe59fe00640 1 -- 172.21.15.27:0/2945174445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8808aad50 con 0x55c8804ed4d0 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe59fe00640 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804ed4d0 0x55c8808ca640 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe58c035040 tx=0x7fe58c0058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c722ef9c725966 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe591200640 1 -- 172.21.15.27:0/2945174445 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c04d070 con 0x55c8804ed4d0 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe591200640 1 -- 172.21.15.27:0/2945174445 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe58c030e70 con 0x55c8804ed4d0 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.996+0000 7fe591200640 1 -- 172.21.15.27:0/2945174445 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c037040 con 0x55c8804ed4d0 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 -- 172.21.15.27:0/2945174445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804ed4d0 msgr2=0x55c8808ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804ed4d0 0x55c8808ca640 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe58c035040 tx=0x7fe58c0058d0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 -- 172.21.15.27:0/2945174445 shutdown_connections 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804ed4d0 0x55c8808ca640 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c8808bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/2945174445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804e52f0 0x55c8808b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 -- 172.21.15.27:0/2945174445 >> 172.21.15.27:0/2945174445 conn(0x55c880502300 msgr2=0x55c8808b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 -- 172.21.15.27:0/2945174445 shutdown_connections 2024-09-21T07:26:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.997+0000 7fe5ae68f740 1 -- 172.21.15.27:0/2945174445 wait complete. 2024-09-21T07:26:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 -- start start 2024-09-21T07:26:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 Processor -- start 2024-09-21T07:26:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c880932970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b16d0 con 0x55c88050e1b0 2024-09-21T07:26:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b2b40 con 0x55c8804ed4d0 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.998+0000 7fe5ae68f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b1da0 con 0x55c8804e52f0 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5ae68f740 10 client.0 _ll_register_callbacks cb 0x7fe5940f9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c880932970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe59fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a4c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c880932970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38728/0 (socket says 172.21.15.27:38728) 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a5600640 1 -- 172.21.15.27:0/3986435448 learned_addr learned my addr 172.21.15.27:0/3986435448 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe59fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808d30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55744/0 (socket says 172.21.15.27:55744) 2024-09-21T07:26:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a4c00640 1 -- 172.21.15.27:0/3986435448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 msgr2=0x55c8808d30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808d30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a4c00640 1 -- 172.21.15.27:0/3986435448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 msgr2=0x55c880932970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c880932970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a4c00640 1 -- 172.21.15.27:0/3986435448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c880932fe0 con 0x55c8804e52f0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe5a4c00640 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808d2b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe5940aab00 tx=0x7fe5940b2480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48f20038ec128ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:13.999+0000 7fe586000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe586000640 1 -- 172.21.15.27:0/3986435448 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940ad000 con 0x55c8804e52f0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe586000640 1 -- 172.21.15.27:0/3986435448 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fe5940037a0 con 0x55c8804e52f0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c8808260b0 con 0x55c8804e52f0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe586000640 1 -- 172.21.15.27:0/3986435448 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5940ad1e0 con 0x55c8804e52f0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 10 client.29425 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c88093a310 con 0x55c8804e52f0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe586000640 1 -- 172.21.15.27:0/3986435448 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c8808260b0 con 0x55c8804e52f0 2024-09-21T07:26:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe586000640 1 -- 172.21.15.27:0/3986435448 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe5940a91e0 con 0x55c8804e52f0 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 10 client.29425 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 10 client.29425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 20 client.29425 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804a21c0 0x55c88093c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 4 client.29425 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.000+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8808234e0 con 0x55c8804a21c0 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.001+0000 7fe59fe00640 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804a21c0 0x55c88093c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.001+0000 7fe59fe00640 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804a21c0 0x55c88093c7f0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0ee1df2de3be20 server_cookie=f983b58985fba852 in_seq=0 out_seq=0 2024-09-21T07:26:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.001+0000 7fe586000640 10 client.29425 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe586000640 1 -- 172.21.15.27:0/3986435448 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8808234e0 con 0x55c8804a21c0 2024-09-21T07:26:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe586000640 10 client.29425 handle_command_reply: tid=0 2024-09-21T07:26:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 client.29425 shutdown 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 msgr2=0x55c8808d2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808d2b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe5940aab00 tx=0x7fe5940b2480 comp rx=0 tx=0).stop 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 shutdown_connections 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c8804a21c0 0x55c88093c7f0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c88050e1b0 0x55c880932970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8804ed4d0 0x55c8808d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 --2- 172.21.15.27:0/3986435448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c8804e52f0 0x55c8808d2b30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe586000640 0 client.29425 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 >> 172.21.15.27:0/3986435448 conn(0x55c880502300 msgr2=0x55c8808b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 shutdown_connections 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.013+0000 7fe5ae68f740 1 -- 172.21.15.27:0/3986435448 wait complete. 2024-09-21T07:26:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5ae68f740 20 client.29425 trim_cache size 0 max 16384 2024-09-21T07:26:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:26:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:26:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:26:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:26:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe5880b6680 msgr2=0x7fe5880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe5880b6680 0x7fe5880b8b90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe5980116b0 tx=0x7fe598009500 comp rx=0 tx=0).stop 2024-09-21T07:26:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 msgr2=0x7fe5a019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 0x7fe5a019c620 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe59400e9c0 tx=0x7fe59400ee80 comp rx=0 tx=0).stop 2024-09-21T07:26:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 shutdown_connections 2024-09-21T07:26:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a01100b0 0x7fe5a01a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe5880b6680 0x7fe5880b8b90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5a00760a0 0x7fe5a019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 --2- 172.21.15.27:0/2638787353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a0075750 0x7fe5a019c620 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.014+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 >> 172.21.15.27:0/2638787353 conn(0x7fe5a00ff500 msgr2=0x7fe5a0109c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.015+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 shutdown_connections 2024-09-21T07:26:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:14.015+0000 7fe5a7600640 1 -- 172.21.15.27:0/2638787353 wait complete. 2024-09-21T07:26:14.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:26:14.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:26:14.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:26:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:26:14.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:26:14.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:26:14.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:26:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:15.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:15 smithi027 ceph-mon[30779]: pgmap v790: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:26:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:15 smithi153 ceph-mon[34747]: pgmap v790: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:26:15.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:15 smithi119 ceph-mon[34519]: pgmap v790: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:26:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 -- 172.21.15.27:0/2047794261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 msgr2=0x7f1ea0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/2047794261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea0112840 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ca0 tx=0x7f1e9802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 -- 172.21.15.27:0/2047794261 shutdown_connections 2024-09-21T07:26:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/2047794261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea0112840 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/2047794261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea010b6c0 0x7f1ea010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/2047794261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea0105d80 0x7f1ea0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 -- 172.21.15.27:0/2047794261 >> 172.21.15.27:0/2047794261 conn(0x7f1ea0101810 msgr2=0x7f1ea0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 -- 172.21.15.27:0/2047794261 shutdown_connections 2024-09-21T07:26:15.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.792+0000 7f1ea7a00640 1 -- 172.21.15.27:0/2047794261 wait complete. 2024-09-21T07:26:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.793+0000 7f1ea7a00640 1 Processor -- start 2024-09-21T07:26:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea7a00640 1 -- start start 2024-09-21T07:26:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea7a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea0105d80 0x7f1ea019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea7a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea010b6c0 0x7f1ea019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea7a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea7a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0115cd0 con 0x7f1ea010c010 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea7a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0115b50 con 0x7f1ea010b6c0 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea7a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0115e50 con 0x7f1ea0105d80 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea01a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38740/0 (socket says 172.21.15.27:38740) 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea5a00640 1 -- 172.21.15.27:0/53389028 learned_addr learned my addr 172.21.15.27:0/53389028 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1e97e00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea010b6c0 0x7f1ea019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.794+0000 7f1ea5000640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea0105d80 0x7f1ea019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.795+0000 7f1ea5a00640 1 -- 172.21.15.27:0/53389028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea0105d80 msgr2=0x7f1ea019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.795+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea0105d80 0x7f1ea019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.795+0000 7f1ea5a00640 1 -- 172.21.15.27:0/53389028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea010b6c0 msgr2=0x7f1ea019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.795+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea010b6c0 0x7f1ea019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.795+0000 7f1ea5a00640 1 -- 172.21.15.27:0/53389028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea01a25d0 con 0x7f1ea010c010 2024-09-21T07:26:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.795+0000 7f1e97e00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea010b6c0 0x7f1ea019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.795+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea01a1ea0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1e98005cc0 tx=0x7f1e98031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.796+0000 7f1e9e000640 1 -- 172.21.15.27:0/53389028 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98003b70 con 0x7f1ea010c010 2024-09-21T07:26:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.796+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea01a2860 con 0x7f1ea010c010 2024-09-21T07:26:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.796+0000 7f1e9e000640 1 -- 172.21.15.27:0/53389028 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e980318f0 con 0x7f1ea010c010 2024-09-21T07:26:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.796+0000 7f1e9e000640 1 -- 172.21.15.27:0/53389028 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9802f950 con 0x7f1ea010c010 2024-09-21T07:26:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.796+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea01a9020 con 0x7f1ea010c010 2024-09-21T07:26:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.799+0000 7f1eae620740 1 Processor -- start 2024-09-21T07:26:15.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1e9e000640 1 -- 172.21.15.27:0/53389028 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1e9802fb50 con 0x7f1ea010c010 2024-09-21T07:26:15.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1eae620740 1 -- start start 2024-09-21T07:26:15.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1eae620740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c10da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1eae620740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb48411b0 0x55ffb4c112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1eae620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c11820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1eae620740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffb4b87fb0 con 0x55ffb486a070 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1eae620740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ffb4b773e0 con 0x55ffb48411b0 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.800+0000 7f1eae620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ffb4b7b1f0 con 0x55ffb4849390 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb48411b0 0x55ffb4c112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1e97e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c10da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb48411b0 0x55ffb4c112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55776/0 (socket says 172.21.15.27:55776) 2024-09-21T07:26:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 -- 172.21.15.27:0/3704490787 learned_addr learned my addr 172.21.15.27:0/3704490787 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c11820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 -- 172.21.15.27:0/3704490787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 msgr2=0x55ffb4c11820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c11820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 -- 172.21.15.27:0/3704490787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 msgr2=0x55ffb4c10da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c10da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1ea5000640 1 -- 172.21.15.27:0/3704490787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb4b41cd0 con 0x55ffb48411b0 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.801+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c11820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.802+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb48411b0 0x55ffb4c112e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009d20 tx=0x7f1e90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e07df56759005d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.802+0000 7f1e9e000640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1e840b6700 0x7f1e840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.802+0000 7f1e96000640 1 -- 172.21.15.27:0/3704490787 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9001e070 con 0x55ffb48411b0 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.802+0000 7f1e96000640 1 -- 172.21.15.27:0/3704490787 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e90019ae0 con 0x55ffb48411b0 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.802+0000 7f1e96000640 1 -- 172.21.15.27:0/3704490787 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000eac0 con 0x55ffb48411b0 2024-09-21T07:26:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.802+0000 7f1e9e000640 1 -- 172.21.15.27:0/53389028 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f1e98108b40 con 0x7f1ea010c010 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.802+0000 7f1e97e00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1e840b6700 0x7f1e840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1e97e00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1e840b6700 0x7f1e840b8c10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009fd0 tx=0x7f1e88005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 -- 172.21.15.27:0/3704490787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb48411b0 msgr2=0x55ffb4c112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb48411b0 0x55ffb4c112e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009d20 tx=0x7f1e90011810 comp rx=0 tx=0).stop 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 -- 172.21.15.27:0/3704490787 shutdown_connections 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c11820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb48411b0 0x55ffb4c112e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 --2- 172.21.15.27:0/3704490787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c10da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 -- 172.21.15.27:0/3704490787 >> 172.21.15.27:0/3704490787 conn(0x55ffb485e460 msgr2=0x55ffb4c0ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 -- 172.21.15.27:0/3704490787 shutdown_connections 2024-09-21T07:26:15.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.803+0000 7f1eae620740 1 -- 172.21.15.27:0/3704490787 wait complete. 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.804+0000 7f1eae620740 1 -- start start 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.804+0000 7f1eae620740 1 Processor -- start 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1eae620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c1c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1eae620740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb4849390 0x55ffb4c1cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1eae620740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1eae620740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffb49d8700 con 0x55ffb4849390 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1eae620740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ffb4b12580 con 0x55ffb486a070 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1eae620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ffb4c1f580 con 0x55ffb48411b0 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1eae620740 10 client.0 _ll_register_callbacks cb 0x55ffb4b22470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c1c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1e97e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb4849390 0x55ffb4c1cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c1c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55012/0 (socket says 172.21.15.27:55012) 2024-09-21T07:26:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 -- 172.21.15.27:0/3499478329 learned_addr learned my addr 172.21.15.27:0/3499478329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 -- 172.21.15.27:0/3499478329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 msgr2=0x55ffb4c28870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 -- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb4849390 msgr2=0x55ffb4c1cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb4849390 0x55ffb4c1cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1ea5000640 1 -- 172.21.15.27:0/3499478329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb48d5650 con 0x55ffb48411b0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.805+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb4849390 0x55ffb4c1cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c1c5d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e760 tx=0x7f1e9000ea30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7518b50d7dcbfec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1e8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3499478329 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9001e030 con 0x55ffb48411b0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffb4b14b90 con 0x55ffb48411b0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3499478329 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f1e9000ee30 con 0x55ffb48411b0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3499478329 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e90030050 con 0x55ffb48411b0 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1eae620740 10 client.29434 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.806+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffb4b2f950 con 0x55ffb48411b0 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3499478329 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ffb4b14b90 con 0x55ffb48411b0 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3499478329 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1e900047e0 con 0x55ffb48411b0 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1eae620740 10 client.29434 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1eae620740 10 client.29434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1eae620740 20 client.29434 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1eae620740 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb48289f0 0x55ffb4c2e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1eae620740 4 client.29434 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.807+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffb4b89f00 con 0x55ffb48289f0 2024-09-21T07:26:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.808+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb48289f0 0x55ffb4c2e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.808+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb48289f0 0x55ffb4c2e510 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cffcf9a792e446c0 server_cookie=e6a5e0e00e095f79 in_seq=0 out_seq=0 2024-09-21T07:26:15.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.808+0000 7f1e8ea00640 10 client.29434 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:15.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.809+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3499478329 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffb4b89f00 con 0x55ffb48289f0 2024-09-21T07:26:15.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.809+0000 7f1e8ea00640 10 client.29434 handle_command_reply: tid=0 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.809+0000 7f1eae620740 1 client.29434 shutdown 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.809+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 msgr2=0x55ffb4c1c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.809+0000 7f1eae620740 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c1c5d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e760 tx=0x7f1e9000ea30 comp rx=0 tx=0).stop 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.809+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 shutdown_connections 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.809+0000 7f1eae620740 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb48289f0 0x55ffb4c2e510 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1eae620740 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1eae620740 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb4849390 0x55ffb4c1cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1eae620740 1 --2- 172.21.15.27:0/3499478329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c1c5d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 >> 172.21.15.27:0/3499478329 conn(0x55ffb485e460 msgr2=0x55ffb4c13f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1e8ea00640 0 client.29434 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 shutdown_connections 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1eae620740 1 -- 172.21.15.27:0/3499478329 wait complete. 2024-09-21T07:26:15.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.810+0000 7f1eae620740 20 client.29434 trim_cache size 0 max 16384 2024-09-21T07:26:15.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 Processor -- start 2024-09-21T07:26:15.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 -- start start 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb48411b0 0x55ffb4c12790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c28770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b5960 con 0x55ffb48411b0 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b3230 con 0x55ffb486a070 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.824+0000 7f1eae620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b10a0 con 0x55ffb4849390 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb48411b0 0x55ffb4c12790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb48411b0 0x55ffb4c12790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38774/0 (socket says 172.21.15.27:38774) 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5000640 1 -- 172.21.15.27:0/3838349677 learned_addr learned my addr 172.21.15.27:0/3838349677 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c28770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5a00640 1 -- 172.21.15.27:0/3838349677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 msgr2=0x55ffb4c28770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c28770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5a00640 1 -- 172.21.15.27:0/3838349677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb48411b0 msgr2=0x55ffb4c12790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb48411b0 0x55ffb4c12790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.825+0000 7f1ea5a00640 1 -- 172.21.15.27:0/3838349677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb4c1f460 con 0x55ffb486a070 2024-09-21T07:26:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.826+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28cb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1e98105eb0 tx=0x7f1e980ce470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1fc6d6e3f0e43e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.826+0000 7f1e95600640 1 -- 172.21.15.27:0/3838349677 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980cece0 con 0x55ffb486a070 2024-09-21T07:26:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.826+0000 7f1e95600640 1 -- 172.21.15.27:0/3838349677 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e98036040 con 0x55ffb486a070 2024-09-21T07:26:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.826+0000 7f1e95600640 1 -- 172.21.15.27:0/3838349677 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980d7b70 con 0x55ffb486a070 2024-09-21T07:26:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 -- 172.21.15.27:0/3838349677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 msgr2=0x55ffb4c28cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28cb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1e98105eb0 tx=0x7f1e980ce470 comp rx=0 tx=0).stop 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 -- 172.21.15.27:0/3838349677 shutdown_connections 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb4849390 0x55ffb4c28770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb486a070 0x55ffb4c28cb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 --2- 172.21.15.27:0/3838349677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb48411b0 0x55ffb4c12790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 -- 172.21.15.27:0/3838349677 >> 172.21.15.27:0/3838349677 conn(0x55ffb485e460 msgr2=0x55ffb4c1daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 -- 172.21.15.27:0/3838349677 shutdown_connections 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 -- 172.21.15.27:0/3838349677 wait complete. 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.827+0000 7f1eae620740 1 -- start start 2024-09-21T07:26:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 1 Processor -- start 2024-09-21T07:26:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb4849390 0x55ffb4c22ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b16f0 con 0x55ffb486a070 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b2b60 con 0x55ffb4849390 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b1dc0 con 0x55ffb48411b0 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1eae620740 10 client.0 _ll_register_callbacks cb 0x7f1e90009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.828+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55036/0 (socket says 172.21.15.27:55036) 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 -- 172.21.15.27:0/3534056906 learned_addr learned my addr 172.21.15.27:0/3534056906 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb4849390 0x55ffb4c22ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 -- 172.21.15.27:0/3534056906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb4849390 msgr2=0x55ffb4c22ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb4849390 0x55ffb4c22ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 -- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 msgr2=0x55ffb4c23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c23430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 -- 172.21.15.27:0/3534056906 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb4c97ff0 con 0x55ffb48411b0 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5a00640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1ea5000640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c229b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1e90006010 tx=0x7f1e900096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccea74bb3584d1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.829+0000 7f1e8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3534056906 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90039060 con 0x55ffb48411b0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e98036040 con 0x55ffb48411b0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3534056906 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f1e90004440 con 0x55ffb48411b0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3534056906 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e90040050 con 0x55ffb48411b0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1eae620740 10 client.29440 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffb4c99250 con 0x55ffb48411b0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3534056906 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e98036040 con 0x55ffb48411b0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3534056906 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1e900051b0 con 0x55ffb48411b0 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.830+0000 7f1eae620740 10 client.29440 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1eae620740 10 client.29440 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1eae620740 20 client.29440 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1eae620740 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb47fdda0 0x55ffb4c9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1eae620740 4 client.29440 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:26:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffb4b859b0 con 0x55ffb47fdda0 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb47fdda0 0x55ffb4c9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1e97e00640 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb47fdda0 0x55ffb4c9b9e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d565e603d42bf48a server_cookie=f80bb5ccb16e44d4 in_seq=0 out_seq=0 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.831+0000 7f1e8ea00640 10 client.29440 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1e8ea00640 1 -- 172.21.15.27:0/3534056906 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffb4b859b0 con 0x55ffb47fdda0 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1e8ea00640 10 client.29440 handle_command_reply: tid=0 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 client.29440 shutdown 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 msgr2=0x55ffb4c229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c229b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1e90006010 tx=0x7f1e900096a0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 shutdown_connections 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ffb47fdda0 0x55ffb4c9b9e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffb486a070 0x55ffb4c23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ffb4849390 0x55ffb4c22ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 --2- 172.21.15.27:0/3534056906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ffb48411b0 0x55ffb4c229b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1e8ea00640 0 client.29440 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:26:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 >> 172.21.15.27:0/3534056906 conn(0x55ffb485e460 msgr2=0x55ffb4c12560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 shutdown_connections 2024-09-21T07:26:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 1 -- 172.21.15.27:0/3534056906 wait complete. 2024-09-21T07:26:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.832+0000 7f1eae620740 20 client.29440 trim_cache size 0 max 16384 2024-09-21T07:26:15.837 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1e840b6700 msgr2=0x7f1e840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1e840b6700 0x7f1e840b8c10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009fd0 tx=0x7f1e88005850 comp rx=0 tx=0).stop 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 msgr2=0x7f1ea01a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea01a1ea0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1e98005cc0 tx=0x7f1e98031700 comp rx=0 tx=0).stop 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 shutdown_connections 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1e840b6700 0x7f1e840b8c10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea010c010 0x7f1ea01a1ea0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea010b6c0 0x7f1ea019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 --2- 172.21.15.27:0/53389028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea0105d80 0x7f1ea019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 >> 172.21.15.27:0/53389028 conn(0x7f1ea0101810 msgr2=0x7f1ea0108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 shutdown_connections 2024-09-21T07:26:15.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:15.833+0000 7f1ea7a00640 1 -- 172.21.15.27:0/53389028 wait complete. 2024-09-21T07:26:16.639 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:26:17.027 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:17.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:17 smithi027 ceph-mon[30779]: pgmap v791: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:26:17.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:17 smithi153 ceph-mon[34747]: pgmap v791: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:26:17.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:17 smithi119 ceph-mon[34519]: pgmap v791: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-21T07:26:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2194239695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 msgr2=0x7fe8bc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2194239695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 0x7fe8bc106190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009860 tx=0x7fe8ac02a340 comp rx=0 tx=0).stop 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2194239695 shutdown_connections 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2194239695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10c010 0x7fe8bc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2194239695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8bc10b6c0 0x7fe8bc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2194239695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 0x7fe8bc106190 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2194239695 >> 172.21.15.27:0/2194239695 conn(0x7fe8bc101810 msgr2=0x7fe8bc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2194239695 shutdown_connections 2024-09-21T07:26:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.746+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2194239695 wait complete. 2024-09-21T07:26:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.747+0000 7fe8c1a00640 1 Processor -- start 2024-09-21T07:26:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8c1a00640 1 -- start start 2024-09-21T07:26:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8c1a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 0x7fe8bc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8c1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10b6c0 0x7fe8bc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8c1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8bc10c010 0x7fe8bc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8c1a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc115cd0 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8c1a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc115b50 con 0x7fe8bc10c010 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8c1a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc115e50 con 0x7fe8bc105d80 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10b6c0 0x7fe8bc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.748+0000 7fe8b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10b6c0 0x7fe8bc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38792/0 (socket says 172.21.15.27:38792) 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8b3e00640 1 -- 172.21.15.27:0/2485826606 learned_addr learned my addr 172.21.15.27:0/2485826606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 0x7fe8bc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8bb400640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8bc10c010 0x7fe8bc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8b3e00640 1 -- 172.21.15.27:0/2485826606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 msgr2=0x7fe8bc19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 0x7fe8bc19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8b3e00640 1 -- 172.21.15.27:0/2485826606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8bc10c010 msgr2=0x7fe8bc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8bc10c010 0x7fe8bc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8b3e00640 1 -- 172.21.15.27:0/2485826606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc1a2650 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.749+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10b6c0 0x7fe8bc19cc80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e9f0 tx=0x7fe8a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.750+0000 7fe8b3400640 1 -- 172.21.15.27:0/2485826606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400ccc0 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.750+0000 7fe8b3400640 1 -- 172.21.15.27:0/2485826606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a40108e0 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.750+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc1a2940 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.750+0000 7fe8b3400640 1 -- 172.21.15.27:0/2485826606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4018860 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.750+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc1a90a0 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.753+0000 7fe8c8604740 1 Processor -- start 2024-09-21T07:26:17.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.753+0000 7fe8c8604740 1 -- start start 2024-09-21T07:26:17.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c28102fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c2810302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c281030820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8c8604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c280fa6fb0 con 0x55c280c68390 2024-09-21T07:26:17.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8c8604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c280f963e0 con 0x55c280c89070 2024-09-21T07:26:17.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8c8604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c280f9a1f0 con 0x55c280c601b0 2024-09-21T07:26:17.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c28102fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c28102fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55838/0 (socket says 172.21.15.27:55838) 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8bb400640 1 -- 172.21.15.27:0/849503189 learned_addr learned my addr 172.21.15.27:0/849503189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8b3400640 1 -- 172.21.15.27:0/2485826606 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe8a4010430 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.754+0000 7fe8baa00640 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c2810302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8bb400640 1 -- 172.21.15.27:0/849503189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 msgr2=0x55c2810302e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8bb400640 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c2810302e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8bb400640 1 -- 172.21.15.27:0/849503189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 msgr2=0x55c281030820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8bb400640 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c281030820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8bb400640 1 -- 172.21.15.27:0/849503189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c280f60cd0 con 0x55c280c89070 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8bb400640 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c28102fda0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400e450 tx=0x7fe8b40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c3f3744e3c19f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8abe00640 1 -- 172.21.15.27:0/849503189 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400d910 con 0x55c280c89070 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8abe00640 1 -- 172.21.15.27:0/849503189 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b401c070 con 0x55c280c89070 2024-09-21T07:26:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.755+0000 7fe8abe00640 1 -- 172.21.15.27:0/849503189 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4017e10 con 0x55c280c89070 2024-09-21T07:26:17.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8b3400640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8a00b6680 0x7fe8a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 -- 172.21.15.27:0/849503189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 msgr2=0x55c28102fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c28102fda0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400e450 tx=0x7fe8b40095c0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8b3400640 1 -- 172.21.15.27:0/2485826606 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fe8a40e7db0 con 0x7fe8bc10b6c0 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 -- 172.21.15.27:0/849503189 shutdown_connections 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c2810302e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c281030820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 --2- 172.21.15.27:0/849503189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c28102fda0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 -- 172.21.15.27:0/849503189 >> 172.21.15.27:0/849503189 conn(0x55c280c7d460 msgr2=0x55c28102dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8a00b6680 0x7fe8a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.756+0000 7fe8c8604740 1 -- 172.21.15.27:0/849503189 shutdown_connections 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.757+0000 7fe8c8604740 1 -- 172.21.15.27:0/849503189 wait complete. 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.757+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8a00b6680 0x7fe8a00b8b90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac02a930 tx=0x7fe8ac02c040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.757+0000 7fe8c8604740 1 -- start start 2024-09-21T07:26:17.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 1 Processor -- start 2024-09-21T07:26:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c601b0 0x55c28103b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c68390 0x55c28103baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c89070 0x55c281047870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c280df7700 con 0x55c280c601b0 2024-09-21T07:26:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c280f31580 con 0x55c280c68390 2024-09-21T07:26:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c28103e580 con 0x55c280c89070 2024-09-21T07:26:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8c8604740 10 client.0 _ll_register_callbacks cb 0x55c280f41470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c601b0 0x55c28103b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8bb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c89070 0x55c281047870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8baa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c601b0 0x55c28103b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38798/0 (socket says 172.21.15.27:38798) 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.758+0000 7fe8baa00640 1 -- 172.21.15.27:0/111726276 learned_addr learned my addr 172.21.15.27:0/111726276 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c68390 0x55c28103baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8bb400640 1 -- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c68390 msgr2=0x55c28103baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8bb400640 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c68390 0x55c28103baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8bb400640 1 -- 172.21.15.27:0/111726276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c601b0 msgr2=0x55c28103b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8bb400640 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c601b0 0x55c28103b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8bb400640 1 -- 172.21.15.27:0/111726276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c280cf4650 con 0x55c280c89070 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c68390 0x55c28103baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8bb400640 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c89070 0x55c281047870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009c20 tx=0x7fe8b4017e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d336de1b21876df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.759+0000 7fe8a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.760+0000 7fe8a8c00640 1 -- 172.21.15.27:0/111726276 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4017620 con 0x55c280c89070 2024-09-21T07:26:17.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.760+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c280f33b90 con 0x55c280c89070 2024-09-21T07:26:17.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.760+0000 7fe8a8c00640 1 -- 172.21.15.27:0/111726276 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fe8b40177c0 con 0x55c280c89070 2024-09-21T07:26:17.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.760+0000 7fe8a8c00640 1 -- 172.21.15.27:0/111726276 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8b4033050 con 0x55c280c89070 2024-09-21T07:26:17.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.760+0000 7fe8c8604740 10 client.29452 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:17.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.760+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c280f4e950 con 0x55c280c89070 2024-09-21T07:26:17.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8a8c00640 1 -- 172.21.15.27:0/111726276 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c280f33b90 con 0x55c280c89070 2024-09-21T07:26:17.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8a8c00640 1 -- 172.21.15.27:0/111726276 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe8b400b040 con 0x55c280c89070 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8c8604740 10 client.29452 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8c8604740 10 client.29452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8c8604740 20 client.29452 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8c8604740 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c479f0 0x55c28104d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8c8604740 4 client.29452 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c280fa8f00 con 0x55c280c479f0 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.761+0000 7fe8baa00640 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c479f0 0x55c28104d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.762+0000 7fe8baa00640 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c479f0 0x55c28104d600 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ef34d71334e73 server_cookie=6462cd9e13c8f8e3 in_seq=0 out_seq=0 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.762+0000 7fe8a8c00640 10 client.29452 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8a8c00640 1 -- 172.21.15.27:0/111726276 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c280fa8f00 con 0x55c280c479f0 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8a8c00640 10 client.29452 handle_command_reply: tid=0 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 client.29452 shutdown 2024-09-21T07:26:17.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c89070 msgr2=0x55c281047870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c89070 0x55c281047870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009c20 tx=0x7fe8b4017e40 comp rx=0 tx=0).stop 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 shutdown_connections 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c479f0 0x55c28104d600 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c89070 0x55c281047870 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c68390 0x55c28103baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 --2- 172.21.15.27:0/111726276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c601b0 0x55c28103b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8a8c00640 0 client.29452 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.763+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 >> 172.21.15.27:0/111726276 conn(0x55c280c7d460 msgr2=0x55c281032f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.764+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 shutdown_connections 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.764+0000 7fe8c8604740 1 -- 172.21.15.27:0/111726276 wait complete. 2024-09-21T07:26:17.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.764+0000 7fe8c8604740 20 client.29452 trim_cache size 0 max 16384 2024-09-21T07:26:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.778+0000 7fe8c8604740 1 Processor -- start 2024-09-21T07:26:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.778+0000 7fe8c8604740 1 -- start start 2024-09-21T07:26:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c28102de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c28103d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c2810477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8c8604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b5a30 con 0x55c280c68390 2024-09-21T07:26:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8c8604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b3300 con 0x55c280c89070 2024-09-21T07:26:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8c8604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b1170 con 0x55c280c601b0 2024-09-21T07:26:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c28103d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8baa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c28102de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8baa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c28102de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55066/0 (socket says 172.21.15.27:55066) 2024-09-21T07:26:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8b3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c28103d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38804/0 (socket says 172.21.15.27:38804) 2024-09-21T07:26:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8baa00640 1 -- 172.21.15.27:0/2043893384 learned_addr learned my addr 172.21.15.27:0/2043893384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.779+0000 7fe8bb400640 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c2810477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8baa00640 1 -- 172.21.15.27:0/2043893384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 msgr2=0x55c2810477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c2810477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8baa00640 1 -- 172.21.15.27:0/2043893384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 msgr2=0x55c28103d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c28103d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8baa00640 1 -- 172.21.15.27:0/2043893384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2810322c0 con 0x55c280c601b0 2024-09-21T07:26:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c28102de50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac035040 tx=0x7fe8ac03e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5971a915049f09c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8ab400640 1 -- 172.21.15.27:0/2043893384 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac058070 con 0x55c280c601b0 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8ab400640 1 -- 172.21.15.27:0/2043893384 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac050070 con 0x55c280c601b0 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.780+0000 7fe8ab400640 1 -- 172.21.15.27:0/2043893384 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac037040 con 0x55c280c601b0 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 -- 172.21.15.27:0/2043893384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 msgr2=0x55c28102de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c28102de50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac035040 tx=0x7fe8ac03e000 comp rx=0 tx=0).stop 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 -- 172.21.15.27:0/2043893384 shutdown_connections 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c89070 0x55c2810477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c68390 0x55c28103d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2043893384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c601b0 0x55c28102de50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 -- 172.21.15.27:0/2043893384 >> 172.21.15.27:0/2043893384 conn(0x55c280c7d460 msgr2=0x55c28103ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 -- 172.21.15.27:0/2043893384 shutdown_connections 2024-09-21T07:26:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.781+0000 7fe8c8604740 1 -- 172.21.15.27:0/2043893384 wait complete. 2024-09-21T07:26:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.782+0000 7fe8c8604740 1 -- start start 2024-09-21T07:26:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.782+0000 7fe8c8604740 1 Processor -- start 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c601b0 0x55c28104e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c68390 0x55c28104eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8c8604740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c89070 0x55c2810b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8c8604740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b17c0 con 0x55c280c89070 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8c8604740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b2c30 con 0x55c280c601b0 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8c8604740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b1e90 con 0x55c280c68390 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8c8604740 10 client.0 _ll_register_callbacks cb 0x7fe8b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8bb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c89070 0x55c2810b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c601b0 0x55c28104e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8b3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c68390 0x55c28104eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c601b0 0x55c28104e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:55874/0 (socket says 172.21.15.27:55874) 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 -- 172.21.15.27:0/2826338758 learned_addr learned my addr 172.21.15.27:0/2826338758 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c68390 0x55c28104eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55082/0 (socket says 172.21.15.27:55082) 2024-09-21T07:26:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 -- 172.21.15.27:0/2826338758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c68390 msgr2=0x55c28104eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c68390 0x55c28104eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 -- 172.21.15.27:0/2826338758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c89070 msgr2=0x55c2810b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c89070 0x55c2810b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8baa00640 1 -- 172.21.15.27:0/2826338758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28104f7b0 con 0x55c280c601b0 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.783+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c68390 0x55c28104eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.784+0000 7fe8baa00640 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c601b0 0x55c28104e980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009830 tx=0x7fe8ac030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a7aa2d2e8eef1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.784+0000 7fe8a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.784+0000 7fe8a8c00640 1 -- 172.21.15.27:0/2826338758 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac058070 con 0x55c280c601b0 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.784+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ac050070 con 0x55c280c601b0 2024-09-21T07:26:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.784+0000 7fe8a8c00640 1 -- 172.21.15.27:0/2826338758 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fe8ac0035f0 con 0x55c280c601b0 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8a8c00640 1 -- 172.21.15.27:0/2826338758 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8ac066050 con 0x55c280c601b0 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 10 client.29507 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2810b83f0 con 0x55c280c601b0 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8a8c00640 1 -- 172.21.15.27:0/2826338758 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac050070 con 0x55c280c601b0 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8a8c00640 1 -- 172.21.15.27:0/2826338758 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe8ac03eb80 con 0x55c280c601b0 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 10 client.29507 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 10 client.29507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 20 client.29507 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c1cda0 0x55c2810bab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 4 client.29507 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.785+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c280fa49b0 con 0x55c280c1cda0 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.786+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c1cda0 0x55c2810bab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.786+0000 7fe8b3e00640 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c1cda0 0x55c2810bab90 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14792e1bedc8c4e server_cookie=b37dca10c0c5e138 in_seq=0 out_seq=0 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.786+0000 7fe8a8c00640 10 client.29507 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.786+0000 7fe8a8c00640 1 -- 172.21.15.27:0/2826338758 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c280fa49b0 con 0x55c280c1cda0 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.786+0000 7fe8a8c00640 10 client.29507 handle_command_reply: tid=0 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.786+0000 7fe8c8604740 1 client.29507 shutdown 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c601b0 msgr2=0x55c28104e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c601b0 0x55c28104e980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009830 tx=0x7fe8ac030650 comp rx=0 tx=0).stop 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 shutdown_connections 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c280c1cda0 0x55c2810bab90 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280c68390 0x55c28104eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c280c89070 0x55c2810b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 --2- 172.21.15.27:0/2826338758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c280c601b0 0x55c28104e980 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8a8c00640 0 client.29507 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 >> 172.21.15.27:0/2826338758 conn(0x55c280c7d460 msgr2=0x55c2810316f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 shutdown_connections 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 1 -- 172.21.15.27:0/2826338758 wait complete. 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.787+0000 7fe8c8604740 20 client.29507 trim_cache size 0 max 16384 2024-09-21T07:26:17.792 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:26:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8a00b6680 msgr2=0x7fe8a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8a00b6680 0x7fe8a00b8b90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac02a930 tx=0x7fe8ac02c040 comp rx=0 tx=0).stop 2024-09-21T07:26:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10b6c0 msgr2=0x7fe8bc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10b6c0 0x7fe8bc19cc80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e9f0 tx=0x7fe8a400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 shutdown_connections 2024-09-21T07:26:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe8a00b6680 0x7fe8a00b8b90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8bc10c010 0x7fe8bc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc10b6c0 0x7fe8bc19cc80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 --2- 172.21.15.27:0/2485826606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8bc105d80 0x7fe8bc19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:17.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 >> 172.21.15.27:0/2485826606 conn(0x7fe8bc101810 msgr2=0x7fe8bc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:17.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 shutdown_connections 2024-09-21T07:26:17.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:17.788+0000 7fe8c1a00640 1 -- 172.21.15.27:0/2485826606 wait complete. 2024-09-21T07:26:19.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:19 smithi153 ceph-mon[34747]: pgmap v792: 129 pgs: 129 active+clean; 3.6 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-09-21T07:26:19.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:19 smithi119 ceph-mon[34519]: pgmap v792: 129 pgs: 129 active+clean; 3.6 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-09-21T07:26:19.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:19 smithi027 ceph-mon[30779]: pgmap v792: 129 pgs: 129 active+clean; 3.6 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-09-21T07:26:19.614 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:26:19.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:26:20.012 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 -- 172.21.15.27:0/2174931468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa030109c20 msgr2=0x7fa030110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 --2- 172.21.15.27:0/2174931468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa030109c20 0x7fa030110450 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ca0 tx=0x7fa02c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 -- 172.21.15.27:0/2174931468 shutdown_connections 2024-09-21T07:26:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 --2- 172.21.15.27:0/2174931468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa030109c20 0x7fa030110450 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 --2- 172.21.15.27:0/2174931468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0301092d0 0x7fa0301096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 --2- 172.21.15.27:0/2174931468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 0x7fa030103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 -- 172.21.15.27:0/2174931468 >> 172.21.15.27:0/2174931468 conn(0x7fa0300ff3e0 msgr2=0x7fa030101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 -- 172.21.15.27:0/2174931468 shutdown_connections 2024-09-21T07:26:20.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.657+0000 7fa037200640 1 -- 172.21.15.27:0/2174931468 wait complete. 2024-09-21T07:26:20.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.658+0000 7fa037200640 1 Processor -- start 2024-09-21T07:26:20.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.658+0000 7fa037200640 1 -- start start 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa037200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 0x7fa03019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa037200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0301092d0 0x7fa03019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa037200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030109c20 0x7fa03019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa037200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa030113930 con 0x7fa0301092d0 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa037200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301137b0 con 0x7fa030109c20 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa037200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa030113ab0 con 0x7fa030103990 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa035e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 0x7fa03019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa036800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030109c20 0x7fa03019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa035e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 0x7fa03019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:44974/0 (socket says 172.21.15.27:44974) 2024-09-21T07:26:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa035400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0301092d0 0x7fa03019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa036800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030109c20 0x7fa03019fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59598/0 (socket says 172.21.15.27:59598) 2024-09-21T07:26:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa035e00640 1 -- 172.21.15.27:0/2354983539 learned_addr learned my addr 172.21.15.27:0/2354983539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa035e00640 1 -- 172.21.15.27:0/2354983539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030109c20 msgr2=0x7fa03019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa035e00640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030109c20 0x7fa03019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.659+0000 7fa035e00640 1 -- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0301092d0 msgr2=0x7fa03019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa035e00640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0301092d0 0x7fa03019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa035e00640 1 -- 172.21.15.27:0/2354983539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0301a02b0 con 0x7fa030103990 2024-09-21T07:26:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa036800640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030109c20 0x7fa03019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:20.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa035400640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0301092d0 0x7fa03019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:20.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa035e00640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 0x7fa03019a340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa02400d770 tx=0x7fa02400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa02aa00640 1 -- 172.21.15.27:0/2354983539 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024017070 con 0x7fa030103990 2024-09-21T07:26:20.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0301a05a0 con 0x7fa030103990 2024-09-21T07:26:20.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.660+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0301a6d00 con 0x7fa030103990 2024-09-21T07:26:20.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.661+0000 7fa02aa00640 1 -- 172.21.15.27:0/2354983539 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0240054d0 con 0x7fa030103990 2024-09-21T07:26:20.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.661+0000 7fa02aa00640 1 -- 172.21.15.27:0/2354983539 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024020060 con 0x7fa030103990 2024-09-21T07:26:20.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.664+0000 7fa03f5c6740 1 Processor -- start 2024-09-21T07:26:20.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.664+0000 7fa03f5c6740 1 -- start start 2024-09-21T07:26:20.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.664+0000 7fa02aa00640 1 -- 172.21.15.27:0/2354983539 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa024009c30 con 0x7fa030103990 2024-09-21T07:26:20.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.664+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af3d350 0x55711b2e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.664+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2e5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55711b25ae90 con 0x55711af1c670 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55711b24a2c0 con 0x55711af14490 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55711b24e0d0 con 0x55711af3d350 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af3d350 0x55711b2e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af3d350 0x55711b2e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:44982/0 (socket says 172.21.15.27:44982) 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 -- 172.21.15.27:0/82518145 learned_addr learned my addr 172.21.15.27:0/82518145 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa035e00640 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa035400640 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 -- 172.21.15.27:0/82518145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 msgr2=0x55711b2e5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2e5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 -- 172.21.15.27:0/82518145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 msgr2=0x55711b2e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa036800640 1 -- 172.21.15.27:0/82518145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55711afa8930 con 0x55711af3d350 2024-09-21T07:26:20.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.665+0000 7fa035400640 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.666+0000 7fa036800640 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af3d350 0x55711b2e4f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ca0 tx=0x7fa02c0098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf01991a27d1f457 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.666+0000 7fa01f000640 1 -- 172.21.15.27:0/82518145 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c030ac0 con 0x55711af3d350 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.666+0000 7fa01f000640 1 -- 172.21.15.27:0/82518145 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c036030 con 0x55711af3d350 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.666+0000 7fa01f000640 1 -- 172.21.15.27:0/82518145 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0046d0 con 0x55711af3d350 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.666+0000 7fa02aa00640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa0180b6700 0x7fa0180b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa02aa00640 1 -- 172.21.15.27:0/2354983539 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fa0240e7480 con 0x7fa030103990 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa035e00640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa0180b6700 0x7fa0180b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 -- 172.21.15.27:0/82518145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af3d350 msgr2=0x55711b2e4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af3d350 0x55711b2e4f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ca0 tx=0x7fa02c0098a0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 -- 172.21.15.27:0/82518145 shutdown_connections 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/82518145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af3d350 0x55711b2e4f00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 -- 172.21.15.27:0/82518145 >> 172.21.15.27:0/82518145 conn(0x55711af31740 msgr2=0x55711b2e2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 -- 172.21.15.27:0/82518145 shutdown_connections 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.667+0000 7fa03f5c6740 1 -- 172.21.15.27:0/82518145 wait complete. 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.668+0000 7fa035e00640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa0180b6700 0x7fa0180b8c10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55711b2e3b80 tx=0x7fa0240eaf30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.668+0000 7fa03f5c6740 1 -- start start 2024-09-21T07:26:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.668+0000 7fa03f5c6740 1 Processor -- start 2024-09-21T07:26:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af14490 0x55711b2f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af3d350 0x55711b300d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55711b0ab890 con 0x55711af1c670 2024-09-21T07:26:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55711b1e5160 con 0x55711af3d350 2024-09-21T07:26:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55711b2f79e0 con 0x55711af14490 2024-09-21T07:26:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa03f5c6740 10 client.0 _ll_register_callbacks cb 0x55711b201a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa035400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa036800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af14490 0x55711b2f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa035400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59228/0 (socket says 172.21.15.27:59228) 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa035400640 1 -- 172.21.15.27:0/3686783354 learned_addr learned my addr 172.21.15.27:0/3686783354 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.669+0000 7fa036800640 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af3d350 0x55711b300d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa035400640 1 -- 172.21.15.27:0/3686783354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af14490 msgr2=0x55711b2f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa035400640 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af14490 0x55711b2f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa035400640 1 -- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af3d350 msgr2=0x55711b300d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa035400640 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af3d350 0x55711b300d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa035400640 1 -- 172.21.15.27:0/3686783354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711b301250 con 0x55711af1c670 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa035400640 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2f5410 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa020012980 tx=0x7fa020012e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ecd165fda933 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa013400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa013400640 1 -- 172.21.15.27:0/3686783354 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020004030 con 0x55711af1c670 2024-09-21T07:26:20.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa013400640 1 -- 172.21.15.27:0/3686783354 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fa0200041d0 con 0x55711af1c670 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.670+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa02c036030 con 0x55711af1c670 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa013400640 1 -- 172.21.15.27:0/3686783354 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0200043d0 con 0x55711af1c670 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 10 client.19491 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55711b303bb0 con 0x55711af1c670 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa013400640 1 -- 172.21.15.27:0/3686783354 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c036030 con 0x55711af1c670 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa013400640 1 -- 172.21.15.27:0/3686783354 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa020003bc0 con 0x55711af1c670 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 10 client.19491 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 10 client.19491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 20 client.19491 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aefbcd0 0x55711b307100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 4 client.19491 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:20.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.671+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55711b25cde0 con 0x55711aefbcd0 2024-09-21T07:26:20.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.672+0000 7fa036800640 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aefbcd0 0x55711b307100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.672+0000 7fa036800640 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aefbcd0 0x55711b307100 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea7a14543fca6c server_cookie=46094f93826ac993 in_seq=0 out_seq=0 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.672+0000 7fa013400640 10 client.19491 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa013400640 1 -- 172.21.15.27:0/3686783354 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55711b25cde0 con 0x55711aefbcd0 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa013400640 10 client.19491 handle_command_reply: tid=0 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 client.19491 shutdown 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 msgr2=0x55711b2f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2f5410 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa020012980 tx=0x7fa020012e40 comp rx=0 tx=0).stop 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 shutdown_connections 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aefbcd0 0x55711b307100 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af3d350 0x55711b300d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af1c670 0x55711b2f5410 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3686783354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af14490 0x55711b2f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 >> 172.21.15.27:0/3686783354 conn(0x55711af31740 msgr2=0x55711b2e1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.673+0000 7fa013400640 0 client.19491 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:20.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.674+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 shutdown_connections 2024-09-21T07:26:20.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.674+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3686783354 wait complete. 2024-09-21T07:26:20.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.674+0000 7fa03f5c6740 20 client.19491 trim_cache size 0 max 16384 2024-09-21T07:26:20.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.692+0000 7fa03f5c6740 1 Processor -- start 2024-09-21T07:26:20.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.692+0000 7fa03f5c6740 1 -- start start 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2e1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301b1320 con 0x55711af3d350 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301aebf0 con 0x55711af14490 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301aca60 con 0x55711af1c670 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa036800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa036800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2e1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59238/0 (socket says 172.21.15.27:59238) 2024-09-21T07:26:20.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa036800640 1 -- 172.21.15.27:0/2233455924 learned_addr learned my addr 172.21.15.27:0/2233455924 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:20.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa035400640 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.693+0000 7fa036800640 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa036800640 1 -- 172.21.15.27:0/2233455924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 msgr2=0x55711b2f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa036800640 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa036800640 1 -- 172.21.15.27:0/2233455924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 msgr2=0x55711b2f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa036800640 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa036800640 1 -- 172.21.15.27:0/2233455924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55711b2e5cf0 con 0x55711af3d350 2024-09-21T07:26:20.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa035400640 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:20.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa036800640 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2e1390 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ca0 tx=0x7fa02c0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f15eeadb57a69d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa01e600640 1 -- 172.21.15.27:0/2233455924 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c004710 con 0x55711af3d350 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa01e600640 1 -- 172.21.15.27:0/2233455924 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c036040 con 0x55711af3d350 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.694+0000 7fa01e600640 1 -- 172.21.15.27:0/2233455924 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c003e50 con 0x55711af3d350 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 -- 172.21.15.27:0/2233455924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 msgr2=0x55711b2e1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2e1390 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ca0 tx=0x7fa02c0098a0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 -- 172.21.15.27:0/2233455924 shutdown_connections 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2e1390 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/2233455924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 -- 172.21.15.27:0/2233455924 >> 172.21.15.27:0/2233455924 conn(0x55711af31740 msgr2=0x55711b2efe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 -- 172.21.15.27:0/2233455924 shutdown_connections 2024-09-21T07:26:20.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.695+0000 7fa03f5c6740 1 -- 172.21.15.27:0/2233455924 wait complete. 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.696+0000 7fa03f5c6740 1 -- start start 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.696+0000 7fa03f5c6740 1 Processor -- start 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.696+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2fd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa03f5c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301ad0b0 con 0x55711af3d350 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301ae520 con 0x55711af14490 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa03f5c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301ad780 con 0x55711af1c670 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa03f5c6740 10 client.0 _ll_register_callbacks cb 0x7fa020009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2fd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59628/0 (socket says 172.21.15.27:59628) 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 -- 172.21.15.27:0/3186760808 learned_addr learned my addr 172.21.15.27:0/3186760808 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa035400640 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 -- 172.21.15.27:0/3186760808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 msgr2=0x55711b2f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 -- 172.21.15.27:0/3186760808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 msgr2=0x55711b2f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.697+0000 7fa036800640 1 -- 172.21.15.27:0/3186760808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711b2f86b0 con 0x55711af14490 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa036800640 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2fd190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa02c030930 tx=0x7fa02c02b030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d50cca2f214487e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa013400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa013400640 1 -- 172.21.15.27:0/3186760808 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c030990 con 0x55711af14490 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa013400640 1 -- 172.21.15.27:0/3186760808 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fa02c030b30 con 0x55711af14490 2024-09-21T07:26:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa02c036040 con 0x55711af14490 2024-09-21T07:26:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa013400640 1 -- 172.21.15.27:0/3186760808 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa02c030d30 con 0x55711af14490 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa03f5c6740 10 client.29525 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.698+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55711b372730 con 0x55711af14490 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa013400640 1 -- 172.21.15.27:0/3186760808 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c036040 con 0x55711af14490 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa013400640 1 -- 172.21.15.27:0/3186760808 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa02c060020 con 0x55711af14490 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa03f5c6740 10 client.29525 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa03f5c6740 10 client.29525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa03f5c6740 20 client.29525 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aed1080 0x55711b374a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa03f5c6740 4 client.29525 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.699+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55711b258890 con 0x55711aed1080 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.700+0000 7fa035400640 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aed1080 0x55711b374a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.700+0000 7fa035400640 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aed1080 0x55711b374a20 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f4efce4183e79bb server_cookie=c826e346c18e23ac in_seq=0 out_seq=0 2024-09-21T07:26:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.700+0000 7fa013400640 10 client.29525 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa013400640 1 -- 172.21.15.27:0/3186760808 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55711b258890 con 0x55711aed1080 2024-09-21T07:26:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa013400640 10 client.29525 handle_command_reply: tid=0 2024-09-21T07:26:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 client.29525 shutdown 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 msgr2=0x55711b2fd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2fd190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa02c030930 tx=0x7fa02c02b030 comp rx=0 tx=0).stop 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 shutdown_connections 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55711aed1080 0x55711b374a20 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55711af1c670 0x55711b2f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55711af3d350 0x55711b2f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 --2- 172.21.15.27:0/3186760808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55711af14490 0x55711b2fd190 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa013400640 0 client.29525 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.702+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 >> 172.21.15.27:0/3186760808 conn(0x55711af31740 msgr2=0x55711b2efe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 shutdown_connections 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa03f5c6740 1 -- 172.21.15.27:0/3186760808 wait complete. 2024-09-21T07:26:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa03f5c6740 20 client.29525 trim_cache size 0 max 16384 2024-09-21T07:26:20.706 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:26:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:26:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "aeb1e447-52d6-4d04-96be-5e4dce252fa0", 2024-09-21T07:26:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:26:20.706 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa0180b6700 msgr2=0x7fa0180b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa0180b6700 0x7fa0180b8c10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55711b2e3b80 tx=0x7fa0240eaf30 comp rx=0 tx=0).stop 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 msgr2=0x7fa03019a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 0x7fa03019a340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa02400d770 tx=0x7fa02400dc30 comp rx=0 tx=0).stop 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 shutdown_connections 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fa0180b6700 0x7fa0180b8c10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030109c20 0x7fa03019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0301092d0 0x7fa03019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 --2- 172.21.15.27:0/2354983539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa030103990 0x7fa03019a340 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.703+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 >> 172.21.15.27:0/2354983539 conn(0x7fa0300ff3e0 msgr2=0x7fa030101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.704+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 shutdown_connections 2024-09-21T07:26:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:20.704+0000 7fa037200640 1 -- 172.21.15.27:0/2354983539 wait complete. 2024-09-21T07:26:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:21 smithi153 ceph-mon[34747]: pgmap v793: 129 pgs: 129 active+clean; 3.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-21T07:26:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:21 smithi119 ceph-mon[34519]: pgmap v793: 129 pgs: 129 active+clean; 3.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-21T07:26:21.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:21 smithi027 ceph-mon[30779]: pgmap v793: 129 pgs: 129 active+clean; 3.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-21T07:26:21.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:26:21.996 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:22 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:26:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:22 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:22.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:22 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:26:22.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:22 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:22.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:22 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:26:22.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:22 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:22.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:22 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:26:22.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:22 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:22 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:26:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:22 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:22 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:26:22.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:22 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:26:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 -- 172.21.15.27:0/1269552252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c109c20 msgr2=0x7f9c0c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 --2- 172.21.15.27:0/1269552252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c109c20 0x7f9c0c110450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009be0 tx=0x7f9c0402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 -- 172.21.15.27:0/1269552252 shutdown_connections 2024-09-21T07:26:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 --2- 172.21.15.27:0/1269552252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0c1092d0 0x7f9c0c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 --2- 172.21.15.27:0/1269552252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c109c20 0x7f9c0c110450 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 --2- 172.21.15.27:0/1269552252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c103990 0x7f9c0c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 -- 172.21.15.27:0/1269552252 >> 172.21.15.27:0/1269552252 conn(0x7f9c0c0ff440 msgr2=0x7f9c0c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 -- 172.21.15.27:0/1269552252 shutdown_connections 2024-09-21T07:26:22.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.648+0000 7f9c12600640 1 -- 172.21.15.27:0/1269552252 wait complete. 2024-09-21T07:26:22.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.649+0000 7f9c12600640 1 Processor -- start 2024-09-21T07:26:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c12600640 1 -- start start 2024-09-21T07:26:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c12600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c103990 0x7f9c0c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c12600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0c1092d0 0x7f9c0c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c12600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c109c20 0x7f9c0c1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c12600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1138e0 con 0x7f9c0c109c20 2024-09-21T07:26:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c12600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c113760 con 0x7f9c0c103990 2024-09-21T07:26:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c12600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c113a60 con 0x7f9c0c1092d0 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c11c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c109c20 0x7f9c0c1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c11200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c103990 0x7f9c0c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c11c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c109c20 0x7f9c0c1a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59266/0 (socket says 172.21.15.27:59266) 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c11200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c103990 0x7f9c0c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59650/0 (socket says 172.21.15.27:59650) 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c11c00640 1 -- 172.21.15.27:0/2371319041 learned_addr learned my addr 172.21.15.27:0/2371319041 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.650+0000 7f9c0be00640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0c1092d0 0x7f9c0c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c11200640 1 -- 172.21.15.27:0/2371319041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0c1092d0 msgr2=0x7f9c0c19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c11200640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0c1092d0 0x7f9c0c19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c11200640 1 -- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c109c20 msgr2=0x7f9c0c1a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c11200640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c109c20 0x7f9c0c1a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c11200640 1 -- 172.21.15.27:0/2371319041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1a23e0 con 0x7f9c0c103990 2024-09-21T07:26:22.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c11c00640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c109c20 0x7f9c0c1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:22.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c11200640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c103990 0x7f9c0c19c470 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e790 tx=0x7f9c0000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.651+0000 7f9c09600640 1 -- 172.21.15.27:0/2371319041 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000eea0 con 0x7f9c0c103990 2024-09-21T07:26:22.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.652+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1a26d0 con 0x7f9c0c103990 2024-09-21T07:26:22.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.652+0000 7f9c09600640 1 -- 172.21.15.27:0/2371319041 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0000ccc0 con 0x7f9c0c103990 2024-09-21T07:26:22.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.652+0000 7f9c09600640 1 -- 172.21.15.27:0/2371319041 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00018660 con 0x7f9c0c103990 2024-09-21T07:26:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.652+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c1a8e30 con 0x7f9c0c103990 2024-09-21T07:26:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 Processor -- start 2024-09-21T07:26:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 -- start start 2024-09-21T07:26:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x564191933cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x564191934230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x564191934770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5641918aaf70 con 0x5641915641b0 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56419189a3a0 con 0x56419158d070 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.655+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56419189e1b0 con 0x56419156c390 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x564191933cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x564191933cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59662/0 (socket says 172.21.15.27:59662) 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 -- 172.21.15.27:0/2827144560 learned_addr learned my addr 172.21.15.27:0/2827144560 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:22.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c0be00640 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x564191934770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 -- 172.21.15.27:0/2827144560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 msgr2=0x564191934770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c09600640 1 -- 172.21.15.27:0/2371319041 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9c00010430 con 0x7f9c0c103990 2024-09-21T07:26:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x564191934770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 -- 172.21.15.27:0/2827144560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 msgr2=0x564191934230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:26:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x564191934230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 -- 172.21.15.27:0/2827144560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564191864c90 con 0x56419158d070 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c0be00640 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x564191934770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.656+0000 7f9c11c00640 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x564191933cf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c04030890 tx=0x7f9c040308c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe295e1b23211d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.657+0000 7f9bfe000640 1 -- 172.21.15.27:0/2827144560 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04041030 con 0x56419158d070 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.657+0000 7f9bfe000640 1 -- 172.21.15.27:0/2827144560 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c04030e20 con 0x56419158d070 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.657+0000 7f9bfe000640 1 -- 172.21.15.27:0/2827144560 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04036040 con 0x56419158d070 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2827144560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 msgr2=0x564191933cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x564191933cf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c04030890 tx=0x7f9c040308c0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2827144560 shutdown_connections 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x564191934770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x564191934230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2827144560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x564191933cf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2827144560 >> 172.21.15.27:0/2827144560 conn(0x564191581460 msgr2=0x564191931b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2827144560 shutdown_connections 2024-09-21T07:26:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2827144560 wait complete. 2024-09-21T07:26:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c09600640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bf40b6610 0x7f9bf40b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c09600640 1 -- 172.21.15.27:0/2371319041 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f9c000e7c00 con 0x7f9c0c103990 2024-09-21T07:26:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.658+0000 7f9c0be00640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bf40b6610 0x7f9bf40b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.659+0000 7f9c1a86c740 1 -- start start 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.659+0000 7f9c1a86c740 1 Processor -- start 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.659+0000 7f9c0be00640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bf40b6610 0x7f9bf40b8b20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c1058e0 tx=0x7f9bf8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5641915641b0 0x56419193f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x56419193fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419158d070 0x56419194b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5641916fb700 con 0x56419158d070 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564191835540 con 0x56419156c390 2024-09-21T07:26:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564191942470 con 0x5641915641b0 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c1a86c740 10 client.0 _ll_register_callbacks cb 0x564191845430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x56419193fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c11200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5641915641b0 0x56419193f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x56419193fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59664/0 (socket says 172.21.15.27:59664) 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 -- 172.21.15.27:0/3997536937 learned_addr learned my addr 172.21.15.27:0/3997536937 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c11c00640 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419158d070 0x56419194b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 -- 172.21.15.27:0/3997536937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5641915641b0 msgr2=0x56419193f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5641915641b0 0x56419193f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 -- 172.21.15.27:0/3997536937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419158d070 msgr2=0x56419194b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419158d070 0x56419194b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.660+0000 7f9c0be00640 1 -- 172.21.15.27:0/3997536937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641915f8650 con 0x56419156c390 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9c0be00640 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x56419193fee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8004600 tx=0x7f9bf8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f72d6a71b09ceaa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9bf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9bf2a00640 1 -- 172.21.15.27:0/3997536937 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8009e40 con 0x56419156c390 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564191837b50 con 0x56419156c390 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9bf2a00640 1 -- 172.21.15.27:0/3997536937 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f9bf801f630 con 0x56419156c390 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9bf2a00640 1 -- 172.21.15.27:0/3997536937 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9bf801f830 con 0x56419156c390 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9c1a86c740 10 client.29537 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.661+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564191852910 con 0x56419156c390 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9bf2a00640 1 -- 172.21.15.27:0/3997536937 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564191837b50 con 0x56419156c390 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9bf2a00640 1 -- 172.21.15.27:0/3997536937 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9bf8027070 con 0x56419156c390 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9c1a86c740 10 client.29537 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9c1a86c740 10 client.29537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9c1a86c740 20 client.29537 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56419154b9f0 0x564191951980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9c1a86c740 4 client.29537 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.662+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641918acec0 con 0x56419154b9f0 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.663+0000 7f9c11200640 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56419154b9f0 0x564191951980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.663+0000 7f9c11200640 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56419154b9f0 0x564191951980 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8810e0e877fa53d2 server_cookie=6be2455678e50569 in_seq=0 out_seq=0 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.663+0000 7f9bf2a00640 10 client.29537 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9bf2a00640 1 -- 172.21.15.27:0/3997536937 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641918acec0 con 0x56419154b9f0 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9bf2a00640 10 client.29537 handle_command_reply: tid=0 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 client.29537 shutdown 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 msgr2=0x56419193fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x56419193fee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8004600 tx=0x7f9bf8004730 comp rx=0 tx=0).stop 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 shutdown_connections 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56419154b9f0 0x564191951980 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419158d070 0x56419194b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x56419193fee0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/3997536937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5641915641b0 0x56419193f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 >> 172.21.15.27:0/3997536937 conn(0x564191581460 msgr2=0x564191936e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9bf2a00640 0 client.29537 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.664+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 shutdown_connections 2024-09-21T07:26:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.665+0000 7f9c1a86c740 1 -- 172.21.15.27:0/3997536937 wait complete. 2024-09-21T07:26:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.665+0000 7f9c1a86c740 20 client.29537 trim_cache size 0 max 16384 2024-09-21T07:26:22.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 Processor -- start 2024-09-21T07:26:22.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 -- start start 2024-09-21T07:26:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419194bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x564191938e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419158d070 0x564191939340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b56f0 con 0x5641915641b0 2024-09-21T07:26:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b2fc0 con 0x56419156c390 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.683+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b0e30 con 0x56419158d070 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419194bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419194bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59284/0 (socket says 172.21.15.27:59284) 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 -- 172.21.15.27:0/2054653736 learned_addr learned my addr 172.21.15.27:0/2054653736 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11c00640 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419158d070 0x564191939340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c0be00640 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x564191938e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 -- 172.21.15.27:0/2054653736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419158d070 msgr2=0x564191939340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419158d070 0x564191939340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 -- 172.21.15.27:0/2054653736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 msgr2=0x564191938e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x564191938e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c11200640 1 -- 172.21.15.27:0/2054653736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564191940250 con 0x5641915641b0 2024-09-21T07:26:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.684+0000 7f9c0be00640 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x564191938e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.685+0000 7f9c11200640 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419194bf10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9c00000c00 tx=0x7f9c00010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00baae8bbd818a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.685+0000 7f9bfd600640 1 -- 172.21.15.27:0/2054653736 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000d4030 con 0x5641915641b0 2024-09-21T07:26:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.685+0000 7f9bfd600640 1 -- 172.21.15.27:0/2054653736 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000b7060 con 0x5641915641b0 2024-09-21T07:26:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.685+0000 7f9bfd600640 1 -- 172.21.15.27:0/2054653736 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000a9040 con 0x5641915641b0 2024-09-21T07:26:22.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2054653736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 msgr2=0x56419194bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419194bf10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9c00000c00 tx=0x7f9c00010d20 comp rx=0 tx=0).stop 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2054653736 shutdown_connections 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419158d070 0x564191939340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419156c390 0x564191938e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/2054653736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419194bf10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2054653736 >> 172.21.15.27:0/2054653736 conn(0x564191581460 msgr2=0x56419193dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2054653736 shutdown_connections 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 -- 172.21.15.27:0/2054653736 wait complete. 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 -- start start 2024-09-21T07:26:22.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.686+0000 7f9c1a86c740 1 Processor -- start 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419193c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x56419193ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c1a86c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x56419193d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b1480 con 0x5641915641b0 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b28f0 con 0x56419158d070 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c1a86c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b1b50 con 0x56419156c390 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c1a86c740 10 client.0 _ll_register_callbacks cb 0x7f9bf80044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c11200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419193c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c0be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x56419193ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c11200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419193c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59290/0 (socket says 172.21.15.27:59290) 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.687+0000 7f9c11200640 1 -- 172.21.15.27:0/1360335383 learned_addr learned my addr 172.21.15.27:0/1360335383 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:22.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9c11c00640 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x56419193d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9c0be00640 1 -- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 msgr2=0x56419193d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9c0be00640 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x56419193d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9c0be00640 1 -- 172.21.15.27:0/1360335383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 msgr2=0x56419193c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9c0be00640 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419193c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9c0be00640 1 -- 172.21.15.27:0/1360335383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641919bb340 con 0x56419156c390 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9c0be00640 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x56419193ce80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8029ee0 tx=0x7f9bf8029670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f111c3e9c6883dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.688+0000 7f9bf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9bf2a00640 1 -- 172.21.15.27:0/1360335383 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf803c060 con 0x56419156c390 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9bf2a00640 1 -- 172.21.15.27:0/1360335383 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f9bf8003e50 con 0x56419156c390 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c000b7060 con 0x56419156c390 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9bf2a00640 1 -- 172.21.15.27:0/1360335383 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9bf8043050 con 0x56419156c390 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9c1a86c740 10 client.29494 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5641919bc580 con 0x56419156c390 2024-09-21T07:26:22.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9bf2a00640 1 -- 172.21.15.27:0/1360335383 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000b7060 con 0x56419156c390 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.689+0000 7f9bf2a00640 1 -- 172.21.15.27:0/1360335383 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9bf8027040 con 0x56419156c390 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.690+0000 7f9c1a86c740 10 client.29494 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.690+0000 7f9c1a86c740 10 client.29494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.690+0000 7f9c1a86c740 20 client.29494 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.690+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564191520da0 0x5641919bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.690+0000 7f9c1a86c740 4 client.29494 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.690+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641918a8970 con 0x564191520da0 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.690+0000 7f9c11200640 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564191520da0 0x5641919bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c11200640 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564191520da0 0x5641919bed10 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=334ec2d12275fb57 server_cookie=fd88179aed6efa00 in_seq=0 out_seq=0 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9bf2a00640 10 client.29494 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9bf2a00640 1 -- 172.21.15.27:0/1360335383 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6534 (crc 0 0 0) 0x5641918a8970 con 0x564191520da0 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9bf2a00640 10 client.29494 handle_command_reply: tid=0 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 client.29494 shutdown 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 msgr2=0x56419193ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x56419193ce80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8029ee0 tx=0x7f9bf8029670 comp rx=0 tx=0).stop 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 shutdown_connections 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x564191520da0 0x5641919bed10 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56419156c390 0x56419193ce80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419158d070 0x56419193d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 --2- 172.21.15.27:0/1360335383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641915641b0 0x56419193c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9bf2a00640 0 client.29494 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.691+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 >> 172.21.15.27:0/1360335383 conn(0x564191581460 msgr2=0x56419193dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.692+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 shutdown_connections 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.692+0000 7f9c1a86c740 1 -- 172.21.15.27:0/1360335383 wait complete. 2024-09-21T07:26:22.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.692+0000 7f9c1a86c740 20 client.29494 trim_cache size 0 max 16384 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "aeb1e447-52d6-4d04-96be-5e4dce252fa0", 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:26:22.697 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.698 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.699 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.702 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.704 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.704 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.704 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:26:22.704 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:26:22.704 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.704 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.705 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.707 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:26:22.707 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:26:22.707 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.707 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.707 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.707 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.707 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.708 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:26:22.708 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:26:22.708 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.708 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.708 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.708 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.709 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.709 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:26:22.709 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:26:22.709 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.709 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.709 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:26:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:26:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:26:22.711 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:26:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.712 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:26:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:26:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:26:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bf40b6610 msgr2=0x7f9bf40b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bf40b6610 0x7f9bf40b8b20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c1058e0 tx=0x7f9bf8005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c103990 msgr2=0x7f9c0c19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c103990 0x7f9c0c19c470 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e790 tx=0x7f9c0000ec50 comp rx=0 tx=0).stop 2024-09-21T07:26:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 shutdown_connections 2024-09-21T07:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0c1092d0 0x7f9c0c19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9bf40b6610 0x7f9bf40b8b20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c0c109c20 0x7f9c0c1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 --2- 172.21.15.27:0/2371319041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c0c103990 0x7f9c0c19c470 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 >> 172.21.15.27:0/2371319041 conn(0x7f9c0c0ff440 msgr2=0x7f9c0c1088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 shutdown_connections 2024-09-21T07:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:22.693+0000 7f9c12600640 1 -- 172.21.15.27:0/2371319041 wait complete. 2024-09-21T07:26:23.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:23 smithi153 ceph-mon[34747]: pgmap v794: 129 pgs: 129 active+clean; 3.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-09-21T07:26:23.507 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:23 smithi027 ceph-mon[30779]: pgmap v794: 129 pgs: 129 active+clean; 3.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-09-21T07:26:23.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:23 smithi119 ceph-mon[34519]: pgmap v794: 129 pgs: 129 active+clean; 3.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-09-21T07:26:23.547 INFO:tasks.cephfs.filesystem:scrub status for tag:aeb1e447-52d6-4d04-96be-5e4dce252fa0 - {'path': '/', 'tag': 'aeb1e447-52d6-4d04-96be-5e4dce252fa0', 'options': 'recursive,force'} 2024-09-21T07:26:24.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:24 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:26:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:24 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:26:24.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:24 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:26:25.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:25 smithi153 ceph-mon[34747]: pgmap v795: 129 pgs: 129 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:26:25.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:25 smithi119 ceph-mon[34519]: pgmap v795: 129 pgs: 129 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:26:25.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:25 smithi027 ceph-mon[30779]: pgmap v795: 129 pgs: 129 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.260+0000 7fb058e00640 1 -- 172.21.15.27:0/1200624916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 msgr2=0x7fb054103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.260+0000 7fb058e00640 1 --2- 172.21.15.27:0/1200624916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 0x7fb054103f10 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb044009900 tx=0x7fb04402a410 comp rx=0 tx=0).stop 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.260+0000 7fb058e00640 1 -- 172.21.15.27:0/1200624916 shutdown_connections 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.260+0000 7fb058e00640 1 --2- 172.21.15.27:0/1200624916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb054109d80 0x7fb0541105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.260+0000 7fb058e00640 1 --2- 172.21.15.27:0/1200624916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109430 0x7fb054109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.260+0000 7fb058e00640 1 --2- 172.21.15.27:0/1200624916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 0x7fb054103f10 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.260+0000 7fb058e00640 1 -- 172.21.15.27:0/1200624916 >> 172.21.15.27:0/1200624916 conn(0x7fb0540ff500 msgr2=0x7fb054101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.261+0000 7fb058e00640 1 -- 172.21.15.27:0/1200624916 shutdown_connections 2024-09-21T07:26:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.261+0000 7fb058e00640 1 -- 172.21.15.27:0/1200624916 wait complete. 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 Processor -- start 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 -- start start 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 0x7fb054075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb054109430 0x7fb054075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109d80 0x7fb054076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb054113a90 con 0x7fb054103b00 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb054113910 con 0x7fb054109d80 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.262+0000 7fb058e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb054113c10 con 0x7fb054109430 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb051e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 0x7fb054075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109d80 0x7fb054076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109d80 0x7fb054076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59704/0 (socket says 172.21.15.27:59704) 2024-09-21T07:26:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 -- 172.21.15.27:0/3675500074 learned_addr learned my addr 172.21.15.27:0/3675500074 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb051400640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb054109430 0x7fb054075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 -- 172.21.15.27:0/3675500074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb054109430 msgr2=0x7fb054075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb054109430 0x7fb054075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 -- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 msgr2=0x7fb054075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 0x7fb054075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 -- 172.21.15.27:0/3675500074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0541a8800 con 0x7fb054109d80 2024-09-21T07:26:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.263+0000 7fb052800640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109d80 0x7fb054076190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00eb10 tx=0x7fb04c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.264+0000 7fb04aa00640 1 -- 172.21.15.27:0/3675500074 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c004b20 con 0x7fb054109d80 2024-09-21T07:26:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.264+0000 7fb04aa00640 1 -- 172.21.15.27:0/3675500074 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb04c004cc0 con 0x7fb054109d80 2024-09-21T07:26:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.264+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0541a8af0 con 0x7fb054109d80 2024-09-21T07:26:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.264+0000 7fb04aa00640 1 -- 172.21.15.27:0/3675500074 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c010670 con 0x7fb054109d80 2024-09-21T07:26:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.264+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541a9010 con 0x7fb054109d80 2024-09-21T07:26:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.267+0000 7fb05fe55740 1 Processor -- start 2024-09-21T07:26:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.267+0000 7fb04aa00640 1 -- 172.21.15.27:0/3675500074 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb04c005020 con 0x7fb054109d80 2024-09-21T07:26:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.267+0000 7fb05fe55740 1 -- start start 2024-09-21T07:26:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa01551c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa01552170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa015526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb05fe55740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fa014c76b0 con 0x55fa011842f0 2024-09-21T07:26:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb05fe55740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fa014b6ae0 con 0x55fa011ad1b0 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb05fe55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fa014ba8f0 con 0x55fa0118c4d0 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa01552170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa01552170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59324/0 (socket says 172.21.15.27:59324) 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 -- 172.21.15.27:0/1531088162 learned_addr learned my addr 172.21.15.27:0/1531088162 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051400640 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa01551c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051400640 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa015526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 -- 172.21.15.27:0/1531088162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 msgr2=0x55fa015526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa015526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 -- 172.21.15.27:0/1531088162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 msgr2=0x55fa01551c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa01551c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051e00640 1 -- 172.21.15.27:0/1531088162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa01483780 con 0x55fa011842f0 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.268+0000 7fb051400640 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa015526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.269+0000 7fb051e00640 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa01552170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb044000c00 tx=0x7fb0440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c805c7ccc6fc68e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.269+0000 7fb04aa00640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0380b66b0 0x7fb0380b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.269+0000 7fb04aa00640 1 -- 172.21.15.27:0/3675500074 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fb04c0e7ff0 con 0x7fb054109d80 2024-09-21T07:26:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.269+0000 7fb03f400640 1 -- 172.21.15.27:0/1531088162 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044035040 con 0x55fa011842f0 2024-09-21T07:26:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.269+0000 7fb03f400640 1 -- 172.21.15.27:0/1531088162 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0440054f0 con 0x55fa011842f0 2024-09-21T07:26:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.269+0000 7fb03f400640 1 -- 172.21.15.27:0/1531088162 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440035f0 con 0x55fa011842f0 2024-09-21T07:26:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.269+0000 7fb051400640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0380b66b0 0x7fb0380b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 -- 172.21.15.27:0/1531088162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 msgr2=0x55fa01552170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb051400640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0380b66b0 0x7fb0380b8bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb054102c00 tx=0x7fb0400096c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa01552170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb044000c00 tx=0x7fb0440040c0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 -- 172.21.15.27:0/1531088162 shutdown_connections 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa015526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa01552170 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 --2- 172.21.15.27:0/1531088162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa01551c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 -- 172.21.15.27:0/1531088162 >> 172.21.15.27:0/1531088162 conn(0x55fa011a1300 msgr2=0x55fa0154fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 -- 172.21.15.27:0/1531088162 shutdown_connections 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.270+0000 7fb05fe55740 1 -- 172.21.15.27:0/1531088162 wait complete. 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.271+0000 7fb05fe55740 1 -- start start 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.271+0000 7fb05fe55740 1 Processor -- start 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.271+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011842f0 0x55fa0155d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.271+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa0118c4d0 0x55fa0155d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011ad1b0 0x55fa0155dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb05fe55740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fa013eeab0 con 0x55fa0118c4d0 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb05fe55740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fa0145d570 con 0x55fa011842f0 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb05fe55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fa01560350 con 0x55fa011ad1b0 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb05fe55740 10 client.0 _ll_register_callbacks cb 0x55fa0146c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa0118c4d0 0x55fa0155d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011842f0 0x55fa0155d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa0118c4d0 0x55fa0155d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59326/0 (socket says 172.21.15.27:59326) 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051400640 1 -- 172.21.15.27:0/2531850203 learned_addr learned my addr 172.21.15.27:0/2531850203 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb052800640 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011ad1b0 0x55fa0155dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051e00640 1 -- 172.21.15.27:0/2531850203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011ad1b0 msgr2=0x55fa0155dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051e00640 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011ad1b0 0x55fa0155dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051e00640 1 -- 172.21.15.27:0/2531850203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa0118c4d0 msgr2=0x55fa0155d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051e00640 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa0118c4d0 0x55fa0155d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.272+0000 7fb051e00640 1 -- 172.21.15.27:0/2531850203 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa01218790 con 0x55fa011842f0 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb051e00640 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011842f0 0x55fa0155d1f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb04402a930 tx=0x7fb044005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c328d2fe4f7598 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb033e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb033e00640 1 -- 172.21.15.27:0/2531850203 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044051020 con 0x55fa011842f0 2024-09-21T07:26:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb033e00640 1 -- 172.21.15.27:0/2531850203 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fb044004d50 con 0x55fa011842f0 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0440054f0 con 0x55fa011842f0 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb033e00640 1 -- 172.21.15.27:0/2531850203 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb044035040 con 0x55fa011842f0 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb05fe55740 10 client.29555 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.273+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa01291620 con 0x55fa011842f0 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb033e00640 1 -- 172.21.15.27:0/2531850203 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0440054f0 con 0x55fa011842f0 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb033e00640 1 -- 172.21.15.27:0/2531850203 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb04404c070 con 0x55fa011842f0 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb05fe55740 10 client.29555 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb05fe55740 10 client.29555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb05fe55740 20 client.29555 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa0116bd80 0x55fa0156f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb05fe55740 4 client.29555 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa014c9600 con 0x55fa0116bd80 2024-09-21T07:26:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.274+0000 7fb051400640 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa0116bd80 0x55fa0156f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.275+0000 7fb051400640 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa0116bd80 0x55fa0156f080 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=893b1975e91f166b server_cookie=eb893dadd03f6063 in_seq=0 out_seq=0 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.275+0000 7fb033e00640 10 client.29555 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb033e00640 1 -- 172.21.15.27:0/2531850203 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa014c9600 con 0x55fa0116bd80 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb033e00640 10 client.29555 handle_command_reply: tid=0 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 client.29555 shutdown 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011842f0 msgr2=0x55fa0155d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011842f0 0x55fa0155d1f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb04402a930 tx=0x7fb044005fd0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 shutdown_connections 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa0116bd80 0x55fa0156f080 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011ad1b0 0x55fa0155dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa0118c4d0 0x55fa0155d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2531850203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011842f0 0x55fa0155d1f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 >> 172.21.15.27:0/2531850203 conn(0x55fa011a1300 msgr2=0x55fa01552c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb033e00640 0 client.29555 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 shutdown_connections 2024-09-21T07:26:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 1 -- 172.21.15.27:0/2531850203 wait complete. 2024-09-21T07:26:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.276+0000 7fb05fe55740 20 client.29555 trim_cache size 0 max 16384 2024-09-21T07:26:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 Processor -- start 2024-09-21T07:26:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 -- start start 2024-09-21T07:26:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa0155ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa0155a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa0155a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b5ab0 con 0x55fa011842f0 2024-09-21T07:26:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b3380 con 0x55fa011ad1b0 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.292+0000 7fb05fe55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b11f0 con 0x55fa0118c4d0 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa0155ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa0155ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59330/0 (socket says 172.21.15.27:59330) 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 -- 172.21.15.27:0/2581793481 learned_addr learned my addr 172.21.15.27:0/2581793481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051400640 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa0155a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb052800640 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa0155a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 -- 172.21.15.27:0/2581793481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 msgr2=0x55fa0155a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa0155a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 -- 172.21.15.27:0/2581793481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 msgr2=0x55fa0155a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa0155a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 -- 172.21.15.27:0/2581793481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa01553f30 con 0x55fa011842f0 2024-09-21T07:26:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.293+0000 7fb051e00640 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa0155ee20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb04402aa00 tx=0x7fb044004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4ecc13f5389de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb03ea00640 1 -- 172.21.15.27:0/2581793481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044051070 con 0x55fa011842f0 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb03ea00640 1 -- 172.21.15.27:0/2581793481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb044062070 con 0x55fa011842f0 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb03ea00640 1 -- 172.21.15.27:0/2581793481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04403cd10 con 0x55fa011842f0 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 -- 172.21.15.27:0/2581793481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 msgr2=0x55fa0155ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa0155ee20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb04402aa00 tx=0x7fb044004140 comp rx=0 tx=0).stop 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 -- 172.21.15.27:0/2581793481 shutdown_connections 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa0118c4d0 0x55fa0155a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa011ad1b0 0x55fa0155a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 --2- 172.21.15.27:0/2581793481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011842f0 0x55fa0155ee20 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 -- 172.21.15.27:0/2581793481 >> 172.21.15.27:0/2581793481 conn(0x55fa011a1300 msgr2=0x55fa0155e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.294+0000 7fb05fe55740 1 -- 172.21.15.27:0/2581793481 shutdown_connections 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.295+0000 7fb05fe55740 1 -- 172.21.15.27:0/2581793481 wait complete. 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.295+0000 7fb05fe55740 1 -- start start 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.295+0000 7fb05fe55740 1 Processor -- start 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011842f0 0x55fa015719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa0118c4d0 0x55fa01571fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb05fe55740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011ad1b0 0x55fa015d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb05fe55740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b1840 con 0x55fa011ad1b0 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb05fe55740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b2cb0 con 0x55fa0118c4d0 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb05fe55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b1f10 con 0x55fa011842f0 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb05fe55740 10 client.0 _ll_register_callbacks cb 0x7fb044009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011ad1b0 0x55fa015d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011ad1b0 0x55fa015d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59336/0 (socket says 172.21.15.27:59336) 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 -- 172.21.15.27:0/63608702 learned_addr learned my addr 172.21.15.27:0/63608702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb051e00640 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011842f0 0x55fa015719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb051400640 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa0118c4d0 0x55fa01571fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 -- 172.21.15.27:0/63608702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011842f0 msgr2=0x55fa015719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011842f0 0x55fa015719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 -- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa0118c4d0 msgr2=0x55fa01571fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa0118c4d0 0x55fa01571fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.296+0000 7fb052800640 1 -- 172.21.15.27:0/63608702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa015d1eb0 con 0x55fa011ad1b0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb052800640 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011ad1b0 0x55fa015d1840 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0ad9c0 tx=0x7fb04c0a98d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b5b43261bc566e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb033e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb033e00640 1 -- 172.21.15.27:0/63608702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c096020 con 0x55fa011ad1b0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb033e00640 1 -- 172.21.15.27:0/63608702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7fb04c0ace90 con 0x55fa011ad1b0 2024-09-21T07:26:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb044062070 con 0x55fa011ad1b0 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb033e00640 1 -- 172.21.15.27:0/63608702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb04c0ad1a0 con 0x55fa011ad1b0 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb05fe55740 10 client.19530 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.297+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa015d9240 con 0x55fa011ad1b0 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb033e00640 1 -- 172.21.15.27:0/63608702 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb044062070 con 0x55fa011ad1b0 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb033e00640 1 -- 172.21.15.27:0/63608702 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb04c0ac700 con 0x55fa011ad1b0 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb05fe55740 10 client.19530 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb05fe55740 10 client.19530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb05fe55740 20 client.19530 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb05fe55740 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa011411c0 0x55fa015dba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb05fe55740 4 client.19530 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa014c50b0 con 0x55fa011411c0 2024-09-21T07:26:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.298+0000 7fb051e00640 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa011411c0 0x55fa015dba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.299+0000 7fb051e00640 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa011411c0 0x55fa015dba30 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc67bfe58d5e0de server_cookie=cf8baede78dfcd32 in_seq=0 out_seq=0 2024-09-21T07:26:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.299+0000 7fb033e00640 10 client.19530 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb033e00640 1 -- 172.21.15.27:0/63608702 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa014c50b0 con 0x55fa011411c0 2024-09-21T07:26:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb033e00640 10 client.19530 handle_command_reply: tid=0 2024-09-21T07:26:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 client.19530 shutdown 2024-09-21T07:26:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011ad1b0 msgr2=0x55fa015d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011ad1b0 0x55fa015d1840 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0ad9c0 tx=0x7fb04c0a98d0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 shutdown_connections 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fa011411c0 0x55fa015dba30 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa011ad1b0 0x55fa015d1840 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa0118c4d0 0x55fa01571fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 --2- 172.21.15.27:0/63608702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa011842f0 0x55fa015719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb033e00640 0 client.19530 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 >> 172.21.15.27:0/63608702 conn(0x55fa011a1300 msgr2=0x55fa01552c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 shutdown_connections 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 1 -- 172.21.15.27:0/63608702 wait complete. 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.312+0000 7fb05fe55740 20 client.19530 trim_cache size 0 max 16384 2024-09-21T07:26:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:26:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:26:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:26:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:26:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0380b66b0 msgr2=0x7fb0380b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0380b66b0 0x7fb0380b8bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb054102c00 tx=0x7fb0400096c0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109d80 msgr2=0x7fb054076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109d80 0x7fb054076190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00eb10 tx=0x7fb04c00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 shutdown_connections 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb054109430 0x7fb054075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb0380b66b0 0x7fb0380b8bc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb054109d80 0x7fb054076190 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 --2- 172.21.15.27:0/3675500074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb054103b00 0x7fb054075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 >> 172.21.15.27:0/3675500074 conn(0x7fb0540ff500 msgr2=0x7fb054108a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 shutdown_connections 2024-09-21T07:26:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:27.313+0000 7fb058e00640 1 -- 172.21.15.27:0/3675500074 wait complete. 2024-09-21T07:26:27.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:27 smithi153 ceph-mon[34747]: pgmap v796: 129 pgs: 129 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:26:27.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:27 smithi119 ceph-mon[34519]: pgmap v796: 129 pgs: 129 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:26:27.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:27 smithi027 ceph-mon[30779]: pgmap v796: 129 pgs: 129 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-21T07:26:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:29 smithi119 ceph-mon[34519]: pgmap v797: 129 pgs: 129 active+clean; 4.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:29.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:29 smithi027 ceph-mon[30779]: pgmap v797: 129 pgs: 129 active+clean; 4.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:29.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:29 smithi153 ceph-mon[34747]: pgmap v797: 129 pgs: 129 active+clean; 4.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:31.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:31 smithi119 ceph-mon[34519]: pgmap v798: 129 pgs: 129 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-09-21T07:26:31.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:31 smithi027 ceph-mon[30779]: pgmap v798: 129 pgs: 129 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-09-21T07:26:31.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:31 smithi153 ceph-mon[34747]: pgmap v798: 129 pgs: 129 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 63 KiB/s wr, 17 op/s 2024-09-21T07:26:33.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:33 smithi119 ceph-mon[34519]: pgmap v799: 129 pgs: 129 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:26:33.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:33 smithi027 ceph-mon[30779]: pgmap v799: 129 pgs: 129 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:26:33.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:33 smithi153 ceph-mon[34747]: pgmap v799: 129 pgs: 129 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:26:35.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:35 smithi119 ceph-mon[34519]: pgmap v800: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:26:35.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:35 smithi027 ceph-mon[30779]: pgmap v800: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:26:35.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:35 smithi153 ceph-mon[34747]: pgmap v800: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-09-21T07:26:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:37 smithi119 ceph-mon[34519]: pgmap v801: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:26:37.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:37 smithi027 ceph-mon[30779]: pgmap v801: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:26:37.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:37 smithi153 ceph-mon[34747]: pgmap v801: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-09-21T07:26:39.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:39 smithi119 ceph-mon[34519]: pgmap v802: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-21T07:26:39.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:39 smithi027 ceph-mon[30779]: pgmap v802: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-21T07:26:39.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:39 smithi153 ceph-mon[34747]: pgmap v802: 129 pgs: 129 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-21T07:26:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.522+0000 7f37cd600640 1 -- 172.21.15.27:0/355815903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c810bf50 msgr2=0x7f37c8112780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.522+0000 7f37cd600640 1 --2- 172.21.15.27:0/355815903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c810bf50 0x7f37c8112780 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009ca0 tx=0x7f37c002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.522+0000 7f37cd600640 1 -- 172.21.15.27:0/355815903 shutdown_connections 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.522+0000 7f37cd600640 1 --2- 172.21.15.27:0/355815903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c810b600 0x7f37c810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.522+0000 7f37cd600640 1 --2- 172.21.15.27:0/355815903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c810bf50 0x7f37c8112780 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.522+0000 7f37cd600640 1 --2- 172.21.15.27:0/355815903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37c8105cd0 0x7f37c81060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.522+0000 7f37cd600640 1 -- 172.21.15.27:0/355815903 >> 172.21.15.27:0/355815903 conn(0x7f37c81016f0 msgr2=0x7f37c8103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.523+0000 7f37cd600640 1 -- 172.21.15.27:0/355815903 shutdown_connections 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.523+0000 7f37cd600640 1 -- 172.21.15.27:0/355815903 wait complete. 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 Processor -- start 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 -- start start 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c8105cd0 0x7f37c819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37c810b600 0x7f37c819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c810bf50 0x7f37c81a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8115c10 con 0x7f37c8105cd0 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8115a90 con 0x7f37c810b600 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37cd600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8115d90 con 0x7f37c810bf50 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c8105cd0 0x7f37c819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c8105cd0 0x7f37c819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52850/0 (socket says 172.21.15.27:52850) 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.524+0000 7f37c6a00640 1 -- 172.21.15.27:0/1110723750 learned_addr learned my addr 172.21.15.27:0/1110723750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c6000640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37c810b600 0x7f37c819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c7400640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c810bf50 0x7f37c81a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c6a00640 1 -- 172.21.15.27:0/1110723750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c810bf50 msgr2=0x7f37c81a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c6a00640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c810bf50 0x7f37c81a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c6a00640 1 -- 172.21.15.27:0/1110723750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37c810b600 msgr2=0x7f37c819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c6a00640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37c810b600 0x7f37c819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c6a00640 1 -- 172.21.15.27:0/1110723750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c81a2520 con 0x7f37c8105cd0 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c7400640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c810bf50 0x7f37c81a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37c6a00640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c8105cd0 0x7f37c819c5f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d950 tx=0x7f37b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37bf400640 1 -- 172.21.15.27:0/1110723750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8017030 con 0x7f37c8105cd0 2024-09-21T07:26:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37bf400640 1 -- 172.21.15.27:0/1110723750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b8004020 con 0x7f37c8105cd0 2024-09-21T07:26:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.525+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c81a2810 con 0x7f37c8105cd0 2024-09-21T07:26:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.526+0000 7f37bf400640 1 -- 172.21.15.27:0/1110723750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80047a0 con 0x7f37c8105cd0 2024-09-21T07:26:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.526+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81a8fa0 con 0x7f37c8105cd0 2024-09-21T07:26:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 Processor -- start 2024-09-21T07:26:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 -- start start 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c58c1b0 0x55ce0c930c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37bf400640 1 -- 172.21.15.27:0/1110723750 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f37b8009a30 con 0x7f37c8105cd0 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c931170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c56b4d0 0x55ce0c9316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ce0c8a66b0 con 0x55ce0c5632f0 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ce0c895ae0 con 0x55ce0c56b4d0 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.529+0000 7f37d466f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ce0c8998f0 con 0x55ce0c58c1b0 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c931170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c56b4d0 0x55ce0c9316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c7400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c58c1b0 0x55ce0c930c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c56b4d0 0x55ce0c9316b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48632/0 (socket says 172.21.15.27:48632) 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 -- 172.21.15.27:0/2878589227 learned_addr learned my addr 172.21.15.27:0/2878589227 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 -- 172.21.15.27:0/2878589227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c58c1b0 msgr2=0x55ce0c930c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c58c1b0 0x55ce0c930c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 -- 172.21.15.27:0/2878589227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 msgr2=0x55ce0c931170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c931170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 -- 172.21.15.27:0/2878589227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0c862780 con 0x55ce0c56b4d0 2024-09-21T07:26:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c7400640 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c58c1b0 0x55ce0c930c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.530+0000 7f37c6000640 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c56b4d0 0x55ce0c9316b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009c30 tx=0x7f37b4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6667625e813a7c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.531+0000 7f37b3400640 1 -- 172.21.15.27:0/2878589227 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4009de0 con 0x55ce0c56b4d0 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.531+0000 7f37b3400640 1 -- 172.21.15.27:0/2878589227 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b4005020 con 0x55ce0c56b4d0 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.531+0000 7f37b3400640 1 -- 172.21.15.27:0/2878589227 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4004a00 con 0x55ce0c56b4d0 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.531+0000 7f37bf400640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37ac0b6680 0x7f37ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.531+0000 7f37bf400640 1 -- 172.21.15.27:0/1110723750 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f37b80e7660 con 0x7f37c8105cd0 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.531+0000 7f37c7400640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37ac0b6680 0x7f37ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 -- 172.21.15.27:0/2878589227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c56b4d0 msgr2=0x55ce0c9316b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c56b4d0 0x55ce0c9316b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009c30 tx=0x7f37b4011810 comp rx=0 tx=0).stop 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 -- 172.21.15.27:0/2878589227 shutdown_connections 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c56b4d0 0x55ce0c9316b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c931170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 --2- 172.21.15.27:0/2878589227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c58c1b0 0x55ce0c930c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 -- 172.21.15.27:0/2878589227 >> 172.21.15.27:0/2878589227 conn(0x55ce0c580300 msgr2=0x55ce0c92eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 -- 172.21.15.27:0/2878589227 shutdown_connections 2024-09-21T07:26:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37d466f740 1 -- 172.21.15.27:0/2878589227 wait complete. 2024-09-21T07:26:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.532+0000 7f37c7400640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37ac0b6680 0x7f37ac0b8b90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f37c0004af0 tx=0x7f37c0005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 -- start start 2024-09-21T07:26:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 Processor -- start 2024-09-21T07:26:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c93c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c9486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ce0c7cdab0 con 0x55ce0c5632f0 2024-09-21T07:26:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ce0c83c570 con 0x55ce0c58c1b0 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ce0c93f3b0 con 0x55ce0c56b4d0 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.533+0000 7f37d466f740 10 client.0 _ll_register_callbacks cb 0x55ce0c84b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c93c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c9486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c9486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:48648/0 (socket says 172.21.15.27:48648) 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c93c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52868/0 (socket says 172.21.15.27:52868) 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c7400640 1 -- 172.21.15.27:0/2244655976 learned_addr learned my addr 172.21.15.27:0/2244655976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6000640 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6a00640 1 -- 172.21.15.27:0/2244655976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 msgr2=0x55ce0c93c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6a00640 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6a00640 1 -- 172.21.15.27:0/2244655976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 msgr2=0x55ce0c9486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6a00640 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c9486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.534+0000 7f37c6a00640 1 -- 172.21.15.27:0/2244655976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0c5f7790 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37c6a00640 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c93c270 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f37b80e49a0 tx=0x7f37b80a97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8f69b0248d5140 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37a7e00640 1 -- 172.21.15.27:0/2244655976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80ac9f0 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37a7e00640 1 -- 172.21.15.27:0/2244655976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f37b80e71e0 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b4005020 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37a7e00640 1 -- 172.21.15.27:0/2244655976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37b80ac810 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37d466f740 10 client.19542 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce0c670620 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.535+0000 7f37a7e00640 1 -- 172.21.15.27:0/2244655976 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b4005020 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37a7e00640 1 -- 172.21.15.27:0/2244655976 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f37b8098190 con 0x55ce0c5632f0 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37d466f740 10 client.19542 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37d466f740 10 client.19542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37d466f740 20 client.19542 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37d466f740 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c54ad80 0x55ce0c94e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37d466f740 4 client.19542 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce0c8a8600 con 0x55ce0c54ad80 2024-09-21T07:26:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.536+0000 7f37c6000640 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c54ad80 0x55ce0c94e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.537+0000 7f37c6000640 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c54ad80 0x55ce0c94e410 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277d70899ad9900d server_cookie=785b2df772e2b3be in_seq=0 out_seq=0 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.537+0000 7f37a7e00640 10 client.19542 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37a7e00640 1 -- 172.21.15.27:0/2244655976 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce0c8a8600 con 0x55ce0c54ad80 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37a7e00640 10 client.19542 handle_command_reply: tid=0 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 client.19542 shutdown 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 msgr2=0x55ce0c93c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c93c270 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f37b80e49a0 tx=0x7f37b80a97b0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 shutdown_connections 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c54ad80 0x55ce0c94e410 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c9486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 --2- 172.21.15.27:0/2244655976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c93c270 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 >> 172.21.15.27:0/2244655976 conn(0x55ce0c580300 msgr2=0x55ce0c931c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37a7e00640 0 client.19542 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 shutdown_connections 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 1 -- 172.21.15.27:0/2244655976 wait complete. 2024-09-21T07:26:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.538+0000 7f37d466f740 20 client.19542 trim_cache size 0 max 16384 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 Processor -- start 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 -- start start 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c5632f0 0x55ce0c92daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c939440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81b58e0 con 0x55ce0c56b4d0 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81b31b0 con 0x55ce0c58c1b0 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37d466f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81b1020 con 0x55ce0c5632f0 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37c6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37c6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52884/0 (socket says 172.21.15.27:52884) 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37c6000640 1 -- 172.21.15.27:0/824660051 learned_addr learned my addr 172.21.15.27:0/824660051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37c6a00640 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c5632f0 0x55ce0c92daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.554+0000 7f37c7400640 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c939440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37c6000640 1 -- 172.21.15.27:0/824660051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c5632f0 msgr2=0x55ce0c92daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37c6000640 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c5632f0 0x55ce0c92daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37c6000640 1 -- 172.21.15.27:0/824660051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 msgr2=0x55ce0c939440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37c6000640 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c939440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37c6000640 1 -- 172.21.15.27:0/824660051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0c928d50 con 0x55ce0c56b4d0 2024-09-21T07:26:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37c6000640 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93dea0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009350 tx=0x7f37b4004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdba6b2d3cbe1375 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37b2a00640 1 -- 172.21.15.27:0/824660051 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b402f020 con 0x55ce0c56b4d0 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37b2a00640 1 -- 172.21.15.27:0/824660051 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b40054b0 con 0x55ce0c56b4d0 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.555+0000 7f37b2a00640 1 -- 172.21.15.27:0/824660051 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4019690 con 0x55ce0c56b4d0 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 -- 172.21.15.27:0/824660051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c56b4d0 msgr2=0x55ce0c93dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93dea0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009350 tx=0x7f37b4004580 comp rx=0 tx=0).stop 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 -- 172.21.15.27:0/824660051 shutdown_connections 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c939440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c56b4d0 0x55ce0c93dea0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 --2- 172.21.15.27:0/824660051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c5632f0 0x55ce0c92daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 -- 172.21.15.27:0/824660051 >> 172.21.15.27:0/824660051 conn(0x55ce0c580300 msgr2=0x55ce0c93d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 -- 172.21.15.27:0/824660051 shutdown_connections 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.556+0000 7f37d466f740 1 -- 172.21.15.27:0/824660051 wait complete. 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 -- start start 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 Processor -- start 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c944c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c942100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c942640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81b1670 con 0x55ce0c5632f0 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81b2ae0 con 0x55ce0c58c1b0 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81b1d40 con 0x55ce0c56b4d0 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37d466f740 10 client.0 _ll_register_callbacks cb 0x7f37b80ed220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.557+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c944c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c944c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52888/0 (socket says 172.21.15.27:52888) 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6a00640 1 -- 172.21.15.27:0/2383522387 learned_addr learned my addr 172.21.15.27:0/2383522387 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6000640 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c942100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c7400640 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c942640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6000640 1 -- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 msgr2=0x55ce0c942640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6000640 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c942640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6000640 1 -- 172.21.15.27:0/2383522387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 msgr2=0x55ce0c944c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6000640 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c944c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6000640 1 -- 172.21.15.27:0/2383522387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0c942c30 con 0x55ce0c56b4d0 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37c6000640 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c942100 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55ce0c8361b0 tx=0x7f37b4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9fd61f4ec96911 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:26:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.558+0000 7f37a7e00640 1 -- 172.21.15.27:0/2383522387 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b402f040 con 0x55ce0c56b4d0 2024-09-21T07:26:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37a7e00640 1 -- 172.21.15.27:0/2383522387 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f37b4024410 con 0x55ce0c56b4d0 2024-09-21T07:26:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b40054b0 con 0x55ce0c56b4d0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37a7e00640 1 -- 172.21.15.27:0/2383522387 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37b4045050 con 0x55ce0c56b4d0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37d466f740 10 client.29527 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce0c9b8b70 con 0x55ce0c56b4d0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37a7e00640 1 -- 172.21.15.27:0/2383522387 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b40054b0 con 0x55ce0c56b4d0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37a7e00640 1 -- 172.21.15.27:0/2383522387 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f37b4034c30 con 0x55ce0c56b4d0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37d466f740 10 client.29527 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37d466f740 10 client.29527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.559+0000 7f37d466f740 20 client.29527 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.560+0000 7f37d466f740 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c5201c0 0x55ce0c9bb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.560+0000 7f37d466f740 4 client.29527 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.560+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce0c8a40b0 con 0x55ce0c5201c0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.560+0000 7f37c6a00640 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c5201c0 0x55ce0c9bb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.560+0000 7f37c6a00640 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c5201c0 0x55ce0c9bb0b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391335f4542a7405 server_cookie=40d5084640d17f55 in_seq=0 out_seq=0 2024-09-21T07:26:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.560+0000 7f37a7e00640 10 client.29527 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37a7e00640 1 -- 172.21.15.27:0/2383522387 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f37b8014040 con 0x55ce0c5201c0 2024-09-21T07:26:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37a7e00640 10 client.29527 handle_command_reply: tid=0 2024-09-21T07:26:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 client.29527 shutdown 2024-09-21T07:26:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 msgr2=0x55ce0c942100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c942100 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55ce0c8361b0 tx=0x7f37b4009780 comp rx=0 tx=0).stop 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 shutdown_connections 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55ce0c5201c0 0x55ce0c9bb0b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ce0c56b4d0 0x55ce0c942100 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce0c58c1b0 0x55ce0c942640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 --2- 172.21.15.27:0/2383522387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ce0c5632f0 0x55ce0c944c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37a7e00640 0 client.29527 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 >> 172.21.15.27:0/2383522387 conn(0x55ce0c580300 msgr2=0x55ce0c932620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 shutdown_connections 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 1 -- 172.21.15.27:0/2383522387 wait complete. 2024-09-21T07:26:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.578+0000 7f37d466f740 20 client.29527 trim_cache size 0 max 16384 2024-09-21T07:26:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:26:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:26:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:26:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37ac0b6680 msgr2=0x7f37ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37ac0b6680 0x7f37ac0b8b90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f37c0004af0 tx=0x7f37c0005c30 comp rx=0 tx=0).stop 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c8105cd0 msgr2=0x7f37c819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c8105cd0 0x7f37c819c5f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d950 tx=0x7f37b800de10 comp rx=0 tx=0).stop 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 shutdown_connections 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c810bf50 0x7f37c81a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f37ac0b6680 0x7f37ac0b8b90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37c810b600 0x7f37c819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 --2- 172.21.15.27:0/1110723750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37c8105cd0 0x7f37c819c5f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 >> 172.21.15.27:0/1110723750 conn(0x7f37c81016f0 msgr2=0x7f37c8103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 shutdown_connections 2024-09-21T07:26:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:40.579+0000 7f37cd600640 1 -- 172.21.15.27:0/1110723750 wait complete. 2024-09-21T07:26:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:41 smithi119 ceph-mon[34519]: pgmap v803: 129 pgs: 129 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:26:41.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:41 smithi027 ceph-mon[30779]: pgmap v803: 129 pgs: 129 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:26:41.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:41 smithi153 ceph-mon[34747]: pgmap v803: 129 pgs: 129 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:26:43.480 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:43 smithi153 ceph-mon[34747]: pgmap v804: 129 pgs: 129 active+clean; 4.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-21T07:26:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:43 smithi119 ceph-mon[34519]: pgmap v804: 129 pgs: 129 active+clean; 4.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-21T07:26:43.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:43 smithi027 ceph-mon[30779]: pgmap v804: 129 pgs: 129 active+clean; 4.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-21T07:26:45.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:45 smithi119 ceph-mon[34519]: pgmap v805: 129 pgs: 129 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:26:45.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:45 smithi027 ceph-mon[30779]: pgmap v805: 129 pgs: 129 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:26:45.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:45 smithi153 ceph-mon[34747]: pgmap v805: 129 pgs: 129 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:26:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:47 smithi119 ceph-mon[34519]: pgmap v806: 129 pgs: 129 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:47.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:47 smithi027 ceph-mon[30779]: pgmap v806: 129 pgs: 129 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:47.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:47 smithi153 ceph-mon[34747]: pgmap v806: 129 pgs: 129 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:49.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:49 smithi119 ceph-mon[34519]: pgmap v807: 129 pgs: 129 active+clean; 4.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-21T07:26:49.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:49 smithi027 ceph-mon[30779]: pgmap v807: 129 pgs: 129 active+clean; 4.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-21T07:26:49.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:49 smithi153 ceph-mon[34747]: pgmap v807: 129 pgs: 129 active+clean; 4.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-21T07:26:51.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:51 smithi119 ceph-mon[34519]: pgmap v808: 129 pgs: 129 active+clean; 5.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:26:51.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:51 smithi027 ceph-mon[30779]: pgmap v808: 129 pgs: 129 active+clean; 5.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:26:51.674 INFO:tasks.workunit.client.0.smithi027.stdout:437/600: open fd = 3 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:438/600: open fd = 3 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:439/600: open fd = 3 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:440/600: open fd = 3 2024-09-21T07:26:51.675 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:441/600: open fd = 3 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:442/600: open fd = 3 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:443/600: open fd = 3 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.676 INFO:tasks.workunit.client.0.smithi027.stdout:444/600: open fd = 3 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:445/600: open fd = 3 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:446/600: open fd = 3 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:447/600: open fd = 3 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:448/600: open fd = 3 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:449/600: open fd = 3 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:450/600: open fd = 3 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:451/600: open fd = 3 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:452/600: open fd = 3 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:453/600: open fd = 3 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:454/600: open fd = 3 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.679 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:455/600: open fd = 3 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:456/600: open fd = 3 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:457/600: open fd = 3 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:458/600: open fd = 3 2024-09-21T07:26:51.680 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:459/600: open fd = 3 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:460/600: open fd = 3 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:461/600: open fd = 3 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:462/600: open fd = 3 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:463/600: open fd = 3 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:464/600: open fd = 3 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:465/600: open fd = 3 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.682 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:466/600: open fd = 3 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:467/600: open fd = 3 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:468/600: open fd = 3 2024-09-21T07:26:51.683 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:469/600: open fd = 3 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:470/600: open fd = 3 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:471/600: open fd = 3 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.684 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:472/600: open fd = 3 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:473/600: open fd = 3 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:474/600: open fd = 3 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:475/600: open fd = 3 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:476/600: open fd = 3 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:477/600: open fd = 3 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:478/600: open fd = 3 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.686 INFO:tasks.workunit.client.0.smithi027.stdout:479/600: open fd = 3 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:480/600: open fd = 3 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:481/600: open fd = 3 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:482/600: open fd = 3 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.687 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:483/600: open fd = 3 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:484/600: open fd = 3 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:485/600: open fd = 3 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:486/600: open fd = 3 2024-09-21T07:26:51.688 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:487/600: open fd = 3 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:488/600: open fd = 3 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:489/600: open fd = 3 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.689 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:490/600: open fd = 3 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:491/600: open fd = 3 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:492/600: open fd = 3 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:493/600: open fd = 3 2024-09-21T07:26:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:494/600: open fd = 3 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:495/600: open fd = 3 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:496/600: open fd = 3 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.691 INFO:tasks.workunit.client.0.smithi027.stdout:497/600: open fd = 3 2024-09-21T07:26:51.692 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.692 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.692 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.692 INFO:tasks.workunit.client.0.smithi027.stdout:498/600: open fd = 3 2024-09-21T07:26:51.692 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.692 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:26:51.692 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:26:51.692 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:51 smithi153 ceph-mon[34747]: pgmap v808: 129 pgs: 129 active+clean; 5.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-09-21T07:26:53.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:53 smithi119 ceph-mon[34519]: pgmap v809: 129 pgs: 129 active+clean; 5.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:53.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:26:53.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:53 smithi027 ceph-mon[30779]: pgmap v809: 129 pgs: 129 active+clean; 5.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:53.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:53 smithi153 ceph-mon[34747]: pgmap v809: 129 pgs: 129 active+clean; 5.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-09-21T07:26:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 -- 172.21.15.27:0/1404545311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804103910 msgr2=0x7f1804103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 --2- 172.21.15.27:0/1404545311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804103910 0x7f1804103d20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009900 tx=0x7f17f802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 -- 172.21.15.27:0/1404545311 shutdown_connections 2024-09-21T07:26:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 --2- 172.21.15.27:0/1404545311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1804109240 0x7f1804109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 --2- 172.21.15.27:0/1404545311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1804109b90 0x7f18041103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 --2- 172.21.15.27:0/1404545311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804103910 0x7f1804103d20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 -- 172.21.15.27:0/1404545311 >> 172.21.15.27:0/1404545311 conn(0x7f18040ff330 msgr2=0x7f1804101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 -- 172.21.15.27:0/1404545311 shutdown_connections 2024-09-21T07:26:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.785+0000 7f180b000640 1 -- 172.21.15.27:0/1404545311 wait complete. 2024-09-21T07:26:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.786+0000 7f180b000640 1 Processor -- start 2024-09-21T07:26:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.786+0000 7f180b000640 1 -- start start 2024-09-21T07:26:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.786+0000 7f180b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1804103910 0x7f180419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1804109240 0x7f180419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804109b90 0x7f180419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18041138a0 con 0x7f1804109b90 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1804113720 con 0x7f1804103910 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1804113a20 con 0x7f1804109240 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804109b90 0x7f180419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804109b90 0x7f180419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46856/0 (socket says 172.21.15.27:46856) 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f1809c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1804103910 0x7f180419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 -- 172.21.15.27:0/2027266674 learned_addr learned my addr 172.21.15.27:0/2027266674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f1809200640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1804109240 0x7f180419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 -- 172.21.15.27:0/2027266674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1804109240 msgr2=0x7f180419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1804109240 0x7f180419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 -- 172.21.15.27:0/2027266674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1804103910 msgr2=0x7f180419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1804103910 0x7f180419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f180a600640 1 -- 172.21.15.27:0/2027266674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18041a0250 con 0x7f1804109b90 2024-09-21T07:26:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.787+0000 7f1809c00640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1804103910 0x7f180419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.788+0000 7f180a600640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804109b90 0x7f180419fb50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00eb10 tx=0x7f17fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.788+0000 7f1802000640 1 -- 172.21.15.27:0/2027266674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc004b20 con 0x7f1804109b90 2024-09-21T07:26:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.788+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18041a0540 con 0x7f1804109b90 2024-09-21T07:26:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.788+0000 7f1802000640 1 -- 172.21.15.27:0/2027266674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17fc004cc0 con 0x7f1804109b90 2024-09-21T07:26:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.788+0000 7f1802000640 1 -- 172.21.15.27:0/2027266674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc010670 con 0x7f1804109b90 2024-09-21T07:26:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.788+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18041a6cd0 con 0x7f1804109b90 2024-09-21T07:26:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.791+0000 7f18136c4740 1 Processor -- start 2024-09-21T07:26:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.792+0000 7f1802000640 1 -- 172.21.15.27:0/2027266674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f17fc005020 con 0x7f1804109b90 2024-09-21T07:26:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.792+0000 7f18136c4740 1 -- start start 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f18136c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d983e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f18136c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d984350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f18136c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d984890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f18136c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fd3d8f96b0 con 0x55fd3d5b62f0 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f18136c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fd3d8e8ae0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f18136c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd3d8ec8f0 con 0x55fd3d5df1b0 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d984350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d984890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d984890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38062/0 (socket says 172.21.15.27:38062) 2024-09-21T07:26:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 -- 172.21.15.27:0/761983030 learned_addr learned my addr 172.21.15.27:0/761983030 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d983e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 -- 172.21.15.27:0/761983030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 msgr2=0x55fd3d983e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d983e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 -- 172.21.15.27:0/761983030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 msgr2=0x55fd3d984350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.793+0000 7f1809200640 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d984350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f1809200640 1 -- 172.21.15.27:0/761983030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3d8b5780 con 0x55fd3d5be4d0 2024-09-21T07:26:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f1809200640 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d984890 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f17f00058c0 tx=0x7f17f0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514a04dddfe45c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f1802000640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17ec0b6680 0x7f17ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f17f6a00640 1 -- 172.21.15.27:0/761983030 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0005020 con 0x55fd3d5be4d0 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f17f6a00640 1 -- 172.21.15.27:0/761983030 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f00047a0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f17f6a00640 1 -- 172.21.15.27:0/761983030 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000fa20 con 0x55fd3d5be4d0 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f1802000640 1 -- 172.21.15.27:0/2027266674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f17fc0e80b0 con 0x7f1804109b90 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.794+0000 7f1809c00640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17ec0b6680 0x7f17ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f18136c4740 1 -- 172.21.15.27:0/761983030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 msgr2=0x55fd3d984890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f18136c4740 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d984890 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f17f00058c0 tx=0x7f17f0005850 comp rx=0 tx=0).stop 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f1809c00640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17ec0b6680 0x7f17ec0b8b90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f17f8005ab0 tx=0x7f17f8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f18136c4740 1 -- 172.21.15.27:0/761983030 shutdown_connections 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f18136c4740 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d984890 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f18136c4740 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d984350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f18136c4740 1 --2- 172.21.15.27:0/761983030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d983e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.795+0000 7f18136c4740 1 -- 172.21.15.27:0/761983030 >> 172.21.15.27:0/761983030 conn(0x55fd3d5d3300 msgr2=0x55fd3d981d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.796+0000 7f18136c4740 1 -- 172.21.15.27:0/761983030 shutdown_connections 2024-09-21T07:26:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.796+0000 7f18136c4740 1 -- 172.21.15.27:0/761983030 wait complete. 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.796+0000 7f18136c4740 1 -- start start 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.796+0000 7f18136c4740 1 Processor -- start 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f18136c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5b62f0 0x55fd3d993890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f18136c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d993dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f18136c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5df1b0 0x55fd3d99fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f18136c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fd3d820ab0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f18136c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fd3d88f570 con 0x55fd3d5df1b0 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f18136c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd3d996970 con 0x55fd3d5b62f0 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f18136c4740 10 client.0 _ll_register_callbacks cb 0x55fd3d89e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f1809200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d993dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f1809200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d993dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46878/0 (socket says 172.21.15.27:46878) 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f1809200640 1 -- 172.21.15.27:0/3827931671 learned_addr learned my addr 172.21.15.27:0/3827931671 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f1809c00640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5b62f0 0x55fd3d993890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.797+0000 7f180a600640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5df1b0 0x55fd3d99fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f1809200640 1 -- 172.21.15.27:0/3827931671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5b62f0 msgr2=0x55fd3d993890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f1809200640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5b62f0 0x55fd3d993890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f1809200640 1 -- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5df1b0 msgr2=0x55fd3d99fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f1809200640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5df1b0 0x55fd3d99fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f1809200640 1 -- 172.21.15.27:0/3827931671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3d64a790 con 0x55fd3d5be4d0 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f180a600640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5df1b0 0x55fd3d99fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f1809200640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d993dd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009c50 tx=0x7f17f0004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39071a789e9a4fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f17eb400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.798+0000 7f17eb400640 1 -- 172.21.15.27:0/3827931671 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0004ce0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.799+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f00047a0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.799+0000 7f17eb400640 1 -- 172.21.15.27:0/3827931671 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f17f0004fb0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.799+0000 7f17eb400640 1 -- 172.21.15.27:0/3827931671 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17f00051b0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.799+0000 7f18136c4740 10 client.19560 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.799+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd3d6c3620 con 0x55fd3d5be4d0 2024-09-21T07:26:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.799+0000 7f17eb400640 1 -- 172.21.15.27:0/3827931671 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f00047a0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.799+0000 7f17eb400640 1 -- 172.21.15.27:0/3827931671 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f17f001d030 con 0x55fd3d5be4d0 2024-09-21T07:26:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.800+0000 7f18136c4740 10 client.19560 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.800+0000 7f18136c4740 10 client.19560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.800+0000 7f18136c4740 20 client.19560 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.800+0000 7f18136c4740 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d59dd80 0x55fd3d9a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.800+0000 7f18136c4740 4 client.19560 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.800+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd3d8fb600 con 0x55fd3d59dd80 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.800+0000 7f1809c00640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d59dd80 0x55fd3d9a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.801+0000 7f17eb400640 10 client.19560 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.801+0000 7f1809c00640 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d59dd80 0x55fd3d9a5870 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32243bdc091b9853 server_cookie=3299335b05912081 in_seq=0 out_seq=0 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.801+0000 7f17eb400640 1 -- 172.21.15.27:0/3827931671 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd3d8fb600 con 0x55fd3d59dd80 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.801+0000 7f17eb400640 10 client.19560 handle_command_reply: tid=0 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 client.19560 shutdown 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 msgr2=0x55fd3d993dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d993dd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009c50 tx=0x7f17f0004710 comp rx=0 tx=0).stop 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 shutdown_connections 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d59dd80 0x55fd3d9a5870 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5df1b0 0x55fd3d99fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d993dd0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 --2- 172.21.15.27:0/3827931671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5b62f0 0x55fd3d993890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f17eb400640 0 client.19560 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 >> 172.21.15.27:0/3827931671 conn(0x55fd3d5d3300 msgr2=0x55fd3d985800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 shutdown_connections 2024-09-21T07:26:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 1 -- 172.21.15.27:0/3827931671 wait complete. 2024-09-21T07:26:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.802+0000 7f18136c4740 20 client.19560 trim_cache size 0 max 16384 2024-09-21T07:26:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 Processor -- start 2024-09-21T07:26:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 -- start start 2024-09-21T07:26:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5b62f0 0x55fd3d980d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d990a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d990fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18041b1420 con 0x55fd3d5be4d0 2024-09-21T07:26:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18041aecf0 con 0x55fd3d5b62f0 2024-09-21T07:26:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.818+0000 7f18136c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18041acb60 con 0x55fd3d5df1b0 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d990a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5b62f0 0x55fd3d980d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d990a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46892/0 (socket says 172.21.15.27:46892) 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809200640 1 -- 172.21.15.27:0/1206207974 learned_addr learned my addr 172.21.15.27:0/1206207974 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f180a600640 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d990fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809c00640 1 -- 172.21.15.27:0/1206207974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 msgr2=0x55fd3d990fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809c00640 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d990fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809c00640 1 -- 172.21.15.27:0/1206207974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 msgr2=0x55fd3d990a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809c00640 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d990a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809c00640 1 -- 172.21.15.27:0/1206207974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3d9967c0 con 0x55fd3d5b62f0 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.819+0000 7f1809c00640 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5b62f0 0x55fd3d980d20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f17f802a9e0 tx=0x7f17f8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5719321f121a4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.820+0000 7f17f6000640 1 -- 172.21.15.27:0/1206207974 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8035040 con 0x55fd3d5b62f0 2024-09-21T07:26:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.820+0000 7f17f6000640 1 -- 172.21.15.27:0/1206207974 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f8004c40 con 0x55fd3d5b62f0 2024-09-21T07:26:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.820+0000 7f17f6000640 1 -- 172.21.15.27:0/1206207974 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8031700 con 0x55fd3d5b62f0 2024-09-21T07:26:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 -- 172.21.15.27:0/1206207974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5b62f0 msgr2=0x55fd3d980d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5b62f0 0x55fd3d980d20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f17f802a9e0 tx=0x7f17f8040000 comp rx=0 tx=0).stop 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 -- 172.21.15.27:0/1206207974 shutdown_connections 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d990fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5be4d0 0x55fd3d990a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 --2- 172.21.15.27:0/1206207974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5b62f0 0x55fd3d980d20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 -- 172.21.15.27:0/1206207974 >> 172.21.15.27:0/1206207974 conn(0x55fd3d5d3300 msgr2=0x55fd3d981c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 -- 172.21.15.27:0/1206207974 shutdown_connections 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.821+0000 7f18136c4740 1 -- 172.21.15.27:0/1206207974 wait complete. 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 -- start start 2024-09-21T07:26:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 Processor -- start 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d99c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d9977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d997d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18041ad1b0 con 0x55fd3d5b62f0 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f18041ae620 con 0x55fd3d5be4d0 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18041ad880 con 0x55fd3d5df1b0 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.822+0000 7f18136c4740 10 client.0 _ll_register_callbacks cb 0x7f17f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d9977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d99c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d9977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38084/0 (socket says 172.21.15.27:38084) 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 -- 172.21.15.27:0/3422591395 learned_addr learned my addr 172.21.15.27:0/3422591395 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f180a600640 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d997d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 -- 172.21.15.27:0/3422591395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 msgr2=0x55fd3d997d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d997d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 -- 172.21.15.27:0/3422591395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 msgr2=0x55fd3d99c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d99c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 -- 172.21.15.27:0/3422591395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3d998410 con 0x55fd3d5be4d0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809c00640 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d99c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f1809200640 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d9977e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f17f0027560 tx=0x7f17f0005580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff43723b4cf74fac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.823+0000 7f17eb400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f17eb400640 1 -- 172.21.15.27:0/3422591395 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f003a060 con 0x55fd3d5be4d0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f17eb400640 1 -- 172.21.15.27:0/3422591395 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f17f00279e0 con 0x55fd3d5be4d0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f8004c40 con 0x55fd3d5be4d0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f17eb400640 1 -- 172.21.15.27:0/3422591395 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17f0011040 con 0x55fd3d5be4d0 2024-09-21T07:26:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f18136c4740 10 client.29597 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd3da0ff20 con 0x55fd3d5be4d0 2024-09-21T07:26:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f17eb400640 1 -- 172.21.15.27:0/3422591395 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f8004c40 con 0x55fd3d5be4d0 2024-09-21T07:26:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.824+0000 7f17eb400640 1 -- 172.21.15.27:0/3422591395 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f17f001d070 con 0x55fd3d5be4d0 2024-09-21T07:26:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f18136c4740 10 client.29597 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f18136c4740 10 client.29597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f18136c4740 20 client.29597 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f18136c4740 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d5731c0 0x55fd3da12710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f18136c4740 4 client.29597 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd3d8f70b0 con 0x55fd3d5731c0 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f1809c00640 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d5731c0 0x55fd3da12710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f1809c00640 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d5731c0 0x55fd3da12710 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772705f7e9576427 server_cookie=6f5ddad938200834 in_seq=0 out_seq=0 2024-09-21T07:26:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.825+0000 7f17eb400640 10 client.29597 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.840+0000 7f17eb400640 1 -- 172.21.15.27:0/3422591395 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd3d8f70b0 con 0x55fd3d5731c0 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f17eb400640 10 client.29597 handle_command_reply: tid=0 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 client.29597 shutdown 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 msgr2=0x55fd3d9977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d9977e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f17f0027560 tx=0x7f17f0005580 comp rx=0 tx=0).stop 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 shutdown_connections 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fd3d5731c0 0x55fd3da12710 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd3d5df1b0 0x55fd3d997d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd3d5be4d0 0x55fd3d9977e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 --2- 172.21.15.27:0/3422591395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd3d5b62f0 0x55fd3d99c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f17eb400640 0 client.29597 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 >> 172.21.15.27:0/3422591395 conn(0x55fd3d5d3300 msgr2=0x55fd3d981c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 shutdown_connections 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 1 -- 172.21.15.27:0/3422591395 wait complete. 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.841+0000 7f18136c4740 20 client.29597 trim_cache size 0 max 16384 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:26:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:26:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17ec0b6680 msgr2=0x7f17ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17ec0b6680 0x7f17ec0b8b90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f17f8005ab0 tx=0x7f17f8005a20 comp rx=0 tx=0).stop 2024-09-21T07:26:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804109b90 msgr2=0x7f180419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804109b90 0x7f180419fb50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00eb10 tx=0x7f17fc00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 shutdown_connections 2024-09-21T07:26:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1804109240 0x7f180419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f17ec0b6680 0x7f17ec0b8b90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1804109b90 0x7f180419fb50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 --2- 172.21.15.27:0/2027266674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1804103910 0x7f180419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 >> 172.21.15.27:0/2027266674 conn(0x7f18040ff330 msgr2=0x7f1804101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 shutdown_connections 2024-09-21T07:26:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:26:53.842+0000 7f180b000640 1 -- 172.21.15.27:0/2027266674 wait complete. 2024-09-21T07:26:53.968 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 -- 172.21.15.27:0/2509263900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 msgr2=0x7f6a94103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/2509263900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 0x7f6a94103f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009900 tx=0x7f6a8402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 -- 172.21.15.27:0/2509263900 shutdown_connections 2024-09-21T07:26:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/2509263900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94109e10 0x7f6a94110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/2509263900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 0x7f6a941098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/2509263900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 0x7f6a94103f90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 -- 172.21.15.27:0/2509263900 >> 172.21.15.27:0/2509263900 conn(0x7f6a940ff610 msgr2=0x7f6a94101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 -- 172.21.15.27:0/2509263900 shutdown_connections 2024-09-21T07:26:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.587+0000 7f6a9bc00640 1 -- 172.21.15.27:0/2509263900 wait complete. 2024-09-21T07:26:54.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.588+0000 7f6a9bc00640 1 Processor -- start 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.588+0000 7f6a9bc00640 1 -- start start 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.588+0000 7f6a9bc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 0x7f6a9419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.588+0000 7f6a9bc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 0x7f6a9419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.588+0000 7f6a9bc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94109e10 0x7f6a9419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.588+0000 7f6a9bc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94113ad0 con 0x7f6a941094c0 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a9bc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94113950 con 0x7f6a94109e10 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a9bc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94113c50 con 0x7f6a94103b80 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 0x7f6a9419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a99200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 0x7f6a9419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 0x7f6a9419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46910/0 (socket says 172.21.15.27:46910) 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a8be00640 1 -- 172.21.15.27:0/735788859 learned_addr learned my addr 172.21.15.27:0/735788859 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a99c00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94109e10 0x7f6a9419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a99200640 1 -- 172.21.15.27:0/735788859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94109e10 msgr2=0x7f6a9419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a99200640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94109e10 0x7f6a9419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a99200640 1 -- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 msgr2=0x7f6a9419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a99200640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 0x7f6a9419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a99200640 1 -- 172.21.15.27:0/735788859 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a941a03d0 con 0x7f6a94103b80 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.589+0000 7f6a8be00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 0x7f6a9419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.590+0000 7f6a99200640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 0x7f6a9419a460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6a8402a910 tx=0x7f6a84031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.590+0000 7f6a92000640 1 -- 172.21.15.27:0/735788859 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8403f030 con 0x7f6a94103b80 2024-09-21T07:26:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.590+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a941a0660 con 0x7f6a94103b80 2024-09-21T07:26:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.590+0000 7f6a92000640 1 -- 172.21.15.27:0/735788859 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a84004490 con 0x7f6a94103b80 2024-09-21T07:26:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.590+0000 7f6a92000640 1 -- 172.21.15.27:0/735788859 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84035070 con 0x7f6a94103b80 2024-09-21T07:26:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.590+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a6e20 con 0x7f6a94103b80 2024-09-21T07:26:54.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.594+0000 7f6aa282c740 1 Processor -- start 2024-09-21T07:26:54.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.594+0000 7f6aa282c740 1 -- start start 2024-09-21T07:26:54.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.595+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b06d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.595+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 0x55c5b4b072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.595+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.595+0000 7f6aa282c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4a7df70 con 0x55c5b473f390 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.595+0000 7f6aa282c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4a6d3a0 con 0x55c5b4760070 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.595+0000 7f6aa282c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4a711b0 con 0x55c5b47371b0 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b06d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b06d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:38100/0 (socket says 172.21.15.27:38100) 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 -- 172.21.15.27:0/3297180418 learned_addr learned my addr 172.21.15.27:0/3297180418 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a92000640 1 -- 172.21.15.27:0/735788859 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6a84042030 con 0x7f6a94103b80 2024-09-21T07:26:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 -- 172.21.15.27:0/3297180418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 msgr2=0x55c5b4b072a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:26:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 0x55c5b4b072a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 -- 172.21.15.27:0/3297180418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 msgr2=0x55c5b4b077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b077e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 -- 172.21.15.27:0/3297180418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5b4a37c90 con 0x55c5b4760070 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a8be00640 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.596+0000 7f6a99c00640 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b06d60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009fd0 tx=0x7f6a8c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1a79b35ff1cec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.597+0000 7f6a8a000640 1 -- 172.21.15.27:0/3297180418 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004d20 con 0x55c5b4760070 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.597+0000 7f6a8a000640 1 -- 172.21.15.27:0/3297180418 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c004050 con 0x55c5b4760070 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.597+0000 7f6a8a000640 1 -- 172.21.15.27:0/3297180418 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c005240 con 0x55c5b4760070 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.597+0000 7f6a92000640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a780b6680 0x7f6a780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.597+0000 7f6aa282c740 1 -- 172.21.15.27:0/3297180418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 msgr2=0x55c5b4b06d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b06d60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009fd0 tx=0x7f6a8c0118d0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 -- 172.21.15.27:0/3297180418 shutdown_connections 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6a92000640 1 -- 172.21.15.27:0/735788859 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f6a84107d80 con 0x7f6a94103b80 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 0x55c5b4b072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 --2- 172.21.15.27:0/3297180418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b06d60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 -- 172.21.15.27:0/3297180418 >> 172.21.15.27:0/3297180418 conn(0x55c5b4754460 msgr2=0x55c5b4b04c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6a99200640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a780b6680 0x7f6a780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 -- 172.21.15.27:0/3297180418 shutdown_connections 2024-09-21T07:26:54.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.598+0000 7f6aa282c740 1 -- 172.21.15.27:0/3297180418 wait complete. 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.599+0000 7f6a99200640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a780b6680 0x7f6a780b8b90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55c5b4b059e0 tx=0x7f6a84005760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.599+0000 7f6aa282c740 1 -- start start 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.599+0000 7f6aa282c740 1 Processor -- start 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.599+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b47371b0 0x55c5b4b16820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b473f390 0x55c5b4b16d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b4760070 0x55c5b4b22b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6aa282c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c5b48ce700 con 0x55c5b4760070 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6aa282c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4a08540 con 0x55c5b47371b0 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6aa282c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4b197e0 con 0x55c5b473f390 2024-09-21T07:26:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6aa282c740 10 client.0 _ll_register_callbacks cb 0x55c5b4a18430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b4760070 0x55c5b4b22b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b4760070 0x55c5b4b22b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46930/0 (socket says 172.21.15.27:46930) 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 -- 172.21.15.27:0/1597651614 learned_addr learned my addr 172.21.15.27:0/1597651614 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b47371b0 0x55c5b4b16820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a8be00640 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b473f390 0x55c5b4b16d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 -- 172.21.15.27:0/1597651614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b473f390 msgr2=0x55c5b4b16d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b473f390 0x55c5b4b16d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 -- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b47371b0 msgr2=0x55c5b4b16820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.600+0000 7f6a99c00640 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b47371b0 0x55c5b4b16820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6a99c00640 1 -- 172.21.15.27:0/1597651614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5b47cb650 con 0x55c5b4760070 2024-09-21T07:26:54.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6a99c00640 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b4760070 0x55c5b4b22b40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x55c5b4b19070 tx=0x7f6a8c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9ffa5e415e85f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6a82a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6a82a00640 1 -- 172.21.15.27:0/1597651614 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c02d060 con 0x55c5b4760070 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5b4a0ab50 con 0x55c5b4760070 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6a82a00640 1 -- 172.21.15.27:0/1597651614 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f6a8c004a40 con 0x55c5b4760070 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6a82a00640 1 -- 172.21.15.27:0/1597651614 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c034050 con 0x55c5b4760070 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6aa282c740 10 client.19578 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.601+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5b4a25910 con 0x55c5b4760070 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6a82a00640 1 -- 172.21.15.27:0/1597651614 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5b4a0ab50 con 0x55c5b4760070 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6a82a00640 1 -- 172.21.15.27:0/1597651614 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a8c019430 con 0x55c5b4760070 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6aa282c740 10 client.19578 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6aa282c740 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6aa282c740 20 client.19578 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6aa282c740 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b471e9f0 0x55c5b4b28770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6aa282c740 4 client.19578 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5b4a7fec0 con 0x55c5b471e9f0 2024-09-21T07:26:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.602+0000 7f6a8be00640 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b471e9f0 0x55c5b4b28770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.603+0000 7f6a8be00640 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b471e9f0 0x55c5b4b28770 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f64289c93c3005 server_cookie=4ca8b0c3fb38b9e3 in_seq=0 out_seq=0 2024-09-21T07:26:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.603+0000 7f6a82a00640 10 client.19578 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6a82a00640 1 -- 172.21.15.27:0/1597651614 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5b4a7fec0 con 0x55c5b471e9f0 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6a82a00640 10 client.19578 handle_command_reply: tid=0 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 client.19578 shutdown 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b4760070 msgr2=0x55c5b4b22b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b4760070 0x55c5b4b22b40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x55c5b4b19070 tx=0x7f6a8c005ad0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 shutdown_connections 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b471e9f0 0x55c5b4b28770 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b473f390 0x55c5b4b16d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b4760070 0x55c5b4b22b40 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 --2- 172.21.15.27:0/1597651614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b47371b0 0x55c5b4b16820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6a82a00640 0 client.19578 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:54.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 >> 172.21.15.27:0/1597651614 conn(0x55c5b4754460 msgr2=0x55c5b4b03210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:54.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 shutdown_connections 2024-09-21T07:26:54.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.604+0000 7f6aa282c740 1 -- 172.21.15.27:0/1597651614 wait complete. 2024-09-21T07:26:54.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.605+0000 7f6aa282c740 20 client.19578 trim_cache size 0 max 16384 2024-09-21T07:26:54.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 Processor -- start 2024-09-21T07:26:54.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 -- start start 2024-09-21T07:26:54.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b47371b0 0x55c5b4b0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b473f390 0x55c5b4b18450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b4760070 0x55c5b4b22a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941b1560 con 0x55c5b47371b0 2024-09-21T07:26:54.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941aee30 con 0x55c5b473f390 2024-09-21T07:26:54.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.623+0000 7f6aa282c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941acca0 con 0x55c5b4760070 2024-09-21T07:26:54.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b47371b0 0x55c5b4b0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b47371b0 0x55c5b4b0be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46934/0 (socket says 172.21.15.27:46934) 2024-09-21T07:26:54.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 -- 172.21.15.27:0/467732732 learned_addr learned my addr 172.21.15.27:0/467732732 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a8be00640 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b473f390 0x55c5b4b18450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b4760070 0x55c5b4b22a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 -- 172.21.15.27:0/467732732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b4760070 msgr2=0x55c5b4b22a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b4760070 0x55c5b4b22a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 -- 172.21.15.27:0/467732732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b473f390 msgr2=0x55c5b4b18450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b473f390 0x55c5b4b18450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.624+0000 7f6a99c00640 1 -- 172.21.15.27:0/467732732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5b4afeea0 con 0x55c5b47371b0 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.625+0000 7f6a99c00640 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b47371b0 0x55c5b4b0be40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00a010 tx=0x7f6a8c004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866a0a2c9dde0bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.625+0000 7f6a89600640 1 -- 172.21.15.27:0/467732732 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c02d070 con 0x55c5b47371b0 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.625+0000 7f6a89600640 1 -- 172.21.15.27:0/467732732 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c003c50 con 0x55c5b47371b0 2024-09-21T07:26:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.625+0000 7f6a89600640 1 -- 172.21.15.27:0/467732732 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004d00 con 0x55c5b47371b0 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 -- 172.21.15.27:0/467732732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b47371b0 msgr2=0x55c5b4b0be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b47371b0 0x55c5b4b0be40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00a010 tx=0x7f6a8c004120 comp rx=0 tx=0).stop 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 -- 172.21.15.27:0/467732732 shutdown_connections 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b4760070 0x55c5b4b22a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b473f390 0x55c5b4b18450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 --2- 172.21.15.27:0/467732732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b47371b0 0x55c5b4b0be40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 -- 172.21.15.27:0/467732732 >> 172.21.15.27:0/467732732 conn(0x55c5b4754460 msgr2=0x55c5b4b17d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 -- 172.21.15.27:0/467732732 shutdown_connections 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 -- 172.21.15.27:0/467732732 wait complete. 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.626+0000 7f6aa282c740 1 -- start start 2024-09-21T07:26:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 1 Processor -- start 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 0x55c5b4b1d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b1d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b91fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ad2f0 con 0x55c5b473f390 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ae760 con 0x55c5b4760070 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ad9c0 con 0x55c5b47371b0 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6aa282c740 10 client.0 _ll_register_callbacks cb 0x7f6a8c004050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6a8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b1d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6a8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b1d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46946/0 (socket says 172.21.15.27:46946) 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.627+0000 7f6a8be00640 1 -- 172.21.15.27:0/2475645968 learned_addr learned my addr 172.21.15.27:0/2475645968 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a99c00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 0x55c5b4b1d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a99c00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b91fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a8be00640 1 -- 172.21.15.27:0/2475645968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 msgr2=0x55c5b4b1d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a8be00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 0x55c5b4b1d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a8be00640 1 -- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 msgr2=0x55c5b4b91fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a8be00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b91fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a8be00640 1 -- 172.21.15.27:0/2475645968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5b4b92520 con 0x55c5b473f390 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a99c00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b91fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a8be00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b1d670 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55c5b4b28d70 tx=0x7f6a7c012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514113a6c9df7d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.628+0000 7f6a82a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.629+0000 7f6a82a00640 1 -- 172.21.15.27:0/2475645968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c01ea00 con 0x55c5b473f390 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.629+0000 7f6a82a00640 1 -- 172.21.15.27:0/2475645968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f6a7c01eba0 con 0x55c5b473f390 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.629+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c003c50 con 0x55c5b473f390 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.629+0000 7f6a82a00640 1 -- 172.21.15.27:0/2475645968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a7c030050 con 0x55c5b473f390 2024-09-21T07:26:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.629+0000 7f6aa282c740 10 client.19584 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.629+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5b4b93710 con 0x55c5b473f390 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.629+0000 7f6a82a00640 1 -- 172.21.15.27:0/2475645968 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c003c50 con 0x55c5b473f390 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6a82a00640 1 -- 172.21.15.27:0/2475645968 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a7c005400 con 0x55c5b473f390 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6aa282c740 10 client.19584 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6aa282c740 10 client.19584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6aa282c740 20 client.19584 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6aa282c740 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b46f3da0 0x55c5b4b95ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6aa282c740 4 client.19584 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5b4a7b970 con 0x55c5b46f3da0 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6a99c00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b46f3da0 0x55c5b4b95ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6a99c00640 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b46f3da0 0x55c5b4b95ef0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a133a708077247a server_cookie=693965194954b867 in_seq=0 out_seq=0 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.630+0000 7f6a82a00640 10 client.19584 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6a82a00640 1 -- 172.21.15.27:0/2475645968 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6338 (crc 0 0 0) 0x55c5b4a7b970 con 0x55c5b46f3da0 2024-09-21T07:26:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6a82a00640 10 client.19584 handle_command_reply: tid=0 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 client.19584 shutdown 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 msgr2=0x55c5b4b1d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b1d670 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55c5b4b28d70 tx=0x7f6a7c012f90 comp rx=0 tx=0).stop 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 shutdown_connections 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c5b46f3da0 0x55c5b4b95ef0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5b4760070 0x55c5b4b91fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5b473f390 0x55c5b4b1d670 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 --2- 172.21.15.27:0/2475645968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5b47371b0 0x55c5b4b1d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 >> 172.21.15.27:0/2475645968 conn(0x55c5b4754460 msgr2=0x55c5b4b184a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6a82a00640 0 client.19584 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.631+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 shutdown_connections 2024-09-21T07:26:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.632+0000 7f6aa282c740 1 -- 172.21.15.27:0/2475645968 wait complete. 2024-09-21T07:26:54.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.632+0000 7f6aa282c740 20 client.19584 trim_cache size 0 max 16384 2024-09-21T07:26:54.638 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:26:54.638 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:26:54.638 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:26:54.638 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:26:54.638 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:26:54.638 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.639 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.641 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.642 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:26:54.643 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:26:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.632+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a780b6680 msgr2=0x7f6a780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.632+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a780b6680 0x7f6a780b8b90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55c5b4b059e0 tx=0x7f6a84005760 comp rx=0 tx=0).stop 2024-09-21T07:26:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.632+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 msgr2=0x7f6a9419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.632+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 0x7f6a9419a460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6a8402a910 tx=0x7f6a84031a20 comp rx=0 tx=0).stop 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 shutdown_connections 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a780b6680 0x7f6a780b8b90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a94109e10 0x7f6a9419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a941094c0 0x7f6a9419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 --2- 172.21.15.27:0/735788859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a94103b80 0x7f6a9419a460 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 >> 172.21.15.27:0/735788859 conn(0x7f6a940ff610 msgr2=0x7f6a94101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 shutdown_connections 2024-09-21T07:26:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:54.633+0000 7f6a9bc00640 1 -- 172.21.15.27:0/735788859 wait complete. 2024-09-21T07:26:55.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:55 smithi119 ceph-mon[34519]: pgmap v810: 129 pgs: 129 active+clean; 5.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:26:55.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:55 smithi027 ceph-mon[30779]: pgmap v810: 129 pgs: 129 active+clean; 5.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:26:55.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:26:55.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:26:55.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:55 smithi153 ceph-mon[34747]: pgmap v810: 129 pgs: 129 active+clean; 5.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:26:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:56.454 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:56 smithi027 ceph-mon[30779]: pgmap v811: 129 pgs: 129 active+clean; 5.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 -- 172.21.15.27:0/3731458984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c105d90 msgr2=0x7f091c1061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 --2- 172.21.15.27:0/3731458984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c105d90 0x7f091c1061a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f090c009900 tx=0x7f090c02a410 comp rx=0 tx=0).stop 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 -- 172.21.15.27:0/3731458984 shutdown_connections 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 --2- 172.21.15.27:0/3731458984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f091c10b6d0 0x7f091c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 --2- 172.21.15.27:0/3731458984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c10c020 0x7f091c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 --2- 172.21.15.27:0/3731458984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c105d90 0x7f091c1061a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 -- 172.21.15.27:0/3731458984 >> 172.21.15.27:0/3731458984 conn(0x7f091c1017c0 msgr2=0x7f091c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 -- 172.21.15.27:0/3731458984 shutdown_connections 2024-09-21T07:26:56.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.636+0000 7f0920e00640 1 -- 172.21.15.27:0/3731458984 wait complete. 2024-09-21T07:26:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.637+0000 7f0920e00640 1 Processor -- start 2024-09-21T07:26:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0920e00640 1 -- start start 2024-09-21T07:26:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0920e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c105d90 0x7f091c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0920e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c10b6d0 0x7f091c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0920e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f091c10c020 0x7f091c1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0920e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f091c115d30 con 0x7f091c10b6d0 2024-09-21T07:26:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0920e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f091c115bb0 con 0x7f091c105d90 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0920e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f091c115eb0 con 0x7f091c10c020 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0913e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c10b6d0 0x7f091c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0913e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c10b6d0 0x7f091c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46970/0 (socket says 172.21.15.27:46970) 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f091a800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f091c10c020 0x7f091c1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0913e00640 1 -- 172.21.15.27:0/3427502820 learned_addr learned my addr 172.21.15.27:0/3427502820 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.638+0000 7f0919e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c105d90 0x7f091c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.639+0000 7f091a800640 1 -- 172.21.15.27:0/3427502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c105d90 msgr2=0x7f091c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.639+0000 7f091a800640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c105d90 0x7f091c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.639+0000 7f091a800640 1 -- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c10b6d0 msgr2=0x7f091c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.639+0000 7f091a800640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c10b6d0 0x7f091c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.639+0000 7f091a800640 1 -- 172.21.15.27:0/3427502820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c1a2670 con 0x7f091c10c020 2024-09-21T07:26:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.639+0000 7f091a800640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f091c10c020 0x7f091c1a1f40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0914009fd0 tx=0x7f091400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.639+0000 7f0912a00640 1 -- 172.21.15.27:0/3427502820 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091400ce20 con 0x7f091c10c020 2024-09-21T07:26:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.640+0000 7f0912a00640 1 -- 172.21.15.27:0/3427502820 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0914005020 con 0x7f091c10c020 2024-09-21T07:26:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.640+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c1a2960 con 0x7f091c10c020 2024-09-21T07:26:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.640+0000 7f0912a00640 1 -- 172.21.15.27:0/3427502820 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914010690 con 0x7f091c10c020 2024-09-21T07:26:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.640+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c1a90c0 con 0x7f091c10c020 2024-09-21T07:26:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.641+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091c1061a0 con 0x7f091c10c020 2024-09-21T07:26:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.643+0000 7f0912a00640 1 -- 172.21.15.27:0/3427502820 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f09140054d0 con 0x7f091c10c020 2024-09-21T07:26:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.645+0000 7f0912a00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09000b6680 0x7f09000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.645+0000 7f0912a00640 1 -- 172.21.15.27:0/3427502820 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f09140e81d0 con 0x7f091c10c020 2024-09-21T07:26:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.645+0000 7f0919e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09000b6680 0x7f09000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.646+0000 7f0919e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09000b6680 0x7f09000b8b90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f090c005ab0 tx=0x7f090c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.647+0000 7f0912a00640 1 -- 172.21.15.27:0/3427502820 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09140ae310 con 0x7f091c10c020 2024-09-21T07:26:56.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:56 smithi153 ceph-mon[34747]: pgmap v811: 129 pgs: 129 active+clean; 5.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-21T07:26:56.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:56 smithi119 ceph-mon[34519]: pgmap v811: 129 pgs: 129 active+clean; 5.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-21T07:26:56.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.891+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f091c19d830 con 0x7f091c10c020 2024-09-21T07:26:56.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.892+0000 7f0912a00640 1 -- 172.21.15.27:0/3427502820 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f091c19d830 con 0x7f091c10c020 2024-09-21T07:26:56.894 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:26:56.895 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:26:56.895 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09000b6680 msgr2=0x7f09000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09000b6680 0x7f09000b8b90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f090c005ab0 tx=0x7f090c005a20 comp rx=0 tx=0).stop 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f091c10c020 msgr2=0x7f091c1a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f091c10c020 0x7f091c1a1f40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0914009fd0 tx=0x7f091400ef10 comp rx=0 tx=0).stop 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 shutdown_connections 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f091c10c020 0x7f091c1a1f40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f09000b6680 0x7f09000b8b90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c10b6d0 0x7f091c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 --2- 172.21.15.27:0/3427502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c105d90 0x7f091c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 >> 172.21.15.27:0/3427502820 conn(0x7f091c1017c0 msgr2=0x7f091c10ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:56.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 shutdown_connections 2024-09-21T07:26:56.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:56.895+0000 7f0920e00640 1 -- 172.21.15.27:0/3427502820 wait complete. 2024-09-21T07:26:57.551 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:57 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3427502820' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:57.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:26:57.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:57 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3427502820' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:57.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:57 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3427502820' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:26:57.981 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:26:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:26:58 smithi027 ceph-mon[30779]: pgmap v812: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-21T07:26:58.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 -- 172.21.15.27:0/2987895025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1810b6c0 msgr2=0x7f8c1810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 --2- 172.21.15.27:0/2987895025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1810b6c0 0x7f8c1810bad0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009900 tx=0x7f8c0002a410 comp rx=0 tx=0).stop 2024-09-21T07:26:58.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 -- 172.21.15.27:0/2987895025 shutdown_connections 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 --2- 172.21.15.27:0/2987895025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c1810c010 0x7f8c18112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 --2- 172.21.15.27:0/2987895025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1810b6c0 0x7f8c1810bad0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 --2- 172.21.15.27:0/2987895025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c18105d80 0x7f8c18106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 -- 172.21.15.27:0/2987895025 >> 172.21.15.27:0/2987895025 conn(0x7f8c18101810 msgr2=0x7f8c18103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 -- 172.21.15.27:0/2987895025 shutdown_connections 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.622+0000 7f8c1e200640 1 -- 172.21.15.27:0/2987895025 wait complete. 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.623+0000 7f8c1e200640 1 Processor -- start 2024-09-21T07:26:58.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.623+0000 7f8c1e200640 1 -- start start 2024-09-21T07:26:58.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c1e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c18105d80 0x7f8c1819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c1e200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c1810b6c0 0x7f8c1819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c1e200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1810c010 0x7f8c181a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c1e200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18115cd0 con 0x7f8c18105d80 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c1e200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18115b50 con 0x7f8c1810b6c0 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c1e200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18115e50 con 0x7f8c1810c010 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c18105d80 0x7f8c1819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c18105d80 0x7f8c1819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38934/0 (socket says 172.21.15.27:38934) 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c17400640 1 -- 172.21.15.27:0/3829491404 learned_addr learned my addr 172.21.15.27:0/3829491404 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c17e00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1810c010 0x7f8c181a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.624+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c1810b6c0 0x7f8c1819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c17e00640 1 -- 172.21.15.27:0/3829491404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c1810b6c0 msgr2=0x7f8c1819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c17e00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c1810b6c0 0x7f8c1819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c17e00640 1 -- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c18105d80 msgr2=0x7f8c1819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c17e00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c18105d80 0x7f8c1819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c17e00640 1 -- 172.21.15.27:0/3829491404 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c181a2630 con 0x7f8c1810c010 2024-09-21T07:26:58.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c17400640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c18105d80 0x7f8c1819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:58.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c1810b6c0 0x7f8c1819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:58.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c17e00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1810c010 0x7f8c181a1f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009fd0 tx=0x7f8c1000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c14c00640 1 -- 172.21.15.27:0/3829491404 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000ce20 con 0x7f8c1810c010 2024-09-21T07:26:58.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c181a2920 con 0x7f8c1810c010 2024-09-21T07:26:58.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c14c00640 1 -- 172.21.15.27:0/3829491404 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c10005020 con 0x7f8c1810c010 2024-09-21T07:26:58.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.625+0000 7f8c14c00640 1 -- 172.21.15.27:0/3829491404 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10010690 con 0x7f8c1810c010 2024-09-21T07:26:58.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.626+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c181a9080 con 0x7f8c1810c010 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.629+0000 7f8c24e30740 1 Processor -- start 2024-09-21T07:26:58.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.629+0000 7f8c14c00640 1 -- 172.21.15.27:0/3829491404 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8c100054d0 con 0x7f8c1810c010 2024-09-21T07:26:58.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c24e30740 1 -- start start 2024-09-21T07:26:58.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea526fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea52702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea8390 0x564ea5270820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c24e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564ea51e6fb0 con 0x564ea4ea01b0 2024-09-21T07:26:58.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c24e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564ea51d63e0 con 0x564ea4ea8390 2024-09-21T07:26:58.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c24e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ea51da1f0 con 0x564ea4ec9070 2024-09-21T07:26:58.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea52702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.630+0000 7f8c0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea526fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea52702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38942/0 (socket says 172.21.15.27:38942) 2024-09-21T07:26:58.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 -- 172.21.15.27:0/2977339122 learned_addr learned my addr 172.21.15.27:0/2977339122 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea8390 0x564ea5270820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 -- 172.21.15.27:0/2977339122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 msgr2=0x564ea526fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea526fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 -- 172.21.15.27:0/2977339122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea8390 msgr2=0x564ea5270820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea8390 0x564ea5270820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 -- 172.21.15.27:0/2977339122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea51a0cd0 con 0x564ea4ea01b0 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea526fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c14c00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bfc0b6680 0x7f8bfc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.631+0000 7f8c17400640 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea52702e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8c08011630 tx=0x7f8c08011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3045c67428b96e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.632+0000 7f8c14c00640 1 -- 172.21.15.27:0/3829491404 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f8c100e81b0 con 0x7f8c1810c010 2024-09-21T07:26:58.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.632+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bfc0b6680 0x7f8bfc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.632+0000 7f8c07e00640 1 -- 172.21.15.27:0/2977339122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0801e070 con 0x564ea4ea01b0 2024-09-21T07:26:58.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.632+0000 7f8c07e00640 1 -- 172.21.15.27:0/2977339122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c0800fba0 con 0x564ea4ea01b0 2024-09-21T07:26:58.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.632+0000 7f8c07e00640 1 -- 172.21.15.27:0/2977339122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08019c30 con 0x564ea4ea01b0 2024-09-21T07:26:58.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 -- 172.21.15.27:0/2977339122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 msgr2=0x564ea52702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bfc0b6680 0x7f8bfc0b8b90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009230 tx=0x7f8c00031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea52702e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8c08011630 tx=0x7f8c08011af0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 -- 172.21.15.27:0/2977339122 shutdown_connections 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea8390 0x564ea5270820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea52702e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 --2- 172.21.15.27:0/2977339122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea526fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 -- 172.21.15.27:0/2977339122 >> 172.21.15.27:0/2977339122 conn(0x564ea4ebd460 msgr2=0x564ea526dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 -- 172.21.15.27:0/2977339122 shutdown_connections 2024-09-21T07:26:58.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.633+0000 7f8c24e30740 1 -- 172.21.15.27:0/2977339122 wait complete. 2024-09-21T07:26:58.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.634+0000 7f8c24e30740 1 -- start start 2024-09-21T07:26:58.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.634+0000 7f8c24e30740 1 Processor -- start 2024-09-21T07:26:58.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea527b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea527bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ec9070 0x564ea52878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c24e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564ea5037700 con 0x564ea4ea01b0 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c24e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564ea5171580 con 0x564ea4ec9070 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c24e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ea527e580 con 0x564ea4ea8390 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c24e30740 10 client.0 _ll_register_callbacks cb 0x564ea5181470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea527b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea527b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38954/0 (socket says 172.21.15.27:38954) 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c17400640 1 -- 172.21.15.27:0/1158440911 learned_addr learned my addr 172.21.15.27:0/1158440911 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c17e00640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ec9070 0x564ea52878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.635+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea527bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c17400640 1 -- 172.21.15.27:0/1158440911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 msgr2=0x564ea527bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c17400640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea527bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c17400640 1 -- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ec9070 msgr2=0x564ea52878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c17400640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ec9070 0x564ea52878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c17400640 1 -- 172.21.15.27:0/1158440911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea4f34650 con 0x564ea4ea01b0 2024-09-21T07:26:58.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c17e00640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ec9070 0x564ea52878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c17400640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea527b5e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8c08019440 tx=0x7f8c08019470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea34703c88aea6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c04c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c04c00640 1 -- 172.21.15.27:0/1158440911 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0801e040 con 0x564ea4ea01b0 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c04c00640 1 -- 172.21.15.27:0/1158440911 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f8c08005540 con 0x564ea4ea01b0 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.636+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564ea5173b90 con 0x564ea4ea01b0 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c04c00640 1 -- 172.21.15.27:0/1158440911 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c08030050 con 0x564ea4ea01b0 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 10 client.19602 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ea518e950 con 0x564ea4ea01b0 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c04c00640 1 -- 172.21.15.27:0/1158440911 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564ea5173b90 con 0x564ea4ea01b0 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c04c00640 1 -- 172.21.15.27:0/1158440911 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8c08021070 con 0x564ea4ea01b0 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 10 client.19602 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 10 client.19602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 20 client.19602 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e879f0 0x564ea528d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 4 client.19602 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.637+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ea51e8f00 con 0x564ea4e879f0 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.638+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e879f0 0x564ea528d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.638+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e879f0 0x564ea528d5d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=927e232d6478d576 server_cookie=3768f6bf3a9c0dfe in_seq=0 out_seq=0 2024-09-21T07:26:58.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.638+0000 7f8c04c00640 10 client.19602 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:58.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.639+0000 7f8c04c00640 1 -- 172.21.15.27:0/1158440911 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ea51e8f00 con 0x564ea4e879f0 2024-09-21T07:26:58.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.639+0000 7f8c04c00640 10 client.19602 handle_command_reply: tid=0 2024-09-21T07:26:58.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.639+0000 7f8c24e30740 1 client.19602 shutdown 2024-09-21T07:26:58.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.639+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 msgr2=0x564ea527b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.639+0000 7f8c24e30740 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea527b5e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8c08019440 tx=0x7f8c08019470 comp rx=0 tx=0).stop 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 shutdown_connections 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e879f0 0x564ea528d5d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea527bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ec9070 0x564ea52878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c04c00640 0 client.19602 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 --2- 172.21.15.27:0/1158440911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea01b0 0x564ea527b5e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 >> 172.21.15.27:0/1158440911 conn(0x564ea4ebd460 msgr2=0x564ea5272f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 shutdown_connections 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 1 -- 172.21.15.27:0/1158440911 wait complete. 2024-09-21T07:26:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.640+0000 7f8c24e30740 20 client.19602 trim_cache size 0 max 16384 2024-09-21T07:26:58.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.654+0000 7f8c24e30740 1 Processor -- start 2024-09-21T07:26:58.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.654+0000 7f8c24e30740 1 -- start start 2024-09-21T07:26:58.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea52735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea8390 0x564ea52787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea5278d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c24e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b5990 con 0x564ea4ea8390 2024-09-21T07:26:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c24e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b3260 con 0x564ea4ea01b0 2024-09-21T07:26:58.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c24e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b10d0 con 0x564ea4ec9070 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea8390 0x564ea52787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea8390 0x564ea52787d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38962/0 (socket says 172.21.15.27:38962) 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c17400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea52735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 -- 172.21.15.27:0/620305311 learned_addr learned my addr 172.21.15.27:0/620305311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c17e00640 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea5278d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 -- 172.21.15.27:0/620305311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 msgr2=0x564ea5278d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea5278d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 -- 172.21.15.27:0/620305311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 msgr2=0x564ea52735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea52735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c0ea00640 1 -- 172.21.15.27:0/620305311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea5273af0 con 0x564ea4ea8390 2024-09-21T07:26:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.655+0000 7f8c17400640 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea52735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.656+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea8390 0x564ea52787d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009900 tx=0x7f8c0002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbca8432ea5a9b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.656+0000 7f8c07400640 1 -- 172.21.15.27:0/620305311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0005a070 con 0x564ea4ea8390 2024-09-21T07:26:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.656+0000 7f8c07400640 1 -- 172.21.15.27:0/620305311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c000054f0 con 0x564ea4ea8390 2024-09-21T07:26:58.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.656+0000 7f8c07400640 1 -- 172.21.15.27:0/620305311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00035040 con 0x564ea4ea8390 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 -- 172.21.15.27:0/620305311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea8390 msgr2=0x564ea52787d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea8390 0x564ea52787d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009900 tx=0x7f8c0002a410 comp rx=0 tx=0).stop 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 -- 172.21.15.27:0/620305311 shutdown_connections 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ec9070 0x564ea5278d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ea8390 0x564ea52787d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 --2- 172.21.15.27:0/620305311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea52735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 -- 172.21.15.27:0/620305311 >> 172.21.15.27:0/620305311 conn(0x564ea4ebd460 msgr2=0x564ea5272ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 -- 172.21.15.27:0/620305311 shutdown_connections 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.657+0000 7f8c24e30740 1 -- 172.21.15.27:0/620305311 wait complete. 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.658+0000 7f8c24e30740 1 -- start start 2024-09-21T07:26:58.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.658+0000 7f8c24e30740 1 Processor -- start 2024-09-21T07:26:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea5284060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea52815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c24e30740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ec9070 0x564ea5281b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c24e30740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b1720 con 0x564ea4ec9070 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c24e30740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b2b90 con 0x564ea4ea01b0 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c24e30740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b1df0 con 0x564ea4ea8390 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c24e30740 10 client.0 _ll_register_callbacks cb 0x7f8c08005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ec9070 0x564ea5281b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea52815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea5284060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ec9070 0x564ea5281b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38978/0 (socket says 172.21.15.27:38978) 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17e00640 1 -- 172.21.15.27:0/3644200391 learned_addr learned my addr 172.21.15.27:0/3644200391 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c0ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea52815d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52906/0 (socket says 172.21.15.27:52906) 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17e00640 1 -- 172.21.15.27:0/3644200391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 msgr2=0x564ea52815d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17e00640 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea52815d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17e00640 1 -- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 msgr2=0x564ea5284060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.659+0000 7f8c17e00640 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea5284060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c17e00640 1 -- 172.21.15.27:0/3644200391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea5282320 con 0x564ea4ec9070 2024-09-21T07:26:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c0ea00640 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea52815d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c17400640 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea5284060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c17e00640 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ec9070 0x564ea5281b90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c100951e0 tx=0x7f8c100aa370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8311ea3e0e49820 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c04c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c04c00640 1 -- 172.21.15.27:0/3644200391 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100d5030 con 0x564ea4ec9070 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c04c00640 1 -- 172.21.15.27:0/3644200391 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f8c100ace50 con 0x564ea4ec9070 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c000054f0 con 0x564ea4ec9070 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c04c00640 1 -- 172.21.15.27:0/3644200391 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c100ad700 con 0x564ea4ec9070 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c24e30740 10 client.19608 fetch_fsmap learned FSMap version 27 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.660+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ea52f7e60 con 0x564ea4ec9070 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c04c00640 1 -- 172.21.15.27:0/3644200391 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c000054f0 con 0x564ea4ec9070 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c04c00640 1 -- 172.21.15.27:0/3644200391 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8c100a9040 con 0x564ea4ec9070 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c24e30740 10 client.19608 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:26:58.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c24e30740 10 client.19608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c24e30740 20 client.19608 populate_metadata read hostname 'smithi027' 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c24e30740 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e5cda0 0x564ea52fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c24e30740 4 client.19608 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ea51e49b0 con 0x564ea4e5cda0 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.661+0000 7f8c17400640 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e5cda0 0x564ea52fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.662+0000 7f8c17400640 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e5cda0 0x564ea52fa670 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32e2cb4ac396dc2d server_cookie=3b3c3b1f71aaf5aa in_seq=0 out_seq=0 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.662+0000 7f8c04c00640 10 client.19608 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.662+0000 7f8c04c00640 1 -- 172.21.15.27:0/3644200391 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ea51e49b0 con 0x564ea4e5cda0 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.662+0000 7f8c04c00640 10 client.19608 handle_command_reply: tid=0 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.662+0000 7f8c24e30740 1 client.19608 shutdown 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.662+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ec9070 msgr2=0x564ea5281b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.662+0000 7f8c24e30740 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ec9070 0x564ea5281b90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c100951e0 tx=0x7f8c100aa370 comp rx=0 tx=0).stop 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 shutdown_connections 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564ea4e5cda0 0x564ea52fa670 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea4ea8390 0x564ea52815d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564ea4ec9070 0x564ea5281b90 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 --2- 172.21.15.27:0/3644200391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea4ea01b0 0x564ea5284060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 >> 172.21.15.27:0/3644200391 conn(0x564ea4ebd460 msgr2=0x564ea5270dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c04c00640 0 client.19608 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 shutdown_connections 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 1 -- 172.21.15.27:0/3644200391 wait complete. 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.663+0000 7f8c24e30740 20 client.19608 trim_cache size 0 max 16384 2024-09-21T07:26:58.667 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bfc0b6680 msgr2=0x7f8bfc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bfc0b6680 0x7f8bfc0b8b90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009230 tx=0x7f8c00031400 comp rx=0 tx=0).stop 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1810c010 msgr2=0x7f8c181a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1810c010 0x7f8c181a1f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009fd0 tx=0x7f8c1000ef10 comp rx=0 tx=0).stop 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 shutdown_connections 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1810c010 0x7f8c181a1f00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bfc0b6680 0x7f8bfc0b8b90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c1810b6c0 0x7f8c1819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 --2- 172.21.15.27:0/3829491404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c18105d80 0x7f8c1819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 >> 172.21.15.27:0/3829491404 conn(0x7f8c18101810 msgr2=0x7f8c18103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 shutdown_connections 2024-09-21T07:26:58.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:26:58.664+0000 7f8c1e200640 1 -- 172.21.15.27:0/3829491404 wait complete. 2024-09-21T07:26:58.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:26:58 smithi153 ceph-mon[34747]: pgmap v812: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-21T07:26:58.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:26:58 smithi119 ceph-mon[34519]: pgmap v812: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-21T07:26:59.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:26:59.828 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:00.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.417+0000 7ff822c00640 1 -- 172.21.15.27:0/2187745667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c1049b0 msgr2=0x7ff81c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.417+0000 7ff822c00640 1 --2- 172.21.15.27:0/2187745667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c1049b0 0x7ff81c104dc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009900 tx=0x7ff80c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.417+0000 7ff822c00640 1 -- 172.21.15.27:0/2187745667 shutdown_connections 2024-09-21T07:27:00.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.417+0000 7ff822c00640 1 --2- 172.21.15.27:0/2187745667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff81c105300 0x7ff81c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.417+0000 7ff822c00640 1 --2- 172.21.15.27:0/2187745667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c10de10 0x7ff81c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.417+0000 7ff822c00640 1 --2- 172.21.15.27:0/2187745667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c1049b0 0x7ff81c104dc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.417+0000 7ff822c00640 1 -- 172.21.15.27:0/2187745667 >> 172.21.15.27:0/2187745667 conn(0x7ff81c079760 msgr2=0x7ff81c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.418+0000 7ff822c00640 1 -- 172.21.15.27:0/2187745667 shutdown_connections 2024-09-21T07:27:00.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.418+0000 7ff822c00640 1 -- 172.21.15.27:0/2187745667 wait complete. 2024-09-21T07:27:00.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 Processor -- start 2024-09-21T07:27:00.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 -- start start 2024-09-21T07:27:00.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c1049b0 0x7ff81c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff81c105300 0x7ff81c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c10de10 0x7ff81c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c113ae0 con 0x7ff81c1049b0 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c113960 con 0x7ff81c10de10 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.419+0000 7ff822c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c113c60 con 0x7ff81c105300 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff81be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c1049b0 0x7ff81c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff820c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c10de10 0x7ff81c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff81be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c1049b0 0x7ff81c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38988/0 (socket says 172.21.15.27:38988) 2024-09-21T07:27:00.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff81be00640 1 -- 172.21.15.27:0/655166379 learned_addr learned my addr 172.21.15.27:0/655166379 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff813400640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff81c105300 0x7ff81c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff820c00640 1 -- 172.21.15.27:0/655166379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff81c105300 msgr2=0x7ff81c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff820c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff81c105300 0x7ff81c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff820c00640 1 -- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c1049b0 msgr2=0x7ff81c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff820c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c1049b0 0x7ff81c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff820c00640 1 -- 172.21.15.27:0/655166379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c1a0440 con 0x7ff81c10de10 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.420+0000 7ff81be00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c1049b0 0x7ff81c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:00.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.421+0000 7ff820c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c10de10 0x7ff81c19fd10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff814009fd0 tx=0x7ff81400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.421+0000 7ff819600640 1 -- 172.21.15.27:0/655166379 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400ce20 con 0x7ff81c10de10 2024-09-21T07:27:00.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.421+0000 7ff819600640 1 -- 172.21.15.27:0/655166379 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814005020 con 0x7ff81c10de10 2024-09-21T07:27:00.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.421+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c1a0730 con 0x7ff81c10de10 2024-09-21T07:27:00.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.421+0000 7ff819600640 1 -- 172.21.15.27:0/655166379 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814010690 con 0x7ff81c10de10 2024-09-21T07:27:00.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.421+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1a6e90 con 0x7ff81c10de10 2024-09-21T07:27:00.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.425+0000 7ff829804740 1 Processor -- start 2024-09-21T07:27:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.426+0000 7ff819600640 1 -- 172.21.15.27:0/655166379 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff8140054d0 con 0x7ff81c10de10 2024-09-21T07:27:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.426+0000 7ff829804740 1 -- start start 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff829804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f6064dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff829804740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f6064e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff829804740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff829804740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562f605c4fb0 con 0x562f6027e1b0 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff829804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562f605b43e0 con 0x562f602a7070 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff829804740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f605b81f0 con 0x562f60286390 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f6064dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f6064dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37122/0 (socket says 172.21.15.27:37122) 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 -- 172.21.15.27:0/3288909133 learned_addr learned my addr 172.21.15.27:0/3288909133 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff81be00640 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f6064e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 -- 172.21.15.27:0/3288909133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 msgr2=0x562f6064e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 -- 172.21.15.27:0/3288909133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 msgr2=0x562f6064e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f6064e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff813400640 1 -- 172.21.15.27:0/3288909133 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6057ecd0 con 0x562f602a7070 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.427+0000 7ff81be00640 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f6064e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.428+0000 7ff813400640 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f6064dda0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff804013780 tx=0x7ff8040096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3daf4148834d7b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.428+0000 7ff811200640 1 -- 172.21.15.27:0/3288909133 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804013d00 con 0x562f602a7070 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.428+0000 7ff819600640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff8000b6680 0x7ff8000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.428+0000 7ff811200640 1 -- 172.21.15.27:0/3288909133 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff804013130 con 0x562f602a7070 2024-09-21T07:27:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.428+0000 7ff811200640 1 -- 172.21.15.27:0/3288909133 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804005620 con 0x562f602a7070 2024-09-21T07:27:00.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.428+0000 7ff819600640 1 -- 172.21.15.27:0/655166379 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7ff8140e81b0 con 0x7ff81c10de10 2024-09-21T07:27:00.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff81be00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff8000b6680 0x7ff8000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 -- 172.21.15.27:0/3288909133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 msgr2=0x562f6064dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f6064dda0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff804013780 tx=0x7ff8040096a0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 -- 172.21.15.27:0/3288909133 shutdown_connections 2024-09-21T07:27:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f6064e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 --2- 172.21.15.27:0/3288909133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f6064dda0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 -- 172.21.15.27:0/3288909133 >> 172.21.15.27:0/3288909133 conn(0x562f6029b460 msgr2=0x562f6064bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 -- 172.21.15.27:0/3288909133 shutdown_connections 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.429+0000 7ff829804740 1 -- 172.21.15.27:0/3288909133 wait complete. 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.430+0000 7ff81be00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff8000b6680 0x7ff8000b8b90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005ec0 tx=0x7ff80c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.430+0000 7ff829804740 1 -- start start 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.430+0000 7ff829804740 1 Processor -- start 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff829804740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f6027e1b0 0x562f6065d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff829804740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f60286390 0x562f6065dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff829804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f60669bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff829804740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562f60415700 con 0x562f60286390 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff829804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562f6054f580 con 0x562f602a7070 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff829804740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f60660850 con 0x562f6027e1b0 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff829804740 10 client.0 _ll_register_callbacks cb 0x562f6055f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff813400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f60286390 0x562f6065dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff81be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f6027e1b0 0x562f6065d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff813400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f60286390 0x562f6065dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39016/0 (socket says 172.21.15.27:39016) 2024-09-21T07:27:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff813400640 1 -- 172.21.15.27:0/3959608574 learned_addr learned my addr 172.21.15.27:0/3959608574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.431+0000 7ff820c00640 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f60669bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff813400640 1 -- 172.21.15.27:0/3959608574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f6027e1b0 msgr2=0x562f6065d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff813400640 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f6027e1b0 0x562f6065d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff813400640 1 -- 172.21.15.27:0/3959608574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 msgr2=0x562f60669bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff813400640 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f60669bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff813400640 1 -- 172.21.15.27:0/3959608574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f60312650 con 0x562f60286390 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff820c00640 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f60669bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff813400640 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f60286390 0x562f6065dd90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff804016040 tx=0x7ff8040045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a0908bac63ded4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff809600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff809600640 1 -- 172.21.15.27:0/3959608574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804024070 con 0x562f60286390 2024-09-21T07:27:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f60551b90 con 0x562f60286390 2024-09-21T07:27:00.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.432+0000 7ff809600640 1 -- 172.21.15.27:0/3959608574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7ff80400f9c0 con 0x562f60286390 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff809600640 1 -- 172.21.15.27:0/3959608574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff804030050 con 0x562f60286390 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 10 client.19620 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f6056c950 con 0x562f60286390 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff809600640 1 -- 172.21.15.27:0/3959608574 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562f60551b90 con 0x562f60286390 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff809600640 1 -- 172.21.15.27:0/3959608574 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff804029440 con 0x562f60286390 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 10 client.19620 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 10 client.19620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:27:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 20 client.19620 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:00.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f602659f0 0x562f6066f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 4 client.19620 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:00.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.433+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f605c6f00 con 0x562f602659f0 2024-09-21T07:27:00.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.434+0000 7ff81be00640 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f602659f0 0x562f6066f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.434+0000 7ff81be00640 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f602659f0 0x562f6066f740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39733cca136bae9e server_cookie=c4fcc579ca219c2b in_seq=0 out_seq=0 2024-09-21T07:27:00.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.434+0000 7ff809600640 10 client.19620 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:00.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff809600640 1 -- 172.21.15.27:0/3959608574 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f605c6f00 con 0x562f602659f0 2024-09-21T07:27:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff809600640 10 client.19620 handle_command_reply: tid=0 2024-09-21T07:27:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 client.19620 shutdown 2024-09-21T07:27:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f60286390 msgr2=0x562f6065dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f60286390 0x562f6065dd90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff804016040 tx=0x7ff8040045a0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 shutdown_connections 2024-09-21T07:27:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f602659f0 0x562f6066f740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f60669bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff809600640 0 client.19620 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:00.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f60286390 0x562f6065dd90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.435+0000 7ff829804740 1 --2- 172.21.15.27:0/3959608574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f6027e1b0 0x562f6065d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.436+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 >> 172.21.15.27:0/3959608574 conn(0x562f6029b460 msgr2=0x562f60650e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.436+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 shutdown_connections 2024-09-21T07:27:00.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.436+0000 7ff829804740 1 -- 172.21.15.27:0/3959608574 wait complete. 2024-09-21T07:27:00.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.436+0000 7ff829804740 20 client.19620 trim_cache size 0 max 16384 2024-09-21T07:27:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 Processor -- start 2024-09-21T07:27:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 -- start start 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f6027e1b0 0x562f6065f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f602a7070 0x562f6064f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1b1550 con 0x562f602a7070 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1aee20 con 0x562f6027e1b0 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.450+0000 7ff829804740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1acc90 con 0x562f60286390 2024-09-21T07:27:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f6027e1b0 0x562f6065f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff820c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f602a7070 0x562f6064f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f6027e1b0 0x562f6065f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37146/0 (socket says 172.21.15.27:37146) 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 -- 172.21.15.27:0/736756991 learned_addr learned my addr 172.21.15.27:0/736756991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff813400640 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 -- 172.21.15.27:0/736756991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 msgr2=0x562f6064a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 -- 172.21.15.27:0/736756991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f602a7070 msgr2=0x562f6064f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f602a7070 0x562f6064f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 -- 172.21.15.27:0/736756991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6065ff90 con 0x562f6027e1b0 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff820c00640 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f602a7070 0x562f6064f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.451+0000 7ff81be00640 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f6027e1b0 0x562f6065f730 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff80c004650 tx=0x7ff80c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8810ade55dd0ca4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.452+0000 7ff80be00640 1 -- 172.21.15.27:0/736756991 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c035040 con 0x562f6027e1b0 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.452+0000 7ff80be00640 1 -- 172.21.15.27:0/736756991 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80c0543f0 con 0x562f6027e1b0 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.452+0000 7ff80be00640 1 -- 172.21.15.27:0/736756991 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c03d6b0 con 0x562f6027e1b0 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 -- 172.21.15.27:0/736756991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f6027e1b0 msgr2=0x562f6065f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f6027e1b0 0x562f6065f730 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff80c004650 tx=0x7ff80c03f000 comp rx=0 tx=0).stop 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 -- 172.21.15.27:0/736756991 shutdown_connections 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f6064a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f602a7070 0x562f6064f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 --2- 172.21.15.27:0/736756991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f6027e1b0 0x562f6065f730 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 -- 172.21.15.27:0/736756991 >> 172.21.15.27:0/736756991 conn(0x562f6029b460 msgr2=0x562f6064edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 -- 172.21.15.27:0/736756991 shutdown_connections 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 -- 172.21.15.27:0/736756991 wait complete. 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 -- start start 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.453+0000 7ff829804740 1 Processor -- start 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff829804740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f60661c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff829804740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f60662160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff829804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f606626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff829804740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ad2e0 con 0x562f6027e1b0 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff829804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ae750 con 0x562f602a7070 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff829804740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ad9b0 con 0x562f60286390 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff829804740 10 client.0 _ll_register_callbacks cb 0x7ff804005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff81be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f60661c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff81be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f60661c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39036/0 (socket says 172.21.15.27:39036) 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff820c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f606626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff81be00640 1 -- 172.21.15.27:0/65211066 learned_addr learned my addr 172.21.15.27:0/65211066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.454+0000 7ff813400640 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f60662160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff820c00640 1 -- 172.21.15.27:0/65211066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 msgr2=0x562f60662160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff820c00640 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f60662160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff820c00640 1 -- 172.21.15.27:0/65211066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 msgr2=0x562f60661c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff820c00640 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f60661c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff820c00640 1 -- 172.21.15.27:0/65211066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f606d9070 con 0x562f602a7070 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff813400640 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f60662160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff820c00640 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f606626a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff8140e54f0 tx=0x7ff814005420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b27094c130b53f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff809600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff809600640 1 -- 172.21.15.27:0/65211066 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8140d5030 con 0x562f602a7070 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff809600640 1 -- 172.21.15.27:0/65211066 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7ff8140ace50 con 0x562f602a7070 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff80c0543f0 con 0x562f602a7070 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff809600640 1 -- 172.21.15.27:0/65211066 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8140b26d0 con 0x562f602a7070 2024-09-21T07:27:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff829804740 10 client.29657 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.455+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f606da350 con 0x562f602a7070 2024-09-21T07:27:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff809600640 1 -- 172.21.15.27:0/65211066 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80c0543f0 con 0x562f602a7070 2024-09-21T07:27:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff809600640 1 -- 172.21.15.27:0/65211066 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8140a9040 con 0x562f602a7070 2024-09-21T07:27:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff829804740 10 client.29657 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff829804740 10 client.29657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:27:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff829804740 20 client.29657 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff829804740 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f6023ada0 0x562f606dcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff829804740 4 client.29657 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f605c29b0 con 0x562f6023ada0 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.456+0000 7ff81be00640 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f6023ada0 0x562f606dcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff81be00640 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f6023ada0 0x562f606dcb60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=605dca05597e9f89 server_cookie=413a25bc1c1f65d9 in_seq=0 out_seq=0 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff809600640 10 client.29657 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff809600640 1 -- 172.21.15.27:0/65211066 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f605c29b0 con 0x562f6023ada0 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff809600640 10 client.29657 handle_command_reply: tid=0 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff829804740 1 client.29657 shutdown 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 msgr2=0x562f606626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff829804740 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f606626a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff8140e54f0 tx=0x7ff814005420 comp rx=0 tx=0).stop 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 shutdown_connections 2024-09-21T07:27:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff829804740 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x562f6023ada0 0x562f606dcb60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.457+0000 7ff829804740 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f60286390 0x562f60662160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.458+0000 7ff829804740 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f602a7070 0x562f606626a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.458+0000 7ff809600640 0 client.29657 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.458+0000 7ff829804740 1 --2- 172.21.15.27:0/65211066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562f6027e1b0 0x562f60661c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.458+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 >> 172.21.15.27:0/65211066 conn(0x562f6029b460 msgr2=0x562f6064f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.458+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 shutdown_connections 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.458+0000 7ff829804740 1 -- 172.21.15.27:0/65211066 wait complete. 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.458+0000 7ff829804740 20 client.29657 trim_cache size 0 max 16384 2024-09-21T07:27:00.467 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:27:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff8000b6680 msgr2=0x7ff8000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff8000b6680 0x7ff8000b8b90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005ec0 tx=0x7ff80c005a20 comp rx=0 tx=0).stop 2024-09-21T07:27:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c10de10 msgr2=0x7ff81c19fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c10de10 0x7ff81c19fd10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff814009fd0 tx=0x7ff81400ef10 comp rx=0 tx=0).stop 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 shutdown_connections 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff81c105300 0x7ff81c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff8000b6680 0x7ff8000b8b90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff81c10de10 0x7ff81c19fd10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 --2- 172.21.15.27:0/655166379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c1049b0 0x7ff81c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 >> 172.21.15.27:0/655166379 conn(0x7ff81c079760 msgr2=0x7ff81c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 shutdown_connections 2024-09-21T07:27:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:00.459+0000 7ff822c00640 1 -- 172.21.15.27:0/655166379 wait complete. 2024-09-21T07:27:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:00 smithi119 ceph-mon[34519]: pgmap v813: 129 pgs: 129 active+clean; 5.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-09-21T07:27:01.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:00 smithi027 ceph-mon[30779]: pgmap v813: 129 pgs: 129 active+clean; 5.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-09-21T07:27:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:00 smithi153 ceph-mon[34747]: pgmap v813: 129 pgs: 129 active+clean; 5.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-09-21T07:27:01.288 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:27:01.680 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 -- 172.21.15.27:0/274364856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1030109c40 msgr2=0x7f1030110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 --2- 172.21.15.27:0/274364856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1030109c40 0x7f1030110470 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1028009ca0 tx=0x7f102802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 -- 172.21.15.27:0/274364856 shutdown_connections 2024-09-21T07:27:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 --2- 172.21.15.27:0/274364856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10301092f0 0x7f1030109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 --2- 172.21.15.27:0/274364856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1030109c40 0x7f1030110470 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 --2- 172.21.15.27:0/274364856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10301039b0 0x7f1030103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 -- 172.21.15.27:0/274364856 >> 172.21.15.27:0/274364856 conn(0x7f10300ff400 msgr2=0x7f1030101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 -- 172.21.15.27:0/274364856 shutdown_connections 2024-09-21T07:27:02.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.351+0000 7f1035000640 1 -- 172.21.15.27:0/274364856 wait complete. 2024-09-21T07:27:02.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 Processor -- start 2024-09-21T07:27:02.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 -- start start 2024-09-21T07:27:02.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10301039b0 0x7f1030075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10301092f0 0x7f1030075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1030109c40 0x7f1030076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1030113950 con 0x7f10301039b0 2024-09-21T07:27:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10301137d0 con 0x7f10301092f0 2024-09-21T07:27:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.353+0000 7f1035000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1030113ad0 con 0x7f1030109c40 2024-09-21T07:27:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10301039b0 0x7f1030075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10301039b0 0x7f1030075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39060/0 (socket says 172.21.15.27:39060) 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 -- 172.21.15.27:0/3947932619 learned_addr learned my addr 172.21.15.27:0/3947932619 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102ea00640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10301092f0 0x7f1030075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102fe00640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1030109c40 0x7f1030076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 -- 172.21.15.27:0/3947932619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1030109c40 msgr2=0x7f1030076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1030109c40 0x7f1030076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 -- 172.21.15.27:0/3947932619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10301092f0 msgr2=0x7f1030075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10301092f0 0x7f1030075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102f400640 1 -- 172.21.15.27:0/3947932619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10301a8520 con 0x7f10301039b0 2024-09-21T07:27:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.354+0000 7f102fe00640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1030109c40 0x7f1030076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.355+0000 7f102f400640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10301039b0 0x7f1030075710 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9f0 tx=0x7f102000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.355+0000 7f1027e00640 1 -- 172.21.15.27:0/3947932619 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102000cd80 con 0x7f10301039b0 2024-09-21T07:27:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.355+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10301a8810 con 0x7f10301039b0 2024-09-21T07:27:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.355+0000 7f1027e00640 1 -- 172.21.15.27:0/3947932619 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10200108e0 con 0x7f10301039b0 2024-09-21T07:27:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.355+0000 7f1027e00640 1 -- 172.21.15.27:0/3947932619 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020018980 con 0x7f10301039b0 2024-09-21T07:27:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.355+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10301a8da0 con 0x7f10301039b0 2024-09-21T07:27:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.358+0000 7f103d26f740 1 Processor -- start 2024-09-21T07:27:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f103d26f740 1 -- start start 2024-09-21T07:27:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f1027e00640 1 -- 172.21.15.27:0/3947932619 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1020010430 con 0x7f10301039b0 2024-09-21T07:27:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f103d26f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b594070 0x555b8b93af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f103d26f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b56b1b0 0x555b8b93b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f103d26f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b573390 0x555b8b93b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f103d26f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555b8b8b1fb0 con 0x555b8b573390 2024-09-21T07:27:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f103d26f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555b8b8a13e0 con 0x555b8b594070 2024-09-21T07:27:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f103d26f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b8b8a51f0 con 0x555b8b56b1b0 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f102ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b573390 0x555b8b93b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f102f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b56b1b0 0x555b8b93b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f102ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b573390 0x555b8b93b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39070/0 (socket says 172.21.15.27:39070) 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.359+0000 7f102ea00640 1 -- 172.21.15.27:0/1730615337 learned_addr learned my addr 172.21.15.27:0/1730615337 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.360+0000 7f102fe00640 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b594070 0x555b8b93af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.360+0000 7f102f400640 1 -- 172.21.15.27:0/1730615337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b594070 msgr2=0x555b8b93af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.360+0000 7f102f400640 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b594070 0x555b8b93af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.360+0000 7f102f400640 1 -- 172.21.15.27:0/1730615337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b573390 msgr2=0x555b8b93b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.360+0000 7f102f400640 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b573390 0x555b8b93b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.360+0000 7f102f400640 1 -- 172.21.15.27:0/1730615337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8b86bcd0 con 0x555b8b56b1b0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.360+0000 7f102f400640 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b56b1b0 0x555b8b93b4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f10200e51a0 tx=0x7f10200f04b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1695777906bad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.361+0000 7f1024c00640 1 -- 172.21.15.27:0/1730615337 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10200fb070 con 0x555b8b56b1b0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.361+0000 7f1027e00640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10140b6680 0x7f10140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.361+0000 7f1024c00640 1 -- 172.21.15.27:0/1730615337 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10200ad460 con 0x555b8b56b1b0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.361+0000 7f1024c00640 1 -- 172.21.15.27:0/1730615337 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10200ad0c0 con 0x555b8b56b1b0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.361+0000 7f1027e00640 1 -- 172.21.15.27:0/3947932619 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f10200e7e60 con 0x7f10301039b0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.361+0000 7f102fe00640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10140b6680 0x7f10140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 -- 172.21.15.27:0/1730615337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b56b1b0 msgr2=0x555b8b93b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b56b1b0 0x555b8b93b4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f10200e51a0 tx=0x7f10200f04b0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f102fe00640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10140b6680 0x7f10140b8b90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1028006010 tx=0x7f1028005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 -- 172.21.15.27:0/1730615337 shutdown_connections 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b56b1b0 0x555b8b93b4b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b573390 0x555b8b93b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 --2- 172.21.15.27:0/1730615337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b594070 0x555b8b93af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 -- 172.21.15.27:0/1730615337 >> 172.21.15.27:0/1730615337 conn(0x555b8b588460 msgr2=0x555b8b938e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 -- 172.21.15.27:0/1730615337 shutdown_connections 2024-09-21T07:27:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.362+0000 7f103d26f740 1 -- 172.21.15.27:0/1730615337 wait complete. 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 -- start start 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 Processor -- start 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b946bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b952aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555b8b702700 con 0x555b8b594070 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555b8b83c580 con 0x555b8b56b1b0 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.363+0000 7f103d26f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b8b949750 con 0x555b8b573390 2024-09-21T07:27:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f103d26f740 10 client.0 _ll_register_callbacks cb 0x555b8b84c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b946bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b952aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102ea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b946bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52990/0 (socket says 172.21.15.27:52990) 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102ea00640 1 -- 172.21.15.27:0/3288923605 learned_addr learned my addr 172.21.15.27:0/3288923605 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102f400640 1 -- 172.21.15.27:0/3288923605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 msgr2=0x555b8b946bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102f400640 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b946bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102f400640 1 -- 172.21.15.27:0/3288923605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 msgr2=0x555b8b952aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102f400640 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b952aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.364+0000 7f102f400640 1 -- 172.21.15.27:0/3288923605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b8b5ff650 con 0x555b8b56b1b0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f102f400640 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9466b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f102009ffa0 tx=0x7f10200e5650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346db2734a09349 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f1019600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f1019600640 1 -- 172.21.15.27:0/3288923605 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020016bf0 con 0x555b8b56b1b0 2024-09-21T07:27:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b8b83eb90 con 0x555b8b56b1b0 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f1019600640 1 -- 172.21.15.27:0/3288923605 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f10200209d0 con 0x555b8b56b1b0 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f1019600640 1 -- 172.21.15.27:0/3288923605 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1020016e00 con 0x555b8b56b1b0 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f103d26f740 10 client.29669 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.365+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b8b859950 con 0x555b8b56b1b0 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f1019600640 1 -- 172.21.15.27:0/3288923605 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555b8b83eb90 con 0x555b8b56b1b0 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f1019600640 1 -- 172.21.15.27:0/3288923605 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f10200aebd0 con 0x555b8b56b1b0 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f103d26f740 10 client.29669 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f103d26f740 10 client.29669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f103d26f740 20 client.29669 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f103d26f740 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b5529f0 0x555b8b9586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f103d26f740 4 client.29669 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b8b8b3f00 con 0x555b8b5529f0 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.366+0000 7f102ea00640 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b5529f0 0x555b8b9586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.367+0000 7f102ea00640 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b5529f0 0x555b8b9586d0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eafc3e4300c9c9f server_cookie=a7bfb84598b98dd9 in_seq=0 out_seq=0 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.367+0000 7f1019600640 10 client.29669 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f1019600640 1 -- 172.21.15.27:0/3288923605 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b8b8b3f00 con 0x555b8b5529f0 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f1019600640 10 client.29669 handle_command_reply: tid=0 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 client.29669 shutdown 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 msgr2=0x555b8b9466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9466b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f102009ffa0 tx=0x7f10200e5650 comp rx=0 tx=0).stop 2024-09-21T07:27:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 shutdown_connections 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b5529f0 0x555b8b9586d0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b946bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b952aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 --2- 172.21.15.27:0/3288923605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9466b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f1019600640 0 client.29669 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 >> 172.21.15.27:0/3288923605 conn(0x555b8b588460 msgr2=0x555b8b937600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 shutdown_connections 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.368+0000 7f103d26f740 1 -- 172.21.15.27:0/3288923605 wait complete. 2024-09-21T07:27:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.369+0000 7f103d26f740 20 client.29669 trim_cache size 0 max 16384 2024-09-21T07:27:02.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.382+0000 7f103d26f740 1 Processor -- start 2024-09-21T07:27:02.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f103d26f740 1 -- start start 2024-09-21T07:27:02.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f103d26f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b56b1b0 0x555b8b9482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f103d26f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b573390 0x555b8b952a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f103d26f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b594070 0x555b8b9438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f103d26f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10301b57f0 con 0x555b8b56b1b0 2024-09-21T07:27:02.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f103d26f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10301b30c0 con 0x555b8b573390 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f103d26f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10301b0f30 con 0x555b8b594070 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f102f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b56b1b0 0x555b8b9482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f102f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b56b1b0 0x555b8b9482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39088/0 (socket says 172.21.15.27:39088) 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f102f400640 1 -- 172.21.15.27:0/4169635739 learned_addr learned my addr 172.21.15.27:0/4169635739 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.383+0000 7f102ea00640 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b573390 0x555b8b952a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102fe00640 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b594070 0x555b8b9438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102f400640 1 -- 172.21.15.27:0/4169635739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b594070 msgr2=0x555b8b9438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102f400640 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b594070 0x555b8b9438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102f400640 1 -- 172.21.15.27:0/4169635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b573390 msgr2=0x555b8b952a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102f400640 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b573390 0x555b8b952a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102f400640 1 -- 172.21.15.27:0/4169635739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8b947970 con 0x555b8b56b1b0 2024-09-21T07:27:02.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102fe00640 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b594070 0x555b8b9438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:02.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f102f400640 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b56b1b0 0x555b8b9482e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f10200eb3b0 tx=0x7f1020000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9f86b79e0d772d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.384+0000 7f101be00640 1 -- 172.21.15.27:0/4169635739 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020026060 con 0x555b8b56b1b0 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f101be00640 1 -- 172.21.15.27:0/4169635739 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10200eb440 con 0x555b8b56b1b0 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f101be00640 1 -- 172.21.15.27:0/4169635739 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020010720 con 0x555b8b56b1b0 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f103d26f740 1 -- 172.21.15.27:0/4169635739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b56b1b0 msgr2=0x555b8b9482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f103d26f740 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b56b1b0 0x555b8b9482e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f10200eb3b0 tx=0x7f1020000bc0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f103d26f740 1 -- 172.21.15.27:0/4169635739 shutdown_connections 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f103d26f740 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b594070 0x555b8b9438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f103d26f740 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b573390 0x555b8b952a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f103d26f740 1 --2- 172.21.15.27:0/4169635739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b56b1b0 0x555b8b9482e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.385+0000 7f103d26f740 1 -- 172.21.15.27:0/4169635739 >> 172.21.15.27:0/4169635739 conn(0x555b8b588460 msgr2=0x555b8b947bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.386+0000 7f103d26f740 1 -- 172.21.15.27:0/4169635739 shutdown_connections 2024-09-21T07:27:02.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.386+0000 7f103d26f740 1 -- 172.21.15.27:0/4169635739 wait complete. 2024-09-21T07:27:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.386+0000 7f103d26f740 1 -- start start 2024-09-21T07:27:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.386+0000 7f103d26f740 1 Processor -- start 2024-09-21T07:27:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f103d26f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f103d26f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b958ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f103d26f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b959420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f103d26f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10301b1580 con 0x555b8b594070 2024-09-21T07:27:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f103d26f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10301b29f0 con 0x555b8b56b1b0 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f103d26f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10301b1c50 con 0x555b8b573390 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f103d26f740 10 client.0 _ll_register_callbacks cb 0x7f101c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f102f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f102f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37216/0 (socket says 172.21.15.27:37216) 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f102f400640 1 -- 172.21.15.27:0/4108856308 learned_addr learned my addr 172.21.15.27:0/4108856308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f102fe00640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b959420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f102ea00640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b958ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f102f400640 1 -- 172.21.15.27:0/4108856308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 msgr2=0x555b8b958ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.387+0000 7f102f400640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b958ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f102f400640 1 -- 172.21.15.27:0/4108856308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 msgr2=0x555b8b959420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f102f400640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b959420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f102f400640 1 -- 172.21.15.27:0/4108856308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b8b9ca6a0 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f102ea00640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b958ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f102f400640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9589a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f10200e57a0 tx=0x7f1020009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b9bd46db8f6aef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f1019600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f1019600640 1 -- 172.21.15.27:0/4108856308 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020010a90 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f1019600640 1 -- 172.21.15.27:0/4108856308 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9461+0+0 (secure 0 0 0) 0x7f10200b7030 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b8b8af9b0 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f1019600640 1 -- 172.21.15.27:0/4108856308 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f102000f3c0 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f103d26f740 10 client.29675 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.388+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b8b9ca6a0 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f1019600640 1 -- 172.21.15.27:0/4108856308 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555b8b8af9b0 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f1019600640 1 -- 172.21.15.27:0/4108856308 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1020018b20 con 0x555b8b56b1b0 2024-09-21T07:27:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f103d26f740 10 client.29675 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f103d26f740 10 client.29675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f103d26f740 20 client.29675 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f103d26f740 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b527da0 0x555b8b9cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f103d26f740 4 client.29675 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b8b8acde0 con 0x555b8b527da0 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.389+0000 7f102ea00640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b527da0 0x555b8b9cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f102ea00640 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b527da0 0x555b8b9cdcc0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb3dbb1a96b61aa server_cookie=4c9f9922518040ab in_seq=0 out_seq=0 2024-09-21T07:27:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f1019600640 10 client.29675 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f1019600640 1 -- 172.21.15.27:0/4108856308 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b8b8acde0 con 0x555b8b527da0 2024-09-21T07:27:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f1019600640 10 client.29675 handle_command_reply: tid=0 2024-09-21T07:27:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 client.29675 shutdown 2024-09-21T07:27:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 msgr2=0x555b8b9589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9589a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f10200e57a0 tx=0x7f1020009310 comp rx=0 tx=0).stop 2024-09-21T07:27:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 shutdown_connections 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555b8b527da0 0x555b8b9cdcc0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b8b573390 0x555b8b958ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555b8b594070 0x555b8b959420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 --2- 172.21.15.27:0/4108856308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b8b56b1b0 0x555b8b9589a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.390+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 >> 172.21.15.27:0/4108856308 conn(0x555b8b588460 msgr2=0x555b8b93bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f1019600640 0 client.29675 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 shutdown_connections 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f103d26f740 1 -- 172.21.15.27:0/4108856308 wait complete. 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f103d26f740 20 client.29675 trim_cache size 0 max 16384 2024-09-21T07:27:02.398 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:27:02.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10140b6680 msgr2=0x7f10140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f1035000640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10140b6680 0x7f10140b8b90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1028006010 tx=0x7f1028005d80 comp rx=0 tx=0).stop 2024-09-21T07:27:02.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10301039b0 msgr2=0x7f1030075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.391+0000 7f1035000640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10301039b0 0x7f1030075710 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9f0 tx=0x7f102000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 shutdown_connections 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1030109c40 0x7f1030076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f10140b6680 0x7f10140b8b90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10301092f0 0x7f1030075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 --2- 172.21.15.27:0/3947932619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10301039b0 0x7f1030075710 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 >> 172.21.15.27:0/3947932619 conn(0x7f10300ff400 msgr2=0x7f1030101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 shutdown_connections 2024-09-21T07:27:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:02.392+0000 7f1035000640 1 -- 172.21.15.27:0/3947932619 wait complete. 2024-09-21T07:27:03.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:02 smithi119 ceph-mon[34519]: pgmap v814: 129 pgs: 129 active+clean; 5.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-21T07:27:03.072 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:02 smithi027 ceph-mon[30779]: pgmap v814: 129 pgs: 129 active+clean; 5.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-21T07:27:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:02 smithi153 ceph-mon[34747]: pgmap v814: 129 pgs: 129 active+clean; 5.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-21T07:27:04.112 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:27:04.112 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:27:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:04.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:04 smithi027 ceph-mon[30779]: pgmap v815: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-21T07:27:04.971 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:04 smithi027 ceph-mon[30779]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T07:27:05.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:04 smithi119 ceph-mon[34519]: pgmap v815: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-21T07:27:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:04 smithi119 ceph-mon[34519]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.155+0000 7f7924200640 1 -- 172.21.15.27:0/1105212113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c105db0 msgr2=0x7f791c1061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.155+0000 7f7924200640 1 --2- 172.21.15.27:0/1105212113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c105db0 0x7f791c1061c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7910009900 tx=0x7f791002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.156+0000 7f7924200640 1 -- 172.21.15.27:0/1105212113 shutdown_connections 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.156+0000 7f7924200640 1 --2- 172.21.15.27:0/1105212113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c10c040 0x7f791c112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.156+0000 7f7924200640 1 --2- 172.21.15.27:0/1105212113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f791c10b6f0 0x7f791c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.156+0000 7f7924200640 1 --2- 172.21.15.27:0/1105212113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c105db0 0x7f791c1061c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.156+0000 7f7924200640 1 -- 172.21.15.27:0/1105212113 >> 172.21.15.27:0/1105212113 conn(0x7f791c1017e0 msgr2=0x7f791c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.156+0000 7f7924200640 1 -- 172.21.15.27:0/1105212113 shutdown_connections 2024-09-21T07:27:05.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.156+0000 7f7924200640 1 -- 172.21.15.27:0/1105212113 wait complete. 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 Processor -- start 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 -- start start 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c105db0 0x7f791c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c10b6f0 0x7f791c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f791c10c040 0x7f791c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f791c115d50 con 0x7f791c10b6f0 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f791c115bd0 con 0x7f791c10c040 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.157+0000 7f7924200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f791c115ed0 con 0x7f791c105db0 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c105db0 0x7f791c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c105db0 0x7f791c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53030/0 (socket says 172.21.15.27:53030) 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 -- 172.21.15.27:0/2488383708 learned_addr learned my addr 172.21.15.27:0/2488383708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7922200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f791c10c040 0x7f791c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7920e00640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c10b6f0 0x7f791c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 -- 172.21.15.27:0/2488383708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f791c10c040 msgr2=0x7f791c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f791c10c040 0x7f791c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 -- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c10b6f0 msgr2=0x7f791c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c10b6f0 0x7f791c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7921800640 1 -- 172.21.15.27:0/2488383708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c1a2690 con 0x7f791c105db0 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.158+0000 7f7922200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f791c10c040 0x7f791c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.159+0000 7f7921800640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c105db0 0x7f791c19c720 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7910005ab0 tx=0x7f7910003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.159+0000 7f791a000640 1 -- 172.21.15.27:0/2488383708 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791003f070 con 0x7f791c105db0 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.159+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c1a2920 con 0x7f791c105db0 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.159+0000 7f791a000640 1 -- 172.21.15.27:0/2488383708 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7910031b60 con 0x7f791c105db0 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.159+0000 7f791a000640 1 -- 172.21.15.27:0/2488383708 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910035030 con 0x7f791c105db0 2024-09-21T07:27:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.159+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1a90e0 con 0x7f791c105db0 2024-09-21T07:27:05.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.162+0000 7f792adce740 1 Processor -- start 2024-09-21T07:27:05.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.162+0000 7f792adce740 1 -- start start 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f792adce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db6ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f792adce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db6b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f792adce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7a2670 0x561d8db6b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f792adce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561d8dae0e90 con 0x561d8d7c3350 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f792adce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561d8dad02c0 con 0x561d8d7a2670 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f792adce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d8dad40d0 con 0x561d8d79a490 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f7922200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db6ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f7922200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db6ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39110/0 (socket says 172.21.15.27:39110) 2024-09-21T07:27:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f7922200640 1 -- 172.21.15.27:0/938012156 learned_addr learned my addr 172.21.15.27:0/938012156 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:05.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f7920e00640 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7a2670 0x561d8db6b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.163+0000 7f791a000640 1 -- 172.21.15.27:0/2488383708 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7910042070 con 0x7f791c105db0 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f7922200640 1 -- 172.21.15.27:0/938012156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 msgr2=0x561d8db6b250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f7922200640 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db6b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f7922200640 1 -- 172.21.15.27:0/938012156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7a2670 msgr2=0x561d8db6b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f7922200640 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7a2670 0x561d8db6b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f7922200640 1 -- 172.21.15.27:0/938012156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8d82e930 con 0x561d8d7c3350 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f7922200640 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db6ad10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f79140117e0 tx=0x7f7914011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2b57659042923d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f790ea00640 1 -- 172.21.15.27:0/938012156 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914011310 con 0x561d8d7c3350 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f790ea00640 1 -- 172.21.15.27:0/938012156 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7914003e20 con 0x561d8d7c3350 2024-09-21T07:27:05.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.164+0000 7f790ea00640 1 -- 172.21.15.27:0/938012156 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79140044f0 con 0x561d8d7c3350 2024-09-21T07:27:05.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f792adce740 1 -- 172.21.15.27:0/938012156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 msgr2=0x561d8db6ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f792adce740 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db6ad10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f79140117e0 tx=0x7f7914011ca0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f791a000640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79040b6680 0x7f79040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f792adce740 1 -- 172.21.15.27:0/938012156 shutdown_connections 2024-09-21T07:27:05.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f792adce740 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db6b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f792adce740 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7a2670 0x561d8db6b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f792adce740 1 --2- 172.21.15.27:0/938012156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db6ad10 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.165+0000 7f792adce740 1 -- 172.21.15.27:0/938012156 >> 172.21.15.27:0/938012156 conn(0x561d8d7b7740 msgr2=0x561d8db68c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.166+0000 7f791a000640 1 -- 172.21.15.27:0/2488383708 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f79101082b0 con 0x7f791c105db0 2024-09-21T07:27:05.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.166+0000 7f792adce740 1 -- 172.21.15.27:0/938012156 shutdown_connections 2024-09-21T07:27:05.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.166+0000 7f7922200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79040b6680 0x7f79040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.166+0000 7f792adce740 1 -- 172.21.15.27:0/938012156 wait complete. 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.167+0000 7f792adce740 1 -- start start 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.167+0000 7f7922200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79040b6680 0x7f79040b8b90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7914005fd0 tx=0x7f7914005bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.167+0000 7f792adce740 1 Processor -- start 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.167+0000 7f792adce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db76880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.167+0000 7f792adce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db76dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f792adce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f792adce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561d8d931890 con 0x561d8d7c3350 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f792adce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561d8da6b160 con 0x561d8d79a490 2024-09-21T07:27:05.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f792adce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d8db794f0 con 0x561d8d7a2670 2024-09-21T07:27:05.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f792adce740 10 client.0 _ll_register_callbacks cb 0x561d8da87a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:05.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7922200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7921800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db76880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7922200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db827e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39118/0 (socket says 172.21.15.27:39118) 2024-09-21T07:27:05.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7922200640 1 -- 172.21.15.27:0/2245364295 learned_addr learned my addr 172.21.15.27:0/2245364295 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:05.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7920e00640 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db76dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7921800640 1 -- 172.21.15.27:0/2245364295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 msgr2=0x561d8db76dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:05.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7921800640 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db76dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7921800640 1 -- 172.21.15.27:0/2245364295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 msgr2=0x561d8db827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7921800640 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.168+0000 7f7921800640 1 -- 172.21.15.27:0/2245364295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8db82d70 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f7921800640 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db76880 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f79101055a0 tx=0x7f79100cdbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4126d6b5b7bd6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f7903400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f7903400640 1 -- 172.21.15.27:0/2245364295 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910049030 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7914003e20 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f7903400640 1 -- 172.21.15.27:0/2245364295 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f7910105820 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f7903400640 1 -- 172.21.15.27:0/2245364295 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7910105a20 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f792adce740 10 client.29687 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d8db85680 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.169+0000 7f7903400640 1 -- 172.21.15.27:0/2245364295 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7914003e20 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f7903400640 1 -- 172.21.15.27:0/2245364295 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f79100f10b0 con 0x561d8d79a490 2024-09-21T07:27:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f792adce740 10 client.29687 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f792adce740 10 client.29687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f792adce740 20 client.29687 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f792adce740 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d781cd0 0x561d8db88a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f792adce740 4 client.29687 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d8dae2de0 con 0x561d8d781cd0 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.170+0000 7f7920e00640 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d781cd0 0x561d8db88a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.171+0000 7f7920e00640 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d781cd0 0x561d8db88a80 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f4927d49a88fb52 server_cookie=912a7c6d33f61ef4 in_seq=0 out_seq=0 2024-09-21T07:27:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.171+0000 7f7903400640 10 client.29687 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.171+0000 7f7903400640 1 -- 172.21.15.27:0/2245364295 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d8dae2de0 con 0x561d8d781cd0 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f7903400640 10 client.29687 handle_command_reply: tid=0 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 client.29687 shutdown 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 msgr2=0x561d8db76880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db76880 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f79101055a0 tx=0x7f79100cdbe0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 shutdown_connections 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d781cd0 0x561d8db88a80 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db76dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7c3350 0x561d8db827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 --2- 172.21.15.27:0/2245364295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db76880 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f7903400640 0 client.29687 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 >> 172.21.15.27:0/2245364295 conn(0x561d8d7b7740 msgr2=0x561d8db67280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 shutdown_connections 2024-09-21T07:27:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 1 -- 172.21.15.27:0/2245364295 wait complete. 2024-09-21T07:27:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.172+0000 7f792adce740 20 client.29687 trim_cache size 0 max 16384 2024-09-21T07:27:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:04 smithi153 ceph-mon[34747]: pgmap v815: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-21T07:27:05.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:04 smithi153 ceph-mon[34747]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T07:27:05.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 Processor -- start 2024-09-21T07:27:05.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 -- start start 2024-09-21T07:27:05.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7b7740 0x561d8db6ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db67280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f791c1b5a10 con 0x561d8d7b7740 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f791c1b32e0 con 0x561d8d79a490 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f792adce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f791c1b1150 con 0x561d8d7a2670 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f7921800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7b7740 0x561d8db6ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f7921800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7b7740 0x561d8db6ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39122/0 (socket says 172.21.15.27:39122) 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f7921800640 1 -- 172.21.15.27:0/2222788825 learned_addr learned my addr 172.21.15.27:0/2222788825 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f7920e00640 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.191+0000 7f7922200640 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db67280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f7921800640 1 -- 172.21.15.27:0/2222788825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 msgr2=0x561d8db67280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f7921800640 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db67280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f7921800640 1 -- 172.21.15.27:0/2222788825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 msgr2=0x561d8db704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f7921800640 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f7921800640 1 -- 172.21.15.27:0/2222788825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8db6db70 con 0x561d8d7b7740 2024-09-21T07:27:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f7921800640 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7b7740 0x561d8db6ff90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f79100b5340 tx=0x7f79100cdc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34301125aee300a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f790e000640 1 -- 172.21.15.27:0/2222788825 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100cd4c0 con 0x561d8d7b7740 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f790e000640 1 -- 172.21.15.27:0/2222788825 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79100d76c0 con 0x561d8d7b7740 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.192+0000 7f790e000640 1 -- 172.21.15.27:0/2222788825 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100cd2e0 con 0x561d8d7b7740 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 -- 172.21.15.27:0/2222788825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7b7740 msgr2=0x561d8db6ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7b7740 0x561d8db6ff90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f79100b5340 tx=0x7f79100cdc50 comp rx=0 tx=0).stop 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 -- 172.21.15.27:0/2222788825 shutdown_connections 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d7a2670 0x561d8db67280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d79a490 0x561d8db704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 --2- 172.21.15.27:0/2222788825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7b7740 0x561d8db6ff90 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 -- 172.21.15.27:0/2222788825 >> 172.21.15.27:0/2222788825 conn(0x561d8db710e0 msgr2=0x561d8db74f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 -- 172.21.15.27:0/2222788825 shutdown_connections 2024-09-21T07:27:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.193+0000 7f792adce740 1 -- 172.21.15.27:0/2222788825 wait complete. 2024-09-21T07:27:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.194+0000 7f792adce740 1 -- start start 2024-09-21T07:27:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.194+0000 7f792adce740 1 Processor -- start 2024-09-21T07:27:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f792adce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db7ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f792adce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7a2670 0x561d8db7c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f792adce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7b7740 0x561d8db7cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f792adce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f791c1b17a0 con 0x561d8d7a2670 2024-09-21T07:27:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f792adce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f791c1b2c10 con 0x561d8d7b7740 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f792adce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f791c1b1e70 con 0x561d8d79a490 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f792adce740 10 client.0 _ll_register_callbacks cb 0x7f7910005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f7920e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7a2670 0x561d8db7c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f7920e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7a2670 0x561d8db7c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39126/0 (socket says 172.21.15.27:39126) 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f7920e00640 1 -- 172.21.15.27:0/3714268619 learned_addr learned my addr 172.21.15.27:0/3714268619 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f7921800640 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db7ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.195+0000 7f7922200640 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7b7740 0x561d8db7cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7921800640 1 -- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7b7740 msgr2=0x561d8db7cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7921800640 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7b7740 0x561d8db7cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7921800640 1 -- 172.21.15.27:0/3714268619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7a2670 msgr2=0x561d8db7c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7921800640 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7a2670 0x561d8db7c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7921800640 1 -- 172.21.15.27:0/3714268619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8db7d600 con 0x561d8d79a490 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7921800640 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db7ef10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f79100ca320 tx=0x7f79100cdbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6215d5d286bc3334 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7903400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7903400640 1 -- 172.21.15.27:0/3714268619 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100cd4c0 con 0x561d8d79a490 2024-09-21T07:27:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7903400640 1 -- 172.21.15.27:0/3714268619 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f79100cd050 con 0x561d8d79a490 2024-09-21T07:27:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79100d76c0 con 0x561d8d79a490 2024-09-21T07:27:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f7903400640 1 -- 172.21.15.27:0/3714268619 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7910105a20 con 0x561d8d79a490 2024-09-21T07:27:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.196+0000 7f792adce740 10 client.29635 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d8db7d600 con 0x561d8d79a490 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f7903400640 1 -- 172.21.15.27:0/3714268619 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79100d76c0 con 0x561d8d79a490 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f7903400640 1 -- 172.21.15.27:0/3714268619 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f79100d5b00 con 0x561d8d79a490 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f792adce740 10 client.29635 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f792adce740 10 client.29635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f792adce740 20 client.29635 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f792adce740 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d7c3350 0x561d8dbf66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f792adce740 4 client.29635 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d8dade890 con 0x561d8d7c3350 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.197+0000 7f7920e00640 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d7c3350 0x561d8dbf66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.198+0000 7f7920e00640 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d7c3350 0x561d8dbf66b0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5237afde433fc5b server_cookie=eababd6f68ee93a4 in_seq=0 out_seq=0 2024-09-21T07:27:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.198+0000 7f7903400640 10 client.29635 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:05.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f7903400640 1 -- 172.21.15.27:0/3714268619 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d8dade890 con 0x561d8d7c3350 2024-09-21T07:27:05.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f7903400640 10 client.29635 handle_command_reply: tid=0 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 client.29635 shutdown 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 msgr2=0x561d8db7ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db7ef10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f79100ca320 tx=0x7f79100cdbe0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 shutdown_connections 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561d8d7c3350 0x561d8dbf66b0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d8d7b7740 0x561d8db7cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561d8d7a2670 0x561d8db7c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 --2- 172.21.15.27:0/3714268619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d8d79a490 0x561d8db7ef10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 >> 172.21.15.27:0/3714268619 conn(0x561d8db710e0 msgr2=0x561d8db74e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f7903400640 0 client.29635 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 shutdown_connections 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.200+0000 7f792adce740 1 -- 172.21.15.27:0/3714268619 wait complete. 2024-09-21T07:27:05.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f792adce740 20 client.29635 trim_cache size 0 max 16384 2024-09-21T07:27:05.204 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:05.204 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:27:05.204 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "0f52bbc6-199d-4b79-9044-1ebcfed058a5", 2024-09-21T07:27:05.204 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:27:05.204 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79040b6680 msgr2=0x7f79040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79040b6680 0x7f79040b8b90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7914005fd0 tx=0x7f7914005bd0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c105db0 msgr2=0x7f791c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c105db0 0x7f791c19c720 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7910005ab0 tx=0x7f7910003c10 comp rx=0 tx=0).stop 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 shutdown_connections 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79040b6680 0x7f79040b8b90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f791c10c040 0x7f791c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c10b6f0 0x7f791c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 --2- 172.21.15.27:0/2488383708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c105db0 0x7f791c19c720 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.201+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 >> 172.21.15.27:0/2488383708 conn(0x7f791c1017e0 msgr2=0x7f791c10ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.202+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 shutdown_connections 2024-09-21T07:27:05.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:05.202+0000 7f7924200640 1 -- 172.21.15.27:0/2488383708 wait complete. 2024-09-21T07:27:06.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:05 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:27:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:05 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:05 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:27:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:05 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:06.058 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:05 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:27:06.058 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:05 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:06.058 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:05 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:27:06.058 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:05 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:06.096 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:27:06.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:05 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:27:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:05 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:05 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:27:06.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:05 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:06.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:06 smithi027 ceph-mon[30779]: pgmap v817: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 101 KiB/s wr, 34 op/s 2024-09-21T07:27:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:06 smithi119 ceph-mon[34519]: pgmap v817: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 101 KiB/s wr, 34 op/s 2024-09-21T07:27:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 -- 172.21.15.27:0/2252720489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda44105090 msgr2=0x7fda4410d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 --2- 172.21.15.27:0/2252720489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda44105090 0x7fda4410d5d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fda30009900 tx=0x7fda3002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 -- 172.21.15.27:0/2252720489 shutdown_connections 2024-09-21T07:27:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 --2- 172.21.15.27:0/2252720489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda4410dba0 0x7fda44110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 --2- 172.21.15.27:0/2252720489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda44105090 0x7fda4410d5d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 --2- 172.21.15.27:0/2252720489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda44104740 0x7fda44104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 -- 172.21.15.27:0/2252720489 >> 172.21.15.27:0/2252720489 conn(0x7fda440fe230 msgr2=0x7fda44100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 -- 172.21.15.27:0/2252720489 shutdown_connections 2024-09-21T07:27:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.041+0000 7fda49400640 1 -- 172.21.15.27:0/2252720489 wait complete. 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.042+0000 7fda49400640 1 Processor -- start 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.042+0000 7fda49400640 1 -- start start 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda49400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda44104740 0x7fda4419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda49400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda44105090 0x7fda4419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda49400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda4410dba0 0x7fda441a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda49400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda441137c0 con 0x7fda44105090 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda49400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda44113640 con 0x7fda4410dba0 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda49400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda44113940 con 0x7fda44104740 2024-09-21T07:27:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda43400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda44105090 0x7fda4419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda43e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda44104740 0x7fda4419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda43400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda44105090 0x7fda4419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39154/0 (socket says 172.21.15.27:39154) 2024-09-21T07:27:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda43400640 1 -- 172.21.15.27:0/3949080576 learned_addr learned my addr 172.21.15.27:0/3949080576 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.043+0000 7fda48a00640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda4410dba0 0x7fda441a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda43e00640 1 -- 172.21.15.27:0/3949080576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda4410dba0 msgr2=0x7fda441a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda43e00640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda4410dba0 0x7fda441a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda43e00640 1 -- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda44105090 msgr2=0x7fda4419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda43e00640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda44105090 0x7fda4419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda43e00640 1 -- 172.21.15.27:0/3949080576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda441a23d0 con 0x7fda44104740 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda43e00640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda44104740 0x7fda4419c4a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e9f0 tx=0x7fda3400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda40c00640 1 -- 172.21.15.27:0/3949080576 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3400ccc0 con 0x7fda44104740 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda441a26c0 con 0x7fda44104740 2024-09-21T07:27:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda40c00640 1 -- 172.21.15.27:0/3949080576 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda340108e0 con 0x7fda44104740 2024-09-21T07:27:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.044+0000 7fda40c00640 1 -- 172.21.15.27:0/3949080576 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34018840 con 0x7fda44104740 2024-09-21T07:27:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.045+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441a8e50 con 0x7fda44104740 2024-09-21T07:27:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 Processor -- start 2024-09-21T07:27:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 -- start start 2024-09-21T07:27:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a7d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e8079f26b0 con 0x55e8076d81b0 2024-09-21T07:27:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e8079e1ae0 con 0x55e8076af2f0 2024-09-21T07:27:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda519bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8079e58f0 con 0x55e8076b74d0 2024-09-21T07:27:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.048+0000 7fda48a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a7ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39156/0 (socket says 172.21.15.27:39156) 2024-09-21T07:27:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 -- 172.21.15.27:0/3902988777 learned_addr learned my addr 172.21.15.27:0/3902988777 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda43400640 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a7d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 -- 172.21.15.27:0/3902988777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 msgr2=0x55e807a7d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a7d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 -- 172.21.15.27:0/3902988777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 msgr2=0x55e807a7d350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a7d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 -- 172.21.15.27:0/3902988777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8079ae780 con 0x55e8076d81b0 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda48a00640 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a7ce10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fda3c011440 tx=0x7fda3c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9ddebbf89d486f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.049+0000 7fda39200640 1 -- 172.21.15.27:0/3902988777 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c003c30 con 0x55e8076d81b0 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.050+0000 7fda39200640 1 -- 172.21.15.27:0/3902988777 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda3c003dd0 con 0x55e8076d81b0 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.050+0000 7fda39200640 1 -- 172.21.15.27:0/3902988777 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00fab0 con 0x55e8076d81b0 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.050+0000 7fda519bd740 1 -- 172.21.15.27:0/3902988777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 msgr2=0x55e807a7ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.050+0000 7fda519bd740 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a7ce10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fda3c011440 tx=0x7fda3c011900 comp rx=0 tx=0).stop 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.050+0000 7fda40c00640 1 -- 172.21.15.27:0/3949080576 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fda34010430 con 0x7fda44104740 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 -- 172.21.15.27:0/3902988777 shutdown_connections 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a7d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 --2- 172.21.15.27:0/3902988777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a7ce10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 -- 172.21.15.27:0/3902988777 >> 172.21.15.27:0/3902988777 conn(0x55e8076cc300 msgr2=0x55e807a7ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 -- 172.21.15.27:0/3902988777 shutdown_connections 2024-09-21T07:27:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 -- 172.21.15.27:0/3902988777 wait complete. 2024-09-21T07:27:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.051+0000 7fda519bd740 1 -- start start 2024-09-21T07:27:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.052+0000 7fda519bd740 1 Processor -- start 2024-09-21T07:27:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda519bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076af2f0 0x55e807a884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda519bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda519bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a94880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda519bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e807919ab0 con 0x55e8076d81b0 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda519bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e807988570 con 0x55e8076b74d0 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda519bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e807a8b590 con 0x55e8076af2f0 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda519bd740 10 client.0 _ll_register_callbacks cb 0x55e807997100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda43400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a94880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda43e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda43400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a94880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39172/0 (socket says 172.21.15.27:39172) 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda43400640 1 -- 172.21.15.27:0/2746628452 learned_addr learned my addr 172.21.15.27:0/2746628452 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda40c00640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda280b6590 0x7fda280b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.053+0000 7fda48a00640 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076af2f0 0x55e807a884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda43400640 1 -- 172.21.15.27:0/2746628452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076af2f0 msgr2=0x55e807a884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda48a00640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda280b6590 0x7fda280b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda43400640 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076af2f0 0x55e807a884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda43400640 1 -- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 msgr2=0x55e807a889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda43400640 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda43400640 1 -- 172.21.15.27:0/2746628452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e807743790 con 0x55e8076d81b0 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda43e00640 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda40c00640 1 -- 172.21.15.27:0/3949080576 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7fda340e7db0 con 0x7fda44104740 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda43400640 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a94880 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fda30009230 tx=0x7fda30031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ec1b2fde2bd02d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda2d600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.054+0000 7fda48a00640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda280b6590 0x7fda280b8aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009df0 tx=0x7fda3c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda2d600640 1 -- 172.21.15.27:0/2746628452 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30035040 con 0x55e8076d81b0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3c003dd0 con 0x55e8076d81b0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda2d600640 1 -- 172.21.15.27:0/2746628452 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7fda30031760 con 0x55e8076d81b0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda2d600640 1 -- 172.21.15.27:0/2746628452 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda30004040 con 0x55e8076d81b0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda519bd740 10 client.19674 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e8077bc620 con 0x55e8076d81b0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda2d600640 1 -- 172.21.15.27:0/2746628452 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda3c003dd0 con 0x55e8076d81b0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.055+0000 7fda2d600640 1 -- 172.21.15.27:0/2746628452 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda30046480 con 0x55e8076d81b0 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.056+0000 7fda519bd740 10 client.19674 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.056+0000 7fda519bd740 10 client.19674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.056+0000 7fda519bd740 20 client.19674 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.056+0000 7fda519bd740 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e807696d80 0x55e807a9a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.056+0000 7fda519bd740 4 client.19674 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.056+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8079f4600 con 0x55e807696d80 2024-09-21T07:27:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.056+0000 7fda43e00640 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e807696d80 0x55e807a9a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.057+0000 7fda43e00640 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e807696d80 0x55e807a9a5a0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fcddd8549de1dd6 server_cookie=6f9516368b7809f4 in_seq=0 out_seq=0 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.057+0000 7fda2d600640 10 client.19674 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.057+0000 7fda2d600640 1 -- 172.21.15.27:0/2746628452 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8079f4600 con 0x55e807696d80 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.057+0000 7fda2d600640 10 client.19674 handle_command_reply: tid=0 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.057+0000 7fda519bd740 1 client.19674 shutdown 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 msgr2=0x55e807a94880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a94880 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fda30009230 tx=0x7fda30031e60 comp rx=0 tx=0).stop 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 shutdown_connections 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e807696d80 0x55e807a9a5a0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a94880 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 --2- 172.21.15.27:0/2746628452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076af2f0 0x55e807a884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda2d600640 0 client.19674 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 >> 172.21.15.27:0/2746628452 conn(0x55e8076cc300 msgr2=0x55e807a7de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 shutdown_connections 2024-09-21T07:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 1 -- 172.21.15.27:0/2746628452 wait complete. 2024-09-21T07:27:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.058+0000 7fda519bd740 20 client.19674 trim_cache size 0 max 16384 2024-09-21T07:27:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 Processor -- start 2024-09-21T07:27:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 -- start start 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076af2f0 0x55e807a79d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a8a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076d81b0 0x55e807a85680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b5920 con 0x55e8076af2f0 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b31f0 con 0x55e8076b74d0 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda519bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b1060 con 0x55e8076d81b0 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076af2f0 0x55e807a79d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076af2f0 0x55e807a79d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39180/0 (socket says 172.21.15.27:39180) 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.074+0000 7fda43e00640 1 -- 172.21.15.27:0/765022781 learned_addr learned my addr 172.21.15.27:0/765022781 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43400640 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a8a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda48a00640 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076d81b0 0x55e807a85680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43400640 1 -- 172.21.15.27:0/765022781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076d81b0 msgr2=0x55e807a85680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43400640 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076d81b0 0x55e807a85680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43400640 1 -- 172.21.15.27:0/765022781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076af2f0 msgr2=0x55e807a79d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43400640 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076af2f0 0x55e807a79d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43400640 1 -- 172.21.15.27:0/765022781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e807a75000 con 0x55e8076b74d0 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43e00640 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076af2f0 0x55e807a79d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.075+0000 7fda43400640 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a8a0e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fda3002a910 tx=0x7fda30005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966268e45421a55a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.076+0000 7fda2fe00640 1 -- 172.21.15.27:0/765022781 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30057020 con 0x55e8076b74d0 2024-09-21T07:27:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.076+0000 7fda2fe00640 1 -- 172.21.15.27:0/765022781 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda30031a10 con 0x55e8076b74d0 2024-09-21T07:27:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.076+0000 7fda2fe00640 1 -- 172.21.15.27:0/765022781 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30035070 con 0x55e8076b74d0 2024-09-21T07:27:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 -- 172.21.15.27:0/765022781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 msgr2=0x55e807a8a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a8a0e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fda3002a910 tx=0x7fda30005e80 comp rx=0 tx=0).stop 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 -- 172.21.15.27:0/765022781 shutdown_connections 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076d81b0 0x55e807a85680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076b74d0 0x55e807a8a0e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 --2- 172.21.15.27:0/765022781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076af2f0 0x55e807a79d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 -- 172.21.15.27:0/765022781 >> 172.21.15.27:0/765022781 conn(0x55e8076cc300 msgr2=0x55e807a899d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 -- 172.21.15.27:0/765022781 shutdown_connections 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 -- 172.21.15.27:0/765022781 wait complete. 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 -- start start 2024-09-21T07:27:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.077+0000 7fda519bd740 1 Processor -- start 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda519bd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a90f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda519bd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a8e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda519bd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a8e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda519bd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b16b0 con 0x55e8076d81b0 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda519bd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b2b20 con 0x55e8076af2f0 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda519bd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b1d80 con 0x55e8076b74d0 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda519bd740 10 client.0 _ll_register_callbacks cb 0x7fda30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda43e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a90f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda43e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a90f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37318/0 (socket says 172.21.15.27:37318) 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda48a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a8e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda43e00640 1 -- 172.21.15.27:0/790257070 learned_addr learned my addr 172.21.15.27:0/790257070 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.078+0000 7fda43400640 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a8e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda43e00640 1 -- 172.21.15.27:0/790257070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 msgr2=0x55e807a8e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda43e00640 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a8e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda43e00640 1 -- 172.21.15.27:0/790257070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 msgr2=0x55e807a8e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda43e00640 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a8e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda43e00640 1 -- 172.21.15.27:0/790257070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e807a8f060 con 0x55e8076af2f0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda43e00640 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a90f50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e9c0 tx=0x7fda340ad030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f8fb4e8a91eeef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda2d600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda2d600640 1 -- 172.21.15.27:0/790257070 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340adf00 con 0x55e8076af2f0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda2d600640 1 -- 172.21.15.27:0/790257070 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7fda340aeb30 con 0x55e8076af2f0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda30031a10 con 0x55e8076af2f0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.079+0000 7fda2d600640 1 -- 172.21.15.27:0/790257070 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda340b22e0 con 0x55e8076af2f0 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 10 client.29708 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e807b04dd0 con 0x55e8076af2f0 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda2d600640 1 -- 172.21.15.27:0/790257070 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda30031a10 con 0x55e8076af2f0 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda2d600640 1 -- 172.21.15.27:0/790257070 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda340ac4d0 con 0x55e8076af2f0 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 10 client.29708 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 10 client.29708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 20 client.29708 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e80766c1c0 0x55e807b07570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 4 client.29708 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8079f00b0 con 0x55e80766c1c0 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.080+0000 7fda43400640 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e80766c1c0 0x55e807b07570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.081+0000 7fda43400640 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e80766c1c0 0x55e807b07570 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6d5e71c2f30a69 server_cookie=175d10cee0812274 in_seq=0 out_seq=0 2024-09-21T07:27:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.081+0000 7fda2d600640 10 client.29708 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda2d600640 1 -- 172.21.15.27:0/790257070 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8079f00b0 con 0x55e80766c1c0 2024-09-21T07:27:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda2d600640 10 client.29708 handle_command_reply: tid=0 2024-09-21T07:27:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 client.29708 shutdown 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 msgr2=0x55e807a90f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a90f50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e9c0 tx=0x7fda340ad030 comp rx=0 tx=0).stop 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 shutdown_connections 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e80766c1c0 0x55e807b07570 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8076b74d0 0x55e807a8e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e8076d81b0 0x55e807a8e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 --2- 172.21.15.27:0/790257070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8076af2f0 0x55e807a90f50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 >> 172.21.15.27:0/790257070 conn(0x55e8076cc300 msgr2=0x55e807a7e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda2d600640 0 client.29708 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 shutdown_connections 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.094+0000 7fda519bd740 1 -- 172.21.15.27:0/790257070 wait complete. 2024-09-21T07:27:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda519bd740 20 client.29708 trim_cache size 0 max 16384 2024-09-21T07:27:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:27:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:27:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:27:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:27:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda280b6590 msgr2=0x7fda280b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda280b6590 0x7fda280b8aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009df0 tx=0x7fda3c009500 comp rx=0 tx=0).stop 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda44104740 msgr2=0x7fda4419c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda44104740 0x7fda4419c4a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e9f0 tx=0x7fda3400edf0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 shutdown_connections 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fda280b6590 0x7fda280b8aa0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda4410dba0 0x7fda441a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda44105090 0x7fda4419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 --2- 172.21.15.27:0/3949080576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda44104740 0x7fda4419c4a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.095+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 >> 172.21.15.27:0/3949080576 conn(0x7fda440fe230 msgr2=0x7fda441023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.096+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 shutdown_connections 2024-09-21T07:27:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:07.096+0000 7fda49400640 1 -- 172.21.15.27:0/3949080576 wait complete. 2024-09-21T07:27:07.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2468304670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0103990 msgr2=0x7f2ce0103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2468304670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0103990 0x7f2ce0103da0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009900 tx=0x7f2cd002a410 comp rx=0 tx=0).stop 2024-09-21T07:27:07.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2468304670 shutdown_connections 2024-09-21T07:27:07.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2468304670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ce01092d0 0x7f2ce01096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2468304670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0109c20 0x7f2ce0110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2468304670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0103990 0x7f2ce0103da0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2468304670 >> 172.21.15.27:0/2468304670 conn(0x7f2ce00ff3e0 msgr2=0x7f2ce0101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2468304670 shutdown_connections 2024-09-21T07:27:07.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.172+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2468304670 wait complete. 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.173+0000 7f2ce5a00640 1 Processor -- start 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.173+0000 7f2ce5a00640 1 -- start start 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.173+0000 7f2ce5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0103990 0x7f2ce019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.173+0000 7f2ce5a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ce01092d0 0x7f2ce019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.173+0000 7f2ce5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0109c20 0x7f2ce01a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2ce5a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01138e0 con 0x7f2ce0109c20 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2ce5a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0113760 con 0x7f2ce0103990 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2ce5a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0113a60 con 0x7f2ce01092d0 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0103990 0x7f2ce019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0103990 0x7f2ce019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:37342/0 (socket says 172.21.15.27:37342) 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 -- 172.21.15.27:0/2923090342 learned_addr learned my addr 172.21.15.27:0/2923090342 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2ce5000640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0109c20 0x7f2ce01a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdf400640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ce01092d0 0x7f2ce019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:06 smithi153 ceph-mon[34747]: pgmap v817: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 101 KiB/s wr, 34 op/s 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 -- 172.21.15.27:0/2923090342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ce01092d0 msgr2=0x7f2ce019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ce01092d0 0x7f2ce019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 -- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0109c20 msgr2=0x7f2ce01a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0109c20 0x7f2ce01a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.174+0000 7f2cdfe00640 1 -- 172.21.15.27:0/2923090342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce01a2480 con 0x7f2ce0103990 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.175+0000 7f2cdf400640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ce01092d0 0x7f2ce019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.175+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0103990 0x7f2ce019c510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005940 tx=0x7f2cd00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.175+0000 7f2cdcc00640 1 -- 172.21.15.27:0/2923090342 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd003f030 con 0x7f2ce0103990 2024-09-21T07:27:07.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.175+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce01a2710 con 0x7f2ce0103990 2024-09-21T07:27:07.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.175+0000 7f2cdcc00640 1 -- 172.21.15.27:0/2923090342 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd00044c0 con 0x7f2ce0103990 2024-09-21T07:27:07.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.175+0000 7f2cdcc00640 1 -- 172.21.15.27:0/2923090342 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0035070 con 0x7f2ce0103990 2024-09-21T07:27:07.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.175+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce01a8ed0 con 0x7f2ce0103990 2024-09-21T07:27:07.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 Processor -- start 2024-09-21T07:27:07.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 -- start start 2024-09-21T07:27:07.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6873f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 0x55daa6874470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa68749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55daa67eaf70 con 0x55daa64cd070 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55daa67da3a0 con 0x55daa64a41b0 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.180+0000 7f2cedc57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55daa67de1b0 con 0x55daa64ac390 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6873f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6873f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39232/0 (socket says 172.21.15.27:39232) 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 -- 172.21.15.27:0/3332097827 learned_addr learned my addr 172.21.15.27:0/3332097827 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2cdf400640 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa68749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2cdcc00640 1 -- 172.21.15.27:0/2923090342 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2cd0042030 con 0x7f2ce0103990 2024-09-21T07:27:07.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 -- 172.21.15.27:0/3332097827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 msgr2=0x55daa68749b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa68749b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 -- 172.21.15.27:0/3332097827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 msgr2=0x55daa6874470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:27:07.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 0x55daa6874470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.181+0000 7f2ce5000640 1 -- 172.21.15.27:0/3332097827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa67a4c90 con 0x55daa64cd070 2024-09-21T07:27:07.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.182+0000 7f2ce5000640 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6873f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009fd0 tx=0x7f2cd80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624d1019d98560f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.182+0000 7f2cd5600640 1 -- 172.21.15.27:0/3332097827 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8004d20 con 0x55daa64cd070 2024-09-21T07:27:07.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.182+0000 7f2cd5600640 1 -- 172.21.15.27:0/3332097827 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd8004050 con 0x55daa64cd070 2024-09-21T07:27:07.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.182+0000 7f2cd5600640 1 -- 172.21.15.27:0/3332097827 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8005260 con 0x55daa64cd070 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 -- 172.21.15.27:0/3332097827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 msgr2=0x55daa6873f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6873f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009fd0 tx=0x7f2cd80118d0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 -- 172.21.15.27:0/3332097827 shutdown_connections 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa68749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 0x55daa6874470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3332097827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6873f30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 -- 172.21.15.27:0/3332097827 >> 172.21.15.27:0/3332097827 conn(0x55daa64c1460 msgr2=0x55daa6871e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cdcc00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2cc40b6680 0x7f2cc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.183+0000 7f2cedc57740 1 -- 172.21.15.27:0/3332097827 shutdown_connections 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.184+0000 7f2cedc57740 1 -- 172.21.15.27:0/3332097827 wait complete. 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.184+0000 7f2cdcc00640 1 -- 172.21.15.27:0/2923090342 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f2cd0107ec0 con 0x7f2ce0103990 2024-09-21T07:27:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.184+0000 7f2cdf400640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2cc40b6680 0x7f2cc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.184+0000 7f2cedc57740 1 -- start start 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.184+0000 7f2cdf400640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2cc40b6680 0x7f2cc40b8b90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0105a20 tx=0x7f2ccc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 1 Processor -- start 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64a41b0 0x55daa687f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa687fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa688ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55daa663b700 con 0x55daa64a41b0 2024-09-21T07:27:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55daa6775540 con 0x55daa64cd070 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55daa6882710 con 0x55daa64ac390 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cedc57740 10 client.0 _ll_register_callbacks cb 0x55daa6785430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.185+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64a41b0 0x55daa687f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64a41b0 0x55daa687f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39244/0 (socket says 172.21.15.27:39244) 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3483375586 learned_addr learned my addr 172.21.15.27:0/3483375586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdf400640 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa687fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2ce5000640 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa688ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3483375586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 msgr2=0x55daa687fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa687fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 msgr2=0x55daa688ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa688ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3483375586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa6538650 con 0x55daa64a41b0 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64a41b0 0x55daa687f750 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0105210 tx=0x7f2cd00ca0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1b119e6fdb0706 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.186+0000 7f2cca000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cca000640 1 -- 172.21.15.27:0/3483375586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0049040 con 0x55daa64a41b0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cca000640 1 -- 172.21.15.27:0/3483375586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f2cd00cebe0 con 0x55daa64a41b0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55daa6777b50 con 0x55daa64a41b0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cca000640 1 -- 172.21.15.27:0/3483375586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2cd00cd2b0 con 0x55daa64a41b0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cedc57740 10 client.19692 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55daa6792910 con 0x55daa64a41b0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cca000640 1 -- 172.21.15.27:0/3483375586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55daa6777b50 con 0x55daa64a41b0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cca000640 1 -- 172.21.15.27:0/3483375586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2cd00c94b0 con 0x55daa64a41b0 2024-09-21T07:27:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cedc57740 10 client.19692 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cedc57740 10 client.19692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cedc57740 20 client.19692 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.187+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa648b9f0 0x55daa6891880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.188+0000 7f2cedc57740 4 client.19692 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.188+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daa67ecec0 con 0x55daa648b9f0 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.188+0000 7f2cdf400640 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa648b9f0 0x55daa6891880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.189+0000 7f2cdf400640 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa648b9f0 0x55daa6891880 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c182f143f38375 server_cookie=bb2a7d14584c729a in_seq=0 out_seq=0 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.189+0000 7f2cca000640 10 client.19692 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.189+0000 7f2cca000640 1 -- 172.21.15.27:0/3483375586 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55daa67ecec0 con 0x55daa648b9f0 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.189+0000 7f2cca000640 10 client.19692 handle_command_reply: tid=0 2024-09-21T07:27:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.189+0000 7f2cedc57740 1 client.19692 shutdown 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64a41b0 msgr2=0x55daa687f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64a41b0 0x55daa687f750 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0105210 tx=0x7f2cd00ca0e0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 shutdown_connections 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa648b9f0 0x55daa6891880 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa687fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa688ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3483375586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64a41b0 0x55daa687f750 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 >> 172.21.15.27:0/3483375586 conn(0x55daa64c1460 msgr2=0x55daa68705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cca000640 0 client.19692 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 shutdown_connections 2024-09-21T07:27:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 1 -- 172.21.15.27:0/3483375586 wait complete. 2024-09-21T07:27:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.190+0000 7f2cedc57740 20 client.19692 trim_cache size 0 max 16384 2024-09-21T07:27:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.208+0000 7f2cedc57740 1 Processor -- start 2024-09-21T07:27:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.208+0000 7f2cedc57740 1 -- start start 2024-09-21T07:27:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 0x55daa687fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa6881630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6875540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cedc57740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01b5830 con 0x55daa64cd070 2024-09-21T07:27:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cedc57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01b3100 con 0x55daa64a41b0 2024-09-21T07:27:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cedc57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01b0f70 con 0x55daa64ac390 2024-09-21T07:27:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa6881630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa6881630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53176/0 (socket says 172.21.15.27:53176) 2024-09-21T07:27:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cdf400640 1 -- 172.21.15.27:0/1399650044 learned_addr learned my addr 172.21.15.27:0/1399650044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6875540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.209+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 0x55daa687fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cdf400640 1 -- 172.21.15.27:0/1399650044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 msgr2=0x55daa687fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cdf400640 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 0x55daa687fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cdf400640 1 -- 172.21.15.27:0/1399650044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 msgr2=0x55daa6875540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cdf400640 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6875540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cdf400640 1 -- 172.21.15.27:0/1399650044 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa648bc20 con 0x55daa64ac390 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2ce5000640 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6875540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cdf400640 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa6881630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2ce006c5a0 tx=0x7f2ccc01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e379f5da2a135a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cd4c00640 1 -- 172.21.15.27:0/1399650044 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc003e10 con 0x55daa64ac390 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cd4c00640 1 -- 172.21.15.27:0/1399650044 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc02a910 con 0x55daa64ac390 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.210+0000 7f2cd4c00640 1 -- 172.21.15.27:0/1399650044 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc02b8f0 con 0x55daa64ac390 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 -- 172.21.15.27:0/1399650044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 msgr2=0x55daa6881630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa6881630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2ce006c5a0 tx=0x7f2ccc01a000 comp rx=0 tx=0).stop 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 -- 172.21.15.27:0/1399650044 shutdown_connections 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64ac390 0x55daa6881630 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64cd070 0x55daa6875540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 --2- 172.21.15.27:0/1399650044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64a41b0 0x55daa687fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 -- 172.21.15.27:0/1399650044 >> 172.21.15.27:0/1399650044 conn(0x55daa64c1460 msgr2=0x55daa6874e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 -- 172.21.15.27:0/1399650044 shutdown_connections 2024-09-21T07:27:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.211+0000 7f2cedc57740 1 -- 172.21.15.27:0/1399650044 wait complete. 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 -- start start 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 Processor -- start 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64a41b0 0x55daa6883b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64ac390 0x55daa6884070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa68845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01b15c0 con 0x55daa64ac390 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01b2a30 con 0x55daa64cd070 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.212+0000 7f2cedc57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01b1c90 con 0x55daa64a41b0 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cedc57740 10 client.0 _ll_register_callbacks cb 0x7f2cd00c9e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64ac390 0x55daa6884070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64a41b0 0x55daa6883b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64a41b0 0x55daa6883b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53190/0 (socket says 172.21.15.27:53190) 2024-09-21T07:27:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3653529214 learned_addr learned my addr 172.21.15.27:0/3653529214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2ce5000640 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa68845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 msgr2=0x55daa68845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa68845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3653529214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64ac390 msgr2=0x55daa6884070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64ac390 0x55daa6884070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdfe00640 1 -- 172.21.15.27:0/3653529214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa68fb480 con 0x55daa64a41b0 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.213+0000 7f2cdf400640 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64ac390 0x55daa6884070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cdfe00640 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64a41b0 0x55daa6883b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00c9ff0 tx=0x7f2cd0031cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b191fbdd0817a837 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cca000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cca000640 1 -- 172.21.15.27:0/3653529214 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0049070 con 0x55daa64a41b0 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cca000640 1 -- 172.21.15.27:0/3653529214 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f2cd00cc980 con 0x55daa64a41b0 2024-09-21T07:27:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ccc02a910 con 0x55daa64a41b0 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cca000640 1 -- 172.21.15.27:0/3653529214 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2cd00c90d0 con 0x55daa64a41b0 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cedc57740 10 client.29671 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.214+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55daa68fb480 con 0x55daa64a41b0 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cca000640 1 -- 172.21.15.27:0/3653529214 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc02a910 con 0x55daa64a41b0 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cca000640 1 -- 172.21.15.27:0/3653529214 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2cd00c92b0 con 0x55daa64a41b0 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cedc57740 10 client.29671 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cedc57740 10 client.29671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cedc57740 20 client.29671 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa6460da0 0x55daa68febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cedc57740 4 client.29671 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55daa67e8970 con 0x55daa6460da0 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.215+0000 7f2cdf400640 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa6460da0 0x55daa68febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.216+0000 7f2cdf400640 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa6460da0 0x55daa68febc0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b445e66de3f3ce server_cookie=1d0071141bfe8cd4 in_seq=0 out_seq=0 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.216+0000 7f2cca000640 10 client.29671 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.216+0000 7f2cca000640 1 -- 172.21.15.27:0/3653529214 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6818 (crc 0 0 0) 0x55daa67e8970 con 0x55daa6460da0 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.216+0000 7f2cca000640 10 client.29671 handle_command_reply: tid=0 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.216+0000 7f2cedc57740 1 client.29671 shutdown 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.216+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64a41b0 msgr2=0x55daa6883b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.216+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64a41b0 0x55daa6883b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00c9ff0 tx=0x7f2cd0031cb0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 shutdown_connections 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55daa6460da0 0x55daa68febc0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daa64cd070 0x55daa68845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55daa64ac390 0x55daa6884070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 --2- 172.21.15.27:0/3653529214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daa64a41b0 0x55daa6883b30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cca000640 0 client.29671 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 >> 172.21.15.27:0/3653529214 conn(0x55daa64c1460 msgr2=0x55daa6874e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 shutdown_connections 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 1 -- 172.21.15.27:0/3653529214 wait complete. 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2cedc57740 20 client.29671 trim_cache size 0 max 16384 2024-09-21T07:27:07.221 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "0f52bbc6-199d-4b79-9044-1ebcfed058a5", 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:27:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:27:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:27:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2cc40b6680 msgr2=0x7f2cc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2cc40b6680 0x7f2cc40b8b90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0105a20 tx=0x7f2ccc005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0103990 msgr2=0x7f2ce019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.217+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0103990 0x7f2ce019c510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005940 tx=0x7f2cd00040c0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 shutdown_connections 2024-09-21T07:27:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ce01092d0 0x7f2ce019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f2cc40b6680 0x7f2cc40b8b90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce0109c20 0x7f2ce01a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 --2- 172.21.15.27:0/2923090342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ce0103990 0x7f2ce019c510 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 >> 172.21.15.27:0/2923090342 conn(0x7f2ce00ff3e0 msgr2=0x7f2ce01089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 shutdown_connections 2024-09-21T07:27:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:07.218+0000 7f2ce5a00640 1 -- 172.21.15.27:0/2923090342 wait complete. 2024-09-21T07:27:07.973 INFO:tasks.cephfs.filesystem:scrub status for tag:0f52bbc6-199d-4b79-9044-1ebcfed058a5 - {'path': '/', 'tag': '0f52bbc6-199d-4b79-9044-1ebcfed058a5', 'options': 'recursive,force'} 2024-09-21T07:27:08.451 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:27:08.451 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-21T07:27:08.451 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:27:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:08 smithi027 ceph-mon[30779]: pgmap v818: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 100 KiB/s wr, 34 op/s 2024-09-21T07:27:09.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:08 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:27:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:08 smithi153 ceph-mon[34747]: pgmap v818: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 100 KiB/s wr, 34 op/s 2024-09-21T07:27:09.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:08 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:27:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:08 smithi119 ceph-mon[34519]: pgmap v818: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 100 KiB/s wr, 34 op/s 2024-09-21T07:27:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:08 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:27:09.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 -- 172.21.15.27:0/62532294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f590010c010 msgr2=0x7f5900112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 --2- 172.21.15.27:0/62532294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f590010c010 0x7f5900112840 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009ca0 tx=0x7f58f802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 -- 172.21.15.27:0/62532294 shutdown_connections 2024-09-21T07:27:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 --2- 172.21.15.27:0/62532294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f590010c010 0x7f5900112840 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 --2- 172.21.15.27:0/62532294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f590010b6c0 0x7f590010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 --2- 172.21.15.27:0/62532294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5900105d80 0x7f5900106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 -- 172.21.15.27:0/62532294 >> 172.21.15.27:0/62532294 conn(0x7f5900101810 msgr2=0x7f5900103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 -- 172.21.15.27:0/62532294 shutdown_connections 2024-09-21T07:27:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.440+0000 7f5907a00640 1 -- 172.21.15.27:0/62532294 wait complete. 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.441+0000 7f5907a00640 1 Processor -- start 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5907a00640 1 -- start start 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5907a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5900105d80 0x7f590019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5907a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f590010b6c0 0x7f590019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5907a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590010c010 0x7f59001a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5907a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5900115cd0 con 0x7f5900105d80 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5907a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5900115b50 con 0x7f590010b6c0 2024-09-21T07:27:09.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5907a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5900115e50 con 0x7f590010c010 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5905000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5900105d80 0x7f590019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5905000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5900105d80 0x7f590019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54202/0 (socket says 172.21.15.27:54202) 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f5905000640 1 -- 172.21.15.27:0/858691140 learned_addr learned my addr 172.21.15.27:0/858691140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.442+0000 7f58f7e00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f590010b6c0 0x7f590019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.443+0000 7f5905a00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590010c010 0x7f59001a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.443+0000 7f5905000640 1 -- 172.21.15.27:0/858691140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590010c010 msgr2=0x7f59001a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.443+0000 7f5905000640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590010c010 0x7f59001a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.443+0000 7f5905000640 1 -- 172.21.15.27:0/858691140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f590010b6c0 msgr2=0x7f590019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.443+0000 7f5905000640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f590010b6c0 0x7f590019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.443+0000 7f5905000640 1 -- 172.21.15.27:0/858691140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59001a25b0 con 0x7f5900105d80 2024-09-21T07:27:09.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.443+0000 7f5905000640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5900105d80 0x7f590019c640 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f58f000e9f0 tx=0x7f58f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:09.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.444+0000 7f58fe000640 1 -- 172.21.15.27:0/858691140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f000cd80 con 0x7f5900105d80 2024-09-21T07:27:09.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.444+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59001a28a0 con 0x7f5900105d80 2024-09-21T07:27:09.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.444+0000 7f58fe000640 1 -- 172.21.15.27:0/858691140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f00108e0 con 0x7f5900105d80 2024-09-21T07:27:09.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.444+0000 7f58fe000640 1 -- 172.21.15.27:0/858691140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0018980 con 0x7f5900105d80 2024-09-21T07:27:09.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.444+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59001a9000 con 0x7f5900105d80 2024-09-21T07:27:09.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.445+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900106190 con 0x7f5900105d80 2024-09-21T07:27:09.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.448+0000 7f58fe000640 1 -- 172.21.15.27:0/858691140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f58f0010430 con 0x7f5900105d80 2024-09-21T07:27:09.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.450+0000 7f58fe000640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58e40b6680 0x7f58e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.450+0000 7f58fe000640 1 -- 172.21.15.27:0/858691140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f58f00e8330 con 0x7f5900105d80 2024-09-21T07:27:09.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.450+0000 7f58f7e00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58e40b6680 0x7f58e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.451+0000 7f58f7e00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58e40b6680 0x7f58e40b8b90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5900105060 tx=0x7f58e8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:09.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.452+0000 7f58fe000640 1 -- 172.21.15.27:0/858691140 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f58f00b2800 con 0x7f5900105d80 2024-09-21T07:27:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.698+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f590019d770 con 0x7f5900105d80 2024-09-21T07:27:09.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.699+0000 7f58fe000640 1 -- 172.21.15.27:0/858691140 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f58f00b7740 con 0x7f5900105d80 2024-09-21T07:27:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:27:09.700 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:27:09.701 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.701+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58e40b6680 msgr2=0x7f58e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.701+0000 7f5907a00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58e40b6680 0x7f58e40b8b90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5900105060 tx=0x7f58e8005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5900105d80 msgr2=0x7f590019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5900105d80 0x7f590019c640 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f58f000e9f0 tx=0x7f58f000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 shutdown_connections 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590010c010 0x7f59001a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f58e40b6680 0x7f58e40b8b90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f590010b6c0 0x7f590019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 --2- 172.21.15.27:0/858691140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5900105d80 0x7f590019c640 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 >> 172.21.15.27:0/858691140 conn(0x7f5900101810 msgr2=0x7f5900108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:09.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 shutdown_connections 2024-09-21T07:27:09.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:09.702+0000 7f5907a00640 1 -- 172.21.15.27:0/858691140 wait complete. 2024-09-21T07:27:10.035 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:09 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/858691140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:10.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:09 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/858691140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:09 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/858691140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:10.415 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-21T07:27:10.416 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-21T07:27:10.802 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:11.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:10 smithi027 ceph-mon[30779]: pgmap v819: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-21T07:27:11.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:10 smithi153 ceph-mon[34747]: pgmap v819: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-21T07:27:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:10 smithi119 ceph-mon[34519]: pgmap v819: 129 pgs: 129 active+clean; 1.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-21T07:27:11.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 -- 172.21.15.27:0/334776301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410b6c0 msgr2=0x7f7ab410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 --2- 172.21.15.27:0/334776301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410b6c0 0x7f7ab410bad0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009900 tx=0x7f7aa002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 -- 172.21.15.27:0/334776301 shutdown_connections 2024-09-21T07:27:11.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 --2- 172.21.15.27:0/334776301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab410c010 0x7f7ab4112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 --2- 172.21.15.27:0/334776301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410b6c0 0x7f7ab410bad0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 --2- 172.21.15.27:0/334776301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab4105d80 0x7f7ab4106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 -- 172.21.15.27:0/334776301 >> 172.21.15.27:0/334776301 conn(0x7f7ab41017d0 msgr2=0x7f7ab4103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 -- 172.21.15.27:0/334776301 shutdown_connections 2024-09-21T07:27:11.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.425+0000 7f7ab9600640 1 -- 172.21.15.27:0/334776301 wait complete. 2024-09-21T07:27:11.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 Processor -- start 2024-09-21T07:27:11.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 -- start start 2024-09-21T07:27:11.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab4105d80 0x7f7ab419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab410b6c0 0x7f7ab419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410c010 0x7f7ab41a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4115d20 con 0x7f7ab410c010 2024-09-21T07:27:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4115ba0 con 0x7f7ab4105d80 2024-09-21T07:27:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.427+0000 7f7ab9600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4115ea0 con 0x7f7ab410b6c0 2024-09-21T07:27:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab4105d80 0x7f7ab419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab4105d80 0x7f7ab419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:53528/0 (socket says 172.21.15.27:53528) 2024-09-21T07:27:11.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2629192584 learned_addr learned my addr 172.21.15.27:0/2629192584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab3400640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410c010 0x7f7ab41a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2000640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab410b6c0 0x7f7ab419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2629192584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab410b6c0 msgr2=0x7f7ab419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab410b6c0 0x7f7ab419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410c010 msgr2=0x7f7ab41a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410c010 0x7f7ab41a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2629192584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab41a2620 con 0x7f7ab4105d80 2024-09-21T07:27:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.428+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab4105d80 0x7f7ab419c6b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400e9f0 tx=0x7f7aa400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.429+0000 7f7aab400640 1 -- 172.21.15.27:0/2629192584 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa400cd80 con 0x7f7ab4105d80 2024-09-21T07:27:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.429+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab41a2910 con 0x7f7ab4105d80 2024-09-21T07:27:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.429+0000 7f7aab400640 1 -- 172.21.15.27:0/2629192584 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa40054d0 con 0x7f7ab4105d80 2024-09-21T07:27:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.429+0000 7f7aab400640 1 -- 172.21.15.27:0/2629192584 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4010690 con 0x7f7ab4105d80 2024-09-21T07:27:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.429+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab41a9070 con 0x7f7ab4105d80 2024-09-21T07:27:11.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.432+0000 7f7ac02a8740 1 Processor -- start 2024-09-21T07:27:11.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.432+0000 7f7ac02a8740 1 -- start start 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de1e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de1eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cf2dd94a90 con 0x55cf2da4e990 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cf2dd83ec0 con 0x55cf2da77850 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf2dd87cd0 con 0x55cf2da56b70 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ab2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ab3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de1e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ab2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de1f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35950/0 (socket says 172.21.15.27:35950) 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7ab2000640 1 -- 172.21.15.27:0/199920908 learned_addr learned my addr 172.21.15.27:0/199920908 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.433+0000 7f7aab400640 1 -- 172.21.15.27:0/2629192584 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7aa4004e90 con 0x7f7ab4105d80 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7ab3400640 1 -- 172.21.15.27:0/199920908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 msgr2=0x55cf2de1f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7ab3400640 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de1f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7ab3400640 1 -- 172.21.15.27:0/199920908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 msgr2=0x55cf2de1eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7ab3400640 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de1eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7ab3400640 1 -- 172.21.15.27:0/199920908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2dd299d0 con 0x55cf2da77850 2024-09-21T07:27:11.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7ab2000640 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7ab3400640 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de1e9b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009f00 tx=0x7f7aac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8efdea41462f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7a9f400640 1 -- 172.21.15.27:0/199920908 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac004d20 con 0x55cf2da77850 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7a9f400640 1 -- 172.21.15.27:0/199920908 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aac004030 con 0x55cf2da77850 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.434+0000 7f7a9f400640 1 -- 172.21.15.27:0/199920908 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac005280 con 0x55cf2da77850 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 -- 172.21.15.27:0/199920908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 msgr2=0x55cf2de1e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de1e9b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009f00 tx=0x7f7aac011810 comp rx=0 tx=0).stop 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 -- 172.21.15.27:0/199920908 shutdown_connections 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de1eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/199920908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de1e9b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 -- 172.21.15.27:0/199920908 >> 172.21.15.27:0/199920908 conn(0x55cf2da6bc40 msgr2=0x55cf2de1c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 -- 172.21.15.27:0/199920908 shutdown_connections 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.435+0000 7f7ac02a8740 1 -- 172.21.15.27:0/199920908 wait complete. 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.436+0000 7f7aab400640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a980b6590 0x7f7a980b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.436+0000 7f7ac02a8740 1 -- start start 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.436+0000 7f7aab400640 1 -- 172.21.15.27:0/2629192584 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f7aa40e7f50 con 0x7f7ab4105d80 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.436+0000 7f7ab3400640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a980b6590 0x7f7a980b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.436+0000 7f7ac02a8740 1 Processor -- start 2024-09-21T07:27:11.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ab3400640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a980b6590 0x7f7a980b8aa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006010 tx=0x7f7aac009c00 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de2a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de36480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cf2dbe57b0 con 0x55cf2da4e990 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cf2dd28650 con 0x55cf2da77850 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf2de2d190 con 0x55cf2da56b70 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ac02a8740 10 client.0 _ll_register_callbacks cb 0x55cf2da8f0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de2a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de2a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54254/0 (socket says 172.21.15.27:54254) 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2845065775 learned_addr learned my addr 172.21.15.27:0/2845065775 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ab2000640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.437+0000 7f7ab3400640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de36480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2845065775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 msgr2=0x55cf2de2aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de2aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 msgr2=0x55cf2de36480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de36480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ab2a00640 1 -- 172.21.15.27:0/2845065775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2daf1ba0 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ab3400640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de36480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de2a530 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40e52a0 tx=0x7f7aa40a98a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d43c377629d692f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7a93e00640 1 -- 172.21.15.27:0/2845065775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40b2070 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7a93e00640 1 -- 172.21.15.27:0/2845065775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f7aa40a9290 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.438+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aac004030 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.439+0000 7f7a93e00640 1 -- 172.21.15.27:0/2845065775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7aa40ad0b0 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.439+0000 7f7ac02a8740 10 client.19713 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.439+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf2de390f0 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.439+0000 7f7a93e00640 1 -- 172.21.15.27:0/2845065775 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aac004030 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.439+0000 7f7a93e00640 1 -- 172.21.15.27:0/2845065775 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7aa40ac610 con 0x55cf2da4e990 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.440+0000 7f7ac02a8740 10 client.19713 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.440+0000 7f7ac02a8740 10 client.19713 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.440+0000 7f7ac02a8740 20 client.19713 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.440+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da361d0 0x55cf2de3c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.440+0000 7f7ac02a8740 4 client.19713 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.440+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2dd969e0 con 0x55cf2da361d0 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.440+0000 7f7ab2000640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da361d0 0x55cf2de3c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.441+0000 7f7ab2000640 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da361d0 0x55cf2de3c5c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3885d98cd505cdb server_cookie=2d46e966dfefde84 in_seq=0 out_seq=0 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.441+0000 7f7a93e00640 10 client.19713 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.441+0000 7f7a93e00640 1 -- 172.21.15.27:0/2845065775 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf2dd969e0 con 0x55cf2da361d0 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.441+0000 7f7a93e00640 10 client.19713 handle_command_reply: tid=0 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.441+0000 7f7ac02a8740 1 client.19713 shutdown 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 msgr2=0x55cf2de2a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de2a530 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40e52a0 tx=0x7f7aa40a98a0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 shutdown_connections 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da361d0 0x55cf2de3c5c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da56b70 0x55cf2de2aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de36480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/2845065775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da4e990 0x55cf2de2a530 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 >> 172.21.15.27:0/2845065775 conn(0x55cf2da6bc40 msgr2=0x55cf2de21b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7a93e00640 0 client.19713 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 shutdown_connections 2024-09-21T07:27:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 1 -- 172.21.15.27:0/2845065775 wait complete. 2024-09-21T07:27:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.442+0000 7f7ac02a8740 20 client.19713 trim_cache size 0 max 16384 2024-09-21T07:27:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.460+0000 7f7ac02a8740 1 Processor -- start 2024-09-21T07:27:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.460+0000 7f7ac02a8740 1 -- start start 2024-09-21T07:27:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da4e990 0x55cf2de200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da77850 0x55cf2de27a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b5a70 con 0x55cf2da56b70 2024-09-21T07:27:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b3340 con 0x55cf2da4e990 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b11b0 con 0x55cf2da77850 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de27540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54270/0 (socket says 172.21.15.27:54270) 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 -- 172.21.15.27:0/538711484 learned_addr learned my addr 172.21.15.27:0/538711484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da4e990 0x55cf2de200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab3400640 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da77850 0x55cf2de27a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 -- 172.21.15.27:0/538711484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da77850 msgr2=0x55cf2de27a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da77850 0x55cf2de27a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 -- 172.21.15.27:0/538711484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da4e990 msgr2=0x55cf2de200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da4e990 0x55cf2de200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2000640 1 -- 172.21.15.27:0/538711484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2de2b5f0 con 0x55cf2da56b70 2024-09-21T07:27:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.461+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da4e990 0x55cf2de200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:11.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.462+0000 7f7ab2000640 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de27540 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7ab406ca20 tx=0x7f7aa0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50996e6ccfdd0bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.462+0000 7f7a9ea00640 1 -- 172.21.15.27:0/538711484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa004a020 con 0x55cf2da56b70 2024-09-21T07:27:11.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.462+0000 7f7a9ea00640 1 -- 172.21.15.27:0/538711484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0004750 con 0x55cf2da56b70 2024-09-21T07:27:11.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.462+0000 7f7a9ea00640 1 -- 172.21.15.27:0/538711484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0035040 con 0x55cf2da56b70 2024-09-21T07:27:11.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 -- 172.21.15.27:0/538711484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 msgr2=0x55cf2de27540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de27540 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7ab406ca20 tx=0x7f7aa0005a20 comp rx=0 tx=0).stop 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 -- 172.21.15.27:0/538711484 shutdown_connections 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da77850 0x55cf2de27a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de27540 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/538711484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da4e990 0x55cf2de200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 -- 172.21.15.27:0/538711484 >> 172.21.15.27:0/538711484 conn(0x55cf2da6bc40 msgr2=0x55cf2de1f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 -- 172.21.15.27:0/538711484 shutdown_connections 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 -- 172.21.15.27:0/538711484 wait complete. 2024-09-21T07:27:11.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.463+0000 7f7ac02a8740 1 -- start start 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 1 Processor -- start 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da4e990 0x55cf2de32bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de2d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de2d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b1800 con 0x55cf2da56b70 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b2c70 con 0x55cf2da77850 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b1ed0 con 0x55cf2da4e990 2024-09-21T07:27:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.464+0000 7f7ac02a8740 10 client.0 _ll_register_callbacks cb 0x7f7aa4009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de2d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da4e990 0x55cf2de32bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da4e990 0x55cf2de32bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35982/0 (socket says 172.21.15.27:35982) 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 -- 172.21.15.27:0/1845783626 learned_addr learned my addr 172.21.15.27:0/1845783626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2000640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de2d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54274/0 (socket says 172.21.15.27:54274) 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab3400640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de2d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 -- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 msgr2=0x55cf2de2d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de2d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 -- 172.21.15.27:0/1845783626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 msgr2=0x55cf2de2d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de2d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 -- 172.21.15.27:0/1845783626 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2de2e070 con 0x55cf2da4e990 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2000640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de2d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7ab2a00640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da4e990 0x55cf2de32bd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40d1170 tx=0x7f7aa40c8f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0a55d086e6ba477 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.465+0000 7f7a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.466+0000 7f7a93e00640 1 -- 172.21.15.27:0/1845783626 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40b2040 con 0x55cf2da4e990 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.466+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa0004750 con 0x55cf2da4e990 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.466+0000 7f7a93e00640 1 -- 172.21.15.27:0/1845783626 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9549+0+0 (secure 0 0 0) 0x7f7aa40c3af0 con 0x55cf2da4e990 2024-09-21T07:27:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.466+0000 7f7a93e00640 1 -- 172.21.15.27:0/1845783626 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7aa4005180 con 0x55cf2da4e990 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.466+0000 7f7ac02a8740 10 client.29695 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.466+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf2dea7e60 con 0x55cf2da4e990 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.466+0000 7f7a93e00640 1 -- 172.21.15.27:0/1845783626 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0004750 con 0x55cf2da4e990 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7a93e00640 1 -- 172.21.15.27:0/1845783626 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7aa40b68a0 con 0x55cf2da4e990 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7ac02a8740 10 client.29695 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7ac02a8740 10 client.29695 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7ac02a8740 20 client.29695 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da0b580 0x55cf2deaa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7ac02a8740 4 client.29695 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-21T07:27:11.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x55cf2dd92490 con 0x55cf2da0b580 2024-09-21T07:27:11.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.467+0000 7f7ab2000640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da0b580 0x55cf2deaa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.468+0000 7f7ab2000640 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da0b580 0x55cf2deaa500 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac28adcf94b62a43 server_cookie=417be0b9d1cf2be3 in_seq=0 out_seq=0 2024-09-21T07:27:11.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.468+0000 7f7a93e00640 10 client.29695 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.476+0000 7f7a93e00640 1 -- 172.21.15.27:0/1845783626 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cf2dd92490 con 0x55cf2da0b580 2024-09-21T07:27:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.476+0000 7f7a93e00640 10 client.29695 handle_command_reply: tid=0 2024-09-21T07:27:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.476+0000 7f7ac02a8740 1 client.29695 shutdown 2024-09-21T07:27:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da4e990 msgr2=0x55cf2de32bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da4e990 0x55cf2de32bd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40d1170 tx=0x7f7aa40c8f90 comp rx=0 tx=0).stop 2024-09-21T07:27:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 shutdown_connections 2024-09-21T07:27:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55cf2da0b580 0x55cf2deaa500 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf2da77850 0x55cf2de2d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf2da56b70 0x55cf2de2d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 --2- 172.21.15.27:0/1845783626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf2da4e990 0x55cf2de32bd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 >> 172.21.15.27:0/1845783626 conn(0x55cf2da6bc40 msgr2=0x55cf2de2b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7a93e00640 0 client.29695 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 shutdown_connections 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 1 -- 172.21.15.27:0/1845783626 wait complete. 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.477+0000 7f7ac02a8740 20 client.29695 trim_cache size 0 max 16384 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 54, 2024-09-21T07:27:11.480 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "a8da313c": { 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 54, 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 138.8, 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:27:11.481 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:11.482 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a980b6590 msgr2=0x7f7a980b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a980b6590 0x7f7a980b8aa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006010 tx=0x7f7aac009c00 comp rx=0 tx=0).stop 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab4105d80 msgr2=0x7f7ab419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab4105d80 0x7f7ab419c6b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400e9f0 tx=0x7f7aa400edf0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 shutdown_connections 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ab410b6c0 0x7f7ab419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7a980b6590 0x7f7a980b8aa0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ab410c010 0x7f7ab41a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 --2- 172.21.15.27:0/2629192584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab4105d80 0x7f7ab419c6b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 >> 172.21.15.27:0/2629192584 conn(0x7f7ab41017d0 msgr2=0x7f7ab4103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 shutdown_connections 2024-09-21T07:27:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:11.478+0000 7f7ab9600640 1 -- 172.21.15.27:0/2629192584 wait complete. 2024-09-21T07:27:12.268 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a8da313c', quiesce took 3.8 seconds. Will release after: 15.05 2024-09-21T07:27:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:12 smithi027 ceph-mon[30779]: pgmap v820: 129 pgs: 129 active+clean; 1.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-21T07:27:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:12 smithi027 ceph-mon[30779]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T07:27:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:12 smithi153 ceph-mon[34747]: pgmap v820: 129 pgs: 129 active+clean; 1.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-21T07:27:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:12 smithi153 ceph-mon[34747]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T07:27:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:12 smithi119 ceph-mon[34519]: pgmap v820: 129 pgs: 129 active+clean; 1.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-21T07:27:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:12 smithi119 ceph-mon[34519]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T07:27:14.078 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:13 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:27:14.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:13 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:27:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:13 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:27:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:14 smithi027 ceph-mon[30779]: pgmap v822: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-21T07:27:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:27:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:27:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:27:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:14 smithi153 ceph-mon[34747]: pgmap v822: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-21T07:27:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:27:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:27:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:27:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:14 smithi119 ceph-mon[34519]: pgmap v822: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-21T07:27:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:27:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:27:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:27:17.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:16 smithi119 ceph-mon[34519]: pgmap v823: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:27:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:16 smithi027 ceph-mon[30779]: pgmap v823: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:27:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:16 smithi153 ceph-mon[34747]: pgmap v823: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:27:19.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:18 smithi027 ceph-mon[30779]: pgmap v824: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:27:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:18 smithi153 ceph-mon[34747]: pgmap v824: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:27:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:18 smithi119 ceph-mon[34519]: pgmap v824: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-21T07:27:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 -- 172.21.15.27:0/983678512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c24105cd0 msgr2=0x7f3c241060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 --2- 172.21.15.27:0/983678512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c24105cd0 0x7f3c241060e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009900 tx=0x7f3c1802a410 comp rx=0 tx=0).stop 2024-09-21T07:27:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 -- 172.21.15.27:0/983678512 shutdown_connections 2024-09-21T07:27:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 --2- 172.21.15.27:0/983678512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c2410b600 0x7f3c2410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 --2- 172.21.15.27:0/983678512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c2410bf50 0x7f3c24112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 --2- 172.21.15.27:0/983678512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c24105cd0 0x7f3c241060e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 -- 172.21.15.27:0/983678512 >> 172.21.15.27:0/983678512 conn(0x7f3c241016f0 msgr2=0x7f3c24103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 -- 172.21.15.27:0/983678512 shutdown_connections 2024-09-21T07:27:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.294+0000 7f3c2c400640 1 -- 172.21.15.27:0/983678512 wait complete. 2024-09-21T07:27:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.295+0000 7f3c2c400640 1 Processor -- start 2024-09-21T07:27:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.295+0000 7f3c2c400640 1 -- start start 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.295+0000 7f3c2c400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24105cd0 0x7f3c2419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.295+0000 7f3c2c400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c2410b600 0x7f3c2419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.295+0000 7f3c2c400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2410bf50 0x7f3c241a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.295+0000 7f3c2c400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24115c10 con 0x7f3c2410bf50 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.295+0000 7f3c2c400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24115a90 con 0x7f3c2410b600 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2c400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24115d90 con 0x7f3c24105cd0 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2410bf50 0x7f3c241a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2410bf50 0x7f3c241a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39040/0 (socket says 172.21.15.27:39040) 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 -- 172.21.15.27:0/1566105910 learned_addr learned my addr 172.21.15.27:0/1566105910 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c29a00640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24105cd0 0x7f3c2419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c29000640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c2410b600 0x7f3c2419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 -- 172.21.15.27:0/1566105910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24105cd0 msgr2=0x7f3c2419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24105cd0 0x7f3c2419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 -- 172.21.15.27:0/1566105910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c2410b600 msgr2=0x7f3c2419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c2410b600 0x7f3c2419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.296+0000 7f3c2a400640 1 -- 172.21.15.27:0/1566105910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c241a2560 con 0x7f3c2410bf50 2024-09-21T07:27:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.297+0000 7f3c2a400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2410bf50 0x7f3c241a1e60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00eb10 tx=0x7f3c1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.297+0000 7f3c22000640 1 -- 172.21.15.27:0/1566105910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c004b20 con 0x7f3c2410bf50 2024-09-21T07:27:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.297+0000 7f3c22000640 1 -- 172.21.15.27:0/1566105910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c1c004cc0 con 0x7f3c2410bf50 2024-09-21T07:27:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.297+0000 7f3c22000640 1 -- 172.21.15.27:0/1566105910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c010670 con 0x7f3c2410bf50 2024-09-21T07:27:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.297+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c241a2850 con 0x7f3c2410bf50 2024-09-21T07:27:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.297+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c241a8fe0 con 0x7f3c2410bf50 2024-09-21T07:27:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.300+0000 7f3c33481740 1 Processor -- start 2024-09-21T07:27:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.301+0000 7f3c22000640 1 -- 172.21.15.27:0/1566105910 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3c1c005020 con 0x7f3c2410bf50 2024-09-21T07:27:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.301+0000 7f3c33481740 1 -- start start 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.301+0000 7f3c33481740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8677c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.301+0000 7f3c33481740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8678170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c33481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82b24d0 0x5644e86786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c33481740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5644e85ed6b0 con 0x5644e82aa2f0 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c33481740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5644e85dcae0 con 0x5644e82d31b0 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c33481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5644e85e08f0 con 0x5644e82b24d0 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8678170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8677c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8678170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39044/0 (socket says 172.21.15.27:39044) 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 -- 172.21.15.27:0/316043829 learned_addr learned my addr 172.21.15.27:0/316043829 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29000640 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82b24d0 0x5644e86786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 -- 172.21.15.27:0/316043829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82b24d0 msgr2=0x5644e86786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82b24d0 0x5644e86786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 -- 172.21.15.27:0/316043829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 msgr2=0x5644e8677c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8677c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29a00640 1 -- 172.21.15.27:0/316043829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e85a9780 con 0x5644e82aa2f0 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.302+0000 7f3c29000640 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8677c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.303+0000 7f3c29a00640 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8678170 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009900 tx=0x7f3c18032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ae1f723e6dbb06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.303+0000 7f3c16a00640 1 -- 172.21.15.27:0/316043829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18040070 con 0x5644e82aa2f0 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.303+0000 7f3c16a00640 1 -- 172.21.15.27:0/316043829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c18004030 con 0x5644e82aa2f0 2024-09-21T07:27:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.303+0000 7f3c16a00640 1 -- 172.21.15.27:0/316043829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18035040 con 0x5644e82aa2f0 2024-09-21T07:27:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.303+0000 7f3c22000640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c0c0b6700 0x7f3c0c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c29000640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c0c0b6700 0x7f3c0c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c22000640 1 -- 172.21.15.27:0/1566105910 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f3c1c0e8080 con 0x7f3c2410bf50 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 -- 172.21.15.27:0/316043829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 msgr2=0x5644e8678170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8678170 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009900 tx=0x7f3c18032b90 comp rx=0 tx=0).stop 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 -- 172.21.15.27:0/316043829 shutdown_connections 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82b24d0 0x5644e86786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8678170 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 --2- 172.21.15.27:0/316043829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8677c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 -- 172.21.15.27:0/316043829 >> 172.21.15.27:0/316043829 conn(0x5644e82c7300 msgr2=0x5644e8675b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 -- 172.21.15.27:0/316043829 shutdown_connections 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.304+0000 7f3c33481740 1 -- 172.21.15.27:0/316043829 wait complete. 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.305+0000 7f3c29000640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c0c0b6700 0x7f3c0c0b8c10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c2410b140 tx=0x7f3c10004730 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.305+0000 7f3c33481740 1 -- start start 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.305+0000 7f3c33481740 1 Processor -- start 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c33481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8683200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c33481740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8683740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c33481740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82d31b0 0x5644e8683c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c33481740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5644e8514ab0 con 0x5644e82d31b0 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c33481740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5644e8583570 con 0x5644e82b24d0 2024-09-21T07:27:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c33481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5644e8686350 con 0x5644e82aa2f0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c33481740 10 client.0 _ll_register_callbacks cb 0x5644e8592100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82d31b0 0x5644e8683c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82d31b0 0x5644e8683c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39052/0 (socket says 172.21.15.27:39052) 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 -- 172.21.15.27:0/3614738472 learned_addr learned my addr 172.21.15.27:0/3614738472 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c29a00640 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8683200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c29000640 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8683740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 -- 172.21.15.27:0/3614738472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 msgr2=0x5644e8683200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8683200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 -- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 msgr2=0x5644e8683740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8683740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.306+0000 7f3c2a400640 1 -- 172.21.15.27:0/3614738472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e833e790 con 0x5644e82d31b0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c2a400640 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82d31b0 0x5644e8683c80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00cfb0 tx=0x7f3c1c0ad950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecd7e470f05786b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c0b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c0b400640 1 -- 172.21.15.27:0/3614738472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0b2020 con 0x5644e82d31b0 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c0b400640 1 -- 172.21.15.27:0/3614738472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f3c1c0b7050 con 0x5644e82d31b0 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c18004030 con 0x5644e82d31b0 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c0b400640 1 -- 172.21.15.27:0/3614738472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c1c0ad5d0 con 0x5644e82d31b0 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c33481740 10 client.19731 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.307+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644e83b7620 con 0x5644e82d31b0 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c0b400640 1 -- 172.21.15.27:0/3614738472 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c18004030 con 0x5644e82d31b0 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c0b400640 1 -- 172.21.15.27:0/3614738472 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c1c0e5b10 con 0x5644e82d31b0 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c33481740 10 client.19731 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c33481740 10 client.19731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c33481740 20 client.19731 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c33481740 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e8291d80 0x5644e8695340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c33481740 4 client.19731 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e85ef600 con 0x5644e8291d80 2024-09-21T07:27:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.308+0000 7f3c29a00640 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e8291d80 0x5644e8695340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.309+0000 7f3c29a00640 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e8291d80 0x5644e8695340 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cb2cc126a370c8 server_cookie=6482d90202e06fec in_seq=0 out_seq=0 2024-09-21T07:27:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.309+0000 7f3c0b400640 10 client.19731 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c0b400640 1 -- 172.21.15.27:0/3614738472 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644e85ef600 con 0x5644e8291d80 2024-09-21T07:27:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c0b400640 10 client.19731 handle_command_reply: tid=0 2024-09-21T07:27:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 client.19731 shutdown 2024-09-21T07:27:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82d31b0 msgr2=0x5644e8683c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82d31b0 0x5644e8683c80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00cfb0 tx=0x7f3c1c0ad950 comp rx=0 tx=0).stop 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 shutdown_connections 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e8291d80 0x5644e8695340 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82d31b0 0x5644e8683c80 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8683740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 --2- 172.21.15.27:0/3614738472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8683200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 >> 172.21.15.27:0/3614738472 conn(0x5644e82c7300 msgr2=0x5644e8678c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c0b400640 0 client.19731 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 shutdown_connections 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 1 -- 172.21.15.27:0/3614738472 wait complete. 2024-09-21T07:27:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.310+0000 7f3c33481740 20 client.19731 trim_cache size 0 max 16384 2024-09-21T07:27:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.325+0000 7f3c33481740 1 Processor -- start 2024-09-21T07:27:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.325+0000 7f3c33481740 1 -- start start 2024-09-21T07:27:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c33481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8679370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c33481740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82b24d0 0x5644e86803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c33481740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8680910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c33481740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c241b5920 con 0x5644e82b24d0 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c33481740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c241b31f0 con 0x5644e82d31b0 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c33481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c241b1060 con 0x5644e82aa2f0 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c29000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82b24d0 0x5644e86803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c29000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82b24d0 0x5644e86803d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39066/0 (socket says 172.21.15.27:39066) 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c29000640 1 -- 172.21.15.27:0/3118709471 learned_addr learned my addr 172.21.15.27:0/3118709471 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c29a00640 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8679370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.326+0000 7f3c2a400640 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8680910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c2a400640 1 -- 172.21.15.27:0/3118709471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 msgr2=0x5644e8679370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c2a400640 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8679370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c2a400640 1 -- 172.21.15.27:0/3118709471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82b24d0 msgr2=0x5644e86803d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c2a400640 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82b24d0 0x5644e86803d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c2a400640 1 -- 172.21.15.27:0/3118709471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e8684080 con 0x5644e82d31b0 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c29000640 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82b24d0 0x5644e86803d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c2a400640 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8680910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0ad770 tx=0x7f3c1c0ad9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8292229a88a190 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c16000640 1 -- 172.21.15.27:0/3118709471 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0b2040 con 0x5644e82d31b0 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c16000640 1 -- 172.21.15.27:0/3118709471 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c1c0ae1c0 con 0x5644e82d31b0 2024-09-21T07:27:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.327+0000 7f3c16000640 1 -- 172.21.15.27:0/3118709471 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0b74e0 con 0x5644e82d31b0 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 -- 172.21.15.27:0/3118709471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 msgr2=0x5644e8680910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8680910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0ad770 tx=0x7f3c1c0ad9c0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 -- 172.21.15.27:0/3118709471 shutdown_connections 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82d31b0 0x5644e8680910 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82b24d0 0x5644e86803d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 --2- 172.21.15.27:0/3118709471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82aa2f0 0x5644e8679370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 -- 172.21.15.27:0/3118709471 >> 172.21.15.27:0/3118709471 conn(0x5644e82c7300 msgr2=0x5644e8678c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 -- 172.21.15.27:0/3118709471 shutdown_connections 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.328+0000 7f3c33481740 1 -- 172.21.15.27:0/3118709471 wait complete. 2024-09-21T07:27:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.329+0000 7f3c33481740 1 -- start start 2024-09-21T07:27:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.329+0000 7f3c33481740 1 Processor -- start 2024-09-21T07:27:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.329+0000 7f3c33481740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8689640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.329+0000 7f3c33481740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8689b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.329+0000 7f3c33481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82d31b0 0x5644e868a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c33481740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c241b16b0 con 0x5644e82aa2f0 2024-09-21T07:27:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c33481740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c241b2b20 con 0x5644e82b24d0 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c33481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c241b1d80 con 0x5644e82d31b0 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c33481740 10 client.0 _ll_register_callbacks cb 0x7f3c1c0a99d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8689640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8689640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39082/0 (socket says 172.21.15.27:39082) 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 -- 172.21.15.27:0/716350733 learned_addr learned my addr 172.21.15.27:0/716350733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29000640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8689b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c2a400640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82d31b0 0x5644e868a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 -- 172.21.15.27:0/716350733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82d31b0 msgr2=0x5644e868a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82d31b0 0x5644e868a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 -- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 msgr2=0x5644e8689b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8689b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c29a00640 1 -- 172.21.15.27:0/716350733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e86fee00 con 0x5644e82aa2f0 2024-09-21T07:27:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.330+0000 7f3c2a400640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82d31b0 0x5644e868a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c29a00640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8689640 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005150 tx=0x7f3c1802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f1b5d595ec29d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c0b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c0b400640 1 -- 172.21.15.27:0/716350733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18040040 con 0x5644e82aa2f0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c1c0ae1c0 con 0x5644e82aa2f0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c0b400640 1 -- 172.21.15.27:0/716350733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f3c18031d10 con 0x5644e82aa2f0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c0b400640 1 -- 172.21.15.27:0/716350733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c1805e050 con 0x5644e82aa2f0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c33481740 10 client.19737 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644e8700120 con 0x5644e82aa2f0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.331+0000 7f3c0b400640 1 -- 172.21.15.27:0/716350733 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c1c0ae1c0 con 0x5644e82aa2f0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c0b400640 1 -- 172.21.15.27:0/716350733 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3c180046c0 con 0x5644e82aa2f0 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c33481740 10 client.19737 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c33481740 10 client.19737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c33481740 20 client.19737 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c33481740 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e82671c0 0x5644e8702530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c33481740 4 client.19737 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:27:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644e85eb0b0 con 0x5644e82671c0 2024-09-21T07:27:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.332+0000 7f3c29000640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e82671c0 0x5644e8702530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.333+0000 7f3c29000640 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e82671c0 0x5644e8702530 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c45c87fe1fae8f server_cookie=3ae2d85f1a4c29cf in_seq=0 out_seq=0 2024-09-21T07:27:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.333+0000 7f3c0b400640 10 client.19737 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c0b400640 1 -- 172.21.15.27:0/716350733 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644e85eb0b0 con 0x5644e82671c0 2024-09-21T07:27:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c0b400640 10 client.19737 handle_command_reply: tid=0 2024-09-21T07:27:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 client.19737 shutdown 2024-09-21T07:27:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 msgr2=0x5644e8689640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8689640 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005150 tx=0x7f3c1802a410 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 shutdown_connections 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5644e82671c0 0x5644e8702530 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644e82d31b0 0x5644e868a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c0b400640 0 client.19737 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644e82b24d0 0x5644e8689b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 --2- 172.21.15.27:0/716350733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5644e82aa2f0 0x5644e8689640 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 >> 172.21.15.27:0/716350733 conn(0x5644e82c7300 msgr2=0x5644e8675ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.346+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 shutdown_connections 2024-09-21T07:27:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c33481740 1 -- 172.21.15.27:0/716350733 wait complete. 2024-09-21T07:27:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c33481740 20 client.19737 trim_cache size 0 max 16384 2024-09-21T07:27:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:27:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:27:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:27:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:27:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c0c0b6700 msgr2=0x7f3c0c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c0c0b6700 0x7f3c0c0b8c10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c2410b140 tx=0x7f3c10004730 comp rx=0 tx=0).stop 2024-09-21T07:27:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2410bf50 msgr2=0x7f3c241a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2410bf50 0x7f3c241a1e60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00eb10 tx=0x7f3c1c00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 shutdown_connections 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3c0c0b6700 0x7f3c0c0b8c10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2410bf50 0x7f3c241a1e60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c2410b600 0x7f3c2419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.347+0000 7f3c2c400640 1 --2- 172.21.15.27:0/1566105910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24105cd0 0x7f3c2419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.348+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 >> 172.21.15.27:0/1566105910 conn(0x7f3c241016f0 msgr2=0x7f3c24103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.348+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 shutdown_connections 2024-09-21T07:27:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:20.348+0000 7f3c2c400640 1 -- 172.21.15.27:0/1566105910 wait complete. 2024-09-21T07:27:21.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:20 smithi027 ceph-mon[30779]: pgmap v825: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-21T07:27:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:20 smithi153 ceph-mon[34747]: pgmap v825: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-21T07:27:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:20 smithi119 ceph-mon[34519]: pgmap v825: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-21T07:27:23.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:22 smithi027 ceph-mon[30779]: pgmap v826: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-21T07:27:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:22 smithi153 ceph-mon[34747]: pgmap v826: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-21T07:27:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:22 smithi119 ceph-mon[34519]: pgmap v826: 129 pgs: 129 active+clean; 2.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-21T07:27:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:24 smithi153 ceph-mon[34747]: pgmap v827: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-09-21T07:27:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:24 smithi119 ceph-mon[34519]: pgmap v827: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-09-21T07:27:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:24 smithi027 ceph-mon[30779]: pgmap v827: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-09-21T07:27:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:26 smithi153 ceph-mon[34747]: pgmap v828: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:27:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:26 smithi119 ceph-mon[34519]: pgmap v828: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:27:27.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:27:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:26 smithi027 ceph-mon[30779]: pgmap v828: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-09-21T07:27:27.721 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:28.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 -- 172.21.15.27:0/2348844547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4078290 msgr2=0x7f3cf4114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/2348844547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4078290 0x7f3cf4114030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009860 tx=0x7f3ce002a340 comp rx=0 tx=0).stop 2024-09-21T07:27:28.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 -- 172.21.15.27:0/2348844547 shutdown_connections 2024-09-21T07:27:28.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/2348844547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4078290 0x7f3cf4114030 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/2348844547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf4114570 0x7f3cf4116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/2348844547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4077940 0x7f3cf4077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 -- 172.21.15.27:0/2348844547 >> 172.21.15.27:0/2348844547 conn(0x7f3cf4101680 msgr2=0x7f3cf4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 -- 172.21.15.27:0/2348844547 shutdown_connections 2024-09-21T07:27:28.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.384+0000 7f3cf8e00640 1 -- 172.21.15.27:0/2348844547 wait complete. 2024-09-21T07:27:28.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.385+0000 7f3cf8e00640 1 Processor -- start 2024-09-21T07:27:28.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.385+0000 7f3cf8e00640 1 -- start start 2024-09-21T07:27:28.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf4077940 0x7f3cf41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4078290 0x7f3cf41a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4114570 0x7f3cf41a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf8e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4119ed0 con 0x7f3cf4077940 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf8e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4119d50 con 0x7f3cf4078290 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf8e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf411a050 con 0x7f3cf4114570 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf4077940 0x7f3cf41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf4077940 0x7f3cf41a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45800/0 (socket says 172.21.15.27:45800) 2024-09-21T07:27:28.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf3400640 1 -- 172.21.15.27:0/404478406 learned_addr learned my addr 172.21.15.27:0/404478406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf2a00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4078290 0x7f3cf41a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.386+0000 7f3cf3e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4114570 0x7f3cf41a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf3e00640 1 -- 172.21.15.27:0/404478406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4078290 msgr2=0x7f3cf41a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf3e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4078290 0x7f3cf41a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf3e00640 1 -- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf4077940 msgr2=0x7f3cf41a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf3e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf4077940 0x7f3cf41a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf3e00640 1 -- 172.21.15.27:0/404478406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf41a6890 con 0x7f3cf4114570 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf3e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4114570 0x7f3cf41a6160 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00efc0 tx=0x7f3cec00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cebe00640 1 -- 172.21.15.27:0/404478406 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec003970 con 0x7f3cf4114570 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cebe00640 1 -- 172.21.15.27:0/404478406 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cec004020 con 0x7f3cf4114570 2024-09-21T07:27:28.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf41a6b80 con 0x7f3cf4114570 2024-09-21T07:27:28.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cebe00640 1 -- 172.21.15.27:0/404478406 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec00b940 con 0x7f3cf4114570 2024-09-21T07:27:28.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.387+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf41ad2e0 con 0x7f3cf4114570 2024-09-21T07:27:28.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.388+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf4077d50 con 0x7f3cf4114570 2024-09-21T07:27:28.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.391+0000 7f3cebe00640 1 -- 172.21.15.27:0/404478406 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3cec01d020 con 0x7f3cf4114570 2024-09-21T07:27:28.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.393+0000 7f3cebe00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3cd80b6700 0x7f3cd80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.393+0000 7f3cebe00640 1 -- 172.21.15.27:0/404478406 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f3cec0e7f90 con 0x7f3cf4114570 2024-09-21T07:27:28.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.393+0000 7f3cf3400640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3cd80b6700 0x7f3cd80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.394+0000 7f3cf3400640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3cd80b6700 0x7f3cd80b8c10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009c90 tx=0x7f3ce4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:28.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.395+0000 7f3cebe00640 1 -- 172.21.15.27:0/404478406 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3cec0ae0d0 con 0x7f3cf4114570 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.633+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cf41a1a50 con 0x7f3cf4114570 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:27:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.635+0000 7f3cebe00640 1 -- 172.21.15.27:0/404478406 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f3cf41a1a50 con 0x7f3cf4114570 2024-09-21T07:27:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:27:28.638 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:27:28.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3cd80b6700 msgr2=0x7f3cd80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3cd80b6700 0x7f3cd80b8c10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009c90 tx=0x7f3ce4005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:28.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4114570 msgr2=0x7f3cf41a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4114570 0x7f3cf41a6160 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00efc0 tx=0x7f3cec00c800 comp rx=0 tx=0).stop 2024-09-21T07:27:28.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 shutdown_connections 2024-09-21T07:27:28.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf4114570 0x7f3cf41a6160 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3cd80b6700 0x7f3cd80b8c10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cf4078290 0x7f3cf41a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 --2- 172.21.15.27:0/404478406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf4077940 0x7f3cf41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.637+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 >> 172.21.15.27:0/404478406 conn(0x7f3cf4101680 msgr2=0x7f3cf4116680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.638+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 shutdown_connections 2024-09-21T07:27:28.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:28.638+0000 7f3cf8e00640 1 -- 172.21.15.27:0/404478406 wait complete. 2024-09-21T07:27:28.989 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:28 smithi027 ceph-mon[30779]: pgmap v829: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-21T07:27:28.989 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:28 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/404478406' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:28 smithi153 ceph-mon[34747]: pgmap v829: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-21T07:27:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:28 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/404478406' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:28 smithi119 ceph-mon[34519]: pgmap v829: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-21T07:27:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:28 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/404478406' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:29.480 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id a8da313c --release --await' 2024-09-21T07:27:29.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id a8da313c --release --await 2024-09-21T07:27:29.908 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 -- 172.21.15.27:0/732617539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 msgr2=0x7fd87c0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 --2- 172.21.15.27:0/732617539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 0x7fd87c0757b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009900 tx=0x7fd86c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 -- 172.21.15.27:0/732617539 shutdown_connections 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 --2- 172.21.15.27:0/732617539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd87c075dd0 0x7fd87c0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 --2- 172.21.15.27:0/732617539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c114280 0x7fd87c116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 --2- 172.21.15.27:0/732617539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 0x7fd87c0757b0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 -- 172.21.15.27:0/732617539 >> 172.21.15.27:0/732617539 conn(0x7fd87c0ff440 msgr2=0x7fd87c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 -- 172.21.15.27:0/732617539 shutdown_connections 2024-09-21T07:27:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.553+0000 7fd880c00640 1 -- 172.21.15.27:0/732617539 wait complete. 2024-09-21T07:27:30.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.555+0000 7fd880c00640 1 Processor -- start 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.555+0000 7fd880c00640 1 -- start start 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.555+0000 7fd880c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c075dd0 0x7fd87c1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.555+0000 7fd880c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 0x7fd87c1a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.555+0000 7fd880c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd87c114280 0x7fd87c1a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd880c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c119ec0 con 0x7fd87c077380 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd880c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c119d40 con 0x7fd87c075dd0 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd880c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c11a040 con 0x7fd87c114280 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c075dd0 0x7fd87c1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c075dd0 0x7fd87c1a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58260/0 (socket says 172.21.15.27:58260) 2024-09-21T07:27:30.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 -- 172.21.15.27:0/98299131 learned_addr learned my addr 172.21.15.27:0/98299131 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:30.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87aa00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 0x7fd87c1a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87be00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd87c114280 0x7fd87c1a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 -- 172.21.15.27:0/98299131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd87c114280 msgr2=0x7fd87c1a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd87c114280 0x7fd87c1a6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 -- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 msgr2=0x7fd87c1a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 0x7fd87c1a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.556+0000 7fd87b400640 1 -- 172.21.15.27:0/98299131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c1a6840 con 0x7fd87c075dd0 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.557+0000 7fd87b400640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c075dd0 0x7fd87c1a08d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005940 tx=0x7fd86c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.557+0000 7fd873e00640 1 -- 172.21.15.27:0/98299131 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c03f030 con 0x7fd87c075dd0 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.557+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c1a6ad0 con 0x7fd87c075dd0 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.557+0000 7fd873e00640 1 -- 172.21.15.27:0/98299131 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86c003cb0 con 0x7fd87c075dd0 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.557+0000 7fd873e00640 1 -- 172.21.15.27:0/98299131 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c035070 con 0x7fd87c075dd0 2024-09-21T07:27:30.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.557+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1ad290 con 0x7fd87c075dd0 2024-09-21T07:27:30.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.561+0000 7fd888fbd740 1 Processor -- start 2024-09-21T07:27:30.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.561+0000 7fd888fbd740 1 -- start start 2024-09-21T07:27:30.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd888fbd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56392993d290 con 0x5639295fe840 2024-09-21T07:27:30.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd888fbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56392992c6c0 con 0x56392961f520 2024-09-21T07:27:30.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd888fbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5639299304d0 con 0x5639295f6660 2024-09-21T07:27:30.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd87be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd87aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.562+0000 7fd87be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58270/0 (socket says 172.21.15.27:58270) 2024-09-21T07:27:30.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd87be00640 1 -- 172.21.15.27:0/2390095707 learned_addr learned my addr 172.21.15.27:0/2390095707 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:30.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd873e00640 1 -- 172.21.15.27:0/98299131 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd86c042030 con 0x7fd87c075dd0 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd87be00640 1 -- 172.21.15.27:0/2390095707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 msgr2=0x5639299c6880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd87be00640 1 --2- 172.21.15.27:0/2390095707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c6880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd87be00640 1 -- 172.21.15.27:0/2390095707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 msgr2=0x5639299c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd87be00640 1 --2- 172.21.15.27:0/2390095707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd87be00640 1 -- 172.21.15.27:0/2390095707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639298b9460 con 0x56392961f520 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.563+0000 7fd87be00640 1 --2- 172.21.15.27:0/2390095707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299c6340 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd874010370 tx=0x7fd874010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d7d32c4f73234b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.564+0000 7fd867e00640 1 -- 172.21.15.27:0/2390095707 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874016070 con 0x56392961f520 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.564+0000 7fd867e00640 1 -- 172.21.15.27:0/2390095707 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd874004d20 con 0x56392961f520 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.564+0000 7fd867e00640 1 -- 172.21.15.27:0/2390095707 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874005420 con 0x56392961f520 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 -- 172.21.15.27:0/2390095707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 msgr2=0x5639299c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2390095707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299c6340 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd874010370 tx=0x7fd874010830 comp rx=0 tx=0).stop 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 -- 172.21.15.27:0/2390095707 shutdown_connections 2024-09-21T07:27:30.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2390095707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2390095707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2390095707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299c6340 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 -- 172.21.15.27:0/2390095707 >> 172.21.15.27:0/2390095707 conn(0x563929613910 msgr2=0x5639299c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 -- 172.21.15.27:0/2390095707 shutdown_connections 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd873e00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8600b6610 0x7fd8600b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd888fbd740 1 -- 172.21.15.27:0/2390095707 wait complete. 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.565+0000 7fd873e00640 1 -- 172.21.15.27:0/98299131 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fd86c1082b0 con 0x7fd87c075dd0 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.566+0000 7fd87aa00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8600b6610 0x7fd8600b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.566+0000 7fd888fbd740 1 -- start start 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 1 Processor -- start 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd87aa00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8600b6610 0x7fd8600b8b20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd87c116210 tx=0x7fd86800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299dde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56392978dbb0 con 0x5639295fe840 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5639298c76c0 con 0x56392961f520 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5639299d4b20 con 0x5639295f6660 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.567+0000 7fd888fbd740 10 client.0 _ll_register_callbacks cb 0x5639298e2490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299d1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39390/0 (socket says 172.21.15.27:39390) 2024-09-21T07:27:30.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 -- 172.21.15.27:0/3216683056 learned_addr learned my addr 172.21.15.27:0/3216683056 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87be00640 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 -- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 msgr2=0x5639299dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299dde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 -- 172.21.15.27:0/3216683056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 msgr2=0x5639299d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.568+0000 7fd87b400640 1 -- 172.21.15.27:0/3216683056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392968ab00 con 0x5639295f6660 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd87b400640 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299d1f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd86c003fd0 tx=0x7fd86c0cdc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ccf6827767cb2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd864c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd864c00640 1 -- 172.21.15.27:0/3216683056 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c049070 con 0x5639295f6660 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd864c00640 1 -- 172.21.15.27:0/3216683056 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fd86c105870 con 0x5639295f6660 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd874004d20 con 0x5639295f6660 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd864c00640 1 -- 172.21.15.27:0/3216683056 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd86c105a70 con 0x5639295f6660 2024-09-21T07:27:30.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd888fbd740 10 client.29728 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.569+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563929701960 con 0x5639295f6660 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd864c00640 1 -- 172.21.15.27:0/3216683056 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd874004d20 con 0x5639295f6660 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd864c00640 1 -- 172.21.15.27:0/3216683056 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd86c03bbf0 con 0x5639295f6660 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd888fbd740 10 client.29728 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd888fbd740 10 client.29728 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd888fbd740 20 client.29728 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd888fbd740 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295ddea0 0x5639299e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd888fbd740 4 client.29728 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56392993f1e0 con 0x5639295ddea0 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.570+0000 7fd87aa00640 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295ddea0 0x5639299e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.571+0000 7fd87aa00640 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295ddea0 0x5639299e3e50 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71661c598807baef server_cookie=1d2461ea8a9facd4 in_seq=0 out_seq=0 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.571+0000 7fd864c00640 10 client.29728 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:30.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd864c00640 1 -- 172.21.15.27:0/3216683056 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56392993f1e0 con 0x5639295ddea0 2024-09-21T07:27:30.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd864c00640 10 client.29728 handle_command_reply: tid=0 2024-09-21T07:27:30.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 client.29728 shutdown 2024-09-21T07:27:30.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 msgr2=0x5639299d1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299d1f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd86c003fd0 tx=0x7fd86c0cdc30 comp rx=0 tx=0).stop 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 shutdown_connections 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295ddea0 0x5639299e3e50 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56392961f520 0x5639299dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 --2- 172.21.15.27:0/3216683056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299d1f50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 >> 172.21.15.27:0/3216683056 conn(0x563929613910 msgr2=0x5639299c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.572+0000 7fd864c00640 0 client.29728 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.573+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 shutdown_connections 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.573+0000 7fd888fbd740 1 -- 172.21.15.27:0/3216683056 wait complete. 2024-09-21T07:27:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.573+0000 7fd888fbd740 20 client.29728 trim_cache size 0 max 16384 2024-09-21T07:27:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 Processor -- start 2024-09-21T07:27:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 -- start start 2024-09-21T07:27:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295fe840 0x5639299c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56392961f520 0x5639299cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1b9b00 con 0x56392961f520 2024-09-21T07:27:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1b73d0 con 0x5639295fe840 2024-09-21T07:27:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.591+0000 7fd888fbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1b5240 con 0x5639295f6660 2024-09-21T07:27:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56392961f520 0x5639299cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56392961f520 0x5639299cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45840/0 (socket says 172.21.15.27:45840) 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87be00640 1 -- 172.21.15.27:0/1275829225 learned_addr learned my addr 172.21.15.27:0/1275829225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87aa00640 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295fe840 0x5639299c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87b400640 1 -- 172.21.15.27:0/1275829225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295fe840 msgr2=0x5639299c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87b400640 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295fe840 0x5639299c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87b400640 1 -- 172.21.15.27:0/1275829225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56392961f520 msgr2=0x5639299cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87b400640 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56392961f520 0x5639299cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87b400640 1 -- 172.21.15.27:0/1275829225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639299c91a0 con 0x5639295f6660 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.592+0000 7fd87aa00640 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295fe840 0x5639299c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.593+0000 7fd87b400640 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c9440 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004650 tx=0x7fd86c0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4677385b604fb42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.593+0000 7fd867400640 1 -- 172.21.15.27:0/1275829225 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0c9c40 con 0x5639295f6660 2024-09-21T07:27:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.593+0000 7fd867400640 1 -- 172.21.15.27:0/1275829225 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86c0cd8b0 con 0x5639295f6660 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.593+0000 7fd867400640 1 -- 172.21.15.27:0/1275829225 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c03bbf0 con 0x5639295f6660 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 -- 172.21.15.27:0/1275829225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 msgr2=0x5639299c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c9440 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004650 tx=0x7fd86c0045a0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 -- 172.21.15.27:0/1275829225 shutdown_connections 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56392961f520 0x5639299cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295fe840 0x5639299c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 --2- 172.21.15.27:0/1275829225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639295f6660 0x5639299c9440 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 -- 172.21.15.27:0/1275829225 >> 172.21.15.27:0/1275829225 conn(0x563929613910 msgr2=0x5639299ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 -- 172.21.15.27:0/1275829225 shutdown_connections 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 -- 172.21.15.27:0/1275829225 wait complete. 2024-09-21T07:27:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.594+0000 7fd888fbd740 1 -- start start 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 1 Processor -- start 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295f6660 0x5639299da330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56392961f520 0x5639299d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1b5890 con 0x5639295fe840 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1b6d00 con 0x5639295f6660 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1b5f60 con 0x56392961f520 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.595+0000 7fd888fbd740 10 client.0 _ll_register_callbacks cb 0x7fd86c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295f6660 0x5639299da330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295f6660 0x5639299da330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58306/0 (socket says 172.21.15.27:58306) 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 -- 172.21.15.27:0/2862010055 learned_addr learned my addr 172.21.15.27:0/2862010055 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87aa00640 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87be00640 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56392961f520 0x5639299d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 -- 172.21.15.27:0/2862010055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56392961f520 msgr2=0x5639299d6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56392961f520 0x5639299d6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 -- 172.21.15.27:0/2862010055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 msgr2=0x5639299d6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 -- 172.21.15.27:0/2862010055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639299c4fc0 con 0x5639295f6660 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd87b400640 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295f6660 0x5639299da330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0cdd30 tx=0x7fd86c03be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74d3a4ee12cc595 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.596+0000 7fd864c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd864c00640 1 -- 172.21.15.27:0/2862010055 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0c9c40 con 0x5639295f6660 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86c0cd8b0 con 0x5639295f6660 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd864c00640 1 -- 172.21.15.27:0/2862010055 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fd86c0f1310 con 0x5639295f6660 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd864c00640 1 -- 172.21.15.27:0/2862010055 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd86c105a80 con 0x5639295f6660 2024-09-21T07:27:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd888fbd740 10 client.29789 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:30.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639299d4490 con 0x5639295f6660 2024-09-21T07:27:30.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd864c00640 1 -- 172.21.15.27:0/2862010055 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86c0cd8b0 con 0x5639295f6660 2024-09-21T07:27:30.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd864c00640 1 -- 172.21.15.27:0/2862010055 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd86c07d0d0 con 0x5639295f6660 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd888fbd740 10 client.29789 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd888fbd740 10 client.29789 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.597+0000 7fd888fbd740 20 client.29789 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.598+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295b3250 0x563929a51530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.598+0000 7fd888fbd740 4 client.29789 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a8da313c", "release": true, "await": true}] 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.598+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a8da313c", "release": true, "await": true}) -- 0x56392993ac90 con 0x5639295b3250 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.598+0000 7fd87aa00640 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295b3250 0x563929a51530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.598+0000 7fd87aa00640 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295b3250 0x563929a51530 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4a5cd46502f5fe server_cookie=7fa5c0f6b7ad4d1 in_seq=0 out_seq=0 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.598+0000 7fd864c00640 10 client.29789 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.601+0000 7fd864c00640 1 -- 172.21.15.27:0/2862010055 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56392993ac90 con 0x5639295b3250 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.601+0000 7fd864c00640 10 client.29789 handle_command_reply: tid=0 2024-09-21T07:27:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.601+0000 7fd888fbd740 1 client.29789 shutdown 2024-09-21T07:27:30.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.601+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295f6660 msgr2=0x5639299da330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.601+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295f6660 0x5639299da330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0cdd30 tx=0x7fd86c03be20 comp rx=0 tx=0).stop 2024-09-21T07:27:30.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 shutdown_connections 2024-09-21T07:27:30.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5639295b3250 0x563929a51530 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56392961f520 0x5639299d6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639295fe840 0x5639299d6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 --2- 172.21.15.27:0/2862010055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639295f6660 0x5639299da330 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 >> 172.21.15.27:0/2862010055 conn(0x563929613910 msgr2=0x5639299c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd864c00640 0 client.29789 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 shutdown_connections 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 1 -- 172.21.15.27:0/2862010055 wait complete. 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.602+0000 7fd888fbd740 20 client.29789 trim_cache size 0 max 16384 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:27:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 56, 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "a8da313c": { 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 56, 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 138.8, 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:27:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:30.608 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8600b6610 msgr2=0x7fd8600b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8600b6610 0x7fd8600b8b20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd87c116210 tx=0x7fd86800b390 comp rx=0 tx=0).stop 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c075dd0 msgr2=0x7fd87c1a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c075dd0 0x7fd87c1a08d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005940 tx=0x7fd86c0042f0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 shutdown_connections 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd87c114280 0x7fd87c1a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd8600b6610 0x7fd8600b8b20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c077380 0x7fd87c1a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 --2- 172.21.15.27:0/98299131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87c075dd0 0x7fd87c1a08d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 >> 172.21.15.27:0/98299131 conn(0x7fd87c0ff440 msgr2=0x7fd87c110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 shutdown_connections 2024-09-21T07:27:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:30.603+0000 7fd880c00640 1 -- 172.21.15.27:0/98299131 wait complete. 2024-09-21T07:27:31.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:30 smithi027 ceph-mon[30779]: pgmap v830: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-21T07:27:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:30 smithi153 ceph-mon[34747]: pgmap v830: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-21T07:27:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:30 smithi119 ceph-mon[34519]: pgmap v830: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-21T07:27:31.451 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a8da313c', total seconds elapsed: 23.0 2024-09-21T07:27:31.451 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-21T07:27:31.451 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:27:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 -- 172.21.15.27:0/2756405610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 msgr2=0x7fbb50109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 --2- 172.21.15.27:0/2756405610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 0x7fbb50109700 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009860 tx=0x7fbb3c02a340 comp rx=0 tx=0).stop 2024-09-21T07:27:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 -- 172.21.15.27:0/2756405610 shutdown_connections 2024-09-21T07:27:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 --2- 172.21.15.27:0/2756405610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb50109c40 0x7fbb50110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 --2- 172.21.15.27:0/2756405610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 0x7fbb50109700 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 --2- 172.21.15.27:0/2756405610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501039b0 0x7fbb50103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 -- 172.21.15.27:0/2756405610 >> 172.21.15.27:0/2756405610 conn(0x7fbb500ff400 msgr2=0x7fbb50101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:32.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 -- 172.21.15.27:0/2756405610 shutdown_connections 2024-09-21T07:27:32.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.430+0000 7fbb54a00640 1 -- 172.21.15.27:0/2756405610 wait complete. 2024-09-21T07:27:32.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.431+0000 7fbb54a00640 1 Processor -- start 2024-09-21T07:27:32.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.431+0000 7fbb54a00640 1 -- start start 2024-09-21T07:27:32.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb54a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb501039b0 0x7fbb5019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb54a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 0x7fbb5019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb54a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb50109c40 0x7fbb501a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb54a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50113900 con 0x7fbb501092f0 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb54a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50113780 con 0x7fbb50109c40 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb54a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50113a80 con 0x7fbb501039b0 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb4de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 0x7fbb5019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb4e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb501039b0 0x7fbb5019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb4de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 0x7fbb5019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45882/0 (socket says 172.21.15.27:45882) 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb4e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb501039b0 0x7fbb5019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39424/0 (socket says 172.21.15.27:39424) 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb4de00640 1 -- 172.21.15.27:0/3817004311 learned_addr learned my addr 172.21.15.27:0/3817004311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:32.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.432+0000 7fbb4f200640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb50109c40 0x7fbb501a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.433+0000 7fbb4e800640 1 -- 172.21.15.27:0/3817004311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb50109c40 msgr2=0x7fbb501a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.433+0000 7fbb4e800640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb50109c40 0x7fbb501a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.433+0000 7fbb4e800640 1 -- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 msgr2=0x7fbb5019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.433+0000 7fbb4e800640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 0x7fbb5019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.433+0000 7fbb4e800640 1 -- 172.21.15.27:0/3817004311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb501a2420 con 0x7fbb501039b0 2024-09-21T07:27:32.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.433+0000 7fbb4e800640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb501039b0 0x7fbb5019c4b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbb40005fd0 tx=0x7fbb4000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:32.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.434+0000 7fbb47400640 1 -- 172.21.15.27:0/3817004311 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000cd00 con 0x7fbb501039b0 2024-09-21T07:27:32.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.434+0000 7fbb47400640 1 -- 172.21.15.27:0/3817004311 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb400035f0 con 0x7fbb501039b0 2024-09-21T07:27:32.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.434+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb501a2710 con 0x7fbb501039b0 2024-09-21T07:27:32.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.434+0000 7fbb47400640 1 -- 172.21.15.27:0/3817004311 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40010690 con 0x7fbb501039b0 2024-09-21T07:27:32.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.434+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb501a8e70 con 0x7fbb501039b0 2024-09-21T07:27:32.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.435+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb50103dc0 con 0x7fbb501039b0 2024-09-21T07:27:32.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.438+0000 7fbb47400640 1 -- 172.21.15.27:0/3817004311 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbb40003a80 con 0x7fbb501039b0 2024-09-21T07:27:32.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.440+0000 7fbb47400640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbb340b6680 0x7fbb340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.440+0000 7fbb47400640 1 -- 172.21.15.27:0/3817004311 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fbb400e7dc0 con 0x7fbb501039b0 2024-09-21T07:27:32.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.440+0000 7fbb4de00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbb340b6680 0x7fbb340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.441+0000 7fbb4de00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbb340b6680 0x7fbb340b8b90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0057c0 tx=0x7fbb3c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:32.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.441+0000 7fbb47400640 1 -- 172.21.15.27:0/3817004311 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbb40009510 con 0x7fbb501039b0 2024-09-21T07:27:32.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.680+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb5019d5e0 con 0x7fbb501039b0 2024-09-21T07:27:32.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.681+0000 7fbb47400640 1 -- 172.21.15.27:0/3817004311 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fbb5019d5e0 con 0x7fbb501039b0 2024-09-21T07:27:32.684 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:27:32.684 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:27:32.684 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:27:32.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbb340b6680 msgr2=0x7fbb340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbb340b6680 0x7fbb340b8b90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0057c0 tx=0x7fbb3c0058d0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb501039b0 msgr2=0x7fbb5019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb501039b0 0x7fbb5019c4b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbb40005fd0 tx=0x7fbb4000ee50 comp rx=0 tx=0).stop 2024-09-21T07:27:32.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 shutdown_connections 2024-09-21T07:27:32.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbb340b6680 0x7fbb340b8b90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb50109c40 0x7fbb501a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb501092f0 0x7fbb5019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 --2- 172.21.15.27:0/3817004311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb501039b0 0x7fbb5019c4b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 >> 172.21.15.27:0/3817004311 conn(0x7fbb500ff400 msgr2=0x7fbb50106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:32.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 shutdown_connections 2024-09-21T07:27:32.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:32.684+0000 7fbb54a00640 1 -- 172.21.15.27:0/3817004311 wait complete. 2024-09-21T07:27:33.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:32 smithi027 ceph-mon[30779]: pgmap v831: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-21T07:27:33.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:32 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3817004311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:32 smithi153 ceph-mon[34747]: pgmap v831: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-21T07:27:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:32 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3817004311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:32 smithi119 ceph-mon[34519]: pgmap v831: 129 pgs: 129 active+clean; 2.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-21T07:27:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:32 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3817004311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:33.505 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-21T07:27:33.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-21T07:27:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 -- 172.21.15.27:0/416921026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 msgr2=0x7f775c103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 --2- 172.21.15.27:0/416921026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 0x7f775c103ef0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f774c009900 tx=0x7f774c02a410 comp rx=0 tx=0).stop 2024-09-21T07:27:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 -- 172.21.15.27:0/416921026 shutdown_connections 2024-09-21T07:27:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 --2- 172.21.15.27:0/416921026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f775c109410 0x7f775c109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 --2- 172.21.15.27:0/416921026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c109d60 0x7f775c110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 --2- 172.21.15.27:0/416921026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 0x7f775c103ef0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 -- 172.21.15.27:0/416921026 >> 172.21.15.27:0/416921026 conn(0x7f775c0ff520 msgr2=0x7f775c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 -- 172.21.15.27:0/416921026 shutdown_connections 2024-09-21T07:27:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.551+0000 7f7762400640 1 -- 172.21.15.27:0/416921026 wait complete. 2024-09-21T07:27:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.552+0000 7f7762400640 1 Processor -- start 2024-09-21T07:27:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.552+0000 7f7762400640 1 -- start start 2024-09-21T07:27:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f7762400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 0x7f775c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f7762400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f775c109410 0x7f775c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f7762400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c109d60 0x7f775c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f7762400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f775c113a70 con 0x7f775c103ae0 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f7762400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1138f0 con 0x7f775c109d60 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f7762400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f775c113bf0 con 0x7f775c109410 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 0x7f775c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 0x7f775c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45896/0 (socket says 172.21.15.27:45896) 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f775c109410 0x7f775c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775b400640 1 -- 172.21.15.27:0/1437532398 learned_addr learned my addr 172.21.15.27:0/1437532398 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775be00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c109d60 0x7f775c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775aa00640 1 -- 172.21.15.27:0/1437532398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c109d60 msgr2=0x7f775c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775aa00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c109d60 0x7f775c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775aa00640 1 -- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 msgr2=0x7f775c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775aa00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 0x7f775c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775aa00640 1 -- 172.21.15.27:0/1437532398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c1a8740 con 0x7f775c109410 2024-09-21T07:27:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.553+0000 7f775be00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c109d60 0x7f775c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.554+0000 7f775aa00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f775c109410 0x7f775c075c50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f775c06c8d0 tx=0x7f774800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.554+0000 7f7753e00640 1 -- 172.21.15.27:0/1437532398 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774800cc40 con 0x7f775c109410 2024-09-21T07:27:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.554+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c1a8a30 con 0x7f775c109410 2024-09-21T07:27:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.554+0000 7f7753e00640 1 -- 172.21.15.27:0/1437532398 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774800cde0 con 0x7f775c109410 2024-09-21T07:27:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.554+0000 7f7753e00640 1 -- 172.21.15.27:0/1437532398 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748004a00 con 0x7f775c109410 2024-09-21T07:27:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.554+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c1a8f80 con 0x7f775c109410 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.558+0000 7f776963f740 1 Processor -- start 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f776963f740 1 -- start start 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f776963f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd85c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f776963f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9b82f0 0x5558bcd86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f776963f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9c04d0 0x5558bcd866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f776963f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5558bccfb6b0 con 0x5558bc9e11b0 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f776963f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5558bcceaae0 con 0x5558bc9c04d0 2024-09-21T07:27:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f776963f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5558bccee8f0 con 0x5558bc9b82f0 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.559+0000 7f775be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd85c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd85c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45898/0 (socket says 172.21.15.27:45898) 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 -- 172.21.15.27:0/3013870358 learned_addr learned my addr 172.21.15.27:0/3013870358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775b400640 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9b82f0 0x5558bcd86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 -- 172.21.15.27:0/3013870358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9b82f0 msgr2=0x5558bcd86170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9b82f0 0x5558bcd86170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 -- 172.21.15.27:0/3013870358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9c04d0 msgr2=0x5558bcd866b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9c04d0 0x5558bcd866b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f7753e00640 1 -- 172.21.15.27:0/1437532398 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7748004ba0 con 0x7f775c109410 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 -- 172.21.15.27:0/3013870358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bccb7780 con 0x5558bc9e11b0 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775b400640 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9b82f0 0x5558bcd86170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.560+0000 7f775be00640 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd85c30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7754009fd0 tx=0x7f77540108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1aece9ad044d3d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.561+0000 7f7750c00640 1 -- 172.21.15.27:0/3013870358 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754011760 con 0x5558bc9e11b0 2024-09-21T07:27:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.561+0000 7f7750c00640 1 -- 172.21.15.27:0/3013870358 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7754011db0 con 0x5558bc9e11b0 2024-09-21T07:27:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.561+0000 7f7750c00640 1 -- 172.21.15.27:0/3013870358 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775400e3c0 con 0x5558bc9e11b0 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 -- 172.21.15.27:0/3013870358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 msgr2=0x5558bcd85c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd85c30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7754009fd0 tx=0x7f77540108e0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 -- 172.21.15.27:0/3013870358 shutdown_connections 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f7753e00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f77400b66b0 0x7f77400b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9b82f0 0x5558bcd86170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9c04d0 0x5558bcd866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 --2- 172.21.15.27:0/3013870358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd85c30 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 -- 172.21.15.27:0/3013870358 >> 172.21.15.27:0/3013870358 conn(0x5558bc9d5300 msgr2=0x5558bcd83b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f7753e00640 1 -- 172.21.15.27:0/1437532398 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f77480e7ab0 con 0x7f775c109410 2024-09-21T07:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 -- 172.21.15.27:0/3013870358 shutdown_connections 2024-09-21T07:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f776963f740 1 -- 172.21.15.27:0/3013870358 wait complete. 2024-09-21T07:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.562+0000 7f775be00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f77400b66b0 0x7f77400b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.563+0000 7f776963f740 1 -- start start 2024-09-21T07:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.563+0000 7f776963f740 1 Processor -- start 2024-09-21T07:27:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.563+0000 7f775be00640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f77400b66b0 0x7f77400b8bc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7754011010 tx=0x7f775401b9f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f776963f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd913a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f776963f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f776963f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd9d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f776963f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5558bcc22ab0 con 0x5558bc9e11b0 2024-09-21T07:27:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f776963f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5558bcc91570 con 0x5558bc9b82f0 2024-09-21T07:27:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f776963f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5558bcd943b0 con 0x5558bc9c04d0 2024-09-21T07:27:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f776963f740 10 client.0 _ll_register_callbacks cb 0x5558bcca0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd9d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd913a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd913a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58344/0 (socket says 172.21.15.27:58344) 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 -- 172.21.15.27:0/1238175312 learned_addr learned my addr 172.21.15.27:0/1238175312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775aa00640 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 -- 172.21.15.27:0/1238175312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 msgr2=0x5558bcd918e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd918e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 -- 172.21.15.27:0/1238175312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 msgr2=0x5558bcd9d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd9d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.564+0000 7f775b400640 1 -- 172.21.15.27:0/1238175312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bca4c790 con 0x5558bc9b82f0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f775b400640 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd913a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f774c02aa00 tx=0x7f774c004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f44a9732cb35e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f7745600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f7745600640 1 -- 172.21.15.27:0/1238175312 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c035040 con 0x5558bc9b82f0 2024-09-21T07:27:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f7745600640 1 -- 172.21.15.27:0/1238175312 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f774c003ac0 con 0x5558bc9b82f0 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7754011db0 con 0x5558bc9b82f0 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f7745600640 1 -- 172.21.15.27:0/1238175312 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f774c003cc0 con 0x5558bc9b82f0 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f776963f740 10 client.29804 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.565+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558bcac5620 con 0x5558bc9b82f0 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f7745600640 1 -- 172.21.15.27:0/1238175312 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7754011db0 con 0x5558bc9b82f0 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f7745600640 1 -- 172.21.15.27:0/1238175312 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f774c03c4d0 con 0x5558bc9b82f0 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f776963f740 10 client.29804 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f776963f740 10 client.29804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f776963f740 20 client.29804 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f776963f740 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc99fd80 0x5558bcda34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f776963f740 4 client.29804 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558bccfd600 con 0x5558bc99fd80 2024-09-21T07:27:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.566+0000 7f775aa00640 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc99fd80 0x5558bcda34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.567+0000 7f775aa00640 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc99fd80 0x5558bcda34d0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f20fec961fee35 server_cookie=ce8a014ad2912507 in_seq=0 out_seq=0 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.567+0000 7f7745600640 10 client.29804 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f7745600640 1 -- 172.21.15.27:0/1238175312 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558bccfd600 con 0x5558bc99fd80 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f7745600640 10 client.29804 handle_command_reply: tid=0 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 client.29804 shutdown 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 msgr2=0x5558bcd913a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd913a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f774c02aa00 tx=0x7f774c004440 comp rx=0 tx=0).stop 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 shutdown_connections 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc99fd80 0x5558bcda34d0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd918e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd9d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f7745600640 0 client.29804 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 --2- 172.21.15.27:0/1238175312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd913a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 >> 172.21.15.27:0/1238175312 conn(0x5558bc9d5300 msgr2=0x5558bcd86c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.568+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 shutdown_connections 2024-09-21T07:27:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.569+0000 7f776963f740 1 -- 172.21.15.27:0/1238175312 wait complete. 2024-09-21T07:27:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.569+0000 7f776963f740 20 client.29804 trim_cache size 0 max 16384 2024-09-21T07:27:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.584+0000 7f776963f740 1 Processor -- start 2024-09-21T07:27:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.584+0000 7f776963f740 1 -- start start 2024-09-21T07:27:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f776963f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd91880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f776963f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd87620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f776963f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd92fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f776963f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b59a0 con 0x5558bc9e11b0 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f776963f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b3270 con 0x5558bc9b82f0 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f776963f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b10e0 con 0x5558bc9c04d0 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd92fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd92fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45910/0 (socket says 172.21.15.27:45910) 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775be00640 1 -- 172.21.15.27:0/3527392 learned_addr learned my addr 172.21.15.27:0/3527392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775b400640 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd91880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775aa00640 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd87620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775be00640 1 -- 172.21.15.27:0/3527392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 msgr2=0x5558bcd87620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775be00640 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd87620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775be00640 1 -- 172.21.15.27:0/3527392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 msgr2=0x5558bcd91880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.585+0000 7f775be00640 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd91880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.586+0000 7f775be00640 1 -- 172.21.15.27:0/3527392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bcd8b830 con 0x5558bc9e11b0 2024-09-21T07:27:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.586+0000 7f775b400640 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd91880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.586+0000 7f775be00640 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd92fd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f775401ba60 tx=0x7f7754013050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a35a96ce937133a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.586+0000 7f7747e00640 1 -- 172.21.15.27:0/3527392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754037070 con 0x5558bc9e11b0 2024-09-21T07:27:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.586+0000 7f7747e00640 1 -- 172.21.15.27:0/3527392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7754011a10 con 0x5558bc9e11b0 2024-09-21T07:27:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.586+0000 7f7747e00640 1 -- 172.21.15.27:0/3527392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77540055b0 con 0x5558bc9e11b0 2024-09-21T07:27:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 -- 172.21.15.27:0/3527392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 msgr2=0x5558bcd92fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd92fd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f775401ba60 tx=0x7f7754013050 comp rx=0 tx=0).stop 2024-09-21T07:27:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 -- 172.21.15.27:0/3527392 shutdown_connections 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd87620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9e11b0 0x5558bcd92fd0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 --2- 172.21.15.27:0/3527392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9b82f0 0x5558bcd91880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 -- 172.21.15.27:0/3527392 >> 172.21.15.27:0/3527392 conn(0x5558bc9d5300 msgr2=0x5558bcd928c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 -- 172.21.15.27:0/3527392 shutdown_connections 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.587+0000 7f776963f740 1 -- 172.21.15.27:0/3527392 wait complete. 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 -- start start 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 Processor -- start 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9b82f0 0x5558bcd99e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd95320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9e11b0 0x5558bcd95920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b1730 con 0x5558bc9b82f0 2024-09-21T07:27:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b2ba0 con 0x5558bc9e11b0 2024-09-21T07:27:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b1e00 con 0x5558bc9c04d0 2024-09-21T07:27:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.588+0000 7f776963f740 10 client.0 _ll_register_callbacks cb 0x7f774c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9b82f0 0x5558bcd99e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd95320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd95320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:39490/0 (socket says 172.21.15.27:39490) 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 -- 172.21.15.27:0/1772909484 learned_addr learned my addr 172.21.15.27:0/1772909484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775be00640 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9e11b0 0x5558bcd95920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 -- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9e11b0 msgr2=0x5558bcd95920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9e11b0 0x5558bcd95920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 -- 172.21.15.27:0/1772909484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9b82f0 msgr2=0x5558bcd99e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9b82f0 0x5558bcd99e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 -- 172.21.15.27:0/1772909484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bcd96080 con 0x5558bc9c04d0 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f775aa00640 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd95320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f774800c6a0 tx=0x7f7748005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10114ad97e774fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.589+0000 7f7745600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f7745600640 1 -- 172.21.15.27:0/1772909484 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774801c040 con 0x5558bc9c04d0 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f7745600640 1 -- 172.21.15.27:0/1772909484 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f77480ac7b0 con 0x5558bc9c04d0 2024-09-21T07:27:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7754011a10 con 0x5558bc9c04d0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f7745600640 1 -- 172.21.15.27:0/1772909484 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77480adc50 con 0x5558bc9c04d0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 10 client.29758 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558bce0df20 con 0x5558bc9c04d0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f7745600640 1 -- 172.21.15.27:0/1772909484 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7754011a10 con 0x5558bc9c04d0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f7745600640 1 -- 172.21.15.27:0/1772909484 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f77480ac220 con 0x5558bc9c04d0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 10 client.29758 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 10 client.29758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 20 client.29758 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc9751c0 0x5558bce10660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 4 client.29758 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.590+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558bccf90b0 con 0x5558bc9751c0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.591+0000 7f775b400640 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc9751c0 0x5558bce10660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.591+0000 7f775b400640 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc9751c0 0x5558bce10660 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76ed1b29b5ab4a0 server_cookie=d81cbdfbad8a4d03 in_seq=0 out_seq=0 2024-09-21T07:27:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.591+0000 7f7745600640 10 client.29758 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.604+0000 7f7745600640 1 -- 172.21.15.27:0/1772909484 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558bccf90b0 con 0x5558bc9751c0 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.604+0000 7f7745600640 10 client.29758 handle_command_reply: tid=0 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.604+0000 7f776963f740 1 client.29758 shutdown 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.604+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 msgr2=0x5558bcd95320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.604+0000 7f776963f740 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd95320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f774800c6a0 tx=0x7f7748005250 comp rx=0 tx=0).stop 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 shutdown_connections 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5558bc9751c0 0x5558bce10660 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558bc9c04d0 0x5558bcd95320 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5558bc9e11b0 0x5558bcd95920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 --2- 172.21.15.27:0/1772909484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5558bc9b82f0 0x5558bcd99e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f7745600640 0 client.29758 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 >> 172.21.15.27:0/1772909484 conn(0x5558bc9d5300 msgr2=0x5558bcd873f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 shutdown_connections 2024-09-21T07:27:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 1 -- 172.21.15.27:0/1772909484 wait complete. 2024-09-21T07:27:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.605+0000 7f776963f740 20 client.29758 trim_cache size 0 max 16384 2024-09-21T07:27:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:27:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:27:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:27:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f77400b66b0 msgr2=0x7f77400b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f77400b66b0 0x7f77400b8bc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7754011010 tx=0x7f775401b9f0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f775c109410 msgr2=0x7f775c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f775c109410 0x7f775c075c50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f775c06c8d0 tx=0x7f774800ee50 comp rx=0 tx=0).stop 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 shutdown_connections 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f775c109410 0x7f775c075c50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f77400b66b0 0x7f77400b8bc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c109d60 0x7f775c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 --2- 172.21.15.27:0/1437532398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f775c103ae0 0x7f775c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 >> 172.21.15.27:0/1437532398 conn(0x7f775c0ff520 msgr2=0x7f775c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 shutdown_connections 2024-09-21T07:27:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:33.606+0000 7f7762400640 1 -- 172.21.15.27:0/1437532398 wait complete. 2024-09-21T07:27:33.906 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:34.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 -- 172.21.15.27:0/573938019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568075dd0 msgr2=0x7f15680761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 --2- 172.21.15.27:0/573938019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568075dd0 0x7f15680761e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1554009900 tx=0x7f155402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 -- 172.21.15.27:0/573938019 shutdown_connections 2024-09-21T07:27:34.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 --2- 172.21.15.27:0/573938019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1568114260 0x7f1568116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 --2- 172.21.15.27:0/573938019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568075dd0 0x7f15680761e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 --2- 172.21.15.27:0/573938019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1568077380 0x7f15680757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 -- 172.21.15.27:0/573938019 >> 172.21.15.27:0/573938019 conn(0x7f15680ff3e0 msgr2=0x7f1568101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:34.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 -- 172.21.15.27:0/573938019 shutdown_connections 2024-09-21T07:27:34.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.576+0000 7f156da00640 1 -- 172.21.15.27:0/573938019 wait complete. 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.577+0000 7f156da00640 1 Processor -- start 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.577+0000 7f156da00640 1 -- start start 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.577+0000 7f156da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1568075dd0 0x7f15681a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.577+0000 7f156da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568077380 0x7f15681a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f156da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1568114260 0x7f15681a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f156da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1568119e60 con 0x7f1568077380 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f156da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1568119ce0 con 0x7f1568114260 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f156da00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1568119fe0 con 0x7f1568075dd0 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568077380 0x7f15681a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568077380 0x7f15681a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45936/0 (socket says 172.21.15.27:45936) 2024-09-21T07:27:34.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 -- 172.21.15.27:0/451875413 learned_addr learned my addr 172.21.15.27:0/451875413 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:34.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567e00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1568075dd0 0x7f15681a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f156d000640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1568114260 0x7f15681a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 -- 172.21.15.27:0/451875413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1568075dd0 msgr2=0x7f15681a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1568075dd0 0x7f15681a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 -- 172.21.15.27:0/451875413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1568114260 msgr2=0x7f15681a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1568114260 0x7f15681a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.578+0000 7f1567400640 1 -- 172.21.15.27:0/451875413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681a6800 con 0x7f1568077380 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.579+0000 7f1567400640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568077380 0x7f15681a0e30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f1554004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.579+0000 7f1564c00640 1 -- 172.21.15.27:0/451875413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155403f030 con 0x7f1568077380 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.579+0000 7f1564c00640 1 -- 172.21.15.27:0/451875413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1554003aa0 con 0x7f1568077380 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.579+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15681a6a90 con 0x7f1568077380 2024-09-21T07:27:34.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.579+0000 7f1564c00640 1 -- 172.21.15.27:0/451875413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554035070 con 0x7f1568077380 2024-09-21T07:27:34.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.580+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15681ad250 con 0x7f1568077380 2024-09-21T07:27:34.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 Processor -- start 2024-09-21T07:27:34.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 -- start start 2024-09-21T07:27:34.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba5a0850 0x55caba947b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba9480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba948610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55caba8bda90 con 0x55caba5a0850 2024-09-21T07:27:34.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55caba8acec0 con 0x55caba57fb70 2024-09-21T07:27:34.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.583+0000 7f1575bdb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55caba8b0cd0 con 0x55caba577990 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f156d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba5a0850 0x55caba947b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f156d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba5a0850 0x55caba947b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45938/0 (socket says 172.21.15.27:45938) 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f156d000640 1 -- 172.21.15.27:0/3041567012 learned_addr learned my addr 172.21.15.27:0/3041567012 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f1567e00640 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba9480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f1564c00640 1 -- 172.21.15.27:0/451875413 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1554042030 con 0x7f1568077380 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f1567e00640 1 -- 172.21.15.27:0/3041567012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 msgr2=0x55caba948610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f1567e00640 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba948610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f1567e00640 1 -- 172.21.15.27:0/3041567012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba5a0850 msgr2=0x55caba947b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f1567e00640 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba5a0850 0x55caba947b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.584+0000 7f1567e00640 1 -- 172.21.15.27:0/3041567012 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caba8529d0 con 0x55caba577990 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.585+0000 7f1567e00640 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba9480d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1558005fd0 tx=0x7f15580107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504aa038659092ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.585+0000 7f155d600640 1 -- 172.21.15.27:0/3041567012 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155801c680 con 0x55caba577990 2024-09-21T07:27:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.585+0000 7f155d600640 1 -- 172.21.15.27:0/3041567012 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f155800be20 con 0x55caba577990 2024-09-21T07:27:34.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.585+0000 7f155d600640 1 -- 172.21.15.27:0/3041567012 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558011b90 con 0x55caba577990 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1564c00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f154c0b6670 0x7f154c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 -- 172.21.15.27:0/3041567012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 msgr2=0x55caba9480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba9480d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1558005fd0 tx=0x7f15580107a0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1564c00640 1 -- 172.21.15.27:0/451875413 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f1554107be0 con 0x7f1568077380 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 -- 172.21.15.27:0/3041567012 shutdown_connections 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba9480d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba948610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 --2- 172.21.15.27:0/3041567012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba5a0850 0x55caba947b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 -- 172.21.15.27:0/3041567012 >> 172.21.15.27:0/3041567012 conn(0x55caba594c40 msgr2=0x55caba945a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f156d000640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f154c0b6670 0x7f154c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 -- 172.21.15.27:0/3041567012 shutdown_connections 2024-09-21T07:27:34.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.586+0000 7f1575bdb740 1 -- 172.21.15.27:0/3041567012 wait complete. 2024-09-21T07:27:34.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.587+0000 7f156d000640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f154c0b6670 0x7f154c0b8b80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1560005fd0 tx=0x7f1560009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:34.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.587+0000 7f1575bdb740 1 -- start start 2024-09-21T07:27:34.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.587+0000 7f1575bdb740 1 Processor -- start 2024-09-21T07:27:34.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba953720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba953c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1575bdb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55caba70e7b0 con 0x55caba57fb70 2024-09-21T07:27:34.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1575bdb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55caba851650 con 0x55caba5a0850 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1575bdb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55caba956370 con 0x55caba577990 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1575bdb740 10 client.0 _ll_register_callbacks cb 0x55caba5b80a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1567400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba953c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f156d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1567400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba953c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45954/0 (socket says 172.21.15.27:45954) 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1567400640 1 -- 172.21.15.27:0/1125262701 learned_addr learned my addr 172.21.15.27:0/1125262701 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.588+0000 7f1567e00640 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba953720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f156d000640 1 -- 172.21.15.27:0/1125262701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 msgr2=0x55caba953720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f156d000640 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba953720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f156d000640 1 -- 172.21.15.27:0/1125262701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 msgr2=0x55caba953c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f156d000640 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba953c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f156d000640 1 -- 172.21.15.27:0/1125262701 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caba61aba0 con 0x55caba5a0850 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f1567400640 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba953c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f156d000640 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f690 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1560003f20 tx=0x7f1560004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38c6bdba1cc3b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:34.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f1552000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:34.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f1552000640 1 -- 172.21.15.27:0/1125262701 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156000f7b0 con 0x55caba5a0850 2024-09-21T07:27:34.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f155800be20 con 0x55caba5a0850 2024-09-21T07:27:34.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.589+0000 7f1552000640 1 -- 172.21.15.27:0/1125262701 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f156000f950 con 0x55caba5a0850 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.590+0000 7f1552000640 1 -- 172.21.15.27:0/1125262701 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f156000fb50 con 0x55caba5a0850 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.590+0000 7f1575bdb740 10 client.29819 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.590+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55caba9623e0 con 0x55caba5a0850 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.590+0000 7f1552000640 1 -- 172.21.15.27:0/1125262701 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f155800be20 con 0x55caba5a0850 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1552000640 1 -- 172.21.15.27:0/1125262701 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1560026070 con 0x55caba5a0850 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1575bdb740 10 client.29819 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1575bdb740 10 client.29819 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:34.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1575bdb740 20 client.29819 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:34.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba55f1d0 0x55caba965950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1575bdb740 4 client.29819 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:34.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caba8bf9e0 con 0x55caba55f1d0 2024-09-21T07:27:34.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.591+0000 7f1567e00640 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba55f1d0 0x55caba965950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.592+0000 7f1567e00640 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba55f1d0 0x55caba965950 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8357f269fc57b9 server_cookie=1fbf6947a034a908 in_seq=0 out_seq=0 2024-09-21T07:27:34.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.592+0000 7f1552000640 10 client.29819 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:34.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1552000640 1 -- 172.21.15.27:0/1125262701 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55caba8bf9e0 con 0x55caba55f1d0 2024-09-21T07:27:34.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1552000640 10 client.29819 handle_command_reply: tid=0 2024-09-21T07:27:34.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 client.29819 shutdown 2024-09-21T07:27:34.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 msgr2=0x55caba95f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f690 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1560003f20 tx=0x7f1560004ab0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 shutdown_connections 2024-09-21T07:27:34.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba55f1d0 0x55caba965950 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f690 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba953c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1125262701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba953720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 >> 172.21.15.27:0/1125262701 conn(0x55caba594c40 msgr2=0x55caba94ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1552000640 0 client.29819 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 shutdown_connections 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 1 -- 172.21.15.27:0/1125262701 wait complete. 2024-09-21T07:27:34.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.593+0000 7f1575bdb740 20 client.29819 trim_cache size 0 max 16384 2024-09-21T07:27:34.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.611+0000 7f1575bdb740 1 Processor -- start 2024-09-21T07:27:34.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.611+0000 7f1575bdb740 1 -- start start 2024-09-21T07:27:34.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba94cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba94d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1575bdb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15681b9b50 con 0x55caba57fb70 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1575bdb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15681b7420 con 0x55caba5a0850 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1575bdb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15681b5290 con 0x55caba577990 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1567400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba94d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1567400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba94d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45964/0 (socket says 172.21.15.27:45964) 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1567400640 1 -- 172.21.15.27:0/1976305910 learned_addr learned my addr 172.21.15.27:0/1976305910 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1567e00640 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba94cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f156d000640 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.612+0000 7f1567400640 1 -- 172.21.15.27:0/1976305910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 msgr2=0x55caba94cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f1567400640 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba94cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f1567400640 1 -- 172.21.15.27:0/1976305910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 msgr2=0x55caba95f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f1567400640 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f1567400640 1 -- 172.21.15.27:0/1976305910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caba94ee20 con 0x55caba57fb70 2024-09-21T07:27:34.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f156d000640 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:34.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f1567400640 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba94d1e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f15540cd3d0 tx=0x7f15540cd560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd42da5feb171c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:34.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f155cc00640 1 -- 172.21.15.27:0/1976305910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554049040 con 0x55caba57fb70 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f155cc00640 1 -- 172.21.15.27:0/1976305910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f155403b460 con 0x55caba57fb70 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.613+0000 7f155cc00640 1 -- 172.21.15.27:0/1976305910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155403bb40 con 0x55caba57fb70 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 -- 172.21.15.27:0/1976305910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 msgr2=0x55caba94d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba94d1e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f15540cd3d0 tx=0x7f15540cd560 comp rx=0 tx=0).stop 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 -- 172.21.15.27:0/1976305910 shutdown_connections 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba5a0850 0x55caba95f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba57fb70 0x55caba94d1e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1976305910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba577990 0x55caba94cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 -- 172.21.15.27:0/1976305910 >> 172.21.15.27:0/1976305910 conn(0x55caba594c40 msgr2=0x55caba953720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 -- 172.21.15.27:0/1976305910 shutdown_connections 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.614+0000 7f1575bdb740 1 -- 172.21.15.27:0/1976305910 wait complete. 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 -- start start 2024-09-21T07:27:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 Processor -- start 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba577990 0x55caba958120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba9586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba5a0850 0x55caba9c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15681b58e0 con 0x55caba577990 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15681b6d50 con 0x55caba57fb70 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15681b5fb0 con 0x55caba5a0850 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.615+0000 7f1575bdb740 10 client.0 _ll_register_callbacks cb 0x7f1560009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba577990 0x55caba958120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba577990 0x55caba958120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45976/0 (socket says 172.21.15.27:45976) 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567e00640 1 -- 172.21.15.27:0/1223721645 learned_addr learned my addr 172.21.15.27:0/1223721645 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:34.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567400640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba9586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f156d000640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba5a0850 0x55caba9c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567400640 1 -- 172.21.15.27:0/1223721645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba5a0850 msgr2=0x55caba9c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567400640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba5a0850 0x55caba9c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567400640 1 -- 172.21.15.27:0/1223721645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba577990 msgr2=0x55caba958120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567400640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba577990 0x55caba958120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567400640 1 -- 172.21.15.27:0/1223721645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caba9c9ac0 con 0x55caba57fb70 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567e00640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba577990 0x55caba958120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1567400640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba9586c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1554004650 tx=0x7f1554003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7217f3313c9ad5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.616+0000 7f1552000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1552000640 1 -- 172.21.15.27:0/1223721645 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554049070 con 0x55caba57fb70 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1552000640 1 -- 172.21.15.27:0/1223721645 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f155403bb40 con 0x55caba57fb70 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55caba8bb490 con 0x55caba57fb70 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1552000640 1 -- 172.21.15.27:0/1223721645 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15540cd1e0 con 0x55caba57fb70 2024-09-21T07:27:34.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 10 client.29825 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:34.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55caba9d0e90 con 0x55caba57fb70 2024-09-21T07:27:34.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1552000640 1 -- 172.21.15.27:0/1223721645 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55caba8bb490 con 0x55caba57fb70 2024-09-21T07:27:34.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1552000640 1 -- 172.21.15.27:0/1223721645 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f155404c060 con 0x55caba57fb70 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 10 client.29825 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 10 client.29825 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 20 client.29825 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba534580 0x55caba9d3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 4 client.29825 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.617+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x55caba8b88c0 con 0x55caba534580 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.618+0000 7f1567e00640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba534580 0x55caba9d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.618+0000 7f1567e00640 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba534580 0x55caba9d3600 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc8a537548cdf59 server_cookie=4ecd91eff91b56ce in_seq=0 out_seq=0 2024-09-21T07:27:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.618+0000 7f1552000640 10 client.29825 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.630+0000 7f1552000640 1 -- 172.21.15.27:0/1223721645 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55caba8b88c0 con 0x55caba534580 2024-09-21T07:27:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.630+0000 7f1552000640 10 client.29825 handle_command_reply: tid=0 2024-09-21T07:27:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.630+0000 7f1575bdb740 1 client.29825 shutdown 2024-09-21T07:27:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.630+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 msgr2=0x55caba9586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.630+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba9586c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1554004650 tx=0x7f1554003f50 comp rx=0 tx=0).stop 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 shutdown_connections 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55caba534580 0x55caba9d3600 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55caba5a0850 0x55caba9c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caba57fb70 0x55caba9586c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 --2- 172.21.15.27:0/1223721645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55caba577990 0x55caba958120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1552000640 0 client.29825 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 >> 172.21.15.27:0/1223721645 conn(0x55caba594c40 msgr2=0x55caba953700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 shutdown_connections 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 1 -- 172.21.15.27:0/1223721645 wait complete. 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.631+0000 7f1575bdb740 20 client.29825 trim_cache size 0 max 16384 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 58, 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:27:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "206a05d": { 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 58, 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 138.8, 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:27:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f154c0b6670 msgr2=0x7f154c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f154c0b6670 0x7f154c0b8b80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1560005fd0 tx=0x7f1560009500 comp rx=0 tx=0).stop 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568077380 msgr2=0x7f15681a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568077380 0x7f15681a0e30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f1554004060 comp rx=0 tx=0).stop 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 shutdown_connections 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f154c0b6670 0x7f154c0b8b80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1568114260 0x7f15681a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1568077380 0x7f15681a0e30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 --2- 172.21.15.27:0/451875413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1568075dd0 0x7f15681a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 >> 172.21.15.27:0/451875413 conn(0x7f15680ff3e0 msgr2=0x7f1568110160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 shutdown_connections 2024-09-21T07:27:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:34.632+0000 7f156da00640 1 -- 172.21.15.27:0/451875413 wait complete. 2024-09-21T07:27:35.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:34 smithi027 ceph-mon[30779]: pgmap v832: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-21T07:27:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:34 smithi153 ceph-mon[34747]: pgmap v832: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-21T07:27:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:34 smithi119 ceph-mon[34519]: pgmap v832: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-21T07:27:35.535 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '206a05d', quiesce took 4.1 seconds. Will release after: 14.750000000000002 2024-09-21T07:27:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:36 smithi153 ceph-mon[34747]: pgmap v833: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:27:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:36 smithi119 ceph-mon[34519]: pgmap v833: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:27:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:36 smithi027 ceph-mon[30779]: pgmap v833: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:27:37.974 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:27:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.035+0000 7f6ab6800640 1 -- 172.21.15.27:0/2209327576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010c010 msgr2=0x7f6ab0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.035+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2209327576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010c010 0x7f6ab0112840 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009ca0 tx=0x7f6aa802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.035+0000 7f6ab6800640 1 -- 172.21.15.27:0/2209327576 shutdown_connections 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.035+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2209327576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ab010b6c0 0x7f6ab010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.035+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2209327576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010c010 0x7f6ab0112840 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.035+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2209327576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ab0105d80 0x7f6ab0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.036+0000 7f6ab6800640 1 -- 172.21.15.27:0/2209327576 >> 172.21.15.27:0/2209327576 conn(0x7f6ab0101810 msgr2=0x7f6ab0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.036+0000 7f6ab6800640 1 -- 172.21.15.27:0/2209327576 shutdown_connections 2024-09-21T07:27:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.036+0000 7f6ab6800640 1 -- 172.21.15.27:0/2209327576 wait complete. 2024-09-21T07:27:39.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 Processor -- start 2024-09-21T07:27:39.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 -- start start 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ab0105d80 0x7f6ab019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010b6c0 0x7f6ab019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ab010c010 0x7f6ab01a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0115cd0 con 0x7f6ab010b6c0 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0115b50 con 0x7f6ab0105d80 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.037+0000 7f6ab6800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0115e50 con 0x7f6ab010c010 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010b6c0 0x7f6ab019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ab0105d80 0x7f6ab019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010b6c0 0x7f6ab019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53982/0 (socket says 172.21.15.27:53982) 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaea00640 1 -- 172.21.15.27:0/2881490426 learned_addr learned my addr 172.21.15.27:0/2881490426 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aafe00640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ab010c010 0x7f6ab01a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaf400640 1 -- 172.21.15.27:0/2881490426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ab010c010 msgr2=0x7f6ab01a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaf400640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ab010c010 0x7f6ab01a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaf400640 1 -- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010b6c0 msgr2=0x7f6ab019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaf400640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010b6c0 0x7f6ab019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.038+0000 7f6aaf400640 1 -- 172.21.15.27:0/2881490426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab01a2650 con 0x7f6ab0105d80 2024-09-21T07:27:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.039+0000 7f6aaf400640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ab0105d80 0x7f6ab019c6e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000e9f0 tx=0x7f6aa000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.039+0000 7f6aa7e00640 1 -- 172.21.15.27:0/2881490426 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000ccc0 con 0x7f6ab0105d80 2024-09-21T07:27:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.039+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab01a2940 con 0x7f6ab0105d80 2024-09-21T07:27:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.039+0000 7f6aa7e00640 1 -- 172.21.15.27:0/2881490426 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa00108e0 con 0x7f6ab0105d80 2024-09-21T07:27:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.039+0000 7f6aa7e00640 1 -- 172.21.15.27:0/2881490426 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0018880 con 0x7f6ab0105d80 2024-09-21T07:27:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.039+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01a90a0 con 0x7f6ab0105d80 2024-09-21T07:27:39.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.043+0000 7f6abd45a740 1 Processor -- start 2024-09-21T07:27:39.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.043+0000 7f6abd45a740 1 -- start start 2024-09-21T07:27:39.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c00b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e46839390 0x556e46c015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6abd45a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556e46b77f70 con 0x556e4685a070 2024-09-21T07:27:39.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6abd45a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556e46b673a0 con 0x556e468311b0 2024-09-21T07:27:39.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6abd45a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e46b6b1b0 con 0x556e46839390 2024-09-21T07:27:39.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c00b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c00b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53988/0 (socket says 172.21.15.27:53988) 2024-09-21T07:27:39.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 -- 172.21.15.27:0/482861016 learned_addr learned my addr 172.21.15.27:0/482861016 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:39.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aaea00640 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e46839390 0x556e46c015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 -- 172.21.15.27:0/482861016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e46839390 msgr2=0x556e46c015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e46839390 0x556e46c015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 -- 172.21.15.27:0/482861016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 msgr2=0x556e46c010a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:27:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c010a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.044+0000 7f6aafe00640 1 -- 172.21.15.27:0/482861016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e46b31c90 con 0x556e4685a070 2024-09-21T07:27:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.045+0000 7f6aa7e00640 1 -- 172.21.15.27:0/2881490426 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6aa0010430 con 0x7f6ab0105d80 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.045+0000 7f6aafe00640 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c00b60 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6aa802f4c0 tx=0x7f6aa8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405d53505c9d4dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.045+0000 7f6aa4c00640 1 -- 172.21.15.27:0/482861016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8031cf0 con 0x556e4685a070 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.045+0000 7f6aa4c00640 1 -- 172.21.15.27:0/482861016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa8036030 con 0x556e4685a070 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.045+0000 7f6aa4c00640 1 -- 172.21.15.27:0/482861016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa80042a0 con 0x556e4685a070 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 -- 172.21.15.27:0/482861016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 msgr2=0x556e46c00b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c00b60 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6aa802f4c0 tx=0x7f6aa8009800 comp rx=0 tx=0).stop 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 -- 172.21.15.27:0/482861016 shutdown_connections 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e46839390 0x556e46c015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 --2- 172.21.15.27:0/482861016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c00b60 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 -- 172.21.15.27:0/482861016 >> 172.21.15.27:0/482861016 conn(0x556e4684e460 msgr2=0x556e46bfea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 -- 172.21.15.27:0/482861016 shutdown_connections 2024-09-21T07:27:39.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.046+0000 7f6abd45a740 1 -- 172.21.15.27:0/482861016 wait complete. 2024-09-21T07:27:39.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.047+0000 7f6aa7e00640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a940b6590 0x7f6a940b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.047+0000 7f6aa7e00640 1 -- 172.21.15.27:0/2881490426 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f6aa00e7e60 con 0x7f6ab0105d80 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.047+0000 7f6abd45a740 1 -- start start 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.047+0000 7f6aafe00640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a940b6590 0x7f6a940b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.047+0000 7f6abd45a740 1 Processor -- start 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6aafe00640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a940b6590 0x7f6a940b8aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009ca0 tx=0x7f6aa802a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e468311b0 0x556e46c0c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c0cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c18450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6abd45a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556e469c8700 con 0x556e468311b0 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6abd45a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556e46b02540 con 0x556e46839390 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6abd45a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e46c0f160 con 0x556e4685a070 2024-09-21T07:27:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6abd45a740 10 client.0 _ll_register_callbacks cb 0x556e46b12430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.048+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e468311b0 0x556e46c0c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c0cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c0cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60796/0 (socket says 172.21.15.27:60796) 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 -- 172.21.15.27:0/506851810 learned_addr learned my addr 172.21.15.27:0/506851810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aafe00640 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c18450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 -- 172.21.15.27:0/506851810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 msgr2=0x556e46c18450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c18450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 -- 172.21.15.27:0/506851810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e468311b0 msgr2=0x556e46c0c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e468311b0 0x556e46c0c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 -- 172.21.15.27:0/506851810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e468c5650 con 0x556e46839390 2024-09-21T07:27:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6aaea00640 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c0cce0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00bb70 tx=0x7f6a9c0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e27ca58b40d8051 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.049+0000 7f6a99600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6a99600640 1 -- 172.21.15.27:0/506851810 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c011440 con 0x556e46839390 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e46b04b50 con 0x556e46839390 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6a99600640 1 -- 172.21.15.27:0/506851810 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f6a9c0115e0 con 0x556e46839390 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6a99600640 1 -- 172.21.15.27:0/506851810 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c0117e0 con 0x556e46839390 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6abd45a740 10 client.29834 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e46b1f910 con 0x556e46839390 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6a99600640 1 -- 172.21.15.27:0/506851810 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556e46b04b50 con 0x556e46839390 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6a99600640 1 -- 172.21.15.27:0/506851810 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a9c01bc00 con 0x556e46839390 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6abd45a740 10 client.29834 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.050+0000 7f6abd45a740 10 client.29834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.051+0000 7f6abd45a740 20 client.29834 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.051+0000 7f6abd45a740 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e468189f0 0x556e46c1e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.051+0000 7f6abd45a740 4 client.29834 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:39.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.051+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e46b79ec0 con 0x556e468189f0 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.051+0000 7f6aaf400640 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e468189f0 0x556e46c1e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.051+0000 7f6aaf400640 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e468189f0 0x556e46c1e5b0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6b9d90c6be7973 server_cookie=f9d88fe10a3d8c74 in_seq=0 out_seq=0 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.051+0000 7f6a99600640 10 client.29834 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.052+0000 7f6a99600640 1 -- 172.21.15.27:0/506851810 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e46b79ec0 con 0x556e468189f0 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.052+0000 7f6a99600640 10 client.29834 handle_command_reply: tid=0 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.052+0000 7f6abd45a740 1 client.29834 shutdown 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 msgr2=0x556e46c0cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c0cce0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00bb70 tx=0x7f6a9c0039c0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 shutdown_connections 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e468189f0 0x556e46c1e5b0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c18450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6a99600640 0 client.29834 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c0cce0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 --2- 172.21.15.27:0/506851810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e468311b0 0x556e46c0c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 >> 172.21.15.27:0/506851810 conn(0x556e4684e460 msgr2=0x556e46c03d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 shutdown_connections 2024-09-21T07:27:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 1 -- 172.21.15.27:0/506851810 wait complete. 2024-09-21T07:27:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.053+0000 7f6abd45a740 20 client.29834 trim_cache size 0 max 16384 2024-09-21T07:27:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.072+0000 7f6abd45a740 1 Processor -- start 2024-09-21T07:27:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.072+0000 7f6abd45a740 1 -- start start 2024-09-21T07:27:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c0e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e46839390 0x556e46c05de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c06320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6abd45a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01b59a0 con 0x556e46839390 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6abd45a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01b3270 con 0x556e468311b0 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6abd45a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01b10e0 con 0x556e4685a070 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e46839390 0x556e46c05de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c0e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c06320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c0e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60798/0 (socket says 172.21.15.27:60798) 2024-09-21T07:27:39.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.073+0000 7f6aaf400640 1 -- 172.21.15.27:0/1908097678 learned_addr learned my addr 172.21.15.27:0/1908097678 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6aaf400640 1 -- 172.21.15.27:0/1908097678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 msgr2=0x556e46c06320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6aaf400640 1 --2- 172.21.15.27:0/1908097678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c06320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6aaf400640 1 -- 172.21.15.27:0/1908097678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e46839390 msgr2=0x556e46c05de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6aaf400640 1 --2- 172.21.15.27:0/1908097678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e46839390 0x556e46c05de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6aaf400640 1 -- 172.21.15.27:0/1908097678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e46c039c0 con 0x556e468311b0 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6aaf400640 1 --2- 172.21.15.27:0/1908097678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c0e1d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00e5200 tx=0x7f6aa00d1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1d382e14336e07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6a9be00640 1 -- 172.21.15.27:0/1908097678 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00ad460 con 0x556e468311b0 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6a9be00640 1 -- 172.21.15.27:0/1908097678 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa00b2380 con 0x556e468311b0 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.074+0000 7f6a9be00640 1 -- 172.21.15.27:0/1908097678 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00e5420 con 0x556e468311b0 2024-09-21T07:27:39.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.075+0000 7f6abd45a740 1 -- 172.21.15.27:0/1908097678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 msgr2=0x556e46c0e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.075+0000 7f6abd45a740 1 --2- 172.21.15.27:0/1908097678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c0e1d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00e5200 tx=0x7f6aa00d1040 comp rx=0 tx=0).stop 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.075+0000 7f6abd45a740 1 -- 172.21.15.27:0/1908097678 shutdown_connections 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.075+0000 7f6abd45a740 1 --2- 172.21.15.27:0/1908097678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e4685a070 0x556e46c06320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.075+0000 7f6abd45a740 1 --2- 172.21.15.27:0/1908097678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e46839390 0x556e46c05de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.075+0000 7f6abd45a740 1 --2- 172.21.15.27:0/1908097678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e468311b0 0x556e46c0e1d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.075+0000 7f6abd45a740 1 -- 172.21.15.27:0/1908097678 >> 172.21.15.27:0/1908097678 conn(0x556e4684e460 msgr2=0x556e46c0dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.076+0000 7f6abd45a740 1 -- 172.21.15.27:0/1908097678 shutdown_connections 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.076+0000 7f6abd45a740 1 -- 172.21.15.27:0/1908097678 wait complete. 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.076+0000 7f6abd45a740 1 -- start start 2024-09-21T07:27:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.076+0000 7f6abd45a740 1 Processor -- start 2024-09-21T07:27:39.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e468311b0 0x556e46c14720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c14ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6abd45a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c0f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6abd45a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01b1730 con 0x556e4685a070 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6abd45a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01b2ba0 con 0x556e46839390 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6abd45a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01b1e00 con 0x556e468311b0 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6abd45a740 10 client.0 _ll_register_callbacks cb 0x7f6a9c0043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c14ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e468311b0 0x556e46c14720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c0f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c14ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60802/0 (socket says 172.21.15.27:60802) 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.077+0000 7f6aaea00640 1 -- 172.21.15.27:0/47959983 learned_addr learned my addr 172.21.15.27:0/47959983 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6aaea00640 1 -- 172.21.15.27:0/47959983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e468311b0 msgr2=0x556e46c14720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6aaea00640 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e468311b0 0x556e46c14720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6aaea00640 1 -- 172.21.15.27:0/47959983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 msgr2=0x556e46c0f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6aaea00640 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c0f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6aaea00640 1 -- 172.21.15.27:0/47959983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e46c0fbc0 con 0x556e46839390 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6aaf400640 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e468311b0 0x556e46c14720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6aaea00640 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c14ce0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005d80 tx=0x7f6a9c011b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c5aefa5fb02070 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6a99600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6a99600640 1 -- 172.21.15.27:0/47959983 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c005040 con 0x556e46839390 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6a99600640 1 -- 172.21.15.27:0/47959983 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f6a9c0051e0 con 0x556e46839390 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.078+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa00b2380 con 0x556e46839390 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6a99600640 1 -- 172.21.15.27:0/47959983 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c0053e0 con 0x556e46839390 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 10 client.29840 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e46c89a80 con 0x556e46839390 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6a99600640 1 -- 172.21.15.27:0/47959983 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa00b2380 con 0x556e46839390 2024-09-21T07:27:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6a99600640 1 -- 172.21.15.27:0/47959983 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a9c004670 con 0x556e46839390 2024-09-21T07:27:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 10 client.29840 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 10 client.29840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 20 client.29840 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e467edda0 0x556e46c8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 4 client.29840 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:27:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.079+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e46b75970 con 0x556e467edda0 2024-09-21T07:27:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.080+0000 7f6aaf400640 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e467edda0 0x556e46c8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.080+0000 7f6aaf400640 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e467edda0 0x556e46c8c240 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e269cfcd9bd78e server_cookie=53cf494e4c6fb8c2 in_seq=0 out_seq=0 2024-09-21T07:27:39.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.080+0000 7f6a99600640 10 client.29840 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:39.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6a99600640 1 -- 172.21.15.27:0/47959983 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6622 (crc 0 0 0) 0x556e46b75970 con 0x556e467edda0 2024-09-21T07:27:39.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6a99600640 10 client.29840 handle_command_reply: tid=0 2024-09-21T07:27:39.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 client.29840 shutdown 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 msgr2=0x556e46c14ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c14ce0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005d80 tx=0x7f6a9c011b90 comp rx=0 tx=0).stop 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 shutdown_connections 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x556e467edda0 0x556e46c8c240 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556e4685a070 0x556e46c0f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e46839390 0x556e46c14ce0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6a99600640 0 client.29840 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 --2- 172.21.15.27:0/47959983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e468311b0 0x556e46c14720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 >> 172.21.15.27:0/47959983 conn(0x556e4684e460 msgr2=0x556e46c01b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 shutdown_connections 2024-09-21T07:27:39.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 1 -- 172.21.15.27:0/47959983 wait complete. 2024-09-21T07:27:39.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.081+0000 7f6abd45a740 20 client.29840 trim_cache size 0 max 16384 2024-09-21T07:27:39.087 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:39.087 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:27:39.087 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:27:39.087 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:27:39.087 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:27:39.087 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.088 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.089 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:27:39.091 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.094 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:27:39.096 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:27:39.099 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:27:39.100 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:39.102 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a940b6590 msgr2=0x7f6a940b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a940b6590 0x7f6a940b8aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009ca0 tx=0x7f6aa802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ab0105d80 msgr2=0x7f6ab019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ab0105d80 0x7f6ab019c6e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000e9f0 tx=0x7f6aa000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 shutdown_connections 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ab010c010 0x7f6ab01a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6a940b6590 0x7f6a940b8aa0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ab010b6c0 0x7f6ab019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 --2- 172.21.15.27:0/2881490426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ab0105d80 0x7f6ab019c6e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 >> 172.21.15.27:0/2881490426 conn(0x7f6ab0101810 msgr2=0x7f6ab0108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 shutdown_connections 2024-09-21T07:27:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:39.082+0000 7f6ab6800640 1 -- 172.21.15.27:0/2881490426 wait complete. 2024-09-21T07:27:39.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:38 smithi027 ceph-mon[30779]: pgmap v834: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-21T07:27:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:38 smithi153 ceph-mon[34747]: pgmap v834: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-21T07:27:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:38 smithi119 ceph-mon[34519]: pgmap v834: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-21T07:27:40.081 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:27:40.081 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:27:40.464 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.102+0000 7f6c19200640 1 -- 172.21.15.27:0/1287236203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14078290 msgr2=0x7f6c14114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.102+0000 7f6c19200640 1 --2- 172.21.15.27:0/1287236203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14078290 0x7f6c14114030 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009900 tx=0x7f6bfc02a400 comp rx=0 tx=0).stop 2024-09-21T07:27:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.103+0000 7f6c19200640 1 -- 172.21.15.27:0/1287236203 shutdown_connections 2024-09-21T07:27:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.103+0000 7f6c19200640 1 --2- 172.21.15.27:0/1287236203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c14114570 0x7f6c14116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.103+0000 7f6c19200640 1 --2- 172.21.15.27:0/1287236203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14078290 0x7f6c14114030 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.103+0000 7f6c19200640 1 --2- 172.21.15.27:0/1287236203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c14077940 0x7f6c14077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.103+0000 7f6c19200640 1 -- 172.21.15.27:0/1287236203 >> 172.21.15.27:0/1287236203 conn(0x7f6c14101680 msgr2=0x7f6c14103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.103+0000 7f6c19200640 1 -- 172.21.15.27:0/1287236203 shutdown_connections 2024-09-21T07:27:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.103+0000 7f6c19200640 1 -- 172.21.15.27:0/1287236203 wait complete. 2024-09-21T07:27:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 Processor -- start 2024-09-21T07:27:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 -- start start 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14077940 0x7f6c141a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c14078290 0x7f6c141a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c14114570 0x7f6c141a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14119ed0 con 0x7f6c14077940 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14119d50 con 0x7f6c14114570 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.104+0000 7f6c19200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1411a050 con 0x7f6c14078290 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14077940 0x7f6c141a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c14114570 0x7f6c141a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14077940 0x7f6c141a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54040/0 (socket says 172.21.15.27:54040) 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13400640 1 -- 172.21.15.27:0/2953421951 learned_addr learned my addr 172.21.15.27:0/2953421951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c0aa00640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c14078290 0x7f6c141a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13e00640 1 -- 172.21.15.27:0/2953421951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c14078290 msgr2=0x7f6c141a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13e00640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c14078290 0x7f6c141a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13e00640 1 -- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14077940 msgr2=0x7f6c141a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13e00640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14077940 0x7f6c141a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13e00640 1 -- 172.21.15.27:0/2953421951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c141a6890 con 0x7f6c14114570 2024-09-21T07:27:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.105+0000 7f6c13400640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14077940 0x7f6c141a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.106+0000 7f6c13e00640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c14114570 0x7f6c141a6160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d680 tx=0x7f6c0c00db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.106+0000 7f6c10c00640 1 -- 172.21.15.27:0/2953421951 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c017070 con 0x7f6c14114570 2024-09-21T07:27:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.106+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c141a6b80 con 0x7f6c14114570 2024-09-21T07:27:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.106+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c141ad2e0 con 0x7f6c14114570 2024-09-21T07:27:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.106+0000 7f6c10c00640 1 -- 172.21.15.27:0/2953421951 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0c009820 con 0x7f6c14114570 2024-09-21T07:27:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.106+0000 7f6c10c00640 1 -- 172.21.15.27:0/2953421951 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c020060 con 0x7f6c14114570 2024-09-21T07:27:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.107+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c14077d50 con 0x7f6c14114570 2024-09-21T07:27:41.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.110+0000 7f6c10c00640 1 -- 172.21.15.27:0/2953421951 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6c0c009c90 con 0x7f6c14114570 2024-09-21T07:27:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.111+0000 7f6c10c00640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6bf80b6680 0x7f6bf80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.112+0000 7f6c10c00640 1 -- 172.21.15.27:0/2953421951 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f6c0c0e7270 con 0x7f6c14114570 2024-09-21T07:27:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.112+0000 7f6c13400640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6bf80b6680 0x7f6bf80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.112+0000 7f6c13400640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6bf80b6680 0x7f6bf80b8b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400f7d0 tx=0x7f6c04009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:41.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.113+0000 7f6c10c00640 1 -- 172.21.15.27:0/2953421951 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6c0c0b1790 con 0x7f6c14114570 2024-09-21T07:27:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:40 smithi153 ceph-mon[34747]: pgmap v835: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:41.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:40 smithi027 ceph-mon[30779]: pgmap v835: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:40 smithi119 ceph-mon[34519]: pgmap v835: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.359+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c1410f730 con 0x7f6c14114570 2024-09-21T07:27:41.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.360+0000 7f6c10c00640 1 -- 172.21.15.27:0/2953421951 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f6c0c0b66d0 con 0x7f6c14114570 2024-09-21T07:27:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:27:41.363 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:27:41.363 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6bf80b6680 msgr2=0x7f6bf80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6bf80b6680 0x7f6bf80b8b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400f7d0 tx=0x7f6c04009500 comp rx=0 tx=0).stop 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c14114570 msgr2=0x7f6c141a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c14114570 0x7f6c141a6160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d680 tx=0x7f6c0c00db40 comp rx=0 tx=0).stop 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 shutdown_connections 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c14078290 0x7f6c141a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6bf80b6680 0x7f6bf80b8b90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c14114570 0x7f6c141a6160 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 --2- 172.21.15.27:0/2953421951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c14077940 0x7f6c141a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 >> 172.21.15.27:0/2953421951 conn(0x7f6c14101680 msgr2=0x7f6c14116660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 shutdown_connections 2024-09-21T07:27:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:41.363+0000 7f6c19200640 1 -- 172.21.15.27:0/2953421951 wait complete. 2024-09-21T07:27:42.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:27:42.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2953421951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2953421951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:42.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2953421951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:43.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 -- 172.21.15.27:0/98532606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4584076100 msgr2=0x7f458410f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 --2- 172.21.15.27:0/98532606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4584076100 0x7f458410f9b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4570009900 tx=0x7f457002a410 comp rx=0 tx=0).stop 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 -- 172.21.15.27:0/98532606 shutdown_connections 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 --2- 172.21.15.27:0/98532606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f458410fef0 0x7f4584112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 --2- 172.21.15.27:0/98532606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4584076100 0x7f458410f9b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 --2- 172.21.15.27:0/98532606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45840757b0 0x7f4584075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 -- 172.21.15.27:0/98532606 >> 172.21.15.27:0/98532606 conn(0x7f45840ff3e0 msgr2=0x7f4584101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 -- 172.21.15.27:0/98532606 shutdown_connections 2024-09-21T07:27:43.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.048+0000 7f458aa00640 1 -- 172.21.15.27:0/98532606 wait complete. 2024-09-21T07:27:43.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.049+0000 7f458aa00640 1 Processor -- start 2024-09-21T07:27:43.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.049+0000 7f458aa00640 1 -- start start 2024-09-21T07:27:43.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f458aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45840757b0 0x7f458419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f458aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4584076100 0x7f458419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f458aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f458410fef0 0x7f45841a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f458aa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4584115af0 con 0x7f45840757b0 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f458aa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4584115970 con 0x7f4584076100 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f458aa00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4584115c70 con 0x7f458410fef0 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f4589600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45840757b0 0x7f458419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f4589600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45840757b0 0x7f458419c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54062/0 (socket says 172.21.15.27:54062) 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f4589600640 1 -- 172.21.15.27:0/1986146976 learned_addr learned my addr 172.21.15.27:0/1986146976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f4588c00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4584076100 0x7f458419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.050+0000 7f458a000640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f458410fef0 0x7f45841a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4589600640 1 -- 172.21.15.27:0/1986146976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f458410fef0 msgr2=0x7f45841a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4589600640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f458410fef0 0x7f45841a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4589600640 1 -- 172.21.15.27:0/1986146976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4584076100 msgr2=0x7f458419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4589600640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4584076100 0x7f458419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4589600640 1 -- 172.21.15.27:0/1986146976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45841a2510 con 0x7f45840757b0 2024-09-21T07:27:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4588c00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4584076100 0x7f458419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4589600640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45840757b0 0x7f458419c5a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f457800e9f0 tx=0x7f457800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4582000640 1 -- 172.21.15.27:0/1986146976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457800cd80 con 0x7f45840757b0 2024-09-21T07:27:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.051+0000 7f4582000640 1 -- 172.21.15.27:0/1986146976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45780108e0 con 0x7f45840757b0 2024-09-21T07:27:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.052+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45841a2800 con 0x7f45840757b0 2024-09-21T07:27:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.052+0000 7f4582000640 1 -- 172.21.15.27:0/1986146976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578018980 con 0x7f45840757b0 2024-09-21T07:27:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.052+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45841a8f60 con 0x7f45840757b0 2024-09-21T07:27:43.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.057+0000 7f4592c4e740 1 Processor -- start 2024-09-21T07:27:43.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.057+0000 7f4592c4e740 1 -- start start 2024-09-21T07:27:43.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.057+0000 7f4582000640 1 -- 172.21.15.27:0/1986146976 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4578010430 con 0x7f45840757b0 2024-09-21T07:27:43.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae4fad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae12b1b0 0x55bfae4fb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae4fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4592c4e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bfae471fb0 con 0x55bfae12b1b0 2024-09-21T07:27:43.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4592c4e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bfae4613e0 con 0x55bfae154070 2024-09-21T07:27:43.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4592c4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bfae4651f0 con 0x55bfae133390 2024-09-21T07:27:43.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4589600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae12b1b0 0x55bfae4fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4589600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae12b1b0 0x55bfae4fb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54064/0 (socket says 172.21.15.27:54064) 2024-09-21T07:27:43.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4589600640 1 -- 172.21.15.27:0/1683519505 learned_addr learned my addr 172.21.15.27:0/1683519505 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:43.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f4588c00640 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae4fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.058+0000 7f458a000640 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae4fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4589600640 1 -- 172.21.15.27:0/1683519505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 msgr2=0x55bfae4fb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4589600640 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae4fb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4589600640 1 -- 172.21.15.27:0/1683519505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 msgr2=0x55bfae4fad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4589600640 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae4fad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4589600640 1 -- 172.21.15.27:0/1683519505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfae42bcd0 con 0x55bfae12b1b0 2024-09-21T07:27:43.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4589600640 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae12b1b0 0x55bfae4fb270 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f45780e53f0 tx=0x7f45780f0910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86db6f47458ca8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4576600640 1 -- 172.21.15.27:0/1683519505 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780fb070 con 0x55bfae12b1b0 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4576600640 1 -- 172.21.15.27:0/1683519505 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45780e8a10 con 0x55bfae12b1b0 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.059+0000 7f4576600640 1 -- 172.21.15.27:0/1683519505 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780f6940 con 0x55bfae12b1b0 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.060+0000 7f4582000640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f456c0b6680 0x7f456c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.060+0000 7f4582000640 1 -- 172.21.15.27:0/1986146976 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f45780e8170 con 0x7f45840757b0 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.060+0000 7f458a000640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f456c0b6680 0x7f456c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 -- 172.21.15.27:0/1683519505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae12b1b0 msgr2=0x55bfae4fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae12b1b0 0x55bfae4fb270 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f45780e53f0 tx=0x7f45780f0910 comp rx=0 tx=0).stop 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 -- 172.21.15.27:0/1683519505 shutdown_connections 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae4fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae12b1b0 0x55bfae4fb270 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1683519505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae4fad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 -- 172.21.15.27:0/1683519505 >> 172.21.15.27:0/1683519505 conn(0x55bfae148460 msgr2=0x55bfae4f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f458a000640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f456c0b6680 0x7f456c0b8b90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f457c005fd0 tx=0x7f457c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:43.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 -- 172.21.15.27:0/1683519505 shutdown_connections 2024-09-21T07:27:43.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 -- 172.21.15.27:0/1683519505 wait complete. 2024-09-21T07:27:43.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.061+0000 7f4592c4e740 1 -- start start 2024-09-21T07:27:43.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 1 Processor -- start 2024-09-21T07:27:43.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae12b1b0 0x55bfae5069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae506f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae512830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bfae2c2700 con 0x55bfae154070 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bfae3fc580 con 0x55bfae12b1b0 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bfae509510 con 0x55bfae133390 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.062+0000 7f4592c4e740 10 client.0 _ll_register_callbacks cb 0x55bfae40c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f458a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae512830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae12b1b0 0x55bfae5069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4588c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae506f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae12b1b0 0x55bfae5069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60854/0 (socket says 172.21.15.27:60854) 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 -- 172.21.15.27:0/3142876511 learned_addr learned my addr 172.21.15.27:0/3142876511 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 -- 172.21.15.27:0/3142876511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 msgr2=0x55bfae506f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae506f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 -- 172.21.15.27:0/3142876511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 msgr2=0x55bfae512830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae512830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.063+0000 7f4589600640 1 -- 172.21.15.27:0/3142876511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfae1bf650 con 0x55bfae12b1b0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f4589600640 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae12b1b0 0x55bfae5069a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f45780aa180 tx=0x7f4578000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc880a62bce390f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f456aa00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f456aa00640 1 -- 172.21.15.27:0/3142876511 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780aa200 con 0x55bfae12b1b0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bfae3feb90 con 0x55bfae12b1b0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f456aa00640 1 -- 172.21.15.27:0/3142876511 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f45780e5730 con 0x55bfae12b1b0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f456aa00640 1 -- 172.21.15.27:0/3142876511 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45780ad320 con 0x55bfae12b1b0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f4592c4e740 10 client.29858 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfae419950 con 0x55bfae12b1b0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.064+0000 7f456aa00640 1 -- 172.21.15.27:0/3142876511 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bfae3feb90 con 0x55bfae12b1b0 2024-09-21T07:27:43.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f456aa00640 1 -- 172.21.15.27:0/3142876511 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f45780b7140 con 0x55bfae12b1b0 2024-09-21T07:27:43.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f4592c4e740 10 client.29858 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:43.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f4592c4e740 10 client.29858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:27:43.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f4592c4e740 20 client.29858 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f4592c4e740 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae1129f0 0x55bfae518730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f4592c4e740 4 client.29858 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfae473f00 con 0x55bfae1129f0 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.065+0000 7f4588c00640 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae1129f0 0x55bfae518730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.066+0000 7f4588c00640 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae1129f0 0x55bfae518730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f92f7f775600824 server_cookie=8ec5a0fa668315e7 in_seq=0 out_seq=0 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.066+0000 7f456aa00640 10 client.29858 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.067+0000 7f456aa00640 1 -- 172.21.15.27:0/3142876511 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bfae473f00 con 0x55bfae1129f0 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.067+0000 7f456aa00640 10 client.29858 handle_command_reply: tid=0 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.067+0000 7f4592c4e740 1 client.29858 shutdown 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae12b1b0 msgr2=0x55bfae5069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae12b1b0 0x55bfae5069a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f45780aa180 tx=0x7f4578000bc0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 shutdown_connections 2024-09-21T07:27:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae1129f0 0x55bfae518730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae133390 0x55bfae506f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae512830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 --2- 172.21.15.27:0/3142876511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae12b1b0 0x55bfae5069a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f456aa00640 0 client.29858 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:27:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 >> 172.21.15.27:0/3142876511 conn(0x55bfae148460 msgr2=0x55bfae4fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 shutdown_connections 2024-09-21T07:27:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 1 -- 172.21.15.27:0/3142876511 wait complete. 2024-09-21T07:27:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.068+0000 7f4592c4e740 20 client.29858 trim_cache size 0 max 16384 2024-09-21T07:27:43.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.082+0000 7f4592c4e740 1 Processor -- start 2024-09-21T07:27:43.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.082+0000 7f4592c4e740 1 -- start start 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.082+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae5127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae133390 0x55bfae512ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae4ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f4592c4e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b58b0 con 0x55bfae154070 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f4592c4e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b3180 con 0x55bfae133390 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f4592c4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b0ff0 con 0x55bfae12b1b0 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae4ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae4ffe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54088/0 (socket says 172.21.15.27:54088) 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 -- 172.21.15.27:0/1435623569 learned_addr learned my addr 172.21.15.27:0/1435623569 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f4589600640 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae5127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f4588c00640 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae133390 0x55bfae512ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 -- 172.21.15.27:0/1435623569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 msgr2=0x55bfae5127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae5127a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 -- 172.21.15.27:0/1435623569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae133390 msgr2=0x55bfae512ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae133390 0x55bfae512ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.083+0000 7f458a000640 1 -- 172.21.15.27:0/1435623569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfae504ca0 con 0x55bfae154070 2024-09-21T07:27:43.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.084+0000 7f458a000640 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae4ffe40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f457c003ee0 tx=0x7f457c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933e0611cd0b59c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:43.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.084+0000 7f4575c00640 1 -- 172.21.15.27:0/1435623569 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c004b20 con 0x55bfae154070 2024-09-21T07:27:43.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.084+0000 7f4575c00640 1 -- 172.21.15.27:0/1435623569 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f457c004cc0 con 0x55bfae154070 2024-09-21T07:27:43.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.084+0000 7f4575c00640 1 -- 172.21.15.27:0/1435623569 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c0053c0 con 0x55bfae154070 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 -- 172.21.15.27:0/1435623569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 msgr2=0x55bfae4ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae4ffe40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f457c003ee0 tx=0x7f457c01a000 comp rx=0 tx=0).stop 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 -- 172.21.15.27:0/1435623569 shutdown_connections 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae154070 0x55bfae4ffe40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae133390 0x55bfae512ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 --2- 172.21.15.27:0/1435623569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae5127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 -- 172.21.15.27:0/1435623569 >> 172.21.15.27:0/1435623569 conn(0x55bfae148460 msgr2=0x55bfae505020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 -- 172.21.15.27:0/1435623569 shutdown_connections 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.085+0000 7f4592c4e740 1 -- 172.21.15.27:0/1435623569 wait complete. 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.086+0000 7f4592c4e740 1 -- start start 2024-09-21T07:27:43.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.086+0000 7f4592c4e740 1 Processor -- start 2024-09-21T07:27:43.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.086+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae51c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.086+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae133390 0x55bfae51a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4592c4e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae51ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4592c4e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b1640 con 0x55bfae133390 2024-09-21T07:27:43.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4592c4e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b2ab0 con 0x55bfae154070 2024-09-21T07:27:43.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4592c4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b1d10 con 0x55bfae12b1b0 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4592c4e740 10 client.0 _ll_register_callbacks cb 0x7f4578005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae51c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4588c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae133390 0x55bfae51a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae51c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35366/0 (socket says 172.21.15.27:35366) 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 -- 172.21.15.27:0/2923127639 learned_addr learned my addr 172.21.15.27:0/2923127639 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f458a000640 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae51ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 -- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 msgr2=0x55bfae51ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae51ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 -- 172.21.15.27:0/2923127639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae133390 msgr2=0x55bfae51a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae133390 0x55bfae51a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4589600640 1 -- 172.21.15.27:0/2923127639 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfae582ec0 con 0x55bfae12b1b0 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.087+0000 7f4588c00640 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae133390 0x55bfae51a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f4589600640 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae51c160 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f45780b7540 tx=0x7f4578009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf14396cb53e83c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f456aa00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:43.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f456aa00640 1 -- 172.21.15.27:0/2923127639 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780ad500 con 0x55bfae12b1b0 2024-09-21T07:27:43.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457c004cc0 con 0x55bfae12b1b0 2024-09-21T07:27:43.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f456aa00640 1 -- 172.21.15.27:0/2923127639 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f45780b7140 con 0x55bfae12b1b0 2024-09-21T07:27:43.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f456aa00640 1 -- 172.21.15.27:0/2923127639 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45780ad980 con 0x55bfae12b1b0 2024-09-21T07:27:43.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f4592c4e740 10 client.29818 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:43.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.088+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfae584010 con 0x55bfae12b1b0 2024-09-21T07:27:43.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f456aa00640 1 -- 172.21.15.27:0/2923127639 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f457c004cc0 con 0x55bfae12b1b0 2024-09-21T07:27:43.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f456aa00640 1 -- 172.21.15.27:0/2923127639 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4578096070 con 0x55bfae12b1b0 2024-09-21T07:27:43.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f4592c4e740 10 client.29818 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:43.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f4592c4e740 10 client.29818 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:27:43.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f4592c4e740 20 client.29818 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:43.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f4592c4e740 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae0e7da0 0x55bfae5865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f4592c4e740 4 client.29818 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfae46f9b0 con 0x55bfae0e7da0 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.089+0000 7f4588c00640 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae0e7da0 0x55bfae5865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4588c00640 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae0e7da0 0x55bfae5865e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b3026b71fbb857f server_cookie=5b1fc4d798ead282 in_seq=0 out_seq=0 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f456aa00640 10 client.29818 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f456aa00640 1 -- 172.21.15.27:0/2923127639 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfae46f9b0 con 0x55bfae0e7da0 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f456aa00640 10 client.29818 handle_command_reply: tid=0 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 client.29818 shutdown 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 msgr2=0x55bfae51c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae51c160 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f45780b7540 tx=0x7f4578009700 comp rx=0 tx=0).stop 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 shutdown_connections 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55bfae0e7da0 0x55bfae5865e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bfae154070 0x55bfae51ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bfae133390 0x55bfae51a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 --2- 172.21.15.27:0/2923127639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bfae12b1b0 0x55bfae51c160 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f456aa00640 0 client.29818 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:27:43.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.090+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 >> 172.21.15.27:0/2923127639 conn(0x55bfae148460 msgr2=0x55bfae4ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:43.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.091+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 shutdown_connections 2024-09-21T07:27:43.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.091+0000 7f4592c4e740 1 -- 172.21.15.27:0/2923127639 wait complete. 2024-09-21T07:27:43.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.091+0000 7f4592c4e740 20 client.29818 trim_cache size 0 max 16384 2024-09-21T07:27:43.096 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.091+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f456c0b6680 msgr2=0x7f456c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.091+0000 7f458aa00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f456c0b6680 0x7f456c0b8b90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f457c005fd0 tx=0x7f457c009500 comp rx=0 tx=0).stop 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45840757b0 msgr2=0x7f458419c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45840757b0 0x7f458419c5a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f457800e9f0 tx=0x7f457800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 shutdown_connections 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f458410fef0 0x7f45841a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f456c0b6680 0x7f456c0b8b90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4584076100 0x7f458419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 --2- 172.21.15.27:0/1986146976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45840757b0 0x7f458419c5a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 >> 172.21.15.27:0/1986146976 conn(0x7f45840ff3e0 msgr2=0x7f4584109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:43.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 shutdown_connections 2024-09-21T07:27:43.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:43.092+0000 7f458aa00640 1 -- 172.21.15.27:0/1986146976 wait complete. 2024-09-21T07:27:43.116 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:42 smithi027 ceph-mon[30779]: pgmap v836: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:43.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:42 smithi153 ceph-mon[34747]: pgmap v836: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:42 smithi119 ceph-mon[34519]: pgmap v836: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:43.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:27:44.381 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.984+0000 7fcc9d600640 1 -- 172.21.15.27:0/1472366472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 msgr2=0x7fcc9810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.984+0000 7fcc9d600640 1 --2- 172.21.15.27:0/1472366472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 0x7fcc9810bae0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009900 tx=0x7fcc8002a410 comp rx=0 tx=0).stop 2024-09-21T07:27:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.985+0000 7fcc9d600640 1 -- 172.21.15.27:0/1472366472 shutdown_connections 2024-09-21T07:27:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.985+0000 7fcc9d600640 1 --2- 172.21.15.27:0/1472366472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 0x7fcc98112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.985+0000 7fcc9d600640 1 --2- 172.21.15.27:0/1472366472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 0x7fcc9810bae0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.985+0000 7fcc9d600640 1 --2- 172.21.15.27:0/1472366472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc98105d90 0x7fcc981061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.985+0000 7fcc9d600640 1 -- 172.21.15.27:0/1472366472 >> 172.21.15.27:0/1472366472 conn(0x7fcc98101860 msgr2=0x7fcc98103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:44.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.985+0000 7fcc9d600640 1 -- 172.21.15.27:0/1472366472 shutdown_connections 2024-09-21T07:27:44.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.985+0000 7fcc9d600640 1 -- 172.21.15.27:0/1472366472 wait complete. 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 Processor -- start 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 -- start start 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc98105d90 0x7fcc9819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 0x7fcc9819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 0x7fcc981a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98115ce0 con 0x7fcc9810b6d0 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98115b60 con 0x7fcc9810c020 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.986+0000 7fcc9d600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98115e60 con 0x7fcc98105d90 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc8e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 0x7fcc9819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 0x7fcc981a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:44.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc98105d90 0x7fcc9819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 0x7fcc981a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60898/0 (socket says 172.21.15.27:60898) 2024-09-21T07:27:44.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 -- 172.21.15.27:0/4146989437 learned_addr learned my addr 172.21.15.27:0/4146989437 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 -- 172.21.15.27:0/4146989437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc98105d90 msgr2=0x7fcc9819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc98105d90 0x7fcc9819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 -- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 msgr2=0x7fcc9819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 0x7fcc9819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc97400640 1 -- 172.21.15.27:0/4146989437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc981a2660 con 0x7fcc9810c020 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc96a00640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc98105d90 0x7fcc9819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.987+0000 7fcc8e000640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 0x7fcc9819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.988+0000 7fcc97400640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 0x7fcc981a1f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e970 tx=0x7fcc9000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.988+0000 7fcc8fe00640 1 -- 172.21.15.27:0/4146989437 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000ccc0 con 0x7fcc9810c020 2024-09-21T07:27:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.988+0000 7fcc8fe00640 1 -- 172.21.15.27:0/4146989437 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc9000ce60 con 0x7fcc9810c020 2024-09-21T07:27:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.988+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc981a2950 con 0x7fcc9810c020 2024-09-21T07:27:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.988+0000 7fcc8fe00640 1 -- 172.21.15.27:0/4146989437 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90010430 con 0x7fcc9810c020 2024-09-21T07:27:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.988+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc981a90b0 con 0x7fcc9810c020 2024-09-21T07:27:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.991+0000 7fcca42a5740 1 Processor -- start 2024-09-21T07:27:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcc8fe00640 1 -- 172.21.15.27:0/4146989437 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcc90004ff0 con 0x7fcc9810c020 2024-09-21T07:27:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcca42a5740 1 -- start start 2024-09-21T07:27:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcca42a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a854948fb0 con 0x55a8546021b0 2024-09-21T07:27:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcca42a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a8549383e0 con 0x55a85462b070 2024-09-21T07:27:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.992+0000 7fcca42a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a85493c1f0 con 0x55a85460a390 2024-09-21T07:27:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc8e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54112/0 (socket says 172.21.15.27:54112) 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 -- 172.21.15.27:0/1157729378 learned_addr learned my addr 172.21.15.27:0/1157729378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc8e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60912/0 (socket says 172.21.15.27:60912) 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc8e000640 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 -- 172.21.15.27:0/1157729378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 msgr2=0x55a8549d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 -- 172.21.15.27:0/1157729378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 msgr2=0x55a8549d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.993+0000 7fcc96a00640 1 -- 172.21.15.27:0/1157729378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a854902cd0 con 0x55a8546021b0 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.994+0000 7fcc96a00640 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549d22e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcc88011860 tx=0x7fcc88011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f472c61164a316d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:44.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.994+0000 7fcc8fe00640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcc7c0b6700 0x7fcc7c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.994+0000 7fcc8fe00640 1 -- 172.21.15.27:0/4146989437 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fcc900e7f00 con 0x7fcc9810c020 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.994+0000 7fcc87400640 1 -- 172.21.15.27:0/1157729378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc880112f0 con 0x55a8546021b0 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.994+0000 7fcc87400640 1 -- 172.21.15.27:0/1157729378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc88003e20 con 0x55a8546021b0 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.994+0000 7fcc87400640 1 -- 172.21.15.27:0/1157729378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88004450 con 0x55a8546021b0 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.994+0000 7fcc8e000640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcc7c0b6700 0x7fcc7c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 -- 172.21.15.27:0/1157729378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 msgr2=0x55a8549d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcc8e000640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcc7c0b6700 0x7fcc7c0b8c10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009230 tx=0x7fcc80000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549d22e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcc88011860 tx=0x7fcc88011d20 comp rx=0 tx=0).stop 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 -- 172.21.15.27:0/1157729378 shutdown_connections 2024-09-21T07:27:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549d22e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 --2- 172.21.15.27:0/1157729378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 -- 172.21.15.27:0/1157729378 >> 172.21.15.27:0/1157729378 conn(0x55a85461f460 msgr2=0x55a8549cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 -- 172.21.15.27:0/1157729378 shutdown_connections 2024-09-21T07:27:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.995+0000 7fcca42a5740 1 -- 172.21.15.27:0/1157729378 wait complete. 2024-09-21T07:27:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.996+0000 7fcca42a5740 1 -- start start 2024-09-21T07:27:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.996+0000 7fcca42a5740 1 Processor -- start 2024-09-21T07:27:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549dd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549dda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcca42a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a854799700 con 0x55a8546021b0 2024-09-21T07:27:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcca42a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a8548d3580 con 0x55a85462b070 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcca42a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8549e0580 con 0x55a85460a390 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcca42a5740 10 client.0 _ll_register_callbacks cb 0x55a8548e3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549dd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549dd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54122/0 (socket says 172.21.15.27:54122) 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcc96a00640 1 -- 172.21.15.27:0/2506402832 learned_addr learned my addr 172.21.15.27:0/2506402832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcc97400640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.997+0000 7fcc8e000640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549dda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc97400640 1 -- 172.21.15.27:0/2506402832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 msgr2=0x55a8549dda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc97400640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549dda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc97400640 1 -- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 msgr2=0x55a8549dd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc97400640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549dd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc97400640 1 -- 172.21.15.27:0/2506402832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a854696650 con 0x55a85462b070 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc96a00640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549dd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc97400640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549e9870 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcc900e5240 tx=0x7fcc900ad0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d82a3eb81de014 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc7be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc7be00640 1 -- 172.21.15.27:0/2506402832 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc900acda0 con 0x55a85462b070 2024-09-21T07:27:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc7be00640 1 -- 172.21.15.27:0/2506402832 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fcc900a9290 con 0x55a85462b070 2024-09-21T07:27:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8548d5b90 con 0x55a85462b070 2024-09-21T07:27:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcc7be00640 1 -- 172.21.15.27:0/2506402832 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc900ad450 con 0x55a85462b070 2024-09-21T07:27:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcca42a5740 10 client.29876 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.998+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a8548f0950 con 0x55a85462b070 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcc7be00640 1 -- 172.21.15.27:0/2506402832 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8548d5b90 con 0x55a85462b070 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcc7be00640 1 -- 172.21.15.27:0/2506402832 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc900a9490 con 0x55a85462b070 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcca42a5740 10 client.29876 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcca42a5740 10 client.29876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcca42a5740 20 client.29876 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545e99f0 0x55a8549ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcca42a5740 4 client.29876 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:44.999+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a85494af00 con 0x55a8545e99f0 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.000+0000 7fcc96a00640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545e99f0 0x55a8549ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.000+0000 7fcc7be00640 10 client.29876 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.000+0000 7fcc96a00640 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545e99f0 0x55a8549ef6a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1019effa6d4ce014 server_cookie=7c0db92ee1bfb19 in_seq=0 out_seq=0 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcc7be00640 1 -- 172.21.15.27:0/2506402832 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a85494af00 con 0x55a8545e99f0 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcc7be00640 10 client.29876 handle_command_reply: tid=0 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 client.29876 shutdown 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 msgr2=0x55a8549e9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549e9870 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcc900e5240 tx=0x7fcc900ad0b0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 shutdown_connections 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545e99f0 0x55a8549ef6a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549dda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85462b070 0x55a8549e9870 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcc7be00640 0 client.29876 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2506402832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8546021b0 0x55a8549dd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.001+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 >> 172.21.15.27:0/2506402832 conn(0x55a85461f460 msgr2=0x55a8549d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.002+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 shutdown_connections 2024-09-21T07:27:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.002+0000 7fcca42a5740 1 -- 172.21.15.27:0/2506402832 wait complete. 2024-09-21T07:27:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.002+0000 7fcca42a5740 20 client.29876 trim_cache size 0 max 16384 2024-09-21T07:27:45.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 Processor -- start 2024-09-21T07:27:45.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 -- start start 2024-09-21T07:27:45.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8546021b0 0x55a8549d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981b5990 con 0x55a85462b070 2024-09-21T07:27:45.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981b3260 con 0x55a8546021b0 2024-09-21T07:27:45.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcca42a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981b10d0 con 0x55a85460a390 2024-09-21T07:27:45.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcc97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549dac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcc97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549dac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54134/0 (socket says 172.21.15.27:54134) 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcc97400640 1 -- 172.21.15.27:0/337343432 learned_addr learned my addr 172.21.15.27:0/337343432 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.016+0000 7fcc96a00640 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8546021b0 0x55a8549d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc8e000640 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc97400640 1 -- 172.21.15.27:0/337343432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 msgr2=0x55a8549da700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc97400640 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549da700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc97400640 1 -- 172.21.15.27:0/337343432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8546021b0 msgr2=0x55a8549d34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc97400640 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8546021b0 0x55a8549d34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc97400640 1 -- 172.21.15.27:0/337343432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8549d4090 con 0x55a85462b070 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc96a00640 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8546021b0 0x55a8549d34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc97400640 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549dac40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcc900aabb0 tx=0x7fcc900ad1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca3ff5a7315ee59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.017+0000 7fcc86a00640 1 -- 172.21.15.27:0/337343432 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc900a90b0 con 0x55a85462b070 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcc86a00640 1 -- 172.21.15.27:0/337343432 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc900ac780 con 0x55a85462b070 2024-09-21T07:27:45.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcc86a00640 1 -- 172.21.15.27:0/337343432 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc900b2420 con 0x55a85462b070 2024-09-21T07:27:45.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcca42a5740 1 -- 172.21.15.27:0/337343432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 msgr2=0x55a8549dac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:45.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcca42a5740 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549dac40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcc900aabb0 tx=0x7fcc900ad1e0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcca42a5740 1 -- 172.21.15.27:0/337343432 shutdown_connections 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcca42a5740 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a85460a390 0x55a8549da700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcca42a5740 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549dac40 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcca42a5740 1 --2- 172.21.15.27:0/337343432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8546021b0 0x55a8549d34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.018+0000 7fcca42a5740 1 -- 172.21.15.27:0/337343432 >> 172.21.15.27:0/337343432 conn(0x55a85461f460 msgr2=0x55a8549d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.019+0000 7fcca42a5740 1 -- 172.21.15.27:0/337343432 shutdown_connections 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.019+0000 7fcca42a5740 1 -- 172.21.15.27:0/337343432 wait complete. 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.019+0000 7fcca42a5740 1 -- start start 2024-09-21T07:27:45.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.019+0000 7fcca42a5740 1 Processor -- start 2024-09-21T07:27:45.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8546021b0 0x55a8549e6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85460a390 0x55a8549e1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcca42a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcca42a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981b1720 con 0x55a85462b070 2024-09-21T07:27:45.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcca42a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981b2b90 con 0x55a85460a390 2024-09-21T07:27:45.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcca42a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981b1df0 con 0x55a8546021b0 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcca42a5740 10 client.0 _ll_register_callbacks cb 0x7fcc900ad290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcc97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8546021b0 0x55a8549e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcc8e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85460a390 0x55a8549e1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8546021b0 0x55a8549e6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35420/0 (socket says 172.21.15.27:35420) 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.020+0000 7fcc96a00640 1 -- 172.21.15.27:0/2251129318 learned_addr learned my addr 172.21.15.27:0/2251129318 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc96a00640 1 -- 172.21.15.27:0/2251129318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85460a390 msgr2=0x55a8549e1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc96a00640 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85460a390 0x55a8549e1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc96a00640 1 -- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 msgr2=0x55a8549e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc96a00640 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc96a00640 1 -- 172.21.15.27:0/2251129318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a854a58fd0 con 0x55a8546021b0 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc8e000640 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85460a390 0x55a8549e1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc97400640 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:45.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc96a00640 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8546021b0 0x55a8549e6160 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005c40 tx=0x7fcc88009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e209e81668a4125a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:45.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.021+0000 7fcc7be00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:45.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.022+0000 7fcc7be00640 1 -- 172.21.15.27:0/2251129318 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88033060 con 0x55a8546021b0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.022+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc900ac780 con 0x55a8546021b0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.022+0000 7fcc7be00640 1 -- 172.21.15.27:0/2251129318 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fcc8800f9a0 con 0x55a8546021b0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.022+0000 7fcc7be00640 1 -- 172.21.15.27:0/2251129318 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc8803a050 con 0x55a8546021b0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.022+0000 7fcca42a5740 10 client.29836 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.022+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a854a5a0c0 con 0x55a8546021b0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcc7be00640 1 -- 172.21.15.27:0/2251129318 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc900ac780 con 0x55a8546021b0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcc7be00640 1 -- 172.21.15.27:0/2251129318 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc88005780 con 0x55a8546021b0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcca42a5740 10 client.29836 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcca42a5740 10 client.29836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcca42a5740 20 client.29836 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545beda0 0x55a854a5c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcca42a5740 4 client.29836 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8549469b0 con 0x55a8545beda0 2024-09-21T07:27:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.023+0000 7fcc8e000640 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545beda0 0x55a854a5c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcc8e000640 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545beda0 0x55a854a5c690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ed7dcb28055d672 server_cookie=324d7ee732579aab in_seq=0 out_seq=0 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcc7be00640 10 client.29836 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcc7be00640 1 -- 172.21.15.27:0/2251129318 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8549469b0 con 0x55a8545beda0 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcc7be00640 10 client.29836 handle_command_reply: tid=0 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 client.29836 shutdown 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8546021b0 msgr2=0x55a8549e6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8546021b0 0x55a8549e6160 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005c40 tx=0x7fcc88009e00 comp rx=0 tx=0).stop 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 shutdown_connections 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55a8545beda0 0x55a854a5c690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a85462b070 0x55a8549e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a85460a390 0x55a8549e1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 --2- 172.21.15.27:0/2251129318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8546021b0 0x55a8549e6160 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcc7be00640 0 client.29836 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:27:45.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.024+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 >> 172.21.15.27:0/2251129318 conn(0x55a85461f460 msgr2=0x55a8549d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:45.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.025+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 shutdown_connections 2024-09-21T07:27:45.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.025+0000 7fcca42a5740 1 -- 172.21.15.27:0/2251129318 wait complete. 2024-09-21T07:27:45.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.025+0000 7fcca42a5740 20 client.29836 trim_cache size 0 max 16384 2024-09-21T07:27:45.029 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.025+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcc7c0b6700 msgr2=0x7fcc7c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.025+0000 7fcc9d600640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcc7c0b6700 0x7fcc7c0b8c10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009230 tx=0x7fcc80000bc0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.025+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 msgr2=0x7fcc981a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.025+0000 7fcc9d600640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 0x7fcc981a1f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e970 tx=0x7fcc9000ed70 comp rx=0 tx=0).stop 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 shutdown_connections 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fcc7c0b6700 0x7fcc7c0b8c10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9810c020 0x7fcc981a1f30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9810b6d0 0x7fcc9819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 --2- 172.21.15.27:0/4146989437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc98105d90 0x7fcc9819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 >> 172.21.15.27:0/4146989437 conn(0x7fcc98101860 msgr2=0x7fcc98103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:45.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 shutdown_connections 2024-09-21T07:27:45.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:45.026+0000 7fcc9d600640 1 -- 172.21.15.27:0/4146989437 wait complete. 2024-09-21T07:27:45.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:44 smithi027 ceph-mon[30779]: pgmap v837: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:44 smithi153 ceph-mon[34747]: pgmap v837: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:44 smithi119 ceph-mon[34519]: pgmap v837: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-21T07:27:45.815 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:27:46.202 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.794+0000 7f1323200640 1 -- 172.21.15.27:0/1601398753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c105d90 msgr2=0x7f131c1061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.794+0000 7f1323200640 1 --2- 172.21.15.27:0/1601398753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c105d90 0x7f131c1061a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f130c009900 tx=0x7f130c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.794+0000 7f1323200640 1 -- 172.21.15.27:0/1601398753 shutdown_connections 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.794+0000 7f1323200640 1 --2- 172.21.15.27:0/1601398753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f131c10c020 0x7f131c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.794+0000 7f1323200640 1 --2- 172.21.15.27:0/1601398753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f131c10b6d0 0x7f131c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.794+0000 7f1323200640 1 --2- 172.21.15.27:0/1601398753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c105d90 0x7f131c1061a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.794+0000 7f1323200640 1 -- 172.21.15.27:0/1601398753 >> 172.21.15.27:0/1601398753 conn(0x7f131c101860 msgr2=0x7f131c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.795+0000 7f1323200640 1 -- 172.21.15.27:0/1601398753 shutdown_connections 2024-09-21T07:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.795+0000 7f1323200640 1 -- 172.21.15.27:0/1601398753 wait complete. 2024-09-21T07:27:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 Processor -- start 2024-09-21T07:27:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 -- start start 2024-09-21T07:27:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f131c105d90 0x7f131c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c10b6d0 0x7f131c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f131c10c020 0x7f131c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f131c115ce0 con 0x7f131c10b6d0 2024-09-21T07:27:46.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f131c115b60 con 0x7f131c105d90 2024-09-21T07:27:46.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.796+0000 7f1323200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f131c115e60 con 0x7f131c10c020 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f131b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c10b6d0 0x7f131c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f131be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f131c105d90 0x7f131c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f1321200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f131c10c020 0x7f131c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f131b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c10b6d0 0x7f131c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54162/0 (socket says 172.21.15.27:54162) 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f131b400640 1 -- 172.21.15.27:0/835821264 learned_addr learned my addr 172.21.15.27:0/835821264 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f131be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f131c105d90 0x7f131c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60936/0 (socket says 172.21.15.27:60936) 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f1321200640 1 -- 172.21.15.27:0/835821264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f131c105d90 msgr2=0x7f131c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f1321200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f131c105d90 0x7f131c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f1321200640 1 -- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c10b6d0 msgr2=0x7f131c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f1321200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c10b6d0 0x7f131c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f1321200640 1 -- 172.21.15.27:0/835821264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c1a2640 con 0x7f131c10c020 2024-09-21T07:27:46.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.797+0000 7f131b400640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c10b6d0 0x7f131c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.798+0000 7f1321200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f131c10c020 0x7f131c1a1f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1314009fd0 tx=0x7f131400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.798+0000 7f1318c00640 1 -- 172.21.15.27:0/835821264 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131400ce20 con 0x7f131c10c020 2024-09-21T07:27:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.798+0000 7f1318c00640 1 -- 172.21.15.27:0/835821264 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1314005020 con 0x7f131c10c020 2024-09-21T07:27:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.798+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131c1a2930 con 0x7f131c10c020 2024-09-21T07:27:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.798+0000 7f1318c00640 1 -- 172.21.15.27:0/835821264 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314010690 con 0x7f131c10c020 2024-09-21T07:27:46.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.798+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131c1a9090 con 0x7f131c10c020 2024-09-21T07:27:46.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.801+0000 7f1329e67740 1 Processor -- start 2024-09-21T07:27:46.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.803+0000 7f1318c00640 1 -- 172.21.15.27:0/835821264 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f13140054d0 con 0x7f131c10c020 2024-09-21T07:27:46.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.803+0000 7f1329e67740 1 -- start start 2024-09-21T07:27:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f1329e67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb5fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f1329e67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb5fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f1329e67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb5fe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f1329e67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d7eb574fb0 con 0x55d7eb22e1b0 2024-09-21T07:27:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f1329e67740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d7eb5643e0 con 0x55d7eb257070 2024-09-21T07:27:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f1329e67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d7eb5681f0 con 0x55d7eb236390 2024-09-21T07:27:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f131b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb5fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f131b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb5fdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60958/0 (socket says 172.21.15.27:60958) 2024-09-21T07:27:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f131b400640 1 -- 172.21.15.27:0/2212670911 learned_addr learned my addr 172.21.15.27:0/2212670911 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.804+0000 7f131b400640 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb5fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f131b400640 1 -- 172.21.15.27:0/2212670911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 msgr2=0x55d7eb5fe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f131b400640 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb5fe620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f131b400640 1 -- 172.21.15.27:0/2212670911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 msgr2=0x55d7eb5fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f131b400640 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb5fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f131b400640 1 -- 172.21.15.27:0/2212670911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7eb52ecd0 con 0x55d7eb257070 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f131b400640 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb5fdba0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f13080095f0 tx=0x7f1308009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b645a3dc5313444 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f1311600640 1 -- 172.21.15.27:0/2212670911 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130800fc50 con 0x55d7eb257070 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f1311600640 1 -- 172.21.15.27:0/2212670911 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f130800fdf0 con 0x55d7eb257070 2024-09-21T07:27:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.805+0000 7f1311600640 1 -- 172.21.15.27:0/2212670911 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080047a0 con 0x55d7eb257070 2024-09-21T07:27:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.806+0000 7f1318c00640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13000b6700 0x7f13000b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.806+0000 7f1318c00640 1 -- 172.21.15.27:0/835821264 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f13140e81b0 con 0x7f131c10c020 2024-09-21T07:27:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.806+0000 7f131be00640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13000b6700 0x7f13000b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 -- 172.21.15.27:0/3790249516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 msgr2=0x7ff2f410f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3790249516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 0x7ff2f410f2a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009be0 tx=0x7ff2ec02a6d0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 -- 172.21.15.27:0/3790249516 shutdown_connections 2024-09-21T07:27:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3790249516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4108120 0x7ff2f4108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3790249516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 0x7ff2f410f2a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3790249516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 0x7ff2f4102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 -- 172.21.15.27:0/3790249516 >> 172.21.15.27:0/3790249516 conn(0x7ff2f40fe230 msgr2=0x7ff2f4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 -- 172.21.15.27:0/3790249516 shutdown_connections 2024-09-21T07:27:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.807+0000 7ff2fac00640 1 -- 172.21.15.27:0/3790249516 wait complete. 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f1329e67740 1 -- 172.21.15.27:0/2212670911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 msgr2=0x55d7eb5fdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f1329e67740 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb5fdba0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f13080095f0 tx=0x7f1308009560 comp rx=0 tx=0).stop 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f1329e67740 1 -- 172.21.15.27:0/2212670911 shutdown_connections 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f1329e67740 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb5fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f1329e67740 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb5fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f1329e67740 1 --2- 172.21.15.27:0/2212670911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb5fdba0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f1329e67740 1 -- 172.21.15.27:0/2212670911 >> 172.21.15.27:0/2212670911 conn(0x55d7eb24b460 msgr2=0x55d7eb5fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.807+0000 7f131be00640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13000b6700 0x7f13000b8c10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f130c031440 tx=0x7f130c0095d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.808+0000 7f1329e67740 1 -- 172.21.15.27:0/2212670911 shutdown_connections 2024-09-21T07:27:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.808+0000 7f1329e67740 1 -- 172.21.15.27:0/2212670911 wait complete. 2024-09-21T07:27:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.808+0000 7ff2fac00640 1 Processor -- start 2024-09-21T07:27:46.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.808+0000 7f1329e67740 1 -- start start 2024-09-21T07:27:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.808+0000 7ff2fac00640 1 -- start start 2024-09-21T07:27:46.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.808+0000 7f1329e67740 1 Processor -- start 2024-09-21T07:27:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2fac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 0x7ff2f419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2fac00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4108120 0x7ff2f419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2fac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 0x7ff2f41a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2fac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4112720 con 0x7ff2f41027f0 2024-09-21T07:27:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2fac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41125a0 con 0x7ff2f4108a70 2024-09-21T07:27:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2fac00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41128a0 con 0x7ff2f4108120 2024-09-21T07:27:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2f9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 0x7ff2f419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2f9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 0x7ff2f419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54190/0 (socket says 172.21.15.27:54190) 2024-09-21T07:27:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2f9800640 1 -- 172.21.15.27:0/3681490188 learned_addr learned my addr 172.21.15.27:0/3681490188 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4108120 0x7ff2f419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.809+0000 7ff2fa200640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 0x7ff2f41a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.809+0000 7f1329e67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb22e1b0 0x55d7eb611f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.809+0000 7f1329e67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb612470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.809+0000 7f1329e67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb257070 0x55d7eb61dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.809+0000 7f1329e67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d7eb3c5700 con 0x55d7eb257070 2024-09-21T07:27:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.809+0000 7f1329e67740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d7eb4ff580 con 0x55d7eb22e1b0 2024-09-21T07:27:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.809+0000 7f1329e67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d7eb614ab0 con 0x55d7eb236390 2024-09-21T07:27:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2fa200640 1 -- 172.21.15.27:0/3681490188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4108120 msgr2=0x7ff2f419c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.809+0000 7f1329e67740 10 client.0 _ll_register_callbacks cb 0x55d7eb50f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb22e1b0 0x55d7eb611f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb612470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb22e1b0 0x55d7eb611f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60970/0 (socket says 172.21.15.27:60970) 2024-09-21T07:27:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2fa200640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4108120 0x7ff2f419c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2fa200640 1 -- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 msgr2=0x7ff2f419c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2fa200640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 0x7ff2f419c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131be00640 1 -- 172.21.15.27:0/2756623536 learned_addr learned my addr 172.21.15.27:0/2756623536 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f1321200640 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb257070 0x55d7eb61dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131b400640 1 -- 172.21.15.27:0/2756623536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb22e1b0 msgr2=0x55d7eb611f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131b400640 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb22e1b0 0x55d7eb611f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131b400640 1 -- 172.21.15.27:0/2756623536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb257070 msgr2=0x55d7eb61dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131b400640 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb257070 0x55d7eb61dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2fa200640 1 -- 172.21.15.27:0/3681490188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f41a2320 con 0x7ff2f4108a70 2024-09-21T07:27:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2f9800640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 0x7ff2f419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4108120 0x7ff2f419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:46.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f131b400640 1 -- 172.21.15.27:0/2756623536 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7eb2c2650 con 0x55d7eb236390 2024-09-21T07:27:46.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.810+0000 7f1321200640 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb257070 0x55d7eb61dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.810+0000 7ff2fa200640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 0x7ff2f41a1c20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009b30 tx=0x7ff2ec003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f131b400640 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb612470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f13080058c0 tx=0x7f130800ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc3daa3698139b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f1306000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.811+0000 7ff2f2000640 1 -- 172.21.15.27:0/3681490188 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec041020 con 0x7ff2f4108a70 2024-09-21T07:27:46.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f1306000640 1 -- 172.21.15.27:0/2756623536 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130802c060 con 0x55d7eb236390 2024-09-21T07:27:46.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f1306000640 1 -- 172.21.15.27:0/2756623536 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f1308019040 con 0x55d7eb236390 2024-09-21T07:27:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.811+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f41a25b0 con 0x7ff2f4108a70 2024-09-21T07:27:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7eb501b90 con 0x55d7eb236390 2024-09-21T07:27:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.811+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41a8da0 con 0x7ff2f4108a70 2024-09-21T07:27:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.811+0000 7ff2f2000640 1 -- 172.21.15.27:0/3681490188 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ec0318f0 con 0x7ff2f4108a70 2024-09-21T07:27:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.811+0000 7ff2f2000640 1 -- 172.21.15.27:0/3681490188 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec036040 con 0x7ff2f4108a70 2024-09-21T07:27:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f1306000640 1 -- 172.21.15.27:0/2756623536 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1308005230 con 0x55d7eb236390 2024-09-21T07:27:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f1329e67740 10 client.29854 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.811+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7eb51c950 con 0x55d7eb236390 2024-09-21T07:27:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1306000640 1 -- 172.21.15.27:0/2756623536 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7eb501b90 con 0x55d7eb236390 2024-09-21T07:27:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1306000640 1 -- 172.21.15.27:0/2756623536 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f130801d030 con 0x55d7eb236390 2024-09-21T07:27:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1329e67740 10 client.29854 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1329e67740 10 client.29854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1329e67740 20 client.29854 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1329e67740 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb2159f0 0x55d7eb623c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1329e67740 4 client.29854 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.812+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7eb576f00 con 0x55d7eb2159f0 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.813+0000 7f131be00640 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb2159f0 0x55d7eb623c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.813+0000 7f131be00640 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb2159f0 0x55d7eb623c60 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee23f8953ec7ad3 server_cookie=d3baf2cede4d8682 in_seq=0 out_seq=0 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.813+0000 7f1306000640 10 client.29854 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.814+0000 7f1306000640 1 -- 172.21.15.27:0/2756623536 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7eb576f00 con 0x55d7eb2159f0 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.814+0000 7f1306000640 10 client.29854 handle_command_reply: tid=0 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.814+0000 7f1329e67740 1 client.29854 shutdown 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 msgr2=0x55d7eb612470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb612470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f13080058c0 tx=0x7f130800ffa0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.815+0000 7ff303406740 1 Processor -- start 2024-09-21T07:27:46.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 shutdown_connections 2024-09-21T07:27:46.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb2159f0 0x55d7eb623c60 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb612470 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb257070 0x55d7eb61dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 --2- 172.21.15.27:0/2756623536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb22e1b0 0x55d7eb611f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1306000640 0 client.29854 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 >> 172.21.15.27:0/2756623536 conn(0x55d7eb24b460 msgr2=0x55d7eb5fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 shutdown_connections 2024-09-21T07:27:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 1 -- 172.21.15.27:0/2756623536 wait complete. 2024-09-21T07:27:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.815+0000 7f1329e67740 20 client.29854 trim_cache size 0 max 16384 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff2f2000640 1 -- 172.21.15.27:0/3681490188 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff2ec044020 con 0x7ff2f4108a70 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff303406740 1 -- start start 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff303406740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff303406740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff303406740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff303406740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559640f466b0 con 0x559640c0b4d0 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff303406740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559640f35ae0 con 0x559640c2c1b0 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.816+0000 7ff303406740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559640f398f0 con 0x559640c032f0 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60984/0 (socket says 172.21.15.27:60984) 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f8e00640 1 -- 172.21.15.27:0/721065411 learned_addr learned my addr 172.21.15.27:0/721065411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f9800640 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f9800640 1 -- 172.21.15.27:0/721065411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 msgr2=0x559640fd0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f9800640 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f9800640 1 -- 172.21.15.27:0/721065411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 msgr2=0x559640fd1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f9800640 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.817+0000 7ff2f9800640 1 -- 172.21.15.27:0/721065411 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559640f02780 con 0x559640c032f0 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.818+0000 7ff2f9800640 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd1350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8011540 tx=0x7ff2e8011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d52841113a9e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.818+0000 7ff2e6a00640 1 -- 172.21.15.27:0/721065411 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004030 con 0x559640c032f0 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.818+0000 7ff2e6a00640 1 -- 172.21.15.27:0/721065411 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e80041d0 con 0x559640c032f0 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.818+0000 7ff2e6a00640 1 -- 172.21.15.27:0/721065411 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8003850 con 0x559640c032f0 2024-09-21T07:27:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff2f2000640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff2dc0b6680 0x7ff2dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 -- 172.21.15.27:0/721065411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 msgr2=0x559640fd1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd1350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8011540 tx=0x7ff2e8011a00 comp rx=0 tx=0).stop 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 -- 172.21.15.27:0/721065411 shutdown_connections 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd1350 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 --2- 172.21.15.27:0/721065411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 -- 172.21.15.27:0/721065411 >> 172.21.15.27:0/721065411 conn(0x559640c20300 msgr2=0x559640fced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff2f2000640 1 -- 172.21.15.27:0/3681490188 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7ff2ec108270 con 0x7ff2f4108a70 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 -- 172.21.15.27:0/721065411 shutdown_connections 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff2dc0b6680 0x7ff2dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.819+0000 7ff303406740 1 -- 172.21.15.27:0/721065411 wait complete. 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.820+0000 7ff303406740 1 -- start start 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.820+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff2dc0b6680 0x7ff2dc0b8b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00058c0 tx=0x7ff2e0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.820+0000 7ff303406740 1 Processor -- start 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff303406740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fdc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff303406740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fdc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff303406740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fdce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff303406740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559640e6dab0 con 0x559640c0b4d0 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff303406740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559640edc570 con 0x559640c2c1b0 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff303406740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559640fdf5e0 con 0x559640c032f0 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff303406740 10 client.0 _ll_register_callbacks cb 0x559640eeb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff2f8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fdc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff2f8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fdc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54216/0 (socket says 172.21.15.27:54216) 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1339760386 learned_addr learned my addr 172.21.15.27:0/1339760386 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff2fa200640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fdce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff2f9800640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fdc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.821+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1339760386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 msgr2=0x559640fdc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.822+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fdc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.822+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 msgr2=0x559640fdce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.822+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fdce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.822+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1339760386 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559640c97790 con 0x559640c0b4d0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.822+0000 7ff2f9800640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fdc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.822+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fdc8f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003bf0 tx=0x7ff2e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff223c2e44df172 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.822+0000 7ff2db400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.823+0000 7ff2db400640 1 -- 172.21.15.27:0/1339760386 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0031060 con 0x559640c0b4d0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.823+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e80041d0 con 0x559640c0b4d0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.823+0000 7ff2db400640 1 -- 172.21.15.27:0/1339760386 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7ff2e001a050 con 0x559640c0b4d0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff2db400640 1 -- 172.21.15.27:0/1339760386 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2e0038050 con 0x559640c0b4d0 2024-09-21T07:27:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 10 client.19881 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559640d10620 con 0x559640c0b4d0 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff2db400640 1 -- 172.21.15.27:0/1339760386 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e80041d0 con 0x559640c0b4d0 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff2db400640 1 -- 172.21.15.27:0/1339760386 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e001f700 con 0x559640c0b4d0 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 10 client.19881 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 10 client.19881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 20 client.19881 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bead80 0x559640fee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 4 client.19881 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.824+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559640f48600 con 0x559640bead80 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.825+0000 7ff2f9800640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bead80 0x559640fee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.825+0000 7ff2db400640 10 client.19881 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.825+0000 7ff2f9800640 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bead80 0x559640fee590 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09a7663f73edbc8 server_cookie=a264f050982b282 in_seq=0 out_seq=0 2024-09-21T07:27:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff2db400640 1 -- 172.21.15.27:0/1339760386 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559640f48600 con 0x559640bead80 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff2db400640 10 client.19881 handle_command_reply: tid=0 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 client.19881 shutdown 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 msgr2=0x559640fdc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fdc8f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003bf0 tx=0x7ff2e0004730 comp rx=0 tx=0).stop 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 shutdown_connections 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bead80 0x559640fee590 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fdce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fdc8f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 --2- 172.21.15.27:0/1339760386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fdc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 >> 172.21.15.27:0/1339760386 conn(0x559640c20300 msgr2=0x559640fd1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff2db400640 0 client.19881 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.826+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 shutdown_connections 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.827+0000 7ff303406740 1 -- 172.21.15.27:0/1339760386 wait complete. 2024-09-21T07:27:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.827+0000 7ff303406740 20 client.19881 trim_cache size 0 max 16384 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.832+0000 7f1329e67740 1 Processor -- start 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1329e67740 1 -- start start 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1329e67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb613c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1329e67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1329e67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb61e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1329e67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1b1570 con 0x55d7eb22e1b0 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1329e67740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1aee40 con 0x55d7eb257070 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1329e67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1accb0 con 0x55d7eb236390 2024-09-21T07:27:46.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f131be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb613c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f131be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb613c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54218/0 (socket says 172.21.15.27:54218) 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f131be00640 1 -- 172.21.15.27:0/721145655 learned_addr learned my addr 172.21.15.27:0/721145655 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f131b400640 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.833+0000 7f1321200640 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb61e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f131b400640 1 -- 172.21.15.27:0/721145655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 msgr2=0x55d7eb61e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f131b400640 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb61e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f131b400640 1 -- 172.21.15.27:0/721145655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 msgr2=0x55d7eb613c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f131b400640 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb613c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f131b400640 1 -- 172.21.15.27:0/721145655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7eb5f6ea0 con 0x55d7eb236390 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f131b400640 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61dce0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1308027a30 tx=0x7f1308011b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df548f82d40aee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f1310c00640 1 -- 172.21.15.27:0/721145655 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130802c040 con 0x55d7eb236390 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.834+0000 7f1310c00640 1 -- 172.21.15.27:0/721145655 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1308027cb0 con 0x55d7eb236390 2024-09-21T07:27:46.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1310c00640 1 -- 172.21.15.27:0/721145655 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308011270 con 0x55d7eb236390 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1329e67740 1 -- 172.21.15.27:0/721145655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 msgr2=0x55d7eb61dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1329e67740 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61dce0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1308027a30 tx=0x7f1308011b90 comp rx=0 tx=0).stop 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1329e67740 1 -- 172.21.15.27:0/721145655 shutdown_connections 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1329e67740 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61dce0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1329e67740 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb61e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1329e67740 1 --2- 172.21.15.27:0/721145655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb613c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.835+0000 7f1329e67740 1 -- 172.21.15.27:0/721145655 >> 172.21.15.27:0/721145655 conn(0x55d7eb24b460 msgr2=0x55d7eb5fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.836+0000 7f1329e67740 1 -- 172.21.15.27:0/721145655 shutdown_connections 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.836+0000 7f1329e67740 1 -- 172.21.15.27:0/721145655 wait complete. 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.836+0000 7f1329e67740 1 -- start start 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.836+0000 7f1329e67740 1 Processor -- start 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1329e67740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb619d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1329e67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1329e67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb614d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1329e67740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1ad300 con 0x55d7eb22e1b0 2024-09-21T07:27:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1329e67740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1ae770 con 0x55d7eb257070 2024-09-21T07:27:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1329e67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1ad9d0 con 0x55d7eb236390 2024-09-21T07:27:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1329e67740 10 client.0 _ll_register_callbacks cb 0x7f1308011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f131be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb619d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f131b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f131b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35492/0 (socket says 172.21.15.27:35492) 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f131b400640 1 -- 172.21.15.27:0/896249659 learned_addr learned my addr 172.21.15.27:0/896249659 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.837+0000 7f1321200640 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb614d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f131b400640 1 -- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 msgr2=0x55d7eb614d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f131b400640 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb614d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f131b400640 1 -- 172.21.15.27:0/896249659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 msgr2=0x55d7eb619d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f131b400640 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb619d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f131b400640 1 -- 172.21.15.27:0/896249659 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7eb615240 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f131be00640 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb619d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f131b400640 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61a2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f13080059b0 tx=0x7f1308005620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3045ba5222808684 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.838+0000 7f1306000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.839+0000 7f1306000640 1 -- 172.21.15.27:0/896249659 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130802c040 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.839+0000 7f1306000640 1 -- 172.21.15.27:0/896249659 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f1308004220 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.839+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1308027cb0 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.839+0000 7f1306000640 1 -- 172.21.15.27:0/896249659 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1308046050 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.839+0000 7f1329e67740 10 client.29866 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.839+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7eb68ec50 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.839+0000 7f1306000640 1 -- 172.21.15.27:0/896249659 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1308027cb0 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f1306000640 1 -- 172.21.15.27:0/896249659 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f130801d070 con 0x55d7eb236390 2024-09-21T07:27:46.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f1329e67740 10 client.29866 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f1329e67740 10 client.29866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f1329e67740 20 client.29866 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f1329e67740 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb1eada0 0x55d7eb6912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f1329e67740 4 client.29866 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7eb5729b0 con 0x55d7eb1eada0 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.840+0000 7f131be00640 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb1eada0 0x55d7eb6912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f131be00640 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb1eada0 0x55d7eb6912c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acc3bb6819865dd server_cookie=ac2a79620066fa1e in_seq=0 out_seq=0 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1306000640 10 client.29866 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1306000640 1 -- 172.21.15.27:0/896249659 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7eb5729b0 con 0x55d7eb1eada0 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1306000640 10 client.29866 handle_command_reply: tid=0 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 client.29866 shutdown 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 msgr2=0x55d7eb61a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61a2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f13080059b0 tx=0x7f1308005620 comp rx=0 tx=0).stop 2024-09-21T07:27:46.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 shutdown_connections 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55d7eb1eada0 0x55d7eb6912c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7eb236390 0x55d7eb61a2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d7eb257070 0x55d7eb614d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1306000640 0 client.29866 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 --2- 172.21.15.27:0/896249659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d7eb22e1b0 0x55d7eb619d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.841+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 >> 172.21.15.27:0/896249659 conn(0x55d7eb24b460 msgr2=0x55d7eb610f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.842+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 shutdown_connections 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.842+0000 7f1329e67740 1 -- 172.21.15.27:0/896249659 wait complete. 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.842+0000 7f1329e67740 20 client.29866 trim_cache size 0 max 16384 2024-09-21T07:27:46.847 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:27:46.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.842+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13000b6700 msgr2=0x7f13000b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.842+0000 7f1323200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13000b6700 0x7f13000b8c10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f130c031440 tx=0x7f130c0095d0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.842+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f131c10c020 msgr2=0x7f131c1a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.842+0000 7f1323200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f131c10c020 0x7f131c1a1f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1314009fd0 tx=0x7f131400ef10 comp rx=0 tx=0).stop 2024-09-21T07:27:46.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 shutdown_connections 2024-09-21T07:27:46.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f131c10c020 0x7f131c1a1f10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f13000b6700 0x7f13000b8c10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c10b6d0 0x7f131c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 --2- 172.21.15.27:0/835821264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f131c105d90 0x7f131c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 >> 172.21.15.27:0/835821264 conn(0x7f131c101860 msgr2=0x7f131c10ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 shutdown_connections 2024-09-21T07:27:46.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:46.843+0000 7f1323200640 1 -- 172.21.15.27:0/835821264 wait complete. 2024-09-21T07:27:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.847+0000 7ff303406740 1 Processor -- start 2024-09-21T07:27:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.847+0000 7ff303406740 1 -- start start 2024-09-21T07:27:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff303406740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff303406740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff303406740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff303406740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b5710 con 0x559640c0b4d0 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff303406740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b2fe0 con 0x559640c2c1b0 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff303406740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b0e50 con 0x559640c032f0 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff2f8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff2f8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54236/0 (socket says 172.21.15.27:54236) 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1323840870 learned_addr learned my addr 172.21.15.27:0/1323840870 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff2f9800640 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.848+0000 7ff2fa200640 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1323840870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 msgr2=0x559640fd2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1323840870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 msgr2=0x559640fd9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2f8e00640 1 -- 172.21.15.27:0/1323840870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559640fd7ee0 con 0x559640c0b4d0 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd9580 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009df0 tx=0x7ff2e0009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec54a8b2a2cfdfa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2e6000640 1 -- 172.21.15.27:0/1323840870 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001fd10 con 0x559640c0b4d0 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2e6000640 1 -- 172.21.15.27:0/1323840870 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e001f3a0 con 0x559640c0b4d0 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.849+0000 7ff2e6000640 1 -- 172.21.15.27:0/1323840870 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00217a0 con 0x559640c0b4d0 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 -- 172.21.15.27:0/1323840870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 msgr2=0x559640fd9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd9580 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009df0 tx=0x7ff2e0009ed0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 -- 172.21.15.27:0/1323840870 shutdown_connections 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c2c1b0 0x559640fd9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c0b4d0 0x559640fd9580 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 --2- 172.21.15.27:0/1323840870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c032f0 0x559640fd2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 -- 172.21.15.27:0/1323840870 >> 172.21.15.27:0/1323840870 conn(0x559640c20300 msgr2=0x559640fd1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 -- 172.21.15.27:0/1323840870 shutdown_connections 2024-09-21T07:27:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.850+0000 7ff303406740 1 -- 172.21.15.27:0/1323840870 wait complete. 2024-09-21T07:27:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.851+0000 7ff303406740 1 -- start start 2024-09-21T07:27:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.851+0000 7ff303406740 1 Processor -- start 2024-09-21T07:27:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff303406740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c032f0 0x559640fe4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff303406740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c0b4d0 0x559640fe2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff303406740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c2c1b0 0x559640fe2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff303406740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b14a0 con 0x559640c2c1b0 2024-09-21T07:27:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff303406740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b2910 con 0x559640c032f0 2024-09-21T07:27:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff303406740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b1b70 con 0x559640c0b4d0 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff303406740 10 client.0 _ll_register_callbacks cb 0x7ff2e0009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c032f0 0x559640fe4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2fa200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c2c1b0 0x559640fe2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c032f0 0x559640fe4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:32790/0 (socket says 172.21.15.27:32790) 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 -- 172.21.15.27:0/3339873685 learned_addr learned my addr 172.21.15.27:0/3339873685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c0b4d0 0x559640fe2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 -- 172.21.15.27:0/3339873685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c0b4d0 msgr2=0x559640fe2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c0b4d0 0x559640fe2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 -- 172.21.15.27:0/3339873685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c2c1b0 msgr2=0x559640fe2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c2c1b0 0x559640fe2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2f9800640 1 -- 172.21.15.27:0/3339873685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559640fe2ff0 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.852+0000 7ff2fa200640 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c2c1b0 0x559640fe2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff2f9800640 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c032f0 0x559640fe4f50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8011f00 tx=0x7ff2e800fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9945e3a711eee24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff2db400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff2db400640 1 -- 172.21.15.27:0/3339873685 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8033060 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e001f3a0 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff2db400640 1 -- 172.21.15.27:0/3339873685 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7ff2e8005040 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff2db400640 1 -- 172.21.15.27:0/3339873685 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2e803a050 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff303406740 10 client.29918 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559641058dc0 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.853+0000 7ff2db400640 1 -- 172.21.15.27:0/3339873685 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e001f3a0 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff2db400640 1 -- 172.21.15.27:0/3339873685 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e8004a30 con 0x559640c032f0 2024-09-21T07:27:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff303406740 10 client.29918 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff303406740 10 client.29918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff303406740 20 client.29918 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff303406740 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bc01c0 0x55964105b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff303406740 4 client.29918 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559640f440b0 con 0x559640bc01c0 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.854+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bc01c0 0x55964105b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.855+0000 7ff2f8e00640 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bc01c0 0x55964105b410 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1dfac8b7ff69a2e server_cookie=839257b5eb318d77 in_seq=0 out_seq=0 2024-09-21T07:27:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.855+0000 7ff2db400640 10 client.29918 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff2db400640 1 -- 172.21.15.27:0/3339873685 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559640f440b0 con 0x559640bc01c0 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff2db400640 10 client.29918 handle_command_reply: tid=0 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 client.29918 shutdown 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c032f0 msgr2=0x559640fe4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c032f0 0x559640fe4f50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8011f00 tx=0x7ff2e800fef0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 shutdown_connections 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559640bc01c0 0x55964105b410 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559640c0b4d0 0x559640fe2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559640c2c1b0 0x559640fe2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 --2- 172.21.15.27:0/3339873685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559640c032f0 0x559640fe4f50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff2db400640 0 client.29918 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.867+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 >> 172.21.15.27:0/3339873685 conn(0x559640c20300 msgr2=0x559640fd1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.868+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 shutdown_connections 2024-09-21T07:27:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.868+0000 7ff303406740 1 -- 172.21.15.27:0/3339873685 wait complete. 2024-09-21T07:27:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.868+0000 7ff303406740 20 client.29918 trim_cache size 0 max 16384 2024-09-21T07:27:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:27:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:27:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:27:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff2dc0b6680 msgr2=0x7ff2dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff2dc0b6680 0x7ff2dc0b8b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00058c0 tx=0x7ff2e0005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 msgr2=0x7ff2f41a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 0x7ff2f41a1c20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009b30 tx=0x7ff2ec003b80 comp rx=0 tx=0).stop 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 shutdown_connections 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4108120 0x7ff2f419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff2dc0b6680 0x7ff2dc0b8b90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2f4108a70 0x7ff2f41a1c20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 --2- 172.21.15.27:0/3681490188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2f41027f0 0x7ff2f419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 >> 172.21.15.27:0/3681490188 conn(0x7ff2f40fe230 msgr2=0x7ff2f406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 shutdown_connections 2024-09-21T07:27:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:27:46.869+0000 7ff2fac00640 1 -- 172.21.15.27:0/3681490188 wait complete. 2024-09-21T07:27:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:46 smithi153 ceph-mon[34747]: pgmap v838: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-21T07:27:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:46 smithi119 ceph-mon[34519]: pgmap v838: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-21T07:27:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:46 smithi027 ceph-mon[30779]: pgmap v838: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-21T07:27:48.718 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:27:48.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:27:49.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:48 smithi027 ceph-mon[30779]: pgmap v839: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 KiB/s wr, 8 op/s 2024-09-21T07:27:49.116 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:48 smithi153 ceph-mon[34747]: pgmap v839: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 KiB/s wr, 8 op/s 2024-09-21T07:27:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:48 smithi119 ceph-mon[34519]: pgmap v839: 129 pgs: 129 active+clean; 2.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 KiB/s wr, 8 op/s 2024-09-21T07:27:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.780+0000 7f5b24e00640 1 -- 172.21.15.27:0/2598900576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 msgr2=0x7f5b201061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.780+0000 7f5b24e00640 1 --2- 172.21.15.27:0/2598900576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 0x7f5b201061a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009ca0 tx=0x7f5b1002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.781+0000 7f5b24e00640 1 -- 172.21.15.27:0/2598900576 shutdown_connections 2024-09-21T07:27:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.781+0000 7f5b24e00640 1 --2- 172.21.15.27:0/2598900576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2010b6d0 0x7f5b2010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.781+0000 7f5b24e00640 1 --2- 172.21.15.27:0/2598900576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2010c020 0x7f5b20112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.781+0000 7f5b24e00640 1 --2- 172.21.15.27:0/2598900576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 0x7f5b201061a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.781+0000 7f5b24e00640 1 -- 172.21.15.27:0/2598900576 >> 172.21.15.27:0/2598900576 conn(0x7f5b20101860 msgr2=0x7f5b20103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.781+0000 7f5b24e00640 1 -- 172.21.15.27:0/2598900576 shutdown_connections 2024-09-21T07:27:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.781+0000 7f5b24e00640 1 -- 172.21.15.27:0/2598900576 wait complete. 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.782+0000 7f5b24e00640 1 Processor -- start 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.782+0000 7f5b24e00640 1 -- start start 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b24e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 0x7f5b2019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b24e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2010b6d0 0x7f5b2019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b24e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2010c020 0x7f5b201a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b24e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20115ce0 con 0x7f5b20105d90 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b24e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20115b60 con 0x7f5b2010c020 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b24e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20115e60 con 0x7f5b2010b6d0 2024-09-21T07:27:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b1de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 0x7f5b2019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b1de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 0x7f5b2019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36732/0 (socket says 172.21.15.27:36732) 2024-09-21T07:27:49.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b15400640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2010b6d0 0x7f5b2019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b1de00640 1 -- 172.21.15.27:0/1701022193 learned_addr learned my addr 172.21.15.27:0/1701022193 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:49.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b1e800640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2010c020 0x7f5b201a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b15400640 1 -- 172.21.15.27:0/1701022193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2010c020 msgr2=0x7f5b201a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b15400640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2010c020 0x7f5b201a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b15400640 1 -- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 msgr2=0x7f5b2019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b15400640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 0x7f5b2019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.783+0000 7f5b15400640 1 -- 172.21.15.27:0/1701022193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b201a26c0 con 0x7f5b2010b6d0 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.784+0000 7f5b15400640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2010b6d0 0x7f5b2019ccf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5b2006ca80 tx=0x7f5b0800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.784+0000 7f5b17400640 1 -- 172.21.15.27:0/1701022193 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800cc40 con 0x7f5b2010b6d0 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.784+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b201a29b0 con 0x7f5b2010b6d0 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.784+0000 7f5b17400640 1 -- 172.21.15.27:0/1701022193 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b0800cde0 con 0x7f5b2010b6d0 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.784+0000 7f5b17400640 1 -- 172.21.15.27:0/1701022193 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08004a00 con 0x7f5b2010b6d0 2024-09-21T07:27:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.784+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b201a9110 con 0x7f5b2010b6d0 2024-09-21T07:27:49.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.788+0000 7f5b2b9d3740 1 Processor -- start 2024-09-21T07:27:49.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.788+0000 7f5b2b9d3740 1 -- start start 2024-09-21T07:27:49.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b50bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b50c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b50c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a9b481e90 con 0x561a9b164350 2024-09-21T07:27:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561a9b4712c0 con 0x561a9b143670 2024-09-21T07:27:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a9b4750d0 con 0x561a9b13b490 2024-09-21T07:27:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b1e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b50bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b1e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b50bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36736/0 (socket says 172.21.15.27:36736) 2024-09-21T07:27:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b1e800640 1 -- 172.21.15.27:0/519163388 learned_addr learned my addr 172.21.15.27:0/519163388 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.789+0000 7f5b1de00640 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b50c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b1e800640 1 -- 172.21.15.27:0/519163388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 msgr2=0x561a9b50c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b1e800640 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b50c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b1e800640 1 -- 172.21.15.27:0/519163388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 msgr2=0x561a9b50c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b1e800640 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b50c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b1e800640 1 -- 172.21.15.27:0/519163388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9b1cf930 con 0x561a9b164350 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b1de00640 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b50c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b17400640 1 -- 172.21.15.27:0/1701022193 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5b08004ba0 con 0x7f5b2010b6d0 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.790+0000 7f5b1e800640 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b50bd10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5b18004620 tx=0x7f5b1800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417c96efaf417980 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.791+0000 7f5b0f400640 1 -- 172.21.15.27:0/519163388 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18003ba0 con 0x561a9b164350 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.791+0000 7f5b0f400640 1 -- 172.21.15.27:0/519163388 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18015070 con 0x561a9b164350 2024-09-21T07:27:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.791+0000 7f5b0f400640 1 -- 172.21.15.27:0/519163388 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b180116a0 con 0x561a9b164350 2024-09-21T07:27:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.791+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/519163388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 msgr2=0x561a9b50bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.791+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b50bd10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5b18004620 tx=0x7f5b1800bb10 comp rx=0 tx=0).stop 2024-09-21T07:27:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/519163388 shutdown_connections 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b50c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b50c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/519163388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b50bd10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/519163388 >> 172.21.15.27:0/519163388 conn(0x561a9b158740 msgr2=0x561a9b509c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/519163388 shutdown_connections 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/519163388 wait complete. 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b2b9d3740 1 -- start start 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b17400640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b040b6590 0x7f5b040b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.792+0000 7f5b17400640 1 -- 172.21.15.27:0/1701022193 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f5b080e7aa0 con 0x7f5b2010b6d0 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 1 Processor -- start 2024-09-21T07:27:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b1e800640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b040b6590 0x7f5b040b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b517960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b517f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b5237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a9b2d2890 con 0x561a9b164350 2024-09-21T07:27:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561a9b40c160 con 0x561a9b143670 2024-09-21T07:27:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a9b51a4f0 con 0x561a9b13b490 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b1e800640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b040b6590 0x7f5b040b8aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5b180043b0 tx=0x7f5b180041b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.793+0000 7f5b2b9d3740 10 client.0 _ll_register_callbacks cb 0x561a9b428a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b517f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b1de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b517960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b517f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60440/0 (socket says 172.21.15.27:60440) 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 -- 172.21.15.27:0/126282044 learned_addr learned my addr 172.21.15.27:0/126282044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b1e800640 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b5237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 -- 172.21.15.27:0/126282044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 msgr2=0x561a9b517960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b517960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 -- 172.21.15.27:0/126282044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 msgr2=0x561a9b5237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b5237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 -- 172.21.15.27:0/126282044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a9b523e50 con 0x561a9b143670 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b15400640 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b517f40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5b080a9ab0 tx=0x7f5b08004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db2d90d3ac516ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.794+0000 7f5b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.795+0000 7f5b03e00640 1 -- 172.21.15.27:0/126282044 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b080d0e60 con 0x561a9b143670 2024-09-21T07:27:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.795+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b18015070 con 0x561a9b143670 2024-09-21T07:27:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.795+0000 7f5b03e00640 1 -- 172.21.15.27:0/126282044 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f5b080ac740 con 0x561a9b143670 2024-09-21T07:27:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.795+0000 7f5b03e00640 1 -- 172.21.15.27:0/126282044 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b0801c040 con 0x561a9b143670 2024-09-21T07:27:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.795+0000 7f5b2b9d3740 10 client.29927 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.795+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a9b526810 con 0x561a9b143670 2024-09-21T07:27:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.795+0000 7f5b03e00640 1 -- 172.21.15.27:0/126282044 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18015070 con 0x561a9b143670 2024-09-21T07:27:49.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b03e00640 1 -- 172.21.15.27:0/126282044 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b080a8e30 con 0x561a9b143670 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b2b9d3740 10 client.29927 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b2b9d3740 10 client.29927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b2b9d3740 20 client.29927 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b122cd0 0x561a9b529c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b2b9d3740 4 client.29927 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a9b483de0 con 0x561a9b122cd0 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.796+0000 7f5b1de00640 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b122cd0 0x561a9b529c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.797+0000 7f5b1de00640 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b122cd0 0x561a9b529c60 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab30c09deeab4476 server_cookie=a187a42e1a3bb22d in_seq=0 out_seq=0 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.797+0000 7f5b03e00640 10 client.29927 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b03e00640 1 -- 172.21.15.27:0/126282044 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561a9b483de0 con 0x561a9b122cd0 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b03e00640 10 client.29927 handle_command_reply: tid=0 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 client.29927 shutdown 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 msgr2=0x561a9b517f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b517f40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5b080a9ab0 tx=0x7f5b08004f60 comp rx=0 tx=0).stop 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 shutdown_connections 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b122cd0 0x561a9b529c60 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b164350 0x561a9b5237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b03e00640 0 client.29927 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b143670 0x561a9b517f40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/126282044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b13b490 0x561a9b517960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 >> 172.21.15.27:0/126282044 conn(0x561a9b158740 msgr2=0x561a9b508280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.798+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 shutdown_connections 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.799+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/126282044 wait complete. 2024-09-21T07:27:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.799+0000 7f5b2b9d3740 20 client.29927 trim_cache size 0 max 16384 2024-09-21T07:27:49.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.817+0000 7f5b2b9d3740 1 Processor -- start 2024-09-21T07:27:49.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.817+0000 7f5b2b9d3740 1 -- start start 2024-09-21T07:27:49.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b13b490 0x561a9b508990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b143670 0x561a9b5236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b164350 0x561a9b510e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b5990 con 0x561a9b143670 2024-09-21T07:27:49.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b3260 con 0x561a9b13b490 2024-09-21T07:27:49.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b10d0 con 0x561a9b164350 2024-09-21T07:27:49.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b15400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b143670 0x561a9b5236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b15400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b143670 0x561a9b5236e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36758/0 (socket says 172.21.15.27:36758) 2024-09-21T07:27:49.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b15400640 1 -- 172.21.15.27:0/2121744220 learned_addr learned my addr 172.21.15.27:0/2121744220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:49.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b1e800640 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b164350 0x561a9b510e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.818+0000 7f5b1de00640 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b13b490 0x561a9b508990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b1e800640 1 -- 172.21.15.27:0/2121744220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b13b490 msgr2=0x561a9b508990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b1e800640 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b13b490 0x561a9b508990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b1e800640 1 -- 172.21.15.27:0/2121744220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b143670 msgr2=0x561a9b5236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b1e800640 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b143670 0x561a9b5236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b1e800640 1 -- 172.21.15.27:0/2121744220 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9b516700 con 0x561a9b164350 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b1e800640 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b164350 0x561a9b510e20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8704c02275db88b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b0ea00640 1 -- 172.21.15.27:0/2121744220 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1801a900 con 0x561a9b164350 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b0ea00640 1 -- 172.21.15.27:0/2121744220 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18015040 con 0x561a9b164350 2024-09-21T07:27:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.819+0000 7f5b0ea00640 1 -- 172.21.15.27:0/2121744220 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18004d60 con 0x561a9b164350 2024-09-21T07:27:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/2121744220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b164350 msgr2=0x561a9b510e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b164350 0x561a9b510e20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18020000 comp rx=0 tx=0).stop 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/2121744220 shutdown_connections 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b164350 0x561a9b510e20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b143670 0x561a9b5236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/2121744220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b13b490 0x561a9b508990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/2121744220 >> 172.21.15.27:0/2121744220 conn(0x561a9b158740 msgr2=0x561a9b508280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/2121744220 shutdown_connections 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.820+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/2121744220 wait complete. 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.821+0000 7f5b2b9d3740 1 -- start start 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.821+0000 7f5b2b9d3740 1 Processor -- start 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b13b490 0x561a9b51f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b143670 0x561a9b51fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b2b9d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b164350 0x561a9b51a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b1720 con 0x561a9b13b490 2024-09-21T07:27:49.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b2b90 con 0x561a9b164350 2024-09-21T07:27:49.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b2b9d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201b1df0 con 0x561a9b143670 2024-09-21T07:27:49.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b2b9d3740 10 client.0 _ll_register_callbacks cb 0x7f5b080046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:49.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b13b490 0x561a9b51f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b13b490 0x561a9b51f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36772/0 (socket says 172.21.15.27:36772) 2024-09-21T07:27:49.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 -- 172.21.15.27:0/3033312589 learned_addr learned my addr 172.21.15.27:0/3033312589 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:49.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b15400640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b143670 0x561a9b51fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1e800640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b164350 0x561a9b51a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 -- 172.21.15.27:0/3033312589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b143670 msgr2=0x561a9b51fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b143670 0x561a9b51fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 -- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b164350 msgr2=0x561a9b51a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b164350 0x561a9b51a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1de00640 1 -- 172.21.15.27:0/3033312589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a9b51acf0 con 0x561a9b13b490 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.822+0000 7f5b1e800640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b164350 0x561a9b51a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b15400640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b143670 0x561a9b51fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b1de00640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b13b490 0x561a9b51f950 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000f80 tx=0x7f5b10031e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b302795402db1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b03e00640 1 -- 172.21.15.27:0/3033312589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10031920 con 0x561a9b13b490 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b18015040 con 0x561a9b13b490 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b03e00640 1 -- 172.21.15.27:0/3033312589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f5b10003970 con 0x561a9b13b490 2024-09-21T07:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b03e00640 1 -- 172.21.15.27:0/3033312589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b10003b70 con 0x561a9b13b490 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.823+0000 7f5b2b9d3740 10 client.19911 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a9b594e90 con 0x561a9b13b490 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b03e00640 1 -- 172.21.15.27:0/3033312589 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18015040 con 0x561a9b13b490 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b03e00640 1 -- 172.21.15.27:0/3033312589 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b1002f780 con 0x561a9b13b490 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b2b9d3740 10 client.19911 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b2b9d3740 10 client.19911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b2b9d3740 20 client.19911 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b0f8080 0x561a9b5976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b2b9d3740 4 client.19911 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.824+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a9b47f890 con 0x561a9b0f8080 2024-09-21T07:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.825+0000 7f5b15400640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b0f8080 0x561a9b5976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:49.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.825+0000 7f5b15400640 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b0f8080 0x561a9b5976d0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6fd5ce1ebc737cf server_cookie=28174b9402694134 in_seq=0 out_seq=0 2024-09-21T07:27:49.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.825+0000 7f5b03e00640 10 client.19911 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b03e00640 1 -- 172.21.15.27:0/3033312589 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a9b47f890 con 0x561a9b0f8080 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b03e00640 10 client.19911 handle_command_reply: tid=0 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 client.19911 shutdown 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b13b490 msgr2=0x561a9b51f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b13b490 0x561a9b51f950 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000f80 tx=0x7f5b10031e10 comp rx=0 tx=0).stop 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 shutdown_connections 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561a9b0f8080 0x561a9b5976d0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a9b143670 0x561a9b51fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a9b164350 0x561a9b51a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 --2- 172.21.15.27:0/3033312589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a9b13b490 0x561a9b51f950 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 >> 172.21.15.27:0/3033312589 conn(0x561a9b158740 msgr2=0x561a9b510e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b03e00640 0 client.19911 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 shutdown_connections 2024-09-21T07:27:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.827+0000 7f5b2b9d3740 1 -- 172.21.15.27:0/3033312589 wait complete. 2024-09-21T07:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.828+0000 7f5b2b9d3740 20 client.19911 trim_cache size 0 max 16384 2024-09-21T07:27:49.830 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:49.830 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:27:49.830 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "23d6d7e9-06b5-4a6e-adfb-86c244736c20", 2024-09-21T07:27:49.830 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:27:49.830 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.828+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b040b6590 msgr2=0x7f5b040b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.828+0000 7f5b24e00640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b040b6590 0x7f5b040b8aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5b180043b0 tx=0x7f5b180041b0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.828+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2010b6d0 msgr2=0x7f5b2019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.828+0000 7f5b24e00640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2010b6d0 0x7f5b2019ccf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5b2006ca80 tx=0x7f5b0800ee50 comp rx=0 tx=0).stop 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 shutdown_connections 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2010b6d0 0x7f5b2019ccf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5b040b6590 0x7f5b040b8aa0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2010c020 0x7f5b201a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 --2- 172.21.15.27:0/1701022193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b20105d90 0x7f5b2019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 >> 172.21.15.27:0/1701022193 conn(0x7f5b20101860 msgr2=0x7f5b20103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:49.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 shutdown_connections 2024-09-21T07:27:49.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:49.829+0000 7f5b24e00640 1 -- 172.21.15.27:0/1701022193 wait complete. 2024-09-21T07:27:50.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:27:50.806 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:27:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:51.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:50 smithi027 ceph-mon[30779]: pgmap v840: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-21T07:27:51.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:50 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:27:51.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:50 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:51.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:50 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:27:51.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:50 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:50 smithi153 ceph-mon[34747]: pgmap v840: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-21T07:27:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:50 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:27:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:50 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:50 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:27:51.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:50 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:50 smithi119 ceph-mon[34519]: pgmap v840: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-21T07:27:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:50 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:27:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:50 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:50 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:27:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:50 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:27:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:51.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 -- 172.21.15.27:0/57620212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc105d80 msgr2=0x7f25fc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:51.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 --2- 172.21.15.27:0/57620212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc105d80 0x7f25fc106190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f25ec009900 tx=0x7f25ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 -- 172.21.15.27:0/57620212 shutdown_connections 2024-09-21T07:27:51.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 --2- 172.21.15.27:0/57620212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc10c010 0x7f25fc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 --2- 172.21.15.27:0/57620212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25fc10b6c0 0x7f25fc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 --2- 172.21.15.27:0/57620212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc105d80 0x7f25fc106190 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 -- 172.21.15.27:0/57620212 >> 172.21.15.27:0/57620212 conn(0x7f25fc101810 msgr2=0x7f25fc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 -- 172.21.15.27:0/57620212 shutdown_connections 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.573+0000 7f2600a00640 1 -- 172.21.15.27:0/57620212 wait complete. 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.574+0000 7f2600a00640 1 Processor -- start 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f2600a00640 1 -- start start 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f2600a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc105d80 0x7f25fc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f2600a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc10b6c0 0x7f25fc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f2600a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25fc10c010 0x7f25fc1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f2600a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc115cd0 con 0x7f25fc105d80 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f2600a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc115b50 con 0x7f25fc10c010 2024-09-21T07:27:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f2600a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc115e50 con 0x7f25fc10b6c0 2024-09-21T07:27:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc105d80 0x7f25fc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc105d80 0x7f25fc19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36794/0 (socket says 172.21.15.27:36794) 2024-09-21T07:27:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.575+0000 7f25f9e00640 1 -- 172.21.15.27:0/2737465301 learned_addr learned my addr 172.21.15.27:0/2737465301 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.576+0000 7f25f9400640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc10b6c0 0x7f25fc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.576+0000 7f25fa800640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25fc10c010 0x7f25fc1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:51.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.576+0000 7f25f9e00640 1 -- 172.21.15.27:0/2737465301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc10b6c0 msgr2=0x7f25fc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:51.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.576+0000 7f25f9e00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc10b6c0 0x7f25fc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.576+0000 7f25f9e00640 1 -- 172.21.15.27:0/2737465301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25fc10c010 msgr2=0x7f25fc1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.576+0000 7f25f9e00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25fc10c010 0x7f25fc1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.576+0000 7f25f9e00640 1 -- 172.21.15.27:0/2737465301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc1a2630 con 0x7f25fc105d80 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.577+0000 7f25f9e00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc105d80 0x7f25fc19c6c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005940 tx=0x7f25ec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.577+0000 7f25f2a00640 1 -- 172.21.15.27:0/2737465301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec03f030 con 0x7f25fc105d80 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.577+0000 7f25f2a00640 1 -- 172.21.15.27:0/2737465301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25ec0044c0 con 0x7f25fc105d80 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.577+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc1a28c0 con 0x7f25fc105d80 2024-09-21T07:27:51.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.577+0000 7f25f2a00640 1 -- 172.21.15.27:0/2737465301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec035070 con 0x7f25fc105d80 2024-09-21T07:27:51.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.577+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc1a9080 con 0x7f25fc105d80 2024-09-21T07:27:51.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.579+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc106190 con 0x7f25fc105d80 2024-09-21T07:27:51.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.581+0000 7f25f2a00640 1 -- 172.21.15.27:0/2737465301 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f25ec042030 con 0x7f25fc105d80 2024-09-21T07:27:51.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.584+0000 7f25f2a00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f25e00b6680 0x7f25e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:51.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.584+0000 7f25f2a00640 1 -- 172.21.15.27:0/2737465301 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f25ec107e60 con 0x7f25fc105d80 2024-09-21T07:27:51.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.584+0000 7f25f9400640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f25e00b6680 0x7f25e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:51.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.585+0000 7f25f9400640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f25e00b6680 0x7f25e00b8b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f25e80095f0 tx=0x7f25e8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:51.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.589+0000 7f25f2a00640 1 -- 172.21.15.27:0/2737465301 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25ec0d23e0 con 0x7f25fc105d80 2024-09-21T07:27:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.829+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25fc10dde0 con 0x7f25fc105d80 2024-09-21T07:27:51.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.830+0000 7f25f2a00640 1 -- 172.21.15.27:0/2737465301 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f25ec0d7320 con 0x7f25fc105d80 2024-09-21T07:27:51.832 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:27:51.833 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:27:51.833 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:27:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.832+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f25e00b6680 msgr2=0x7f25e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f25e00b6680 0x7f25e00b8b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f25e80095f0 tx=0x7f25e8005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc105d80 msgr2=0x7f25fc19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc105d80 0x7f25fc19c6c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005940 tx=0x7f25ec0040c0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 shutdown_connections 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25fc10b6c0 0x7f25fc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f25e00b6680 0x7f25e00b8b90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25fc10c010 0x7f25fc1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 --2- 172.21.15.27:0/2737465301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25fc105d80 0x7f25fc19c6c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 >> 172.21.15.27:0/2737465301 conn(0x7f25fc101810 msgr2=0x7f25fc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 shutdown_connections 2024-09-21T07:27:51.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:51.833+0000 7f2600a00640 1 -- 172.21.15.27:0/2737465301 wait complete. 2024-09-21T07:27:52.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:51 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2737465301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:52.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:51 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2737465301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:52.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:51 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2737465301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:27:52.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 -- 172.21.15.27:0/2469417380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c105bb0 msgr2=0x7f383c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 --2- 172.21.15.27:0/2469417380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c105bb0 0x7f383c105fc0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f382c009900 tx=0x7f382c02a410 comp rx=0 tx=0).stop 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 -- 172.21.15.27:0/2469417380 shutdown_connections 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 --2- 172.21.15.27:0/2469417380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c10be40 0x7f383c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 --2- 172.21.15.27:0/2469417380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f383c10b4f0 0x7f383c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 --2- 172.21.15.27:0/2469417380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c105bb0 0x7f383c105fc0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 -- 172.21.15.27:0/2469417380 >> 172.21.15.27:0/2469417380 conn(0x7f383c101680 msgr2=0x7f383c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.376+0000 7f383be00640 1 -- 172.21.15.27:0/2469417380 shutdown_connections 2024-09-21T07:27:52.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.377+0000 7f383be00640 1 -- 172.21.15.27:0/2469417380 wait complete. 2024-09-21T07:27:52.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 Processor -- start 2024-09-21T07:27:52.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 -- start start 2024-09-21T07:27:52.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c105bb0 0x7f383c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f383c10b4f0 0x7f383c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c10be40 0x7f383c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f383c115b50 con 0x7f383c10be40 2024-09-21T07:27:52.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1159d0 con 0x7f383c10b4f0 2024-09-21T07:27:52.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.378+0000 7f383be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f383c115cd0 con 0x7f383c105bb0 2024-09-21T07:27:52.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c105bb0 0x7f383c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c10be40 0x7f383c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c105bb0 0x7f383c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57904/0 (socket says 172.21.15.27:57904) 2024-09-21T07:27:52.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 -- 172.21.15.27:0/3157746485 learned_addr learned my addr 172.21.15.27:0/3157746485 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f3833e00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f383c10b4f0 0x7f383c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 -- 172.21.15.27:0/3157746485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f383c10b4f0 msgr2=0x7f383c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f383c10b4f0 0x7f383c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 -- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c10be40 msgr2=0x7f383c1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c10be40 0x7f383c1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383aa00640 1 -- 172.21.15.27:0/3157746485 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c1a2490 con 0x7f383c105bb0 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.379+0000 7f383b400640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c10be40 0x7f383c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.380+0000 7f383aa00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c105bb0 0x7f383c19c520 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f382c005ab0 tx=0x7f382c003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:52.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.380+0000 7f3833400640 1 -- 172.21.15.27:0/3157746485 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c03f070 con 0x7f383c105bb0 2024-09-21T07:27:52.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.380+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c1a2720 con 0x7f383c105bb0 2024-09-21T07:27:52.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.380+0000 7f3833400640 1 -- 172.21.15.27:0/3157746485 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382c031d60 con 0x7f383c105bb0 2024-09-21T07:27:52.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.380+0000 7f3833400640 1 -- 172.21.15.27:0/3157746485 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c035030 con 0x7f383c105bb0 2024-09-21T07:27:52.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.380+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c1a8ee0 con 0x7f383c105bb0 2024-09-21T07:27:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.383+0000 7f3848400740 1 Processor -- start 2024-09-21T07:27:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.383+0000 7f3848400740 1 -- start start 2024-09-21T07:27:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3848400740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0cab12070 0x55a0caeb8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3848400740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caeb9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3848400740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caeb99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3848400740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a0cae2ff70 con 0x55a0cab12070 2024-09-21T07:27:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3848400740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a0cae1f3a0 con 0x55a0caaf1390 2024-09-21T07:27:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3848400740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a0cae231b0 con 0x55a0caae91b0 2024-09-21T07:27:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3833400640 1 -- 172.21.15.27:0/3157746485 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f382c042070 con 0x7f383c105bb0 2024-09-21T07:27:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f383b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0cab12070 0x55a0caeb8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f383b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0cab12070 0x55a0caeb8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36818/0 (socket says 172.21.15.27:36818) 2024-09-21T07:27:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f383b400640 1 -- 172.21.15.27:0/1536616599 learned_addr learned my addr 172.21.15.27:0/1536616599 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f3833e00640 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caeb99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.384+0000 7f383aa00640 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caeb9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f383b400640 1 -- 172.21.15.27:0/1536616599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 msgr2=0x55a0caeb9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f383b400640 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caeb9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f383b400640 1 -- 172.21.15.27:0/1536616599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 msgr2=0x55a0caeb99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f383b400640 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caeb99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f383b400640 1 -- 172.21.15.27:0/1536616599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cade9c90 con 0x55a0cab12070 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f383b400640 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0cab12070 0x55a0caeb8f30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3834010750 tx=0x7f3834010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1137b5367f6e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f382be00640 1 -- 172.21.15.27:0/1536616599 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383400ead0 con 0x55a0cab12070 2024-09-21T07:27:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f382be00640 1 -- 172.21.15.27:0/1536616599 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3834010e60 con 0x55a0cab12070 2024-09-21T07:27:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.385+0000 7f382be00640 1 -- 172.21.15.27:0/1536616599 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340113e0 con 0x55a0cab12070 2024-09-21T07:27:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3833400640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f38200b6680 0x7f38200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3848400740 1 -- 172.21.15.27:0/1536616599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0cab12070 msgr2=0x55a0caeb8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3848400740 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0cab12070 0x55a0caeb8f30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3834010750 tx=0x7f3834010c10 comp rx=0 tx=0).stop 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3833400640 1 -- 172.21.15.27:0/3157746485 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f382c108320 con 0x7f383c105bb0 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3848400740 1 -- 172.21.15.27:0/1536616599 shutdown_connections 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3848400740 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caeb9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3848400740 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caeb99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f383aa00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f38200b6680 0x7f38200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.386+0000 7f3848400740 1 --2- 172.21.15.27:0/1536616599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0cab12070 0x55a0caeb8f30 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.387+0000 7f3848400740 1 -- 172.21.15.27:0/1536616599 >> 172.21.15.27:0/1536616599 conn(0x55a0cab06460 msgr2=0x55a0caeb6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.387+0000 7f3848400740 1 -- 172.21.15.27:0/1536616599 shutdown_connections 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.387+0000 7f3848400740 1 -- 172.21.15.27:0/1536616599 wait complete. 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.387+0000 7f383aa00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f38200b6680 0x7f38200b8b90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55a0caeb7bb0 tx=0x7f382c03b890 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 -- start start 2024-09-21T07:27:52.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 Processor -- start 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caae91b0 0x55a0caec4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caec4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0cab12070 0x55a0caed0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a0cac80700 con 0x55a0caae91b0 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a0cadba540 con 0x55a0caaf1390 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a0caec7710 con 0x55a0cab12070 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.388+0000 7f3848400740 10 client.0 _ll_register_callbacks cb 0x55a0cadca430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caec4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caec4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60512/0 (socket says 172.21.15.27:60512) 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 -- 172.21.15.27:0/1716175864 learned_addr learned my addr 172.21.15.27:0/1716175864 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f383b400640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0cab12070 0x55a0caed0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f383b400640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caae91b0 0x55a0caec4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 -- 172.21.15.27:0/1716175864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0cab12070 msgr2=0x55a0caed0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0cab12070 0x55a0caed0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 -- 172.21.15.27:0/1716175864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caae91b0 msgr2=0x55a0caec4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caae91b0 0x55a0caec4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.389+0000 7f3833e00640 1 -- 172.21.15.27:0/1716175864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cab7d650 con 0x55a0caaf1390 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.390+0000 7f383b400640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caae91b0 0x55a0caec4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.390+0000 7f3833e00640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caec4bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3824012700 tx=0x7f3824012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc5673d8ae6bb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.390+0000 7f3828c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.390+0000 7f3828c00640 1 -- 172.21.15.27:0/1716175864 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382400f920 con 0x55a0caaf1390 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.390+0000 7f3828c00640 1 -- 172.21.15.27:0/1716175864 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f382400fac0 con 0x55a0caaf1390 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.390+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0cadbcb50 con 0x55a0caaf1390 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.391+0000 7f3828c00640 1 -- 172.21.15.27:0/1716175864 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f382400fcc0 con 0x55a0caaf1390 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.391+0000 7f3848400740 10 client.29948 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.391+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0cadd7910 con 0x55a0caaf1390 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.391+0000 7f3828c00640 1 -- 172.21.15.27:0/1716175864 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a0cadbcb50 con 0x55a0caaf1390 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.391+0000 7f3828c00640 1 -- 172.21.15.27:0/1716175864 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f382401c420 con 0x55a0caaf1390 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.392+0000 7f3848400740 10 client.29948 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.392+0000 7f3848400740 10 client.29948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.392+0000 7f3848400740 20 client.29948 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.392+0000 7f3848400740 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0caad09f0 0x55a0caed68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.392+0000 7f3848400740 4 client.29948 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.392+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0cae31ec0 con 0x55a0caad09f0 2024-09-21T07:27:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.392+0000 7f383b400640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0caad09f0 0x55a0caed68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.393+0000 7f383b400640 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0caad09f0 0x55a0caed68f0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2652e0237cd0658 server_cookie=28552d6d3ede952d in_seq=0 out_seq=0 2024-09-21T07:27:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.393+0000 7f3828c00640 10 client.29948 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:52.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3828c00640 1 -- 172.21.15.27:0/1716175864 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a0cae31ec0 con 0x55a0caad09f0 2024-09-21T07:27:52.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3828c00640 10 client.29948 handle_command_reply: tid=0 2024-09-21T07:27:52.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 client.29948 shutdown 2024-09-21T07:27:52.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 msgr2=0x55a0caec4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caec4bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3824012700 tx=0x7f3824012bc0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 shutdown_connections 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0caad09f0 0x55a0caed68f0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0cab12070 0x55a0caed0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caaf1390 0x55a0caec4bb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3828c00640 0 client.29948 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 --2- 172.21.15.27:0/1716175864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caae91b0 0x55a0caec4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 >> 172.21.15.27:0/1716175864 conn(0x55a0cab06460 msgr2=0x55a0caebc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 shutdown_connections 2024-09-21T07:27:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.394+0000 7f3848400740 1 -- 172.21.15.27:0/1716175864 wait complete. 2024-09-21T07:27:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.395+0000 7f3848400740 20 client.29948 trim_cache size 0 max 16384 2024-09-21T07:27:52.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 Processor -- start 2024-09-21T07:27:52.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 -- start start 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caad09f0 0x55a0caebfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caec4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caaf1390 0x55a0caed0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b5830 con 0x55a0caaf1390 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b3100 con 0x55a0caad09f0 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.414+0000 7f3848400740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b0f70 con 0x55a0caae91b0 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caec4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caec4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57918/0 (socket says 172.21.15.27:57918) 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 -- 172.21.15.27:0/1956761686 learned_addr learned my addr 172.21.15.27:0/1956761686 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f383b400640 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caaf1390 0x55a0caed0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f383b400640 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caad09f0 0x55a0caebfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 -- 172.21.15.27:0/1956761686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caad09f0 msgr2=0x55a0caebfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caad09f0 0x55a0caebfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 -- 172.21.15.27:0/1956761686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caaf1390 msgr2=0x55a0caed0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caaf1390 0x55a0caed0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f3833e00640 1 -- 172.21.15.27:0/1956761686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0caed17a0 con 0x55a0caae91b0 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.415+0000 7f383b400640 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caaf1390 0x55a0caed0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.416+0000 7f3833e00640 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caec4b30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3824001eb0 tx=0x7f382401dfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c78adba7be3701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.416+0000 7f382b400640 1 -- 172.21.15.27:0/1956761686 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382401cad0 con 0x55a0caae91b0 2024-09-21T07:27:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.416+0000 7f382b400640 1 -- 172.21.15.27:0/1956761686 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382401cc70 con 0x55a0caae91b0 2024-09-21T07:27:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.416+0000 7f382b400640 1 -- 172.21.15.27:0/1956761686 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824003990 con 0x55a0caae91b0 2024-09-21T07:27:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 -- 172.21.15.27:0/1956761686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 msgr2=0x55a0caec4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caec4b30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3824001eb0 tx=0x7f382401dfb0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 -- 172.21.15.27:0/1956761686 shutdown_connections 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caae91b0 0x55a0caec4b30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caaf1390 0x55a0caed0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 --2- 172.21.15.27:0/1956761686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caad09f0 0x55a0caebfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 -- 172.21.15.27:0/1956761686 >> 172.21.15.27:0/1956761686 conn(0x55a0cab06460 msgr2=0x55a0caec4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 -- 172.21.15.27:0/1956761686 shutdown_connections 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 -- 172.21.15.27:0/1956761686 wait complete. 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.417+0000 7f3848400740 1 -- start start 2024-09-21T07:27:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 1 Processor -- start 2024-09-21T07:27:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caad09f0 0x55a0caecc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caae91b0 0x55a0caecceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caaf1390 0x55a0caec7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b15c0 con 0x55a0caad09f0 2024-09-21T07:27:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b2a30 con 0x55a0caae91b0 2024-09-21T07:27:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b1c90 con 0x55a0caaf1390 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.418+0000 7f3848400740 10 client.0 _ll_register_callbacks cb 0x7f3824009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f383b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caad09f0 0x55a0caecc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f3833e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caae91b0 0x55a0caecceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f383b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caad09f0 0x55a0caecc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36844/0 (socket says 172.21.15.27:36844) 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f383b400640 1 -- 172.21.15.27:0/1402034557 learned_addr learned my addr 172.21.15.27:0/1402034557 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f383b400640 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caaf1390 0x55a0caec7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f3833e00640 1 -- 172.21.15.27:0/1402034557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caaf1390 msgr2=0x55a0caec7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f3833e00640 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caaf1390 0x55a0caec7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f3833e00640 1 -- 172.21.15.27:0/1402034557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caad09f0 msgr2=0x55a0caecc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f3833e00640 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caad09f0 0x55a0caecc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.419+0000 7f3833e00640 1 -- 172.21.15.27:0/1402034557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0caecd470 con 0x55a0caae91b0 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3833e00640 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caae91b0 0x55a0caecceb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f382401c9d0 tx=0x7f3824005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e17eab790060356a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3828c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3828c00640 1 -- 172.21.15.27:0/1402034557 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382401cad0 con 0x55a0caae91b0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382401cc70 con 0x55a0caae91b0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3828c00640 1 -- 172.21.15.27:0/1402034557 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f38240054d0 con 0x55a0caae91b0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3828c00640 1 -- 172.21.15.27:0/1402034557 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3824039050 con 0x55a0caae91b0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3848400740 10 client.29954 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.420+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0caf41600 con 0x55a0caae91b0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3828c00640 1 -- 172.21.15.27:0/1402034557 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382401cc70 con 0x55a0caae91b0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3828c00640 1 -- 172.21.15.27:0/1402034557 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f382402a4d0 con 0x55a0caae91b0 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3848400740 10 client.29954 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3848400740 10 client.29954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3848400740 20 client.29954 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3848400740 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0cab12070 0x55a0caf43c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:52.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3848400740 4 client.29954 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:27:52.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0cae2d970 con 0x55a0cab12070 2024-09-21T07:27:52.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.421+0000 7f383b400640 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0cab12070 0x55a0caf43c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:52.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f383b400640 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0cab12070 0x55a0caf43c10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6581a81cb082de1a server_cookie=82d52c8a676bcfa2 in_seq=0 out_seq=0 2024-09-21T07:27:52.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3828c00640 10 client.29954 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:52.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3828c00640 1 -- 172.21.15.27:0/1402034557 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7102 (crc 0 0 0) 0x55a0cae2d970 con 0x55a0cab12070 2024-09-21T07:27:52.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3828c00640 10 client.29954 handle_command_reply: tid=0 2024-09-21T07:27:52.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 client.29954 shutdown 2024-09-21T07:27:52.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caae91b0 msgr2=0x55a0caecceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caae91b0 0x55a0caecceb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f382401c9d0 tx=0x7f3824005250 comp rx=0 tx=0).stop 2024-09-21T07:27:52.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 shutdown_connections 2024-09-21T07:27:52.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a0cab12070 0x55a0caf43c10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0caaf1390 0x55a0caec7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0caae91b0 0x55a0caecceb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 --2- 172.21.15.27:0/1402034557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a0caad09f0 0x55a0caecc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3828c00640 0 client.29954 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.422+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 >> 172.21.15.27:0/1402034557 conn(0x55a0cab06460 msgr2=0x55a0caeb55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.423+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 shutdown_connections 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.423+0000 7f3848400740 1 -- 172.21.15.27:0/1402034557 wait complete. 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.423+0000 7f3848400740 20 client.29954 trim_cache size 0 max 16384 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "23d6d7e9-06b5-4a6e-adfb-86c244736c20": { 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "23d6d7e9-06b5-4a6e-adfb-86c244736c20", 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "23d6d7e9-06b5-4a6e-adfb-86c244736c20": { 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:49.824406+0000", 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:27:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:27:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:27:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:52.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.423+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f38200b6680 msgr2=0x7f38200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.423+0000 7f383be00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f38200b6680 0x7f38200b8b90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55a0caeb7bb0 tx=0x7f382c03b890 comp rx=0 tx=0).stop 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.423+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c105bb0 msgr2=0x7f383c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.423+0000 7f383be00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c105bb0 0x7f383c19c520 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f382c005ab0 tx=0x7f382c003c80 comp rx=0 tx=0).stop 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 shutdown_connections 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f38200b6680 0x7f38200b8b90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383c10be40 0x7f383c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f383c10b4f0 0x7f383c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 --2- 172.21.15.27:0/3157746485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383c105bb0 0x7f383c19c520 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 >> 172.21.15.27:0/3157746485 conn(0x7f383c101680 msgr2=0x7f383c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 shutdown_connections 2024-09-21T07:27:52.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:52.424+0000 7f383be00640 1 -- 172.21.15.27:0/3157746485 wait complete. 2024-09-21T07:27:52.803 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 206a05d --release --await' 2024-09-21T07:27:52.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 206a05d --release --await 2024-09-21T07:27:53.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:52 smithi027 ceph-mon[30779]: pgmap v841: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:52 smithi153 ceph-mon[34747]: pgmap v841: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:52 smithi119 ceph-mon[34519]: pgmap v841: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:53.555 INFO:tasks.cephfs.filesystem:scrub status for tag:23d6d7e9-06b5-4a6e-adfb-86c244736c20 - {'path': '/', 'tag': '23d6d7e9-06b5-4a6e-adfb-86c244736c20', 'options': 'recursive,force'} 2024-09-21T07:27:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:27:54.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 -- 172.21.15.27:0/1894632515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 msgr2=0x7fab8c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 --2- 172.21.15.27:0/1894632515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 0x7fab8c110450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fab84009be0 tx=0x7fab8402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 -- 172.21.15.27:0/1894632515 shutdown_connections 2024-09-21T07:27:54.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 --2- 172.21.15.27:0/1894632515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 0x7fab8c110450 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 --2- 172.21.15.27:0/1894632515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab8c1092d0 0x7fab8c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 --2- 172.21.15.27:0/1894632515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c103990 0x7fab8c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 -- 172.21.15.27:0/1894632515 >> 172.21.15.27:0/1894632515 conn(0x7fab8c0ff440 msgr2=0x7fab8c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:54.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 -- 172.21.15.27:0/1894632515 shutdown_connections 2024-09-21T07:27:54.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.306+0000 7fab90e00640 1 -- 172.21.15.27:0/1894632515 wait complete. 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.307+0000 7fab90e00640 1 Processor -- start 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.307+0000 7fab90e00640 1 -- start start 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab90e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab8c103990 0x7fab8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab90e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c1092d0 0x7fab8c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab90e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 0x7fab8c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab90e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1138e0 con 0x7fab8c1092d0 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab90e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c113760 con 0x7fab8c103990 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab90e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c113a60 con 0x7fab8c109c20 2024-09-21T07:27:54.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab8aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c1092d0 0x7fab8c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab8aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c1092d0 0x7fab8c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36868/0 (socket says 172.21.15.27:36868) 2024-09-21T07:27:54.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab8aa00640 1 -- 172.21.15.27:0/1344545065 learned_addr learned my addr 172.21.15.27:0/1344545065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:54.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab8b400640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab8c103990 0x7fab8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.308+0000 7fab8be00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 0x7fab8c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab8b400640 1 -- 172.21.15.27:0/1344545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 msgr2=0x7fab8c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab8b400640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 0x7fab8c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab8b400640 1 -- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c1092d0 msgr2=0x7fab8c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab8b400640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c1092d0 0x7fab8c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab8b400640 1 -- 172.21.15.27:0/1344545065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c1a8650 con 0x7fab8c103990 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab8aa00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c1092d0 0x7fab8c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab8b400640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab8c103990 0x7fab8c075710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fab7c005fd0 tx=0x7fab7c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab83e00640 1 -- 172.21.15.27:0/1344545065 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00cd00 con 0x7fab8c103990 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.309+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c1a8940 con 0x7fab8c103990 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.310+0000 7fab83e00640 1 -- 172.21.15.27:0/1344545065 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c0035f0 con 0x7fab8c103990 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.310+0000 7fab83e00640 1 -- 172.21.15.27:0/1344545065 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c010670 con 0x7fab8c103990 2024-09-21T07:27:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.310+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a8ed0 con 0x7fab8c103990 2024-09-21T07:27:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 Processor -- start 2024-09-21T07:27:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 -- start start 2024-09-21T07:27:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098506340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098506880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c098506dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c09847d290 con 0x55c098136660 2024-09-21T07:27:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c09846c6c0 con 0x55c09815f520 2024-09-21T07:27:54.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.313+0000 7fab99033740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c0984704d0 con 0x55c09813e840 2024-09-21T07:27:54.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098506340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab83e00640 1 -- 172.21.15.27:0/1344545065 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fab7c0037a0 con 0x7fab8c103990 2024-09-21T07:27:54.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098506340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60556/0 (socket says 172.21.15.27:60556) 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 -- 172.21.15.27:0/3559265746 learned_addr learned my addr 172.21.15.27:0/3559265746 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8aa00640 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c098506dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 -- 172.21.15.27:0/3559265746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 msgr2=0x55c098506dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c098506dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 -- 172.21.15.27:0/3559265746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 msgr2=0x55c098506880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098506880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8be00640 1 -- 172.21.15.27:0/3559265746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0983f9460 con 0x55c09815f520 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.314+0000 7fab8aa00640 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c098506dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:54.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.315+0000 7fab8be00640 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098506340 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fab8402acb0 tx=0x7fab84030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4688a0c1cdc2cb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.315+0000 7fab80c00640 1 -- 172.21.15.27:0/3559265746 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84041030 con 0x55c09815f520 2024-09-21T07:27:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.315+0000 7fab80c00640 1 -- 172.21.15.27:0/3559265746 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84003e20 con 0x55c09815f520 2024-09-21T07:27:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.315+0000 7fab80c00640 1 -- 172.21.15.27:0/3559265746 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84036040 con 0x55c09815f520 2024-09-21T07:27:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab83e00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab700b6680 0x7fab700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 -- 172.21.15.27:0/3559265746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 msgr2=0x55c098506340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098506340 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fab8402acb0 tx=0x7fab84030420 comp rx=0 tx=0).stop 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 -- 172.21.15.27:0/3559265746 shutdown_connections 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c098506dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab83e00640 1 -- 172.21.15.27:0/1344545065 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fab7c0e7cf0 con 0x7fab8c103990 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098506880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 --2- 172.21.15.27:0/3559265746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098506340 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 -- 172.21.15.27:0/3559265746 >> 172.21.15.27:0/3559265746 conn(0x55c098153910 msgr2=0x55c098504230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab8b400640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab700b6680 0x7fab700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 -- 172.21.15.27:0/3559265746 shutdown_connections 2024-09-21T07:27:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.316+0000 7fab99033740 1 -- 172.21.15.27:0/3559265746 wait complete. 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.317+0000 7fab8b400640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab700b6680 0x7fab700b8b90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55c098504fc0 tx=0x7fab7c00cf50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.317+0000 7fab99033740 1 -- start start 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.317+0000 7fab99033740 1 Processor -- start 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab99033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c098136660 0x55c098511f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab99033740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c09813e840 0x55c098512480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab99033740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab99033740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c0982cdbb0 con 0x55c09813e840 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab99033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c0984076c0 con 0x55c098136660 2024-09-21T07:27:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab99033740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c098514ac0 con 0x55c09815f520 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab99033740 10 client.0 _ll_register_callbacks cb 0x55c098422490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab8aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c09813e840 0x55c098512480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c098136660 0x55c098511f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab8aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c09813e840 0x55c098512480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36896/0 (socket says 172.21.15.27:36896) 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab8aa00640 1 -- 172.21.15.27:0/1014507261 learned_addr learned my addr 172.21.15.27:0/1014507261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c098136660 0x55c098511f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:60568/0 (socket says 172.21.15.27:60568) 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.318+0000 7fab8be00640 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab8aa00640 1 -- 172.21.15.27:0/1014507261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 msgr2=0x55c09851ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab8aa00640 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab8aa00640 1 -- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c098136660 msgr2=0x55c098511f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab8aa00640 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c098136660 0x55c098511f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab8aa00640 1 -- 172.21.15.27:0/1014507261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0981cab00 con 0x55c09813e840 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab8be00640 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c098136660 0x55c098511f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:54.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab8aa00640 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c09813e840 0x55c098512480 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fab780129e0 tx=0x7fab78012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90e349c633b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab75600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:27:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab75600640 1 -- 172.21.15.27:0/1014507261 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800fa30 con 0x55c09813e840 2024-09-21T07:27:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab75600640 1 -- 172.21.15.27:0/1014507261 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fab7800fbd0 con 0x55c09813e840 2024-09-21T07:27:54.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab84003e20 con 0x55c09813e840 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.319+0000 7fab75600640 1 -- 172.21.15.27:0/1014507261 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab7800fdd0 con 0x55c09813e840 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 10 client.19944 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c098241960 con 0x55c09813e840 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab75600640 1 -- 172.21.15.27:0/1014507261 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84003e20 con 0x55c09813e840 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab75600640 1 -- 172.21.15.27:0/1014507261 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab78004c60 con 0x55c09813e840 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 10 client.19944 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 10 client.19944 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 20 client.19944 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c09811dea0 0x55c098523e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 4 client.19944 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.320+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c09847f1e0 con 0x55c09811dea0 2024-09-21T07:27:54.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.321+0000 7fab8be00640 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c09811dea0 0x55c098523e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.321+0000 7fab8be00640 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c09811dea0 0x55c098523e00 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a7988507807702 server_cookie=aca3f4c8cdaa5b61 in_seq=0 out_seq=0 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.321+0000 7fab75600640 10 client.19944 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab75600640 1 -- 172.21.15.27:0/1014507261 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c09847f1e0 con 0x55c09811dea0 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab75600640 10 client.19944 handle_command_reply: tid=0 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 client.19944 shutdown 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c09813e840 msgr2=0x55c098512480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c09813e840 0x55c098512480 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fab780129e0 tx=0x7fab78012ea0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 shutdown_connections 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c09811dea0 0x55c098523e00 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c09813e840 0x55c098512480 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 --2- 172.21.15.27:0/1014507261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c098136660 0x55c098511f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 >> 172.21.15.27:0/1014507261 conn(0x55c098153910 msgr2=0x55c098509520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.322+0000 7fab75600640 0 client.19944 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.323+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 shutdown_connections 2024-09-21T07:27:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.323+0000 7fab99033740 1 -- 172.21.15.27:0/1014507261 wait complete. 2024-09-21T07:27:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.323+0000 7fab99033740 20 client.19944 trim_cache size 0 max 16384 2024-09-21T07:27:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 Processor -- start 2024-09-21T07:27:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 -- start start 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c09850c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09813e840 0x55c09851dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1b5710 con 0x55c098136660 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1b2fe0 con 0x55c09813e840 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.341+0000 7fab99033740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1b0e50 con 0x55c09815f520 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c09850c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c09850c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36900/0 (socket says 172.21.15.27:36900) 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 -- 172.21.15.27:0/560605862 learned_addr learned my addr 172.21.15.27:0/560605862 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09813e840 0x55c09851dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 -- 172.21.15.27:0/560605862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 msgr2=0x55c09851e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 -- 172.21.15.27:0/560605862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09813e840 msgr2=0x55c09851dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09813e840 0x55c09851dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 -- 172.21.15.27:0/560605862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09850bfa0 con 0x55c098136660 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8aa00640 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09813e840 0x55c09851dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.342+0000 7fab8be00640 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c09850c400 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fab84005c60 tx=0x7fab8402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a21029ad20a2a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.343+0000 7fab77e00640 1 -- 172.21.15.27:0/560605862 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84036030 con 0x55c098136660 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.343+0000 7fab77e00640 1 -- 172.21.15.27:0/560605862 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab8404a070 con 0x55c098136660 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.343+0000 7fab77e00640 1 -- 172.21.15.27:0/560605862 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84041040 con 0x55c098136660 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 -- 172.21.15.27:0/560605862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 msgr2=0x55c09850c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c09850c400 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fab84005c60 tx=0x7fab8402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 -- 172.21.15.27:0/560605862 shutdown_connections 2024-09-21T07:27:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09815f520 0x55c09851e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09813e840 0x55c09851dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 --2- 172.21.15.27:0/560605862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c09850c400 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 -- 172.21.15.27:0/560605862 >> 172.21.15.27:0/560605862 conn(0x55c098153910 msgr2=0x55c0985105c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 -- 172.21.15.27:0/560605862 shutdown_connections 2024-09-21T07:27:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 -- 172.21.15.27:0/560605862 wait complete. 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 -- start start 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.344+0000 7fab99033740 1 Processor -- start 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab99033740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098515d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab99033740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c0985162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab99033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098516810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab99033740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1b14a0 con 0x55c098136660 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab99033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1b2910 con 0x55c09815f520 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab99033740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1b1b70 con 0x55c09813e840 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab99033740 10 client.0 _ll_register_callbacks cb 0x7fab78005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098515d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098515d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36902/0 (socket says 172.21.15.27:36902) 2024-09-21T07:27:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab8be00640 1 -- 172.21.15.27:0/435741025 learned_addr learned my addr 172.21.15.27:0/435741025 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.345+0000 7fab8aa00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c0985162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8be00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098516810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8aa00640 1 -- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 msgr2=0x55c098516810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8aa00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098516810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8aa00640 1 -- 172.21.15.27:0/435741025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 msgr2=0x55c098515d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8aa00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098515d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8aa00640 1 -- 172.21.15.27:0/435741025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09858d8b0 con 0x55c09813e840 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8be00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098515d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab8aa00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c0985162d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fab78012fd0 tx=0x7fab7801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecdb90e9d2a366bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab75600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.346+0000 7fab75600640 1 -- 172.21.15.27:0/435741025 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800f790 con 0x55c09813e840 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.347+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8404a070 con 0x55c09813e840 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.347+0000 7fab75600640 1 -- 172.21.15.27:0/435741025 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7fab7801cc60 con 0x55c09813e840 2024-09-21T07:27:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.347+0000 7fab75600640 1 -- 172.21.15.27:0/435741025 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab7801ce60 con 0x55c09813e840 2024-09-21T07:27:54.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.347+0000 7fab99033740 10 client.29929 fetch_fsmap learned FSMap version 27 2024-09-21T07:27:54.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.347+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c09858eb70 con 0x55c09813e840 2024-09-21T07:27:54.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.347+0000 7fab75600640 1 -- 172.21.15.27:0/435741025 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab8404a070 con 0x55c09813e840 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab75600640 1 -- 172.21.15.27:0/435741025 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab78031030 con 0x55c09813e840 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab99033740 10 client.29929 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab99033740 10 client.29929 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab99033740 20 client.29929 populate_metadata read hostname 'smithi027' 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab99033740 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c0980f3250 0x55c098591370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab99033740 4 client.29929 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "206a05d", "release": true, "await": true}] 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "206a05d", "release": true, "await": true}) -- 0x55c09847ac90 con 0x55c0980f3250 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.348+0000 7fab8be00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c0980f3250 0x55c098591370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.349+0000 7fab8be00640 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c0980f3250 0x55c098591370 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ddea2d8f25ae server_cookie=4a78036a7071cf45 in_seq=0 out_seq=0 2024-09-21T07:27:54.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.349+0000 7fab75600640 10 client.29929 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:54.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.352+0000 7fab75600640 1 -- 172.21.15.27:0/435741025 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c09847ac90 con 0x55c0980f3250 2024-09-21T07:27:54.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.352+0000 7fab75600640 10 client.29929 handle_command_reply: tid=0 2024-09-21T07:27:54.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.352+0000 7fab99033740 1 client.29929 shutdown 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 msgr2=0x55c0985162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c0985162d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fab78012fd0 tx=0x7fab7801c8b0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 shutdown_connections 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55c0980f3250 0x55c098591370 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c09813e840 0x55c0985162d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c09815f520 0x55c098516810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 --2- 172.21.15.27:0/435741025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c098136660 0x55c098515d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab75600640 0 client.29929 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 >> 172.21.15.27:0/435741025 conn(0x55c098153910 msgr2=0x55c098509440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:54.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 shutdown_connections 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 1 -- 172.21.15.27:0/435741025 wait complete. 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.353+0000 7fab99033740 20 client.29929 trim_cache size 0 max 16384 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 60, 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "206a05d": { 2024-09-21T07:27:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 60, 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 138.8, 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:27:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:27:54.358 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:27:54.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab700b6680 msgr2=0x7fab700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab700b6680 0x7fab700b8b90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55c098504fc0 tx=0x7fab7c00cf50 comp rx=0 tx=0).stop 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab8c103990 msgr2=0x7fab8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab8c103990 0x7fab8c075710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fab7c005fd0 tx=0x7fab7c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 shutdown_connections 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c109c20 0x7fab8c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fab700b6680 0x7fab700b8b90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab8c1092d0 0x7fab8c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 --2- 172.21.15.27:0/1344545065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab8c103990 0x7fab8c075710 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 >> 172.21.15.27:0/1344545065 conn(0x7fab8c0ff440 msgr2=0x7fab8c1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 shutdown_connections 2024-09-21T07:27:54.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:27:54.354+0000 7fab90e00640 1 -- 172.21.15.27:0/1344545065 wait complete. 2024-09-21T07:27:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:54 smithi153 ceph-mon[34747]: pgmap v842: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:55.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:54 smithi027 ceph-mon[30779]: pgmap v842: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:55.228 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '206a05d', total seconds elapsed: 23.8 2024-09-21T07:27:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:54 smithi119 ceph-mon[34519]: pgmap v842: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:56 smithi153 ceph-mon[34747]: pgmap v843: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:56 smithi119 ceph-mon[34519]: pgmap v843: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:57.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:56 smithi027 ceph-mon[30779]: pgmap v843: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:58 smithi153 ceph-mon[34747]: pgmap v844: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:59.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:27:58 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:27:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:58 smithi119 ceph-mon[34519]: pgmap v844: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:27:58 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:27:59.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:58 smithi027 ceph-mon[30779]: pgmap v844: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 KiB/s wr, 9 op/s 2024-09-21T07:27:59.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:27:58 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.071+0000 7f23eee00640 1 -- 172.21.15.27:0/3446877375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8109d50 msgr2=0x7f23e8110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.071+0000 7f23eee00640 1 --2- 172.21.15.27:0/3446877375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8109d50 0x7f23e8110580 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009ca0 tx=0x7f23e002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.072+0000 7f23eee00640 1 -- 172.21.15.27:0/3446877375 shutdown_connections 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.072+0000 7f23eee00640 1 --2- 172.21.15.27:0/3446877375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23e8109400 0x7f23e8109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.072+0000 7f23eee00640 1 --2- 172.21.15.27:0/3446877375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8109d50 0x7f23e8110580 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.072+0000 7f23eee00640 1 --2- 172.21.15.27:0/3446877375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23e8103ad0 0x7f23e8103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.072+0000 7f23eee00640 1 -- 172.21.15.27:0/3446877375 >> 172.21.15.27:0/3446877375 conn(0x7f23e80ff510 msgr2=0x7f23e8101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.072+0000 7f23eee00640 1 -- 172.21.15.27:0/3446877375 shutdown_connections 2024-09-21T07:28:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.072+0000 7f23eee00640 1 -- 172.21.15.27:0/3446877375 wait complete. 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 Processor -- start 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 -- start start 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8103ad0 0x7f23e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23e8109400 0x7f23e8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23e8109d50 0x7f23e8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8113a60 con 0x7f23e8103ad0 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81138e0 con 0x7f23e8109d50 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.073+0000 7f23eee00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8113be0 con 0x7f23e8109400 2024-09-21T07:28:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8103ad0 0x7f23e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8103ad0 0x7f23e8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48714/0 (socket says 172.21.15.27:48714) 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 -- 172.21.15.27:0/1968856016 learned_addr learned my addr 172.21.15.27:0/1968856016 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23ece00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23e8109d50 0x7f23e8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23df400640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23e8109400 0x7f23e8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 -- 172.21.15.27:0/1968856016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23e8109400 msgr2=0x7f23e8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23e8109400 0x7f23e8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 -- 172.21.15.27:0/1968856016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23e8109d50 msgr2=0x7f23e8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23e8109d50 0x7f23e8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23e7e00640 1 -- 172.21.15.27:0/1968856016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81a86b0 con 0x7f23e8103ad0 2024-09-21T07:28:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.074+0000 7f23ece00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23e8109d50 0x7f23e8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.075+0000 7f23e7e00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8103ad0 0x7f23e8075710 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f23d800e9f0 tx=0x7f23d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.075+0000 7f23e5600640 1 -- 172.21.15.27:0/1968856016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d800cd80 con 0x7f23e8103ad0 2024-09-21T07:28:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.075+0000 7f23e5600640 1 -- 172.21.15.27:0/1968856016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d80108e0 con 0x7f23e8103ad0 2024-09-21T07:28:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.075+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e81a89a0 con 0x7f23e8103ad0 2024-09-21T07:28:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.075+0000 7f23e5600640 1 -- 172.21.15.27:0/1968856016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8018980 con 0x7f23e8103ad0 2024-09-21T07:28:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.075+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81a8ef0 con 0x7f23e8103ad0 2024-09-21T07:28:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.078+0000 7f23f5fbe740 1 Processor -- start 2024-09-21T07:28:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.078+0000 7f23f5fbe740 1 -- start start 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.078+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda320c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.078+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda321170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.078+0000 7f23e5600640 1 -- 172.21.15.27:0/1968856016 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f23d8010430 con 0x7f23e8103ad0 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.078+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda3216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555fda2966b0 con 0x555fd9f5b4d0 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555fda285ae0 con 0x555fd9f532f0 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555fda2898f0 con 0x555fd9f7c1b0 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda320c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda320c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:37850/0 (socket says 172.21.15.27:37850) 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 -- 172.21.15.27:0/1260763156 learned_addr learned my addr 172.21.15.27:0/1260763156 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23df400640 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda3216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23e7e00640 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda321170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 -- 172.21.15.27:0/1260763156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 msgr2=0x555fda321170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda321170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 -- 172.21.15.27:0/1260763156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 msgr2=0x555fda3216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda3216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.079+0000 7f23ece00640 1 -- 172.21.15.27:0/1260763156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fda252780 con 0x555fd9f7c1b0 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.080+0000 7f23ece00640 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda320c30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f23e0035000 tx=0x7f23e0009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b2eea99ddb6e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.080+0000 7f23dd600640 1 -- 172.21.15.27:0/1260763156 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e002f3d0 con 0x555fd9f7c1b0 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.080+0000 7f23dd600640 1 -- 172.21.15.27:0/1260763156 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0036040 con 0x555fd9f7c1b0 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.080+0000 7f23dd600640 1 -- 172.21.15.27:0/1260763156 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0004480 con 0x555fd9f7c1b0 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.080+0000 7f23e5600640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23cc0b66b0 0x7f23cc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.080+0000 7f23e5600640 1 -- 172.21.15.27:0/1968856016 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f23d80e7de0 con 0x7f23e8103ad0 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23e7e00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23cc0b66b0 0x7f23cc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 -- 172.21.15.27:0/1260763156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 msgr2=0x555fda320c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda320c30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f23e0035000 tx=0x7f23e0009490 comp rx=0 tx=0).stop 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 -- 172.21.15.27:0/1260763156 shutdown_connections 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda3216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda321170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/1260763156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda320c30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 -- 172.21.15.27:0/1260763156 >> 172.21.15.27:0/1260763156 conn(0x555fd9f70300 msgr2=0x555fda31eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 -- 172.21.15.27:0/1260763156 shutdown_connections 2024-09-21T07:28:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23f5fbe740 1 -- 172.21.15.27:0/1260763156 wait complete. 2024-09-21T07:28:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.081+0000 7f23e7e00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23cc0b66b0 0x7f23cc0b8bc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f23d80e5230 tx=0x7f23d8018420 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 -- start start 2024-09-21T07:28:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 Processor -- start 2024-09-21T07:28:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda32c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda32c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda32cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555fda1bdab0 con 0x555fd9f5b4d0 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555fda22c570 con 0x555fd9f532f0 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555fda32f1d0 con 0x555fd9f7c1b0 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.082+0000 7f23f5fbe740 10 client.0 _ll_register_callbacks cb 0x555fda23b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda32c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda32c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48736/0 (socket says 172.21.15.27:48736) 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 -- 172.21.15.27:0/407584441 learned_addr learned my addr 172.21.15.27:0/407584441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23ece00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda32cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23ece00640 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda32c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 -- 172.21.15.27:0/407584441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 msgr2=0x555fda32cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda32cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 -- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 msgr2=0x555fda32c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda32c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 -- 172.21.15.27:0/407584441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd9fe7790 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23df400640 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda32c710 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f23d00126e0 tx=0x7f23d0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6ea288292aac14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.083+0000 7f23d6000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23d6000640 1 -- 172.21.15.27:0/407584441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0012190 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23d6000640 1 -- 172.21.15.27:0/407584441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f23d000f920 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e0036040 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23d6000640 1 -- 172.21.15.27:0/407584441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23d000fb20 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23f5fbe740 10 client.19962 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fda060620 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23d6000640 1 -- 172.21.15.27:0/407584441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0036040 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.084+0000 7f23d6000640 1 -- 172.21.15.27:0/407584441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f23d001c420 con 0x555fd9f5b4d0 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.085+0000 7f23f5fbe740 10 client.19962 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.085+0000 7f23f5fbe740 10 client.19962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.085+0000 7f23f5fbe740 20 client.19962 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.085+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f3ad80 0x555fda33e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.085+0000 7f23f5fbe740 4 client.19962 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.085+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fda298600 con 0x555fd9f3ad80 2024-09-21T07:28:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.085+0000 7f23ece00640 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f3ad80 0x555fda33e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.086+0000 7f23ece00640 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f3ad80 0x555fda33e180 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5744fe3e880cd9ae server_cookie=86c86e7b867b77ff in_seq=0 out_seq=0 2024-09-21T07:28:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.086+0000 7f23d6000640 10 client.19962 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.086+0000 7f23d6000640 1 -- 172.21.15.27:0/407584441 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fda298600 con 0x555fd9f3ad80 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.086+0000 7f23d6000640 10 client.19962 handle_command_reply: tid=0 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.086+0000 7f23f5fbe740 1 client.19962 shutdown 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 msgr2=0x555fda32c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda32c710 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f23d00126e0 tx=0x7f23d0012ba0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 shutdown_connections 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f3ad80 0x555fda33e180 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f7c1b0 0x555fda32cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f5b4d0 0x555fda32c710 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/407584441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda32c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23d6000640 0 client.19962 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 >> 172.21.15.27:0/407584441 conn(0x555fd9f70300 msgr2=0x555fda321c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 shutdown_connections 2024-09-21T07:28:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 1 -- 172.21.15.27:0/407584441 wait complete. 2024-09-21T07:28:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.087+0000 7f23f5fbe740 20 client.19962 trim_cache size 0 max 16384 2024-09-21T07:28:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.102+0000 7f23f5fbe740 1 Processor -- start 2024-09-21T07:28:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23f5fbe740 1 -- start start 2024-09-21T07:28:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda322620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f5b4d0 0x555fda32de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda3293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b5970 con 0x555fd9f7c1b0 2024-09-21T07:28:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b3240 con 0x555fd9f532f0 2024-09-21T07:28:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b10b0 con 0x555fd9f5b4d0 2024-09-21T07:28:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23ece00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda3293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23ece00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda3293a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48742/0 (socket says 172.21.15.27:48742) 2024-09-21T07:28:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23ece00640 1 -- 172.21.15.27:0/756508815 learned_addr learned my addr 172.21.15.27:0/756508815 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23ece00640 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda322620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.103+0000 7f23df400640 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f5b4d0 0x555fda32de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23ece00640 1 -- 172.21.15.27:0/756508815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f5b4d0 msgr2=0x555fda32de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23ece00640 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f5b4d0 0x555fda32de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23ece00640 1 -- 172.21.15.27:0/756508815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 msgr2=0x555fda322620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23ece00640 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda322620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23ece00640 1 -- 172.21.15.27:0/756508815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fda321a40 con 0x555fd9f7c1b0 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23ece00640 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda3293a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f23e00093d0 tx=0x7f23e002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3e0a650742bc9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23dcc00640 1 -- 172.21.15.27:0/756508815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0004160 con 0x555fd9f7c1b0 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23dcc00640 1 -- 172.21.15.27:0/756508815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0036040 con 0x555fd9f7c1b0 2024-09-21T07:28:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.104+0000 7f23dcc00640 1 -- 172.21.15.27:0/756508815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0003d60 con 0x555fd9f7c1b0 2024-09-21T07:28:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 -- 172.21.15.27:0/756508815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 msgr2=0x555fda3293a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda3293a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f23e00093d0 tx=0x7f23e002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 -- 172.21.15.27:0/756508815 shutdown_connections 2024-09-21T07:28:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f5b4d0 0x555fda32de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda3293a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/756508815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f532f0 0x555fda322620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 -- 172.21.15.27:0/756508815 >> 172.21.15.27:0/756508815 conn(0x555fd9f70300 msgr2=0x555fda32d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 -- 172.21.15.27:0/756508815 shutdown_connections 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 -- 172.21.15.27:0/756508815 wait complete. 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.105+0000 7f23f5fbe740 1 -- start start 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 1 Processor -- start 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f532f0 0x555fda334c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f5b4d0 0x555fda332140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda332700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b1700 con 0x555fd9f7c1b0 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b2b70 con 0x555fd9f5b4d0 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b1dd0 con 0x555fd9f532f0 2024-09-21T07:28:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23f5fbe740 10 client.0 _ll_register_callbacks cb 0x7f23d0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.106+0000 7f23ece00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda332700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23ece00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda332700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48746/0 (socket says 172.21.15.27:48746) 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23ece00640 1 -- 172.21.15.27:0/265066432 learned_addr learned my addr 172.21.15.27:0/265066432 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23df400640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f5b4d0 0x555fda332140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23ece00640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f532f0 0x555fda334c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23df400640 1 -- 172.21.15.27:0/265066432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f532f0 msgr2=0x555fda334c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23df400640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f532f0 0x555fda334c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23df400640 1 -- 172.21.15.27:0/265066432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 msgr2=0x555fda332700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23df400640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda332700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23df400640 1 -- 172.21.15.27:0/265066432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fda332e90 con 0x555fd9f5b4d0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23ece00640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda332700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23df400640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f5b4d0 0x555fda332140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000c00 tx=0x7f23d000fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f181864b179f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.107+0000 7f23d6000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23d6000640 1 -- 172.21.15.27:0/265066432 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d001c630 con 0x555fd9f5b4d0 2024-09-21T07:28:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23d6000640 1 -- 172.21.15.27:0/265066432 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9461+0+0 (secure 0 0 0) 0x7f23d001cad0 con 0x555fd9f5b4d0 2024-09-21T07:28:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e0036040 con 0x555fd9f5b4d0 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23d6000640 1 -- 172.21.15.27:0/265066432 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23d001ccd0 con 0x555fd9f5b4d0 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 10 client.29990 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fda3a8b00 con 0x555fd9f5b4d0 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23d6000640 1 -- 172.21.15.27:0/265066432 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0036040 con 0x555fd9f5b4d0 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23d6000640 1 -- 172.21.15.27:0/265066432 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f23d00037b0 con 0x555fd9f5b4d0 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 10 client.29990 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 10 client.29990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 20 client.29990 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f101c0 0x555fda3ab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 4 client.29990 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.108+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fda2940b0 con 0x555fd9f101c0 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.109+0000 7f23ece00640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f101c0 0x555fda3ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.109+0000 7f23ece00640 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f101c0 0x555fda3ab340 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184d3ef90d8df98e server_cookie=ecc3b90f984a31fa in_seq=0 out_seq=0 2024-09-21T07:28:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.109+0000 7f23d6000640 10 client.29990 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.120+0000 7f23d6000640 1 -- 172.21.15.27:0/265066432 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fda2940b0 con 0x555fd9f101c0 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.120+0000 7f23d6000640 10 client.29990 handle_command_reply: tid=0 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.120+0000 7f23f5fbe740 1 client.29990 shutdown 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f5b4d0 msgr2=0x555fda332140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f5b4d0 0x555fda332140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000c00 tx=0x7f23d000fa80 comp rx=0 tx=0).stop 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 shutdown_connections 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x555fd9f101c0 0x555fda3ab340 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fd9f7c1b0 0x555fda332700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fd9f5b4d0 0x555fda332140 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23d6000640 0 client.29990 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 --2- 172.21.15.27:0/265066432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fd9f532f0 0x555fda334c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 >> 172.21.15.27:0/265066432 conn(0x555fd9f70300 msgr2=0x555fda337b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 shutdown_connections 2024-09-21T07:28:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 1 -- 172.21.15.27:0/265066432 wait complete. 2024-09-21T07:28:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.121+0000 7f23f5fbe740 20 client.29990 trim_cache size 0 max 16384 2024-09-21T07:28:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:28:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:28:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:28:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23cc0b66b0 msgr2=0x7f23cc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23cc0b66b0 0x7f23cc0b8bc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f23d80e5230 tx=0x7f23d8018420 comp rx=0 tx=0).stop 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8103ad0 msgr2=0x7f23e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8103ad0 0x7f23e8075710 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f23d800e9f0 tx=0x7f23d800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 shutdown_connections 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23e8109400 0x7f23e8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f23cc0b66b0 0x7f23cc0b8bc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23e8109d50 0x7f23e8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 --2- 172.21.15.27:0/1968856016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23e8103ad0 0x7f23e8075710 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 >> 172.21.15.27:0/1968856016 conn(0x7f23e80ff510 msgr2=0x7f23e8108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 shutdown_connections 2024-09-21T07:28:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:00.122+0000 7f23eee00640 1 -- 172.21.15.27:0/1968856016 wait complete. 2024-09-21T07:28:01.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:00 smithi153 ceph-mon[34747]: pgmap v845: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-09-21T07:28:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:00 smithi119 ceph-mon[34519]: pgmap v845: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-09-21T07:28:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:00 smithi027 ceph-mon[30779]: pgmap v845: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-09-21T07:28:03.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:02 smithi153 ceph-mon[34747]: pgmap v846: 129 pgs: 129 active+clean; 2.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-09-21T07:28:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:02 smithi119 ceph-mon[34519]: pgmap v846: 129 pgs: 129 active+clean; 2.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-09-21T07:28:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:02 smithi027 ceph-mon[30779]: pgmap v846: 129 pgs: 129 active+clean; 2.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 KiB/s wr, 11 op/s 2024-09-21T07:28:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:04 smithi119 ceph-mon[34519]: pgmap v847: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2024-09-21T07:28:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:04 smithi119 ceph-mon[34519]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T07:28:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:04 smithi027 ceph-mon[30779]: pgmap v847: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2024-09-21T07:28:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:04 smithi027 ceph-mon[30779]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T07:28:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:04 smithi153 ceph-mon[34747]: pgmap v847: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2024-09-21T07:28:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:04 smithi153 ceph-mon[34747]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T07:28:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:06 smithi119 ceph-mon[34519]: pgmap v849: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 34 op/s 2024-09-21T07:28:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:06 smithi027 ceph-mon[30779]: pgmap v849: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 34 op/s 2024-09-21T07:28:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:06 smithi153 ceph-mon[34747]: pgmap v849: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 34 op/s 2024-09-21T07:28:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:08 smithi119 ceph-mon[34519]: pgmap v850: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 34 op/s 2024-09-21T07:28:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:08 smithi027 ceph-mon[30779]: pgmap v850: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 34 op/s 2024-09-21T07:28:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:08 smithi153 ceph-mon[34747]: pgmap v850: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 34 op/s 2024-09-21T07:28:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:10 smithi119 ceph-mon[34519]: pgmap v851: 129 pgs: 129 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 103 KiB/s wr, 34 op/s 2024-09-21T07:28:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:10 smithi027 ceph-mon[30779]: pgmap v851: 129 pgs: 129 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 103 KiB/s wr, 34 op/s 2024-09-21T07:28:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:10 smithi153 ceph-mon[34747]: pgmap v851: 129 pgs: 129 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 103 KiB/s wr, 34 op/s 2024-09-21T07:28:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:12 smithi027 ceph-mon[30779]: pgmap v852: 129 pgs: 129 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 76 KiB/s wr, 31 op/s 2024-09-21T07:28:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:12 smithi027 ceph-mon[30779]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T07:28:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:12 smithi153 ceph-mon[34747]: pgmap v852: 129 pgs: 129 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 76 KiB/s wr, 31 op/s 2024-09-21T07:28:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:12 smithi153 ceph-mon[34747]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T07:28:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:12 smithi119 ceph-mon[34519]: pgmap v852: 129 pgs: 129 active+clean; 3.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 76 KiB/s wr, 31 op/s 2024-09-21T07:28:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:12 smithi119 ceph-mon[34519]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.328+0000 7f0f19600640 1 -- 172.21.15.27:0/1644609127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14103910 msgr2=0x7f0f14103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.328+0000 7f0f19600640 1 --2- 172.21.15.27:0/1644609127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14103910 0x7f0f14103d20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009be0 tx=0x7f0f0402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.328+0000 7f0f19600640 1 -- 172.21.15.27:0/1644609127 shutdown_connections 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.328+0000 7f0f19600640 1 --2- 172.21.15.27:0/1644609127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f14109240 0x7f0f14109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.328+0000 7f0f19600640 1 --2- 172.21.15.27:0/1644609127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14109b90 0x7f0f141103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.328+0000 7f0f19600640 1 --2- 172.21.15.27:0/1644609127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14103910 0x7f0f14103d20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.329+0000 7f0f19600640 1 -- 172.21.15.27:0/1644609127 >> 172.21.15.27:0/1644609127 conn(0x7f0f140ff330 msgr2=0x7f0f14101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.329+0000 7f0f19600640 1 -- 172.21.15.27:0/1644609127 shutdown_connections 2024-09-21T07:28:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.329+0000 7f0f19600640 1 -- 172.21.15.27:0/1644609127 wait complete. 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 Processor -- start 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 -- start start 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14103910 0x7f0f1419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14109240 0x7f0f1419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f14109b90 0x7f0f141a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14113850 con 0x7f0f14103910 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141136d0 con 0x7f0f14109240 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f19600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141139d0 con 0x7f0f14109b90 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f13e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14103910 0x7f0f1419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f13e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14103910 0x7f0f1419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45530/0 (socket says 172.21.15.27:45530) 2024-09-21T07:28:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f13e00640 1 -- 172.21.15.27:0/3154348819 learned_addr learned my addr 172.21.15.27:0/3154348819 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.330+0000 7f0f18c00640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f14109b90 0x7f0f141a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.331+0000 7f0f13400640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14109240 0x7f0f1419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.331+0000 7f0f13e00640 1 -- 172.21.15.27:0/3154348819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f14109b90 msgr2=0x7f0f141a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.331+0000 7f0f13e00640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f14109b90 0x7f0f141a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.331+0000 7f0f13e00640 1 -- 172.21.15.27:0/3154348819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14109240 msgr2=0x7f0f1419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.331+0000 7f0f13e00640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14109240 0x7f0f1419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.331+0000 7f0f13e00640 1 -- 172.21.15.27:0/3154348819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f141a2420 con 0x7f0f14103910 2024-09-21T07:28:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.331+0000 7f0f13e00640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14103910 0x7f0f1419c4f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0f04004af0 tx=0x7f0f04003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.332+0000 7f0f10c00640 1 -- 172.21.15.27:0/3154348819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04040020 con 0x7f0f14103910 2024-09-21T07:28:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.332+0000 7f0f10c00640 1 -- 172.21.15.27:0/3154348819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f04030e20 con 0x7f0f14103910 2024-09-21T07:28:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.332+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f141a26b0 con 0x7f0f14103910 2024-09-21T07:28:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.332+0000 7f0f10c00640 1 -- 172.21.15.27:0/3154348819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04036040 con 0x7f0f14103910 2024-09-21T07:28:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.332+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141a8ea0 con 0x7f0f14103910 2024-09-21T07:28:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 Processor -- start 2024-09-21T07:28:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 -- start start 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043026e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043027350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043027890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562042f9c6b0 con 0x562042c614d0 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562042f8bae0 con 0x562042c821b0 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f10c00640 1 -- 172.21.15.27:0/3154348819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0f04043020 con 0x7f0f14103910 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.335+0000 7f0f21c77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562042f8f8f0 con 0x562042c592f0 2024-09-21T07:28:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043026e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043026e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59282/0 (socket says 172.21.15.27:59282) 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 -- 172.21.15.27:0/4215577713 learned_addr learned my addr 172.21.15.27:0/4215577713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f13e00640 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043027350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f13400640 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043027890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 -- 172.21.15.27:0/4215577713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 msgr2=0x562043027350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043027350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 -- 172.21.15.27:0/4215577713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 msgr2=0x562043027890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043027890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f18c00640 1 -- 172.21.15.27:0/4215577713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562042f58780 con 0x562042c821b0 2024-09-21T07:28:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.336+0000 7f0f13400640 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043027890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.337+0000 7f0f18c00640 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043026e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009c10 tx=0x7f0f0c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc637aba7c924b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.337+0000 7f0f09200640 1 -- 172.21.15.27:0/4215577713 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c004230 con 0x562042c821b0 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.337+0000 7f0f09200640 1 -- 172.21.15.27:0/4215577713 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f0c0043d0 con 0x562042c821b0 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.337+0000 7f0f09200640 1 -- 172.21.15.27:0/4215577713 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c019790 con 0x562042c821b0 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.337+0000 7f0f10c00640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0ef80b6680 0x7f0ef80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f10c00640 1 -- 172.21.15.27:0/3154348819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f0f041087b0 con 0x7f0f14103910 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 -- 172.21.15.27:0/4215577713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 msgr2=0x562043026e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043026e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009c10 tx=0x7f0f0c011810 comp rx=0 tx=0).stop 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f13e00640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0ef80b6680 0x7f0ef80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 -- 172.21.15.27:0/4215577713 shutdown_connections 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043027350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043027890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 --2- 172.21.15.27:0/4215577713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043026e10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 -- 172.21.15.27:0/4215577713 >> 172.21.15.27:0/4215577713 conn(0x562042c76300 msgr2=0x562043024d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 -- 172.21.15.27:0/4215577713 shutdown_connections 2024-09-21T07:28:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f21c77740 1 -- 172.21.15.27:0/4215577713 wait complete. 2024-09-21T07:28:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.338+0000 7f0f13e00640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0ef80b6680 0x7f0ef80b8b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x562043025a90 tx=0x7f0f0410e530 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 -- start start 2024-09-21T07:28:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 Processor -- start 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043032490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x5620430329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043032f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562042ec3ab0 con 0x562042c614d0 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562042f32570 con 0x562042c821b0 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.339+0000 7f0f21c77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562043035590 con 0x562042c592f0 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f21c77740 10 client.0 _ll_register_callbacks cb 0x562042f41100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043032490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043032490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:52686/0 (socket says 172.21.15.27:52686) 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 -- 172.21.15.27:0/53093821 learned_addr learned my addr 172.21.15.27:0/53093821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f13400640 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x5620430329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043032f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 -- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 msgr2=0x562043032f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043032f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 -- 172.21.15.27:0/53093821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 msgr2=0x5620430329d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x5620430329d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 -- 172.21.15.27:0/53093821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562042ced790 con 0x562042c592f0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0f18c00640 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043032490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0099f0 tx=0x7f0f0c003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee49bf6c9e49a16c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.340+0000 7f0efd600640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0efd600640 1 -- 172.21.15.27:0/53093821 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c02d060 con 0x562042c592f0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f0c0043d0 con 0x562042c592f0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0efd600640 1 -- 172.21.15.27:0/53093821 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f0f0c019530 con 0x562042c592f0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0efd600640 1 -- 172.21.15.27:0/53093821 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f0c005020 con 0x562042c592f0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0f21c77740 10 client.29959 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562042d66620 con 0x562042c592f0 2024-09-21T07:28:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0efd600640 1 -- 172.21.15.27:0/53093821 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f0c0043d0 con 0x562042c592f0 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.341+0000 7f0efd600640 1 -- 172.21.15.27:0/53093821 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f0c025040 con 0x562042c592f0 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f21c77740 10 client.29959 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f21c77740 10 client.29959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f21c77740 20 client.29959 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f21c77740 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c40d80 0x5620430444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f21c77740 4 client.29959 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562042f9e600 con 0x562042c40d80 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f13400640 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c40d80 0x5620430444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0efd600640 10 client.29959 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.342+0000 7f0f13400640 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c40d80 0x5620430444f0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e0e8104bc5445a server_cookie=932b892451c5b48e in_seq=0 out_seq=0 2024-09-21T07:28:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.343+0000 7f0efd600640 1 -- 172.21.15.27:0/53093821 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562042f9e600 con 0x562042c40d80 2024-09-21T07:28:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.343+0000 7f0efd600640 10 client.29959 handle_command_reply: tid=0 2024-09-21T07:28:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.343+0000 7f0f21c77740 1 client.29959 shutdown 2024-09-21T07:28:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 msgr2=0x562043032490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043032490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0099f0 tx=0x7f0f0c003e70 comp rx=0 tx=0).stop 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 shutdown_connections 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c40d80 0x5620430444f0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x562043032f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x5620430329d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 --2- 172.21.15.27:0/53093821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c592f0 0x562043032490 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0efd600640 0 client.29959 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 >> 172.21.15.27:0/53093821 conn(0x562042c76300 msgr2=0x562043027e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 shutdown_connections 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.344+0000 7f0f21c77740 1 -- 172.21.15.27:0/53093821 wait complete. 2024-09-21T07:28:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.345+0000 7f0f21c77740 20 client.29959 trim_cache size 0 max 16384 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 Processor -- start 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 -- start start 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c592f0 0x5620430340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c614d0 0x56204302f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x56204302fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141b5860 con 0x562042c592f0 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141b3130 con 0x562042c821b0 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.360+0000 7f0f21c77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141b0fa0 con 0x562042c614d0 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f18c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c592f0 0x5620430340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f18c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c592f0 0x5620430340c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45548/0 (socket says 172.21.15.27:45548) 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f18c00640 1 -- 172.21.15.27:0/407245565 learned_addr learned my addr 172.21.15.27:0/407245565 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f13400640 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c614d0 0x56204302f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f18c00640 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x56204302fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f13400640 1 -- 172.21.15.27:0/407245565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 msgr2=0x56204302fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f13400640 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x56204302fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f13400640 1 -- 172.21.15.27:0/407245565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c592f0 msgr2=0x5620430340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f13400640 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c592f0 0x5620430340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f13400640 1 -- 172.21.15.27:0/407245565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56204303e7b0 con 0x562042c614d0 2024-09-21T07:28:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.361+0000 7f0f13400640 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c614d0 0x56204302f660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x562043044ae0 tx=0x7f0f000138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62eed67ed0ce69ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.362+0000 7f0effe00640 1 -- 172.21.15.27:0/407245565 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00013190 con 0x562042c614d0 2024-09-21T07:28:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.362+0000 7f0effe00640 1 -- 172.21.15.27:0/407245565 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f000035f0 con 0x562042c614d0 2024-09-21T07:28:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.362+0000 7f0effe00640 1 -- 172.21.15.27:0/407245565 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0001d690 con 0x562042c614d0 2024-09-21T07:28:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 -- 172.21.15.27:0/407245565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c614d0 msgr2=0x56204302f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c614d0 0x56204302f660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x562043044ae0 tx=0x7f0f000138f0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 -- 172.21.15.27:0/407245565 shutdown_connections 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c614d0 0x56204302f660 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c821b0 0x56204302fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 --2- 172.21.15.27:0/407245565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c592f0 0x5620430340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 -- 172.21.15.27:0/407245565 >> 172.21.15.27:0/407245565 conn(0x562042c76300 msgr2=0x5620430339b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 -- 172.21.15.27:0/407245565 shutdown_connections 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 -- 172.21.15.27:0/407245565 wait complete. 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.363+0000 7f0f21c77740 1 -- start start 2024-09-21T07:28:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 1 Processor -- start 2024-09-21T07:28:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c592f0 0x5620430382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043038870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c821b0 0x5620430a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141b15f0 con 0x562042c614d0 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141b2a60 con 0x562042c592f0 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141b1cc0 con 0x562042c821b0 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f21c77740 10 client.0 _ll_register_callbacks cb 0x7f0f0410e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f13400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043038870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f13400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043038870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45550/0 (socket says 172.21.15.27:45550) 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f13400640 1 -- 172.21.15.27:0/1141289115 learned_addr learned my addr 172.21.15.27:0/1141289115 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.364+0000 7f0f18c00640 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c592f0 0x5620430382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0f18c00640 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c821b0 0x5620430a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0f13400640 1 -- 172.21.15.27:0/1141289115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c821b0 msgr2=0x5620430a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0f13400640 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c821b0 0x5620430a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0f13400640 1 -- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c592f0 msgr2=0x5620430382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0f13400640 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c592f0 0x5620430382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0f13400640 1 -- 172.21.15.27:0/1141289115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620430a6370 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0f13400640 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043038870 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0f000131e0 tx=0x7f0f00013210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f0f9fec7962948 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0efd600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.365+0000 7f0efd600640 1 -- 172.21.15.27:0/1141289115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00004230 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0efd600640 1 -- 172.21.15.27:0/1141289115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f0f000043d0 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f000035f0 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0efd600640 1 -- 172.21.15.27:0/1141289115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f00038050 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 10 client.19986 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620430ae8a0 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0efd600640 1 -- 172.21.15.27:0/1141289115 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f000035f0 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0efd600640 1 -- 172.21.15.27:0/1141289115 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f00003be0 con 0x562042c614d0 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 10 client.19986 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 10 client.19986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 20 client.19986 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c161c0 0x5620430b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 4 client.19986 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:28:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.366+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562042f9a0b0 con 0x562042c161c0 2024-09-21T07:28:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.367+0000 7f0f18c00640 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c161c0 0x5620430b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.367+0000 7f0f18c00640 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c161c0 0x5620430b1090 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7dde50cf270c80 server_cookie=bad9f8bb8d4ca58d in_seq=0 out_seq=0 2024-09-21T07:28:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.367+0000 7f0efd600640 10 client.19986 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0efd600640 1 -- 172.21.15.27:0/1141289115 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562042f9a0b0 con 0x562042c161c0 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0efd600640 10 client.19986 handle_command_reply: tid=0 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 client.19986 shutdown 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 msgr2=0x562043038870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043038870 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0f000131e0 tx=0x7f0f00013210 comp rx=0 tx=0).stop 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 shutdown_connections 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x562042c161c0 0x5620430b1090 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562042c821b0 0x5620430a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562042c614d0 0x562043038870 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0efd600640 0 client.19986 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 --2- 172.21.15.27:0/1141289115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562042c592f0 0x5620430382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.382+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 >> 172.21.15.27:0/1141289115 conn(0x562042c76300 msgr2=0x562043023360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.383+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 shutdown_connections 2024-09-21T07:28:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.383+0000 7f0f21c77740 1 -- 172.21.15.27:0/1141289115 wait complete. 2024-09-21T07:28:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.383+0000 7f0f21c77740 20 client.19986 trim_cache size 0 max 16384 2024-09-21T07:28:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:28:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:28:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:28:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.383+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0ef80b6680 msgr2=0x7f0ef80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.383+0000 7f0f19600640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0ef80b6680 0x7f0ef80b8b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x562043025a90 tx=0x7f0f0410e530 comp rx=0 tx=0).stop 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.383+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14103910 msgr2=0x7f0f1419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.383+0000 7f0f19600640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14103910 0x7f0f1419c4f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0f04004af0 tx=0x7f0f04003d60 comp rx=0 tx=0).stop 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 shutdown_connections 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f14109b90 0x7f0f141a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0ef80b6680 0x7f0ef80b8b90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f14109240 0x7f0f1419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 --2- 172.21.15.27:0/3154348819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f14103910 0x7f0f1419c4f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 >> 172.21.15.27:0/3154348819 conn(0x7f0f140ff330 msgr2=0x7f0f14108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 shutdown_connections 2024-09-21T07:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:13.384+0000 7f0f19600640 1 -- 172.21.15.27:0/3154348819 wait complete. 2024-09-21T07:28:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:14 smithi027 ceph-mon[30779]: pgmap v854: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 12 op/s 2024-09-21T07:28:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:28:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:28:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:28:15.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:28:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:14 smithi153 ceph-mon[34747]: pgmap v854: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 12 op/s 2024-09-21T07:28:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:28:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:28:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:28:15.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:28:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:14 smithi119 ceph-mon[34519]: pgmap v854: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 12 op/s 2024-09-21T07:28:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:28:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:28:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:28:15.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:28:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:16 smithi153 ceph-mon[34747]: pgmap v855: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-09-21T07:28:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:16 smithi119 ceph-mon[34519]: pgmap v855: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-09-21T07:28:17.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:16 smithi027 ceph-mon[30779]: pgmap v855: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-09-21T07:28:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:18 smithi153 ceph-mon[34747]: pgmap v856: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-21T07:28:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:18 smithi119 ceph-mon[34519]: pgmap v856: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-21T07:28:19.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:18 smithi027 ceph-mon[30779]: pgmap v856: 129 pgs: 129 active+clean; 3.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-21T07:28:21.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:20 smithi153 ceph-mon[34747]: pgmap v857: 129 pgs: 129 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-21T07:28:21.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:20 smithi119 ceph-mon[34519]: pgmap v857: 129 pgs: 129 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-21T07:28:21.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:20 smithi027 ceph-mon[30779]: pgmap v857: 129 pgs: 129 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-21T07:28:23.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:22 smithi153 ceph-mon[34747]: pgmap v858: 129 pgs: 129 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-21T07:28:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:22 smithi119 ceph-mon[34519]: pgmap v858: 129 pgs: 129 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-21T07:28:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:22 smithi027 ceph-mon[30779]: pgmap v858: 129 pgs: 129 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-21T07:28:23.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:28:23.956 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 -- 172.21.15.27:0/155464301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 msgr2=0x7fc3ac10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/155464301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 0x7fc3ac10bad0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc398009900 tx=0x7fc39802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 -- 172.21.15.27:0/155464301 shutdown_connections 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/155464301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ac10c010 0x7fc3ac112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/155464301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 0x7fc3ac10bad0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/155464301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ac105d80 0x7fc3ac106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 -- 172.21.15.27:0/155464301 >> 172.21.15.27:0/155464301 conn(0x7fc3ac1017d0 msgr2=0x7fc3ac103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 -- 172.21.15.27:0/155464301 shutdown_connections 2024-09-21T07:28:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.695+0000 7fc3b1c00640 1 -- 172.21.15.27:0/155464301 wait complete. 2024-09-21T07:28:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.696+0000 7fc3b1c00640 1 Processor -- start 2024-09-21T07:28:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3b1c00640 1 -- start start 2024-09-21T07:28:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3b1c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3b1c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 0x7fc3ac19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3b1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ac10c010 0x7fc3ac1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3b1c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac115cd0 con 0x7fc3ac10b6c0 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3b1c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac115b50 con 0x7fc3ac10c010 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3b1c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac115e50 con 0x7fc3ac105d80 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46692/0 (socket says 172.21.15.27:46692) 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3aaa00640 1 -- 172.21.15.27:0/1006963715 learned_addr learned my addr 172.21.15.27:0/1006963715 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3aa000640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 0x7fc3ac19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.697+0000 7fc3ab400640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ac10c010 0x7fc3ac1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.698+0000 7fc3ab400640 1 -- 172.21.15.27:0/1006963715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ac105d80 msgr2=0x7fc3ac19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.698+0000 7fc3ab400640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.698+0000 7fc3ab400640 1 -- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 msgr2=0x7fc3ac19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.698+0000 7fc3ab400640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 0x7fc3ac19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.698+0000 7fc3ab400640 1 -- 172.21.15.27:0/1006963715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1a2630 con 0x7fc3ac10c010 2024-09-21T07:28:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.698+0000 7fc3ab400640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ac10c010 0x7fc3ac1a1f00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009fd0 tx=0x7fc3a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:24.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.698+0000 7fc3a3400640 1 -- 172.21.15.27:0/1006963715 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a400ce20 con 0x7fc3ac10c010 2024-09-21T07:28:24.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.699+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1a2920 con 0x7fc3ac10c010 2024-09-21T07:28:24.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.699+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1a9080 con 0x7fc3ac10c010 2024-09-21T07:28:24.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.699+0000 7fc3a3400640 1 -- 172.21.15.27:0/1006963715 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a4005020 con 0x7fc3ac10c010 2024-09-21T07:28:24.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.699+0000 7fc3a3400640 1 -- 172.21.15.27:0/1006963715 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4010690 con 0x7fc3ac10c010 2024-09-21T07:28:24.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.702+0000 7fc3b8851740 1 Processor -- start 2024-09-21T07:28:24.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3a3400640 1 -- 172.21.15.27:0/1006963715 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc3a4004a40 con 0x7fc3ac10c010 2024-09-21T07:28:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3b8851740 1 -- start start 2024-09-21T07:28:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefa8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefa92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefa97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3b8851740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eefef1ff70 con 0x55eefec02070 2024-09-21T07:28:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3b8851740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eefef0f3a0 con 0x55eefebe1390 2024-09-21T07:28:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.703+0000 7fc3b8851740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eefef131b0 con 0x55eefebd91b0 2024-09-21T07:28:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aa000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefa8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aa000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefa8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50082/0 (socket says 172.21.15.27:50082) 2024-09-21T07:28:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefa92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aa000640 1 -- 172.21.15.27:0/2859355785 learned_addr learned my addr 172.21.15.27:0/2859355785 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefa97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aaa00640 1 -- 172.21.15.27:0/2859355785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 msgr2=0x55eefefa97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefa97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aaa00640 1 -- 172.21.15.27:0/2859355785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 msgr2=0x55eefefa8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefa8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.704+0000 7fc3aaa00640 1 -- 172.21.15.27:0/2859355785 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eefeed9c90 con 0x55eefebd91b0 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.705+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefa92a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009cf0 tx=0x7fc39c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b70cb0ea6aaf96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:24.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.705+0000 7fc3a3400640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3900b6700 0x7fc3900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.705+0000 7fc3a3400640 1 -- 172.21.15.27:0/1006963715 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7fc3a40e7fe0 con 0x7fc3ac10c010 2024-09-21T07:28:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.705+0000 7fc397e00640 1 -- 172.21.15.27:0/2859355785 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c005220 con 0x55eefebd91b0 2024-09-21T07:28:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.705+0000 7fc397e00640 1 -- 172.21.15.27:0/2859355785 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc39c004c50 con 0x55eefebd91b0 2024-09-21T07:28:24.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.705+0000 7fc3aa000640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3900b6700 0x7fc3900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.705+0000 7fc397e00640 1 -- 172.21.15.27:0/2859355785 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c019780 con 0x55eefebd91b0 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 -- 172.21.15.27:0/2859355785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 msgr2=0x55eefefa92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3aa000640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3900b6700 0x7fc3900b8c10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc3980096c0 tx=0x7fc398000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefa92a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009cf0 tx=0x7fc39c011810 comp rx=0 tx=0).stop 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 -- 172.21.15.27:0/2859355785 shutdown_connections 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefa92a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefa97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2859355785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefa8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 -- 172.21.15.27:0/2859355785 >> 172.21.15.27:0/2859355785 conn(0x55eefebf6460 msgr2=0x55eefefa6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 -- 172.21.15.27:0/2859355785 shutdown_connections 2024-09-21T07:28:24.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.706+0000 7fc3b8851740 1 -- 172.21.15.27:0/2859355785 wait complete. 2024-09-21T07:28:24.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.707+0000 7fc3b8851740 1 -- start start 2024-09-21T07:28:24.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.707+0000 7fc3b8851740 1 Processor -- start 2024-09-21T07:28:24.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefb4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefb4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefc08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3b8851740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eefed70700 con 0x55eefec02070 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3b8851740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eefeeaa540 con 0x55eefebe1390 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3b8851740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eefefb7540 con 0x55eefebd91b0 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3b8851740 10 client.0 _ll_register_callbacks cb 0x55eefeeba430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefc08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefb4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefb4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46704/0 (socket says 172.21.15.27:46704) 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefc08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50094/0 (socket says 172.21.15.27:50094) 2024-09-21T07:28:24.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3aaa00640 1 -- 172.21.15.27:0/2518431113 learned_addr learned my addr 172.21.15.27:0/2518431113 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefb4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.708+0000 7fc3aaa00640 1 -- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 msgr2=0x55eefefb4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefb4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc3aaa00640 1 -- 172.21.15.27:0/2518431113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 msgr2=0x55eefefc08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefc08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc3aaa00640 1 -- 172.21.15.27:0/2518431113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eefec6d650 con 0x55eefebd91b0 2024-09-21T07:28:24.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefb4510 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc39c019bb0 tx=0x7fc39c019be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57755c1e5648bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:24.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc394c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:24.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc394c00640 1 -- 172.21.15.27:0/2518431113 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c029070 con 0x55eefebd91b0 2024-09-21T07:28:24.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eefeeacb50 con 0x55eefebd91b0 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.709+0000 7fc394c00640 1 -- 172.21.15.27:0/2518431113 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7fc39c023430 con 0x55eefebd91b0 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.710+0000 7fc394c00640 1 -- 172.21.15.27:0/2518431113 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc39c035050 con 0x55eefebd91b0 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.710+0000 7fc3b8851740 10 client.29977 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.710+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eefeec7910 con 0x55eefebd91b0 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.710+0000 7fc394c00640 1 -- 172.21.15.27:0/2518431113 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eefeeacb50 con 0x55eefebd91b0 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.710+0000 7fc394c00640 1 -- 172.21.15.27:0/2518431113 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc39c0224d0 con 0x55eefebd91b0 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.710+0000 7fc3b8851740 10 client.29977 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.710+0000 7fc3b8851740 10 client.29977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.711+0000 7fc3b8851740 20 client.29977 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:24.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.711+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefebc09f0 0x55eefefc6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.711+0000 7fc3b8851740 4 client.29977 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.711+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eefef21ec0 con 0x55eefebc09f0 2024-09-21T07:28:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.711+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefebc09f0 0x55eefefc6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.712+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefebc09f0 0x55eefefc6510 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b2b1296e530b6 server_cookie=3f89ed75f2634df in_seq=0 out_seq=0 2024-09-21T07:28:24.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.712+0000 7fc394c00640 10 client.29977 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc394c00640 1 -- 172.21.15.27:0/2518431113 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eefef21ec0 con 0x55eefebc09f0 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc394c00640 10 client.29977 handle_command_reply: tid=0 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 client.29977 shutdown 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 msgr2=0x55eefefb4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefb4510 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc39c019bb0 tx=0x7fc39c019be0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 shutdown_connections 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefebc09f0 0x55eefefc6510 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefec02070 0x55eefefc08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebe1390 0x55eefefb4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2518431113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefb4510 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc394c00640 0 client.29977 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 >> 172.21.15.27:0/2518431113 conn(0x55eefebf6460 msgr2=0x55eefefabef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 shutdown_connections 2024-09-21T07:28:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 1 -- 172.21.15.27:0/2518431113 wait complete. 2024-09-21T07:28:24.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.713+0000 7fc3b8851740 20 client.29977 trim_cache size 0 max 16384 2024-09-21T07:28:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 Processor -- start 2024-09-21T07:28:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 -- start start 2024-09-21T07:28:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefc07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefec02070 0x55eefefb1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c012820 con 0x55eefebe1390 2024-09-21T07:28:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c0100f0 con 0x55eefec02070 2024-09-21T07:28:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3b8851740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00df60 con 0x55eefebd91b0 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3aa000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefc07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3aa000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefc07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50108/0 (socket says 172.21.15.27:50108) 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.732+0000 7fc3aa000640 1 -- 172.21.15.27:0/2642008982 learned_addr learned my addr 172.21.15.27:0/2642008982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3ab400640 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefec02070 0x55eefefb1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3aa000640 1 -- 172.21.15.27:0/2642008982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 msgr2=0x55eefefac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3aa000640 1 -- 172.21.15.27:0/2642008982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefec02070 msgr2=0x55eefefb1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefec02070 0x55eefefb1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3aa000640 1 -- 172.21.15.27:0/2642008982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eefefb7420 con 0x55eefebe1390 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3ab400640 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefec02070 0x55eefefb1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefc07c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc398009900 tx=0x7fc398009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59dac5b26c68dfdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.733+0000 7fc397400640 1 -- 172.21.15.27:0/2642008982 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398035040 con 0x55eefebe1390 2024-09-21T07:28:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc397400640 1 -- 172.21.15.27:0/2642008982 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc398005590 con 0x55eefebe1390 2024-09-21T07:28:24.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc397400640 1 -- 172.21.15.27:0/2642008982 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398004d80 con 0x55eefebe1390 2024-09-21T07:28:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 -- 172.21.15.27:0/2642008982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 msgr2=0x55eefefc07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefc07c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc398009900 tx=0x7fc398009260 comp rx=0 tx=0).stop 2024-09-21T07:28:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 -- 172.21.15.27:0/2642008982 shutdown_connections 2024-09-21T07:28:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefec02070 0x55eefefb1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefc07c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2642008982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefebd91b0 0x55eefefac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 -- 172.21.15.27:0/2642008982 >> 172.21.15.27:0/2642008982 conn(0x55eefebf6460 msgr2=0x55eefefabe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.734+0000 7fc3b8851740 1 -- 172.21.15.27:0/2642008982 shutdown_connections 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.735+0000 7fc3b8851740 1 -- 172.21.15.27:0/2642008982 wait complete. 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.735+0000 7fc3b8851740 1 -- start start 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.735+0000 7fc3b8851740 1 Processor -- start 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebd91b0 0x55eefefb9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefb9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3b8851740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefec02070 0x55eeff02fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3b8851740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00e5b0 con 0x55eefebe1390 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3b8851740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00fa20 con 0x55eefebd91b0 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3b8851740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c00ec80 con 0x55eefec02070 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3b8851740 10 client.0 _ll_register_callbacks cb 0x7fc39c019840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3aa000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefb9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebd91b0 0x55eefefb9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3aa000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefb9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50110/0 (socket says 172.21.15.27:50110) 2024-09-21T07:28:24.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3aa000640 1 -- 172.21.15.27:0/2236473586 learned_addr learned my addr 172.21.15.27:0/2236473586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:24.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.736+0000 7fc3ab400640 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefec02070 0x55eeff02fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3aa000640 1 -- 172.21.15.27:0/2236473586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefec02070 msgr2=0x55eeff02fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefec02070 0x55eeff02fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3aa000640 1 -- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebd91b0 msgr2=0x55eefefb9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:24.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebd91b0 0x55eefefb9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3aa000640 1 -- 172.21.15.27:0/2236473586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeff030420 con 0x55eefebe1390 2024-09-21T07:28:24.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebd91b0 0x55eefefb9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:24.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3aa000640 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefb9610 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc398009230 tx=0x7fc398031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1279c6d5a7f30ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc394c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc394c00640 1 -- 172.21.15.27:0/2236473586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398060060 con 0x55eefebe1390 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc394c00640 1 -- 172.21.15.27:0/2236473586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7fc398051040 con 0x55eefebe1390 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.737+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc398005590 con 0x55eefebe1390 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc394c00640 1 -- 172.21.15.27:0/2236473586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc398035040 con 0x55eefebe1390 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 10 client.20004 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eeff031690 con 0x55eefebe1390 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc394c00640 1 -- 172.21.15.27:0/2236473586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc398005590 con 0x55eefebe1390 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc394c00640 1 -- 172.21.15.27:0/2236473586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc398048050 con 0x55eefebe1390 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 10 client.20004 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 10 client.20004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 20 client.20004 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:24.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefeb95da0 0x55eeff033e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 4 client.20004 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.738+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eefef1d970 con 0x55eefeb95da0 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.739+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefeb95da0 0x55eeff033e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.739+0000 7fc3aaa00640 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefeb95da0 0x55eeff033e20 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c01efbf7654a7a server_cookie=4784189ac4c77c1d in_seq=0 out_seq=0 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.739+0000 7fc394c00640 10 client.20004 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc394c00640 1 -- 172.21.15.27:0/2236473586 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6906 (crc 0 0 0) 0x55eefef1d970 con 0x55eefeb95da0 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc394c00640 10 client.20004 handle_command_reply: tid=0 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 client.20004 shutdown 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 msgr2=0x55eefefb9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefb9610 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc398009230 tx=0x7fc398031400 comp rx=0 tx=0).stop 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 shutdown_connections 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55eefeb95da0 0x55eeff033e20 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eefec02070 0x55eeff02fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eefebe1390 0x55eefefb9610 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 --2- 172.21.15.27:0/2236473586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eefebd91b0 0x55eefefb9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc394c00640 0 client.20004 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 >> 172.21.15.27:0/2236473586 conn(0x55eefebf6460 msgr2=0x55eefefac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 shutdown_connections 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 1 -- 172.21.15.27:0/2236473586 wait complete. 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.740+0000 7fc3b8851740 20 client.20004 trim_cache size 0 max 16384 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:28:24.745 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "23d6d7e9-06b5-4a6e-adfb-86c244736c20": { 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:49.824406+0000", 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:28:24.747 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.749 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.750 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:28:24.751 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.752 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.753 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.754 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.756 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.756 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.756 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:28:24.756 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:28:24.756 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.756 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.757 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.757 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.757 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.758 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.759 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:28:24.760 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3900b6700 msgr2=0x7fc3900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3900b6700 0x7fc3900b8c10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc3980096c0 tx=0x7fc398000bc0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ac10c010 msgr2=0x7fc3ac1a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ac10c010 0x7fc3ac1a1f00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009fd0 tx=0x7fc3a400ef10 comp rx=0 tx=0).stop 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 shutdown_connections 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc3900b6700 0x7fc3900b8c10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3ac10c010 0x7fc3ac1a1f00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac10b6c0 0x7fc3ac19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 --2- 172.21.15.27:0/1006963715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3ac105d80 0x7fc3ac19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 >> 172.21.15.27:0/1006963715 conn(0x7fc3ac1017d0 msgr2=0x7fc3ac103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:24.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 shutdown_connections 2024-09-21T07:28:24.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:24.741+0000 7fc3b1c00640 1 -- 172.21.15.27:0/1006963715 wait complete. 2024-09-21T07:28:25.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:24 smithi027 ceph-mon[30779]: pgmap v859: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:28:25.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:24 smithi153 ceph-mon[34747]: pgmap v859: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:28:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:24 smithi119 ceph-mon[34519]: pgmap v859: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-09-21T07:28:25.609 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:28:25.609 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:28:25.999 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 -- 172.21.15.27:0/1602596783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 msgr2=0x7f04d0104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 --2- 172.21.15.27:0/1602596783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 0x7f04d0104b50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009900 tx=0x7f04c002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 -- 172.21.15.27:0/1602596783 shutdown_connections 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 --2- 172.21.15.27:0/1602596783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d010dba0 0x7f04d0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 --2- 172.21.15.27:0/1602596783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d0105090 0x7f04d010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 --2- 172.21.15.27:0/1602596783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 0x7f04d0104b50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 -- 172.21.15.27:0/1602596783 >> 172.21.15.27:0/1602596783 conn(0x7f04d00fe230 msgr2=0x7f04d0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 -- 172.21.15.27:0/1602596783 shutdown_connections 2024-09-21T07:28:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.581+0000 7f04cfe00640 1 -- 172.21.15.27:0/1602596783 wait complete. 2024-09-21T07:28:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 Processor -- start 2024-09-21T07:28:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 -- start start 2024-09-21T07:28:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 0x7f04d0102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d0105090 0x7f04d01029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d010dba0 0x7f04d0102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0113850 con 0x7f04d0104740 2024-09-21T07:28:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01136d0 con 0x7f04d010dba0 2024-09-21T07:28:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.583+0000 7f04cfe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01139d0 con 0x7f04d0105090 2024-09-21T07:28:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d0105090 0x7f04d01029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04cea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 0x7f04d0102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d0105090 0x7f04d01029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46748/0 (socket says 172.21.15.27:46748) 2024-09-21T07:28:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 -- 172.21.15.27:0/3842733309 learned_addr learned my addr 172.21.15.27:0/3842733309 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04cf400640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d010dba0 0x7f04d0102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 -- 172.21.15.27:0/3842733309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d010dba0 msgr2=0x7f04d0102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d010dba0 0x7f04d0102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 -- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 msgr2=0x7f04d0102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 0x7f04d0102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 -- 172.21.15.27:0/3842733309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d01a8540 con 0x7f04d0105090 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.584+0000 7f04ce000640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d0105090 0x7f04d01029b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00e9f0 tx=0x7f04bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.585+0000 7f04c7400640 1 -- 172.21.15.27:0/3842733309 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00ccc0 con 0x7f04d0105090 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.585+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d01a8830 con 0x7f04d0105090 2024-09-21T07:28:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.585+0000 7f04c7400640 1 -- 172.21.15.27:0/3842733309 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04bc0108e0 con 0x7f04d0105090 2024-09-21T07:28:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.585+0000 7f04c7400640 1 -- 172.21.15.27:0/3842733309 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc018860 con 0x7f04d0105090 2024-09-21T07:28:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.585+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d01a8d50 con 0x7f04d0105090 2024-09-21T07:28:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 Processor -- start 2024-09-21T07:28:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 -- start start 2024-09-21T07:28:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b0727de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b0727e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b0727e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558b071f36b0 con 0x558b06eb02f0 2024-09-21T07:28:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558b071e2ae0 con 0x558b06ed91b0 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.588+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b071e68f0 con 0x558b06eb84d0 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b0727de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b0727e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b0727de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44902/0 (socket says 172.21.15.27:44902) 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 -- 172.21.15.27:0/1463522817 learned_addr learned my addr 172.21.15.27:0/1463522817 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04c7400640 1 -- 172.21.15.27:0/3842733309 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f04bc010430 con 0x7f04d0105090 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 -- 172.21.15.27:0/1463522817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 msgr2=0x558b0727e890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 --2- 172.21.15.27:0/1463522817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b0727e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 -- 172.21.15.27:0/1463522817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 msgr2=0x558b0727e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 --2- 172.21.15.27:0/1463522817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b0727e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.589+0000 7f04cf400640 1 -- 172.21.15.27:0/1463522817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b071af780 con 0x558b06ed91b0 2024-09-21T07:28:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.590+0000 7f04cf400640 1 --2- 172.21.15.27:0/1463522817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b0727de10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f04c8011440 tx=0x7f04c8011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2081ab7bc06f70f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.590+0000 7f04bbe00640 1 -- 172.21.15.27:0/1463522817 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8004d20 con 0x558b06ed91b0 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.590+0000 7f04bbe00640 1 -- 172.21.15.27:0/1463522817 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c8004050 con 0x558b06ed91b0 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.590+0000 7f04bbe00640 1 -- 172.21.15.27:0/1463522817 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800f7a0 con 0x558b06ed91b0 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04c7400640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04b40b66b0 0x7f04b40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1463522817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 msgr2=0x558b0727de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1463522817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b0727de10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f04c8011440 tx=0x7f04c8011900 comp rx=0 tx=0).stop 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04c7400640 1 -- 172.21.15.27:0/3842733309 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f04bc0e7d40 con 0x7f04d0105090 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1463522817 shutdown_connections 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1463522817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b0727e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1463522817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b0727e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1463522817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b0727de10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.591+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1463522817 >> 172.21.15.27:0/1463522817 conn(0x558b06ecd300 msgr2=0x558b0727bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.592+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1463522817 shutdown_connections 2024-09-21T07:28:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.592+0000 7f04cea00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04b40b66b0 0x7f04b40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.592+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1463522817 wait complete. 2024-09-21T07:28:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 -- start start 2024-09-21T07:28:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04cea00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04b40b66b0 0x7f04b40b8bc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f04c0005ab0 tx=0x7f04c0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 Processor -- start 2024-09-21T07:28:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b07289570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b07289ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07295880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558b0711aab0 con 0x558b06eb02f0 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558b07189570 con 0x558b06ed91b0 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b0728c590 con 0x558b06eb84d0 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.593+0000 7f04dcbc2740 10 client.0 _ll_register_callbacks cb 0x558b07198100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b07289570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b07289570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50154/0 (socket says 172.21.15.27:50154) 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 -- 172.21.15.27:0/360470180 learned_addr learned my addr 172.21.15.27:0/360470180 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cf400640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07295880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04ce000640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b07289ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 -- 172.21.15.27:0/360470180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 msgr2=0x558b07289ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b07289ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 -- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 msgr2=0x558b07295880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07295880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cea00640 1 -- 172.21.15.27:0/360470180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b06f44790 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.594+0000 7f04cf400640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07295880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04cea00640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b07289570 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f04c0033000 tx=0x7f04c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31951a3a2185f8ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04b8c00640 1 -- 172.21.15.27:0/360470180 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c004e070 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04b8c00640 1 -- 172.21.15.27:0/360470180 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f04c0004b80 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c8004050 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04b8c00640 1 -- 172.21.15.27:0/360470180 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04c005a050 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04dcbc2740 10 client.20016 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.595+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b06fbd620 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04b8c00640 1 -- 172.21.15.27:0/360470180 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c8004050 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04b8c00640 1 -- 172.21.15.27:0/360470180 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f04c0032540 con 0x558b06eb02f0 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04dcbc2740 10 client.20016 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04dcbc2740 10 client.20016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04dcbc2740 20 client.20016 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e97d80 0x558b0729b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04dcbc2740 4 client.20016 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.596+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b071f5600 con 0x558b06e97d80 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.597+0000 7f04ce000640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e97d80 0x558b0729b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.597+0000 7f04ce000640 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e97d80 0x558b0729b660 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b54e30596d7c84 server_cookie=9d6b34875ce0f2af in_seq=0 out_seq=0 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.597+0000 7f04b8c00640 10 client.20016 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04b8c00640 1 -- 172.21.15.27:0/360470180 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b071f5600 con 0x558b06e97d80 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04b8c00640 10 client.20016 handle_command_reply: tid=0 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 client.20016 shutdown 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 msgr2=0x558b07289570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b07289570 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f04c0033000 tx=0x7f04c0009260 comp rx=0 tx=0).stop 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 shutdown_connections 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e97d80 0x558b0729b660 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b07289ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07295880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/360470180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb02f0 0x558b07289570 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 >> 172.21.15.27:0/360470180 conn(0x558b06ecd300 msgr2=0x558b0727ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04b8c00640 0 client.20016 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 shutdown_connections 2024-09-21T07:28:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.598+0000 7f04dcbc2740 1 -- 172.21.15.27:0/360470180 wait complete. 2024-09-21T07:28:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.600+0000 7f04dcbc2740 20 client.20016 trim_cache size 0 max 16384 2024-09-21T07:28:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.614+0000 7f04dcbc2740 1 Processor -- start 2024-09-21T07:28:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.614+0000 7f04dcbc2740 1 -- start start 2024-09-21T07:28:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06eb02f0 0x558b07284cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b0727ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06ed91b0 0x558b07289a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b5790 con 0x558b06ed91b0 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b3060 con 0x558b06eb02f0 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b0ed0 con 0x558b06eb84d0 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06ed91b0 0x558b07289a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06eb02f0 0x558b07284cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06eb02f0 0x558b07284cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:44916/0 (socket says 172.21.15.27:44916) 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cea00640 1 -- 172.21.15.27:0/1599867386 learned_addr learned my addr 172.21.15.27:0/1599867386 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cf400640 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06ed91b0 0x558b07289a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50166/0 (socket says 172.21.15.27:50166) 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04ce000640 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b0727ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cea00640 1 -- 172.21.15.27:0/1599867386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 msgr2=0x558b0727ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cea00640 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b0727ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cea00640 1 -- 172.21.15.27:0/1599867386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06ed91b0 msgr2=0x558b07289a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.615+0000 7f04cea00640 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06ed91b0 0x558b07289a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.616+0000 7f04cea00640 1 -- 172.21.15.27:0/1599867386 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b0729bbc0 con 0x558b06eb02f0 2024-09-21T07:28:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.616+0000 7f04cea00640 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06eb02f0 0x558b07284cb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f04c0032930 tx=0x7f04c0032ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5110018f1f6dfa15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.616+0000 7f04bb400640 1 -- 172.21.15.27:0/1599867386 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0062070 con 0x558b06eb02f0 2024-09-21T07:28:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.616+0000 7f04bb400640 1 -- 172.21.15.27:0/1599867386 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c00326b0 con 0x558b06eb02f0 2024-09-21T07:28:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.616+0000 7f04bb400640 1 -- 172.21.15.27:0/1599867386 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c004e050 con 0x558b06eb02f0 2024-09-21T07:28:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1599867386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06eb02f0 msgr2=0x558b07284cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06eb02f0 0x558b07284cb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f04c0032930 tx=0x7f04c0032ef0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1599867386 shutdown_connections 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb84d0 0x558b0727ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06ed91b0 0x558b07289a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/1599867386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06eb02f0 0x558b07284cb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1599867386 >> 172.21.15.27:0/1599867386 conn(0x558b06ecd300 msgr2=0x558b0727bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1599867386 shutdown_connections 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.617+0000 7f04dcbc2740 1 -- 172.21.15.27:0/1599867386 wait complete. 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 -- start start 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 Processor -- start 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb02f0 0x558b07283490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb84d0 0x558b072839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07283f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b1520 con 0x558b06eb84d0 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b2990 con 0x558b06ed91b0 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b1bf0 con 0x558b06eb02f0 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.618+0000 7f04dcbc2740 10 client.0 _ll_register_callbacks cb 0x7f04c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb02f0 0x558b07283490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb02f0 0x558b07283490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46774/0 (socket says 172.21.15.27:46774) 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 -- 172.21.15.27:0/2161028377 learned_addr learned my addr 172.21.15.27:0/2161028377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04ce000640 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb84d0 0x558b072839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cf400640 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07283f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 -- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 msgr2=0x558b07283f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07283f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 -- 172.21.15.27:0/2161028377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb84d0 msgr2=0x558b072839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb84d0 0x558b072839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 -- 172.21.15.27:0/2161028377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b07290990 con 0x558b06eb02f0 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04cea00640 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb02f0 0x558b07283490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f04c0033000 tx=0x7f04c0032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cfc8aeabc17df52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.619+0000 7f04b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.620+0000 7f04b8c00640 1 -- 172.21.15.27:0/2161028377 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c006b020 con 0x558b06eb02f0 2024-09-21T07:28:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.620+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c00326b0 con 0x558b06eb02f0 2024-09-21T07:28:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.620+0000 7f04b8c00640 1 -- 172.21.15.27:0/2161028377 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f04c0004d90 con 0x558b06eb02f0 2024-09-21T07:28:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.620+0000 7f04b8c00640 1 -- 172.21.15.27:0/2161028377 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04c0076050 con 0x558b06eb02f0 2024-09-21T07:28:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.620+0000 7f04dcbc2740 10 client.29998 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.620+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b07291c20 con 0x558b06eb02f0 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04b8c00640 1 -- 172.21.15.27:0/2161028377 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c00326b0 con 0x558b06eb02f0 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04b8c00640 1 -- 172.21.15.27:0/2161028377 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f04c004a6f0 con 0x558b06eb02f0 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04dcbc2740 10 client.29998 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04dcbc2740 10 client.29998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04dcbc2740 20 client.29998 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e6d1c0 0x558b07308010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04dcbc2740 4 client.29998 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.621+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b071f10b0 con 0x558b06e6d1c0 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.622+0000 7f04ce000640 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e6d1c0 0x558b07308010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.622+0000 7f04ce000640 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e6d1c0 0x558b07308010 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcdccb8a142585a server_cookie=dd7c390d171dda15 in_seq=0 out_seq=0 2024-09-21T07:28:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.622+0000 7f04b8c00640 10 client.29998 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04b8c00640 1 -- 172.21.15.27:0/2161028377 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b071f10b0 con 0x558b06e6d1c0 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04b8c00640 10 client.29998 handle_command_reply: tid=0 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 client.29998 shutdown 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb02f0 msgr2=0x558b07283490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb02f0 0x558b07283490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f04c0033000 tx=0x7f04c0032d50 comp rx=0 tx=0).stop 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 shutdown_connections 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x558b06e6d1c0 0x558b07308010 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b06ed91b0 0x558b07283f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b06eb84d0 0x558b072839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 --2- 172.21.15.27:0/2161028377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b06eb02f0 0x558b07283490 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04b8c00640 0 client.29998 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 >> 172.21.15.27:0/2161028377 conn(0x558b06ecd300 msgr2=0x558b0727bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 shutdown_connections 2024-09-21T07:28:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 1 -- 172.21.15.27:0/2161028377 wait complete. 2024-09-21T07:28:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.637+0000 7f04dcbc2740 20 client.29998 trim_cache size 0 max 16384 2024-09-21T07:28:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:28:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:28:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:28:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04b40b66b0 msgr2=0x7f04b40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04b40b66b0 0x7f04b40b8bc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f04c0005ab0 tx=0x7f04c0005a20 comp rx=0 tx=0).stop 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d0105090 msgr2=0x7f04d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d0105090 0x7f04d01029b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00e9f0 tx=0x7f04bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 shutdown_connections 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d0105090 0x7f04d01029b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04b40b66b0 0x7f04b40b8bc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d010dba0 0x7f04d0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 --2- 172.21.15.27:0/3842733309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04d0104740 0x7f04d0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 >> 172.21.15.27:0/3842733309 conn(0x7f04d00fe230 msgr2=0x7f04d0107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 shutdown_connections 2024-09-21T07:28:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:26.638+0000 7f04cfe00640 1 -- 172.21.15.27:0/3842733309 wait complete. 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 -- 172.21.15.27:0/2202685835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a8110040 msgr2=0x7f21a8112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 --2- 172.21.15.27:0/2202685835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a8110040 0x7f21a8112550 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009ca0 tx=0x7f21a002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 -- 172.21.15.27:0/2202685835 shutdown_connections 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 --2- 172.21.15.27:0/2202685835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a8110040 0x7f21a8112550 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 --2- 172.21.15.27:0/2202685835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806c0e0 0x7f21a806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 --2- 172.21.15.27:0/2202685835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21a81046a0 0x7f21a806bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 -- 172.21.15.27:0/2202685835 >> 172.21.15.27:0/2202685835 conn(0x7f21a80799a0 msgr2=0x7f21a8079db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.770+0000 7f21afe00640 1 -- 172.21.15.27:0/2202685835 shutdown_connections 2024-09-21T07:28:26.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.771+0000 7f21afe00640 1 -- 172.21.15.27:0/2202685835 wait complete. 2024-09-21T07:28:26.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.772+0000 7f21afe00640 1 Processor -- start 2024-09-21T07:28:26.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.772+0000 7f21afe00640 1 -- start start 2024-09-21T07:28:26.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806c0e0 0x7f21a819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21afe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a81046a0 0x7f21a819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21afe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21a8110040 0x7f21a81a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21afe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8115ce0 con 0x7f21a81046a0 2024-09-21T07:28:26.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21afe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8115b60 con 0x7f21a806c0e0 2024-09-21T07:28:26.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21afe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8115e60 con 0x7f21a8110040 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21aca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a81046a0 0x7f21a819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21ad400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806c0e0 0x7f21a819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21aca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a81046a0 0x7f21a819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50202/0 (socket says 172.21.15.27:50202) 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21aca00640 1 -- 172.21.15.27:0/2643915586 learned_addr learned my addr 172.21.15.27:0/2643915586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.773+0000 7f21ade00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21a8110040 0x7f21a81a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21aca00640 1 -- 172.21.15.27:0/2643915586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21a8110040 msgr2=0x7f21a81a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21aca00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21a8110040 0x7f21a81a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21aca00640 1 -- 172.21.15.27:0/2643915586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806c0e0 msgr2=0x7f21a819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:26.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21aca00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806c0e0 0x7f21a819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:26.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21aca00640 1 -- 172.21.15.27:0/2643915586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a81a25a0 con 0x7f21a81046a0 2024-09-21T07:28:26.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21aca00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a81046a0 0x7f21a819cbd0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f219400e970 tx=0x7f219400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21a6000640 1 -- 172.21.15.27:0/2643915586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219400cd80 con 0x7f21a81046a0 2024-09-21T07:28:26.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a81a2890 con 0x7f21a81046a0 2024-09-21T07:28:26.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21a6000640 1 -- 172.21.15.27:0/2643915586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21940035f0 con 0x7f21a81046a0 2024-09-21T07:28:26.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.774+0000 7f21a6000640 1 -- 172.21.15.27:0/2643915586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194010690 con 0x7f21a81046a0 2024-09-21T07:28:26.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.775+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a81a8ff0 con 0x7f21a81046a0 2024-09-21T07:28:26.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.776+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a806c4f0 con 0x7f21a81046a0 2024-09-21T07:28:26.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.778+0000 7f21a6000640 1 -- 172.21.15.27:0/2643915586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2194003aa0 con 0x7f21a81046a0 2024-09-21T07:28:26.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.781+0000 7f21a6000640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21900b6680 0x7f21900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:26.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.781+0000 7f21a6000640 1 -- 172.21.15.27:0/2643915586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f21940e7e40 con 0x7f21a81046a0 2024-09-21T07:28:26.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.781+0000 7f21ad400640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21900b6680 0x7f21900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:26.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.782+0000 7f21ad400640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21900b6680 0x7f21900b8b90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f219c009bd0 tx=0x7f219c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:26.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:26.783+0000 7f21a6000640 1 -- 172.21.15.27:0/2643915586 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f21940adf80 con 0x7f21a81046a0 2024-09-21T07:28:27.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.022+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21a819d760 con 0x7f21a81046a0 2024-09-21T07:28:27.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.023+0000 7f21a6000640 1 -- 172.21.15.27:0/2643915586 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f21a819d760 con 0x7f21a81046a0 2024-09-21T07:28:27.024 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:28:27.024 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:28:27.024 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:28:27.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21900b6680 msgr2=0x7f21900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21900b6680 0x7f21900b8b90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f219c009bd0 tx=0x7f219c005f90 comp rx=0 tx=0).stop 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a81046a0 msgr2=0x7f21a819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a81046a0 0x7f21a819cbd0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f219400e970 tx=0x7f219400ee30 comp rx=0 tx=0).stop 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 shutdown_connections 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21a8110040 0x7f21a81a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f21900b6680 0x7f21900b8b90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21a81046a0 0x7f21a819cbd0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 --2- 172.21.15.27:0/2643915586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806c0e0 0x7f21a819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.025+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 >> 172.21.15.27:0/2643915586 conn(0x7f21a80799a0 msgr2=0x7f21a81014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.026+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 shutdown_connections 2024-09-21T07:28:27.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:27.026+0000 7f21afe00640 1 -- 172.21.15.27:0/2643915586 wait complete. 2024-09-21T07:28:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:26 smithi153 ceph-mon[34747]: pgmap v860: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:26 smithi119 ceph-mon[34519]: pgmap v860: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:26 smithi027 ceph-mon[30779]: pgmap v860: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:27.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:28:28.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:27 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2643915586' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:28:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:28:28.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:27 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2643915586' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:28:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:27 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2643915586' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:28:28.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 -- 172.21.15.27:0/812329010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 msgr2=0x7f36ec110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 --2- 172.21.15.27:0/812329010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec110450 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009ca0 tx=0x7f36e402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 -- 172.21.15.27:0/812329010 shutdown_connections 2024-09-21T07:28:28.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 --2- 172.21.15.27:0/812329010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec110450 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 --2- 172.21.15.27:0/812329010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36ec1092d0 0x7f36ec1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 --2- 172.21.15.27:0/812329010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec103990 0x7f36ec103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 -- 172.21.15.27:0/812329010 >> 172.21.15.27:0/812329010 conn(0x7f36ec0ff3e0 msgr2=0x7f36ec101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:28.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 -- 172.21.15.27:0/812329010 shutdown_connections 2024-09-21T07:28:28.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.723+0000 7f36eb200640 1 -- 172.21.15.27:0/812329010 wait complete. 2024-09-21T07:28:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.724+0000 7f36eb200640 1 Processor -- start 2024-09-21T07:28:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36eb200640 1 -- start start 2024-09-21T07:28:28.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36eb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec103990 0x7f36ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36eb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36ec1092d0 0x7f36ec075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36eb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36eb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec113930 con 0x7f36ec109c20 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36eb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1137b0 con 0x7f36ec1092d0 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36eb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec113ab0 con 0x7f36ec103990 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36ea800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36ea800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57862/0 (socket says 172.21.15.27:57862) 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36ea800640 1 -- 172.21.15.27:0/1708946948 learned_addr learned my addr 172.21.15.27:0/1708946948 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36e9400640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36ec1092d0 0x7f36ec075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.725+0000 7f36e9e00640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec103990 0x7f36ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.726+0000 7f36ea800640 1 -- 172.21.15.27:0/1708946948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec103990 msgr2=0x7f36ec075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.726+0000 7f36ea800640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec103990 0x7f36ec075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.726+0000 7f36ea800640 1 -- 172.21.15.27:0/1708946948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36ec1092d0 msgr2=0x7f36ec075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.726+0000 7f36ea800640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36ec1092d0 0x7f36ec075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.726+0000 7f36ea800640 1 -- 172.21.15.27:0/1708946948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec1a8580 con 0x7f36ec109c20 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.726+0000 7f36ea800640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec076190 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f36e402f540 tx=0x7f36e40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.727+0000 7f36e2a00640 1 -- 172.21.15.27:0/1708946948 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003b70 con 0x7f36ec109c20 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.727+0000 7f36e2a00640 1 -- 172.21.15.27:0/1708946948 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e4003d10 con 0x7f36ec109c20 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.727+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec1a8810 con 0x7f36ec109c20 2024-09-21T07:28:28.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.727+0000 7f36e2a00640 1 -- 172.21.15.27:0/1708946948 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e402f990 con 0x7f36ec109c20 2024-09-21T07:28:28.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.727+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a8da0 con 0x7f36ec109c20 2024-09-21T07:28:28.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.730+0000 7f36f7e4b740 1 Processor -- start 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36e2a00640 1 -- 172.21.15.27:0/1708946948 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f36e402fb30 con 0x7f36ec109c20 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36f7e4b740 1 -- start start 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd0ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd0b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd0b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5623ddc81fb0 con 0x5623dd93b1b0 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5623ddc713e0 con 0x5623dd964070 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.731+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5623ddc751f0 con 0x5623dd943390 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd0ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd0b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd0ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47464/0 (socket says 172.21.15.27:47464) 2024-09-21T07:28:28.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9400640 1 -- 172.21.15.27:0/1006017141 learned_addr learned my addr 172.21.15.27:0/1006017141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:28.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9400640 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd0b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9e00640 1 -- 172.21.15.27:0/1006017141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 msgr2=0x5623ddd0b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9e00640 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd0b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9e00640 1 -- 172.21.15.27:0/1006017141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 msgr2=0x5623ddd0ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9e00640 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd0ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.732+0000 7f36e9e00640 1 -- 172.21.15.27:0/1006017141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ddc3bcd0 con 0x5623dd93b1b0 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.733+0000 7f36e2a00640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f36d00b6700 0x7f36d00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.733+0000 7f36e9e00640 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd0b270 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009bd0 tx=0x7f36dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dc3fc0fa4c87b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.733+0000 7f36e2a00640 1 -- 172.21.15.27:0/1708946948 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f36e4108e60 con 0x7f36ec109c20 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.733+0000 7f36d7400640 1 -- 172.21.15.27:0/1006017141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc004230 con 0x5623dd93b1b0 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.733+0000 7f36e9400640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f36d00b6700 0x7f36d00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.733+0000 7f36d7400640 1 -- 172.21.15.27:0/1006017141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36dc009e40 con 0x5623dd93b1b0 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.733+0000 7f36d7400640 1 -- 172.21.15.27:0/1006017141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc004be0 con 0x5623dd93b1b0 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 -- 172.21.15.27:0/1006017141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 msgr2=0x5623ddd0b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36e9400640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f36d00b6700 0x7f36d00b8c10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009fd0 tx=0x7f36d8015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd0b270 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009bd0 tx=0x7f36dc011810 comp rx=0 tx=0).stop 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 -- 172.21.15.27:0/1006017141 shutdown_connections 2024-09-21T07:28:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd0b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd0b270 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/1006017141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd0ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 -- 172.21.15.27:0/1006017141 >> 172.21.15.27:0/1006017141 conn(0x5623dd958460 msgr2=0x5623ddd08bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:28.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 -- 172.21.15.27:0/1006017141 shutdown_connections 2024-09-21T07:28:28.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.734+0000 7f36f7e4b740 1 -- 172.21.15.27:0/1006017141 wait complete. 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.735+0000 7f36f7e4b740 1 -- start start 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.735+0000 7f36f7e4b740 1 Processor -- start 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd16e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd964070 0x5623ddd22860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5623ddad2700 con 0x5623dd964070 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5623ddc0c580 con 0x5623dd93b1b0 2024-09-21T07:28:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5623ddd19510 con 0x5623dd943390 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36f7e4b740 10 client.0 _ll_register_callbacks cb 0x5623ddc1c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36ea800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd964070 0x5623ddd22860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47474/0 (socket says 172.21.15.27:47474) 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36e9e00640 1 -- 172.21.15.27:0/2165415478 learned_addr learned my addr 172.21.15.27:0/2165415478 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36e9400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd16e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.736+0000 7f36ea800640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd964070 0x5623ddd22860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57876/0 (socket says 172.21.15.27:57876) 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36e9e00640 1 -- 172.21.15.27:0/2165415478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 msgr2=0x5623ddd16e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36e9e00640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd16e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36e9e00640 1 -- 172.21.15.27:0/2165415478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd964070 msgr2=0x5623ddd22860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36e9e00640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd964070 0x5623ddd22860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36e9e00640 1 -- 172.21.15.27:0/2165415478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623dd9cf650 con 0x5623dd93b1b0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36ea800640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd964070 0x5623ddd22860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36e9400640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd16e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36e9e00640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f36dc004880 tx=0x7f36dc019b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9d1a20fdcb4140 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36cbe00640 1 -- 172.21.15.27:0/2165415478 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc028020 con 0x5623dd93b1b0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.737+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623ddc0eb90 con 0x5623dd93b1b0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.738+0000 7f36cbe00640 1 -- 172.21.15.27:0/2165415478 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f36dc024470 con 0x5623dd93b1b0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.738+0000 7f36cbe00640 1 -- 172.21.15.27:0/2165415478 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36dc033050 con 0x5623dd93b1b0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.738+0000 7f36f7e4b740 10 client.30062 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.738+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623ddc29950 con 0x5623dd93b1b0 2024-09-21T07:28:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.738+0000 7f36cbe00640 1 -- 172.21.15.27:0/2165415478 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5623ddc0eb90 con 0x5623dd93b1b0 2024-09-21T07:28:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.738+0000 7f36cbe00640 1 -- 172.21.15.27:0/2165415478 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36dc018070 con 0x5623dd93b1b0 2024-09-21T07:28:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.739+0000 7f36f7e4b740 10 client.30062 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.739+0000 7f36f7e4b740 10 client.30062 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:28:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.739+0000 7f36f7e4b740 20 client.30062 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.739+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623dd9229f0 0x5623ddd28740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.739+0000 7f36f7e4b740 4 client.30062 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.739+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623ddc83f00 con 0x5623dd9229f0 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.739+0000 7f36e9400640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623dd9229f0 0x5623ddd28740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.740+0000 7f36e9400640 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623dd9229f0 0x5623ddd28740 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a70afda2e90b5f8 server_cookie=c220120f06863cf3 in_seq=0 out_seq=0 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.740+0000 7f36cbe00640 10 client.30062 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.740+0000 7f36cbe00640 1 -- 172.21.15.27:0/2165415478 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5623ddc83f00 con 0x5623dd9229f0 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.740+0000 7f36cbe00640 10 client.30062 handle_command_reply: tid=0 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.740+0000 7f36f7e4b740 1 client.30062 shutdown 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 msgr2=0x5623ddd16920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f36dc004880 tx=0x7f36dc019b80 comp rx=0 tx=0).stop 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 shutdown_connections 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623dd9229f0 0x5623ddd28740 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd943390 0x5623ddd16e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd964070 0x5623ddd22860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/2165415478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16920 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36cbe00640 0 client.30062 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:28:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 >> 172.21.15.27:0/2165415478 conn(0x5623dd958460 msgr2=0x5623ddd0dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 shutdown_connections 2024-09-21T07:28:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 1 -- 172.21.15.27:0/2165415478 wait complete. 2024-09-21T07:28:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.741+0000 7f36f7e4b740 20 client.30062 trim_cache size 0 max 16384 2024-09-21T07:28:28.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 Processor -- start 2024-09-21T07:28:28.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 -- start start 2024-09-21T07:28:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd943390 0x5623ddd08c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd964070 0x5623ddd22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b5700 con 0x5623dd943390 2024-09-21T07:28:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b2fd0 con 0x5623dd93b1b0 2024-09-21T07:28:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.757+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b0e40 con 0x5623dd964070 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd943390 0x5623ddd08c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd943390 0x5623ddd08c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57882/0 (socket says 172.21.15.27:57882) 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 -- 172.21.15.27:0/873317077 learned_addr learned my addr 172.21.15.27:0/873317077 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36ea800640 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd964070 0x5623ddd22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 -- 172.21.15.27:0/873317077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd964070 msgr2=0x5623ddd22760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd964070 0x5623ddd22760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 -- 172.21.15.27:0/873317077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 msgr2=0x5623ddd16f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9400640 1 -- 172.21.15.27:0/873317077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ddd179e0 con 0x5623dd943390 2024-09-21T07:28:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.758+0000 7f36e9e00640 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:28.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.759+0000 7f36e9400640 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd943390 0x5623ddd08c40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x5623ddd28e10 tx=0x7f36d8012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3f5ed7e1dc5f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.759+0000 7f36d6a00640 1 -- 172.21.15.27:0/873317077 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8022590 con 0x5623dd943390 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.759+0000 7f36d6a00640 1 -- 172.21.15.27:0/873317077 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d8022be0 con 0x5623dd943390 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.759+0000 7f36d6a00640 1 -- 172.21.15.27:0/873317077 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8021b50 con 0x5623dd943390 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 -- 172.21.15.27:0/873317077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd943390 msgr2=0x5623ddd08c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd943390 0x5623ddd08c40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x5623ddd28e10 tx=0x7f36d8012040 comp rx=0 tx=0).stop 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 -- 172.21.15.27:0/873317077 shutdown_connections 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd964070 0x5623ddd22760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd943390 0x5623ddd08c40 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/873317077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd93b1b0 0x5623ddd16f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 -- 172.21.15.27:0/873317077 >> 172.21.15.27:0/873317077 conn(0x5623dd958460 msgr2=0x5623ddd08530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:28.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 -- 172.21.15.27:0/873317077 shutdown_connections 2024-09-21T07:28:28.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.760+0000 7f36f7e4b740 1 -- 172.21.15.27:0/873317077 wait complete. 2024-09-21T07:28:28.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.761+0000 7f36f7e4b740 1 -- start start 2024-09-21T07:28:28.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.761+0000 7f36f7e4b740 1 Processor -- start 2024-09-21T07:28:28.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.761+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd1ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.761+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd1a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36f7e4b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd8f7da0 0x5623ddd1a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b1490 con 0x5623dd93b1b0 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b2900 con 0x5623dd964070 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36f7e4b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b1b60 con 0x5623dd8f7da0 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36f7e4b740 10 client.0 _ll_register_callbacks cb 0x7f36dc019790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd1ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd1ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57890/0 (socket says 172.21.15.27:57890) 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 -- 172.21.15.27:0/3889904015 learned_addr learned my addr 172.21.15.27:0/3889904015 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9400640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd1a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36ea800640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd8f7da0 0x5623ddd1a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 -- 172.21.15.27:0/3889904015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd8f7da0 msgr2=0x5623ddd1a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd8f7da0 0x5623ddd1a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 -- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 msgr2=0x5623ddd1a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:28.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd1a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9e00640 1 -- 172.21.15.27:0/3889904015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623ddd1ae30 con 0x5623dd93b1b0 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.762+0000 7f36e9400640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd1a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36e9e00640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd1ee30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0236b0 tx=0x7f36dc005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21548319e1b9aca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36cbe00640 1 -- 172.21.15.27:0/3889904015 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc028040 con 0x5623dd93b1b0 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36cbe00640 1 -- 172.21.15.27:0/3889904015 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f36dc023940 con 0x5623dd93b1b0 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d8022be0 con 0x5623dd93b1b0 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36cbe00640 1 -- 172.21.15.27:0/3889904015 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36dc01e040 con 0x5623dd93b1b0 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36f7e4b740 10 client.20046 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:28.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.763+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623ddd92e40 con 0x5623dd93b1b0 2024-09-21T07:28:28.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36cbe00640 1 -- 172.21.15.27:0/3889904015 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d8022be0 con 0x5623dd93b1b0 2024-09-21T07:28:28.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36cbe00640 1 -- 172.21.15.27:0/3889904015 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36dc018040 con 0x5623dd93b1b0 2024-09-21T07:28:28.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36f7e4b740 10 client.20046 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:28.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36f7e4b740 10 client.20046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:28:28.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36f7e4b740 20 client.20046 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:28.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623ddd935e0 0x5623ddd95a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:28.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36f7e4b740 4 client.20046 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.764+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623ddc7f9b0 con 0x5623ddd935e0 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.765+0000 7f36e9400640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623ddd935e0 0x5623ddd95a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.765+0000 7f36e9400640 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623ddd935e0 0x5623ddd95a00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=814f48a7731ac574 server_cookie=2e75ea5845017864 in_seq=0 out_seq=0 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.765+0000 7f36cbe00640 10 client.20046 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.765+0000 7f36cbe00640 1 -- 172.21.15.27:0/3889904015 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623ddc7f9b0 con 0x5623ddd935e0 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.765+0000 7f36cbe00640 10 client.20046 handle_command_reply: tid=0 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 client.20046 shutdown 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 msgr2=0x5623ddd1ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd1ee30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0236b0 tx=0x7f36dc005ce0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 shutdown_connections 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x5623ddd935e0 0x5623ddd95a00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623dd8f7da0 0x5623ddd1a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623dd964070 0x5623ddd1a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 --2- 172.21.15.27:0/3889904015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623dd93b1b0 0x5623ddd1ee30 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 >> 172.21.15.27:0/3889904015 conn(0x5623dd958460 msgr2=0x5623ddd17bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36cbe00640 0 client.20046 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 shutdown_connections 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 1 -- 172.21.15.27:0/3889904015 wait complete. 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.766+0000 7f36f7e4b740 20 client.20046 trim_cache size 0 max 16384 2024-09-21T07:28:28.770 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:28:28.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f36d00b6700 msgr2=0x7f36d00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f36d00b6700 0x7f36d00b8c10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009fd0 tx=0x7f36d8015040 comp rx=0 tx=0).stop 2024-09-21T07:28:28.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 msgr2=0x7f36ec076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:28.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec076190 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f36e402f540 tx=0x7f36e40316f0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 shutdown_connections 2024-09-21T07:28:28.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f36d00b6700 0x7f36d00b8c10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ec109c20 0x7f36ec076190 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36ec1092d0 0x7f36ec075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 --2- 172.21.15.27:0/1708946948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36ec103990 0x7f36ec075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:28.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 >> 172.21.15.27:0/1708946948 conn(0x7f36ec0ff3e0 msgr2=0x7f36ec101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:28.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 shutdown_connections 2024-09-21T07:28:28.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:28.767+0000 7f36eb200640 1 -- 172.21.15.27:0/1708946948 wait complete. 2024-09-21T07:28:29.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:28 smithi027 ceph-mon[30779]: pgmap v861: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-21T07:28:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:28 smithi153 ceph-mon[34747]: pgmap v861: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-21T07:28:29.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:28 smithi119 ceph-mon[34519]: pgmap v861: 129 pgs: 129 active+clean; 3.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-21T07:28:29.454 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:28:29.841 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.454+0000 7ff1e0c00640 1 -- 172.21.15.27:0/3341607742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc105d90 msgr2=0x7ff1dc1061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.454+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/3341607742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc105d90 0x7ff1dc1061a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009860 tx=0x7ff1cc02a340 comp rx=0 tx=0).stop 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.455+0000 7ff1e0c00640 1 -- 172.21.15.27:0/3341607742 shutdown_connections 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.455+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/3341607742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc10b6d0 0x7ff1dc10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.455+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/3341607742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1dc10c020 0x7ff1dc112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.455+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/3341607742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc105d90 0x7ff1dc1061a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.455+0000 7ff1e0c00640 1 -- 172.21.15.27:0/3341607742 >> 172.21.15.27:0/3341607742 conn(0x7ff1dc101860 msgr2=0x7ff1dc103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.455+0000 7ff1e0c00640 1 -- 172.21.15.27:0/3341607742 shutdown_connections 2024-09-21T07:28:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.455+0000 7ff1e0c00640 1 -- 172.21.15.27:0/3341607742 wait complete. 2024-09-21T07:28:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 Processor -- start 2024-09-21T07:28:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 -- start start 2024-09-21T07:28:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc105d90 0x7ff1dc19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc10b6d0 0x7ff1dc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1dc10c020 0x7ff1dc1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc115ce0 con 0x7ff1dc10b6d0 2024-09-21T07:28:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc115b60 con 0x7ff1dc10c020 2024-09-21T07:28:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.456+0000 7ff1e0c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc115e60 con 0x7ff1dc105d90 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc10b6d0 0x7ff1dc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc105d90 0x7ff1dc19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc105d90 0x7ff1dc19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33286/0 (socket says 172.21.15.27:33286) 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc10b6d0 0x7ff1dc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57908/0 (socket says 172.21.15.27:57908) 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 -- 172.21.15.27:0/2644988882 learned_addr learned my addr 172.21.15.27:0/2644988882 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 -- 172.21.15.27:0/2644988882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1dc10c020 msgr2=0x7ff1dc1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1dc10c020 0x7ff1dc1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 -- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc10b6d0 msgr2=0x7ff1dc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc10b6d0 0x7ff1dc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d9e00640 1 -- 172.21.15.27:0/2644988882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc1a2640 con 0x7ff1dc105d90 2024-09-21T07:28:30.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.457+0000 7ff1d1400640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc10b6d0 0x7ff1dc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:30.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.458+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc105d90 0x7ff1dc19c6d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc035040 tx=0x7ff1cc032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:30.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.458+0000 7ff1d3400640 1 -- 172.21.15.27:0/2644988882 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc037040 con 0x7ff1dc105d90 2024-09-21T07:28:30.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.458+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc1a28d0 con 0x7ff1dc105d90 2024-09-21T07:28:30.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.458+0000 7ff1d3400640 1 -- 172.21.15.27:0/2644988882 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc041070 con 0x7ff1dc105d90 2024-09-21T07:28:30.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.458+0000 7ff1d3400640 1 -- 172.21.15.27:0/2644988882 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0035f0 con 0x7ff1dc105d90 2024-09-21T07:28:30.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.458+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc1a9090 con 0x7ff1dc105d90 2024-09-21T07:28:30.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.462+0000 7ff1e780d740 1 Processor -- start 2024-09-21T07:28:30.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.462+0000 7ff1e780d740 1 -- start start 2024-09-21T07:28:30.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a08546da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a085472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a0817f390 0x559a08547820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1e780d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559a084bdfb0 con 0x559a081a0070 2024-09-21T07:28:30.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1e780d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a084ad3e0 con 0x559a081771b0 2024-09-21T07:28:30.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1e780d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559a084b11f0 con 0x559a0817f390 2024-09-21T07:28:30.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1da800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a08546da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1da800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a08546da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57914/0 (socket says 172.21.15.27:57914) 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1da800640 1 -- 172.21.15.27:0/4196092537 learned_addr learned my addr 172.21.15.27:0/4196092537 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.463+0000 7ff1d1400640 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a0817f390 0x559a08547820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1d1400640 1 -- 172.21.15.27:0/4196092537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 msgr2=0x559a085472e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1d1400640 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a085472e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1d1400640 1 -- 172.21.15.27:0/4196092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 msgr2=0x559a08546da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1d1400640 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a08546da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1d1400640 1 -- 172.21.15.27:0/4196092537 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a08477cd0 con 0x559a0817f390 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1da800640 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a08546da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1d3400640 1 -- 172.21.15.27:0/2644988882 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff1cc004050 con 0x7ff1dc105d90 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.464+0000 7ff1d1400640 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a0817f390 0x559a08547820 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc06c7f0 tx=0x7ff1c40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c35c6e20c537eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.465+0000 7ff1cb000640 1 -- 172.21.15.27:0/4196092537 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004310 con 0x559a0817f390 2024-09-21T07:28:30.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.465+0000 7ff1cb000640 1 -- 172.21.15.27:0/4196092537 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c40044b0 con 0x559a0817f390 2024-09-21T07:28:30.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.465+0000 7ff1cb000640 1 -- 172.21.15.27:0/4196092537 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400fa50 con 0x559a0817f390 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.465+0000 7ff1e780d740 1 -- 172.21.15.27:0/4196092537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a0817f390 msgr2=0x559a08547820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.465+0000 7ff1e780d740 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a0817f390 0x559a08547820 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc06c7f0 tx=0x7ff1c40118d0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.465+0000 7ff1e780d740 1 -- 172.21.15.27:0/4196092537 shutdown_connections 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1e780d740 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a0817f390 0x559a08547820 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1e780d740 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a085472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1e780d740 1 --2- 172.21.15.27:0/4196092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a08546da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1e780d740 1 -- 172.21.15.27:0/4196092537 >> 172.21.15.27:0/4196092537 conn(0x559a08194460 msgr2=0x559a08544c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1e780d740 1 -- 172.21.15.27:0/4196092537 shutdown_connections 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1e780d740 1 -- 172.21.15.27:0/4196092537 wait complete. 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1d3400640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1c00b6610 0x7ff1c00b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1e780d740 1 -- start start 2024-09-21T07:28:30.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1d3400640 1 -- 172.21.15.27:0/2644988882 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7ff1cc107520 con 0x7ff1dc105d90 2024-09-21T07:28:30.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.466+0000 7ff1da800640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1c00b6610 0x7ff1c00b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 1 Processor -- start 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a08552590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a0817f390 0x559a08552ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a0855e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559a0830e700 con 0x559a0817f390 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a08448580 con 0x559a081771b0 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559a08555580 con 0x559a081a0070 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.467+0000 7ff1e780d740 10 client.0 _ll_register_callbacks cb 0x559a08458470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:30.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1da800640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1c00b6610 0x7ff1c00b8b20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40049a0 tx=0x7ff1d40099b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a0817f390 0x559a08552ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a08552590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1da800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a0855e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a0817f390 0x559a08552ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57926/0 (socket says 172.21.15.27:57926) 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d1400640 1 -- 172.21.15.27:0/243188387 learned_addr learned my addr 172.21.15.27:0/243188387 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d9e00640 1 -- 172.21.15.27:0/243188387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 msgr2=0x559a0855e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a0855e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d9e00640 1 -- 172.21.15.27:0/243188387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a0817f390 msgr2=0x559a08552ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:30.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a0817f390 0x559a08552ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.468+0000 7ff1d9e00640 1 -- 172.21.15.27:0/243188387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a0820b650 con 0x559a081771b0 2024-09-21T07:28:30.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a08552590 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc104860 tx=0x7ff1cc0cce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b59814bc50953b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1bf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1bf400640 1 -- 172.21.15.27:0/243188387 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc030450 con 0x559a081771b0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1bf400640 1 -- 172.21.15.27:0/243188387 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7ff1cc0b40b0 con 0x559a081771b0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a0844ab90 con 0x559a081771b0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1bf400640 1 -- 172.21.15.27:0/243188387 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1cc0cca70 con 0x559a081771b0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1e780d740 10 client.30074 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a08465950 con 0x559a081771b0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.469+0000 7ff1bf400640 1 -- 172.21.15.27:0/243188387 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559a0844ab90 con 0x559a081771b0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1bf400640 1 -- 172.21.15.27:0/243188387 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff1cc0c88b0 con 0x559a081771b0 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1e780d740 10 client.30074 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1e780d740 10 client.30074 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:28:30.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1e780d740 20 client.30074 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1e780d740 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a0815e9f0 0x559a08564600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1e780d740 4 client.30074 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a084bff00 con 0x559a0815e9f0 2024-09-21T07:28:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.470+0000 7ff1d1400640 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a0815e9f0 0x559a08564600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.471+0000 7ff1d1400640 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a0815e9f0 0x559a08564600 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4d3edf8980cb030 server_cookie=12cec08e3c4ce9a3 in_seq=0 out_seq=0 2024-09-21T07:28:30.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.471+0000 7ff1bf400640 10 client.30074 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.471+0000 7ff1bf400640 1 -- 172.21.15.27:0/243188387 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a084bff00 con 0x559a0815e9f0 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.471+0000 7ff1bf400640 10 client.30074 handle_command_reply: tid=0 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 client.30074 shutdown 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 msgr2=0x559a08552590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a08552590 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc104860 tx=0x7ff1cc0cce60 comp rx=0 tx=0).stop 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 shutdown_connections 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a0815e9f0 0x559a08564600 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a0855e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a0817f390 0x559a08552ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 --2- 172.21.15.27:0/243188387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a081771b0 0x559a08552590 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1bf400640 0 client.30074 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:28:30.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 >> 172.21.15.27:0/243188387 conn(0x559a08194460 msgr2=0x559a08549f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:30.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 shutdown_connections 2024-09-21T07:28:30.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 1 -- 172.21.15.27:0/243188387 wait complete. 2024-09-21T07:28:30.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.472+0000 7ff1e780d740 20 client.30074 trim_cache size 0 max 16384 2024-09-21T07:28:30.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 Processor -- start 2024-09-21T07:28:30.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 -- start start 2024-09-21T07:28:30.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081771b0 0x559a08544d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a0855e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a0855ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1b57e0 con 0x559a081a0070 2024-09-21T07:28:30.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1b30b0 con 0x559a0817f390 2024-09-21T07:28:30.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1e780d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1b0f20 con 0x559a081771b0 2024-09-21T07:28:30.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1da800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a0855ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1da800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a0855ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57938/0 (socket says 172.21.15.27:57938) 2024-09-21T07:28:30.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1da800640 1 -- 172.21.15.27:0/2151810369 learned_addr learned my addr 172.21.15.27:0/2151810369 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:30.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1d1400640 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a0855e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.487+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081771b0 0x559a08544d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1d9e00640 1 -- 172.21.15.27:0/2151810369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 msgr2=0x559a0855e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:30.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a0855e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1d9e00640 1 -- 172.21.15.27:0/2151810369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 msgr2=0x559a0855ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a0855ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1d9e00640 1 -- 172.21.15.27:0/2151810369 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a08549090 con 0x559a081771b0 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1d1400640 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a0855e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081771b0 0x559a08544d00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc030d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c2216b433b1440 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1ca600640 1 -- 172.21.15.27:0/2151810369 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0cc6d0 con 0x559a081771b0 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1ca600640 1 -- 172.21.15.27:0/2151810369 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc030450 con 0x559a081771b0 2024-09-21T07:28:30.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.488+0000 7ff1ca600640 1 -- 172.21.15.27:0/2151810369 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0c88b0 con 0x559a081771b0 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 -- 172.21.15.27:0/2151810369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081771b0 msgr2=0x559a08544d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081771b0 0x559a08544d00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc030d50 comp rx=0 tx=0).stop 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 -- 172.21.15.27:0/2151810369 shutdown_connections 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081a0070 0x559a0855ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a0855e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 --2- 172.21.15.27:0/2151810369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081771b0 0x559a08544d00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 -- 172.21.15.27:0/2151810369 >> 172.21.15.27:0/2151810369 conn(0x559a08194460 msgr2=0x559a085445f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 -- 172.21.15.27:0/2151810369 shutdown_connections 2024-09-21T07:28:30.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.489+0000 7ff1e780d740 1 -- 172.21.15.27:0/2151810369 wait complete. 2024-09-21T07:28:30.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.490+0000 7ff1e780d740 1 -- start start 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.490+0000 7ff1e780d740 1 Processor -- start 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081771b0 0x559a08556eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a085573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1e780d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a085cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1e780d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1b1570 con 0x559a081771b0 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1e780d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1b29e0 con 0x559a0817f390 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1e780d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1b1c40 con 0x559a081a0070 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1e780d740 10 client.0 _ll_register_callbacks cb 0x7ff1cc0c8e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:30.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081771b0 0x559a08556eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081771b0 0x559a08556eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57940/0 (socket says 172.21.15.27:57940) 2024-09-21T07:28:30.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1da800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a085cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1d9e00640 1 -- 172.21.15.27:0/1941416216 learned_addr learned my addr 172.21.15.27:0/1941416216 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:30.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1d1400640 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a085573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1da800640 1 -- 172.21.15.27:0/1941416216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 msgr2=0x559a085573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:30.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1da800640 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a085573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1da800640 1 -- 172.21.15.27:0/1941416216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081771b0 msgr2=0x559a08556eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.491+0000 7ff1da800640 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081771b0 0x559a08556eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.492+0000 7ff1da800640 1 -- 172.21.15.27:0/1941416216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a085ce500 con 0x559a081a0070 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.492+0000 7ff1da800640 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a085cdfc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40049d0 tx=0x7ff1d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=125bfb6e60408ded server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.492+0000 7ff1bf400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.492+0000 7ff1bf400640 1 -- 172.21.15.27:0/1941416216 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d401db30 con 0x559a081a0070 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.492+0000 7ff1bf400640 1 -- 172.21.15.27:0/1941416216 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7ff1d4009020 con 0x559a081a0070 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.492+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a084bb9b0 con 0x559a081a0070 2024-09-21T07:28:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1bf400640 1 -- 172.21.15.27:0/1941416216 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1d4034050 con 0x559a081a0070 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 10 client.30040 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a085cf7b0 con 0x559a081a0070 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1bf400640 1 -- 172.21.15.27:0/1941416216 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559a084bb9b0 con 0x559a081a0070 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1bf400640 1 -- 172.21.15.27:0/1941416216 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff1d401c400 con 0x559a081a0070 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 10 client.30040 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 10 client.30040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 20 client.30040 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a08133da0 0x559a085d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:30.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 4 client.30040 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:28:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.493+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a084b8de0 con 0x559a08133da0 2024-09-21T07:28:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.494+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a08133da0 0x559a085d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.494+0000 7ff1d9e00640 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a08133da0 0x559a085d1ff0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73803057fbccc028 server_cookie=ba082600f1d04b4d in_seq=0 out_seq=0 2024-09-21T07:28:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.494+0000 7ff1bf400640 10 client.30040 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:28:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.494+0000 7ff1bf400640 1 -- 172.21.15.27:0/1941416216 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a084b8de0 con 0x559a08133da0 2024-09-21T07:28:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.494+0000 7ff1bf400640 10 client.30040 handle_command_reply: tid=0 2024-09-21T07:28:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.494+0000 7ff1e780d740 1 client.30040 shutdown 2024-09-21T07:28:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 msgr2=0x559a085cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a085cdfc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40049d0 tx=0x7ff1d4004a40 comp rx=0 tx=0).stop 2024-09-21T07:28:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 shutdown_connections 2024-09-21T07:28:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x559a08133da0 0x559a085d1ff0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a081a0070 0x559a085cdfc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a0817f390 0x559a085573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1bf400640 0 client.30040 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:28:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 --2- 172.21.15.27:0/1941416216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a081771b0 0x559a08556eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 >> 172.21.15.27:0/1941416216 conn(0x559a08194460 msgr2=0x559a08549ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 shutdown_connections 2024-09-21T07:28:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 1 -- 172.21.15.27:0/1941416216 wait complete. 2024-09-21T07:28:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.495+0000 7ff1e780d740 20 client.30040 trim_cache size 0 max 16384 2024-09-21T07:28:30.505 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1c00b6610 msgr2=0x7ff1c00b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1c00b6610 0x7ff1c00b8b20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40049a0 tx=0x7ff1d40099b0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc105d90 msgr2=0x7ff1dc19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc105d90 0x7ff1dc19c6d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc035040 tx=0x7ff1cc032d50 comp rx=0 tx=0).stop 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 shutdown_connections 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff1c00b6610 0x7ff1c00b8b20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1dc10c020 0x7ff1dc1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc10b6d0 0x7ff1dc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 --2- 172.21.15.27:0/2644988882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc105d90 0x7ff1dc19c6d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:30.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 >> 172.21.15.27:0/2644988882 conn(0x7ff1dc101860 msgr2=0x7ff1dc103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:30.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 shutdown_connections 2024-09-21T07:28:30.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:30.496+0000 7ff1e0c00640 1 -- 172.21.15.27:0/2644988882 wait complete. 2024-09-21T07:28:31.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:30 smithi027 ceph-mon[30779]: pgmap v862: 129 pgs: 129 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 16 op/s 2024-09-21T07:28:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:30 smithi153 ceph-mon[34747]: pgmap v862: 129 pgs: 129 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 16 op/s 2024-09-21T07:28:31.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:28:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:30 smithi119 ceph-mon[34519]: pgmap v862: 129 pgs: 129 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 16 op/s 2024-09-21T07:28:31.568 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 -- 172.21.15.27:0/617933408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68101920 msgr2=0x7f7e68101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 --2- 172.21.15.27:0/617933408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68101920 0x7f7e68101d30 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009900 tx=0x7f7e5802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 -- 172.21.15.27:0/617933408 shutdown_connections 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 --2- 172.21.15.27:0/617933408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e68102300 0x7f7e6810f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 --2- 172.21.15.27:0/617933408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6810fe10 0x7f7e68112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 --2- 172.21.15.27:0/617933408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68101920 0x7f7e68101d30 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 -- 172.21.15.27:0/617933408 >> 172.21.15.27:0/617933408 conn(0x7f7e68078680 msgr2=0x7f7e68078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 -- 172.21.15.27:0/617933408 shutdown_connections 2024-09-21T07:28:32.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.179+0000 7f7e67e00640 1 -- 172.21.15.27:0/617933408 wait complete. 2024-09-21T07:28:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.180+0000 7f7e67e00640 1 Processor -- start 2024-09-21T07:28:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.180+0000 7f7e67e00640 1 -- start start 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e67e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e68101920 0x7f7e6819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e67e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68102300 0x7f7e6819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e67e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e6810fe10 0x7f7e681a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e67e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e68115b00 con 0x7f7e68102300 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e67e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e68115980 con 0x7f7e68101920 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e67e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e68115c80 con 0x7f7e6810fe10 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68102300 0x7f7e6819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68102300 0x7f7e6819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57952/0 (socket says 172.21.15.27:57952) 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e5fe00640 1 -- 172.21.15.27:0/3393236677 learned_addr learned my addr 172.21.15.27:0/3393236677 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e66a00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e68101920 0x7f7e6819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.181+0000 7f7e67400640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e6810fe10 0x7f7e681a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5fe00640 1 -- 172.21.15.27:0/3393236677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e6810fe10 msgr2=0x7f7e681a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e6810fe10 0x7f7e681a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5fe00640 1 -- 172.21.15.27:0/3393236677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e68101920 msgr2=0x7f7e6819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e68101920 0x7f7e6819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5fe00640 1 -- 172.21.15.27:0/3393236677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e681a2460 con 0x7f7e68102300 2024-09-21T07:28:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68102300 0x7f7e6819ca90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9c0 tx=0x7f7e5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5f400640 1 -- 172.21.15.27:0/3393236677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000ccc0 con 0x7f7e68102300 2024-09-21T07:28:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5f400640 1 -- 172.21.15.27:0/3393236677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e500054d0 con 0x7f7e68102300 2024-09-21T07:28:32.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e681a2750 con 0x7f7e68102300 2024-09-21T07:28:32.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.182+0000 7f7e5f400640 1 -- 172.21.15.27:0/3393236677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50010430 con 0x7f7e68102300 2024-09-21T07:28:32.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.183+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e681a8eb0 con 0x7f7e68102300 2024-09-21T07:28:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 Processor -- start 2024-09-21T07:28:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 -- start start 2024-09-21T07:28:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56465092a070 0x564650cd0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cd19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564650c47fb0 con 0x564650909390 2024-09-21T07:28:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564650c373e0 con 0x56465092a070 2024-09-21T07:28:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.186+0000 7f7e74647740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564650c3b1f0 con 0x5646509011b0 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e67400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56465092a070 0x564650cd0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33360/0 (socket says 172.21.15.27:33360) 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e66a00640 1 -- 172.21.15.27:0/643347539 learned_addr learned my addr 172.21.15.27:0/643347539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e5f400640 1 -- 172.21.15.27:0/3393236677 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7e50005020 con 0x7f7e68102300 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e67400640 1 -- 172.21.15.27:0/643347539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 msgr2=0x564650cd14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e67400640 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e67400640 1 -- 172.21.15.27:0/643347539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 msgr2=0x564650cd19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cd19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e67400640 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cd19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e67400640 1 -- 172.21.15.27:0/643347539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564650c01cd0 con 0x56465092a070 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.187+0000 7f7e67400640 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56465092a070 0x564650cd0f70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009f30 tx=0x7f7e60010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c015e0febb2e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.188+0000 7f7e57e00640 1 -- 172.21.15.27:0/643347539 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60016070 con 0x56465092a070 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.188+0000 7f7e57e00640 1 -- 172.21.15.27:0/643347539 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e60004d20 con 0x56465092a070 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.188+0000 7f7e57e00640 1 -- 172.21.15.27:0/643347539 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e600127d0 con 0x56465092a070 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.188+0000 7f7e5f400640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e4c0b6680 0x7f7e4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.188+0000 7f7e74647740 1 -- 172.21.15.27:0/643347539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56465092a070 msgr2=0x564650cd0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56465092a070 0x564650cd0f70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009f30 tx=0x7f7e60010800 comp rx=0 tx=0).stop 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e5f400640 1 -- 172.21.15.27:0/3393236677 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f7e500e7eb0 con 0x7f7e68102300 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 -- 172.21.15.27:0/643347539 shutdown_connections 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cd19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 --2- 172.21.15.27:0/643347539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56465092a070 0x564650cd0f70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 -- 172.21.15.27:0/643347539 >> 172.21.15.27:0/643347539 conn(0x56465091e460 msgr2=0x564650ccee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e66a00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e4c0b6680 0x7f7e4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 -- 172.21.15.27:0/643347539 shutdown_connections 2024-09-21T07:28:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.189+0000 7f7e74647740 1 -- 172.21.15.27:0/643347539 wait complete. 2024-09-21T07:28:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 -- start start 2024-09-21T07:28:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e66a00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e4c0b6680 0x7f7e4c0b8b90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7e5802a9e0 tx=0x7f7e58005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 Processor -- start 2024-09-21T07:28:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650cdc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564650909390 0x564650cdcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56465092a070 0x564650ce8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564650a98700 con 0x56465092a070 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564650bd2580 con 0x5646509011b0 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.190+0000 7f7e74647740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564650cdf750 con 0x564650909390 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e74647740 10 client.0 _ll_register_callbacks cb 0x564650be2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650cdc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e67400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56465092a070 0x564650ce8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e67400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56465092a070 0x564650ce8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57970/0 (socket says 172.21.15.27:57970) 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e67400640 1 -- 172.21.15.27:0/2529345246 learned_addr learned my addr 172.21.15.27:0/2529345246 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564650909390 0x564650cdcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650cdc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47604/0 (socket says 172.21.15.27:47604) 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e5fe00640 1 -- 172.21.15.27:0/2529345246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 msgr2=0x564650cdc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650cdc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e5fe00640 1 -- 172.21.15.27:0/2529345246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56465092a070 msgr2=0x564650ce8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56465092a070 0x564650ce8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e5fe00640 1 -- 172.21.15.27:0/2529345246 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564650995650 con 0x564650909390 2024-09-21T07:28:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.191+0000 7f7e67400640 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56465092a070 0x564650ce8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564650909390 0x564650cdcc00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7e500e51f0 tx=0x7f7e500ad300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ed281d46171869 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e54c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e54c00640 1 -- 172.21.15.27:0/2529345246 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50005310 con 0x564650909390 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564650bd4b90 con 0x564650909390 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e54c00640 1 -- 172.21.15.27:0/2529345246 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f7e500a9240 con 0x564650909390 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e54c00640 1 -- 172.21.15.27:0/2529345246 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e500b21b0 con 0x564650909390 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e74647740 10 client.30052 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.192+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564650bef950 con 0x564650909390 2024-09-21T07:28:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e54c00640 1 -- 172.21.15.27:0/2529345246 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564650bd4b90 con 0x564650909390 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e54c00640 1 -- 172.21.15.27:0/2529345246 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e500a9440 con 0x564650909390 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e74647740 10 client.30052 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e74647740 10 client.30052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e74647740 20 client.30052 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e74647740 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5646508e89f0 0x564650cee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e74647740 4 client.30052 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564650c49f00 con 0x5646508e89f0 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.193+0000 7f7e66a00640 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5646508e89f0 0x564650cee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.194+0000 7f7e66a00640 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5646508e89f0 0x564650cee870 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e50038f751926a server_cookie=98394c28d28b668a in_seq=0 out_seq=0 2024-09-21T07:28:32.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.194+0000 7f7e54c00640 10 client.30052 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.195+0000 7f7e54c00640 1 -- 172.21.15.27:0/2529345246 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564650c49f00 con 0x5646508e89f0 2024-09-21T07:28:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.195+0000 7f7e54c00640 10 client.30052 handle_command_reply: tid=0 2024-09-21T07:28:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.195+0000 7f7e74647740 1 client.30052 shutdown 2024-09-21T07:28:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.195+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564650909390 msgr2=0x564650cdcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.195+0000 7f7e74647740 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564650909390 0x564650cdcc00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7e500e51f0 tx=0x7f7e500ad300 comp rx=0 tx=0).stop 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 shutdown_connections 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5646508e89f0 0x564650cee870 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564650909390 0x564650cdcc00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56465092a070 0x564650ce8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 --2- 172.21.15.27:0/2529345246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650cdc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e54c00640 0 client.30052 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 >> 172.21.15.27:0/2529345246 conn(0x56465091e460 msgr2=0x564650cd4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 shutdown_connections 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 1 -- 172.21.15.27:0/2529345246 wait complete. 2024-09-21T07:28:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.196+0000 7f7e74647740 20 client.30052 trim_cache size 0 max 16384 2024-09-21T07:28:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.210+0000 7f7e74647740 1 Processor -- start 2024-09-21T07:28:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.210+0000 7f7e74647740 1 -- start start 2024-09-21T07:28:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.210+0000 7f7e74647740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646508e89f0 0x564650cde6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.210+0000 7f7e74647740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.210+0000 7f7e74647740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cdb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e74647740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681b5730 con 0x564650909390 2024-09-21T07:28:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e74647740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681b3000 con 0x5646508e89f0 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e74647740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681b0e70 con 0x5646509011b0 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646508e89f0 0x564650cde6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646508e89f0 0x564650cde6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47616/0 (socket says 172.21.15.27:47616) 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 -- 172.21.15.27:0/2699533676 learned_addr learned my addr 172.21.15.27:0/2699533676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e67400640 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cdb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 -- 172.21.15.27:0/2699533676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 msgr2=0x564650cd2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 -- 172.21.15.27:0/2699533676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 msgr2=0x564650cdb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cdb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.211+0000 7f7e66a00640 1 -- 172.21.15.27:0/2699533676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564650cd3dd0 con 0x5646508e89f0 2024-09-21T07:28:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.212+0000 7f7e66a00640 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646508e89f0 0x564650cde6b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7e58005ab0 tx=0x7f7e5803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa48134e9ab97c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.212+0000 7f7e57400640 1 -- 172.21.15.27:0/2699533676 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58035040 con 0x5646508e89f0 2024-09-21T07:28:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.212+0000 7f7e57400640 1 -- 172.21.15.27:0/2699533676 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e58004ec0 con 0x5646508e89f0 2024-09-21T07:28:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.212+0000 7f7e57400640 1 -- 172.21.15.27:0/2699533676 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58032870 con 0x5646508e89f0 2024-09-21T07:28:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 -- 172.21.15.27:0/2699533676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646508e89f0 msgr2=0x564650cde6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646508e89f0 0x564650cde6b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7e58005ab0 tx=0x7f7e5803f000 comp rx=0 tx=0).stop 2024-09-21T07:28:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 -- 172.21.15.27:0/2699533676 shutdown_connections 2024-09-21T07:28:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646509011b0 0x564650cd2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650cdb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 --2- 172.21.15.27:0/2699533676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646508e89f0 0x564650cde6b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 -- 172.21.15.27:0/2699533676 >> 172.21.15.27:0/2699533676 conn(0x56465091e460 msgr2=0x564650cdaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 -- 172.21.15.27:0/2699533676 shutdown_connections 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 -- 172.21.15.27:0/2699533676 wait complete. 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.213+0000 7f7e74647740 1 -- start start 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 1 Processor -- start 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646508e89f0 0x564650ce4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650ce1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650ce1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681b14c0 con 0x564650909390 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681b2930 con 0x5646509011b0 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681b1b90 con 0x5646508e89f0 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e74647740 10 client.0 _ll_register_callbacks cb 0x7f7e500ad6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.214+0000 7f7e67400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650ce1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e67400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650ce1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57984/0 (socket says 172.21.15.27:57984) 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e67400640 1 -- 172.21.15.27:0/353940503 learned_addr learned my addr 172.21.15.27:0/353940503 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650ce1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e66a00640 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646508e89f0 0x564650ce4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e5fe00640 1 -- 172.21.15.27:0/353940503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646508e89f0 msgr2=0x564650ce4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646508e89f0 0x564650ce4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e5fe00640 1 -- 172.21.15.27:0/353940503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 msgr2=0x564650ce1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650ce1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e5fe00640 1 -- 172.21.15.27:0/353940503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564650cd5080 con 0x5646509011b0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e5fe00640 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650ce1400 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c6a0 tx=0x7f7e500e8130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd06d585237ead server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.215+0000 7f7e54c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e54c00640 1 -- 172.21.15.27:0/353940503 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500c7070 con 0x5646509011b0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e54c00640 1 -- 172.21.15.27:0/353940503 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f7e500ac5c0 con 0x5646509011b0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e58004ec0 con 0x5646509011b0 2024-09-21T07:28:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e54c00640 1 -- 172.21.15.27:0/353940503 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e500a9f40 con 0x5646509011b0 2024-09-21T07:28:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e74647740 10 client.30098 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564650d59c60 con 0x5646509011b0 2024-09-21T07:28:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e54c00640 1 -- 172.21.15.27:0/353940503 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e58004ec0 con 0x5646509011b0 2024-09-21T07:28:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e54c00640 1 -- 172.21.15.27:0/353940503 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e500ac7c0 con 0x5646509011b0 2024-09-21T07:28:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e74647740 10 client.30098 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.216+0000 7f7e74647740 10 client.30098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.217+0000 7f7e74647740 20 client.30098 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.217+0000 7f7e74647740 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56465092a070 0x564650d5c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.217+0000 7f7e74647740 4 client.30098 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:28:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.217+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564650cd8270 con 0x56465092a070 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.217+0000 7f7e66a00640 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56465092a070 0x564650d5c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e66a00640 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56465092a070 0x564650d5c440 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717e4dd67059002a server_cookie=5e946051c42f3551 in_seq=0 out_seq=0 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e54c00640 10 client.30098 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e54c00640 1 -- 172.21.15.27:0/353940503 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564650cd8270 con 0x56465092a070 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e54c00640 10 client.30098 handle_command_reply: tid=0 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 client.30098 shutdown 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 msgr2=0x564650ce1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650ce1400 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c6a0 tx=0x7f7e500e8130 comp rx=0 tx=0).stop 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 shutdown_connections 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56465092a070 0x564650d5c440 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564650909390 0x564650ce1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646509011b0 0x564650ce1400 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 --2- 172.21.15.27:0/353940503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646508e89f0 0x564650ce4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 >> 172.21.15.27:0/353940503 conn(0x56465091e460 msgr2=0x564650cd4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e54c00640 0 client.30098 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 shutdown_connections 2024-09-21T07:28:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 1 -- 172.21.15.27:0/353940503 wait complete. 2024-09-21T07:28:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.218+0000 7f7e74647740 20 client.30098 trim_cache size 0 max 16384 2024-09-21T07:28:32.225 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e4c0b6680 msgr2=0x7f7e4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e4c0b6680 0x7f7e4c0b8b90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7e5802a9e0 tx=0x7f7e58005a20 comp rx=0 tx=0).stop 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68102300 msgr2=0x7f7e6819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68102300 0x7f7e6819ca90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9c0 tx=0x7f7e5000ee80 comp rx=0 tx=0).stop 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 shutdown_connections 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e6810fe10 0x7f7e681a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7e4c0b6680 0x7f7e4c0b8b90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e68102300 0x7f7e6819ca90 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 --2- 172.21.15.27:0/3393236677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e68101920 0x7f7e6819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 >> 172.21.15.27:0/3393236677 conn(0x7f7e68078680 msgr2=0x7f7e68110530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.219+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 shutdown_connections 2024-09-21T07:28:32.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:32.220+0000 7f7e67e00640 1 -- 172.21.15.27:0/3393236677 wait complete. 2024-09-21T07:28:32.726 INFO:tasks.workunit.client.0.smithi027.stdout:499/600: open fd = 3 2024-09-21T07:28:32.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.726 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.726 INFO:tasks.workunit.client.0.smithi027.stdout:500/600: open fd = 3 2024-09-21T07:28:32.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.726 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:501/600: open fd = 3 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:502/600: open fd = 3 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.727 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:503/600: open fd = 3 2024-09-21T07:28:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:504/600: open fd = 3 2024-09-21T07:28:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.728 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.729 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.729 INFO:tasks.workunit.client.0.smithi027.stdout:505/600: open fd = 3 2024-09-21T07:28:32.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.729 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.729 INFO:tasks.workunit.client.0.smithi027.stdout:506/600: open fd = 3 2024-09-21T07:28:32.729 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.730 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.730 INFO:tasks.workunit.client.0.smithi027.stdout:507/600: open fd = 3 2024-09-21T07:28:32.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.730 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.730 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:508/600: open fd = 3 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:509/600: open fd = 3 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:510/600: open fd = 3 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.731 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:511/600: open fd = 3 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:512/600: open fd = 3 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:513/600: open fd = 3 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.732 INFO:tasks.workunit.client.0.smithi027.stdout:514/600: open fd = 3 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:515/600: open fd = 3 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:516/600: open fd = 3 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:517/600: open fd = 3 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.733 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:518/600: open fd = 3 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:519/600: open fd = 3 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:520/600: open fd = 3 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:521/600: open fd = 3 2024-09-21T07:28:32.734 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:522/600: open fd = 3 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:523/600: open fd = 3 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.735 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:524/600: open fd = 3 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:525/600: open fd = 3 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:526/600: open fd = 3 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.736 INFO:tasks.workunit.client.0.smithi027.stdout:527/600: open fd = 3 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:528/600: open fd = 3 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:529/600: open fd = 3 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:530/600: open fd = 3 2024-09-21T07:28:32.737 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:531/600: open fd = 3 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:532/600: open fd = 3 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:533/600: open fd = 3 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.738 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:534/600: open fd = 3 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:535/600: open fd = 3 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.739 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:536/600: open fd = 3 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:537/600: open fd = 3 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:538/600: open fd = 3 2024-09-21T07:28:32.740 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:539/600: open fd = 3 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:540/600: open fd = 3 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.741 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:541/600: open fd = 3 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:542/600: open fd = 3 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.742 INFO:tasks.workunit.client.0.smithi027.stdout:543/600: open fd = 3 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:544/600: open fd = 3 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:545/600: open fd = 3 2024-09-21T07:28:32.743 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:546/600: open fd = 3 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:547/600: open fd = 3 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:548/600: open fd = 3 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.744 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:549/600: open fd = 3 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:550/600: open fd = 3 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:551/600: open fd = 3 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:552/600: open fd = 3 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.745 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:553/600: open fd = 3 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:554/600: open fd = 3 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:555/600: open fd = 3 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.746 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:556/600: open fd = 3 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:557/600: open fd = 3 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:558/600: open fd = 3 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:559/600: open fd = 3 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.747 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.748 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:32.748 INFO:tasks.workunit.client.0.smithi027.stdout:560/600: open fd = 3 2024-09-21T07:28:32.748 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.748 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:28:32.748 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:28:33.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:32 smithi153 ceph-mon[34747]: pgmap v863: 129 pgs: 129 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:32 smithi119 ceph-mon[34519]: pgmap v863: 129 pgs: 129 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:32 smithi027 ceph-mon[30779]: pgmap v863: 129 pgs: 129 active+clean; 3.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:33.968 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:28:33.968 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:28:34.370 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:28:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 -- 172.21.15.27:0/242216549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2680075750 msgr2=0x7f2680075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 --2- 172.21.15.27:0/242216549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2680075750 0x7f2680075b60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2670009900 tx=0x7f267002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 -- 172.21.15.27:0/242216549 shutdown_connections 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 --2- 172.21.15.27:0/242216549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f268010ff50 0x7f2680112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 --2- 172.21.15.27:0/242216549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26800760a0 0x7f268010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 --2- 172.21.15.27:0/242216549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2680075750 0x7f2680075b60 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 -- 172.21.15.27:0/242216549 >> 172.21.15.27:0/242216549 conn(0x7f26800ff3e0 msgr2=0x7f2680101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 -- 172.21.15.27:0/242216549 shutdown_connections 2024-09-21T07:28:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.052+0000 7f2685800640 1 -- 172.21.15.27:0/242216549 wait complete. 2024-09-21T07:28:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.054+0000 7f2685800640 1 Processor -- start 2024-09-21T07:28:35.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.054+0000 7f2685800640 1 -- start start 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.054+0000 7f2685800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2680075750 0x7f2680106a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.054+0000 7f2685800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26800760a0 0x7f2680106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.054+0000 7f2685800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f268010ff50 0x7f26801074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f2685800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2680115b10 con 0x7f26800760a0 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f2685800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2680115990 con 0x7f268010ff50 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f2685800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2680115c90 con 0x7f2680075750 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2680075750 0x7f2680106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26800760a0 0x7f2680106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2680075750 0x7f2680106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33410/0 (socket says 172.21.15.27:33410) 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 -- 172.21.15.27:0/1653750194 learned_addr learned my addr 172.21.15.27:0/1653750194 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f2684e00640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f268010ff50 0x7f26801074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 -- 172.21.15.27:0/1653750194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f268010ff50 msgr2=0x7f26801074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f268010ff50 0x7f26801074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 -- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26800760a0 msgr2=0x7f2680106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26800760a0 0x7f2680106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.055+0000 7f267fe00640 1 -- 172.21.15.27:0/1653750194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26801ac9c0 con 0x7f2680075750 2024-09-21T07:28:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.056+0000 7f267fe00640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2680075750 0x7f2680106a30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2670005ab0 tx=0x7f2670003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.056+0000 7f267cc00640 1 -- 172.21.15.27:0/1653750194 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267003f070 con 0x7f2680075750 2024-09-21T07:28:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.056+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26801acc50 con 0x7f2680075750 2024-09-21T07:28:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.057+0000 7f267cc00640 1 -- 172.21.15.27:0/1653750194 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2670031b60 con 0x7f2680075750 2024-09-21T07:28:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.057+0000 7f267cc00640 1 -- 172.21.15.27:0/1653750194 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670035030 con 0x7f2680075750 2024-09-21T07:28:35.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.057+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801ad180 con 0x7f2680075750 2024-09-21T07:28:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.060+0000 7f268da03740 1 Processor -- start 2024-09-21T07:28:35.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.060+0000 7f268da03740 1 -- start start 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f268da03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef804350 0x563befbabf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f268da03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7db490 0x563befbac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f268da03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbac980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f268da03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563befb21e90 con 0x563bef804350 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f268da03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563befb112c0 con 0x563bef7db490 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f268da03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563befb150d0 con 0x563bef7e3670 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f2684e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef804350 0x563befbabf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f2684e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef804350 0x563befbabf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58024/0 (socket says 172.21.15.27:58024) 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f2684e00640 1 -- 172.21.15.27:0/2248947263 learned_addr learned my addr 172.21.15.27:0/2248947263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f267f400640 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f267cc00640 1 -- 172.21.15.27:0/1653750194 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2670042070 con 0x7f2680075750 2024-09-21T07:28:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.061+0000 7f267f400640 1 -- 172.21.15.27:0/2248947263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7db490 msgr2=0x563befbac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f267f400640 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7db490 0x563befbac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f267f400640 1 -- 172.21.15.27:0/2248947263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef804350 msgr2=0x563befbabf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f267f400640 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef804350 0x563befbabf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f267f400640 1 -- 172.21.15.27:0/2248947263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bef86f930 con 0x563bef7e3670 2024-09-21T07:28:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f2684e00640 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef804350 0x563befbabf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:35.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f267f400640 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbac980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f268006c810 tx=0x7f266c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=823a8c9a9ae20549 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:35.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f2675600640 1 -- 172.21.15.27:0/2248947263 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c004050 con 0x563bef7e3670 2024-09-21T07:28:35.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f2675600640 1 -- 172.21.15.27:0/2248947263 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0054d0 con 0x563bef7e3670 2024-09-21T07:28:35.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.062+0000 7f2675600640 1 -- 172.21.15.27:0/2248947263 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c004c80 con 0x563bef7e3670 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f267cc00640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26640b6700 0x7f26640b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f268da03740 1 -- 172.21.15.27:0/2248947263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 msgr2=0x563befbac980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f268da03740 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbac980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f268006c810 tx=0x7f266c011810 comp rx=0 tx=0).stop 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f267cc00640 1 -- 172.21.15.27:0/1653750194 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f26701081d0 con 0x7f2680075750 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f2684e00640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26640b6700 0x7f26640b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f268da03740 1 -- 172.21.15.27:0/2248947263 shutdown_connections 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f268da03740 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbac980 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f268da03740 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7db490 0x563befbac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f268da03740 1 --2- 172.21.15.27:0/2248947263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef804350 0x563befbabf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.063+0000 7f268da03740 1 -- 172.21.15.27:0/2248947263 >> 172.21.15.27:0/2248947263 conn(0x563bef7f8740 msgr2=0x563befba9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.064+0000 7f268da03740 1 -- 172.21.15.27:0/2248947263 shutdown_connections 2024-09-21T07:28:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.064+0000 7f268da03740 1 -- 172.21.15.27:0/2248947263 wait complete. 2024-09-21T07:28:35.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.064+0000 7f2684e00640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26640b6700 0x7f26640b8c10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2678009df0 tx=0x7f2678009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.064+0000 7f268da03740 1 -- start start 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 1 Processor -- start 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbb7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7e3670 0x563befbb80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef804350 0x563befbc39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563bef972890 con 0x563bef7db490 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563befaac160 con 0x563bef7e3670 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563befbba6e0 con 0x563bef804350 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f268da03740 10 client.0 _ll_register_callbacks cb 0x563befac8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f2684e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef804350 0x563befbc39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f267f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7e3670 0x563befbb80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f2684e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef804350 0x563befbc39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33428/0 (socket says 172.21.15.27:33428) 2024-09-21T07:28:35.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f2684e00640 1 -- 172.21.15.27:0/4229503142 learned_addr learned my addr 172.21.15.27:0/4229503142 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:35.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.065+0000 7f267fe00640 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbb7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.066+0000 7f2684e00640 1 -- 172.21.15.27:0/4229503142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7e3670 msgr2=0x563befbb80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.066+0000 7f2684e00640 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7e3670 0x563befbb80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.066+0000 7f2684e00640 1 -- 172.21.15.27:0/4229503142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 msgr2=0x563befbb7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.066+0000 7f2684e00640 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbb7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.066+0000 7f2684e00640 1 -- 172.21.15.27:0/4229503142 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563befbc3fc0 con 0x563bef804350 2024-09-21T07:28:35.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.066+0000 7f2684e00640 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef804350 0x563befbc39d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2678003f80 tx=0x7f2678004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae5a83c8379c2fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:35.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.066+0000 7f266a000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:35.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.067+0000 7f266a000640 1 -- 172.21.15.27:0/4229503142 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267800f890 con 0x563bef804350 2024-09-21T07:28:35.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.067+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266c0054d0 con 0x563bef804350 2024-09-21T07:28:35.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.067+0000 7f266a000640 1 -- 172.21.15.27:0/4229503142 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f267800fa30 con 0x563bef804350 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.067+0000 7f266a000640 1 -- 172.21.15.27:0/4229503142 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f267800fc30 con 0x563bef804350 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.067+0000 7f268da03740 10 client.30070 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.067+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563befbc69a0 con 0x563bef804350 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.067+0000 7f266a000640 1 -- 172.21.15.27:0/4229503142 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0054d0 con 0x563bef804350 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f266a000640 1 -- 172.21.15.27:0/4229503142 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2678027070 con 0x563bef804350 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f268da03740 10 client.30070 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f268da03740 10 client.30070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f268da03740 20 client.30070 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f268da03740 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef7c2cd0 0x563befbc9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f268da03740 4 client.30070 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563befb23de0 con 0x563bef7c2cd0 2024-09-21T07:28:35.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.068+0000 7f267fe00640 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef7c2cd0 0x563befbc9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.069+0000 7f267fe00640 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef7c2cd0 0x563befbc9ef0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc667e4788427952 server_cookie=aac86395e8e85b9 in_seq=0 out_seq=0 2024-09-21T07:28:35.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.069+0000 7f266a000640 10 client.30070 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:35.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f266a000640 1 -- 172.21.15.27:0/4229503142 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563befb23de0 con 0x563bef7c2cd0 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f266a000640 10 client.30070 handle_command_reply: tid=0 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 client.30070 shutdown 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef804350 msgr2=0x563befbc39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef804350 0x563befbc39d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2678003f80 tx=0x7f2678004ab0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 shutdown_connections 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef7c2cd0 0x563befbc9ef0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef804350 0x563befbc39d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef7e3670 0x563befbb80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 --2- 172.21.15.27:0/4229503142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbb7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f266a000640 0 client.30070 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 >> 172.21.15.27:0/4229503142 conn(0x563bef7f8740 msgr2=0x563befbaf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 shutdown_connections 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.070+0000 7f268da03740 1 -- 172.21.15.27:0/4229503142 wait complete. 2024-09-21T07:28:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.071+0000 7f268da03740 20 client.30070 trim_cache size 0 max 16384 2024-09-21T07:28:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.089+0000 7f268da03740 1 Processor -- start 2024-09-21T07:28:35.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.089+0000 7f268da03740 1 -- start start 2024-09-21T07:28:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f268da03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7db490 0x563befbb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f268da03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7e3670 0x563befbad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f268da03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befba93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f268da03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26801b9be0 con 0x563bef7e3670 2024-09-21T07:28:35.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f268da03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26801b74b0 con 0x563bef804350 2024-09-21T07:28:35.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f268da03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26801b5320 con 0x563bef7db490 2024-09-21T07:28:35.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f267f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7e3670 0x563befbad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f267f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7e3670 0x563befbad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58040/0 (socket says 172.21.15.27:58040) 2024-09-21T07:28:35.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f267f400640 1 -- 172.21.15.27:0/2724193626 learned_addr learned my addr 172.21.15.27:0/2724193626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:35.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f267fe00640 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7db490 0x563befbb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.090+0000 7f2684e00640 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befba93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f267fe00640 1 -- 172.21.15.27:0/2724193626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 msgr2=0x563befba93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f267fe00640 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befba93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f267fe00640 1 -- 172.21.15.27:0/2724193626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7e3670 msgr2=0x563befbad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f267fe00640 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7e3670 0x563befbad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f267fe00640 1 -- 172.21.15.27:0/2724193626 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563befbb5e40 con 0x563bef7db490 2024-09-21T07:28:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f267f400640 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7e3670 0x563befbad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f267fe00640 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7db490 0x563befbb1680 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2670105510 tx=0x7f26700045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0531c915fa7268e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f2674c00640 1 -- 172.21.15.27:0/2724193626 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670049040 con 0x563bef7db490 2024-09-21T07:28:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f2674c00640 1 -- 172.21.15.27:0/2724193626 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26700cd720 con 0x563bef7db490 2024-09-21T07:28:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.091+0000 7f2674c00640 1 -- 172.21.15.27:0/2724193626 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267003b580 con 0x563bef7db490 2024-09-21T07:28:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 -- 172.21.15.27:0/2724193626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7db490 msgr2=0x563befbb1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7db490 0x563befbb1680 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2670105510 tx=0x7f26700045e0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 -- 172.21.15.27:0/2724193626 shutdown_connections 2024-09-21T07:28:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befba93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7e3670 0x563befbad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 --2- 172.21.15.27:0/2724193626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7db490 0x563befbb1680 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 -- 172.21.15.27:0/2724193626 >> 172.21.15.27:0/2724193626 conn(0x563bef7f8740 msgr2=0x563befbacf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:35.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 -- 172.21.15.27:0/2724193626 shutdown_connections 2024-09-21T07:28:35.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.092+0000 7f268da03740 1 -- 172.21.15.27:0/2724193626 wait complete. 2024-09-21T07:28:35.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.093+0000 7f268da03740 1 -- start start 2024-09-21T07:28:35.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.093+0000 7f268da03740 1 Processor -- start 2024-09-21T07:28:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.093+0000 7f268da03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbbbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.093+0000 7f268da03740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbbbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.093+0000 7f268da03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befbbc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f268da03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26801b5970 con 0x563bef7db490 2024-09-21T07:28:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f268da03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26801b6de0 con 0x563bef804350 2024-09-21T07:28:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f268da03740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26801b6040 con 0x563bef7e3670 2024-09-21T07:28:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f268da03740 10 client.0 _ll_register_callbacks cb 0x7f2678009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbbbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbbbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbbbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58054/0 (socket says 172.21.15.27:58054) 2024-09-21T07:28:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267fe00640 1 -- 172.21.15.27:0/4116535757 learned_addr learned my addr 172.21.15.27:0/4116535757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f2684e00640 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befbbc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267f400640 1 -- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 msgr2=0x563befbbc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267f400640 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befbbc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267f400640 1 -- 172.21.15.27:0/4116535757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 msgr2=0x563befbbbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267f400640 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbbbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.094+0000 7f267f400640 1 -- 172.21.15.27:0/4116535757 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563befc33a60 con 0x563bef7e3670 2024-09-21T07:28:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f267f400640 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbbbfe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f266c014040 tx=0x7f266c009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d23e2fcc98773ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f266a000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f266a000640 1 -- 172.21.15.27:0/4116535757 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c01d040 con 0x563bef7e3670 2024-09-21T07:28:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f266a000640 1 -- 172.21.15.27:0/4116535757 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f266c004520 con 0x563bef7e3670 2024-09-21T07:28:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26700cd720 con 0x563bef7e3670 2024-09-21T07:28:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f266a000640 1 -- 172.21.15.27:0/4116535757 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f266c036050 con 0x563bef7e3670 2024-09-21T07:28:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f268da03740 10 client.30076 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.095+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563befc34d00 con 0x563bef7e3670 2024-09-21T07:28:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f266a000640 1 -- 172.21.15.27:0/4116535757 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26700cd720 con 0x563bef7e3670 2024-09-21T07:28:35.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f266a000640 1 -- 172.21.15.27:0/4116535757 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f266c02b020 con 0x563bef7e3670 2024-09-21T07:28:35.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f268da03740 10 client.30076 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:35.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f268da03740 10 client.30076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:35.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f268da03740 20 client.30076 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:35.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f268da03740 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef798080 0x563befc37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:35.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f268da03740 4 client.30076 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:28:35.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563befb1f890 con 0x563bef798080 2024-09-21T07:28:35.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.096+0000 7f267fe00640 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef798080 0x563befc37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:35.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.097+0000 7f267fe00640 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef798080 0x563befc37280 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc0a140cc34989d server_cookie=de596c14882b71a3 in_seq=0 out_seq=0 2024-09-21T07:28:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.097+0000 7f266a000640 10 client.30076 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.098+0000 7f266a000640 1 -- 172.21.15.27:0/4116535757 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563befb1f890 con 0x563bef798080 2024-09-21T07:28:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.098+0000 7f266a000640 10 client.30076 handle_command_reply: tid=0 2024-09-21T07:28:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.098+0000 7f268da03740 1 client.30076 shutdown 2024-09-21T07:28:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 msgr2=0x563befbbbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbbbfe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f266c014040 tx=0x7f266c009ed0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 shutdown_connections 2024-09-21T07:28:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x563bef798080 0x563befc37280 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563bef7e3670 0x563befbbbfe0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bef804350 0x563befbbc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 --2- 172.21.15.27:0/4116535757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563bef7db490 0x563befbbbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f266a000640 0 client.30076 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:35.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 >> 172.21.15.27:0/4116535757 conn(0x563bef7f8740 msgr2=0x563befbacf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:35.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 shutdown_connections 2024-09-21T07:28:35.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 1 -- 172.21.15.27:0/4116535757 wait complete. 2024-09-21T07:28:35.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.099+0000 7f268da03740 20 client.30076 trim_cache size 0 max 16384 2024-09-21T07:28:35.115 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:28:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:28:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "594683c9-bab5-4ebb-ae50-7e80b1ad2ba8", 2024-09-21T07:28:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:28:35.115 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:28:35.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26640b6700 msgr2=0x7f26640b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26640b6700 0x7f26640b8c10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2678009df0 tx=0x7f2678009500 comp rx=0 tx=0).stop 2024-09-21T07:28:35.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2680075750 msgr2=0x7f2680106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2680075750 0x7f2680106a30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2670005ab0 tx=0x7f2670003c10 comp rx=0 tx=0).stop 2024-09-21T07:28:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 shutdown_connections 2024-09-21T07:28:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f26640b6700 0x7f26640b8c10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f268010ff50 0x7f26801074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26800760a0 0x7f2680106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 --2- 172.21.15.27:0/1653750194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2680075750 0x7f2680106a30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:35.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 >> 172.21.15.27:0/1653750194 conn(0x7f26800ff3e0 msgr2=0x7f2680109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:35.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.100+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 shutdown_connections 2024-09-21T07:28:35.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:35.101+0000 7f2685800640 1 -- 172.21.15.27:0/1653750194 wait complete. 2024-09-21T07:28:35.124 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:34 smithi027 ceph-mon[30779]: pgmap v864: 129 pgs: 129 active+clean; 4.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:28:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:34 smithi153 ceph-mon[34747]: pgmap v864: 129 pgs: 129 active+clean; 4.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:28:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:34 smithi119 ceph-mon[34519]: pgmap v864: 129 pgs: 129 active+clean; 4.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-21T07:28:36.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:28:36.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:35 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:28:36.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:35 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:28:36.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:35 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:28:36.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:35 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:28:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:35 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:28:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:35 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:28:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:35 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:28:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:35 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:28:36.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:35 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:28:36.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:35 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:28:36.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:35 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:28:36.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:35 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:28:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:28:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.035+0000 7f4d3e000640 1 -- 172.21.15.27:0/2254407770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d38109e10 msgr2=0x7f4d38110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.035+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2254407770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d38109e10 0x7f4d38110640 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009900 tx=0x7f4d3002a410 comp rx=0 tx=0).stop 2024-09-21T07:28:37.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.036+0000 7f4d3e000640 1 -- 172.21.15.27:0/2254407770 shutdown_connections 2024-09-21T07:28:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.036+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2254407770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d38109e10 0x7f4d38110640 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.036+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2254407770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d381094c0 0x7f4d381098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.036+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2254407770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d38103b80 0x7f4d38103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.036+0000 7f4d3e000640 1 -- 172.21.15.27:0/2254407770 >> 172.21.15.27:0/2254407770 conn(0x7f4d380ff5d0 msgr2=0x7f4d38101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.036+0000 7f4d3e000640 1 -- 172.21.15.27:0/2254407770 shutdown_connections 2024-09-21T07:28:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.036+0000 7f4d3e000640 1 -- 172.21.15.27:0/2254407770 wait complete. 2024-09-21T07:28:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 Processor -- start 2024-09-21T07:28:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 -- start start 2024-09-21T07:28:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38103b80 0x7f4d3819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d381094c0 0x7f4d3819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d38109e10 0x7f4d3819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38113b20 con 0x7f4d38109e10 2024-09-21T07:28:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381139a0 con 0x7f4d38103b80 2024-09-21T07:28:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.037+0000 7f4d3e000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38113ca0 con 0x7f4d381094c0 2024-09-21T07:28:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38103b80 0x7f4d3819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38103b80 0x7f4d3819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:47694/0 (socket says 172.21.15.27:47694) 2024-09-21T07:28:37.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 -- 172.21.15.27:0/2454617624 learned_addr learned my addr 172.21.15.27:0/2454617624 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:37.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37e00640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d38109e10 0x7f4d3819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d381094c0 0x7f4d3819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 -- 172.21.15.27:0/2454617624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d381094c0 msgr2=0x7f4d3819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d381094c0 0x7f4d3819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 -- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d38109e10 msgr2=0x7f4d3819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d38109e10 0x7f4d3819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d37400640 1 -- 172.21.15.27:0/2454617624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d381a0400 con 0x7f4d38103b80 2024-09-21T07:28:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.038+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d381094c0 0x7f4d3819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:28:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.039+0000 7f4d37400640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38103b80 0x7f4d3819a490 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009d50 tx=0x7f4d2800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.039+0000 7f4d34c00640 1 -- 172.21.15.27:0/2454617624 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28003b70 con 0x7f4d38103b80 2024-09-21T07:28:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.039+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d381a06f0 con 0x7f4d38103b80 2024-09-21T07:28:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.039+0000 7f4d34c00640 1 -- 172.21.15.27:0/2454617624 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d28003d10 con 0x7f4d38103b80 2024-09-21T07:28:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.039+0000 7f4d34c00640 1 -- 172.21.15.27:0/2454617624 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d280108b0 con 0x7f4d38103b80 2024-09-21T07:28:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.039+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d380755f0 con 0x7f4d38103b80 2024-09-21T07:28:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.042+0000 7f4d44c6b740 1 Processor -- start 2024-09-21T07:28:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.042+0000 7f4d44c6b740 1 -- start start 2024-09-21T07:28:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d5048dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d5048e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5048e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559d50404f70 con 0x559d500e7070 2024-09-21T07:28:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559d503f43a0 con 0x559d500be1b0 2024-09-21T07:28:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559d503f81b0 con 0x559d500c6390 2024-09-21T07:28:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d37e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d5048dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d37e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d5048dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58094/0 (socket says 172.21.15.27:58094) 2024-09-21T07:28:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d37e00640 1 -- 172.21.15.27:0/1759414743 learned_addr learned my addr 172.21.15.27:0/1759414743 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5048e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d2ea00640 1 -- 172.21.15.27:0/1759414743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 msgr2=0x559d5048e2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:28:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d5048e2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d2ea00640 1 -- 172.21.15.27:0/1759414743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 msgr2=0x559d5048dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d5048dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.043+0000 7f4d2ea00640 1 -- 172.21.15.27:0/1759414743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d503bec90 con 0x559d500c6390 2024-09-21T07:28:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.044+0000 7f4d34c00640 1 -- 172.21.15.27:0/2454617624 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4d28004000 con 0x7f4d38103b80 2024-09-21T07:28:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.044+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5048e7e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4d3806c780 tx=0x7f4d20010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9f8c974112a5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.044+0000 7f4d27e00640 1 -- 172.21.15.27:0/1759414743 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20004030 con 0x559d500c6390 2024-09-21T07:28:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.044+0000 7f4d27e00640 1 -- 172.21.15.27:0/1759414743 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d200035f0 con 0x559d500c6390 2024-09-21T07:28:37.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.044+0000 7f4d27e00640 1 -- 172.21.15.27:0/1759414743 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d200126b0 con 0x559d500c6390 2024-09-21T07:28:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.045+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1759414743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 msgr2=0x559d5048e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.045+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5048e7e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4d3806c780 tx=0x7f4d20010740 comp rx=0 tx=0).stop 2024-09-21T07:28:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.045+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1759414743 shutdown_connections 2024-09-21T07:28:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.045+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5048e7e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.045+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d5048e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.045+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1759414743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d5048dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.045+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1759414743 >> 172.21.15.27:0/1759414743 conn(0x559d500db460 msgr2=0x559d5048bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.046+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1759414743 shutdown_connections 2024-09-21T07:28:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.046+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1759414743 wait complete. 2024-09-21T07:28:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.046+0000 7f4d34c00640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4d1c0b6590 0x7f4d1c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.046+0000 7f4d34c00640 1 -- 172.21.15.27:0/2454617624 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f4d280e8270 con 0x7f4d38103b80 2024-09-21T07:28:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.046+0000 7f4d44c6b740 1 -- start start 2024-09-21T07:28:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.046+0000 7f4d37e00640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4d1c0b6590 0x7f4d1c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 1 Processor -- start 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d50499570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d50499ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d37e00640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4d1c0b6590 0x7f4d1c0b8aa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4d300096c0 tx=0x7f4d30005a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d504a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559d50255700 con 0x559d500c6390 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559d5038f540 con 0x559d500e7070 2024-09-21T07:28:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559d5049c540 con 0x559d500be1b0 2024-09-21T07:28:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d44c6b740 10 client.0 _ll_register_callbacks cb 0x559d5039f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.047+0000 7f4d37400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d50499570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d50499570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33476/0 (socket says 172.21.15.27:33476) 2024-09-21T07:28:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 -- 172.21.15.27:0/1780377693 learned_addr learned my addr 172.21.15.27:0/1780377693 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37e00640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d504a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d50499ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 -- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 msgr2=0x559d504a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d504a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 -- 172.21.15.27:0/1780377693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 msgr2=0x559d50499ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d50499ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 -- 172.21.15.27:0/1780377693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d50152650 con 0x559d500be1b0 2024-09-21T07:28:37.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37e00640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d504a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d37400640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d50499570 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009d20 tx=0x7f4d28004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f8816a020f53f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.048+0000 7f4d24c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d24c00640 1 -- 172.21.15.27:0/1780377693 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d280ae3b0 con 0x559d500be1b0 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d24c00640 1 -- 172.21.15.27:0/1780377693 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f4d280a9f20 con 0x559d500be1b0 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d50391b50 con 0x559d500be1b0 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d24c00640 1 -- 172.21.15.27:0/1780377693 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d280c3e60 con 0x559d500be1b0 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d44c6b740 10 client.30085 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559d503ac910 con 0x559d500be1b0 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d24c00640 1 -- 172.21.15.27:0/1780377693 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559d50391b50 con 0x559d500be1b0 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.049+0000 7f4d24c00640 1 -- 172.21.15.27:0/1780377693 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d280a9600 con 0x559d500be1b0 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.050+0000 7f4d44c6b740 10 client.30085 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.050+0000 7f4d44c6b740 10 client.30085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.050+0000 7f4d44c6b740 20 client.30085 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.050+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d500a59f0 0x559d504ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.050+0000 7f4d44c6b740 4 client.30085 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:37.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.050+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d50406ec0 con 0x559d500a59f0 2024-09-21T07:28:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.050+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d500a59f0 0x559d504ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.051+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d500a59f0 0x559d504ab610 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9797eea8110d5b9a server_cookie=fd063b30d41b04d7 in_seq=0 out_seq=0 2024-09-21T07:28:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.051+0000 7f4d24c00640 10 client.30085 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d24c00640 1 -- 172.21.15.27:0/1780377693 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d50406ec0 con 0x559d500a59f0 2024-09-21T07:28:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d24c00640 10 client.30085 handle_command_reply: tid=0 2024-09-21T07:28:37.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 client.30085 shutdown 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 msgr2=0x559d50499570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d50499570 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009d20 tx=0x7f4d28004790 comp rx=0 tx=0).stop 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 shutdown_connections 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d500a59f0 0x559d504ab610 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d504a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d50499ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d24c00640 0 client.30085 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:37.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/1780377693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d50499570 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 >> 172.21.15.27:0/1780377693 conn(0x559d500db460 msgr2=0x559d50490f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 shutdown_connections 2024-09-21T07:28:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 1 -- 172.21.15.27:0/1780377693 wait complete. 2024-09-21T07:28:37.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.052+0000 7f4d44c6b740 20 client.30085 trim_cache size 0 max 16384 2024-09-21T07:28:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 Processor -- start 2024-09-21T07:28:37.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 -- start start 2024-09-21T07:28:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d50491820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5049b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d504a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b5990 con 0x559d500e7070 2024-09-21T07:28:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b3260 con 0x559d500be1b0 2024-09-21T07:28:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.071+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b10d0 con 0x559d500c6390 2024-09-21T07:28:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d504a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d504a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58118/0 (socket says 172.21.15.27:58118) 2024-09-21T07:28:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 -- 172.21.15.27:0/2177623840 learned_addr learned my addr 172.21.15.27:0/2177623840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37400640 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d50491820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5049b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 -- 172.21.15.27:0/2177623840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 msgr2=0x559d5049b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5049b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 -- 172.21.15.27:0/2177623840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 msgr2=0x559d50491820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d50491820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37e00640 1 -- 172.21.15.27:0/2177623840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d5049a3f0 con 0x559d500e7070 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.072+0000 7f4d37400640 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d50491820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.073+0000 7f4d37e00640 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d504a5760 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009900 tx=0x7f4d30040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40dad778bbc1784 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.073+0000 7f4d27400640 1 -- 172.21.15.27:0/2177623840 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30035040 con 0x559d500e7070 2024-09-21T07:28:37.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.073+0000 7f4d27400640 1 -- 172.21.15.27:0/2177623840 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d30003cf0 con 0x559d500e7070 2024-09-21T07:28:37.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.073+0000 7f4d27400640 1 -- 172.21.15.27:0/2177623840 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30052460 con 0x559d500e7070 2024-09-21T07:28:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 -- 172.21.15.27:0/2177623840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 msgr2=0x559d504a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d504a5760 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009900 tx=0x7f4d30040000 comp rx=0 tx=0).stop 2024-09-21T07:28:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 -- 172.21.15.27:0/2177623840 shutdown_connections 2024-09-21T07:28:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500c6390 0x559d5049b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500e7070 0x559d504a5760 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/2177623840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500be1b0 0x559d50491820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 -- 172.21.15.27:0/2177623840 >> 172.21.15.27:0/2177623840 conn(0x559d500db460 msgr2=0x559d5049aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 -- 172.21.15.27:0/2177623840 shutdown_connections 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 -- 172.21.15.27:0/2177623840 wait complete. 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.074+0000 7f4d44c6b740 1 -- start start 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 1 Processor -- start 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d5049dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d5049e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d50514fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b1720 con 0x559d500c6390 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b2b90 con 0x559d500e7070 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b1df0 con 0x559d500be1b0 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d44c6b740 10 client.0 _ll_register_callbacks cb 0x7f4d280ada80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.075+0000 7f4d37400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d5049dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d5049dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:33506/0 (socket says 172.21.15.27:33506) 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 -- 172.21.15.27:0/3718528657 learned_addr learned my addr 172.21.15.27:0/3718528657 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37e00640 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d50514fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d5049e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 -- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 msgr2=0x559d50514fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d50514fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 -- 172.21.15.27:0/3718528657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 msgr2=0x559d5049e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d5049e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 -- 172.21.15.27:0/3718528657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d50515510 con 0x559d500be1b0 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d37400640 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d5049dba0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d280048e0 tx=0x7f4d28004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25fcd55f5c45f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.076+0000 7f4d24c00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:37.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d24c00640 1 -- 172.21.15.27:0/3718528657 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d280ad420 con 0x559d500be1b0 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d24c00640 1 -- 172.21.15.27:0/3718528657 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f4d280a9f20 con 0x559d500be1b0 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d30003cf0 con 0x559d500be1b0 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d24c00640 1 -- 172.21.15.27:0/3718528657 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d280ae3b0 con 0x559d500be1b0 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 10 client.30091 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559d50516670 con 0x559d500be1b0 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d24c00640 1 -- 172.21.15.27:0/3718528657 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d30003cf0 con 0x559d500be1b0 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d24c00640 1 -- 172.21.15.27:0/3718528657 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d280b5b80 con 0x559d500be1b0 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 10 client.30091 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:37.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 10 client.30091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:37.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 20 client.30091 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:37.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d5007ada0 0x559d50518e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:37.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 4 client.30091 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:28:37.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.077+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d50402970 con 0x559d5007ada0 2024-09-21T07:28:37.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.078+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d5007ada0 0x559d50518e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.078+0000 7f4d2ea00640 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d5007ada0 0x559d50518e30 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22598a25bf93c5df server_cookie=ad36102dbbeab483 in_seq=0 out_seq=0 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.078+0000 7f4d24c00640 10 client.30091 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d24c00640 1 -- 172.21.15.27:0/3718528657 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7386 (crc 0 0 0) 0x559d50402970 con 0x559d5007ada0 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d24c00640 10 client.30091 handle_command_reply: tid=0 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 client.30091 shutdown 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 msgr2=0x559d5049dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d5049dba0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d280048e0 tx=0x7f4d28004930 comp rx=0 tx=0).stop 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 shutdown_connections 2024-09-21T07:28:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x559d5007ada0 0x559d50518e30 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d500e7070 0x559d50514fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559d500c6390 0x559d5049e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d24c00640 0 client.30091 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 --2- 172.21.15.27:0/3718528657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559d500be1b0 0x559d5049dba0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 >> 172.21.15.27:0/3718528657 conn(0x559d500db460 msgr2=0x559d504915f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 shutdown_connections 2024-09-21T07:28:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 1 -- 172.21.15.27:0/3718528657 wait complete. 2024-09-21T07:28:37.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.079+0000 7f4d44c6b740 20 client.30091 trim_cache size 0 max 16384 2024-09-21T07:28:37.091 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:28:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:28:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:28:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "594683c9-bab5-4ebb-ae50-7e80b1ad2ba8": { 2024-09-21T07:28:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "594683c9-bab5-4ebb-ae50-7e80b1ad2ba8", 2024-09-21T07:28:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:28:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:28:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:28:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "23d6d7e9-06b5-4a6e-adfb-86c244736c20": { 2024-09-21T07:28:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:49.824406+0000", 2024-09-21T07:28:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:28:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:28:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "594683c9-bab5-4ebb-ae50-7e80b1ad2ba8": { 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:28:35.097223+0000", 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:28:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:28:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:28:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:28:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:28:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4d1c0b6590 msgr2=0x7f4d1c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4d1c0b6590 0x7f4d1c0b8aa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4d300096c0 tx=0x7f4d30005a40 comp rx=0 tx=0).stop 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38103b80 msgr2=0x7f4d3819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38103b80 0x7f4d3819a490 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009d50 tx=0x7f4d2800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 shutdown_connections 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d381094c0 0x7f4d3819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f4d1c0b6590 0x7f4d1c0b8aa0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d38109e10 0x7f4d3819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 --2- 172.21.15.27:0/2454617624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38103b80 0x7f4d3819a490 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:37.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 >> 172.21.15.27:0/2454617624 conn(0x7f4d380ff5d0 msgr2=0x7f4d38108b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:37.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 shutdown_connections 2024-09-21T07:28:37.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:28:37.080+0000 7f4d3e000640 1 -- 172.21.15.27:0/2454617624 wait complete. 2024-09-21T07:28:37.114 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:36 smithi027 ceph-mon[30779]: pgmap v865: 129 pgs: 129 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:36 smithi153 ceph-mon[34747]: pgmap v865: 129 pgs: 129 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:36 smithi119 ceph-mon[34519]: pgmap v865: 129 pgs: 129 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-21T07:28:37.939 INFO:tasks.cephfs.filesystem:scrub status for tag:594683c9-bab5-4ebb-ae50-7e80b1ad2ba8 - {'path': '/', 'tag': '594683c9-bab5-4ebb-ae50-7e80b1ad2ba8', 'options': 'recursive,force'} 2024-09-21T07:28:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:38 smithi153 ceph-mon[34747]: pgmap v866: 129 pgs: 129 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:28:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:38 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:28:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:38 smithi119 ceph-mon[34519]: pgmap v866: 129 pgs: 129 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:28:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:38 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:28:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:38 smithi027 ceph-mon[30779]: pgmap v866: 129 pgs: 129 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-21T07:28:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:38 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:28:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 -- 172.21.15.27:0/2172040758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 msgr2=0x7ffb00103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 --2- 172.21.15.27:0/2172040758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb00103d20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009900 tx=0x7ffaf002a410 comp rx=0 tx=0).stop 2024-09-21T07:28:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 -- 172.21.15.27:0/2172040758 shutdown_connections 2024-09-21T07:28:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 --2- 172.21.15.27:0/2172040758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb00109b90 0x7ffb001103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 --2- 172.21.15.27:0/2172040758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb00109240 0x7ffb00109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 --2- 172.21.15.27:0/2172040758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb00103d20 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 -- 172.21.15.27:0/2172040758 >> 172.21.15.27:0/2172040758 conn(0x7ffb000ff330 msgr2=0x7ffb00101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 -- 172.21.15.27:0/2172040758 shutdown_connections 2024-09-21T07:28:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.844+0000 7ffaff200640 1 -- 172.21.15.27:0/2172040758 wait complete. 2024-09-21T07:28:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.845+0000 7ffaff200640 1 Processor -- start 2024-09-21T07:28:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffaff200640 1 -- start start 2024-09-21T07:28:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffaff200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb0019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffaff200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb00109240 0x7ffb0019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffaff200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb00109b90 0x7ffb0019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffaff200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00113850 con 0x7ffb00103910 2024-09-21T07:28:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffaff200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001136d0 con 0x7ffb00109240 2024-09-21T07:28:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffaff200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001139d0 con 0x7ffb00109b90 2024-09-21T07:28:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffafde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb0019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffafde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb0019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44228/0 (socket says 172.21.15.27:44228) 2024-09-21T07:28:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffafde00640 1 -- 172.21.15.27:0/2701799761 learned_addr learned my addr 172.21.15.27:0/2701799761 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffafd400640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb00109240 0x7ffb0019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.846+0000 7ffafe800640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb00109b90 0x7ffb0019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffafde00640 1 -- 172.21.15.27:0/2701799761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb00109b90 msgr2=0x7ffb0019fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffafde00640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb00109b90 0x7ffb0019fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffafde00640 1 -- 172.21.15.27:0/2701799761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb00109240 msgr2=0x7ffb0019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffafde00640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb00109240 0x7ffb0019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffafde00640 1 -- 172.21.15.27:0/2701799761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb001a0220 con 0x7ffb00103910 2024-09-21T07:28:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffafd400640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb00109240 0x7ffb0019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffafde00640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb0019a2f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005940 tx=0x7ffaf00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.847+0000 7ffaf6a00640 1 -- 172.21.15.27:0/2701799761 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf003f030 con 0x7ffb00103910 2024-09-21T07:28:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.848+0000 7ffaf6a00640 1 -- 172.21.15.27:0/2701799761 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaf00044c0 con 0x7ffb00103910 2024-09-21T07:28:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.848+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001a04b0 con 0x7ffb00103910 2024-09-21T07:28:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.848+0000 7ffaf6a00640 1 -- 172.21.15.27:0/2701799761 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0035070 con 0x7ffb00103910 2024-09-21T07:28:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.848+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001a6ca0 con 0x7ffb00103910 2024-09-21T07:28:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 Processor -- start 2024-09-21T07:28:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 -- start start 2024-09-21T07:28:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x564734663e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734664350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x564734664890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5647345d96b0 con 0x5647342bf1b0 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5647345c8ae0 con 0x5647342962f0 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffb0be39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647345cc8f0 con 0x56473429e4d0 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.851+0000 7ffafe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x564734663e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x564734664890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x564734663e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44230/0 (socket says 172.21.15.27:44230) 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 -- 172.21.15.27:0/33741506 learned_addr learned my addr 172.21.15.27:0/33741506 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffaf6a00640 1 -- 172.21.15.27:0/2701799761 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ffaf0042030 con 0x7ffb00103910 2024-09-21T07:28:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafde00640 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734664350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 -- 172.21.15.27:0/33741506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 msgr2=0x564734664890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x564734664890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 -- 172.21.15.27:0/33741506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 msgr2=0x564734664350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734664350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 -- 172.21.15.27:0/33741506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564734595780 con 0x5647342bf1b0 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafd400640 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x564734664890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.852+0000 7ffafe800640 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x564734663e10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009fd0 tx=0x7ffaf80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad13e58bb6397b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.853+0000 7ffaeb000640 1 -- 172.21.15.27:0/33741506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8004d20 con 0x5647342bf1b0 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.853+0000 7ffaeb000640 1 -- 172.21.15.27:0/33741506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaf8004050 con 0x5647342bf1b0 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.853+0000 7ffaeb000640 1 -- 172.21.15.27:0/33741506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8005220 con 0x5647342bf1b0 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffaf6a00640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffae40b6680 0x7ffae40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 -- 172.21.15.27:0/33741506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 msgr2=0x564734663e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x564734663e10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009fd0 tx=0x7ffaf80118d0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 -- 172.21.15.27:0/33741506 shutdown_connections 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x564734664890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734664350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 --2- 172.21.15.27:0/33741506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x564734663e10 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 -- 172.21.15.27:0/33741506 >> 172.21.15.27:0/33741506 conn(0x5647342b3300 msgr2=0x564734661d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffaf6a00640 1 -- 172.21.15.27:0/2701799761 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7ffaf0107e90 con 0x7ffb00103910 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffafde00640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffae40b6680 0x7ffae40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 -- 172.21.15.27:0/33741506 shutdown_connections 2024-09-21T07:28:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffb0be39740 1 -- 172.21.15.27:0/33741506 wait complete. 2024-09-21T07:28:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.854+0000 7ffafde00640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffae40b6680 0x7ffae40b8b90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x564734662a90 tx=0x7ffaf002a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 -- start start 2024-09-21T07:28:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 Processor -- start 2024-09-21T07:28:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x56473466f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x56473466f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x56473466fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564734500ab0 con 0x5647342bf1b0 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56473456f570 con 0x5647342962f0 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564734672590 con 0x56473429e4d0 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.855+0000 7ffb0be39740 10 client.0 _ll_register_callbacks cb 0x56473457e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x56473466f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x56473466f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51094/0 (socket says 172.21.15.27:51094) 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafe800640 1 -- 172.21.15.27:0/2053901519 learned_addr learned my addr 172.21.15.27:0/2053901519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafd400640 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x56473466f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafe800640 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x56473466fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafd400640 1 -- 172.21.15.27:0/2053901519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 msgr2=0x56473466f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafd400640 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x56473466f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafd400640 1 -- 172.21.15.27:0/2053901519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 msgr2=0x56473466fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafd400640 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x56473466fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafd400640 1 -- 172.21.15.27:0/2053901519 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56473432a790 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffafd400640 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x56473466f930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffb0006c400 tx=0x7ffaec00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa5af52ae9ceb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.856+0000 7ffadf400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.857+0000 7ffadf400640 1 -- 172.21.15.27:0/2053901519 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0120d0 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.857+0000 7ffadf400640 1 -- 172.21.15.27:0/2053901519 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7ffaec004030 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.857+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaf8004050 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.857+0000 7ffadf400640 1 -- 172.21.15.27:0/2053901519 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffaec025050 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.857+0000 7ffb0be39740 10 client.30103 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.857+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647343a3620 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.857+0000 7ffadf400640 1 -- 172.21.15.27:0/2053901519 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaf8004050 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffadf400640 1 -- 172.21.15.27:0/2053901519 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffaec010430 con 0x56473429e4d0 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffb0be39740 10 client.30103 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffb0be39740 10 client.30103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffb0be39740 20 client.30103 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56473427dd80 0x564734681610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffb0be39740 4 client.30103 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647345db600 con 0x56473427dd80 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.858+0000 7ffafe800640 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56473427dd80 0x564734681610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.859+0000 7ffafe800640 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56473427dd80 0x564734681610 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5231ef2f500a5f2 server_cookie=bf8954caa419b8b in_seq=0 out_seq=0 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.859+0000 7ffadf400640 10 client.30103 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.859+0000 7ffadf400640 1 -- 172.21.15.27:0/2053901519 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5647345db600 con 0x56473427dd80 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.859+0000 7ffadf400640 10 client.30103 handle_command_reply: tid=0 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 client.30103 shutdown 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 msgr2=0x56473466f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x56473466f930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffb0006c400 tx=0x7ffaec00f790 comp rx=0 tx=0).stop 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 shutdown_connections 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56473427dd80 0x564734681610 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56473429e4d0 0x56473466f930 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffadf400640 0 client.30103 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342bf1b0 0x56473466fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2053901519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x56473466f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 >> 172.21.15.27:0/2053901519 conn(0x5647342b3300 msgr2=0x564734664e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 shutdown_connections 2024-09-21T07:28:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 1 -- 172.21.15.27:0/2053901519 wait complete. 2024-09-21T07:28:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.860+0000 7ffb0be39740 20 client.30103 trim_cache size 0 max 16384 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 Processor -- start 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 -- start start 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734665550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56473429e4d0 0x56473466c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x56473466cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001b35d0 con 0x56473429e4d0 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001b0ea0 con 0x5647342962f0 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffb0be39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001aed10 con 0x5647342bf1b0 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffafe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734665550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.876+0000 7ffafd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56473429e4d0 0x56473466c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734665550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:51096/0 (socket says 172.21.15.27:51096) 2024-09-21T07:28:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafe800640 1 -- 172.21.15.27:0/3241954092 learned_addr learned my addr 172.21.15.27:0/3241954092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafe800640 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x56473466cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafd400640 1 -- 172.21.15.27:0/3241954092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 msgr2=0x56473466cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafd400640 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x56473466cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafd400640 1 -- 172.21.15.27:0/3241954092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 msgr2=0x564734665550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafd400640 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734665550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafd400640 1 -- 172.21.15.27:0/3241954092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56473467b7b0 con 0x56473429e4d0 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafe800640 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x56473466cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:28:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffafd400640 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56473429e4d0 0x56473466c5c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ffaec010880 tx=0x7ffaec012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa62886dac57b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffaea600640 1 -- 172.21.15.27:0/3241954092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0104f0 con 0x56473429e4d0 2024-09-21T07:28:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffaea600640 1 -- 172.21.15.27:0/3241954092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaec0120d0 con 0x56473429e4d0 2024-09-21T07:28:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.877+0000 7ffaea600640 1 -- 172.21.15.27:0/3241954092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0056b0 con 0x56473429e4d0 2024-09-21T07:28:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 -- 172.21.15.27:0/3241954092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56473429e4d0 msgr2=0x56473466c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56473429e4d0 0x56473466c5c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ffaec010880 tx=0x7ffaec012910 comp rx=0 tx=0).stop 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 -- 172.21.15.27:0/3241954092 shutdown_connections 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x56473466cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56473429e4d0 0x56473466c5c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 --2- 172.21.15.27:0/3241954092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647342962f0 0x564734665550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 -- 172.21.15.27:0/3241954092 >> 172.21.15.27:0/3241954092 conn(0x5647342b3300 msgr2=0x564734664e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 -- 172.21.15.27:0/3241954092 shutdown_connections 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.878+0000 7ffb0be39740 1 -- 172.21.15.27:0/3241954092 wait complete. 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 -- start start 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 Processor -- start 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342962f0 0x564734677fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56473429e4d0 0x564734675490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x5647346759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001af360 con 0x5647342962f0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001b07d0 con 0x56473429e4d0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001afa30 con 0x5647342bf1b0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.879+0000 7ffb0be39740 10 client.0 _ll_register_callbacks cb 0x7ffaec009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342962f0 0x564734677fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56473429e4d0 0x564734675490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342962f0 0x564734677fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44252/0 (socket says 172.21.15.27:44252) 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafe800640 1 -- 172.21.15.27:0/2062336850 learned_addr learned my addr 172.21.15.27:0/2062336850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafe800640 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x5647346759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafd400640 1 -- 172.21.15.27:0/2062336850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 msgr2=0x5647346759d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafd400640 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x5647346759d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafd400640 1 -- 172.21.15.27:0/2062336850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342962f0 msgr2=0x564734677fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafd400640 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342962f0 0x564734677fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafd400640 1 -- 172.21.15.27:0/2062336850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564734676140 con 0x56473429e4d0 2024-09-21T07:28:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.880+0000 7ffafe800640 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342962f0 0x564734677fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffafd400640 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56473429e4d0 0x564734675490 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ffaec010920 tx=0x7ffaec005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecda1bc613461e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffadf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:28:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffadf400640 1 -- 172.21.15.27:0/2062336850 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec012280 con 0x56473429e4d0 2024-09-21T07:28:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaec0120d0 con 0x56473429e4d0 2024-09-21T07:28:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffadf400640 1 -- 172.21.15.27:0/2062336850 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7ffaec0035f0 con 0x56473429e4d0 2024-09-21T07:28:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffadf400640 1 -- 172.21.15.27:0/2062336850 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffaec03c050 con 0x56473429e4d0 2024-09-21T07:28:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffb0be39740 10 client.30146 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.881+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647346ebf30 con 0x56473429e4d0 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffadf400640 1 -- 172.21.15.27:0/2062336850 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaec0120d0 con 0x56473429e4d0 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffadf400640 1 -- 172.21.15.27:0/2062336850 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffaec028070 con 0x56473429e4d0 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffb0be39740 10 client.30146 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffb0be39740 10 client.30146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffb0be39740 20 client.30146 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647342531c0 0x5647346ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffb0be39740 4 client.30146 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647345d70b0 con 0x5647342531c0 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.882+0000 7ffafe800640 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647342531c0 0x5647346ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.883+0000 7ffafe800640 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647342531c0 0x5647346ee5d0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8632c52a4593d156 server_cookie=951391b91b3c2eef in_seq=0 out_seq=0 2024-09-21T07:28:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.883+0000 7ffadf400640 10 client.30146 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.896+0000 7ffadf400640 1 -- 172.21.15.27:0/2062336850 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647345d70b0 con 0x5647342531c0 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.896+0000 7ffadf400640 10 client.30146 handle_command_reply: tid=0 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.896+0000 7ffb0be39740 1 client.30146 shutdown 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56473429e4d0 msgr2=0x564734675490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56473429e4d0 0x564734675490 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ffaec010920 tx=0x7ffaec005880 comp rx=0 tx=0).stop 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 shutdown_connections 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5647342531c0 0x5647346ee5d0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647342bf1b0 0x5647346759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56473429e4d0 0x564734675490 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 --2- 172.21.15.27:0/2062336850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5647342962f0 0x564734677fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffadf400640 0 client.30146 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 >> 172.21.15.27:0/2062336850 conn(0x5647342b3300 msgr2=0x564734660360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 shutdown_connections 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 1 -- 172.21.15.27:0/2062336850 wait complete. 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.897+0000 7ffb0be39740 20 client.30146 trim_cache size 0 max 16384 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:28:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffae40b6680 msgr2=0x7ffae40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffae40b6680 0x7ffae40b8b90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x564734662a90 tx=0x7ffaf002a410 comp rx=0 tx=0).stop 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 msgr2=0x7ffb0019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb0019a2f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005940 tx=0x7ffaf00040c0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 shutdown_connections 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb00109b90 0x7ffb0019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ffae40b6680 0x7ffae40b8b90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb00109240 0x7ffb0019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 --2- 172.21.15.27:0/2701799761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb00103910 0x7ffb0019a2f0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 >> 172.21.15.27:0/2701799761 conn(0x7ffb000ff330 msgr2=0x7ffb00101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 shutdown_connections 2024-09-21T07:28:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:39.898+0000 7ffaff200640 1 -- 172.21.15.27:0/2701799761 wait complete. 2024-09-21T07:28:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:40 smithi153 ceph-mon[34747]: pgmap v867: 129 pgs: 129 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 16 op/s 2024-09-21T07:28:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:40 smithi119 ceph-mon[34519]: pgmap v867: 129 pgs: 129 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 16 op/s 2024-09-21T07:28:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:40 smithi027 ceph-mon[30779]: pgmap v867: 129 pgs: 129 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 16 op/s 2024-09-21T07:28:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:42 smithi153 ceph-mon[34747]: pgmap v868: 129 pgs: 129 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:28:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:42 smithi119 ceph-mon[34519]: pgmap v868: 129 pgs: 129 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:28:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:42 smithi027 ceph-mon[30779]: pgmap v868: 129 pgs: 129 active+clean; 4.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-21T07:28:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:44 smithi153 ceph-mon[34747]: pgmap v869: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-09-21T07:28:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:44 smithi119 ceph-mon[34519]: pgmap v869: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-09-21T07:28:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:44 smithi027 ceph-mon[30779]: pgmap v869: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-09-21T07:28:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:46 smithi153 ceph-mon[34747]: pgmap v870: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:28:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:46 smithi119 ceph-mon[34519]: pgmap v870: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:28:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:46 smithi027 ceph-mon[30779]: pgmap v870: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-21T07:28:49.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:48 smithi153 ceph-mon[34747]: pgmap v871: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:28:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:48 smithi119 ceph-mon[34519]: pgmap v871: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:28:49.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:48 smithi027 ceph-mon[30779]: pgmap v871: 129 pgs: 129 active+clean; 4.5 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-21T07:28:51.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:50 smithi153 ceph-mon[34747]: pgmap v872: 129 pgs: 129 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-09-21T07:28:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:50 smithi119 ceph-mon[34519]: pgmap v872: 129 pgs: 129 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-09-21T07:28:51.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:50 smithi027 ceph-mon[30779]: pgmap v872: 129 pgs: 129 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-09-21T07:28:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 -- 172.21.15.27:0/3616867299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 msgr2=0x7f14fc10f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 --2- 172.21.15.27:0/3616867299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc10f9b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009900 tx=0x7f14e802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 -- 172.21.15.27:0/3616867299 shutdown_connections 2024-09-21T07:28:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 --2- 172.21.15.27:0/3616867299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14fc10fef0 0x7f14fc112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 --2- 172.21.15.27:0/3616867299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc10f9b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 --2- 172.21.15.27:0/3616867299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14fc075730 0x7f14fc075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 -- 172.21.15.27:0/3616867299 >> 172.21.15.27:0/3616867299 conn(0x7f14fc0ff320 msgr2=0x7f14fc101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 -- 172.21.15.27:0/3616867299 shutdown_connections 2024-09-21T07:28:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.101+0000 7f14fb200640 1 -- 172.21.15.27:0/3616867299 wait complete. 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.102+0000 7f14fb200640 1 Processor -- start 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fb200640 1 -- start start 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14fc075730 0x7f14fc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14fc10fef0 0x7f14fc1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc115a30 con 0x7f14fc076080 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1158b0 con 0x7f14fc10fef0 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc115bb0 con 0x7f14fc075730 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53606/0 (socket says 172.21.15.27:53606) 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14f9400640 1 -- 172.21.15.27:0/614961882 learned_addr learned my addr 172.21.15.27:0/614961882 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14f9e00640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14fc075730 0x7f14fc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.103+0000 7f14fa800640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14fc10fef0 0x7f14fc1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14f9400640 1 -- 172.21.15.27:0/614961882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14fc075730 msgr2=0x7f14fc19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14f9400640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14fc075730 0x7f14fc19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14f9400640 1 -- 172.21.15.27:0/614961882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14fc10fef0 msgr2=0x7f14fc1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14f9400640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14fc10fef0 0x7f14fc1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14f9400640 1 -- 172.21.15.27:0/614961882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc1a2380 con 0x7f14fc076080 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14f9e00640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14fc075730 0x7f14fc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14fa800640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14fc10fef0 0x7f14fc1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.104+0000 7f14f9400640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc19c9f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f14e8032fd0 tx=0x7f14e8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.105+0000 7f14f2a00640 1 -- 172.21.15.27:0/614961882 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e803f030 con 0x7f14fc076080 2024-09-21T07:28:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.105+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc1a2610 con 0x7f14fc076080 2024-09-21T07:28:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.105+0000 7f14f2a00640 1 -- 172.21.15.27:0/614961882 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e8004490 con 0x7f14fc076080 2024-09-21T07:28:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.105+0000 7f14f2a00640 1 -- 172.21.15.27:0/614961882 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8035070 con 0x7f14fc076080 2024-09-21T07:28:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.105+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a8e00 con 0x7f14fc076080 2024-09-21T07:28:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 Processor -- start 2024-09-21T07:28:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 -- start start 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cd31b0 0x55a395077e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a395078350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cb24d0 0x55a395078890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f14f2a00640 1 -- 172.21.15.27:0/614961882 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f14e8042030 con 0x7f14fc076080 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a394fed6b0 con 0x55a394cb24d0 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a394fdcae0 con 0x55a394caa2f0 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.108+0000 7f1507dcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a394fe08f0 con 0x55a394cd31b0 2024-09-21T07:28:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cd31b0 0x55a395077e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a395078350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cd31b0 0x55a395077e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54190/0 (socket says 172.21.15.27:54190) 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 -- 172.21.15.27:0/2818781923 learned_addr learned my addr 172.21.15.27:0/2818781923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14f9400640 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cb24d0 0x55a395078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 -- 172.21.15.27:0/2818781923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 msgr2=0x55a395078350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a395078350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 -- 172.21.15.27:0/2818781923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cb24d0 msgr2=0x55a395078890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cb24d0 0x55a395078890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14fa800640 1 -- 172.21.15.27:0/2818781923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a394fa9780 con 0x55a394cd31b0 2024-09-21T07:28:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.109+0000 7f14f9e00640 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a395078350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.110+0000 7f14fa800640 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cd31b0 0x55a395077e10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14f4011810 tx=0x7f14f4011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5ae3fd28f3c24a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.110+0000 7f14e7000640 1 -- 172.21.15.27:0/2818781923 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4011310 con 0x55a394cd31b0 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.110+0000 7f14e7000640 1 -- 172.21.15.27:0/2818781923 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f4003e20 con 0x55a394cd31b0 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.110+0000 7f14e7000640 1 -- 172.21.15.27:0/2818781923 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40044f0 con 0x55a394cd31b0 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f14f2a00640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14e00b6680 0x7f14e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f14f9e00640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14e00b6680 0x7f14e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f14f2a00640 1 -- 172.21.15.27:0/614961882 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f14e8107e40 con 0x7f14fc076080 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 -- 172.21.15.27:0/2818781923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cd31b0 msgr2=0x55a395077e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cd31b0 0x55a395077e10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14f4011810 tx=0x7f14f4011cd0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 -- 172.21.15.27:0/2818781923 shutdown_connections 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cb24d0 0x55a395078890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a395078350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 --2- 172.21.15.27:0/2818781923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cd31b0 0x55a395077e10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 -- 172.21.15.27:0/2818781923 >> 172.21.15.27:0/2818781923 conn(0x55a394cc7300 msgr2=0x55a395075d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.111+0000 7f1507dcf740 1 -- 172.21.15.27:0/2818781923 shutdown_connections 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.112+0000 7f1507dcf740 1 -- 172.21.15.27:0/2818781923 wait complete. 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.112+0000 7f14f9e00640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14e00b6680 0x7f14e00b8b90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009fd0 tx=0x7f14ec005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.112+0000 7f1507dcf740 1 -- start start 2024-09-21T07:28:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.112+0000 7f1507dcf740 1 Processor -- start 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a395083390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cd31b0 0x55a395083e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f1507dcf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a394f14ab0 con 0x55a394caa2f0 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f1507dcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a394f83570 con 0x55a394cd31b0 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f1507dcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a395086590 con 0x55a394cb24d0 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f1507dcf740 10 client.0 _ll_register_callbacks cb 0x55a394f92100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a395083390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a395083390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53628/0 (socket says 172.21.15.27:53628) 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f14f9e00640 1 -- 172.21.15.27:0/1320143448 learned_addr learned my addr 172.21.15.27:0/1320143448 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f14f9400640 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.113+0000 7f14fa800640 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cd31b0 0x55a395083e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14f9e00640 1 -- 172.21.15.27:0/1320143448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 msgr2=0x55a3950838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14f9e00640 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14f9e00640 1 -- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cd31b0 msgr2=0x55a395083e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14f9e00640 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cd31b0 0x55a395083e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14f9e00640 1 -- 172.21.15.27:0/1320143448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a394d3e790 con 0x55a394caa2f0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14f9e00640 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a395083390 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00fac0 tx=0x7f14ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda6e525d984126c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14db400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14db400640 1 -- 172.21.15.27:0/1320143448 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec012400 con 0x55a394caa2f0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f14db400640 1 -- 172.21.15.27:0/1320143448 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f14ec004240 con 0x55a394caa2f0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.114+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f4003e20 con 0x55a394caa2f0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f14db400640 1 -- 172.21.15.27:0/1320143448 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14ec004440 con 0x55a394caa2f0 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 10 client.20145 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a394db7620 con 0x55a394caa2f0 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f14db400640 1 -- 172.21.15.27:0/1320143448 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f4003e20 con 0x55a394caa2f0 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f14db400640 1 -- 172.21.15.27:0/1320143448 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f14ec027020 con 0x55a394caa2f0 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 10 client.20145 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 10 client.20145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 20 client.20145 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394c91d80 0x55a395095590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 4 client.20145 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.115+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a394fef600 con 0x55a394c91d80 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.116+0000 7f14f9400640 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394c91d80 0x55a395095590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.116+0000 7f14f9400640 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394c91d80 0x55a395095590 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9a480861788f58 server_cookie=b79e8dd7366d1055 in_seq=0 out_seq=0 2024-09-21T07:28:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.116+0000 7f14db400640 10 client.20145 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f14db400640 1 -- 172.21.15.27:0/1320143448 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a394fef600 con 0x55a394c91d80 2024-09-21T07:28:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f14db400640 10 client.20145 handle_command_reply: tid=0 2024-09-21T07:28:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 client.20145 shutdown 2024-09-21T07:28:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 msgr2=0x55a395083390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a395083390 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00fac0 tx=0x7f14ec004730 comp rx=0 tx=0).stop 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 shutdown_connections 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394c91d80 0x55a395095590 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f14db400640 0 client.20145 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cd31b0 0x55a395083e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1320143448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a395083390 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 >> 172.21.15.27:0/1320143448 conn(0x55a394cc7300 msgr2=0x55a395078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 shutdown_connections 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 1 -- 172.21.15.27:0/1320143448 wait complete. 2024-09-21T07:28:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.117+0000 7f1507dcf740 20 client.20145 trim_cache size 0 max 16384 2024-09-21T07:28:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 Processor -- start 2024-09-21T07:28:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 -- start start 2024-09-21T07:28:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cc7300 0x55a395084fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a39509ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a39509b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b5750 con 0x55a394cc7300 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b3020 con 0x55a394caa2f0 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f1507dcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b0e90 con 0x55a394cb24d0 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cc7300 0x55a395084fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cc7300 0x55a395084fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53638/0 (socket says 172.21.15.27:53638) 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.133+0000 7f14f9e00640 1 -- 172.21.15.27:0/1843596348 learned_addr learned my addr 172.21.15.27:0/1843596348 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14f9400640 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a39509ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14fa800640 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a39509b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14f9e00640 1 -- 172.21.15.27:0/1843596348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 msgr2=0x55a39509b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14f9e00640 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a39509b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14f9e00640 1 -- 172.21.15.27:0/1843596348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 msgr2=0x55a39509ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14f9e00640 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a39509ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14f9e00640 1 -- 172.21.15.27:0/1843596348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39507a910 con 0x55a394cc7300 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14f9e00640 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cc7300 0x55a395084fc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f14ec003d80 tx=0x7f14ec003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b90242dda566ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14e6600640 1 -- 172.21.15.27:0/1843596348 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0035f0 con 0x55a394cc7300 2024-09-21T07:28:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14e6600640 1 -- 172.21.15.27:0/1843596348 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ec011810 con 0x55a394cc7300 2024-09-21T07:28:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.134+0000 7f14e6600640 1 -- 172.21.15.27:0/1843596348 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0296a0 con 0x55a394cc7300 2024-09-21T07:28:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 -- 172.21.15.27:0/1843596348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cc7300 msgr2=0x55a395084fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cc7300 0x55a395084fc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f14ec003d80 tx=0x7f14ec003fb0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 -- 172.21.15.27:0/1843596348 shutdown_connections 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a39509b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394caa2f0 0x55a39509ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 --2- 172.21.15.27:0/1843596348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394cc7300 0x55a395084fc0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 -- 172.21.15.27:0/1843596348 >> 172.21.15.27:0/1843596348 conn(0x55a39508fca0 msgr2=0x55a3950848b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 -- 172.21.15.27:0/1843596348 shutdown_connections 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.135+0000 7f1507dcf740 1 -- 172.21.15.27:0/1843596348 wait complete. 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 -- start start 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 Processor -- start 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a39508bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cc7300 0x55a395089800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b14e0 con 0x55a394caa2f0 2024-09-21T07:28:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b2950 con 0x55a394cc7300 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b1bb0 con 0x55a394cb24d0 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.136+0000 7f1507dcf740 10 client.0 _ll_register_callbacks cb 0x7f14ec012170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a39508bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a39508bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53646/0 (socket says 172.21.15.27:53646) 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 -- 172.21.15.27:0/3806460472 learned_addr learned my addr 172.21.15.27:0/3806460472 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14fa800640 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cc7300 0x55a395089800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9400640 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 -- 172.21.15.27:0/3806460472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 msgr2=0x55a3950892c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950892c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 -- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cc7300 msgr2=0x55a395089800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cc7300 0x55a395089800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 -- 172.21.15.27:0/3806460472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a395089d40 con 0x55a394caa2f0 2024-09-21T07:28:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14f9e00640 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a39508bdb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0038d0 tx=0x7f14ec003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a2362fb8a206f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.137+0000 7f14db400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.138+0000 7f14db400640 1 -- 172.21.15.27:0/3806460472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec003b20 con 0x55a394caa2f0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.138+0000 7f14db400640 1 -- 172.21.15.27:0/3806460472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9461+0+0 (secure 0 0 0) 0x7f14ec005260 con 0x55a394caa2f0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.138+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec011810 con 0x55a394caa2f0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.138+0000 7f14db400640 1 -- 172.21.15.27:0/3806460472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14ec04a050 con 0x55a394caa2f0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.138+0000 7f1507dcf740 10 client.20151 fetch_fsmap learned FSMap version 27 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.138+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3950ffcb0 con 0x55a394caa2f0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.138+0000 7f14db400640 1 -- 172.21.15.27:0/3806460472 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ec011810 con 0x55a394caa2f0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f14db400640 1 -- 172.21.15.27:0/3806460472 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f14ec027040 con 0x55a394caa2f0 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f1507dcf740 10 client.20151 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f1507dcf740 10 client.20151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f1507dcf740 20 client.20151 populate_metadata read hostname 'smithi027' 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f1507dcf740 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394cd31b0 0x55a3951023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f1507dcf740 4 client.20151 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:28:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a394feb0b0 con 0x55a394cd31b0 2024-09-21T07:28:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.139+0000 7f14f9400640 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394cd31b0 0x55a3951023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.140+0000 7f14f9400640 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394cd31b0 0x55a3951023b0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7443055d67202507 server_cookie=ef5ca01116b2bf93 in_seq=0 out_seq=0 2024-09-21T07:28:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.140+0000 7f14db400640 10 client.20151 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.154+0000 7f14db400640 1 -- 172.21.15.27:0/3806460472 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a394feb0b0 con 0x55a394cd31b0 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.154+0000 7f14db400640 10 client.20151 handle_command_reply: tid=0 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.154+0000 7f1507dcf740 1 client.20151 shutdown 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.154+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 msgr2=0x55a39508bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.154+0000 7f1507dcf740 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a39508bdb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0038d0 tx=0x7f14ec003900 comp rx=0 tx=0).stop 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.154+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 shutdown_connections 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.154+0000 7f1507dcf740 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55a394cd31b0 0x55a3951023b0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f1507dcf740 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a394cb24d0 0x55a3950892c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f1507dcf740 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a394cc7300 0x55a395089800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f1507dcf740 1 --2- 172.21.15.27:0/3806460472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a394caa2f0 0x55a39508bdb0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f14db400640 0 client.20151 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 >> 172.21.15.27:0/3806460472 conn(0x55a39508fca0 msgr2=0x55a39508e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 shutdown_connections 2024-09-21T07:28:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f1507dcf740 1 -- 172.21.15.27:0/3806460472 wait complete. 2024-09-21T07:28:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.155+0000 7f1507dcf740 20 client.20151 trim_cache size 0 max 16384 2024-09-21T07:28:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:28:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:28:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:28:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14e00b6680 msgr2=0x7f14e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14e00b6680 0x7f14e00b8b90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009fd0 tx=0x7f14ec005f90 comp rx=0 tx=0).stop 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 msgr2=0x7f14fc19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc19c9f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f14e8032fd0 tx=0x7f14e8031a20 comp rx=0 tx=0).stop 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 shutdown_connections 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f14e00b6680 0x7f14e00b8b90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14fc10fef0 0x7f14fc1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14fc076080 0x7f14fc19c9f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 --2- 172.21.15.27:0/614961882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14fc075730 0x7f14fc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 >> 172.21.15.27:0/614961882 conn(0x7f14fc0ff320 msgr2=0x7f14fc109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 shutdown_connections 2024-09-21T07:28:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:28:53.156+0000 7f14fb200640 1 -- 172.21.15.27:0/614961882 wait complete. 2024-09-21T07:28:53.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:52 smithi153 ceph-mon[34747]: pgmap v873: 129 pgs: 129 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:28:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:52 smithi119 ceph-mon[34519]: pgmap v873: 129 pgs: 129 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:28:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:52 smithi027 ceph-mon[30779]: pgmap v873: 129 pgs: 129 active+clean; 4.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-21T07:28:55.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:54 smithi153 ceph-mon[34747]: pgmap v874: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-09-21T07:28:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:54 smithi119 ceph-mon[34519]: pgmap v874: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-09-21T07:28:55.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:54 smithi027 ceph-mon[30779]: pgmap v874: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-09-21T07:28:57.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:56 smithi153 ceph-mon[34747]: pgmap v875: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:28:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:56 smithi119 ceph-mon[34519]: pgmap v875: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:28:57.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:56 smithi027 ceph-mon[30779]: pgmap v875: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-21T07:28:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:28:58 smithi119 ceph-mon[34519]: pgmap v876: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-21T07:28:59.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:28:58 smithi027 ceph-mon[30779]: pgmap v876: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-21T07:28:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:28:58 smithi153 ceph-mon[34747]: pgmap v876: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-09-21T07:29:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:00 smithi119 ceph-mon[34519]: pgmap v877: 129 pgs: 129 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-09-21T07:29:01.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:00 smithi027 ceph-mon[30779]: pgmap v877: 129 pgs: 129 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-09-21T07:29:01.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:00 smithi153 ceph-mon[34747]: pgmap v877: 129 pgs: 129 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-09-21T07:29:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:02 smithi119 ceph-mon[34519]: pgmap v878: 129 pgs: 129 active+clean; 5.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-21T07:29:03.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:02 smithi027 ceph-mon[30779]: pgmap v878: 129 pgs: 129 active+clean; 5.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-21T07:29:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:02 smithi153 ceph-mon[34747]: pgmap v878: 129 pgs: 129 active+clean; 5.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-21T07:29:05.244 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.2 2024-09-21T07:29:05.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:29:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:04 smithi119 ceph-mon[34519]: pgmap v879: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 107 KiB/s wr, 32 op/s 2024-09-21T07:29:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:04 smithi119 ceph-mon[34519]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T07:29:05.300 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:04 smithi027 ceph-mon[30779]: pgmap v879: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 107 KiB/s wr, 32 op/s 2024-09-21T07:29:05.300 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:04 smithi027 ceph-mon[30779]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T07:29:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:04 smithi153 ceph-mon[34747]: pgmap v879: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 107 KiB/s wr, 32 op/s 2024-09-21T07:29:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:04 smithi153 ceph-mon[34747]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T07:29:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 -- 172.21.15.27:0/2611202293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97181047c0 msgr2=0x7f9718104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 --2- 172.21.15.27:0/2611202293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97181047c0 0x7f9718104bd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9708009900 tx=0x7f970802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 -- 172.21.15.27:0/2611202293 shutdown_connections 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 --2- 172.21.15.27:0/2611202293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9718105110 0x7f971810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 --2- 172.21.15.27:0/2611202293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f971810dc20 0x7f9718110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 --2- 172.21.15.27:0/2611202293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97181047c0 0x7f9718104bd0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 -- 172.21.15.27:0/2611202293 >> 172.21.15.27:0/2611202293 conn(0x7f97180fe2c0 msgr2=0x7f9718100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 -- 172.21.15.27:0/2611202293 shutdown_connections 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.234+0000 7f9717200640 1 -- 172.21.15.27:0/2611202293 wait complete. 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.235+0000 7f9717200640 1 Processor -- start 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.235+0000 7f9717200640 1 -- start start 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9717200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97181047c0 0x7f971819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9717200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9718105110 0x7f971819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9717200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971810dc20 0x7f971819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9717200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97181138d0 con 0x7f9718105110 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9717200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113750 con 0x7f97181047c0 2024-09-21T07:29:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9717200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113a50 con 0x7f971810dc20 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9715400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9718105110 0x7f971819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9715400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9718105110 0x7f971819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42772/0 (socket says 172.21.15.27:42772) 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9715400640 1 -- 172.21.15.27:0/2878057684 learned_addr learned my addr 172.21.15.27:0/2878057684 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9715e00640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97181047c0 0x7f971819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9716800640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971810dc20 0x7f971819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9715400640 1 -- 172.21.15.27:0/2878057684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971810dc20 msgr2=0x7f971819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.236+0000 7f9715400640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971810dc20 0x7f971819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f9715400640 1 -- 172.21.15.27:0/2878057684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97181047c0 msgr2=0x7f971819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f9715400640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97181047c0 0x7f971819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f9715400640 1 -- 172.21.15.27:0/2878057684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97181a02b0 con 0x7f9718105110 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f9715400640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9718105110 0x7f971819a8e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f970400e9f0 tx=0x7f970400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f970ea00640 1 -- 172.21.15.27:0/2878057684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970400ccc0 con 0x7f9718105110 2024-09-21T07:29:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f970ea00640 1 -- 172.21.15.27:0/2878057684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97040108e0 con 0x7f9718105110 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f970ea00640 1 -- 172.21.15.27:0/2878057684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704018860 con 0x7f9718105110 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97181a05a0 con 0x7f9718105110 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.237+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a6d00 con 0x7f9718105110 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.239+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9718104bd0 con 0x7f9718105110 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.242+0000 7f970ea00640 1 -- 172.21.15.27:0/2878057684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9704010430 con 0x7f9718105110 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.243+0000 7f970ea00640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96fc0b6680 0x7f96fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.243+0000 7f970ea00640 1 -- 172.21.15.27:0/2878057684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7f97040e7e00 con 0x7f9718105110 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.244+0000 7f9715e00640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96fc0b6680 0x7f96fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.245+0000 7f9715e00640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96fc0b6680 0x7f96fc0b8b90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9708005ab0 tx=0x7f9708005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.248+0000 7f970ea00640 1 -- 172.21.15.27:0/2878057684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f97040b22d0 con 0x7f9718105110 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 -- 172.21.15.27:0/702714906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160103910 msgr2=0x7fc160103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 --2- 172.21.15.27:0/702714906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160103910 0x7fc160103d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc150009900 tx=0x7fc15002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 -- 172.21.15.27:0/702714906 shutdown_connections 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 --2- 172.21.15.27:0/702714906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc160109240 0x7fc160109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 --2- 172.21.15.27:0/702714906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160109b90 0x7fc1601103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 --2- 172.21.15.27:0/702714906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160103910 0x7fc160103d20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 -- 172.21.15.27:0/702714906 >> 172.21.15.27:0/702714906 conn(0x7fc1600ff330 msgr2=0x7fc160101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 -- 172.21.15.27:0/702714906 shutdown_connections 2024-09-21T07:29:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.358+0000 7fc15f200640 1 -- 172.21.15.27:0/702714906 wait complete. 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.359+0000 7fc15f200640 1 Processor -- start 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.359+0000 7fc15f200640 1 -- start start 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.359+0000 7fc15f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160103910 0x7fc16019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160109240 0x7fc16019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc160109b90 0x7fc1601a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc160113850 con 0x7fc160103910 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601136d0 con 0x7fc160109240 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601139d0 con 0x7fc160109b90 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160103910 0x7fc16019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160103910 0x7fc16019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42800/0 (socket says 172.21.15.27:42800) 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15de00640 1 -- 172.21.15.27:0/1342617390 learned_addr learned my addr 172.21.15.27:0/1342617390 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15e800640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc160109b90 0x7fc1601a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15d400640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160109240 0x7fc16019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15de00640 1 -- 172.21.15.27:0/1342617390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc160109b90 msgr2=0x7fc1601a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15de00640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc160109b90 0x7fc1601a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.360+0000 7fc15de00640 1 -- 172.21.15.27:0/1342617390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160109240 msgr2=0x7fc16019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc15de00640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160109240 0x7fc16019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc15de00640 1 -- 172.21.15.27:0/1342617390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1601a2400 con 0x7fc160103910 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc15e800640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc160109b90 0x7fc1601a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc15d400640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160109240 0x7fc16019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc15de00640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160103910 0x7fc16019c4d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc150005940 tx=0x7fc1500040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc156a00640 1 -- 172.21.15.27:0/1342617390 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15003f030 con 0x7fc160103910 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1601a2690 con 0x7fc160103910 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc156a00640 1 -- 172.21.15.27:0/1342617390 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1500044c0 con 0x7fc160103910 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.361+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1601a8e80 con 0x7fc160103910 2024-09-21T07:29:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.362+0000 7fc156a00640 1 -- 172.21.15.27:0/1342617390 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150049060 con 0x7fc160103910 2024-09-21T07:29:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 Processor -- start 2024-09-21T07:29:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 -- start start 2024-09-21T07:29:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb7170fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb71710350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb71710890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fb716856b0 con 0x55fb7136b1b0 2024-09-21T07:29:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fb71674ae0 con 0x55fb7134a4d0 2024-09-21T07:29:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc16c02e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb716788f0 con 0x55fb713422f0 2024-09-21T07:29:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.365+0000 7fc15e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb7170fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb7170fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42810/0 (socket says 172.21.15.27:42810) 2024-09-21T07:29:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 -- 172.21.15.27:0/4149556867 learned_addr learned my addr 172.21.15.27:0/4149556867 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15d400640 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb71710890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 -- 172.21.15.27:0/4149556867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 msgr2=0x55fb71710350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:29:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb71710350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 -- 172.21.15.27:0/4149556867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 msgr2=0x55fb71710890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb71710890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc156a00640 1 -- 172.21.15.27:0/1342617390 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc150042030 con 0x7fc160103910 2024-09-21T07:29:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 -- 172.21.15.27:0/4149556867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb71641780 con 0x55fb7136b1b0 2024-09-21T07:29:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.366+0000 7fc15e800640 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb7170fe10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc158011810 tx=0x7fc158011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8031cd568bf362 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.367+0000 7fc14b400640 1 -- 172.21.15.27:0/4149556867 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158011310 con 0x55fb7136b1b0 2024-09-21T07:29:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.367+0000 7fc14b400640 1 -- 172.21.15.27:0/4149556867 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc15800f9a0 con 0x55fb7136b1b0 2024-09-21T07:29:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.367+0000 7fc14b400640 1 -- 172.21.15.27:0/4149556867 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1580056a0 con 0x55fb7136b1b0 2024-09-21T07:29:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 -- 172.21.15.27:0/4149556867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 msgr2=0x55fb7170fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb7170fe10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc158011810 tx=0x7fc158011cd0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc156a00640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1440b6680 0x7fc1440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc156a00640 1 -- 172.21.15.27:0/1342617390 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fc150107df0 con 0x7fc160103910 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 -- 172.21.15.27:0/4149556867 shutdown_connections 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb71710350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb71710890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 --2- 172.21.15.27:0/4149556867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb7170fe10 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 -- 172.21.15.27:0/4149556867 >> 172.21.15.27:0/4149556867 conn(0x55fb7135f300 msgr2=0x55fb7170dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc15e800640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1440b6680 0x7fc1440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 -- 172.21.15.27:0/4149556867 shutdown_connections 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.368+0000 7fc16c02e740 1 -- 172.21.15.27:0/4149556867 wait complete. 2024-09-21T07:29:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.369+0000 7fc15e800640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1440b6680 0x7fc1440b8b90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc158005300 tx=0x7fc158000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.369+0000 7fc16c02e740 1 -- start start 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.369+0000 7fc16c02e740 1 Processor -- start 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.370+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb7171b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.370+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb7171b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.370+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb71727880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.370+0000 7fc16c02e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fb715acab0 con 0x55fb7136b1b0 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.370+0000 7fc16c02e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fb7161b570 con 0x55fb7134a4d0 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.370+0000 7fc16c02e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb7171e590 con 0x55fb713422f0 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb7171b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc16c02e740 10 client.0 _ll_register_callbacks cb 0x55fb7162a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb7171b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:51378/0 (socket says 172.21.15.27:51378) 2024-09-21T07:29:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 -- 172.21.15.27:0/358437338 learned_addr learned my addr 172.21.15.27:0/358437338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15e800640 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb71727880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15d400640 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb7171b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 -- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 msgr2=0x55fb7171b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb7171b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 -- 172.21.15.27:0/358437338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 msgr2=0x55fb71727880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb71727880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 -- 172.21.15.27:0/358437338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb713d6790 con 0x55fb713422f0 2024-09-21T07:29:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc15de00640 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb7171b490 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc150105740 tx=0x7fc1500c9d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6652ed360394074 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.371+0000 7fc13fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc13fe00640 1 -- 172.21.15.27:0/358437338 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150035070 con 0x55fb713422f0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc15800f9a0 con 0x55fb713422f0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc13fe00640 1 -- 172.21.15.27:0/358437338 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fc1500c9190 con 0x55fb713422f0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc13fe00640 1 -- 172.21.15.27:0/358437338 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1500ccfb0 con 0x55fb713422f0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc16c02e740 10 client.30142 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb7144f620 con 0x55fb713422f0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc13fe00640 1 -- 172.21.15.27:0/358437338 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc15800f9a0 con 0x55fb713422f0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc13fe00640 1 -- 172.21.15.27:0/358437338 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc1500ceb70 con 0x55fb713422f0 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.372+0000 7fc16c02e740 10 client.30142 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc16c02e740 10 client.30142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc16c02e740 20 client.30142 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc16c02e740 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb71329d80 0x55fb7172d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc16c02e740 4 client.30142 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb71687600 con 0x55fb71329d80 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc15d400640 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb71329d80 0x55fb7172d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc15d400640 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb71329d80 0x55fb7172d4e0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e8e83432266016 server_cookie=f60542c44eb3a8cb in_seq=0 out_seq=0 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.373+0000 7fc13fe00640 10 client.30142 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.374+0000 7fc13fe00640 1 -- 172.21.15.27:0/358437338 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb71687600 con 0x55fb71329d80 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.374+0000 7fc13fe00640 10 client.30142 handle_command_reply: tid=0 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.374+0000 7fc16c02e740 1 client.30142 shutdown 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.374+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 msgr2=0x55fb7171b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.374+0000 7fc16c02e740 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb7171b490 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc150105740 tx=0x7fc1500c9d30 comp rx=0 tx=0).stop 2024-09-21T07:29:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 shutdown_connections 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb71329d80 0x55fb7172d4e0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb7136b1b0 0x55fb71727880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7134a4d0 0x55fb7171b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 --2- 172.21.15.27:0/358437338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb713422f0 0x55fb7171b490 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc13fe00640 0 client.30142 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 >> 172.21.15.27:0/358437338 conn(0x55fb7135f300 msgr2=0x55fb71710e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 shutdown_connections 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 1 -- 172.21.15.27:0/358437338 wait complete. 2024-09-21T07:29:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.375+0000 7fc16c02e740 20 client.30142 trim_cache size 0 max 16384 2024-09-21T07:29:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.390+0000 7fc16c02e740 1 Processor -- start 2024-09-21T07:29:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.390+0000 7fc16c02e740 1 -- start start 2024-09-21T07:29:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb7170cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71711550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71718660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc16c02e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601b5820 con 0x55fb713422f0 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc16c02e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601b30f0 con 0x55fb7136b1b0 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc16c02e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601b0f60 con 0x55fb7134a4d0 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb7170cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb7170cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42820/0 (socket says 172.21.15.27:42820) 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 -- 172.21.15.27:0/1917468008 learned_addr learned my addr 172.21.15.27:0/1917468008 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15d400640 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71711550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15e800640 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71718660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 -- 172.21.15.27:0/1917468008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 msgr2=0x55fb71711550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71711550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 -- 172.21.15.27:0/1917468008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 msgr2=0x55fb71718660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71718660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15de00640 1 -- 172.21.15.27:0/1917468008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb71712c20 con 0x55fb713422f0 2024-09-21T07:29:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.391+0000 7fc15d400640 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71711550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.392+0000 7fc15de00640 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb7170cd20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc150006010 tx=0x7fc15002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22766eed10fffe28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.392+0000 7fc14aa00640 1 -- 172.21.15.27:0/1917468008 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150035070 con 0x55fb713422f0 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.392+0000 7fc14aa00640 1 -- 172.21.15.27:0/1917468008 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1500cd350 con 0x55fb713422f0 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.392+0000 7fc14aa00640 1 -- 172.21.15.27:0/1917468008 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1500c9500 con 0x55fb713422f0 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 -- 172.21.15.27:0/1917468008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 msgr2=0x55fb7170cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb7170cd20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc150006010 tx=0x7fc15002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 -- 172.21.15.27:0/1917468008 shutdown_connections 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71711550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71718660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 --2- 172.21.15.27:0/1917468008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb7170cd20 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 -- 172.21.15.27:0/1917468008 >> 172.21.15.27:0/1917468008 conn(0x55fb7135f300 msgr2=0x55fb71710e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 -- 172.21.15.27:0/1917468008 shutdown_connections 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 -- 172.21.15.27:0/1917468008 wait complete. 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.393+0000 7fc16c02e740 1 -- start start 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 1 Processor -- start 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb71723d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71721200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71721740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601b15b0 con 0x55fb713422f0 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601b2a20 con 0x55fb7136b1b0 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1601b1c80 con 0x55fb7134a4d0 2024-09-21T07:29:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc16c02e740 10 client.0 _ll_register_callbacks cb 0x7fc1501055f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc15de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb71723d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.394+0000 7fc15de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb71723d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42830/0 (socket says 172.21.15.27:42830) 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15de00640 1 -- 172.21.15.27:0/3161735686 learned_addr learned my addr 172.21.15.27:0/3161735686 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15d400640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71721200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15e800640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71721740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15d400640 1 -- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 msgr2=0x55fb71721740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15d400640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71721740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15d400640 1 -- 172.21.15.27:0/3161735686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 msgr2=0x55fb71723d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15d400640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb71723d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15d400640 1 -- 172.21.15.27:0/3161735686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb71721c80 con 0x55fb7134a4d0 2024-09-21T07:29:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15de00640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb71723d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc15d400640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71721200 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55fb7172dc50 tx=0x7fc14c012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee40832ae1c8b1f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.395+0000 7fc13fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.396+0000 7fc13fe00640 1 -- 172.21.15.27:0/3161735686 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c028030 con 0x55fb7134a4d0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.396+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1500cd350 con 0x55fb7134a4d0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.396+0000 7fc13fe00640 1 -- 172.21.15.27:0/3161735686 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fc14c010af0 con 0x55fb7134a4d0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.396+0000 7fc13fe00640 1 -- 172.21.15.27:0/3161735686 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc14c032050 con 0x55fb7134a4d0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.396+0000 7fc16c02e740 10 client.30148 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.396+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb71797c10 con 0x55fb7134a4d0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.396+0000 7fc13fe00640 1 -- 172.21.15.27:0/3161735686 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1500cd350 con 0x55fb7134a4d0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc13fe00640 1 -- 172.21.15.27:0/3161735686 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc14c004930 con 0x55fb7134a4d0 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc16c02e740 10 client.30148 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc16c02e740 10 client.30148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc16c02e740 20 client.30148 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc16c02e740 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb712ff1c0 0x55fb7179a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc16c02e740 4 client.30148 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:29:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb716830b0 con 0x55fb712ff1c0 2024-09-21T07:29:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.397+0000 7fc15de00640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb712ff1c0 0x55fb7179a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.398+0000 7fc13fe00640 10 client.30148 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.398+0000 7fc15de00640 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb712ff1c0 0x55fb7179a3a0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f01d16a14b00f79e server_cookie=8ba04f0be4cc583c in_seq=0 out_seq=0 2024-09-21T07:29:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc13fe00640 1 -- 172.21.15.27:0/3161735686 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb716830b0 con 0x55fb712ff1c0 2024-09-21T07:29:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc13fe00640 10 client.30148 handle_command_reply: tid=0 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 client.30148 shutdown 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 msgr2=0x55fb71721200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71721200 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55fb7172dc50 tx=0x7fc14c012bc0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 shutdown_connections 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55fb712ff1c0 0x55fb7179a3a0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb7134a4d0 0x55fb71721200 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb7136b1b0 0x55fb71721740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 --2- 172.21.15.27:0/3161735686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fb713422f0 0x55fb71723d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.412+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 >> 172.21.15.27:0/3161735686 conn(0x55fb7135f300 msgr2=0x55fb7170caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.413+0000 7fc13fe00640 0 client.30148 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.413+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 shutdown_connections 2024-09-21T07:29:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.414+0000 7fc16c02e740 1 -- 172.21.15.27:0/3161735686 wait complete. 2024-09-21T07:29:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.414+0000 7fc16c02e740 20 client.30148 trim_cache size 0 max 16384 2024-09-21T07:29:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:29:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:29:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:29:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:29:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1440b6680 msgr2=0x7fc1440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1440b6680 0x7fc1440b8b90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc158005300 tx=0x7fc158000f40 comp rx=0 tx=0).stop 2024-09-21T07:29:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160103910 msgr2=0x7fc16019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160103910 0x7fc16019c4d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc150005940 tx=0x7fc1500040c0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 shutdown_connections 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc160109b90 0x7fc1601a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc1440b6680 0x7fc1440b8b90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc160109240 0x7fc16019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 --2- 172.21.15.27:0/1342617390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc160103910 0x7fc16019c4d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 >> 172.21.15.27:0/1342617390 conn(0x7fc1600ff330 msgr2=0x7fc16010e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 shutdown_connections 2024-09-21T07:29:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:06.415+0000 7fc15f200640 1 -- 172.21.15.27:0/1342617390 wait complete. 2024-09-21T07:29:06.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.500+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97181a6f90 con 0x7f9718105110 2024-09-21T07:29:06.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.501+0000 7f970ea00640 1 -- 172.21.15.27:0/2878057684 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f97040b7210 con 0x7f9718105110 2024-09-21T07:29:06.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:29:06.503 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:29:06.503 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:29:06.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96fc0b6680 msgr2=0x7f96fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96fc0b6680 0x7f96fc0b8b90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9708005ab0 tx=0x7f9708005a20 comp rx=0 tx=0).stop 2024-09-21T07:29:06.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9718105110 msgr2=0x7f971819a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:06.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9718105110 0x7f971819a8e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f970400e9f0 tx=0x7f970400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 shutdown_connections 2024-09-21T07:29:06.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971810dc20 0x7f971819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f96fc0b6680 0x7f96fc0b8b90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9718105110 0x7f971819a8e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 --2- 172.21.15.27:0/2878057684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97181047c0 0x7f971819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:06.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 >> 172.21.15.27:0/2878057684 conn(0x7f97180fe2c0 msgr2=0x7f9718106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:06.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 shutdown_connections 2024-09-21T07:29:06.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:06.504+0000 7f9717200640 1 -- 172.21.15.27:0/2878057684 wait complete. 2024-09-21T07:29:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:06 smithi119 ceph-mon[34519]: pgmap v881: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 34 op/s 2024-09-21T07:29:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:06 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2878057684' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:06 smithi027 ceph-mon[30779]: pgmap v881: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 34 op/s 2024-09-21T07:29:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:06 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2878057684' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:06 smithi153 ceph-mon[34747]: pgmap v881: 129 pgs: 129 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 34 op/s 2024-09-21T07:29:07.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:06 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2878057684' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:07.433 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db / --timeout 90 --expiration 143.2 --await-for 45.0' 2024-09-21T07:29:07.433 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db / --timeout 90 --expiration 143.2 --await-for 45.0 2024-09-21T07:29:07.818 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:07.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.800+0000 7fbc75800640 1 -- 172.21.15.27:0/2579283217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc7010c010 msgr2=0x7fbc70112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.800+0000 7fbc75800640 1 --2- 172.21.15.27:0/2579283217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc7010c010 0x7fbc70112840 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009ca0 tx=0x7fbc6802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.800+0000 7fbc75800640 1 -- 172.21.15.27:0/2579283217 shutdown_connections 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.800+0000 7fbc75800640 1 --2- 172.21.15.27:0/2579283217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc7010c010 0x7fbc70112840 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.800+0000 7fbc75800640 1 --2- 172.21.15.27:0/2579283217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc7010b6c0 0x7fbc7010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.800+0000 7fbc75800640 1 --2- 172.21.15.27:0/2579283217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc70105d80 0x7fbc70106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.801+0000 7fbc75800640 1 -- 172.21.15.27:0/2579283217 >> 172.21.15.27:0/2579283217 conn(0x7fbc701017d0 msgr2=0x7fbc70103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.801+0000 7fbc75800640 1 -- 172.21.15.27:0/2579283217 shutdown_connections 2024-09-21T07:29:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.801+0000 7fbc75800640 1 -- 172.21.15.27:0/2579283217 wait complete. 2024-09-21T07:29:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 Processor -- start 2024-09-21T07:29:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 -- start start 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc70105d80 0x7fbc7019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc7010b6c0 0x7fbc7019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc7010c010 0x7fbc701a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70115d20 con 0x7fbc70105d80 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70115ba0 con 0x7fbc7010b6c0 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.802+0000 7fbc75800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70115ea0 con 0x7fbc7010c010 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.803+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc70105d80 0x7fbc7019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.803+0000 7fbc6f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc7010c010 0x7fbc701a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.803+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc70105d80 0x7fbc7019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55850/0 (socket says 172.21.15.27:55850) 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.803+0000 7fbc6ea00640 1 -- 172.21.15.27:0/1508212473 learned_addr learned my addr 172.21.15.27:0/1508212473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.803+0000 7fbc6e000640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc7010b6c0 0x7fbc7019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6f400640 1 -- 172.21.15.27:0/1508212473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc7010b6c0 msgr2=0x7fbc7019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6f400640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc7010b6c0 0x7fbc7019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6f400640 1 -- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc70105d80 msgr2=0x7fbc7019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6f400640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc70105d80 0x7fbc7019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6f400640 1 -- 172.21.15.27:0/1508212473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc701a2660 con 0x7fbc7010c010 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc70105d80 0x7fbc7019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6e000640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc7010b6c0 0x7fbc7019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:29:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc6f400640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc7010c010 0x7fbc701a1f30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbc68005e30 tx=0x7fbc680317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.804+0000 7fbc67400640 1 -- 172.21.15.27:0/1508212473 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68003b70 con 0x7fbc7010c010 2024-09-21T07:29:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.805+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc701a28f0 con 0x7fbc7010c010 2024-09-21T07:29:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.805+0000 7fbc67400640 1 -- 172.21.15.27:0/1508212473 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc68003d10 con 0x7fbc7010c010 2024-09-21T07:29:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.805+0000 7fbc67400640 1 -- 172.21.15.27:0/1508212473 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6802f970 con 0x7fbc7010c010 2024-09-21T07:29:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.805+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc701a90b0 con 0x7fbc7010c010 2024-09-21T07:29:08.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.808+0000 7fbc7c478740 1 Processor -- start 2024-09-21T07:29:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc67400640 1 -- 172.21.15.27:0/1508212473 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbc6802fb10 con 0x7fbc7010c010 2024-09-21T07:29:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc7c478740 1 -- start start 2024-09-21T07:29:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7bddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc7c478740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56516a732ca0 con 0x56516a3ec980 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc7c478740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56516a7220d0 con 0x56516a3f4b60 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.810+0000 7fbc7c478740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56516a725ee0 con 0x56516a415840 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34848/0 (socket says 172.21.15.27:34848) 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 -- 172.21.15.27:0/2362432395 learned_addr learned my addr 172.21.15.27:0/2362432395 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7bddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 -- 172.21.15.27:0/2362432395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 msgr2=0x56516a7bddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7bddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 -- 172.21.15.27:0/2362432395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 msgr2=0x56516a7bd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7bd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.811+0000 7fbc6e000640 1 -- 172.21.15.27:0/2362432395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516a6d4400 con 0x56516a415840 2024-09-21T07:29:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.812+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7bd350 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c012740 tx=0x7fbc5c009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebbb2029e5908e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.812+0000 7fbc67400640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbc540b6680 0x7fbc540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.812+0000 7fbc67400640 1 -- 172.21.15.27:0/1508212473 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fbc68108bc0 con 0x7fbc7010c010 2024-09-21T07:29:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.812+0000 7fbc5be00640 1 -- 172.21.15.27:0/2362432395 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c012cc0 con 0x56516a415840 2024-09-21T07:29:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.812+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbc540b6680 0x7fbc540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.812+0000 7fbc5be00640 1 -- 172.21.15.27:0/2362432395 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5c0107d0 con 0x56516a415840 2024-09-21T07:29:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.812+0000 7fbc5be00640 1 -- 172.21.15.27:0/2362432395 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0044e0 con 0x56516a415840 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.813+0000 7fbc7c478740 1 -- 172.21.15.27:0/2362432395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 msgr2=0x56516a7bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.813+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbc540b6680 0x7fbc540b8b90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x56516a7bbfd0 tx=0x7fbc60005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.813+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7bd350 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c012740 tx=0x7fbc5c009640 comp rx=0 tx=0).stop 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 -- 172.21.15.27:0/2362432395 shutdown_connections 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7bddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7bd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2362432395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7bd350 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 -- 172.21.15.27:0/2362432395 >> 172.21.15.27:0/2362432395 conn(0x56516a409990 msgr2=0x56516a7bb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 -- 172.21.15.27:0/2362432395 shutdown_connections 2024-09-21T07:29:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 -- 172.21.15.27:0/2362432395 wait complete. 2024-09-21T07:29:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.814+0000 7fbc7c478740 1 -- start start 2024-09-21T07:29:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 1 Processor -- start 2024-09-21T07:29:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3f4b60 0x56516a7c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56516a5837a0 con 0x56516a3f4b60 2024-09-21T07:29:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56516a6c7a10 con 0x56516a3ec980 2024-09-21T07:29:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56516a7cbb30 con 0x56516a415840 2024-09-21T07:29:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.815+0000 7fbc7c478740 10 client.0 _ll_register_callbacks cb 0x56516a6d7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41212/0 (socket says 172.21.15.27:41212) 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3f4b60 0x56516a7c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 -- 172.21.15.27:0/4220960108 learned_addr learned my addr 172.21.15.27:0/4220960108 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6f400640 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 -- 172.21.15.27:0/4220960108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 msgr2=0x56516a7d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 -- 172.21.15.27:0/4220960108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3f4b60 msgr2=0x56516a7c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3f4b60 0x56516a7c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6ea00640 1 -- 172.21.15.27:0/4220960108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56516a48fb90 con 0x56516a3ec980 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.816+0000 7fbc6e000640 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3f4b60 0x56516a7c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.817+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7c8a30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000f960 tx=0x7fbc60004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af890186596505a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.817+0000 7fbc58c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.817+0000 7fbc58c00640 1 -- 172.21.15.27:0/4220960108 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60011b50 con 0x56516a3ec980 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.817+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5c0107d0 con 0x56516a3ec980 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.817+0000 7fbc58c00640 1 -- 172.21.15.27:0/4220960108 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fbc60011cf0 con 0x56516a3ec980 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.817+0000 7fbc58c00640 1 -- 172.21.15.27:0/4220960108 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc6002e6b0 con 0x56516a3ec980 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.818+0000 7fbc7c478740 10 client.30200 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.818+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56516a7d7920 con 0x56516a3ec980 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.818+0000 7fbc58c00640 1 -- 172.21.15.27:0/4220960108 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5c0107d0 con 0x56516a3ec980 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.818+0000 7fbc58c00640 1 -- 172.21.15.27:0/4220960108 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbc60027070 con 0x56516a3ec980 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.818+0000 7fbc7c478740 10 client.30200 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.818+0000 7fbc7c478740 10 client.30200 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.818+0000 7fbc7c478740 20 client.30200 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.819+0000 7fbc7c478740 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3d41c0 0x56516a7dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.819+0000 7fbc7c478740 4 client.30200 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.819+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56516a734bf0 con 0x56516a3d41c0 2024-09-21T07:29:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.819+0000 7fbc6e000640 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3d41c0 0x56516a7dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.820+0000 7fbc6e000640 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3d41c0 0x56516a7dad60 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e30d73dd04314feb server_cookie=8d96852873a893fc in_seq=0 out_seq=0 2024-09-21T07:29:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.820+0000 7fbc58c00640 10 client.30200 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.820+0000 7fbc58c00640 1 -- 172.21.15.27:0/4220960108 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56516a734bf0 con 0x56516a3d41c0 2024-09-21T07:29:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.820+0000 7fbc58c00640 10 client.30200 handle_command_reply: tid=0 2024-09-21T07:29:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.820+0000 7fbc7c478740 1 client.30200 shutdown 2024-09-21T07:29:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 msgr2=0x56516a7c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7c8a30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000f960 tx=0x7fbc60004730 comp rx=0 tx=0).stop 2024-09-21T07:29:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 shutdown_connections 2024-09-21T07:29:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3d41c0 0x56516a7dad60 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3f4b60 0x56516a7c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 --2- 172.21.15.27:0/4220960108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7c8a30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 >> 172.21.15.27:0/4220960108 conn(0x56516a409990 msgr2=0x56516a7b99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc58c00640 0 client.30200 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 shutdown_connections 2024-09-21T07:29:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 1 -- 172.21.15.27:0/4220960108 wait complete. 2024-09-21T07:29:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.821+0000 7fbc7c478740 20 client.30200 trim_cache size 0 max 16384 2024-09-21T07:29:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.840+0000 7fbc7c478740 1 Processor -- start 2024-09-21T07:29:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.840+0000 7fbc7c478740 1 -- start start 2024-09-21T07:29:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a3f4b60 0x56516a7d4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a415840 0x56516a7c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc7c478740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701b5a90 con 0x56516a415840 2024-09-21T07:29:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc7c478740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701b3360 con 0x56516a3ec980 2024-09-21T07:29:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc7c478740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701b11d0 con 0x56516a3f4b60 2024-09-21T07:29:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7bea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41226/0 (socket says 172.21.15.27:41226) 2024-09-21T07:29:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc6ea00640 1 -- 172.21.15.27:0/190020076 learned_addr learned my addr 172.21.15.27:0/190020076 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc6f400640 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a415840 0x56516a7c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.841+0000 7fbc6e000640 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a3f4b60 0x56516a7d4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc6ea00640 1 -- 172.21.15.27:0/190020076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a3f4b60 msgr2=0x56516a7d4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a3f4b60 0x56516a7d4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc6ea00640 1 -- 172.21.15.27:0/190020076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a415840 msgr2=0x56516a7c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a415840 0x56516a7c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc6ea00640 1 -- 172.21.15.27:0/190020076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516a7bfbe0 con 0x56516a3ec980 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc6f400640 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a415840 0x56516a7c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7bea90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbc60013010 tx=0x7fbc600092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240783fd42b74663 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.842+0000 7fbc5b400640 1 -- 172.21.15.27:0/190020076 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6002e450 con 0x56516a3ec980 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.843+0000 7fbc5b400640 1 -- 172.21.15.27:0/190020076 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc60029440 con 0x56516a3ec980 2024-09-21T07:29:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.843+0000 7fbc5b400640 1 -- 172.21.15.27:0/190020076 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6002ecf0 con 0x56516a3ec980 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.843+0000 7fbc7c478740 1 -- 172.21.15.27:0/190020076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 msgr2=0x56516a7bea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.843+0000 7fbc7c478740 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7bea90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbc60013010 tx=0x7fbc600092a0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.844+0000 7fbc7c478740 1 -- 172.21.15.27:0/190020076 shutdown_connections 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.844+0000 7fbc7c478740 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a3f4b60 0x56516a7d4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.844+0000 7fbc7c478740 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a415840 0x56516a7c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.844+0000 7fbc7c478740 1 --2- 172.21.15.27:0/190020076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3ec980 0x56516a7bea90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.844+0000 7fbc7c478740 1 -- 172.21.15.27:0/190020076 >> 172.21.15.27:0/190020076 conn(0x56516a409990 msgr2=0x56516a7be380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.844+0000 7fbc7c478740 1 -- 172.21.15.27:0/190020076 shutdown_connections 2024-09-21T07:29:08.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.844+0000 7fbc7c478740 1 -- 172.21.15.27:0/190020076 wait complete. 2024-09-21T07:29:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.845+0000 7fbc7c478740 1 -- start start 2024-09-21T07:29:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.845+0000 7fbc7c478740 1 Processor -- start 2024-09-21T07:29:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.845+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.845+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc7c478740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc7c478740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701b1820 con 0x56516a3ec980 2024-09-21T07:29:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc7c478740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701b2c90 con 0x56516a3f4b60 2024-09-21T07:29:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc7c478740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701b1ef0 con 0x56516a415840 2024-09-21T07:29:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc7c478740 10 client.0 _ll_register_callbacks cb 0x7fbc60012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55896/0 (socket says 172.21.15.27:55896) 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 -- 172.21.15.27:0/2659216251 learned_addr learned my addr 172.21.15.27:0/2659216251 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6f400640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 -- 172.21.15.27:0/2659216251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 msgr2=0x56516a7cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 -- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 msgr2=0x56516a7cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.846+0000 7fbc6ea00640 1 -- 172.21.15.27:0/2659216251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56516a8447a0 con 0x56516a3ec980 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.847+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.847+0000 7fbc6f400640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:29:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.847+0000 7fbc6ea00640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7ccd70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbc60019040 tx=0x7fbc60011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c468f0847bb7ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.847+0000 7fbc58c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.847+0000 7fbc58c00640 1 -- 172.21.15.27:0/2659216251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60029a00 con 0x56516a3ec980 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.847+0000 7fbc58c00640 1 -- 172.21.15.27:0/2659216251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fbc6002e850 con 0x56516a3ec980 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.847+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc60029440 con 0x56516a3ec980 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc58c00640 1 -- 172.21.15.27:0/2659216251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc6004a050 con 0x56516a3ec980 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 10 client.20193 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56516a8459e0 con 0x56516a3ec980 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc58c00640 1 -- 172.21.15.27:0/2659216251 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc60029440 con 0x56516a3ec980 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc58c00640 1 -- 172.21.15.27:0/2659216251 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbc60035020 con 0x56516a3ec980 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 10 client.20193 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 10 client.20193 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 20 client.20193 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3a9570 0x56516a848100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 4 client.20193 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.2, "await_for": 45.0}] 2024-09-21T07:29:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.848+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.2, "await_for": 45.0}) -- 0x56516a7306a0 con 0x56516a3a9570 2024-09-21T07:29:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.849+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3a9570 0x56516a848100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.849+0000 7fbc6e000640 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3a9570 0x56516a848100 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2bccfb8c8e3819a server_cookie=380829245f6b972 in_seq=0 out_seq=0 2024-09-21T07:29:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.849+0000 7fbc58c00640 10 client.20193 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.860+0000 7fbc58c00640 1 -- 172.21.15.27:0/2659216251 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56516a7306a0 con 0x56516a3a9570 2024-09-21T07:29:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.860+0000 7fbc58c00640 10 client.20193 handle_command_reply: tid=0 2024-09-21T07:29:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.860+0000 7fbc7c478740 1 client.20193 shutdown 2024-09-21T07:29:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 msgr2=0x56516a7ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7ccd70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbc60019040 tx=0x7fbc60011d80 comp rx=0 tx=0).stop 2024-09-21T07:29:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 shutdown_connections 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x56516a3a9570 0x56516a848100 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56516a415840 0x56516a7cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56516a3f4b60 0x56516a7cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 --2- 172.21.15.27:0/2659216251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56516a3ec980 0x56516a7ccd70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 >> 172.21.15.27:0/2659216251 conn(0x56516a409990 msgr2=0x56516a7be380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc58c00640 0 client.20193 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 shutdown_connections 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.861+0000 7fbc7c478740 1 -- 172.21.15.27:0/2659216251 wait complete. 2024-09-21T07:29:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.862+0000 7fbc7c478740 20 client.20193 trim_cache size 0 max 16384 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 62, 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "820df98c": { 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 62, 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:29:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 143.2, 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:08.866 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:29:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbc540b6680 msgr2=0x7fbc540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbc540b6680 0x7fbc540b8b90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x56516a7bbfd0 tx=0x7fbc60005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc7010c010 msgr2=0x7fbc701a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc7010c010 0x7fbc701a1f30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbc68005e30 tx=0x7fbc680317c0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 shutdown_connections 2024-09-21T07:29:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc7010c010 0x7fbc701a1f30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbc540b6680 0x7fbc540b8b90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc7010b6c0 0x7fbc7019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 --2- 172.21.15.27:0/1508212473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc70105d80 0x7fbc7019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 >> 172.21.15.27:0/1508212473 conn(0x7fbc701017d0 msgr2=0x7fbc7010acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 shutdown_connections 2024-09-21T07:29:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:08.863+0000 7fbc75800640 1 -- 172.21.15.27:0/1508212473 wait complete. 2024-09-21T07:29:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:09.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:08 smithi027 ceph-mon[30779]: pgmap v882: 129 pgs: 129 active+clean; 5.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 KiB/s wr, 37 op/s 2024-09-21T07:29:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:08 smithi119 ceph-mon[34519]: pgmap v882: 129 pgs: 129 active+clean; 5.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 KiB/s wr, 37 op/s 2024-09-21T07:29:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:08 smithi153 ceph-mon[34747]: pgmap v882: 129 pgs: 129 active+clean; 5.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 KiB/s wr, 37 op/s 2024-09-21T07:29:10.127 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '820df98c', quiesce took 4.9 seconds. Will release after: 18.299999999999997 2024-09-21T07:29:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 -- 172.21.15.27:0/1431223037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810b6f0 msgr2=0x7fb9f810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 --2- 172.21.15.27:0/1431223037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810b6f0 0x7fb9f810bb00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009860 tx=0x7fb9e402a340 comp rx=0 tx=0).stop 2024-09-21T07:29:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 -- 172.21.15.27:0/1431223037 shutdown_connections 2024-09-21T07:29:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 --2- 172.21.15.27:0/1431223037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810c040 0x7fb9f8112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 --2- 172.21.15.27:0/1431223037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810b6f0 0x7fb9f810bb00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 --2- 172.21.15.27:0/1431223037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f8105db0 0x7fb9f81061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 -- 172.21.15.27:0/1431223037 >> 172.21.15.27:0/1431223037 conn(0x7fb9f81017e0 msgr2=0x7fb9f8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 -- 172.21.15.27:0/1431223037 shutdown_connections 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.826+0000 7fb9fd000640 1 -- 172.21.15.27:0/1431223037 wait complete. 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.828+0000 7fb9fd000640 1 Processor -- start 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.828+0000 7fb9fd000640 1 -- start start 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9fd000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f8105db0 0x7fb9f819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9fd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810b6f0 0x7fb9f819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9fd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810c040 0x7fb9f81a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9fd000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8115d00 con 0x7fb9f810b6f0 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9fd000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8115b80 con 0x7fb9f810c040 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9fd000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8115e80 con 0x7fb9f8105db0 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810b6f0 0x7fb9f819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810b6f0 0x7fb9f819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55906/0 (socket says 172.21.15.27:55906) 2024-09-21T07:29:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9f5400640 1 -- 172.21.15.27:0/3112317434 learned_addr learned my addr 172.21.15.27:0/3112317434 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f8105db0 0x7fb9f819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.829+0000 7fb9f6800640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810c040 0x7fb9f81a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9f5400640 1 -- 172.21.15.27:0/3112317434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f8105db0 msgr2=0x7fb9f819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9f5400640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f8105db0 0x7fb9f819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9f5400640 1 -- 172.21.15.27:0/3112317434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810c040 msgr2=0x7fb9f81a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9f5400640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810c040 0x7fb9f81a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9f5400640 1 -- 172.21.15.27:0/3112317434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f81a2660 con 0x7fb9f810b6f0 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f8105db0 0x7fb9f819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9f5400640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810b6f0 0x7fb9f819cc90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4035040 tx=0x7fb9e4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9eea00640 1 -- 172.21.15.27:0/3112317434 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4040020 con 0x7fb9f810b6f0 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9eea00640 1 -- 172.21.15.27:0/3112317434 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9e4030e10 con 0x7fb9f810b6f0 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f81a28f0 con 0x7fb9f810b6f0 2024-09-21T07:29:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.830+0000 7fb9eea00640 1 -- 172.21.15.27:0/3112317434 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4037040 con 0x7fb9f810b6f0 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.831+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f81a90b0 con 0x7fb9f810b6f0 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.834+0000 7fba03dc3740 1 Processor -- start 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fba03dc3740 1 -- start start 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fb9eea00640 1 -- 172.21.15.27:0/3112317434 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb9e4043020 con 0x7fb9f810b6f0 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd809d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd80a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd442390 0x55bddd80a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fba03dc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bddd780f70 con 0x55bddd442390 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fba03dc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bddd7703a0 con 0x55bddd43a1b0 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fba03dc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bddd7741b0 con 0x55bddd463070 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.835+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd442390 0x55bddd80a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd80a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f6800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd809d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd442390 0x55bddd80a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55908/0 (socket says 172.21.15.27:55908) 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5400640 1 -- 172.21.15.27:0/3427221832 learned_addr learned my addr 172.21.15.27:0/3427221832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5e00640 1 -- 172.21.15.27:0/3427221832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 msgr2=0x55bddd809d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/3427221832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd809d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5e00640 1 -- 172.21.15.27:0/3427221832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd442390 msgr2=0x55bddd80a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/3427221832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd442390 0x55bddd80a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5e00640 1 -- 172.21.15.27:0/3427221832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bddd73ac90 con 0x55bddd43a1b0 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.836+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/3427221832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd80a2a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009fd0 tx=0x7fb9e80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff47c1d302f119a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.837+0000 7fb9e3400640 1 -- 172.21.15.27:0/3427221832 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8004310 con 0x55bddd43a1b0 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.837+0000 7fb9e3400640 1 -- 172.21.15.27:0/3427221832 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9e80044b0 con 0x55bddd43a1b0 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.837+0000 7fb9e3400640 1 -- 172.21.15.27:0/3427221832 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e80035f0 con 0x55bddd43a1b0 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.837+0000 7fb9eea00640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9dc0b6680 0x7fb9dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.837+0000 7fb9f6800640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9dc0b6680 0x7fb9dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.837+0000 7fb9eea00640 1 -- 172.21.15.27:0/3112317434 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fb9e4107680 con 0x7fb9f810b6f0 2024-09-21T07:29:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 -- 172.21.15.27:0/3427221832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 msgr2=0x55bddd80a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 --2- 172.21.15.27:0/3427221832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd80a2a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009fd0 tx=0x7fb9e80118d0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 -- 172.21.15.27:0/3427221832 shutdown_connections 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 --2- 172.21.15.27:0/3427221832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd442390 0x55bddd80a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 --2- 172.21.15.27:0/3427221832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd80a2a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 --2- 172.21.15.27:0/3427221832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd809d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 -- 172.21.15.27:0/3427221832 >> 172.21.15.27:0/3427221832 conn(0x55bddd457460 msgr2=0x55bddd807c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fb9f6800640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9dc0b6680 0x7fb9dc0b8b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005fd0 tx=0x7fb9f0010040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 -- 172.21.15.27:0/3427221832 shutdown_connections 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.838+0000 7fba03dc3740 1 -- 172.21.15.27:0/3427221832 wait complete. 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.839+0000 7fba03dc3740 1 -- start start 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.839+0000 7fba03dc3740 1 Processor -- start 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.839+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd8154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd815a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd821860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fba03dc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bddd5d1700 con 0x55bddd43a1b0 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fba03dc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bddd70b540 con 0x55bddd442390 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fba03dc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bddd818540 con 0x55bddd463070 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fba03dc3740 10 client.0 _ll_register_callbacks cb 0x55bddd71b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd8154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd8154d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55914/0 (socket says 172.21.15.27:55914) 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 -- 172.21.15.27:0/1100643114 learned_addr learned my addr 172.21.15.27:0/1100643114 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f6800640 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd821860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5400640 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd815a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 -- 172.21.15.27:0/1100643114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 msgr2=0x55bddd821860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd821860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 -- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 msgr2=0x55bddd815a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd815a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.840+0000 7fb9f5e00640 1 -- 172.21.15.27:0/1100643114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bddd4ce650 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd8154d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb9e801b040 tx=0x7fb9e8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896cf6d6fbd1b7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fb9d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fb9d7e00640 1 -- 172.21.15.27:0/1100643114 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8029070 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fb9d7e00640 1 -- 172.21.15.27:0/1100643114 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fb9e8003a00 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bddd70db50 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fb9d7e00640 1 -- 172.21.15.27:0/1100643114 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb9e8035050 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fba03dc3740 10 client.20205 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.841+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bddd728910 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fb9d7e00640 1 -- 172.21.15.27:0/1100643114 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bddd70db50 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fb9d7e00640 1 -- 172.21.15.27:0/1100643114 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb9e8004c70 con 0x55bddd43a1b0 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fba03dc3740 10 client.20205 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fba03dc3740 10 client.20205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fba03dc3740 20 client.20205 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fba03dc3740 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd4219f0 0x55bddd8274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fba03dc3740 4 client.20205 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bddd782ec0 con 0x55bddd4219f0 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.842+0000 7fb9f5400640 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd4219f0 0x55bddd8274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.843+0000 7fb9f5400640 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd4219f0 0x55bddd8274d0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d245ab40eec92bf7 server_cookie=8e812975498f7a2f in_seq=0 out_seq=0 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.843+0000 7fb9d7e00640 10 client.20205 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fb9d7e00640 1 -- 172.21.15.27:0/1100643114 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bddd782ec0 con 0x55bddd4219f0 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fb9d7e00640 10 client.20205 handle_command_reply: tid=0 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 client.20205 shutdown 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 msgr2=0x55bddd8154d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd8154d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb9e801b040 tx=0x7fb9e8004340 comp rx=0 tx=0).stop 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 shutdown_connections 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd4219f0 0x55bddd8274d0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd821860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd815a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 --2- 172.21.15.27:0/1100643114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd8154d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 >> 172.21.15.27:0/1100643114 conn(0x55bddd457460 msgr2=0x55bddd80cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fb9d7e00640 0 client.20205 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 shutdown_connections 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.844+0000 7fba03dc3740 1 -- 172.21.15.27:0/1100643114 wait complete. 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.845+0000 7fba03dc3740 20 client.20205 trim_cache size 0 max 16384 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.863+0000 7fba03dc3740 1 Processor -- start 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.863+0000 7fba03dc3740 1 -- start start 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.863+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd810c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.863+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd442390 0x55bddd815670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd463070 0x55bddd80b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fba03dc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81b5a30 con 0x55bddd463070 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fba03dc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81b3300 con 0x55bddd43a1b0 2024-09-21T07:29:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fba03dc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81b1170 con 0x55bddd442390 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd463070 0x55bddd80b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd810c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd442390 0x55bddd815670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd463070 0x55bddd80b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55922/0 (socket says 172.21.15.27:55922) 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f6800640 1 -- 172.21.15.27:0/2571053713 learned_addr learned my addr 172.21.15.27:0/2571053713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f5400640 1 -- 172.21.15.27:0/2571053713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 msgr2=0x55bddd810c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f5400640 1 --2- 172.21.15.27:0/2571053713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd810c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f5400640 1 -- 172.21.15.27:0/2571053713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd463070 msgr2=0x55bddd80b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f5400640 1 --2- 172.21.15.27:0/2571053713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd463070 0x55bddd80b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.864+0000 7fb9f5400640 1 -- 172.21.15.27:0/2571053713 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bddd816790 con 0x55bddd442390 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.865+0000 7fb9f5400640 1 --2- 172.21.15.27:0/2571053713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd442390 0x55bddd815670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40ccdd0 tx=0x7fb9e4109b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f5eacd5c3fcdb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.865+0000 7fb9e2a00640 1 -- 172.21.15.27:0/2571053713 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40c8f70 con 0x55bddd442390 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.865+0000 7fb9e2a00640 1 -- 172.21.15.27:0/2571053713 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9e4107070 con 0x55bddd442390 2024-09-21T07:29:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.865+0000 7fb9e2a00640 1 -- 172.21.15.27:0/2571053713 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40cc840 con 0x55bddd442390 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 -- 172.21.15.27:0/2571053713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd442390 msgr2=0x55bddd815670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 --2- 172.21.15.27:0/2571053713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd442390 0x55bddd815670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40ccdd0 tx=0x7fb9e4109b90 comp rx=0 tx=0).stop 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 -- 172.21.15.27:0/2571053713 shutdown_connections 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 --2- 172.21.15.27:0/2571053713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd442390 0x55bddd815670 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 --2- 172.21.15.27:0/2571053713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd463070 0x55bddd80b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 --2- 172.21.15.27:0/2571053713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd43a1b0 0x55bddd810c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 -- 172.21.15.27:0/2571053713 >> 172.21.15.27:0/2571053713 conn(0x55bddd457460 msgr2=0x55bddd80ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 -- 172.21.15.27:0/2571053713 shutdown_connections 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 -- 172.21.15.27:0/2571053713 wait complete. 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 -- start start 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.866+0000 7fba03dc3740 1 Processor -- start 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd81dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd81e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fba03dc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd8191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fba03dc3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81b17c0 con 0x55bddd43a1b0 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fba03dc3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81b2c30 con 0x55bddd442390 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fba03dc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81b1e90 con 0x55bddd463070 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fba03dc3740 10 client.0 _ll_register_callbacks cb 0x7fb9e8009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd81dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd81dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55936/0 (socket says 172.21.15.27:55936) 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fb9f5e00640 1 -- 172.21.15.27:0/976179200 learned_addr learned my addr 172.21.15.27:0/976179200 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fb9f5400640 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd81e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.867+0000 7fb9f6800640 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd8191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9f5e00640 1 -- 172.21.15.27:0/976179200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 msgr2=0x55bddd8191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd8191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9f5e00640 1 -- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 msgr2=0x55bddd81e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd81e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9f5e00640 1 -- 172.21.15.27:0/976179200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bddd819920 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9f5e00640 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd81dba0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x55bddd817d20 tx=0x7fb9e8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98862544bc0651 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9d7e00640 1 -- 172.21.15.27:0/976179200 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8029040 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9d7e00640 1 -- 172.21.15.27:0/976179200 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9549+0+0 (secure 0 0 0) 0x7fb9e8004aa0 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e4107070 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.868+0000 7fb9d7e00640 1 -- 172.21.15.27:0/976179200 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb9e8042050 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 10 client.20211 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bddd8918f0 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fb9d7e00640 1 -- 172.21.15.27:0/976179200 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9e4107070 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fb9d7e00640 1 -- 172.21.15.27:0/976179200 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb9e800f4e0 con 0x55bddd43a1b0 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 10 client.20211 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 10 client.20211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 20 client.20211 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd3f6da0 0x55bddd894080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 4 client.20211 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bddd77e970 con 0x55bddd3f6da0 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.869+0000 7fb9f5400640 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd3f6da0 0x55bddd894080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.870+0000 7fb9f5400640 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd3f6da0 0x55bddd894080 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=946649243d0135d1 server_cookie=d251863383408d9c in_seq=0 out_seq=0 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.870+0000 7fb9d7e00640 10 client.20211 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.870+0000 7fb9d7e00640 1 -- 172.21.15.27:0/976179200 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7190 (crc 0 0 0) 0x55bddd77e970 con 0x55bddd3f6da0 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.870+0000 7fb9d7e00640 10 client.20211 handle_command_reply: tid=0 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.870+0000 7fba03dc3740 1 client.20211 shutdown 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 msgr2=0x55bddd81dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd81dba0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x55bddd817d20 tx=0x7fb9e8005f10 comp rx=0 tx=0).stop 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 shutdown_connections 2024-09-21T07:29:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55bddd3f6da0 0x55bddd894080 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bddd463070 0x55bddd8191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bddd442390 0x55bddd81e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 --2- 172.21.15.27:0/976179200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bddd43a1b0 0x55bddd81dba0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fb9d7e00640 0 client.20211 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 >> 172.21.15.27:0/976179200 conn(0x55bddd457460 msgr2=0x55bddd813d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 shutdown_connections 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 1 -- 172.21.15.27:0/976179200 wait complete. 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:29:10.298 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "23d6d7e9-06b5-4a6e-adfb-86c244736c20": { 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:49.824406+0000", 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.299 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:29:10.300 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "594683c9-bab5-4ebb-ae50-7e80b1ad2ba8": { 2024-09-21T07:29:10.301 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:28:35.097223+0000", 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.303 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.304 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.305 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.306 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.307 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.308 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:29:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.871+0000 7fba03dc3740 20 client.20211 trim_cache size 0 max 16384 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9dc0b6680 msgr2=0x7fb9dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9dc0b6680 0x7fb9dc0b8b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005fd0 tx=0x7fb9f0010040 comp rx=0 tx=0).stop 2024-09-21T07:29:10.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810b6f0 msgr2=0x7fb9f819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810b6f0 0x7fb9f819cc90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4035040 tx=0x7fb9e4032d50 comp rx=0 tx=0).stop 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 shutdown_connections 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb9dc0b6680 0x7fb9dc0b8b90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f810c040 0x7fb9f81a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9f810b6f0 0x7fb9f819cc90 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 --2- 172.21.15.27:0/3112317434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f8105db0 0x7fb9f819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 >> 172.21.15.27:0/3112317434 conn(0x7fb9f81017e0 msgr2=0x7fb9f810ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 shutdown_connections 2024-09-21T07:29:10.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:09.872+0000 7fb9fd000640 1 -- 172.21.15.27:0/3112317434 wait complete. 2024-09-21T07:29:10.975 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:29:10.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:29:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:10 smithi119 ceph-mon[34519]: pgmap v883: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-09-21T07:29:11.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:10 smithi027 ceph-mon[30779]: pgmap v883: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-09-21T07:29:11.359 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:10 smithi153 ceph-mon[34747]: pgmap v883: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.968+0000 7f646b000640 1 -- 172.21.15.27:0/2791830525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6464103990 msgr2=0x7f6464103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.968+0000 7f646b000640 1 --2- 172.21.15.27:0/2791830525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6464103990 0x7f6464103da0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6458009900 tx=0x7f645802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.969+0000 7f646b000640 1 -- 172.21.15.27:0/2791830525 shutdown_connections 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.969+0000 7f646b000640 1 --2- 172.21.15.27:0/2791830525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64641092d0 0x7f64641096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.969+0000 7f646b000640 1 --2- 172.21.15.27:0/2791830525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6464109c20 0x7f6464110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.969+0000 7f646b000640 1 --2- 172.21.15.27:0/2791830525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6464103990 0x7f6464103da0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.969+0000 7f646b000640 1 -- 172.21.15.27:0/2791830525 >> 172.21.15.27:0/2791830525 conn(0x7f64640ff440 msgr2=0x7f6464101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.969+0000 7f646b000640 1 -- 172.21.15.27:0/2791830525 shutdown_connections 2024-09-21T07:29:11.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.969+0000 7f646b000640 1 -- 172.21.15.27:0/2791830525 wait complete. 2024-09-21T07:29:11.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 Processor -- start 2024-09-21T07:29:11.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 -- start start 2024-09-21T07:29:11.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6464103990 0x7f646419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:11.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64641092d0 0x7f646419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:11.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6464109c20 0x7f64641a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:11.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6464113930 con 0x7f64641092d0 2024-09-21T07:29:11.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f64641137b0 con 0x7f6464109c20 2024-09-21T07:29:11.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.970+0000 7f646b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6464113ab0 con 0x7f6464103990 2024-09-21T07:29:11.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64641092d0 0x7f646419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:11.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6464103990 0x7f646419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:11.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64641092d0 0x7f646419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55946/0 (socket says 172.21.15.27:55946) 2024-09-21T07:29:11.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6464103990 0x7f646419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34942/0 (socket says 172.21.15.27:34942) 2024-09-21T07:29:11.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469200640 1 -- 172.21.15.27:0/3283318519 learned_addr learned my addr 172.21.15.27:0/3283318519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:11.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f646a600640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6464109c20 0x7f64641a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:11.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469200640 1 -- 172.21.15.27:0/3283318519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6464103990 msgr2=0x7f646419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:11.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469200640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6464103990 0x7f646419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:11.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.971+0000 7f6469200640 1 -- 172.21.15.27:0/3283318519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6464109c20 msgr2=0x7f64641a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f6469200640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6464109c20 0x7f64641a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f6469200640 1 -- 172.21.15.27:0/3283318519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64641a2490 con 0x7f64641092d0 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f646a600640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6464109c20 0x7f64641a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f6469200640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64641092d0 0x7f646419cac0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f645000d740 tx=0x7f645000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f6462000640 1 -- 172.21.15.27:0/3283318519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450017070 con 0x7f64641092d0 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64641a2780 con 0x7f64641092d0 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f6462000640 1 -- 172.21.15.27:0/3283318519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f645000de50 con 0x7f64641092d0 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.972+0000 7f6462000640 1 -- 172.21.15.27:0/3283318519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645000f620 con 0x7f64641092d0 2024-09-21T07:29:11.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.973+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641a8ee0 con 0x7f64641092d0 2024-09-21T07:29:11.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.974+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6464103da0 con 0x7f64641092d0 2024-09-21T07:29:11.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.976+0000 7f6462000640 1 -- 172.21.15.27:0/3283318519 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f645001a070 con 0x7f64641092d0 2024-09-21T07:29:11.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.978+0000 7f6462000640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f644c0b6680 0x7f644c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:11.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.979+0000 7f6462000640 1 -- 172.21.15.27:0/3283318519 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f64500e7460 con 0x7f64641092d0 2024-09-21T07:29:11.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.979+0000 7f6469c00640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f644c0b6680 0x7f644c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:11.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.980+0000 7f6469c00640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f644c0b6680 0x7f644c0b8b90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6458005ab0 tx=0x7f6458005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:11.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:11.982+0000 7f6462000640 1 -- 172.21.15.27:0/3283318519 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64500b1980 con 0x7f64641092d0 2024-09-21T07:29:12.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.220+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64641a91c0 con 0x7f64641092d0 2024-09-21T07:29:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.221+0000 7f6462000640 1 -- 172.21.15.27:0/3283318519 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f64500b68c0 con 0x7f64641092d0 2024-09-21T07:29:12.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:29:12.223 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:29:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:29:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f644c0b6680 msgr2=0x7f644c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f644c0b6680 0x7f644c0b8b90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6458005ab0 tx=0x7f6458005a20 comp rx=0 tx=0).stop 2024-09-21T07:29:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64641092d0 msgr2=0x7f646419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64641092d0 0x7f646419cac0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f645000d740 tx=0x7f645000dc00 comp rx=0 tx=0).stop 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 shutdown_connections 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f644c0b6680 0x7f644c0b8b90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6464109c20 0x7f64641a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64641092d0 0x7f646419cac0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 --2- 172.21.15.27:0/3283318519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6464103990 0x7f646419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.223+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 >> 172.21.15.27:0/3283318519 conn(0x7f64640ff440 msgr2=0x7f6464108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.224+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 shutdown_connections 2024-09-21T07:29:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:12.224+0000 7f646b000640 1 -- 172.21.15.27:0/3283318519 wait complete. 2024-09-21T07:29:13.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:29:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:12 smithi153 ceph-mon[34747]: pgmap v884: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-09-21T07:29:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:12 smithi153 ceph-mon[34747]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T07:29:13.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:12 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3283318519' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:13.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:12 smithi119 ceph-mon[34519]: pgmap v884: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-09-21T07:29:13.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:12 smithi119 ceph-mon[34519]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T07:29:13.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:12 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3283318519' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:12 smithi027 ceph-mon[30779]: pgmap v884: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-09-21T07:29:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:12 smithi027 ceph-mon[30779]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T07:29:13.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:12 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3283318519' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:13.402 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:14.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.006+0000 7fd34be00640 1 -- 172.21.15.27:0/2030367354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 msgr2=0x7fd34c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.006+0000 7fd34be00640 1 --2- 172.21.15.27:0/2030367354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 0x7fd34c10b900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd338009900 tx=0x7fd33802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.006+0000 7fd34be00640 1 -- 172.21.15.27:0/2030367354 shutdown_connections 2024-09-21T07:29:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.006+0000 7fd34be00640 1 --2- 172.21.15.27:0/2030367354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34c10be40 0x7fd34c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.006+0000 7fd34be00640 1 --2- 172.21.15.27:0/2030367354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 0x7fd34c10b900 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.006+0000 7fd34be00640 1 --2- 172.21.15.27:0/2030367354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c105bb0 0x7fd34c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.006+0000 7fd34be00640 1 -- 172.21.15.27:0/2030367354 >> 172.21.15.27:0/2030367354 conn(0x7fd34c101680 msgr2=0x7fd34c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.007+0000 7fd34be00640 1 -- 172.21.15.27:0/2030367354 shutdown_connections 2024-09-21T07:29:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.007+0000 7fd34be00640 1 -- 172.21.15.27:0/2030367354 wait complete. 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 Processor -- start 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 -- start start 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34c105bb0 0x7fd34c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 0x7fd34c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c10be40 0x7fd34c1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c115b00 con 0x7fd34c10be40 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c115980 con 0x7fd34c10b4f0 2024-09-21T07:29:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.008+0000 7fd34be00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c115c80 con 0x7fd34c105bb0 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34c105bb0 0x7fd34c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 0x7fd34c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c10be40 0x7fd34c1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34c105bb0 0x7fd34c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:34960/0 (socket says 172.21.15.27:34960) 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34aa00640 1 -- 172.21.15.27:0/771735308 learned_addr learned my addr 172.21.15.27:0/771735308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34a000640 1 -- 172.21.15.27:0/771735308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34c105bb0 msgr2=0x7fd34c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34a000640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34c105bb0 0x7fd34c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34a000640 1 -- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c10be40 msgr2=0x7fd34c1a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34a000640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c10be40 0x7fd34c1a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34a000640 1 -- 172.21.15.27:0/771735308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c1a24a0 con 0x7fd34c10b4f0 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.009+0000 7fd34b400640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c10be40 0x7fd34c1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.010+0000 7fd34a000640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 0x7fd34c19cad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd33802a910 tx=0x7fd338031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.010+0000 7fd343400640 1 -- 172.21.15.27:0/771735308 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33803f070 con 0x7fd34c10b4f0 2024-09-21T07:29:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.010+0000 7fd343400640 1 -- 172.21.15.27:0/771735308 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd338004490 con 0x7fd34c10b4f0 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.010+0000 7fd343400640 1 -- 172.21.15.27:0/771735308 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338035070 con 0x7fd34c10b4f0 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.010+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c1a2730 con 0x7fd34c10b4f0 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.010+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c1a2bc0 con 0x7fd34c10b4f0 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.013+0000 7fd3587d5740 1 Processor -- start 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.013+0000 7fd3587d5740 1 -- start start 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8682f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b86834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b86839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd3587d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f2b85f9fb0 con 0x55f2b82bb390 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd3587d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f2b85e93e0 con 0x55f2b82dc070 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd3587d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f2b85ed1f0 con 0x55f2b82b31b0 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd34b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8682f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd34b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8682f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41328/0 (socket says 172.21.15.27:41328) 2024-09-21T07:29:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd34b400640 1 -- 172.21.15.27:0/1392446449 learned_addr learned my addr 172.21.15.27:0/1392446449 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd343400640 1 -- 172.21.15.27:0/771735308 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd33803b460 con 0x7fd34c10b4f0 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.014+0000 7fd34aa00640 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b86834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd34b400640 1 -- 172.21.15.27:0/1392446449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 msgr2=0x55f2b86834b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd34b400640 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b86834b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd34b400640 1 -- 172.21.15.27:0/1392446449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 msgr2=0x55f2b86839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd34b400640 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b86839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd34b400640 1 -- 172.21.15.27:0/1392446449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b85b3cd0 con 0x55f2b82dc070 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd34b400640 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8682f70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd3440117e0 tx=0x7fd344011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be215f568971e45e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd337e00640 1 -- 172.21.15.27:0/1392446449 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344011310 con 0x55f2b82dc070 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd337e00640 1 -- 172.21.15.27:0/1392446449 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd344003e20 con 0x55f2b82dc070 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.015+0000 7fd337e00640 1 -- 172.21.15.27:0/1392446449 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440044f0 con 0x55f2b82dc070 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.016+0000 7fd3587d5740 1 -- 172.21.15.27:0/1392446449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 msgr2=0x55f2b8682f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.016+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8682f70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd3440117e0 tx=0x7fd344011ca0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.016+0000 7fd343400640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3300b6680 0x7fd3300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd343400640 1 -- 172.21.15.27:0/771735308 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fd338107be0 con 0x7fd34c10b4f0 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 -- 172.21.15.27:0/1392446449 shutdown_connections 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b86834b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b86839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1392446449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8682f70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 -- 172.21.15.27:0/1392446449 >> 172.21.15.27:0/1392446449 conn(0x55f2b82d0460 msgr2=0x55f2b8680e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd34b400640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3300b6680 0x7fd3300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 -- 172.21.15.27:0/1392446449 shutdown_connections 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 -- 172.21.15.27:0/1392446449 wait complete. 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd3587d5740 1 -- start start 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.017+0000 7fd34b400640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3300b6680 0x7fd3300b8b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd344006010 tx=0x7fd344005bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 1 Processor -- start 2024-09-21T07:29:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b868e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82bb390 0x55f2b868ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82dc070 0x55f2b869aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f2b844a700 con 0x55f2b82dc070 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8584580 con 0x55f2b82bb390 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f2b86917a0 con 0x55f2b82b31b0 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.018+0000 7fd3587d5740 10 client.0 _ll_register_callbacks cb 0x55f2b8594470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82bb390 0x55f2b868ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b868e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82bb390 0x55f2b868ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41338/0 (socket says 172.21.15.27:41338) 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 -- 172.21.15.27:0/2009028951 learned_addr learned my addr 172.21.15.27:0/2009028951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34b400640 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82dc070 0x55f2b869aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 -- 172.21.15.27:0/2009028951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 msgr2=0x55f2b868e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b868e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 -- 172.21.15.27:0/2009028951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82dc070 msgr2=0x55f2b869aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82dc070 0x55f2b869aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 -- 172.21.15.27:0/2009028951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b8347650 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd34a000640 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82bb390 0x55f2b868ecf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd3380cd3d0 tx=0x7fd3381054a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e72137e4cf5179a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.019+0000 7fd334c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.020+0000 7fd334c00640 1 -- 172.21.15.27:0/2009028951 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338048040 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.020+0000 7fd334c00640 1 -- 172.21.15.27:0/2009028951 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fd3380c9020 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.020+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2b8586b90 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.020+0000 7fd334c00640 1 -- 172.21.15.27:0/2009028951 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd3380cce40 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.020+0000 7fd3587d5740 10 client.30242 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.020+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2b85a1950 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.020+0000 7fd334c00640 1 -- 172.21.15.27:0/2009028951 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f2b8586b90 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd334c00640 1 -- 172.21.15.27:0/2009028951 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd338031a90 con 0x55f2b82bb390 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd3587d5740 10 client.30242 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd3587d5740 10 client.30242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd3587d5740 20 client.30242 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b829a9f0 0x55f2b86a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd3587d5740 4 client.30242 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b85fbf00 con 0x55f2b829a9f0 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.021+0000 7fd34aa00640 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b829a9f0 0x55f2b86a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.022+0000 7fd34aa00640 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b829a9f0 0x55f2b86a08f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a4fe6227dde974d server_cookie=e4218c7f1c9b55da in_seq=0 out_seq=0 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.022+0000 7fd334c00640 10 client.30242 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.022+0000 7fd334c00640 1 -- 172.21.15.27:0/2009028951 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2b85fbf00 con 0x55f2b829a9f0 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.022+0000 7fd334c00640 10 client.30242 handle_command_reply: tid=0 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 client.30242 shutdown 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82bb390 msgr2=0x55f2b868ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82bb390 0x55f2b868ecf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd3380cd3d0 tx=0x7fd3381054a0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 shutdown_connections 2024-09-21T07:29:14.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b829a9f0 0x55f2b86a08f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82dc070 0x55f2b869aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82bb390 0x55f2b868ecf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2009028951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b868e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 >> 172.21.15.27:0/2009028951 conn(0x55f2b82d0460 msgr2=0x55f2b867f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd334c00640 0 client.30242 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 shutdown_connections 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 1 -- 172.21.15.27:0/2009028951 wait complete. 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.023+0000 7fd3587d5740 20 client.30242 trim_cache size 0 max 16384 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.037+0000 7fd3587d5740 1 Processor -- start 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd3587d5740 1 -- start start 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b8690650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b8686780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8680610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd3587d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b5730 con 0x55f2b82bb390 2024-09-21T07:29:14.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd3587d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b3000 con 0x55f2b82dc070 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd3587d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b0e70 con 0x55f2b82b31b0 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd34a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b8686780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd34a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b8686780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55988/0 (socket says 172.21.15.27:55988) 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd34a000640 1 -- 172.21.15.27:0/2673399549 learned_addr learned my addr 172.21.15.27:0/2673399549 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd34aa00640 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b8690650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.038+0000 7fd34b400640 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8680610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd34a000640 1 -- 172.21.15.27:0/2673399549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 msgr2=0x55f2b8690650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd34a000640 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b8690650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd34a000640 1 -- 172.21.15.27:0/2673399549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 msgr2=0x55f2b8680610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd34a000640 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8680610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd34a000640 1 -- 172.21.15.27:0/2673399549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b8685260 con 0x55f2b82bb390 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd34aa00640 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b8690650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:14.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd34a000640 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b8686780 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd3380d0040 tx=0x7fd338000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3e28ca4ed9e53b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd337400640 1 -- 172.21.15.27:0/2673399549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338048030 con 0x55f2b82bb390 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.039+0000 7fd337400640 1 -- 172.21.15.27:0/2673399549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3380d3290 con 0x55f2b82bb390 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd337400640 1 -- 172.21.15.27:0/2673399549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3380c9020 con 0x55f2b82bb390 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 -- 172.21.15.27:0/2673399549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 msgr2=0x55f2b8686780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b8686780 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd3380d0040 tx=0x7fd338000bc0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 -- 172.21.15.27:0/2673399549 shutdown_connections 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8680610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82bb390 0x55f2b8686780 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 --2- 172.21.15.27:0/2673399549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82b31b0 0x55f2b8690650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 -- 172.21.15.27:0/2673399549 >> 172.21.15.27:0/2673399549 conn(0x55f2b82d0460 msgr2=0x55f2b8686070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 -- 172.21.15.27:0/2673399549 shutdown_connections 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.040+0000 7fd3587d5740 1 -- 172.21.15.27:0/2673399549 wait complete. 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.041+0000 7fd3587d5740 1 -- start start 2024-09-21T07:29:14.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.041+0000 7fd3587d5740 1 Processor -- start 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82b31b0 0x55f2b8697420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82bb390 0x55f2b8692950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd3587d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8693060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd3587d5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b14c0 con 0x55f2b82b31b0 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd3587d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b2930 con 0x55f2b82dc070 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd3587d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b1b90 con 0x55f2b82bb390 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd3587d5740 10 client.0 _ll_register_callbacks cb 0x7fd3380c9b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82b31b0 0x55f2b8697420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82b31b0 0x55f2b8697420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55990/0 (socket says 172.21.15.27:55990) 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 -- 172.21.15.27:0/1145379613 learned_addr learned my addr 172.21.15.27:0/1145379613 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34b400640 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8693060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34a000640 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82bb390 0x55f2b8692950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 -- 172.21.15.27:0/1145379613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82bb390 msgr2=0x55f2b8692950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82bb390 0x55f2b8692950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 -- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 msgr2=0x55f2b8693060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8693060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.042+0000 7fd34aa00640 1 -- 172.21.15.27:0/1145379613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b870a250 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.043+0000 7fd34aa00640 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82b31b0 0x55f2b8697420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005e00 tx=0x7fd33c012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58890a59b7ea6d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.043+0000 7fd334c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.043+0000 7fd334c00640 1 -- 172.21.15.27:0/1145379613 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c028020 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.043+0000 7fd334c00640 1 -- 172.21.15.27:0/1145379613 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fd33c01e9c0 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.043+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3380d3290 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.043+0000 7fd334c00640 1 -- 172.21.15.27:0/1145379613 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd33c033050 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.043+0000 7fd3587d5740 10 client.20232 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2b870b470 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd334c00640 1 -- 172.21.15.27:0/1145379613 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3380d3290 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd334c00640 1 -- 172.21.15.27:0/1145379613 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd33c010c40 con 0x55f2b82b31b0 2024-09-21T07:29:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd3587d5740 10 client.20232 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd3587d5740 10 client.20232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd3587d5740 20 client.20232 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b826fda0 0x55f2b870dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd3587d5740 4 client.20232 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.044+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2b85f79b0 con 0x55f2b826fda0 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.045+0000 7fd34a000640 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b826fda0 0x55f2b870dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.045+0000 7fd34a000640 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b826fda0 0x55f2b870dc60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19a21177d7c9d0a2 server_cookie=25630156784e5027 in_seq=0 out_seq=0 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.045+0000 7fd334c00640 10 client.20232 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.045+0000 7fd334c00640 1 -- 172.21.15.27:0/1145379613 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2b85f79b0 con 0x55f2b826fda0 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.045+0000 7fd334c00640 10 client.20232 handle_command_reply: tid=0 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.045+0000 7fd3587d5740 1 client.20232 shutdown 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82b31b0 msgr2=0x55f2b8697420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82b31b0 0x55f2b8697420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005e00 tx=0x7fd33c012b40 comp rx=0 tx=0).stop 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 shutdown_connections 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x55f2b826fda0 0x55f2b870dc60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2b82bb390 0x55f2b8692950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f2b82dc070 0x55f2b8693060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 --2- 172.21.15.27:0/1145379613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f2b82b31b0 0x55f2b8697420 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 >> 172.21.15.27:0/1145379613 conn(0x55f2b82d0460 msgr2=0x55f2b8684630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd334c00640 0 client.20232 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 shutdown_connections 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 1 -- 172.21.15.27:0/1145379613 wait complete. 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.046+0000 7fd3587d5740 20 client.20232 trim_cache size 0 max 16384 2024-09-21T07:29:14.152 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:29:14.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3300b6680 msgr2=0x7fd3300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3300b6680 0x7fd3300b8b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd344006010 tx=0x7fd344005bd0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 msgr2=0x7fd34c19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:14.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 0x7fd34c19cad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd33802a910 tx=0x7fd338031a20 comp rx=0 tx=0).stop 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 shutdown_connections 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd3300b6680 0x7fd3300b8b90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c10be40 0x7fd34c1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd34c10b4f0 0x7fd34c19cad0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 --2- 172.21.15.27:0/771735308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34c105bb0 0x7fd34c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 >> 172.21.15.27:0/771735308 conn(0x7fd34c101680 msgr2=0x7fd34c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 shutdown_connections 2024-09-21T07:29:14.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:14.047+0000 7fd34be00640 1 -- 172.21.15.27:0/771735308 wait complete. 2024-09-21T07:29:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:14 smithi027 ceph-mon[30779]: pgmap v886: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-21T07:29:15.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:14 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:29:15.156 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:29:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:14 smithi153 ceph-mon[34747]: pgmap v886: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-21T07:29:15.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:14 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:29:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:14 smithi119 ceph-mon[34519]: pgmap v886: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-21T07:29:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:14 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:29:15.585 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:16.051 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.051 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.051 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.051 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.051 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: Unable to set osd_memory_target on smithi153 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:29:16.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:15 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:29:16.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:29:16.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: Unable to set osd_memory_target on smithi153 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-21T07:29:16.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:29:16.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:29:16.179 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:15 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 -- 172.21.15.27:0/2662514693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780075dd0 msgr2=0x7fb7800761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 --2- 172.21.15.27:0/2662514693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780075dd0 0x7fb7800761e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009860 tx=0x7fb76c02a340 comp rx=0 tx=0).stop 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 -- 172.21.15.27:0/2662514693 shutdown_connections 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 --2- 172.21.15.27:0/2662514693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb780114280 0x7fb780116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 --2- 172.21.15.27:0/2662514693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780075dd0 0x7fb7800761e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 --2- 172.21.15.27:0/2662514693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb780077380 0x7fb7800757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 -- 172.21.15.27:0/2662514693 >> 172.21.15.27:0/2662514693 conn(0x7fb7800ff440 msgr2=0x7fb780101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.233+0000 7fb785e00640 1 -- 172.21.15.27:0/2662514693 shutdown_connections 2024-09-21T07:29:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.234+0000 7fb785e00640 1 -- 172.21.15.27:0/2662514693 wait complete. 2024-09-21T07:29:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 Processor -- start 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 -- start start 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb780075dd0 0x7fb7801a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780077380 0x7fb7801a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb780114280 0x7fb7801a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb780119ec0 con 0x7fb780075dd0 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb780119d40 con 0x7fb780077380 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.235+0000 7fb785e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb78011a040 con 0x7fb780114280 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb784a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb780075dd0 0x7fb7801a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb77fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780077380 0x7fb7801a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb784a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb780075dd0 0x7fb7801a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56008/0 (socket says 172.21.15.27:56008) 2024-09-21T07:29:16.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb784a00640 1 -- 172.21.15.27:0/2062911737 learned_addr learned my addr 172.21.15.27:0/2062911737 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:16.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb785400640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb780114280 0x7fb7801a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb77fe00640 1 -- 172.21.15.27:0/2062911737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb780114280 msgr2=0x7fb7801a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb780114280 0x7fb7801a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb77fe00640 1 -- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb780075dd0 msgr2=0x7fb7801a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb780075dd0 0x7fb7801a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.236+0000 7fb77fe00640 1 -- 172.21.15.27:0/2062911737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7801a6860 con 0x7fb780077380 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.237+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780077380 0x7fb7801a0e90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02a860 tx=0x7fb76c032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.237+0000 7fb77d600640 1 -- 172.21.15.27:0/2062911737 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c037040 con 0x7fb780077380 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.237+0000 7fb77d600640 1 -- 172.21.15.27:0/2062911737 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb76c030c10 con 0x7fb780077380 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.237+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7801a6af0 con 0x7fb780077380 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.237+0000 7fb77d600640 1 -- 172.21.15.27:0/2062911737 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0035f0 con 0x7fb780077380 2024-09-21T07:29:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.237+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7801ad2b0 con 0x7fb780077380 2024-09-21T07:29:16.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.240+0000 7fb78e20e740 1 Processor -- start 2024-09-21T07:29:16.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.240+0000 7fb78e20e740 1 -- start start 2024-09-21T07:29:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429b1d070 0x55c429ec3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ec44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429ec49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb78e20e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c429e3afb0 con 0x55c429af41b0 2024-09-21T07:29:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb78e20e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c429e2a3e0 con 0x55c429afc390 2024-09-21T07:29:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb78e20e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c429e2e1f0 con 0x55c429b1d070 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb784a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ec44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb784a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ec44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56018/0 (socket says 172.21.15.27:56018) 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb784a00640 1 -- 172.21.15.27:0/3493932313 learned_addr learned my addr 172.21.15.27:0/3493932313 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb77d600640 1 -- 172.21.15.27:0/2062911737 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb76c043020 con 0x7fb780077380 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.241+0000 7fb785400640 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429b1d070 0x55c429ec3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb785400640 1 -- 172.21.15.27:0/3493932313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 msgr2=0x55c429ec49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb77fe00640 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429ec49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb785400640 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429ec49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb785400640 1 -- 172.21.15.27:0/3493932313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 msgr2=0x55c429ec44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb785400640 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ec44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb785400640 1 -- 172.21.15.27:0/3493932313 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c429df4cd0 con 0x55c429b1d070 2024-09-21T07:29:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb784a00640 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ec44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb77fe00640 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429ec49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:29:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb785400640 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429b1d070 0x55c429ec3f70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7780099b0 tx=0x7fb7780095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3bb57ed322866a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.242+0000 7fb771c00640 1 -- 172.21.15.27:0/3493932313 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778003970 con 0x55c429b1d070 2024-09-21T07:29:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.243+0000 7fb771c00640 1 -- 172.21.15.27:0/3493932313 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb77800d6b0 con 0x55c429b1d070 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.243+0000 7fb771c00640 1 -- 172.21.15.27:0/3493932313 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778004280 con 0x55c429b1d070 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.243+0000 7fb78e20e740 1 -- 172.21.15.27:0/3493932313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429b1d070 msgr2=0x55c429ec3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.243+0000 7fb78e20e740 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429b1d070 0x55c429ec3f70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7780099b0 tx=0x7fb7780095c0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 -- 172.21.15.27:0/3493932313 shutdown_connections 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429ec49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ec44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 --2- 172.21.15.27:0/3493932313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429b1d070 0x55c429ec3f70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 -- 172.21.15.27:0/3493932313 >> 172.21.15.27:0/3493932313 conn(0x55c429b11460 msgr2=0x55c429ec1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 -- 172.21.15.27:0/3493932313 shutdown_connections 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 -- 172.21.15.27:0/3493932313 wait complete. 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb77d600640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7680b6610 0x7fb7680b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb77d600640 1 -- 172.21.15.27:0/2062911737 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fb76c106bc0 con 0x7fb780077380 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb784a00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7680b6610 0x7fb7680b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.244+0000 7fb78e20e740 1 -- start start 2024-09-21T07:29:16.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb78e20e740 1 Processor -- start 2024-09-21T07:29:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb784a00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7680b6610 0x7fb7680b8b20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097e0 tx=0x7fb774005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ecf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ecfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429b1d070 0x55c429edba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb78e20e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c429c8b700 con 0x55c429af41b0 2024-09-21T07:29:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb78e20e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c429dc5580 con 0x55c429b1d070 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.245+0000 7fb78e20e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c429ed2750 con 0x55c429afc390 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb78e20e740 10 client.0 _ll_register_callbacks cb 0x55c429dd5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb784a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ecf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb784a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ecf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56034/0 (socket says 172.21.15.27:56034) 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb784a00640 1 -- 172.21.15.27:0/2385021064 learned_addr learned my addr 172.21.15.27:0/2385021064 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ecfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb785400640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429b1d070 0x55c429edba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb77fe00640 1 -- 172.21.15.27:0/2385021064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429b1d070 msgr2=0x55c429edba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429b1d070 0x55c429edba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb77fe00640 1 -- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 msgr2=0x55c429ecf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ecf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb77fe00640 1 -- 172.21.15.27:0/2385021064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c429b88650 con 0x55c429afc390 2024-09-21T07:29:16.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.246+0000 7fb784a00640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ecf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ecfc00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb76c103f00 tx=0x7fb76c0cc440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503fe3a063f446cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb766000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb766000640 1 -- 172.21.15.27:0/2385021064 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c030450 con 0x55c429afc390 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c429dc7b90 con 0x55c429afc390 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb766000640 1 -- 172.21.15.27:0/2385021064 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fb76c0cb860 con 0x55c429afc390 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb766000640 1 -- 172.21.15.27:0/2385021064 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb76c0cc110 con 0x55c429afc390 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb78e20e740 10 client.30217 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.247+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c429de2950 con 0x55c429afc390 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb766000640 1 -- 172.21.15.27:0/2385021064 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c429dc7b90 con 0x55c429afc390 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb766000640 1 -- 172.21.15.27:0/2385021064 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb76c0cb2d0 con 0x55c429afc390 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb78e20e740 10 client.30217 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb78e20e740 10 client.30217 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb78e20e740 20 client.30217 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429adb9f0 0x55c429ee1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb78e20e740 4 client.30217 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c429e3cf00 con 0x55c429adb9f0 2024-09-21T07:29:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.248+0000 7fb784a00640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429adb9f0 0x55c429ee1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.249+0000 7fb784a00640 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429adb9f0 0x55c429ee1700 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3632a2ff140c559a server_cookie=c497a5986d960014 in_seq=0 out_seq=0 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.249+0000 7fb766000640 10 client.30217 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb766000640 1 -- 172.21.15.27:0/2385021064 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c429e3cf00 con 0x55c429adb9f0 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb766000640 10 client.30217 handle_command_reply: tid=0 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 client.30217 shutdown 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 msgr2=0x55c429ecfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ecfc00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb76c103f00 tx=0x7fb76c0cc440 comp rx=0 tx=0).stop 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 shutdown_connections 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429adb9f0 0x55c429ee1700 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ecfc00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429b1d070 0x55c429edba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2385021064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429af41b0 0x55c429ecf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 >> 172.21.15.27:0/2385021064 conn(0x55c429b11460 msgr2=0x55c429ec7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb766000640 0 client.30217 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:29:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 shutdown_connections 2024-09-21T07:29:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 1 -- 172.21.15.27:0/2385021064 wait complete. 2024-09-21T07:29:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.250+0000 7fb78e20e740 20 client.30217 trim_cache size 0 max 16384 2024-09-21T07:29:16.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 Processor -- start 2024-09-21T07:29:16.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 -- start start 2024-09-21T07:29:16.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429af41b0 0x55c429ed12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429edb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ecc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801b9b00 con 0x55c429b1d070 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801b73d0 con 0x55c429afc390 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb78e20e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801b5240 con 0x55c429af41b0 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.265+0000 7fb785400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ecc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb784a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429af41b0 0x55c429ed12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb785400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ecc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56048/0 (socket says 172.21.15.27:56048) 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb77fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429edb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb785400640 1 -- 172.21.15.27:0/2660399386 learned_addr learned my addr 172.21.15.27:0/2660399386 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb784a00640 1 -- 172.21.15.27:0/2660399386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 msgr2=0x55c429edb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb784a00640 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429edb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb784a00640 1 -- 172.21.15.27:0/2660399386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 msgr2=0x55c429ecc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb784a00640 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ecc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb784a00640 1 -- 172.21.15.27:0/2660399386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c429ec6800 con 0x55c429af41b0 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb785400640 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ecc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb77fe00640 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429edb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.266+0000 7fb784a00640 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429af41b0 0x55c429ed12f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55c429ee1e00 tx=0x7fb77401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=143b52a9b473398c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:16.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.267+0000 7fb771200640 1 -- 172.21.15.27:0/2660399386 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774005020 con 0x55c429af41b0 2024-09-21T07:29:16.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.267+0000 7fb771200640 1 -- 172.21.15.27:0/2660399386 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb77400f810 con 0x55c429af41b0 2024-09-21T07:29:16.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.267+0000 7fb771200640 1 -- 172.21.15.27:0/2660399386 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774004a00 con 0x55c429af41b0 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 -- 172.21.15.27:0/2660399386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429af41b0 msgr2=0x55c429ed12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429af41b0 0x55c429ed12f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55c429ee1e00 tx=0x7fb77401b000 comp rx=0 tx=0).stop 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 -- 172.21.15.27:0/2660399386 shutdown_connections 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ecc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429afc390 0x55c429edb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 --2- 172.21.15.27:0/2660399386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429af41b0 0x55c429ed12f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 -- 172.21.15.27:0/2660399386 >> 172.21.15.27:0/2660399386 conn(0x55c429b11460 msgr2=0x55c429ed0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 -- 172.21.15.27:0/2660399386 shutdown_connections 2024-09-21T07:29:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 -- 172.21.15.27:0/2660399386 wait complete. 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.268+0000 7fb78e20e740 1 -- start start 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 1 Processor -- start 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429af41b0 0x55c429ed8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ed2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ed2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801b5890 con 0x55c429b1d070 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801b6d00 con 0x55c429af41b0 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801b5f60 con 0x55c429afc390 2024-09-21T07:29:16.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.269+0000 7fb78e20e740 10 client.0 _ll_register_callbacks cb 0x7fb76c0c8530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ed2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ed2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56064/0 (socket says 172.21.15.27:56064) 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 -- 172.21.15.27:0/4062864551 learned_addr learned my addr 172.21.15.27:0/4062864551 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb77fe00640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ed2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb784a00640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429af41b0 0x55c429ed8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 -- 172.21.15.27:0/4062864551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 msgr2=0x55c429ed2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ed2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 -- 172.21.15.27:0/4062864551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429af41b0 msgr2=0x55c429ed8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429af41b0 0x55c429ed8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 -- 172.21.15.27:0/4062864551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c429ed34b0 con 0x55c429b1d070 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb784a00640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429af41b0 0x55c429ed8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb785400640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ed2ea0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb7780099b0 tx=0x7fb778009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe91bb9829231ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.270+0000 7fb766000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.271+0000 7fb766000640 1 -- 172.21.15.27:0/4062864551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778030060 con 0x55c429b1d070 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.271+0000 7fb766000640 1 -- 172.21.15.27:0/4062864551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fb77801c040 con 0x55c429b1d070 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.271+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77400f810 con 0x55c429b1d070 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.271+0000 7fb766000640 1 -- 172.21.15.27:0/4062864551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb77800d6b0 con 0x55c429b1d070 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.271+0000 7fb78e20e740 10 client.20250 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.271+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c429f4ce90 con 0x55c429b1d070 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.271+0000 7fb766000640 1 -- 172.21.15.27:0/4062864551 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb77400f810 con 0x55c429b1d070 2024-09-21T07:29:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb766000640 1 -- 172.21.15.27:0/4062864551 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb77800b040 con 0x55c429b1d070 2024-09-21T07:29:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb78e20e740 10 client.20250 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb78e20e740 10 client.20250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:29:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb78e20e740 20 client.20250 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb78e20e740 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429ab0da0 0x55c429f4f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb78e20e740 4 client.20250 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:29:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c429e389b0 con 0x55c429ab0da0 2024-09-21T07:29:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.272+0000 7fb784a00640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429ab0da0 0x55c429f4f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb784a00640 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429ab0da0 0x55c429f4f370 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d2ad5a279b5a0b2 server_cookie=1f97b492c341cd8d in_seq=0 out_seq=0 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb766000640 10 client.20250 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb766000640 1 -- 172.21.15.27:0/4062864551 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c429e389b0 con 0x55c429ab0da0 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb766000640 10 client.20250 handle_command_reply: tid=0 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 client.20250 shutdown 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 msgr2=0x55c429ed2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ed2ea0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb7780099b0 tx=0x7fb778009de0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 shutdown_connections 2024-09-21T07:29:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55c429ab0da0 0x55c429f4f370 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c429afc390 0x55c429ed2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c429b1d070 0x55c429ed2ea0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 --2- 172.21.15.27:0/4062864551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c429af41b0 0x55c429ed8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 >> 172.21.15.27:0/4062864551 conn(0x55c429b11460 msgr2=0x55c429ed1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.273+0000 7fb766000640 0 client.20250 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.274+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 shutdown_connections 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.274+0000 7fb78e20e740 1 -- 172.21.15.27:0/4062864551 wait complete. 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.274+0000 7fb78e20e740 20 client.20250 trim_cache size 0 max 16384 2024-09-21T07:29:16.278 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:29:16.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.274+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7680b6610 msgr2=0x7fb7680b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.274+0000 7fb785e00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7680b6610 0x7fb7680b8b20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097e0 tx=0x7fb774005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:16.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.274+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780077380 msgr2=0x7fb7801a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:16.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.274+0000 7fb785e00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780077380 0x7fb7801a0e90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02a860 tx=0x7fb76c032d50 comp rx=0 tx=0).stop 2024-09-21T07:29:16.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 shutdown_connections 2024-09-21T07:29:16.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb780114280 0x7fb7801a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb7680b6610 0x7fb7680b8b20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb780077380 0x7fb7801a0e90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 --2- 172.21.15.27:0/2062911737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb780075dd0 0x7fb7801a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 >> 172.21.15.27:0/2062911737 conn(0x7fb7800ff440 msgr2=0x7fb780110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 shutdown_connections 2024-09-21T07:29:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:16.275+0000 7fb785e00640 1 -- 172.21.15.27:0/2062911737 wait complete. 2024-09-21T07:29:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: Unable to set osd_memory_target on smithi153 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:29:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:29:16.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:29:16.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:15 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:29:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:16 smithi027 ceph-mon[30779]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T07:29:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:16 smithi027 ceph-mon[30779]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T07:29:17.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:16 smithi027 ceph-mon[30779]: pgmap v887: 129 pgs: 129 active+clean; 1.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:29:17.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:29:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:16 smithi153 ceph-mon[34747]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T07:29:17.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:16 smithi153 ceph-mon[34747]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T07:29:17.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:16 smithi153 ceph-mon[34747]: pgmap v887: 129 pgs: 129 active+clean; 1.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:29:17.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:16 smithi119 ceph-mon[34519]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T07:29:17.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:16 smithi119 ceph-mon[34519]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T07:29:17.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:16 smithi119 ceph-mon[34519]: pgmap v887: 129 pgs: 129 active+clean; 1.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-21T07:29:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 -- 172.21.15.27:0/1991045519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694810c010 msgr2=0x7f6948112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 --2- 172.21.15.27:0/1991045519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694810c010 0x7f6948112840 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f6940009ca0 tx=0x7f694002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 -- 172.21.15.27:0/1991045519 shutdown_connections 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 --2- 172.21.15.27:0/1991045519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694810b6c0 0x7f694810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 --2- 172.21.15.27:0/1991045519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694810c010 0x7f6948112840 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 --2- 172.21.15.27:0/1991045519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6948105d80 0x7f6948106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 -- 172.21.15.27:0/1991045519 >> 172.21.15.27:0/1991045519 conn(0x7f6948101810 msgr2=0x7f6948103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 -- 172.21.15.27:0/1991045519 shutdown_connections 2024-09-21T07:29:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.161+0000 7f694da00640 1 -- 172.21.15.27:0/1991045519 wait complete. 2024-09-21T07:29:18.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 Processor -- start 2024-09-21T07:29:18.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 -- start start 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6948105d80 0x7f694819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694810b6c0 0x7f694819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694810c010 0x7f69481a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6948115cd0 con 0x7f6948105d80 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6948115b50 con 0x7f694810b6c0 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f694da00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6948115e50 con 0x7f694810c010 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.163+0000 7f6946a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6948105d80 0x7f694819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6948105d80 0x7f694819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56800/0 (socket says 172.21.15.27:56800) 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 -- 172.21.15.27:0/2814272266 learned_addr learned my addr 172.21.15.27:0/2814272266 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f693fe00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694810b6c0 0x7f694819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6947400640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694810c010 0x7f69481a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 -- 172.21.15.27:0/2814272266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694810c010 msgr2=0x7f69481a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694810c010 0x7f69481a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 -- 172.21.15.27:0/2814272266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694810b6c0 msgr2=0x7f694819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694810b6c0 0x7f694819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 -- 172.21.15.27:0/2814272266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69481a2690 con 0x7f6948105d80 2024-09-21T07:29:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.164+0000 7f6946a00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6948105d80 0x7f694819c720 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f693800e9f0 tx=0x7f693800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.165+0000 7f693f400640 1 -- 172.21.15.27:0/2814272266 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693800cd80 con 0x7f6948105d80 2024-09-21T07:29:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.165+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69481a2980 con 0x7f6948105d80 2024-09-21T07:29:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.165+0000 7f693f400640 1 -- 172.21.15.27:0/2814272266 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69380108e0 con 0x7f6948105d80 2024-09-21T07:29:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.165+0000 7f693f400640 1 -- 172.21.15.27:0/2814272266 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938018980 con 0x7f6948105d80 2024-09-21T07:29:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.165+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69481a90e0 con 0x7f6948105d80 2024-09-21T07:29:18.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.168+0000 7f6954684740 1 Processor -- start 2024-09-21T07:29:18.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.168+0000 7f6954684740 1 -- start start 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6954684740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6954684740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c5a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6954684740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c5a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6954684740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b574bd0fb0 con 0x55b5748b3070 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6954684740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b574bc03e0 con 0x55b57488a1b0 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6954684740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b574bc41f0 con 0x55b574892390 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6947400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6947400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c59da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56810/0 (socket says 172.21.15.27:56810) 2024-09-21T07:29:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f693f400640 1 -- 172.21.15.27:0/2814272266 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6938010430 con 0x7f6948105d80 2024-09-21T07:29:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6947400640 1 -- 172.21.15.27:0/1834295049 learned_addr learned my addr 172.21.15.27:0/1834295049 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f693fe00640 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c5a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.169+0000 7f6947400640 1 -- 172.21.15.27:0/1834295049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 msgr2=0x55b574c5a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6947400640 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c5a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6947400640 1 -- 172.21.15.27:0/1834295049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 msgr2=0x55b574c5a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6947400640 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c5a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6947400640 1 -- 172.21.15.27:0/1834295049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b574b8acd0 con 0x55b5748b3070 2024-09-21T07:29:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6947400640 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c59da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6940005cc0 tx=0x7f694002f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a0670bf51ba48d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6937e00640 1 -- 172.21.15.27:0/1834295049 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940031cf0 con 0x55b5748b3070 2024-09-21T07:29:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6937e00640 1 -- 172.21.15.27:0/1834295049 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6940036040 con 0x55b5748b3070 2024-09-21T07:29:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.170+0000 7f6937e00640 1 -- 172.21.15.27:0/1834295049 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940004040 con 0x55b5748b3070 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f693f400640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f693f400640 1 -- 172.21.15.27:0/2814272266 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f69380e7e90 con 0x7f6948105d80 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f6954684740 1 -- 172.21.15.27:0/1834295049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 msgr2=0x55b574c59da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f6954684740 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c59da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6940005cc0 tx=0x7f694002f420 comp rx=0 tx=0).stop 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f6946a00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f6954684740 1 -- 172.21.15.27:0/1834295049 shutdown_connections 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f6954684740 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c5a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.171+0000 7f6954684740 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c5a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.172+0000 7f6954684740 1 --2- 172.21.15.27:0/1834295049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c59da0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.172+0000 7f6954684740 1 -- 172.21.15.27:0/1834295049 >> 172.21.15.27:0/1834295049 conn(0x55b5748a7460 msgr2=0x55b574c57c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:18.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.172+0000 7f6954684740 1 -- 172.21.15.27:0/1834295049 shutdown_connections 2024-09-21T07:29:18.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.172+0000 7f6954684740 1 -- 172.21.15.27:0/1834295049 wait complete. 2024-09-21T07:29:18.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.172+0000 7f6946a00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f692c0b6700 0x7f692c0b8c10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55b574c58a20 tx=0x7f693800cf20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.172+0000 7f6954684740 1 -- start start 2024-09-21T07:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 1 Processor -- start 2024-09-21T07:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b574892390 0x55b574c65ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5748b3070 0x55b574c71840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b574a21700 con 0x55b574892390 2024-09-21T07:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b574b5b580 con 0x55b57488a1b0 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b574c68520 con 0x55b5748b3070 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.173+0000 7f6954684740 10 client.0 _ll_register_callbacks cb 0x55b574b6b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f693fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b574892390 0x55b574c65ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58802/0 (socket says 172.21.15.27:58802) 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 -- 172.21.15.27:0/2692642575 learned_addr learned my addr 172.21.15.27:0/2692642575 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5748b3070 0x55b574c71840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 -- 172.21.15.27:0/2692642575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5748b3070 msgr2=0x55b574c71840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5748b3070 0x55b574c71840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 -- 172.21.15.27:0/2692642575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b574892390 msgr2=0x55b574c65ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b574892390 0x55b574c65ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.174+0000 7f6947400640 1 -- 172.21.15.27:0/2692642575 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57491e650 con 0x55b57488a1b0 2024-09-21T07:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6947400640 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c655a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f69400096c0 tx=0x7f6940003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf5355f3f31585f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6934c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6934c00640 1 -- 172.21.15.27:0/2692642575 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940058060 con 0x55b57488a1b0 2024-09-21T07:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b574b5db90 con 0x55b57488a1b0 2024-09-21T07:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6934c00640 1 -- 172.21.15.27:0/2692642575 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f6940031cf0 con 0x55b57488a1b0 2024-09-21T07:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6934c00640 1 -- 172.21.15.27:0/2692642575 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f694004a040 con 0x55b57488a1b0 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6954684740 10 client.30272 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.175+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b574b78950 con 0x55b57488a1b0 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6934c00640 1 -- 172.21.15.27:0/2692642575 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b574b5db90 con 0x55b57488a1b0 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6934c00640 1 -- 172.21.15.27:0/2692642575 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6940050070 con 0x55b57488a1b0 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6954684740 10 client.30272 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6954684740 10 client.30272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6954684740 20 client.30272 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6954684740 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b5748719f0 0x55b574c774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6954684740 4 client.30272 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b574bd2f00 con 0x55b5748719f0 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.176+0000 7f693fe00640 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b5748719f0 0x55b574c774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.177+0000 7f693fe00640 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b5748719f0 0x55b574c774e0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80bfdead666b349d server_cookie=4e972d1d93ca7a3d in_seq=0 out_seq=0 2024-09-21T07:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.177+0000 7f6934c00640 10 client.30272 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6934c00640 1 -- 172.21.15.27:0/2692642575 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b574bd2f00 con 0x55b5748719f0 2024-09-21T07:29:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6934c00640 10 client.30272 handle_command_reply: tid=0 2024-09-21T07:29:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 client.30272 shutdown 2024-09-21T07:29:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 msgr2=0x55b574c655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c655a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f69400096c0 tx=0x7f6940003c30 comp rx=0 tx=0).stop 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 shutdown_connections 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b5748719f0 0x55b574c774e0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b5748b3070 0x55b574c71840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b574892390 0x55b574c65ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 --2- 172.21.15.27:0/2692642575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c655a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6934c00640 0 client.30272 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 >> 172.21.15.27:0/2692642575 conn(0x55b5748a7460 msgr2=0x55b574c5cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 shutdown_connections 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 1 -- 172.21.15.27:0/2692642575 wait complete. 2024-09-21T07:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.178+0000 7f6954684740 20 client.30272 trim_cache size 0 max 16384 2024-09-21T07:29:18.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.192+0000 7f6954684740 1 Processor -- start 2024-09-21T07:29:18.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6954684740 1 -- start start 2024-09-21T07:29:18.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6954684740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6954684740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c62790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6954684740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6954684740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b5a90 con 0x55b5748b3070 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6954684740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b3360 con 0x55b57488a1b0 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6954684740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b11d0 con 0x55b574892390 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6947400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f693fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c62790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6947400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c62cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56820/0 (socket says 172.21.15.27:56820) 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.193+0000 7f6947400640 1 -- 172.21.15.27:0/225183186 learned_addr learned my addr 172.21.15.27:0/225183186 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f6947400640 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f693fe00640 1 -- 172.21.15.27:0/225183186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 msgr2=0x55b574c671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f693fe00640 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f693fe00640 1 -- 172.21.15.27:0/225183186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 msgr2=0x55b574c62cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f693fe00640 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c62cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f693fe00640 1 -- 172.21.15.27:0/225183186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b574c66860 con 0x55b574892390 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f6947400640 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c62cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.194+0000 7f693fe00640 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c62790 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55b574bc04c0 tx=0x7f6930005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bdc24a634653074 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.195+0000 7f6937400640 1 -- 172.21.15.27:0/225183186 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930005220 con 0x55b574892390 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.195+0000 7f6937400640 1 -- 172.21.15.27:0/225183186 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6930004c50 con 0x55b574892390 2024-09-21T07:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.195+0000 7f6937400640 1 -- 172.21.15.27:0/225183186 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693001e9a0 con 0x55b574892390 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 -- 172.21.15.27:0/225183186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 msgr2=0x55b574c62790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c62790 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55b574bc04c0 tx=0x7f6930005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 -- 172.21.15.27:0/225183186 shutdown_connections 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c62790 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5748b3070 0x55b574c62cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 --2- 172.21.15.27:0/225183186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b57488a1b0 0x55b574c671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 -- 172.21.15.27:0/225183186 >> 172.21.15.27:0/225183186 conn(0x55b5748a7460 msgr2=0x55b574c66ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:18.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 -- 172.21.15.27:0/225183186 shutdown_connections 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 -- 172.21.15.27:0/225183186 wait complete. 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 -- start start 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.196+0000 7f6954684740 1 Processor -- start 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6954684740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b57488a1b0 0x55b574c7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6954684740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c78420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6954684740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5748b3070 0x55b574c78b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6954684740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b1820 con 0x55b57488a1b0 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6954684740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b2c90 con 0x55b5748b3070 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6954684740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b1ef0 con 0x55b574892390 2024-09-21T07:29:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6954684740 10 client.0 _ll_register_callbacks cb 0x7f6940003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:18.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6947400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b57488a1b0 0x55b574c7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6947400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b57488a1b0 0x55b574c7aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56822/0 (socket says 172.21.15.27:56822) 2024-09-21T07:29:18.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f6947400640 1 -- 172.21.15.27:0/21445198 learned_addr learned my addr 172.21.15.27:0/21445198 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.197+0000 7f693fe00640 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c78420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6947400640 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5748b3070 0x55b574c78b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6947400640 1 -- 172.21.15.27:0/21445198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 msgr2=0x55b574c78420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6947400640 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c78420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6947400640 1 -- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5748b3070 msgr2=0x55b574c78b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6947400640 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5748b3070 0x55b574c78b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6947400640 1 -- 172.21.15.27:0/21445198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b574ce0c60 con 0x55b57488a1b0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6947400640 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b57488a1b0 0x55b574c7aee0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f6940000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f935f88aa7cccaf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.198+0000 7f6934c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.199+0000 7f6934c00640 1 -- 172.21.15.27:0/21445198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940058030 con 0x55b57488a1b0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.199+0000 7f6934c00640 1 -- 172.21.15.27:0/21445198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f6940031d00 con 0x55b57488a1b0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.199+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6930004c50 con 0x55b57488a1b0 2024-09-21T07:29:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.199+0000 7f6934c00640 1 -- 172.21.15.27:0/21445198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6940009360 con 0x55b57488a1b0 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.199+0000 7f6954684740 10 client.20268 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.199+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b574ce1d90 con 0x55b57488a1b0 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.199+0000 7f6934c00640 1 -- 172.21.15.27:0/21445198 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6930004c50 con 0x55b57488a1b0 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f6934c00640 1 -- 172.21.15.27:0/21445198 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6940050070 con 0x55b57488a1b0 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f6954684740 10 client.20268 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f6954684740 10 client.20268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f6954684740 20 client.20268 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f6954684740 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b574846da0 0x55b574ce4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f6954684740 4 client.20268 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b574bce9b0 con 0x55b574846da0 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.200+0000 7f693fe00640 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b574846da0 0x55b574ce4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f693fe00640 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b574846da0 0x55b574ce4550 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd13559e2c9ed1 server_cookie=e4ecc4fc5541a1cc in_seq=0 out_seq=0 2024-09-21T07:29:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6934c00640 10 client.20268 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:18.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6934c00640 1 -- 172.21.15.27:0/21445198 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b574bce9b0 con 0x55b574846da0 2024-09-21T07:29:18.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6934c00640 10 client.20268 handle_command_reply: tid=0 2024-09-21T07:29:18.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 client.20268 shutdown 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b57488a1b0 msgr2=0x55b574c7aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b57488a1b0 0x55b574c7aee0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f6940000f40 comp rx=0 tx=0).stop 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 shutdown_connections 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55b574846da0 0x55b574ce4550 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b574892390 0x55b574c78420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5748b3070 0x55b574c78b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 --2- 172.21.15.27:0/21445198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b57488a1b0 0x55b574c7aee0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 >> 172.21.15.27:0/21445198 conn(0x55b5748a7460 msgr2=0x55b574c647e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6934c00640 0 client.20268 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.201+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 shutdown_connections 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f6954684740 1 -- 172.21.15.27:0/21445198 wait complete. 2024-09-21T07:29:18.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f6954684740 20 client.20268 trim_cache size 0 max 16384 2024-09-21T07:29:18.207 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:29:18.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f692c0b6700 msgr2=0x7f692c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f692c0b6700 0x7f692c0b8c10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55b574c58a20 tx=0x7f693800cf20 comp rx=0 tx=0).stop 2024-09-21T07:29:18.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6948105d80 msgr2=0x7f694819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:18.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6948105d80 0x7f694819c720 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f693800e9f0 tx=0x7f693800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 shutdown_connections 2024-09-21T07:29:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f694810c010 0x7f69481a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694810b6c0 0x7f694819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 --2- 172.21.15.27:0/2814272266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6948105d80 0x7f694819c720 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.202+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 >> 172.21.15.27:0/2814272266 conn(0x7f6948101810 msgr2=0x7f6948109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.203+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 shutdown_connections 2024-09-21T07:29:18.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:18.203+0000 7f694da00640 1 -- 172.21.15.27:0/2814272266 wait complete. 2024-09-21T07:29:19.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:18 smithi027 ceph-mon[30779]: pgmap v888: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-21T07:29:19.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:18 smithi153 ceph-mon[34747]: pgmap v888: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-21T07:29:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:18 smithi119 ceph-mon[34519]: pgmap v888: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-21T07:29:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 -- 172.21.15.27:0/3272339314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24103910 msgr2=0x7f9d24103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 --2- 172.21.15.27:0/3272339314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24103910 0x7f9d24103d20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d1802a410 comp rx=0 tx=0).stop 2024-09-21T07:29:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 -- 172.21.15.27:0/3272339314 shutdown_connections 2024-09-21T07:29:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 --2- 172.21.15.27:0/3272339314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d24109b90 0x7f9d241103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 --2- 172.21.15.27:0/3272339314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d24109240 0x7f9d24109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 --2- 172.21.15.27:0/3272339314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24103910 0x7f9d24103d20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 -- 172.21.15.27:0/3272339314 >> 172.21.15.27:0/3272339314 conn(0x7f9d240ff330 msgr2=0x7f9d24101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 -- 172.21.15.27:0/3272339314 shutdown_connections 2024-09-21T07:29:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.620+0000 7f9d2b000640 1 -- 172.21.15.27:0/3272339314 wait complete. 2024-09-21T07:29:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.621+0000 7f9d2b000640 1 Processor -- start 2024-09-21T07:29:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.621+0000 7f9d2b000640 1 -- start start 2024-09-21T07:29:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d2b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d24103910 0x7f9d24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d2b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24109240 0x7f9d24075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d2b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d24109b90 0x7f9d24076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d2b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241138a0 con 0x7f9d24109240 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d2b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24113720 con 0x7f9d24109b90 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d2b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24113a20 con 0x7f9d24103910 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d29200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24109240 0x7f9d24075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d29200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24109240 0x7f9d24075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56840/0 (socket says 172.21.15.27:56840) 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d29200640 1 -- 172.21.15.27:0/1425977938 learned_addr learned my addr 172.21.15.27:0/1425977938 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d2a600640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d24109b90 0x7f9d24076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.622+0000 7f9d29c00640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d24103910 0x7f9d24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d29200640 1 -- 172.21.15.27:0/1425977938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d24103910 msgr2=0x7f9d24075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d29200640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d24103910 0x7f9d24075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d29200640 1 -- 172.21.15.27:0/1425977938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d24109b90 msgr2=0x7f9d24076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d29200640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d24109b90 0x7f9d24076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d29200640 1 -- 172.21.15.27:0/1425977938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d241a8550 con 0x7f9d24109240 2024-09-21T07:29:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d29200640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24109240 0x7f9d24075c50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000e9f0 tx=0x7f9d1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d22000640 1 -- 172.21.15.27:0/1425977938 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000ccc0 con 0x7f9d24109240 2024-09-21T07:29:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d241a8840 con 0x7f9d24109240 2024-09-21T07:29:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d22000640 1 -- 172.21.15.27:0/1425977938 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d100108e0 con 0x7f9d24109240 2024-09-21T07:29:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d241a8d60 con 0x7f9d24109240 2024-09-21T07:29:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.623+0000 7f9d22000640 1 -- 172.21.15.27:0/1425977938 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10018800 con 0x7f9d24109240 2024-09-21T07:29:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 Processor -- start 2024-09-21T07:29:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 -- start start 2024-09-21T07:29:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d22000640 1 -- 172.21.15.27:0/1425977938 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9d10010430 con 0x7f9d24109240 2024-09-21T07:29:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625049221b0 0x562504cc6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625048f92f0 0x562504cc7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049014d0 0x562504cc7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562504c3c6b0 con 0x5625049221b0 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562504c2bae0 con 0x5625049014d0 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d3368a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562504c2f8f0 con 0x5625048f92f0 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d2a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625049221b0 0x562504cc6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d2a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625049221b0 0x562504cc6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56848/0 (socket says 172.21.15.27:56848) 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d2a600640 1 -- 172.21.15.27:0/2637101954 learned_addr learned my addr 172.21.15.27:0/2637101954 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.627+0000 7f9d29c00640 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625048f92f0 0x562504cc7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d29200640 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049014d0 0x562504cc7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d2a600640 1 -- 172.21.15.27:0/2637101954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625048f92f0 msgr2=0x562504cc7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d2a600640 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625048f92f0 0x562504cc7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d2a600640 1 -- 172.21.15.27:0/2637101954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049014d0 msgr2=0x562504cc7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d2a600640 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049014d0 0x562504cc7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d2a600640 1 -- 172.21.15.27:0/2637101954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562504bf8780 con 0x5625049221b0 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d2a600640 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625049221b0 0x562504cc6e10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196f7b53e655fe06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.628+0000 7f9d16a00640 1 -- 172.21.15.27:0/2637101954 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c004d20 con 0x5625049221b0 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d16a00640 1 -- 172.21.15.27:0/2637101954 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d1c004050 con 0x5625049221b0 2024-09-21T07:29:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d16a00640 1 -- 172.21.15.27:0/2637101954 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c005220 con 0x5625049221b0 2024-09-21T07:29:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d22000640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d0c0b66b0 0x7f9d0c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d22000640 1 -- 172.21.15.27:0/1425977938 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f9d100e7d60 con 0x7f9d24109240 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d29c00640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d0c0b66b0 0x7f9d0c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d3368a740 1 -- 172.21.15.27:0/2637101954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625049221b0 msgr2=0x562504cc6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625049221b0 0x562504cc6e10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c0118d0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d3368a740 1 -- 172.21.15.27:0/2637101954 shutdown_connections 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.629+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625048f92f0 0x562504cc7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.630+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049014d0 0x562504cc7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.630+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2637101954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625049221b0 0x562504cc6e10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.630+0000 7f9d3368a740 1 -- 172.21.15.27:0/2637101954 >> 172.21.15.27:0/2637101954 conn(0x562504916300 msgr2=0x562504cc4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.630+0000 7f9d3368a740 1 -- 172.21.15.27:0/2637101954 shutdown_connections 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.630+0000 7f9d3368a740 1 -- 172.21.15.27:0/2637101954 wait complete. 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.630+0000 7f9d29c00640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d0c0b66b0 0x7f9d0c0b8bc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9d18005ab0 tx=0x7f9d18005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.630+0000 7f9d3368a740 1 -- start start 2024-09-21T07:29:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 1 Processor -- start 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562504b63ab0 con 0x5625048f92f0 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562504bd2570 con 0x5625049221b0 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562504cd5590 con 0x5625049014d0 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d3368a740 10 client.0 _ll_register_callbacks cb 0x562504be1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.631+0000 7f9d29c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56856/0 (socket says 172.21.15.27:56856) 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 -- 172.21.15.27:0/219281816 learned_addr learned my addr 172.21.15.27:0/219281816 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d2a600640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29200640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 -- 172.21.15.27:0/219281816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 msgr2=0x562504cd2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 -- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 msgr2=0x562504cd2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 -- 172.21.15.27:0/219281816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250498d790 con 0x5625048f92f0 2024-09-21T07:29:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d2a600640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d29c00640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd23c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d18009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154d3ec3af72c829 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.632+0000 7f9d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d0b400640 1 -- 172.21.15.27:0/219281816 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1804e070 con 0x5625048f92f0 2024-09-21T07:29:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d0b400640 1 -- 172.21.15.27:0/219281816 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f9d18004970 con 0x5625048f92f0 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d1c004050 con 0x5625048f92f0 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d0b400640 1 -- 172.21.15.27:0/219281816 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d1805a050 con 0x5625048f92f0 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d3368a740 10 client.20280 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562504a06620 con 0x5625048f92f0 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d0b400640 1 -- 172.21.15.27:0/219281816 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d1c004050 con 0x5625048f92f0 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.633+0000 7f9d0b400640 1 -- 172.21.15.27:0/219281816 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9d180324b0 con 0x5625048f92f0 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d3368a740 10 client.20280 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d3368a740 10 client.20280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d3368a740 20 client.20280 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d3368a740 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048e0d80 0x562504ce4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d3368a740 4 client.20280 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562504c3e600 con 0x5625048e0d80 2024-09-21T07:29:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d29200640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048e0d80 0x562504ce4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d29200640 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048e0d80 0x562504ce4530 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49703827a76983f server_cookie=721c93a89bd3a580 in_seq=0 out_seq=0 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.634+0000 7f9d0b400640 10 client.20280 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.635+0000 7f9d0b400640 1 -- 172.21.15.27:0/219281816 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562504c3e600 con 0x5625048e0d80 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.635+0000 7f9d0b400640 10 client.20280 handle_command_reply: tid=0 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.635+0000 7f9d3368a740 1 client.20280 shutdown 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.635+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 msgr2=0x562504cd23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.635+0000 7f9d3368a740 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd23c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d18009260 comp rx=0 tx=0).stop 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 shutdown_connections 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048e0d80 0x562504ce4530 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 --2- 172.21.15.27:0/219281816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd23c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 >> 172.21.15.27:0/219281816 conn(0x562504916300 msgr2=0x562504cc7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d0b400640 0 client.20280 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 shutdown_connections 2024-09-21T07:29:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 1 -- 172.21.15.27:0/219281816 wait complete. 2024-09-21T07:29:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.636+0000 7f9d3368a740 20 client.20280 trim_cache size 0 max 16384 2024-09-21T07:29:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.651+0000 7f9d3368a740 1 Processor -- start 2024-09-21T07:29:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.651+0000 7f9d3368a740 1 -- start start 2024-09-21T07:29:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.651+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.651+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cc3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504ce9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d3368a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241b58e0 con 0x5625048f92f0 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d3368a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241b31b0 con 0x5625049221b0 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d3368a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241b1020 con 0x5625049014d0 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56862/0 (socket says 172.21.15.27:56862) 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 -- 172.21.15.27:0/1447701995 learned_addr learned my addr 172.21.15.27:0/1447701995 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29200640 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cc3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d2a600640 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504ce9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 -- 172.21.15.27:0/1447701995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 msgr2=0x562504cc3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cc3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 -- 172.21.15.27:0/1447701995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 msgr2=0x562504ce9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504ce9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.652+0000 7f9d29c00640 1 -- 172.21.15.27:0/1447701995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562504cc9c20 con 0x5625048f92f0 2024-09-21T07:29:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.653+0000 7f9d29c00640 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd1290 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9d1802a930 tx=0x7f9d18004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27116845b3645ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.653+0000 7f9d16000640 1 -- 172.21.15.27:0/1447701995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1804e070 con 0x5625048f92f0 2024-09-21T07:29:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.653+0000 7f9d16000640 1 -- 172.21.15.27:0/1447701995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d180043a0 con 0x5625048f92f0 2024-09-21T07:29:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.653+0000 7f9d16000640 1 -- 172.21.15.27:0/1447701995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18035040 con 0x5625048f92f0 2024-09-21T07:29:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 -- 172.21.15.27:0/1447701995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 msgr2=0x562504cd1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd1290 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9d1802a930 tx=0x7f9d18004610 comp rx=0 tx=0).stop 2024-09-21T07:29:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 -- 172.21.15.27:0/1447701995 shutdown_connections 2024-09-21T07:29:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cc3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504ce9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 --2- 172.21.15.27:0/1447701995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cd1290 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 -- 172.21.15.27:0/1447701995 >> 172.21.15.27:0/1447701995 conn(0x562504916300 msgr2=0x562504cc3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 -- 172.21.15.27:0/1447701995 shutdown_connections 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 -- 172.21.15.27:0/1447701995 wait complete. 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 -- start start 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.654+0000 7f9d3368a740 1 Processor -- start 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cdad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d3368a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d3368a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241b1670 con 0x5625048f92f0 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d3368a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241b2ae0 con 0x5625049221b0 2024-09-21T07:29:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d3368a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241b1d40 con 0x5625049014d0 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d3368a740 10 client.0 _ll_register_callbacks cb 0x7f9d18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d29c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cdad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d29c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cdad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56868/0 (socket says 172.21.15.27:56868) 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d29c00640 1 -- 172.21.15.27:0/2457931723 learned_addr learned my addr 172.21.15.27:0/2457931723 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d2a600640 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d29200640 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d29c00640 1 -- 172.21.15.27:0/2457931723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 msgr2=0x562504cd8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.655+0000 7f9d29c00640 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d29c00640 1 -- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 msgr2=0x562504cd8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d29c00640 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d29c00640 1 -- 172.21.15.27:0/2457931723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562504cd8c80 con 0x5625048f92f0 2024-09-21T07:29:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d29c00640 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cdad40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d180324b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a667a7f129963 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d0b400640 1 -- 172.21.15.27:0/2457931723 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1806b020 con 0x5625048f92f0 2024-09-21T07:29:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d0b400640 1 -- 172.21.15.27:0/2457931723 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f9d180048e0 con 0x5625048f92f0 2024-09-21T07:29:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d180043a0 con 0x5625048f92f0 2024-09-21T07:29:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d0b400640 1 -- 172.21.15.27:0/2457931723 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d18076050 con 0x5625048f92f0 2024-09-21T07:29:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d3368a740 10 client.20286 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.656+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562504d4ebc0 con 0x5625048f92f0 2024-09-21T07:29:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d0b400640 1 -- 172.21.15.27:0/2457931723 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d180043a0 con 0x5625048f92f0 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d0b400640 1 -- 172.21.15.27:0/2457931723 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9d18067460 con 0x5625048f92f0 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d3368a740 10 client.20286 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d3368a740 10 client.20286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d3368a740 20 client.20286 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048b61c0 0x562504d51340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d3368a740 4 client.20286 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562504c3a0b0 con 0x5625048b61c0 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.657+0000 7f9d29200640 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048b61c0 0x562504d51340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.658+0000 7f9d29200640 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048b61c0 0x562504d51340 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba8688d6c3275ad server_cookie=72e63da3881a39dd in_seq=0 out_seq=0 2024-09-21T07:29:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.658+0000 7f9d0b400640 10 client.20286 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.671+0000 7f9d0b400640 1 -- 172.21.15.27:0/2457931723 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562504c3a0b0 con 0x5625048b61c0 2024-09-21T07:29:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.671+0000 7f9d0b400640 10 client.20286 handle_command_reply: tid=0 2024-09-21T07:29:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.671+0000 7f9d3368a740 1 client.20286 shutdown 2024-09-21T07:29:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 msgr2=0x562504cdad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cdad40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d180324b0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 shutdown_connections 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5625048b61c0 0x562504d51340 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625049014d0 0x562504cd8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625049221b0 0x562504cd8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d0b400640 0 client.20286 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 --2- 172.21.15.27:0/2457931723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5625048f92f0 0x562504cdad40 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 >> 172.21.15.27:0/2457931723 conn(0x562504916300 msgr2=0x562504cc7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 shutdown_connections 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 1 -- 172.21.15.27:0/2457931723 wait complete. 2024-09-21T07:29:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.672+0000 7f9d3368a740 20 client.20286 trim_cache size 0 max 16384 2024-09-21T07:29:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:29:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:29:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:29:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:29:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d0c0b66b0 msgr2=0x7f9d0c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d0c0b66b0 0x7f9d0c0b8bc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9d18005ab0 tx=0x7f9d18005a20 comp rx=0 tx=0).stop 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24109240 msgr2=0x7f9d24075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24109240 0x7f9d24075c50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000e9f0 tx=0x7f9d1000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 shutdown_connections 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9d0c0b66b0 0x7f9d0c0b8bc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d24109b90 0x7f9d24076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d24109240 0x7f9d24075c50 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 --2- 172.21.15.27:0/1425977938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d24103910 0x7f9d24075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 >> 172.21.15.27:0/1425977938 conn(0x7f9d240ff330 msgr2=0x7f9d24101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 shutdown_connections 2024-09-21T07:29:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:19.673+0000 7f9d2b000640 1 -- 172.21.15.27:0/1425977938 wait complete. 2024-09-21T07:29:20.115 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:29:20.115 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:29:20.503 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:20.971 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:20 smithi027 ceph-mon[30779]: pgmap v889: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:29:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 -- 172.21.15.27:0/2522613492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41981092d0 msgr2=0x7f41981096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 --2- 172.21.15.27:0/2522613492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41981092d0 0x7f41981096e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4180009900 tx=0x7f418002a410 comp rx=0 tx=0).stop 2024-09-21T07:29:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 -- 172.21.15.27:0/2522613492 shutdown_connections 2024-09-21T07:29:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 --2- 172.21.15.27:0/2522613492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198109c20 0x7f4198110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 --2- 172.21.15.27:0/2522613492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41981092d0 0x7f41981096e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 --2- 172.21.15.27:0/2522613492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4198103990 0x7f4198103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 -- 172.21.15.27:0/2522613492 >> 172.21.15.27:0/2522613492 conn(0x7f41980ff3e0 msgr2=0x7f4198101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 -- 172.21.15.27:0/2522613492 shutdown_connections 2024-09-21T07:29:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.158+0000 7f419dc00640 1 -- 172.21.15.27:0/2522613492 wait complete. 2024-09-21T07:29:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.159+0000 7f419dc00640 1 Processor -- start 2024-09-21T07:29:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.159+0000 7f419dc00640 1 -- start start 2024-09-21T07:29:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f419dc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198103990 0x7f419819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f419dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41981092d0 0x7f419819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f419dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4198109c20 0x7f419819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f419dc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4198113930 con 0x7f4198109c20 2024-09-21T07:29:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f419dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41981137b0 con 0x7f41981092d0 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f419dc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4198113ab0 con 0x7f4198103990 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f4197e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198103990 0x7f419819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f4197e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198103990 0x7f419819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35088/0 (socket says 172.21.15.27:35088) 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f4197e00640 1 -- 172.21.15.27:0/658231053 learned_addr learned my addr 172.21.15.27:0/658231053 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f419d200640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4198109c20 0x7f419819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.160+0000 7f418fe00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41981092d0 0x7f419819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f4197e00640 1 -- 172.21.15.27:0/658231053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41981092d0 msgr2=0x7f419819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f4197e00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41981092d0 0x7f419819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f4197e00640 1 -- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4198109c20 msgr2=0x7f419819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f4197e00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4198109c20 0x7f419819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f4197e00640 1 -- 172.21.15.27:0/658231053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41981a02d0 con 0x7f4198103990 2024-09-21T07:29:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f4197e00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198103990 0x7f419819a360 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f418800e9f0 tx=0x7f418800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f4195600640 1 -- 172.21.15.27:0/658231053 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800cd80 con 0x7f4198103990 2024-09-21T07:29:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.161+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41981a05c0 con 0x7f4198103990 2024-09-21T07:29:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.162+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41981a6d20 con 0x7f4198103990 2024-09-21T07:29:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.162+0000 7f4195600640 1 -- 172.21.15.27:0/658231053 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41880108e0 con 0x7f4198103990 2024-09-21T07:29:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.162+0000 7f4195600640 1 -- 172.21.15.27:0/658231053 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880188e0 con 0x7f4198103990 2024-09-21T07:29:21.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.165+0000 7f41a5e7a740 1 Processor -- start 2024-09-21T07:29:21.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.166+0000 7f41a5e7a740 1 -- start start 2024-09-21T07:29:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.166+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd6bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.166+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd6c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.166+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd6c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.166+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f49dce1e90 con 0x55f49d99b490 2024-09-21T07:29:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.166+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f49dcd12c0 con 0x55f49d9c4350 2024-09-21T07:29:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.166+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f49dcd50d0 con 0x55f49d9a3670 2024-09-21T07:29:21.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f419d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd6bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f418fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd6c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f419d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd6bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58924/0 (socket says 172.21.15.27:58924) 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f418fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd6c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35100/0 (socket says 172.21.15.27:35100) 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f419d200640 1 -- 172.21.15.27:0/3998478349 learned_addr learned my addr 172.21.15.27:0/3998478349 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f4195600640 1 -- 172.21.15.27:0/658231053 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4188010430 con 0x7f4198103990 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f418fe00640 1 -- 172.21.15.27:0/3998478349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 msgr2=0x55f49dd6bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f418fe00640 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd6bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f418fe00640 1 -- 172.21.15.27:0/3998478349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 msgr2=0x55f49dd6c440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f418fe00640 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd6c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f418fe00640 1 -- 172.21.15.27:0/3998478349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49da2f930 con 0x55f49d9a3670 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.167+0000 7f419d200640 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd6bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.168+0000 7f418fe00640 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd6c980 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4180009230 tx=0x7f41800040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a480124b1664609b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:21.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.168+0000 7f418d600640 1 -- 172.21.15.27:0/3998478349 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180035040 con 0x55f49d9a3670 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.168+0000 7f418d600640 1 -- 172.21.15.27:0/3998478349 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41800054f0 con 0x55f49d9a3670 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.168+0000 7f418d600640 1 -- 172.21.15.27:0/3998478349 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800035f0 con 0x55f49d9a3670 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3998478349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 msgr2=0x55f49dd6c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd6c980 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4180009230 tx=0x7f41800040c0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f4195600640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f417c0b6700 0x7f417c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f4195600640 1 -- 172.21.15.27:0/658231053 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f41880e7bd0 con 0x7f4198103990 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3998478349 shutdown_connections 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd6c980 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd6c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3998478349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd6bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3998478349 >> 172.21.15.27:0/3998478349 conn(0x55f49d9b8740 msgr2=0x55f49dd69df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.169+0000 7f419d200640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f417c0b6700 0x7f417c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.170+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3998478349 shutdown_connections 2024-09-21T07:29:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.170+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3998478349 wait complete. 2024-09-21T07:29:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.170+0000 7f419d200640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f417c0b6700 0x7f417c0b8c10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4190005f40 tx=0x7f41900096a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.170+0000 7f41a5e7a740 1 -- start start 2024-09-21T07:29:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 1 Processor -- start 2024-09-21T07:29:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd7c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9a3670 0x55f49dd7cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd81bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f49db32890 con 0x55f49d9a3670 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f49dc6c160 con 0x55f49d9c4350 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f49dd7a680 con 0x55f49d99b490 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f41a5e7a740 10 client.0 _ll_register_callbacks cb 0x55f49dc88a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f418fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9a3670 0x55f49dd7cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f4197e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd7c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f418fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9a3670 0x55f49dd7cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56906/0 (socket says 172.21.15.27:56906) 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.171+0000 7f418fe00640 1 -- 172.21.15.27:0/3989150288 learned_addr learned my addr 172.21.15.27:0/3989150288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f419d200640 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd81bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4197e00640 1 -- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 msgr2=0x55f49dd81bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4197e00640 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd81bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4197e00640 1 -- 172.21.15.27:0/3989150288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9a3670 msgr2=0x55f49dd7cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4197e00640 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9a3670 0x55f49dd7cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4197e00640 1 -- 172.21.15.27:0/3989150288 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49dd82230 con 0x55f49d99b490 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f418fe00640 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9a3670 0x55f49dd7cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4197e00640 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd7c5c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f41880e4f10 tx=0x7f41880b6410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe514aff1d18106 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4186000640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.172+0000 7f4186000640 1 -- 172.21.15.27:0/3989150288 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880ace30 con 0x55f49d99b490 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41800054f0 con 0x55f49d99b490 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f4186000640 1 -- 172.21.15.27:0/3989150288 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f41880ac390 con 0x55f49d99b490 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f4186000640 1 -- 172.21.15.27:0/3989150288 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41880b6e20 con 0x55f49d99b490 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 10 client.30271 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f49dd8ad80 con 0x55f49d99b490 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f4186000640 1 -- 172.21.15.27:0/3989150288 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41800054f0 con 0x55f49d99b490 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f4186000640 1 -- 172.21.15.27:0/3989150288 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f41880ac590 con 0x55f49d99b490 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 10 client.30271 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 10 client.30271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 20 client.30271 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d982cd0 0x55f49dd8e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 4 client.30271 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.173+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f49dce3de0 con 0x55f49d982cd0 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.174+0000 7f418fe00640 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d982cd0 0x55f49dd8e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.174+0000 7f418fe00640 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d982cd0 0x55f49dd8e1a0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520e57a58f00611b server_cookie=3f1158459b301a3a in_seq=0 out_seq=0 2024-09-21T07:29:21.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.174+0000 7f4186000640 10 client.30271 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:21.180 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:20 smithi153 ceph-mon[34747]: pgmap v889: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.175+0000 7f4186000640 1 -- 172.21.15.27:0/3989150288 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f49dce3de0 con 0x55f49d982cd0 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.175+0000 7f4186000640 10 client.30271 handle_command_reply: tid=0 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.175+0000 7f41a5e7a740 1 client.30271 shutdown 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 msgr2=0x55f49dd7c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd7c5c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f41880e4f10 tx=0x7f41880b6410 comp rx=0 tx=0).stop 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 shutdown_connections 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d982cd0 0x55f49dd8e1a0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd81bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9a3670 0x55f49dd7cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/3989150288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd7c5c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f4186000640 0 client.30271 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 >> 172.21.15.27:0/3989150288 conn(0x55f49d9b8740 msgr2=0x55f49dd6f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 shutdown_connections 2024-09-21T07:29:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 1 -- 172.21.15.27:0/3989150288 wait complete. 2024-09-21T07:29:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.176+0000 7f41a5e7a740 20 client.30271 trim_cache size 0 max 16384 2024-09-21T07:29:21.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 Processor -- start 2024-09-21T07:29:21.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 -- start start 2024-09-21T07:29:21.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd75b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd7ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4174012820 con 0x55f49d99b490 2024-09-21T07:29:21.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41740100f0 con 0x55f49d9c4350 2024-09-21T07:29:21.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f417400df60 con 0x55f49d9a3670 2024-09-21T07:29:21.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f4197e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.195+0000 7f4197e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd75600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56920/0 (socket says 172.21.15.27:56920) 2024-09-21T07:29:21.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f4197e00640 1 -- 172.21.15.27:0/2394482196 learned_addr learned my addr 172.21.15.27:0/2394482196 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:21.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f418fe00640 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd75b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f419d200640 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd7ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f418fe00640 1 -- 172.21.15.27:0/2394482196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 msgr2=0x55f49dd7ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f418fe00640 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd7ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f418fe00640 1 -- 172.21.15.27:0/2394482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 msgr2=0x55f49dd75600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f418fe00640 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd75600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f418fe00640 1 -- 172.21.15.27:0/2394482196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49dd70010 con 0x55f49d9a3670 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f4197e00640 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd75600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.196+0000 7f418fe00640 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd75b40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f41800092a0 tx=0x7f4180031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83bda5f120be9d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f418cc00640 1 -- 172.21.15.27:0/2394482196 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180035040 con 0x55f49d9a3670 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f418cc00640 1 -- 172.21.15.27:0/2394482196 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4180004e70 con 0x55f49d9a3670 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f418cc00640 1 -- 172.21.15.27:0/2394482196 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800038e0 con 0x55f49d9a3670 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f41a5e7a740 1 -- 172.21.15.27:0/2394482196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 msgr2=0x55f49dd75b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd75b40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f41800092a0 tx=0x7f4180031400 comp rx=0 tx=0).stop 2024-09-21T07:29:21.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f41a5e7a740 1 -- 172.21.15.27:0/2394482196 shutdown_connections 2024-09-21T07:29:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d9a3670 0x55f49dd75b40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9c4350 0x55f49dd7ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/2394482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d99b490 0x55f49dd75600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.197+0000 7f41a5e7a740 1 -- 172.21.15.27:0/2394482196 >> 172.21.15.27:0/2394482196 conn(0x55f49d9b8740 msgr2=0x55f49dd76e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.198+0000 7f41a5e7a740 1 -- 172.21.15.27:0/2394482196 shutdown_connections 2024-09-21T07:29:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.198+0000 7f41a5e7a740 1 -- 172.21.15.27:0/2394482196 wait complete. 2024-09-21T07:29:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.198+0000 7f41a5e7a740 1 -- start start 2024-09-21T07:29:21.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.198+0000 7f41a5e7a740 1 Processor -- start 2024-09-21T07:29:21.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9a3670 0x55f49dd85f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f41a5e7a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9c4350 0x55f49dd84980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f417400e5b0 con 0x55f49d9c4350 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f417400fa20 con 0x55f49d9a3670 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f41a5e7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f417400ec80 con 0x55f49d99b490 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f41a5e7a740 10 client.0 _ll_register_callbacks cb 0x7f418800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f4197e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f419d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9c4350 0x55f49dd84980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f4197e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd859d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35120/0 (socket says 172.21.15.27:35120) 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f4197e00640 1 -- 172.21.15.27:0/4052714314 learned_addr learned my addr 172.21.15.27:0/4052714314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.199+0000 7f418fe00640 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9a3670 0x55f49dd85f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f419d200640 1 -- 172.21.15.27:0/4052714314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 msgr2=0x55f49dd859d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f419d200640 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd859d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f419d200640 1 -- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9a3670 msgr2=0x55f49dd85f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f419d200640 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9a3670 0x55f49dd85f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f419d200640 1 -- 172.21.15.27:0/4052714314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49dd86740 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f418fe00640 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9a3670 0x55f49dd85f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f419d200640 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9c4350 0x55f49dd84980 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4190005f10 tx=0x7f4190004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb6fb3c8d9c544 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f4186000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f4186000640 1 -- 172.21.15.27:0/4052714314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190004b20 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f4186000640 1 -- 172.21.15.27:0/4052714314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f4190004cc0 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.200+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4180004e70 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f4186000640 1 -- 172.21.15.27:0/4052714314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4190034050 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 10 client.20301 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f49ddf9110 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f4186000640 1 -- 172.21.15.27:0/4052714314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4180004e70 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f4186000640 1 -- 172.21.15.27:0/4052714314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f419001d510 con 0x55f49d9c4350 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 10 client.20301 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 10 client.20301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:21.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 20 client.20301 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:21.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d958080 0x55f49ddfb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:21.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 4 client.20301 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:29:21.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.201+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f49dcdf890 con 0x55f49d958080 2024-09-21T07:29:21.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.202+0000 7f4197e00640 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d958080 0x55f49ddfb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.202+0000 7f4197e00640 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d958080 0x55f49ddfb950 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6bafbaa00d785c4 server_cookie=476c0a586e8796f1 in_seq=0 out_seq=0 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.202+0000 7f4186000640 10 client.20301 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f4186000640 1 -- 172.21.15.27:0/4052714314 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f49dcdf890 con 0x55f49d958080 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f4186000640 10 client.20301 handle_command_reply: tid=0 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 client.20301 shutdown 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9c4350 msgr2=0x55f49dd84980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9c4350 0x55f49dd84980 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4190005f10 tx=0x7f4190004a40 comp rx=0 tx=0).stop 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 shutdown_connections 2024-09-21T07:29:21.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55f49d958080 0x55f49ddfb950 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f49d9c4350 0x55f49dd84980 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f49d9a3670 0x55f49dd85f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 --2- 172.21.15.27:0/4052714314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f49d99b490 0x55f49dd859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 >> 172.21.15.27:0/4052714314 conn(0x55f49d9b8740 msgr2=0x55f49dd7ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f4186000640 0 client.20301 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 shutdown_connections 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.204+0000 7f41a5e7a740 1 -- 172.21.15.27:0/4052714314 wait complete. 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f41a5e7a740 20 client.20301 trim_cache size 0 max 16384 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-09-21T07:29:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "7da61c44-5e2e-4487-b8b6-2df9843a330d", 2024-09-21T07:29:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-09-21T07:29:21.209 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:29:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f417c0b6700 msgr2=0x7f417c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f417c0b6700 0x7f417c0b8c10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4190005f40 tx=0x7f41900096a0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198103990 msgr2=0x7f419819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198103990 0x7f419819a360 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f418800e9f0 tx=0x7f418800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 shutdown_connections 2024-09-21T07:29:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f417c0b6700 0x7f417c0b8c10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4198109c20 0x7f419819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41981092d0 0x7f419819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 --2- 172.21.15.27:0/658231053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4198103990 0x7f419819a360 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.205+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 >> 172.21.15.27:0/658231053 conn(0x7f41980ff3e0 msgr2=0x7f4198101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.206+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 shutdown_connections 2024-09-21T07:29:21.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:21.206+0000 7f419dc00640 1 -- 172.21.15.27:0/658231053 wait complete. 2024-09-21T07:29:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:20 smithi119 ceph-mon[34519]: pgmap v889: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-09-21T07:29:22.060 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:29:22.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:21 smithi153 ceph-mon[34747]: scrub queued for path: / 2024-09-21T07:29:22.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:21 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:29:22.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:21 smithi153 ceph-mon[34747]: scrub summary: active paths [/] 2024-09-21T07:29:22.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:21 smithi153 ceph-mon[34747]: scrub summary: idle+waiting paths [/] 2024-09-21T07:29:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:21 smithi119 ceph-mon[34519]: scrub queued for path: / 2024-09-21T07:29:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:21 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:29:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:21 smithi119 ceph-mon[34519]: scrub summary: active paths [/] 2024-09-21T07:29:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:21 smithi119 ceph-mon[34519]: scrub summary: idle+waiting paths [/] 2024-09-21T07:29:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:21 smithi027 ceph-mon[30779]: scrub queued for path: / 2024-09-21T07:29:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:21 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:29:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:21 smithi027 ceph-mon[30779]: scrub summary: active paths [/] 2024-09-21T07:29:22.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:21 smithi027 ceph-mon[30779]: scrub summary: idle+waiting paths [/] 2024-09-21T07:29:22.444 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.098+0000 7f0fff200640 1 -- 172.21.15.27:0/4277883262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010b500 msgr2=0x7f100010b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.098+0000 7f0fff200640 1 --2- 172.21.15.27:0/4277883262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010b500 0x7f100010b910 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009900 tx=0x7f0fec02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.099+0000 7f0fff200640 1 -- 172.21.15.27:0/4277883262 shutdown_connections 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.099+0000 7f0fff200640 1 --2- 172.21.15.27:0/4277883262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f100010be50 0x7f1000112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.099+0000 7f0fff200640 1 --2- 172.21.15.27:0/4277883262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010b500 0x7f100010b910 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.099+0000 7f0fff200640 1 --2- 172.21.15.27:0/4277883262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1000105bc0 0x7f1000105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.099+0000 7f0fff200640 1 -- 172.21.15.27:0/4277883262 >> 172.21.15.27:0/4277883262 conn(0x7f1000101670 msgr2=0x7f1000103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.099+0000 7f0fff200640 1 -- 172.21.15.27:0/4277883262 shutdown_connections 2024-09-21T07:29:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.099+0000 7f0fff200640 1 -- 172.21.15.27:0/4277883262 wait complete. 2024-09-21T07:29:23.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 Processor -- start 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 -- start start 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1000105bc0 0x7f100019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f100010b500 0x7f100019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010be50 0x7f10001a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1000115b10 con 0x7f100010be50 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1000115990 con 0x7f1000105bc0 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.100+0000 7f0fff200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1000115c90 con 0x7f100010b500 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1000105bc0 0x7f100019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1000105bc0 0x7f100019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58988/0 (socket says 172.21.15.27:58988) 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 -- 172.21.15.27:0/969389518 learned_addr learned my addr 172.21.15.27:0/969389518 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffe800640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010be50 0x7f10001a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffd400640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f100010b500 0x7f100019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 -- 172.21.15.27:0/969389518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f100010b500 msgr2=0x7f100019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f100010b500 0x7f100019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 -- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010be50 msgr2=0x7f10001a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010be50 0x7f10001a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 -- 172.21.15.27:0/969389518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10001a2470 con 0x7f1000105bc0 2024-09-21T07:29:23.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.101+0000 7f0ffde00640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1000105bc0 0x7f100019c500 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000d950 tx=0x7f0ff000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:23.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.102+0000 7f0ff6a00640 1 -- 172.21.15.27:0/969389518 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0017030 con 0x7f1000105bc0 2024-09-21T07:29:23.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.102+0000 7f0ff6a00640 1 -- 172.21.15.27:0/969389518 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff0004020 con 0x7f1000105bc0 2024-09-21T07:29:23.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.102+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10001a2760 con 0x7f1000105bc0 2024-09-21T07:29:23.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.102+0000 7f0ff6a00640 1 -- 172.21.15.27:0/969389518 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00047a0 con 0x7f1000105bc0 2024-09-21T07:29:23.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.102+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10001a8ec0 con 0x7f1000105bc0 2024-09-21T07:29:23.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.105+0000 7f100b7ba740 1 Processor -- start 2024-09-21T07:29:23.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.105+0000 7f100b7ba740 1 -- start start 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2acbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e26fc1b0 0x5635e2acc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2704390 0x5635e2acc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f100b7ba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5635e2a42f70 con 0x5635e26fc1b0 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f100b7ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5635e2a323a0 con 0x5635e2704390 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f100b7ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635e2a361b0 con 0x5635e2725070 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2704390 0x5635e2acc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffe800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2acbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffe800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2acbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35144/0 (socket says 172.21.15.27:35144) 2024-09-21T07:29:23.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2704390 0x5635e2acc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58996/0 (socket says 172.21.15.27:58996) 2024-09-21T07:29:23.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffe800640 1 -- 172.21.15.27:0/465462987 learned_addr learned my addr 172.21.15.27:0/465462987 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:23.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ff6a00640 1 -- 172.21.15.27:0/969389518 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0ff0009a30 con 0x7f1000105bc0 2024-09-21T07:29:23.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffd400640 1 -- 172.21.15.27:0/465462987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 msgr2=0x5635e2acbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffd400640 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2acbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffd400640 1 -- 172.21.15.27:0/465462987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e26fc1b0 msgr2=0x5635e2acc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.106+0000 7f0ffde00640 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e26fc1b0 0x5635e2acc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0ffd400640 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e26fc1b0 0x5635e2acc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0ffd400640 1 -- 172.21.15.27:0/465462987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635e29fcc90 con 0x5635e2704390 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0ffe800640 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2acbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0ffde00640 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e26fc1b0 0x5635e2acc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:29:23.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0ffd400640 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2704390 0x5635e2acc770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0fec02a910 tx=0x7f0fec000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c568d644b5f65848 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0feb000640 1 -- 172.21.15.27:0/465462987 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec040070 con 0x5635e2704390 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0feb000640 1 -- 172.21.15.27:0/465462987 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fec004030 con 0x5635e2704390 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.107+0000 7f0feb000640 1 -- 172.21.15.27:0/465462987 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec035040 con 0x5635e2704390 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f0ff6a00640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fe40b6680 0x7f0fe40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 -- 172.21.15.27:0/465462987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2704390 msgr2=0x5635e2acc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2704390 0x5635e2acc770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0fec02a910 tx=0x7f0fec000bc0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f0ff6a00640 1 -- 172.21.15.27:0/969389518 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f0ff00e7520 con 0x7f1000105bc0 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 -- 172.21.15.27:0/465462987 shutdown_connections 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2704390 0x5635e2acc770 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e26fc1b0 0x5635e2acc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 --2- 172.21.15.27:0/465462987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2acbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 -- 172.21.15.27:0/465462987 >> 172.21.15.27:0/465462987 conn(0x5635e2719460 msgr2=0x5635e2ac9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:23.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f0ffe800640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fe40b6680 0x7f0fe40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.108+0000 7f100b7ba740 1 -- 172.21.15.27:0/465462987 shutdown_connections 2024-09-21T07:29:23.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.109+0000 7f100b7ba740 1 -- 172.21.15.27:0/465462987 wait complete. 2024-09-21T07:29:23.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.109+0000 7f100b7ba740 1 -- start start 2024-09-21T07:29:23.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.109+0000 7f0ffe800640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fe40b6680 0x7f0fe40b8b90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005fd0 tx=0x7f0ff8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:23.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 1 Processor -- start 2024-09-21T07:29:23.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2ad7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ad7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2ae3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5635e2893700 con 0x5635e2704390 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5635e29cd540 con 0x5635e26fc1b0 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635e2ada470 con 0x5635e2725070 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.110+0000 7f100b7ba740 10 client.0 _ll_register_callbacks cb 0x5635e29dd430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ad7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2ad7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffe800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2ae3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2ad7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:59002/0 (socket says 172.21.15.27:59002) 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 -- 172.21.15.27:0/3090504651 learned_addr learned my addr 172.21.15.27:0/3090504651 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 -- 172.21.15.27:0/3090504651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 msgr2=0x5635e2ae3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2ae3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 -- 172.21.15.27:0/3090504651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 msgr2=0x5635e2ad7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ad7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 -- 172.21.15.27:0/3090504651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635e2790650 con 0x5635e26fc1b0 2024-09-21T07:29:23.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0ffde00640 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2ad7850 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00aa230 tx=0x7f0ff00ac9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969b22d6ebae7d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.111+0000 7f0fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f0fdf400640 1 -- 172.21.15.27:0/3090504651 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00ad6c0 con 0x5635e26fc1b0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f0fdf400640 1 -- 172.21.15.27:0/3090504651 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f0ff00e7110 con 0x5635e26fc1b0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5635e29cfb50 con 0x5635e26fc1b0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f0fdf400640 1 -- 172.21.15.27:0/3090504651 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ff00b1aa0 con 0x5635e26fc1b0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f100b7ba740 10 client.30326 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635e29ea910 con 0x5635e26fc1b0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f0fdf400640 1 -- 172.21.15.27:0/3090504651 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5635e29cfb50 con 0x5635e26fc1b0 2024-09-21T07:29:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.112+0000 7f0fdf400640 1 -- 172.21.15.27:0/3090504651 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ff00af070 con 0x5635e26fc1b0 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.113+0000 7f100b7ba740 10 client.30326 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.113+0000 7f100b7ba740 10 client.30326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.113+0000 7f100b7ba740 20 client.30326 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.113+0000 7f100b7ba740 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26e39f0 0x5635e2ae9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.113+0000 7f100b7ba740 4 client.30326 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.113+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635e2a44ec0 con 0x5635e26e39f0 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.113+0000 7f0ffd400640 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26e39f0 0x5635e2ae9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.114+0000 7f0ffd400640 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26e39f0 0x5635e2ae9740 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb54819be4c31870 server_cookie=6e329960bf1e70ac in_seq=0 out_seq=0 2024-09-21T07:29:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.114+0000 7f0fdf400640 10 client.30326 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:23.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f0fdf400640 1 -- 172.21.15.27:0/3090504651 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635e2a44ec0 con 0x5635e26e39f0 2024-09-21T07:29:23.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f0fdf400640 10 client.30326 handle_command_reply: tid=0 2024-09-21T07:29:23.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 client.30326 shutdown 2024-09-21T07:29:23.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 msgr2=0x5635e2ad7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2ad7850 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00aa230 tx=0x7f0ff00ac9d0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 shutdown_connections 2024-09-21T07:29:23.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26e39f0 0x5635e2ae9740 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2725070 0x5635e2ae3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ad7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 --2- 172.21.15.27:0/3090504651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2ad7850 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f0fdf400640 0 client.30326 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 >> 172.21.15.27:0/3090504651 conn(0x5635e2719460 msgr2=0x5635e2aceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 shutdown_connections 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 1 -- 172.21.15.27:0/3090504651 wait complete. 2024-09-21T07:29:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.115+0000 7f100b7ba740 20 client.30326 trim_cache size 0 max 16384 2024-09-21T07:29:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 Processor -- start 2024-09-21T07:29:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 -- start start 2024-09-21T07:29:23.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:22 smithi027 ceph-mon[30779]: pgmap v890: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2acd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2704390 0x5635e2ad0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2725070 0x5635e2ad14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10001b5810 con 0x5635e2725070 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10001b30e0 con 0x5635e26fc1b0 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.134+0000 7f100b7ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10001b0f50 con 0x5635e2704390 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2725070 0x5635e2ad14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2725070 0x5635e2ad14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56966/0 (socket says 172.21.15.27:56966) 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 -- 172.21.15.27:0/113587015 learned_addr learned my addr 172.21.15.27:0/113587015 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffde00640 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2acd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffd400640 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2704390 0x5635e2ad0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 -- 172.21.15.27:0/113587015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2704390 msgr2=0x5635e2ad0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2704390 0x5635e2ad0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 -- 172.21.15.27:0/113587015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 msgr2=0x5635e2acd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2acd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffe800640 1 -- 172.21.15.27:0/113587015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635e2ad8910 con 0x5635e2725070 2024-09-21T07:29:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.135+0000 7f0ffd400640 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2704390 0x5635e2ad0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.136+0000 7f0ffe800640 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2725070 0x5635e2ad14b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800fa20 tx=0x7f0ff8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51faaac8f7db93ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.136+0000 7f0fea600640 1 -- 172.21.15.27:0/113587015 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8004b20 con 0x5635e2725070 2024-09-21T07:29:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.136+0000 7f0fea600640 1 -- 172.21.15.27:0/113587015 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff8003e20 con 0x5635e2725070 2024-09-21T07:29:23.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.136+0000 7f0fea600640 1 -- 172.21.15.27:0/113587015 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8004560 con 0x5635e2725070 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 -- 172.21.15.27:0/113587015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2725070 msgr2=0x5635e2ad14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2725070 0x5635e2ad14b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800fa20 tx=0x7f0ff8016000 comp rx=0 tx=0).stop 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 -- 172.21.15.27:0/113587015 shutdown_connections 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e2704390 0x5635e2ad0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2725070 0x5635e2ad14b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 --2- 172.21.15.27:0/113587015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e26fc1b0 0x5635e2acd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 -- 172.21.15.27:0/113587015 >> 172.21.15.27:0/113587015 conn(0x5635e2719460 msgr2=0x5635e2accd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 -- 172.21.15.27:0/113587015 shutdown_connections 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.137+0000 7f100b7ba740 1 -- 172.21.15.27:0/113587015 wait complete. 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 -- start start 2024-09-21T07:29:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 Processor -- start 2024-09-21T07:29:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e26fc1b0 0x5635e2adfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ae0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2725070 0x5635e2ada7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10001b15a0 con 0x5635e2704390 2024-09-21T07:29:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10001b2a10 con 0x5635e2725070 2024-09-21T07:29:23.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.138+0000 7f100b7ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10001b1c70 con 0x5635e26fc1b0 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f100b7ba740 10 client.0 _ll_register_callbacks cb 0x7f0ff00a8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ae0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e26fc1b0 0x5635e2adfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ae0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56974/0 (socket says 172.21.15.27:56974) 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffd400640 1 -- 172.21.15.27:0/2766432386 learned_addr learned my addr 172.21.15.27:0/2766432386 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffe800640 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2725070 0x5635e2ada7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffde00640 1 -- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2725070 msgr2=0x5635e2ada7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffde00640 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2725070 0x5635e2ada7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffde00640 1 -- 172.21.15.27:0/2766432386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 msgr2=0x5635e2ae0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffde00640 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ae0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.139+0000 7f0ffde00640 1 -- 172.21.15.27:0/2766432386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635e2adaf50 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f0ffde00640 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e26fc1b0 0x5635e2adfaa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005fd0 tx=0x7f0ff0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b61a36b40fcf07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f0fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f0fdf400640 1 -- 172.21.15.27:0/2766432386 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00c4030 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff8003e20 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f0fdf400640 1 -- 172.21.15.27:0/2766432386 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f0ff00e7320 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f0fdf400640 1 -- 172.21.15.27:0/2766432386 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ff00ad6c0 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f100b7ba740 10 client.30295 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.140+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635e2b54c50 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f0fdf400640 1 -- 172.21.15.27:0/2766432386 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff8003e20 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f0fdf400640 1 -- 172.21.15.27:0/2766432386 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ff00af030 con 0x5635e26fc1b0 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f100b7ba740 10 client.30295 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f100b7ba740 10 client.30295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:23.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f100b7ba740 20 client.30295 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:23.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f100b7ba740 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26b8da0 0x5635e2b57380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:23.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f100b7ba740 4 client.30295 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:29:23.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635e2a40970 con 0x5635e26b8da0 2024-09-21T07:29:23.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.141+0000 7f0ffd400640 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26b8da0 0x5635e2b57380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:23.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.142+0000 7f0ffd400640 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26b8da0 0x5635e2b57380 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32bb62778d8e5461 server_cookie=c0fb9d158a20bf28 in_seq=0 out_seq=0 2024-09-21T07:29:23.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.142+0000 7f0fdf400640 10 client.30295 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.142+0000 7f0fdf400640 1 -- 172.21.15.27:0/2766432386 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7670 (crc 0 0 0) 0x5635e2a40970 con 0x5635e26b8da0 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.142+0000 7f0fdf400640 10 client.30295 handle_command_reply: tid=0 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.142+0000 7f100b7ba740 1 client.30295 shutdown 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e26fc1b0 msgr2=0x5635e2adfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e26fc1b0 0x5635e2adfaa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005fd0 tx=0x7f0ff0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 shutdown_connections 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5635e26b8da0 0x5635e2b57380 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e2725070 0x5635e2ada7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5635e2704390 0x5635e2ae0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 --2- 172.21.15.27:0/2766432386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635e26fc1b0 0x5635e2adfaa0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f0fdf400640 0 client.30295 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:23.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 >> 172.21.15.27:0/2766432386 conn(0x5635e2719460 msgr2=0x5635e2accd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 shutdown_connections 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 1 -- 172.21.15.27:0/2766432386 wait complete. 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.143+0000 7f100b7ba740 20 client.30295 trim_cache size 0 max 16384 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stdout: "7da61c44-5e2e-4487-b8b6-2df9843a330d": { 2024-09-21T07:29:23.148 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "7da61c44-5e2e-4487-b8b6-2df9843a330d", 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "7da61c44-5e2e-4487-b8b6-2df9843a330d": { 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:29:21.203018+0000", 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "23d6d7e9-06b5-4a6e-adfb-86c244736c20": { 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:49.824406+0000", 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.151 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "594683c9-bab5-4ebb-ae50-7e80b1ad2ba8": { 2024-09-21T07:29:23.152 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:28:35.097223+0000", 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.153 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:29:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:29:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.157 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:29:23.158 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.159 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:29:23.160 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.161 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:29:23.162 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.164 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:23.165 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fe40b6680 msgr2=0x7f0fe40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fe40b6680 0x7f0fe40b8b90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005fd0 tx=0x7f0ff8009500 comp rx=0 tx=0).stop 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1000105bc0 msgr2=0x7f100019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1000105bc0 0x7f100019c500 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000d950 tx=0x7f0ff000de10 comp rx=0 tx=0).stop 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 shutdown_connections 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f100010b500 0x7f100019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f0fe40b6680 0x7f0fe40b8b90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100010be50 0x7f10001a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 --2- 172.21.15.27:0/969389518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1000105bc0 0x7f100019c500 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 >> 172.21.15.27:0/969389518 conn(0x7f1000101670 msgr2=0x7f1000103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 shutdown_connections 2024-09-21T07:29:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:23.144+0000 7f0fff200640 1 -- 172.21.15.27:0/969389518 wait complete. 2024-09-21T07:29:23.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:22 smithi153 ceph-mon[34747]: pgmap v890: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-21T07:29:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:22 smithi119 ceph-mon[34519]: pgmap v890: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-21T07:29:24.172 INFO:tasks.cephfs.filesystem:scrub status for tag:7da61c44-5e2e-4487-b8b6-2df9843a330d - {'path': '/', 'tag': '7da61c44-5e2e-4487-b8b6-2df9843a330d', 'options': 'recursive,force'} 2024-09-21T07:29:25.178 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:24 smithi153 ceph-mon[34747]: pgmap v891: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-21T07:29:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:24 smithi119 ceph-mon[34519]: pgmap v891: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-21T07:29:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:24 smithi027 ceph-mon[30779]: pgmap v891: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-21T07:29:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:26 smithi153 ceph-mon[34747]: pgmap v892: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-21T07:29:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:26 smithi119 ceph-mon[34519]: pgmap v892: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-21T07:29:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:26 smithi027 ceph-mon[30779]: pgmap v892: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-21T07:29:28.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:29:28.824 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:29.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:28 smithi027 ceph-mon[30779]: pgmap v893: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-09-21T07:29:29.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:28 smithi153 ceph-mon[34747]: pgmap v893: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-09-21T07:29:29.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:28 smithi119 ceph-mon[34519]: pgmap v893: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-09-21T07:29:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.452+0000 7f5de6e00640 1 -- 172.21.15.27:0/4143587076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010c010 msgr2=0x7f5de0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.452+0000 7f5de6e00640 1 --2- 172.21.15.27:0/4143587076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010c010 0x7f5de0112840 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009ca0 tx=0x7f5dd802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.453+0000 7f5de6e00640 1 -- 172.21.15.27:0/4143587076 shutdown_connections 2024-09-21T07:29:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.453+0000 7f5de6e00640 1 --2- 172.21.15.27:0/4143587076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5de010b6c0 0x7f5de010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.453+0000 7f5de6e00640 1 --2- 172.21.15.27:0/4143587076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010c010 0x7f5de0112840 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.453+0000 7f5de6e00640 1 --2- 172.21.15.27:0/4143587076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de0105d80 0x7f5de0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.453+0000 7f5de6e00640 1 -- 172.21.15.27:0/4143587076 >> 172.21.15.27:0/4143587076 conn(0x7f5de0101810 msgr2=0x7f5de0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.453+0000 7f5de6e00640 1 -- 172.21.15.27:0/4143587076 shutdown_connections 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.453+0000 7f5de6e00640 1 -- 172.21.15.27:0/4143587076 wait complete. 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.454+0000 7f5de6e00640 1 Processor -- start 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5de6e00640 1 -- start start 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5de6e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5de0105d80 0x7f5de019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5de6e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010b6c0 0x7f5de019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5de6e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de010c010 0x7f5de01a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5de6e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0115d20 con 0x7f5de010b6c0 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5de6e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0115ba0 con 0x7f5de010c010 2024-09-21T07:29:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5de6e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0115ea0 con 0x7f5de0105d80 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5ddf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010b6c0 0x7f5de019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5ddf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010b6c0 0x7f5de019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37896/0 (socket says 172.21.15.27:37896) 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5ddf400640 1 -- 172.21.15.27:0/1104816946 learned_addr learned my addr 172.21.15.27:0/1104816946 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.455+0000 7f5ddfe00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5de0105d80 0x7f5de019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5de4e00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de010c010 0x7f5de01a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddf400640 1 -- 172.21.15.27:0/1104816946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5de0105d80 msgr2=0x7f5de019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddf400640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5de0105d80 0x7f5de019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddf400640 1 -- 172.21.15.27:0/1104816946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de010c010 msgr2=0x7f5de01a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddf400640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de010c010 0x7f5de01a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddf400640 1 -- 172.21.15.27:0/1104816946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de01a26a0 con 0x7f5de010b6c0 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddfe00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5de0105d80 0x7f5de019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddf400640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010b6c0 0x7f5de019ccd0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00e9c0 tx=0x7f5dcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.456+0000 7f5ddcc00640 1 -- 172.21.15.27:0/1104816946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00ccc0 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.457+0000 7f5ddcc00640 1 -- 172.21.15.27:0/1104816946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc0054d0 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.457+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de01a2990 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.457+0000 7f5ddcc00640 1 -- 172.21.15.27:0/1104816946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc010430 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.457+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de01a90f0 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.458+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de0106190 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.461+0000 7f5ddcc00640 1 -- 172.21.15.27:0/1104816946 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5dcc005000 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.463+0000 7f5ddcc00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5dc40b6700 0x7f5dc40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.464+0000 7f5ddcc00640 1 -- 172.21.15.27:0/1104816946 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f5dcc0e7f60 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.464+0000 7f5ddfe00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5dc40b6700 0x7f5dc40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.464+0000 7f5ddcc00640 1 -- 172.21.15.27:0/1104816946 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5dcc056710 con 0x7f5de010b6c0 2024-09-21T07:29:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.464+0000 7f5ddfe00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5dc40b6700 0x7f5dc40b8c10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009fc0 tx=0x7f5dd0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:29.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.702+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5de019d860 con 0x7f5de010b6c0 2024-09-21T07:29:29.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.703+0000 7f5ddcc00640 1 -- 172.21.15.27:0/1104816946 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f5de019d860 con 0x7f5de010b6c0 2024-09-21T07:29:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:29:29.705 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:29:29.705 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:29:29.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.705+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5dc40b6700 msgr2=0x7f5dc40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:29.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.705+0000 7f5de6e00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5dc40b6700 0x7f5dc40b8c10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009fc0 tx=0x7f5dd0005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:29.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.705+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010b6c0 msgr2=0x7f5de019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:29.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.705+0000 7f5de6e00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010b6c0 0x7f5de019ccd0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00e9c0 tx=0x7f5dcc00ee80 comp rx=0 tx=0).stop 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.705+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 shutdown_connections 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.706+0000 7f5de6e00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f5dc40b6700 0x7f5dc40b8c10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.706+0000 7f5de6e00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de010c010 0x7f5de01a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.706+0000 7f5de6e00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5de010b6c0 0x7f5de019ccd0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.706+0000 7f5de6e00640 1 --2- 172.21.15.27:0/1104816946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5de0105d80 0x7f5de019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.706+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 >> 172.21.15.27:0/1104816946 conn(0x7f5de0101810 msgr2=0x7f5de0109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.706+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 shutdown_connections 2024-09-21T07:29:29.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:29.706+0000 7f5de6e00640 1 -- 172.21.15.27:0/1104816946 wait complete. 2024-09-21T07:29:30.051 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:29 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1104816946' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:30.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:29 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1104816946' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:29 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1104816946' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:30.460 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --set-id 820df98c --release --await' 2024-09-21T07:29:30.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --set-id 820df98c --release --await 2024-09-21T07:29:30.842 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:31.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:30 smithi153 ceph-mon[34747]: pgmap v894: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-21T07:29:31.272 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:30 smithi027 ceph-mon[30779]: pgmap v894: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-21T07:29:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:30 smithi119 ceph-mon[34519]: pgmap v894: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 -- 172.21.15.27:0/970560663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 msgr2=0x7f6b941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 --2- 172.21.15.27:0/970560663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b941098b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009900 tx=0x7f6b7c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 -- 172.21.15.27:0/970560663 shutdown_connections 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 --2- 172.21.15.27:0/970560663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b94109df0 0x7f6b94110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 --2- 172.21.15.27:0/970560663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b941098b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 --2- 172.21.15.27:0/970560663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b94103b60 0x7f6b94103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 -- 172.21.15.27:0/970560663 >> 172.21.15.27:0/970560663 conn(0x7f6b9407b850 msgr2=0x7f6b9407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 -- 172.21.15.27:0/970560663 shutdown_connections 2024-09-21T07:29:31.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.464+0000 7f6b99000640 1 -- 172.21.15.27:0/970560663 wait complete. 2024-09-21T07:29:31.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 Processor -- start 2024-09-21T07:29:31.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 -- start start 2024-09-21T07:29:31.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b94103b60 0x7f6b94198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b94198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b94109df0 0x7f6b9419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941118f0 con 0x7f6b941094a0 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94111770 con 0x7f6b94109df0 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b99000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94111a70 con 0x7f6b94103b60 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.466+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b94198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b92800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b94109df0 0x7f6b9419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b94198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37924/0 (socket says 172.21.15.27:37924) 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b8be00640 1 -- 172.21.15.27:0/1521904575 learned_addr learned my addr 172.21.15.27:0/1521904575 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:31.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b91e00640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b94103b60 0x7f6b94198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b92800640 1 -- 172.21.15.27:0/1521904575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b94103b60 msgr2=0x7f6b94198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b92800640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b94103b60 0x7f6b94198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b92800640 1 -- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 msgr2=0x7f6b94198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b92800640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b94198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b92800640 1 -- 172.21.15.27:0/1521904575 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9419e2f0 con 0x7f6b94109df0 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b8be00640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b94198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b91e00640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b94103b60 0x7f6b94198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.467+0000 7f6b92800640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b94109df0 0x7f6b9419dbc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bfb0 tx=0x7f6b8c0114a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.468+0000 7f6b8aa00640 1 -- 172.21.15.27:0/1521904575 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c01c070 con 0x7f6b94109df0 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.468+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9419e5e0 con 0x7f6b94109df0 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.468+0000 7f6b8aa00640 1 -- 172.21.15.27:0/1521904575 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8c003e20 con 0x7f6b94109df0 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.468+0000 7f6b8aa00640 1 -- 172.21.15.27:0/1521904575 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c017650 con 0x7f6b94109df0 2024-09-21T07:29:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.468+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941a4d40 con 0x7f6b94109df0 2024-09-21T07:29:31.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.471+0000 7f6b9fdea740 1 Processor -- start 2024-09-21T07:29:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b8aa00640 1 -- 172.21.15.27:0/1521904575 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6b8c01f070 con 0x7f6b94109df0 2024-09-21T07:29:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b9fdea740 1 -- start start 2024-09-21T07:29:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432ec520 0x55e1436931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432c3660 0x55e143693700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432cb840 0x55e143693c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e14360a290 con 0x55e1432cb840 2024-09-21T07:29:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1435f96c0 con 0x55e1432c3660 2024-09-21T07:29:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.472+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1435fd4d0 con 0x55e1432ec520 2024-09-21T07:29:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432c3660 0x55e143693700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432ec520 0x55e1436931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432c3660 0x55e143693700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58290/0 (socket says 172.21.15.27:58290) 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 -- 172.21.15.27:0/1394680143 learned_addr learned my addr 172.21.15.27:0/1394680143 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b8be00640 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432cb840 0x55e143693c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 -- 172.21.15.27:0/1394680143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432ec520 msgr2=0x55e1436931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432ec520 0x55e1436931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 -- 172.21.15.27:0/1394680143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432cb840 msgr2=0x55e143693c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432cb840 0x55e143693c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.473+0000 7f6b91e00640 1 -- 172.21.15.27:0/1394680143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e143586460 con 0x55e1432c3660 2024-09-21T07:29:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.474+0000 7f6b91e00640 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432c3660 0x55e143693700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b840115c0 tx=0x7f6b84011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d734209be57f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.474+0000 7f6b8aa00640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6b780b6680 0x7f6b780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.474+0000 7f6b83400640 1 -- 172.21.15.27:0/1394680143 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400f640 con 0x55e1432c3660 2024-09-21T07:29:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.474+0000 7f6b83400640 1 -- 172.21.15.27:0/1394680143 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8400fc90 con 0x55e1432c3660 2024-09-21T07:29:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.474+0000 7f6b83400640 1 -- 172.21.15.27:0/1394680143 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84019c50 con 0x55e1432c3660 2024-09-21T07:29:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.474+0000 7f6b8aa00640 1 -- 172.21.15.27:0/1521904575 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f6b8c0eaed0 con 0x7f6b94109df0 2024-09-21T07:29:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.474+0000 7f6b8be00640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6b780b6680 0x7f6b780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 -- 172.21.15.27:0/1394680143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432c3660 msgr2=0x55e143693700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432c3660 0x55e143693700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b840115c0 tx=0x7f6b84011a80 comp rx=0 tx=0).stop 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b8be00640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6b780b6680 0x7f6b780b8b90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c006010 tx=0x7f6b7c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 -- 172.21.15.27:0/1394680143 shutdown_connections 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432cb840 0x55e143693c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432c3660 0x55e143693700 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/1394680143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432ec520 0x55e1436931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 -- 172.21.15.27:0/1394680143 >> 172.21.15.27:0/1394680143 conn(0x55e1432e0910 msgr2=0x55e1436910b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 -- 172.21.15.27:0/1394680143 shutdown_connections 2024-09-21T07:29:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.475+0000 7f6b9fdea740 1 -- 172.21.15.27:0/1394680143 wait complete. 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.476+0000 7f6b9fdea740 1 -- start start 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.476+0000 7f6b9fdea740 1 Processor -- start 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e14369ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e14369f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e1436aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e14345abb0 con 0x55e1432c3660 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1435946c0 con 0x55e1432ec520 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1436a19a0 con 0x55e1432cb840 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b9fdea740 10 client.0 _ll_register_callbacks cb 0x55e1435af490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e14369ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b92800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e1436aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b92800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e1436aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58294/0 (socket says 172.21.15.27:58294) 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e14369ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37930/0 (socket says 172.21.15.27:37930) 2024-09-21T07:29:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b92800640 1 -- 172.21.15.27:0/3218801979 learned_addr learned my addr 172.21.15.27:0/3218801979 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.477+0000 7f6b8be00640 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e14369f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b92800640 1 -- 172.21.15.27:0/3218801979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 msgr2=0x55e14369f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b92800640 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e14369f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b92800640 1 -- 172.21.15.27:0/3218801979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 msgr2=0x55e14369ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b92800640 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e14369ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b92800640 1 -- 172.21.15.27:0/3218801979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e143357b00 con 0x55e1432ec520 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b92800640 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e1436aacc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00dfd0 tx=0x7f6b8c004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30296ec09fa739d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b77e00640 1 -- 172.21.15.27:0/3218801979 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0b0130 con 0x55e1432ec520 2024-09-21T07:29:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b77e00640 1 -- 172.21.15.27:0/3218801979 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f6b8c004d50 con 0x55e1432ec520 2024-09-21T07:29:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8400fc90 con 0x55e1432ec520 2024-09-21T07:29:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b77e00640 1 -- 172.21.15.27:0/3218801979 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c004000 con 0x55e1432ec520 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b9fdea740 10 client.30350 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.478+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1433ce960 con 0x55e1432ec520 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b77e00640 1 -- 172.21.15.27:0/3218801979 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8400fc90 con 0x55e1432ec520 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b77e00640 1 -- 172.21.15.27:0/3218801979 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6b8c05f050 con 0x55e1432ec520 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b9fdea740 10 client.30350 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b9fdea740 10 client.30350 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b9fdea740 20 client.30350 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1432aaea0 0x55e1436b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b9fdea740 4 client.30350 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.479+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e14360c1e0 con 0x55e1432aaea0 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.480+0000 7f6b91e00640 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1432aaea0 0x55e1436b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.480+0000 7f6b91e00640 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1432aaea0 0x55e1436b0e00 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe1fcc8bbaf03fe server_cookie=32991b9df9648cdf in_seq=0 out_seq=0 2024-09-21T07:29:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.480+0000 7f6b77e00640 10 client.30350 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:31.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.481+0000 7f6b77e00640 1 -- 172.21.15.27:0/3218801979 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e14360c1e0 con 0x55e1432aaea0 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.481+0000 7f6b77e00640 10 client.30350 handle_command_reply: tid=0 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.481+0000 7f6b9fdea740 1 client.30350 shutdown 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.481+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 msgr2=0x55e1436aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e1436aacc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00dfd0 tx=0x7f6b8c004490 comp rx=0 tx=0).stop 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 shutdown_connections 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e1432aaea0 0x55e1436b0e00 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e14369f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e1436aacc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/3218801979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e14369ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 >> 172.21.15.27:0/3218801979 conn(0x55e1432e0910 msgr2=0x55e143696350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b77e00640 0 client.30350 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 shutdown_connections 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 1 -- 172.21.15.27:0/3218801979 wait complete. 2024-09-21T07:29:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.482+0000 7f6b9fdea740 20 client.30350 trim_cache size 0 max 16384 2024-09-21T07:29:31.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.500+0000 7f6b9fdea740 1 Processor -- start 2024-09-21T07:29:31.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.500+0000 7f6b9fdea740 1 -- start start 2024-09-21T07:29:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e1436a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e143690660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e143690ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941b1640 con 0x55e1432c3660 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941aef10 con 0x55e1432ec520 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941acd80 con 0x55e1432cb840 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e1436a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e1436a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37942/0 (socket says 172.21.15.27:37942) 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b91e00640 1 -- 172.21.15.27:0/4060190732 learned_addr learned my addr 172.21.15.27:0/4060190732 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b8be00640 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e143690660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.501+0000 7f6b92800640 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e143690ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b91e00640 1 -- 172.21.15.27:0/4060190732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 msgr2=0x55e143690660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b91e00640 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e143690660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b91e00640 1 -- 172.21.15.27:0/4060190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 msgr2=0x55e143690ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b91e00640 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e143690ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b91e00640 1 -- 172.21.15.27:0/4060190732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14369a3d0 con 0x55e1432c3660 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b91e00640 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e1436a0770 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009380 tx=0x7f6b84009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd0a8253a5d418e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b82a00640 1 -- 172.21.15.27:0/4060190732 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84019630 con 0x55e1432c3660 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b82a00640 1 -- 172.21.15.27:0/4060190732 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84019d90 con 0x55e1432c3660 2024-09-21T07:29:31.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.502+0000 7f6b82a00640 1 -- 172.21.15.27:0/4060190732 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400f8a0 con 0x55e1432c3660 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 -- 172.21.15.27:0/4060190732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 msgr2=0x55e1436a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e1436a0770 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009380 tx=0x7f6b84009310 comp rx=0 tx=0).stop 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 -- 172.21.15.27:0/4060190732 shutdown_connections 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432cb840 0x55e143690660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432ec520 0x55e143690ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/4060190732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432c3660 0x55e1436a0770 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 -- 172.21.15.27:0/4060190732 >> 172.21.15.27:0/4060190732 conn(0x55e1432e0910 msgr2=0x55e1436a0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 -- 172.21.15.27:0/4060190732 shutdown_connections 2024-09-21T07:29:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.503+0000 7f6b9fdea740 1 -- 172.21.15.27:0/4060190732 wait complete. 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 -- start start 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 Processor -- start 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432c3660 0x55e1436a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432cb840 0x55e1436a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432ec520 0x55e14371a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ad3d0 con 0x55e1432ec520 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ae840 con 0x55e1432cb840 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941adaa0 con 0x55e1432c3660 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.504+0000 7f6b9fdea740 10 client.0 _ll_register_callbacks cb 0x7f6b8c00d390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432cb840 0x55e1436a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b92800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432ec520 0x55e14371a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432cb840 0x55e1436a34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58312/0 (socket says 172.21.15.27:58312) 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b8be00640 1 -- 172.21.15.27:0/2060149447 learned_addr learned my addr 172.21.15.27:0/2060149447 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b91e00640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432c3660 0x55e1436a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b92800640 1 -- 172.21.15.27:0/2060149447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432c3660 msgr2=0x55e1436a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b92800640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432c3660 0x55e1436a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b92800640 1 -- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432cb840 msgr2=0x55e1436a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b92800640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432cb840 0x55e1436a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b92800640 1 -- 172.21.15.27:0/2060149447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14371ad40 con 0x55e1432ec520 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b8be00640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432cb840 0x55e1436a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.505+0000 7f6b91e00640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432c3660 0x55e1436a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b92800640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432ec520 0x55e14371a800 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0ac550 tx=0x7f6b8c00d690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99159f80511bbd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b77e00640 1 -- 172.21.15.27:0/2060149447 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0ca070 con 0x55e1432ec520 2024-09-21T07:29:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b77e00640 1 -- 172.21.15.27:0/2060149447 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f6b8c0afe20 con 0x55e1432ec520 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b84019d90 con 0x55e1432ec520 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b77e00640 1 -- 172.21.15.27:0/2060149447 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c0b0130 con 0x55e1432ec520 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b9fdea740 10 client.20340 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e14371be50 con 0x55e1432ec520 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.506+0000 7f6b77e00640 1 -- 172.21.15.27:0/2060149447 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84019d90 con 0x55e1432ec520 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b77e00640 1 -- 172.21.15.27:0/2060149447 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6b8c05f070 con 0x55e1432ec520 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b9fdea740 10 client.20340 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b9fdea740 10 client.20340 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b9fdea740 20 client.20340 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e143280250 0x55e14371e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b9fdea740 4 client.20340 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "820df98c", "release": true, "await": true}] 2024-09-21T07:29:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "set_id": "820df98c", "release": true, "await": true}) -- 0x55e143607c90 con 0x55e143280250 2024-09-21T07:29:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.507+0000 7f6b91e00640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e143280250 0x55e14371e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.508+0000 7f6b91e00640 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e143280250 0x55e14371e450 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf57a675c99adf5 server_cookie=873badb321252500 in_seq=0 out_seq=0 2024-09-21T07:29:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.508+0000 7f6b77e00640 10 client.20340 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.511+0000 7f6b77e00640 1 -- 172.21.15.27:0/2060149447 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e143607c90 con 0x55e143280250 2024-09-21T07:29:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.511+0000 7f6b77e00640 10 client.20340 handle_command_reply: tid=0 2024-09-21T07:29:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.511+0000 7f6b9fdea740 1 client.20340 shutdown 2024-09-21T07:29:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432ec520 msgr2=0x55e14371a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432ec520 0x55e14371a800 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0ac550 tx=0x7f6b8c00d690 comp rx=0 tx=0).stop 2024-09-21T07:29:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 shutdown_connections 2024-09-21T07:29:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55e143280250 0x55e14371e450 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e1432ec520 0x55e14371a800 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1432cb840 0x55e1436a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 --2- 172.21.15.27:0/2060149447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1432c3660 0x55e1436a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 >> 172.21.15.27:0/2060149447 conn(0x55e1432e0910 msgr2=0x55e14369fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b77e00640 0 client.20340 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 shutdown_connections 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 1 -- 172.21.15.27:0/2060149447 wait complete. 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.512+0000 7f6b9fdea740 20 client.20340 trim_cache size 0 max 16384 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:29:31.515 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 64, 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "820df98c": { 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 64, 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T07:29:31.516 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 143.2, 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:31.517 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:29:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6b780b6680 msgr2=0x7f6b780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6b780b6680 0x7f6b780b8b90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c006010 tx=0x7f6b7c005a20 comp rx=0 tx=0).stop 2024-09-21T07:29:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b94109df0 msgr2=0x7f6b9419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b94109df0 0x7f6b9419dbc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bfb0 tx=0x7f6b8c0114a0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 shutdown_connections 2024-09-21T07:29:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f6b780b6680 0x7f6b780b8b90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b94109df0 0x7f6b9419dbc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b941094a0 0x7f6b94198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 --2- 172.21.15.27:0/1521904575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b94103b60 0x7f6b94198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:31.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 >> 172.21.15.27:0/1521904575 conn(0x7f6b9407b850 msgr2=0x7f6b941055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:31.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 shutdown_connections 2024-09-21T07:29:31.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:31.513+0000 7f6b99000640 1 -- 172.21.15.27:0/1521904575 wait complete. 2024-09-21T07:29:32.300 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '820df98c', total seconds elapsed: 27.1 2024-09-21T07:29:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 -- 172.21.15.27:0/882783585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 msgr2=0x7f91d01125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 --2- 172.21.15.27:0/882783585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 0x7f91d01125c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009ca0 tx=0x7f91c802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 -- 172.21.15.27:0/882783585 shutdown_connections 2024-09-21T07:29:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 --2- 172.21.15.27:0/882783585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 0x7f91d01125c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 --2- 172.21.15.27:0/882783585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010b440 0x7f91d010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 --2- 172.21.15.27:0/882783585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d0105b10 0x7f91d0105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 -- 172.21.15.27:0/882783585 >> 172.21.15.27:0/882783585 conn(0x7f91d0101550 msgr2=0x7f91d01039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 -- 172.21.15.27:0/882783585 shutdown_connections 2024-09-21T07:29:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.870+0000 7f91cf200640 1 -- 172.21.15.27:0/882783585 wait complete. 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.871+0000 7f91cf200640 1 Processor -- start 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.871+0000 7f91cf200640 1 -- start start 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cf200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d0105b10 0x7f91d019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010b440 0x7f91d019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cf200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 0x7f91d01a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cf200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0115aa0 con 0x7f91d010bd90 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0115920 con 0x7f91d010b440 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cf200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0115c20 con 0x7f91d0105b10 2024-09-21T07:29:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91ce800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 0x7f91d01a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d0105b10 0x7f91d019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91c5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010b440 0x7f91d019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d0105b10 0x7f91d019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35110/0 (socket says 172.21.15.27:35110) 2024-09-21T07:29:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.872+0000 7f91cde00640 1 -- 172.21.15.27:0/3471102975 learned_addr learned my addr 172.21.15.27:0/3471102975 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c5400640 1 -- 172.21.15.27:0/3471102975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d0105b10 msgr2=0x7f91d019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c5400640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d0105b10 0x7f91d019c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c5400640 1 -- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 msgr2=0x7f91d01a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c5400640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 0x7f91d01a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c5400640 1 -- 172.21.15.27:0/3471102975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d01a2410 con 0x7f91d010b440 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91ce800640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 0x7f91d01a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c5400640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010b440 0x7f91d019ca80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f91d006c8b0 tx=0x7f91b800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c7400640 1 -- 172.21.15.27:0/3471102975 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b800cd00 con 0x7f91d010b440 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d01a2700 con 0x7f91d010b440 2024-09-21T07:29:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c7400640 1 -- 172.21.15.27:0/3471102975 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b80035f0 con 0x7f91d010b440 2024-09-21T07:29:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.873+0000 7f91c7400640 1 -- 172.21.15.27:0/3471102975 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8010690 con 0x7f91d010b440 2024-09-21T07:29:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.874+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01a8e90 con 0x7f91d010b440 2024-09-21T07:29:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.877+0000 7f91dbc20740 1 Processor -- start 2024-09-21T07:29:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.877+0000 7f91dbc20740 1 -- start start 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85ebe1b0 0x561b86262e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b86263350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e9d4d0 0x561b86263890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91dbc20740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561b861d86b0 con 0x561b85e952f0 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91dbc20740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561b861c7ae0 con 0x561b85ebe1b0 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91dbc20740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561b861cb8f0 con 0x561b85e9d4d0 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91cde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b86263350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91cde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b86263350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37970/0 (socket says 172.21.15.27:37970) 2024-09-21T07:29:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91cde00640 1 -- 172.21.15.27:0/1154620742 learned_addr learned my addr 172.21.15.27:0/1154620742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.878+0000 7f91ce800640 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85ebe1b0 0x561b86262e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91cde00640 1 -- 172.21.15.27:0/1154620742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e9d4d0 msgr2=0x561b86263890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91cde00640 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e9d4d0 0x561b86263890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91cde00640 1 -- 172.21.15.27:0/1154620742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85ebe1b0 msgr2=0x561b86262e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91cde00640 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85ebe1b0 0x561b86262e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91cde00640 1 -- 172.21.15.27:0/1154620742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b86194780 con 0x561b85e952f0 2024-09-21T07:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91cde00640 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b86263350 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e10 tx=0x7f91c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfe4b6c381cbe6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91c7400640 1 -- 172.21.15.27:0/3471102975 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f91b80037a0 con 0x7f91d010b440 2024-09-21T07:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91bf000640 1 -- 172.21.15.27:0/1154620742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c001e070 con 0x561b85e952f0 2024-09-21T07:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91bf000640 1 -- 172.21.15.27:0/1154620742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0019ae0 con 0x561b85e952f0 2024-09-21T07:29:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.879+0000 7f91bf000640 1 -- 172.21.15.27:0/1154620742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000fb30 con 0x561b85e952f0 2024-09-21T07:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 -- 172.21.15.27:0/1154620742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 msgr2=0x561b86263350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b86263350 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e10 tx=0x7f91c0011810 comp rx=0 tx=0).stop 2024-09-21T07:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 -- 172.21.15.27:0/1154620742 shutdown_connections 2024-09-21T07:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e9d4d0 0x561b86263890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b86263350 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 --2- 172.21.15.27:0/1154620742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85ebe1b0 0x561b86262e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 -- 172.21.15.27:0/1154620742 >> 172.21.15.27:0/1154620742 conn(0x561b85eb2300 msgr2=0x561b86260d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 -- 172.21.15.27:0/1154620742 shutdown_connections 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91dbc20740 1 -- 172.21.15.27:0/1154620742 wait complete. 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.880+0000 7f91c7400640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91b40b6590 0x7f91b40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.881+0000 7f91c7400640 1 -- 172.21.15.27:0/3471102975 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f91b80e7c60 con 0x7f91d010b440 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.881+0000 7f91dbc20740 1 -- start start 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.881+0000 7f91ce800640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91b40b6590 0x7f91b40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.881+0000 7f91dbc20740 1 Processor -- start 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91ce800640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91b40b6590 0x7f91b40b8aa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c8005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b8626e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b8626e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b8627a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91dbc20740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561b860ffab0 con 0x561b85ebe1b0 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91dbc20740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561b8616e570 con 0x561b85e9d4d0 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91dbc20740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561b86271590 con 0x561b85e952f0 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91dbc20740 10 client.0 _ll_register_callbacks cb 0x561b8617d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91ce800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b8627a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91c5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b8626e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91ce800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b8627a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37972/0 (socket says 172.21.15.27:37972) 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91c5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b8626e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58362/0 (socket says 172.21.15.27:58362) 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91ce800640 1 -- 172.21.15.27:0/415546300 learned_addr learned my addr 172.21.15.27:0/415546300 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91cde00640 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b8626e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91c5400640 1 -- 172.21.15.27:0/415546300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 msgr2=0x561b8626e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91c5400640 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b8626e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91c5400640 1 -- 172.21.15.27:0/415546300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 msgr2=0x561b8627a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91c5400640 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b8627a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.882+0000 7f91c5400640 1 -- 172.21.15.27:0/415546300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b85f29790 con 0x561b85e9d4d0 2024-09-21T07:29:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91c5400640 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b8626e9e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f91b80e5000 tx=0x7f91b80ace70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8a3b713c6104b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91b3400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91b3400640 1 -- 172.21.15.27:0/415546300 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80d4060 con 0x561b85e9d4d0 2024-09-21T07:29:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c0019ae0 con 0x561b85e9d4d0 2024-09-21T07:29:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91b3400640 1 -- 172.21.15.27:0/415546300 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f91b80a9050 con 0x561b85e9d4d0 2024-09-21T07:29:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91b3400640 1 -- 172.21.15.27:0/415546300 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f91b80ade00 con 0x561b85e9d4d0 2024-09-21T07:29:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91dbc20740 10 client.30368 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.883+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b85fa2620 con 0x561b85e9d4d0 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91b3400640 1 -- 172.21.15.27:0/415546300 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0019ae0 con 0x561b85e9d4d0 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91b3400640 1 -- 172.21.15.27:0/415546300 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f91b80a9250 con 0x561b85e9d4d0 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91dbc20740 10 client.30368 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91dbc20740 10 client.30368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91dbc20740 20 client.30368 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91dbc20740 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e7cd80 0x561b862805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91dbc20740 4 client.30368 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b861da600 con 0x561b85e7cd80 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.884+0000 7f91cde00640 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e7cd80 0x561b862805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.885+0000 7f91cde00640 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e7cd80 0x561b862805e0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b04a8c531bc5e9 server_cookie=3df12dea5237081f in_seq=0 out_seq=0 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.885+0000 7f91b3400640 10 client.30368 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91b3400640 1 -- 172.21.15.27:0/415546300 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b861da600 con 0x561b85e7cd80 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91b3400640 10 client.30368 handle_command_reply: tid=0 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 client.30368 shutdown 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 msgr2=0x561b8626e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b8626e9e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f91b80e5000 tx=0x7f91b80ace70 comp rx=0 tx=0).stop 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 shutdown_connections 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e7cd80 0x561b862805e0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b8627a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b8626e9e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 --2- 172.21.15.27:0/415546300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b8626e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91b3400640 0 client.30368 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 >> 172.21.15.27:0/415546300 conn(0x561b85eb2300 msgr2=0x561b86263e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 shutdown_connections 2024-09-21T07:29:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.886+0000 7f91dbc20740 1 -- 172.21.15.27:0/415546300 wait complete. 2024-09-21T07:29:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.887+0000 7f91dbc20740 20 client.30368 trim_cache size 0 max 16384 2024-09-21T07:29:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.902+0000 7f91dbc20740 1 Processor -- start 2024-09-21T07:29:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.902+0000 7f91dbc20740 1 -- start start 2024-09-21T07:29:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b8625fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86264800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85ebe1b0 0x561b862700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91dbc20740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b57c0 con 0x561b85e952f0 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91dbc20740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b3090 con 0x561b85e9d4d0 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91dbc20740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b0f00 con 0x561b85ebe1b0 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91cde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b8625fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91ce800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85ebe1b0 0x561b862700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91cde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b8625fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37984/0 (socket says 172.21.15.27:37984) 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91cde00640 1 -- 172.21.15.27:0/4225446029 learned_addr learned my addr 172.21.15.27:0/4225446029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91c5400640 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86264800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91ce800640 1 -- 172.21.15.27:0/4225446029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 msgr2=0x561b86264800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91ce800640 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86264800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91ce800640 1 -- 172.21.15.27:0/4225446029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 msgr2=0x561b8625fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91ce800640 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b8625fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91ce800640 1 -- 172.21.15.27:0/4225446029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8625f360 con 0x561b85ebe1b0 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.903+0000 7f91c5400640 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86264800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.904+0000 7f91ce800640 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85ebe1b0 0x561b862700d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f91c802aeb0 tx=0x7f91c8042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83587fe5c109bb1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.904+0000 7f91be600640 1 -- 172.21.15.27:0/4225446029 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8003970 con 0x561b85ebe1b0 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.904+0000 7f91be600640 1 -- 172.21.15.27:0/4225446029 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c8036040 con 0x561b85ebe1b0 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.904+0000 7f91be600640 1 -- 172.21.15.27:0/4225446029 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c802f610 con 0x561b85ebe1b0 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 -- 172.21.15.27:0/4225446029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85ebe1b0 msgr2=0x561b862700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85ebe1b0 0x561b862700d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f91c802aeb0 tx=0x7f91c8042000 comp rx=0 tx=0).stop 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 -- 172.21.15.27:0/4225446029 shutdown_connections 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85ebe1b0 0x561b862700d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86264800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4225446029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85e952f0 0x561b8625fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 -- 172.21.15.27:0/4225446029 >> 172.21.15.27:0/4225446029 conn(0x561b85eb2300 msgr2=0x561b8626f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 -- 172.21.15.27:0/4225446029 shutdown_connections 2024-09-21T07:29:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.905+0000 7f91dbc20740 1 -- 172.21.15.27:0/4225446029 wait complete. 2024-09-21T07:29:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.906+0000 7f91dbc20740 1 -- start start 2024-09-21T07:29:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.906+0000 7f91dbc20740 1 Processor -- start 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.906+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b862752b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.906+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86275850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.906+0000 7f91dbc20740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b862e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.906+0000 7f91dbc20740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b1550 con 0x561b85ebe1b0 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91dbc20740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b29c0 con 0x561b85e9d4d0 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91dbc20740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b1c20 con 0x561b85e952f0 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91dbc20740 10 client.0 _ll_register_callbacks cb 0x7f91b800cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86275850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91cde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b862752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86275850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:58376/0 (socket says 172.21.15.27:58376) 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91cde00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b862752b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35142/0 (socket says 172.21.15.27:35142) 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 -- 172.21.15.27:0/4241666782 learned_addr learned my addr 172.21.15.27:0/4241666782 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91ce800640 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b862e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 -- 172.21.15.27:0/4241666782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 msgr2=0x561b862752b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b862752b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 -- 172.21.15.27:0/4241666782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 msgr2=0x561b862e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b862e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 -- 172.21.15.27:0/4241666782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b862e35e0 con 0x561b85e9d4d0 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91cde00640 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b862752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91c5400640 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86275850 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f91b80b33c0 tx=0x7f91b80b3860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ede2a753c1ad96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.907+0000 7f91b3400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91b3400640 1 -- 172.21.15.27:0/4241666782 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80b7120 con 0x561b85e9d4d0 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c8036040 con 0x561b85e9d4d0 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91b3400640 1 -- 172.21.15.27:0/4241666782 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f91b80ac5e0 con 0x561b85e9d4d0 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91b3400640 1 -- 172.21.15.27:0/4241666782 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f91b80b21e0 con 0x561b85e9d4d0 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91dbc20740 10 client.30374 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b862ebb10 con 0x561b85e9d4d0 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91b3400640 1 -- 172.21.15.27:0/4241666782 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c8036040 con 0x561b85e9d4d0 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91b3400640 1 -- 172.21.15.27:0/4241666782 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f91b80b6660 con 0x561b85e9d4d0 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91dbc20740 10 client.30374 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91dbc20740 10 client.30374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91dbc20740 20 client.30374 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.908+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e521c0 0x561b862ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.909+0000 7f91dbc20740 4 client.30374 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:29:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.909+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b861d60b0 con 0x561b85e521c0 2024-09-21T07:29:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.909+0000 7f91cde00640 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e521c0 0x561b862ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.909+0000 7f91cde00640 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e521c0 0x561b862ee0b0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79289784cb7d3f2 server_cookie=5e706b82538442b1 in_seq=0 out_seq=0 2024-09-21T07:29:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.909+0000 7f91b3400640 10 client.30374 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91b3400640 1 -- 172.21.15.27:0/4241666782 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b861d60b0 con 0x561b85e521c0 2024-09-21T07:29:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91b3400640 10 client.30374 handle_command_reply: tid=0 2024-09-21T07:29:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 client.30374 shutdown 2024-09-21T07:29:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 msgr2=0x561b86275850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86275850 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f91b80b33c0 tx=0x7f91b80b3860 comp rx=0 tx=0).stop 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 shutdown_connections 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x561b85e521c0 0x561b862ee0b0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b85ebe1b0 0x561b862e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91b3400640 0 client.30374 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b85e9d4d0 0x561b86275850 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 --2- 172.21.15.27:0/4241666782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b85e952f0 0x561b862752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 >> 172.21.15.27:0/4241666782 conn(0x561b85eb2300 msgr2=0x561b86264560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 shutdown_connections 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.920+0000 7f91dbc20740 1 -- 172.21.15.27:0/4241666782 wait complete. 2024-09-21T07:29:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91dbc20740 20 client.30374 trim_cache size 0 max 16384 2024-09-21T07:29:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-09-21T07:29:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "message": "", 2024-09-21T07:29:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-09-21T07:29:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91b40b6590 msgr2=0x7f91b40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91b40b6590 0x7f91b40b8aa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c8005d80 comp rx=0 tx=0).stop 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010b440 msgr2=0x7f91d019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010b440 0x7f91d019ca80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f91d006c8b0 tx=0x7f91b800ee50 comp rx=0 tx=0).stop 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 shutdown_connections 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f91b40b6590 0x7f91b40b8aa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91d010bd90 0x7f91d01a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010b440 0x7f91d019ca80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 --2- 172.21.15.27:0/3471102975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d0105b10 0x7f91d019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.921+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 >> 172.21.15.27:0/3471102975 conn(0x7f91d0101550 msgr2=0x7f91d0108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.922+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 shutdown_connections 2024-09-21T07:29:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-09-21T07:29:32.922+0000 7f91cf200640 1 -- 172.21.15.27:0/3471102975 wait complete. 2024-09-21T07:29:33.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:32 smithi153 ceph-mon[34747]: pgmap v895: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-21T07:29:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:32 smithi119 ceph-mon[34519]: pgmap v895: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-21T07:29:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:32 smithi027 ceph-mon[30779]: pgmap v895: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-21T07:29:34.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:33 smithi153 ceph-mon[34747]: scrub summary: idle 2024-09-21T07:29:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:33 smithi119 ceph-mon[34519]: scrub summary: idle 2024-09-21T07:29:34.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:33 smithi027 ceph-mon[30779]: scrub summary: idle 2024-09-21T07:29:35.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:34 smithi153 ceph-mon[34747]: pgmap v896: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T07:29:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:34 smithi119 ceph-mon[34519]: pgmap v896: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T07:29:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:34 smithi027 ceph-mon[30779]: pgmap v896: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T07:29:35.548 INFO:tasks.workunit.client.0.smithi027.stdout:561/600: open fd = 3 2024-09-21T07:29:35.548 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.548 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.548 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:562/600: open fd = 3 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:563/600: open fd = 3 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:564/600: open fd = 3 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:565/600: open fd = 3 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:566/600: open fd = 3 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:567/600: open fd = 3 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:568/600: open fd = 3 2024-09-21T07:29:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:569/600: open fd = 3 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:570/600: open fd = 3 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.551 INFO:tasks.workunit.client.0.smithi027.stdout:571/600: open fd = 3 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:572/600: open fd = 3 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:573/600: open fd = 3 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:574/600: open fd = 3 2024-09-21T07:29:35.552 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:575/600: open fd = 3 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:576/600: open fd = 3 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:577/600: open fd = 3 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.553 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:578/600: open fd = 3 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:579/600: open fd = 3 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:580/600: open fd = 3 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.554 INFO:tasks.workunit.client.0.smithi027.stdout:581/600: open fd = 3 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:582/600: open fd = 3 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:583/600: open fd = 3 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:584/600: open fd = 3 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:585/600: open fd = 3 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:586/600: open fd = 3 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.556 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:587/600: open fd = 3 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:588/600: open fd = 3 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:589/600: open fd = 3 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:590/600: open fd = 3 2024-09-21T07:29:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:591/600: open fd = 3 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:592/600: open fd = 3 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:593/600: open fd = 3 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.558 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:594/600: open fd = 3 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:595/600: open fd = 3 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:596/600: open fd = 3 2024-09-21T07:29:35.559 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:597/600: open fd = 3 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:598/600: open fd = 3 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:599/600: open fd = 3 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.560 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.561 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.561 INFO:tasks.workunit.client.0.smithi027.stdout:600/600: open fd = 3 2024-09-21T07:29:35.561 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.561 INFO:tasks.workunit.client.0.smithi027.stdout:write ret = 32 2024-09-21T07:29:35.561 INFO:tasks.workunit.client.0.smithi027.stdout:pread ret = 64 2024-09-21T07:29:35.562 INFO:tasks.workunit.client.0.smithi027.stdout:OK 2024-09-21T07:29:35.563 INFO:tasks.workunit.client.0.smithi027.stderr:+ echo OK 2024-09-21T07:29:35.563 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T07:29:35.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T07:29:35.644 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-21T07:29:35.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T07:29:36.741 DEBUG:teuthology.parallel:result is None 2024-09-21T07:29:36.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T07:29:36.810 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T07:29:36.810 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T07:29:36.824 INFO:tasks.quiescer:joining Quiescers 2024-09-21T07:29:36.824 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T07:29:36.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:29:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:36 smithi027 ceph-mon[30779]: pgmap v897: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-09-21T07:29:37.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:36 smithi153 ceph-mon[34747]: pgmap v897: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-09-21T07:29:37.214 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:36 smithi119 ceph-mon[34519]: pgmap v897: 129 pgs: 129 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 -- 172.21.15.27:0/359296602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc10b500 msgr2=0x7f7dcc10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 --2- 172.21.15.27:0/359296602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc10b500 0x7f7dcc10b910 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009900 tx=0x7f7db402a410 comp rx=0 tx=0).stop 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 -- 172.21.15.27:0/359296602 shutdown_connections 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 --2- 172.21.15.27:0/359296602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dcc10be50 0x7f7dcc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 --2- 172.21.15.27:0/359296602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc10b500 0x7f7dcc10b910 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 --2- 172.21.15.27:0/359296602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc105bc0 0x7f7dcc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 -- 172.21.15.27:0/359296602 >> 172.21.15.27:0/359296602 conn(0x7f7dcc101670 msgr2=0x7f7dcc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 -- 172.21.15.27:0/359296602 shutdown_connections 2024-09-21T07:29:37.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.849+0000 7f7dd2200640 1 -- 172.21.15.27:0/359296602 wait complete. 2024-09-21T07:29:37.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 Processor -- start 2024-09-21T07:29:37.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 -- start start 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc105bc0 0x7f7dcc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc10b500 0x7f7dcc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dcc10be50 0x7f7dcc1a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc115b10 con 0x7f7dcc105bc0 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc115990 con 0x7f7dcc10be50 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.851+0000 7f7dd2200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc115c90 con 0x7f7dcc10b500 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dd0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc105bc0 0x7f7dcc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc10b500 0x7f7dcc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc10b500 0x7f7dcc19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:35158/0 (socket says 172.21.15.27:35158) 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 -- 172.21.15.27:0/4069312802 learned_addr learned my addr 172.21.15.27:0/4069312802 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:37.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dd1800640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dcc10be50 0x7f7dcc1a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:37.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 -- 172.21.15.27:0/4069312802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dcc10be50 msgr2=0x7f7dcc1a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:37.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dcc10be50 0x7f7dcc1a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:37.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 -- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc105bc0 msgr2=0x7f7dcc19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:37.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc105bc0 0x7f7dcc19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dc3e00640 1 -- 172.21.15.27:0/4069312802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc1a24d0 con 0x7f7dcc10b500 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.852+0000 7f7dd0e00640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc105bc0 0x7f7dcc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.853+0000 7f7dc3e00640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc10b500 0x7f7dcc19cb00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7db4006010 tx=0x7f7db4004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.853+0000 7f7dca000640 1 -- 172.21.15.27:0/4069312802 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db403f030 con 0x7f7dcc10b500 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.853+0000 7f7dca000640 1 -- 172.21.15.27:0/4069312802 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db4003aa0 con 0x7f7dcc10b500 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.853+0000 7f7dca000640 1 -- 172.21.15.27:0/4069312802 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4035070 con 0x7f7dcc10b500 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.853+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc1a2760 con 0x7f7dcc10b500 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.853+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1a8f20 con 0x7f7dcc10b500 2024-09-21T07:29:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.854+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc106e40 con 0x7f7dcc10b500 2024-09-21T07:29:37.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.857+0000 7f7dca000640 1 -- 172.21.15.27:0/4069312802 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7db4042030 con 0x7f7dcc10b500 2024-09-21T07:29:37.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.858+0000 7f7dca000640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7db00b6700 0x7f7db00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.859+0000 7f7dca000640 1 -- 172.21.15.27:0/4069312802 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f7db41082c0 con 0x7f7dcc10b500 2024-09-21T07:29:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.859+0000 7f7dd0e00640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7db00b6700 0x7f7db00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.859+0000 7f7dd0e00640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7db00b6700 0x7f7db00b8c10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009fc0 tx=0x7f7dbc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:37.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:37.860+0000 7f7dca000640 1 -- 172.21.15.27:0/4069312802 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7db40d27e0 con 0x7f7dcc10b500 2024-09-21T07:29:38.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.103+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dcc10dc20 con 0x7f7dcc10b500 2024-09-21T07:29:38.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.104+0000 7f7dca000640 1 -- 172.21.15.27:0/4069312802 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f7db40d7720 con 0x7f7dcc10b500 2024-09-21T07:29:38.107 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:29:38.107 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:29:38.107 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.106+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7db00b6700 msgr2=0x7f7db00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.106+0000 7f7dd2200640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7db00b6700 0x7f7db00b8c10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009fc0 tx=0x7f7dbc005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.106+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc10b500 msgr2=0x7f7dcc19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.106+0000 7f7dd2200640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc10b500 0x7f7dcc19cb00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7db4006010 tx=0x7f7db4004060 comp rx=0 tx=0).stop 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 shutdown_connections 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dcc10b500 0x7f7dcc19cb00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7db00b6700 0x7f7db00b8c10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dcc10be50 0x7f7dcc1a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 --2- 172.21.15.27:0/4069312802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dcc105bc0 0x7f7dcc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 >> 172.21.15.27:0/4069312802 conn(0x7f7dcc101670 msgr2=0x7f7dcc103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 shutdown_connections 2024-09-21T07:29:38.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:38.107+0000 7f7dd2200640 1 -- 172.21.15.27:0/4069312802 wait complete. 2024-09-21T07:29:38.907 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24773 quiesce db --cancel --all' 2024-09-21T07:29:38.908 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24773 quiesce db --cancel --all 2024-09-21T07:29:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:38 smithi153 ceph-mon[34747]: pgmap v898: 129 pgs: 129 active+clean; 1.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-21T07:29:39.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:38 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4069312802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:38 smithi119 ceph-mon[34519]: pgmap v898: 129 pgs: 129 active+clean; 1.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-21T07:29:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:38 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4069312802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:39.293 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:38 smithi027 ceph-mon[30779]: pgmap v898: 129 pgs: 129 active+clean; 1.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-21T07:29:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:38 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4069312802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.031+0000 7f1a85200640 1 -- 172.21.15.27:0/2092577322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 msgr2=0x7f1a80110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.031+0000 7f1a85200640 1 --2- 172.21.15.27:0/2092577322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 0x7f1a80110620 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009ca0 tx=0x7f1a7802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.031+0000 7f1a85200640 1 -- 172.21.15.27:0/2092577322 shutdown_connections 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.031+0000 7f1a85200640 1 --2- 172.21.15.27:0/2092577322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a801094a0 0x7f1a801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.031+0000 7f1a85200640 1 --2- 172.21.15.27:0/2092577322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 0x7f1a80110620 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.031+0000 7f1a85200640 1 --2- 172.21.15.27:0/2092577322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a80103b60 0x7f1a80103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.031+0000 7f1a85200640 1 -- 172.21.15.27:0/2092577322 >> 172.21.15.27:0/2092577322 conn(0x7f1a8007b850 msgr2=0x7f1a8007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.032+0000 7f1a85200640 1 -- 172.21.15.27:0/2092577322 shutdown_connections 2024-09-21T07:29:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.032+0000 7f1a85200640 1 -- 172.21.15.27:0/2092577322 wait complete. 2024-09-21T07:29:40.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.033+0000 7f1a85200640 1 Processor -- start 2024-09-21T07:29:40.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.033+0000 7f1a85200640 1 -- start start 2024-09-21T07:29:40.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.033+0000 7f1a85200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a80103b60 0x7f1a80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.033+0000 7f1a85200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a801094a0 0x7f1a80075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.033+0000 7f1a85200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 0x7f1a80076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a85200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80113b00 con 0x7f1a80109df0 2024-09-21T07:29:40.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a85200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80113980 con 0x7f1a80103b60 2024-09-21T07:29:40.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a85200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80113c80 con 0x7f1a801094a0 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a80103b60 0x7f1a80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 0x7f1a80076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a80103b60 0x7f1a80075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36490/0 (socket says 172.21.15.27:36490) 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7de00640 1 -- 172.21.15.27:0/2735357211 learned_addr learned my addr 172.21.15.27:0/2735357211 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7d400640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a801094a0 0x7f1a80075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7e800640 1 -- 172.21.15.27:0/2735357211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a801094a0 msgr2=0x7f1a80075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7e800640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a801094a0 0x7f1a80075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7e800640 1 -- 172.21.15.27:0/2735357211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a80103b60 msgr2=0x7f1a80075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7e800640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a80103b60 0x7f1a80075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7e800640 1 -- 172.21.15.27:0/2735357211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a801a86d0 con 0x7f1a80109df0 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.034+0000 7f1a7d400640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a801094a0 0x7f1a80075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:40.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.035+0000 7f1a7e800640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 0x7f1a80076190 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1a78005e30 tx=0x7f1a780317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.035+0000 7f1a76a00640 1 -- 172.21.15.27:0/2735357211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78003b70 con 0x7f1a80109df0 2024-09-21T07:29:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.035+0000 7f1a76a00640 1 -- 172.21.15.27:0/2735357211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a78003d10 con 0x7f1a80109df0 2024-09-21T07:29:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.035+0000 7f1a76a00640 1 -- 172.21.15.27:0/2735357211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7802f970 con 0x7f1a80109df0 2024-09-21T07:29:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.035+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a801a8960 con 0x7f1a80109df0 2024-09-21T07:29:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.035+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a801a8ea0 con 0x7f1a80109df0 2024-09-21T07:29:40.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.039+0000 7f1a8be96740 1 Processor -- start 2024-09-21T07:29:40.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.039+0000 7f1a76a00640 1 -- 172.21.15.27:0/2735357211 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1a7802fb10 con 0x7f1a80109df0 2024-09-21T07:29:40.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a8be96740 1 -- start start 2024-09-21T07:29:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f35290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b66410 0x5597b2f357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b6e5f0 0x5597b2f35d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a8be96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5597b2eacff0 con 0x5597b2b6e5f0 2024-09-21T07:29:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a8be96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5597b2e9c420 con 0x5597b2b66410 2024-09-21T07:29:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a8be96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5597b2ea0230 con 0x5597b2b8f2d0 2024-09-21T07:29:40.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a7d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b6e5f0 0x5597b2f35d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a7d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b6e5f0 0x5597b2f35d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42170/0 (socket says 172.21.15.27:42170) 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a7d400640 1 -- 172.21.15.27:0/2385191581 learned_addr learned my addr 172.21.15.27:0/2385191581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b66410 0x5597b2f357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.040+0000 7f1a7d400640 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f35290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a7d400640 1 -- 172.21.15.27:0/2385191581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 msgr2=0x5597b2f35290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a7d400640 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f35290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a7d400640 1 -- 172.21.15.27:0/2385191581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b66410 msgr2=0x5597b2f357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a7d400640 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b66410 0x5597b2f357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a7d400640 1 -- 172.21.15.27:0/2385191581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b2bfa8b0 con 0x5597b2b6e5f0 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a76a00640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a640b6680 0x7f1a640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a7d400640 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b6e5f0 0x5597b2f35d10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c011b60 tx=0x7f1a6c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8bc72e3b557476 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.041+0000 7f1a76a00640 1 -- 172.21.15.27:0/2735357211 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f1a78108b10 con 0x7f1a80109df0 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.042+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a640b6680 0x7f1a640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.042+0000 7f1a6b400640 1 -- 172.21.15.27:0/2385191581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00fc50 con 0x5597b2b6e5f0 2024-09-21T07:29:40.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.042+0000 7f1a6b400640 1 -- 172.21.15.27:0/2385191581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6c00fdf0 con 0x5597b2b6e5f0 2024-09-21T07:29:40.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.042+0000 7f1a6b400640 1 -- 172.21.15.27:0/2385191581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0039f0 con 0x5597b2b6e5f0 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 -- 172.21.15.27:0/2385191581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b6e5f0 msgr2=0x5597b2f35d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b6e5f0 0x5597b2f35d10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c011b60 tx=0x7f1a6c005850 comp rx=0 tx=0).stop 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a640b6680 0x7f1a640b8b90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x5597b2f33f10 tx=0x7f1a70005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 -- 172.21.15.27:0/2385191581 shutdown_connections 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b6e5f0 0x5597b2f35d10 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b66410 0x5597b2f357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2385191581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f35290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 -- 172.21.15.27:0/2385191581 >> 172.21.15.27:0/2385191581 conn(0x5597b2b836c0 msgr2=0x5597b2f33180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 -- 172.21.15.27:0/2385191581 shutdown_connections 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.043+0000 7f1a8be96740 1 -- 172.21.15.27:0/2385191581 wait complete. 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 -- start start 2024-09-21T07:29:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 Processor -- start 2024-09-21T07:29:40.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f40970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f40eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5597b2cfd810 con 0x5597b2b66410 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5597b2e37550 con 0x5597b2b6e5f0 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.044+0000 7f1a8be96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5597b2f43a70 con 0x5597b2b8f2d0 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a8be96740 10 client.0 _ll_register_callbacks cb 0x5597b2e47040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f40970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f40eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f40970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42180/0 (socket says 172.21.15.27:42180) 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7de00640 1 -- 172.21.15.27:0/2224716628 learned_addr learned my addr 172.21.15.27:0/2224716628 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7e800640 1 -- 172.21.15.27:0/2224716628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 msgr2=0x5597b2f40eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7e800640 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f40eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7e800640 1 -- 172.21.15.27:0/2224716628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 msgr2=0x5597b2f40970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7e800640 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f40970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.045+0000 7f1a7e800640 1 -- 172.21.15.27:0/2224716628 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b2e29200 con 0x5597b2b8f2d0 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a7e800640 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f4cd90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1a78105e50 tx=0x7f1a780ca4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949cd44f35ff020 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a5fe00640 1 -- 172.21.15.27:0/2224716628 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780d7ce0 con 0x5597b2b8f2d0 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a5fe00640 1 -- 172.21.15.27:0/2224716628 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f1a780cd7b0 con 0x5597b2b8f2d0 2024-09-21T07:29:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a6c00fdf0 con 0x5597b2b8f2d0 2024-09-21T07:29:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a5fe00640 1 -- 172.21.15.27:0/2224716628 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a780ce060 con 0x5597b2b8f2d0 2024-09-21T07:29:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a8be96740 10 client.30352 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.046+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597b2f4f7e0 con 0x5597b2b8f2d0 2024-09-21T07:29:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a5fe00640 1 -- 172.21.15.27:0/2224716628 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6c00fdf0 con 0x5597b2b8f2d0 2024-09-21T07:29:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a5fe00640 1 -- 172.21.15.27:0/2224716628 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1a780cd220 con 0x5597b2b8f2d0 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a8be96740 10 client.30352 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a8be96740 10 client.30352 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a8be96740 20 client.30352 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b4dc50 0x5597b2f52d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a8be96740 4 client.30352 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597b2eaef40 con 0x5597b2b4dc50 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.047+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b4dc50 0x5597b2f52d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.048+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b4dc50 0x5597b2f52d20 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96bd670d7e7e1891 server_cookie=689b7e324e87c3ff in_seq=0 out_seq=0 2024-09-21T07:29:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.048+0000 7f1a5fe00640 10 client.30352 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a5fe00640 1 -- 172.21.15.27:0/2224716628 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597b2eaef40 con 0x5597b2b4dc50 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a5fe00640 10 client.30352 handle_command_reply: tid=0 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 client.30352 shutdown 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 msgr2=0x5597b2f4cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f4cd90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1a78105e50 tx=0x7f1a780ca4b0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 shutdown_connections 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b4dc50 0x5597b2f52d20 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b8f2d0 0x5597b2f4cd90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f40eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a5fe00640 0 client.30352 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:40.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2224716628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f40970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.049+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 >> 172.21.15.27:0/2224716628 conn(0x5597b2b836c0 msgr2=0x5597b2f38420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:40.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.050+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 shutdown_connections 2024-09-21T07:29:40.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.050+0000 7f1a8be96740 1 -- 172.21.15.27:0/2224716628 wait complete. 2024-09-21T07:29:40.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.050+0000 7f1a8be96740 20 client.30352 trim_cache size 0 max 16384 2024-09-21T07:29:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.068+0000 7f1a8be96740 1 Processor -- start 2024-09-21T07:29:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.068+0000 7f1a8be96740 1 -- start start 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.068+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f42500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.068+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b6e5f0 0x5597b2f3f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.068+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b8f2d0 0x5597b2f32070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a8be96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b5840 con 0x5597b2b66410 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a8be96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b3110 con 0x5597b2b8f2d0 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a8be96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b0f80 con 0x5597b2b6e5f0 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f42500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f42500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42192/0 (socket says 172.21.15.27:42192) 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 -- 172.21.15.27:0/2011192872 learned_addr learned my addr 172.21.15.27:0/2011192872 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7d400640 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b6e5f0 0x5597b2f3f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7e800640 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b8f2d0 0x5597b2f32070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 -- 172.21.15.27:0/2011192872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b6e5f0 msgr2=0x5597b2f3f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b6e5f0 0x5597b2f3f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 -- 172.21.15.27:0/2011192872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b8f2d0 msgr2=0x5597b2f32070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b8f2d0 0x5597b2f32070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.069+0000 7f1a7de00640 1 -- 172.21.15.27:0/2011192872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b2f3ee70 con 0x5597b2b66410 2024-09-21T07:29:40.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.070+0000 7f1a7de00640 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f42500 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009620 tx=0x7f1a7001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c715d7b6b9b61af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:40.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.070+0000 7f1a6aa00640 1 -- 172.21.15.27:0/2011192872 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7001fc80 con 0x5597b2b66410 2024-09-21T07:29:40.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.070+0000 7f1a6aa00640 1 -- 172.21.15.27:0/2011192872 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7001fe20 con 0x5597b2b66410 2024-09-21T07:29:40.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.070+0000 7f1a6aa00640 1 -- 172.21.15.27:0/2011192872 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7001e880 con 0x5597b2b66410 2024-09-21T07:29:40.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 -- 172.21.15.27:0/2011192872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 msgr2=0x5597b2f42500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f42500 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009620 tx=0x7f1a7001b000 comp rx=0 tx=0).stop 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 -- 172.21.15.27:0/2011192872 shutdown_connections 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b6e5f0 0x5597b2f3f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b8f2d0 0x5597b2f32070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 --2- 172.21.15.27:0/2011192872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b66410 0x5597b2f42500 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 -- 172.21.15.27:0/2011192872 >> 172.21.15.27:0/2011192872 conn(0x5597b2b836c0 msgr2=0x5597b2f327c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 -- 172.21.15.27:0/2011192872 shutdown_connections 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.071+0000 7f1a8be96740 1 -- 172.21.15.27:0/2011192872 wait complete. 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 -- start start 2024-09-21T07:29:40.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 Processor -- start 2024-09-21T07:29:40.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b66410 0x5597b2f495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f43cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b8f2d0 0x5597b2f442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b15d0 con 0x5597b2b8f2d0 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b2a40 con 0x5597b2b6e5f0 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.072+0000 7f1a8be96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801b1ca0 con 0x5597b2b66410 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a8be96740 10 client.0 _ll_register_callbacks cb 0x7f1a780cac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b66410 0x5597b2f495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b66410 0x5597b2f495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:41708/0 (socket says 172.21.15.27:41708) 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f43cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 -- 172.21.15.27:0/3195157167 learned_addr learned my addr 172.21.15.27:0/3195157167 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7e800640 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b8f2d0 0x5597b2f442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 -- 172.21.15.27:0/3195157167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 msgr2=0x5597b2f43cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f43cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 -- 172.21.15.27:0/3195157167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b8f2d0 msgr2=0x5597b2f442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b8f2d0 0x5597b2f442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.073+0000 7f1a7de00640 1 -- 172.21.15.27:0/3195157167 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b2f44a40 con 0x5597b2b66410 2024-09-21T07:29:40.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a7de00640 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b66410 0x5597b2f495a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009380 tx=0x7f1a7001fd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa43a9d8f7aef81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:40.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a5fe00640 1 -- 172.21.15.27:0/3195157167 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7003f060 con 0x5597b2b66410 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a7001fe20 con 0x5597b2b66410 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a5fe00640 1 -- 172.21.15.27:0/3195157167 <== mon.1 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f1a700051c0 con 0x5597b2b66410 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a5fe00640 1 -- 172.21.15.27:0/3195157167 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a70046050 con 0x5597b2b66410 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a8be96740 10 client.30358 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.074+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597b2fbe6d0 con 0x5597b2b66410 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a5fe00640 1 -- 172.21.15.27:0/3195157167 <== mon.1 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7001fe20 con 0x5597b2b66410 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a5fe00640 1 -- 172.21.15.27:0/3195157167 <== mon.1 v2:172.21.15.153:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1a70030030 con 0x5597b2b66410 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a8be96740 10 client.30358 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:40.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a8be96740 10 client.30358 resolve_mds: validated gid 24773 aka daemon mds.b 2024-09-21T07:29:40.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a8be96740 20 client.30358 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:40.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a8be96740 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b23000 0x5597b2fc0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:40.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a8be96740 4 client.30358 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T07:29:40.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5597b2eaa9f0 con 0x5597b2b23000 2024-09-21T07:29:40.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.075+0000 7f1a7d400640 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b23000 0x5597b2fc0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:40.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.076+0000 7f1a7d400640 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b23000 0x5597b2fc0e10 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b24b2109fbb9001 server_cookie=1b2b7ab81a5e405b in_seq=0 out_seq=0 2024-09-21T07:29:40.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.076+0000 7f1a5fe00640 10 client.30358 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.076+0000 7f1a5fe00640 1 -- 172.21.15.27:0/3195157167 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5597b2eaa9f0 con 0x5597b2b23000 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.076+0000 7f1a5fe00640 10 client.30358 handle_command_reply: tid=0 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.076+0000 7f1a8be96740 1 client.30358 shutdown 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b66410 msgr2=0x5597b2f495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b66410 0x5597b2f495a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009380 tx=0x7f1a7001fd50 comp rx=0 tx=0).stop 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 shutdown_connections 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x5597b2b23000 0x5597b2fc0e10 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5597b2b8f2d0 0x5597b2f442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597b2b6e5f0 0x5597b2f43cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a5fe00640 0 client.30358 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 --2- 172.21.15.27:0/3195157167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5597b2b66410 0x5597b2f495a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 >> 172.21.15.27:0/3195157167 conn(0x5597b2b836c0 msgr2=0x5597b2f36a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 shutdown_connections 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 1 -- 172.21.15.27:0/3195157167 wait complete. 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.077+0000 7f1a8be96740 20 client.30358 trim_cache size 0 max 16384 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24773, 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 64, 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": {} 2024-09-21T07:29:40.081 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:29:40.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a640b6680 msgr2=0x7f1a640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a640b6680 0x7f1a640b8b90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x5597b2f33f10 tx=0x7f1a70005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:40.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 msgr2=0x7f1a80076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:40.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 0x7f1a80076190 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1a78005e30 tx=0x7f1a780317c0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 shutdown_connections 2024-09-21T07:29:40.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a801094a0 0x7f1a80075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f1a640b6680 0x7f1a640b8b90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a80109df0 0x7f1a80076190 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 --2- 172.21.15.27:0/2735357211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a80103b60 0x7f1a80075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:40.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 >> 172.21.15.27:0/2735357211 conn(0x7f1a8007b850 msgr2=0x7f1a80107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:40.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 shutdown_connections 2024-09-21T07:29:40.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:40.078+0000 7f1a85200640 1 -- 172.21.15.27:0/2735357211 wait complete. 2024-09-21T07:29:40.896 INFO:tasks.quiescer:done joining Quiescers 2024-09-21T07:29:40.896 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T07:29:40.907 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:29:40.919 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T07:29:40.929 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T07:29:40.929 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T07:29:40.929 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:29:40.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T07:29:41.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T07:29:41.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T07:29:41.132 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:40 smithi027 ceph-mon[30779]: pgmap v899: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-09-21T07:29:41.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:40 smithi153 ceph-mon[34747]: pgmap v899: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-09-21T07:29:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:40 smithi119 ceph-mon[34519]: pgmap v899: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-09-21T07:29:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.307+0000 7f16f1a00640 1 -- 172.21.15.27:0/796507974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 msgr2=0x7f16ec1060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.307+0000 7f16f1a00640 1 --2- 172.21.15.27:0/796507974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 0x7f16ec1060d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009860 tx=0x7f16dc02a340 comp rx=0 tx=0).stop 2024-09-21T07:29:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.308+0000 7f16f1a00640 1 -- 172.21.15.27:0/796507974 shutdown_connections 2024-09-21T07:29:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.308+0000 7f16f1a00640 1 --2- 172.21.15.27:0/796507974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec10b5f0 0x7f16ec10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.308+0000 7f16f1a00640 1 --2- 172.21.15.27:0/796507974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10bf40 0x7f16ec112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.308+0000 7f16f1a00640 1 --2- 172.21.15.27:0/796507974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 0x7f16ec1060d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.308+0000 7f16f1a00640 1 -- 172.21.15.27:0/796507974 >> 172.21.15.27:0/796507974 conn(0x7f16ec1016e0 msgr2=0x7f16ec103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.308+0000 7f16f1a00640 1 -- 172.21.15.27:0/796507974 shutdown_connections 2024-09-21T07:29:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.308+0000 7f16f1a00640 1 -- 172.21.15.27:0/796507974 wait complete. 2024-09-21T07:29:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 Processor -- start 2024-09-21T07:29:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 -- start start 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 0x7f16ec19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10b5f0 0x7f16ec19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec10bf40 0x7f16ec1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec115c00 con 0x7f16ec10b5f0 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec115a80 con 0x7f16ec105cc0 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.309+0000 7f16f1a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec115d80 con 0x7f16ec10bf40 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16ea000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10b5f0 0x7f16ec19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16ea000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10b5f0 0x7f16ec19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42212/0 (socket says 172.21.15.27:42212) 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16ea000640 1 -- 172.21.15.27:0/3799177602 learned_addr learned my addr 172.21.15.27:0/3799177602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eaa00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 0x7f16ec19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eb400640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec10bf40 0x7f16ec1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eaa00640 1 -- 172.21.15.27:0/3799177602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec10bf40 msgr2=0x7f16ec1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eaa00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec10bf40 0x7f16ec1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eaa00640 1 -- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10b5f0 msgr2=0x7f16ec19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eaa00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10b5f0 0x7f16ec19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eaa00640 1 -- 172.21.15.27:0/3799177602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec1a25b0 con 0x7f16ec105cc0 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16ea000640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10b5f0 0x7f16ec19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.310+0000 7f16eaa00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 0x7f16ec19c680 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f16dc035040 tx=0x7f16dc032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.311+0000 7f16e3400640 1 -- 172.21.15.27:0/3799177602 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc037040 con 0x7f16ec105cc0 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.311+0000 7f16e3400640 1 -- 172.21.15.27:0/3799177602 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16dc030c10 con 0x7f16ec105cc0 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.311+0000 7f16e3400640 1 -- 172.21.15.27:0/3799177602 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0035f0 con 0x7f16ec105cc0 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.311+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ec1a2840 con 0x7f16ec105cc0 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.311+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ec1a2c80 con 0x7f16ec105cc0 2024-09-21T07:29:41.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.312+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ec1060d0 con 0x7f16ec105cc0 2024-09-21T07:29:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.314+0000 7f16e3400640 1 -- 172.21.15.27:0/3799177602 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f16dc043020 con 0x7f16ec105cc0 2024-09-21T07:29:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.316+0000 7f16e3400640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f16d00b6700 0x7f16d00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.316+0000 7f16e3400640 1 -- 172.21.15.27:0/3799177602 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f16dc106ed0 con 0x7f16ec105cc0 2024-09-21T07:29:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.316+0000 7f16ea000640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f16d00b6700 0x7f16d00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.317+0000 7f16ea000640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f16d00b6700 0x7f16d00b8c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f16ec10b1a0 tx=0x7f16d8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.318+0000 7f16e3400640 1 -- 172.21.15.27:0/3799177602 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f16dc04a060 con 0x7f16ec105cc0 2024-09-21T07:29:41.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.543+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f16ec02e6d0 con 0x7f16d00b6700 2024-09-21T07:29:41.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.553+0000 7f16e3400640 1 -- 172.21.15.27:0/3799177602 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f16ec02e6d0 con 0x7f16d00b6700 2024-09-21T07:29:41.555 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T07:12:56", "first": "2024-09-21T07:13:00", "last": "2024-09-21T07:29:00", "last_pruned": "2024-09-21T07:29:00", "created_count": 17, "pruned_count": 11, "active": true} 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f16d00b6700 msgr2=0x7f16d00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f16d00b6700 0x7f16d00b8c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f16ec10b1a0 tx=0x7f16d8005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 msgr2=0x7f16ec19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 0x7f16ec19c680 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f16dc035040 tx=0x7f16dc032d50 comp rx=0 tx=0).stop 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 shutdown_connections 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16ec10bf40 0x7f16ec1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f16d00b6700 0x7f16d00b8c10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ec10b5f0 0x7f16ec19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 --2- 172.21.15.27:0/3799177602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ec105cc0 0x7f16ec19c680 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.555+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 >> 172.21.15.27:0/3799177602 conn(0x7f16ec1016e0 msgr2=0x7f16ec103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.556+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 shutdown_connections 2024-09-21T07:29:41.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.556+0000 7f16f1a00640 1 -- 172.21.15.27:0/3799177602 wait complete. 2024-09-21T07:29:41.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T07:29:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 -- 172.21.15.27:0/2464340429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac105b20 msgr2=0x7f7dac105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 --2- 172.21.15.27:0/2464340429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac105b20 0x7f7dac105f30 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009900 tx=0x7f7da002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 -- 172.21.15.27:0/2464340429 shutdown_connections 2024-09-21T07:29:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 --2- 172.21.15.27:0/2464340429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dac10b450 0x7f7dac10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 --2- 172.21.15.27:0/2464340429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dac10bda0 0x7f7dac1125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 --2- 172.21.15.27:0/2464340429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac105b20 0x7f7dac105f30 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 -- 172.21.15.27:0/2464340429 >> 172.21.15.27:0/2464340429 conn(0x7f7dac101560 msgr2=0x7f7dac1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 -- 172.21.15.27:0/2464340429 shutdown_connections 2024-09-21T07:29:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.810+0000 7f7db2000640 1 -- 172.21.15.27:0/2464340429 wait complete. 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.811+0000 7f7db2000640 1 Processor -- start 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.811+0000 7f7db2000640 1 -- start start 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db2000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dac105b20 0x7f7dac19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac10b450 0x7f7dac19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dac10bda0 0x7f7dac1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db2000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac115a60 con 0x7f7dac10b450 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db2000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac1158e0 con 0x7f7dac10bda0 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db2000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac115be0 con 0x7f7dac105b20 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac10b450 0x7f7dac19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac10b450 0x7f7dac19ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42242/0 (socket says 172.21.15.27:42242) 2024-09-21T07:29:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 -- 172.21.15.27:0/1210223325 learned_addr learned my addr 172.21.15.27:0/1210223325 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:41.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db1600640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dac10bda0 0x7f7dac1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db0c00640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dac105b20 0x7f7dac19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 -- 172.21.15.27:0/1210223325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dac105b20 msgr2=0x7f7dac19c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dac105b20 0x7f7dac19c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 -- 172.21.15.27:0/1210223325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dac10bda0 msgr2=0x7f7dac1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dac10bda0 0x7f7dac1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7dabe00640 1 -- 172.21.15.27:0/1210223325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac1a23f0 con 0x7f7dac10b450 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.812+0000 7f7db1600640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dac10bda0 0x7f7dac1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.813+0000 7f7dabe00640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac10b450 0x7f7dac19ca60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800e9c0 tx=0x7f7d9800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.813+0000 7f7da9600640 1 -- 172.21.15.27:0/1210223325 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800ccc0 con 0x7f7dac10b450 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.813+0000 7f7da9600640 1 -- 172.21.15.27:0/1210223325 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d980054d0 con 0x7f7dac10b450 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.813+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac1a26e0 con 0x7f7dac10b450 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.813+0000 7f7da9600640 1 -- 172.21.15.27:0/1210223325 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98010430 con 0x7f7dac10b450 2024-09-21T07:29:41.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.813+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1a8e70 con 0x7f7dac10b450 2024-09-21T07:29:41.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.814+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac106d90 con 0x7f7dac10b450 2024-09-21T07:29:41.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.817+0000 7f7da9600640 1 -- 172.21.15.27:0/1210223325 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7d98004f40 con 0x7f7dac10b450 2024-09-21T07:29:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.819+0000 7f7da9600640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d940b6700 0x7f7d940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.819+0000 7f7da9600640 1 -- 172.21.15.27:0/1210223325 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f7d980e7f20 con 0x7f7dac10b450 2024-09-21T07:29:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.819+0000 7f7db0c00640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d940b6700 0x7f7d940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.820+0000 7f7db0c00640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d940b6700 0x7f7d940b8c10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7da0031440 tx=0x7f7da0009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:41.822+0000 7f7da9600640 1 -- 172.21.15.27:0/1210223325 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7d980ae060 con 0x7f7dac10b450 2024-09-21T07:29:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.048+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 --> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7dac10dba0 con 0x7f7d940b6700 2024-09-21T07:29:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.056+0000 7f7da9600640 1 -- 172.21.15.27:0/1210223325 <== mgr.14226 v2:172.21.15.27:6800/3854502753 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7dac10dba0 con 0x7f7d940b6700 2024-09-21T07:29:42.058 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d940b6700 msgr2=0x7f7d940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d940b6700 0x7f7d940b8c10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7da0031440 tx=0x7f7da0009560 comp rx=0 tx=0).stop 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac10b450 msgr2=0x7f7dac19ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac10b450 0x7f7dac19ca60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800e9c0 tx=0x7f7d9800edc0 comp rx=0 tx=0).stop 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 shutdown_connections 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f7d940b6700 0x7f7d940b8c10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dac10bda0 0x7f7dac1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac10b450 0x7f7dac19ca60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 --2- 172.21.15.27:0/1210223325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dac105b20 0x7f7dac19c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 >> 172.21.15.27:0/1210223325 conn(0x7f7dac101560 msgr2=0x7f7dac103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 shutdown_connections 2024-09-21T07:29:42.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:42.059+0000 7f7db2000640 1 -- 172.21.15.27:0/1210223325 wait complete. 2024-09-21T07:29:42.086 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T07:29:42.097 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T07:29:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:42 smithi153 ceph-mon[34747]: from='client.30404 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:29:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:42 smithi153 ceph-mon[34747]: pgmap v900: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T07:29:43.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:42 smithi153 ceph-mon[34747]: from='client.20394 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:29:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:42 smithi119 ceph-mon[34519]: from='client.30404 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:29:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:42 smithi119 ceph-mon[34519]: pgmap v900: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T07:29:43.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:42 smithi119 ceph-mon[34519]: from='client.20394 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:29:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:42 smithi027 ceph-mon[30779]: from='client.30404 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:29:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:42 smithi027 ceph-mon[30779]: pgmap v900: 129 pgs: 129 active+clean; 1.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T07:29:43.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:42 smithi027 ceph-mon[30779]: from='client.20394 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:29:45.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:44 smithi153 ceph-mon[34747]: pgmap v901: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-21T07:29:45.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:44 smithi119 ceph-mon[34519]: pgmap v901: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-21T07:29:45.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:44 smithi027 ceph-mon[30779]: pgmap v901: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-21T07:29:47.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:46 smithi153 ceph-mon[34747]: pgmap v902: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-21T07:29:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:46 smithi119 ceph-mon[34519]: pgmap v902: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-21T07:29:47.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:46 smithi027 ceph-mon[30779]: pgmap v902: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-21T07:29:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:48 smithi119 ceph-mon[34519]: pgmap v903: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-21T07:29:49.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:48 smithi027 ceph-mon[30779]: pgmap v903: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-21T07:29:49.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:48 smithi153 ceph-mon[34747]: pgmap v903: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-21T07:29:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:50 smithi119 ceph-mon[34519]: pgmap v904: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-21T07:29:51.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:50 smithi027 ceph-mon[30779]: pgmap v904: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-21T07:29:51.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:50 smithi153 ceph-mon[34747]: pgmap v904: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-09-21T07:29:53.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:52 smithi119 ceph-mon[34519]: pgmap v905: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-21T07:29:53.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:52 smithi027 ceph-mon[30779]: pgmap v905: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-21T07:29:53.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:52 smithi153 ceph-mon[34747]: pgmap v905: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-21T07:29:54.173 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:29:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:55.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:54 smithi027 ceph-mon[30779]: pgmap v906: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-21T07:29:55.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:54 smithi119 ceph-mon[34519]: pgmap v906: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-21T07:29:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 -- 172.21.15.27:0/618133942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4103990 msgr2=0x7f8bd4103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 --2- 172.21.15.27:0/618133942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4103990 0x7f8bd4103da0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009860 tx=0x7f8bbc02a340 comp rx=0 tx=0).stop 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 -- 172.21.15.27:0/618133942 shutdown_connections 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 --2- 172.21.15.27:0/618133942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd4109c20 0x7f8bd4110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 --2- 172.21.15.27:0/618133942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd41092d0 0x7f8bd41096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 --2- 172.21.15.27:0/618133942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4103990 0x7f8bd4103da0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 -- 172.21.15.27:0/618133942 >> 172.21.15.27:0/618133942 conn(0x7f8bd40ff3e0 msgr2=0x7f8bd4101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 -- 172.21.15.27:0/618133942 shutdown_connections 2024-09-21T07:29:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.400+0000 7f8bda000640 1 -- 172.21.15.27:0/618133942 wait complete. 2024-09-21T07:29:55.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.401+0000 7f8bda000640 1 Processor -- start 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.401+0000 7f8bda000640 1 -- start start 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bda000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4103990 0x7f8bd419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bda000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd41092d0 0x7f8bd419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bda000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4109c20 0x7f8bd41a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bda000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4115ae0 con 0x7f8bd4103990 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bda000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4115960 con 0x7f8bd4109c20 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bda000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4115c60 con 0x7f8bd41092d0 2024-09-21T07:29:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bd8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4103990 0x7f8bd419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bd8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4103990 0x7f8bd419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53084/0 (socket says 172.21.15.27:53084) 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bd8c00640 1 -- 172.21.15.27:0/1393696843 learned_addr learned my addr 172.21.15.27:0/1393696843 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4109c20 0x7f8bd41a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.402+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd41092d0 0x7f8bd419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd8c00640 1 -- 172.21.15.27:0/1393696843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd41092d0 msgr2=0x7f8bd419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd8c00640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd41092d0 0x7f8bd419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd8c00640 1 -- 172.21.15.27:0/1393696843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4109c20 msgr2=0x7f8bd41a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd8c00640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4109c20 0x7f8bd41a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd8c00640 1 -- 172.21.15.27:0/1393696843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd41a2480 con 0x7f8bd4103990 2024-09-21T07:29:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4109c20 0x7f8bd41a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd8c00640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4103990 0x7f8bd419c510 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc035040 tx=0x7f8bbc032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd1600640 1 -- 172.21.15.27:0/1393696843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc037040 con 0x7f8bd4103990 2024-09-21T07:29:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd1600640 1 -- 172.21.15.27:0/1393696843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bbc041070 con 0x7f8bd4103990 2024-09-21T07:29:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd41a2710 con 0x7f8bd4103990 2024-09-21T07:29:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.403+0000 7f8bd1600640 1 -- 172.21.15.27:0/1393696843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0035f0 con 0x7f8bd4103990 2024-09-21T07:29:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.404+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd41a2b90 con 0x7f8bd4103990 2024-09-21T07:29:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 Processor -- start 2024-09-21T07:29:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 -- start start 2024-09-21T07:29:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9189cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb918a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55efb9100f70 con 0x55efb8dc2390 2024-09-21T07:29:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55efb90f03a0 con 0x55efb8dba1b0 2024-09-21T07:29:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8be21e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55efb90f41b0 con 0x55efb8de3070 2024-09-21T07:29:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.407+0000 7f8bd1600640 1 -- 172.21.15.27:0/1393696843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8bbc004050 con 0x7f8bd4103990 2024-09-21T07:29:55.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9189cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9189cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36674/0 (socket says 172.21.15.27:36674) 2024-09-21T07:29:55.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 -- 172.21.15.27:0/1925733163 learned_addr learned my addr 172.21.15.27:0/1925733163 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:55.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd8c00640 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb918a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 -- 172.21.15.27:0/1925733163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 msgr2=0x55efb918a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb918a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 -- 172.21.15.27:0/1925733163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 msgr2=0x55efb918a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 -- 172.21.15.27:0/1925733163 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb90bac90 con 0x55efb8de3070 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd8c00640 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb918a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.408+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9189cf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0117e0 tx=0x7f8bcc011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd5240d0510192e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.409+0000 7f8bc9c00640 1 -- 172.21.15.27:0/1925733163 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc011310 con 0x55efb8de3070 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.409+0000 7f8bc9c00640 1 -- 172.21.15.27:0/1925733163 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc003e20 con 0x55efb8de3070 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.409+0000 7f8bc9c00640 1 -- 172.21.15.27:0/1925733163 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0044d0 con 0x55efb8de3070 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.409+0000 7f8bd1600640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bb80b6680 0x7f8bb80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 -- 172.21.15.27:0/1925733163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 msgr2=0x55efb9189cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9189cf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0117e0 tx=0x7f8bcc011ca0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8bd1600640 1 -- 172.21.15.27:0/1393696843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f8bbc107520 con 0x7f8bd4103990 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 -- 172.21.15.27:0/1925733163 shutdown_connections 2024-09-21T07:29:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb918a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1925733163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9189cf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 -- 172.21.15.27:0/1925733163 >> 172.21.15.27:0/1925733163 conn(0x55efb8dd7460 msgr2=0x55efb9187b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8bd8c00640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bb80b6680 0x7f8bb80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 -- 172.21.15.27:0/1925733163 shutdown_connections 2024-09-21T07:29:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.410+0000 7f8be21e2740 1 -- 172.21.15.27:0/1925733163 wait complete. 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.411+0000 7f8bd8c00640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bb80b6680 0x7f8bb80b8b90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55efb9188970 tx=0x7f8bbc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.411+0000 7f8be21e2740 1 -- start start 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.411+0000 7f8be21e2740 1 Processor -- start 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb91958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9195df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb91a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8be21e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55efb8f51700 con 0x55efb8dba1b0 2024-09-21T07:29:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8be21e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55efb908b540 con 0x55efb8de3070 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8be21e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55efb91984d0 con 0x55efb8dc2390 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8be21e2740 10 client.0 _ll_register_callbacks cb 0x55efb909b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb91958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb91958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53100/0 (socket says 172.21.15.27:53100) 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 -- 172.21.15.27:0/3867700063 learned_addr learned my addr 172.21.15.27:0/3867700063 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb91a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 -- 172.21.15.27:0/3867700063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 msgr2=0x55efb9195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 -- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 msgr2=0x55efb91a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.412+0000 7f8bd9600640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb91a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.413+0000 7f8bd9600640 1 -- 172.21.15.27:0/3867700063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb8e4e650 con 0x55efb8dba1b0 2024-09-21T07:29:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.413+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.413+0000 7f8bd9600640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb91958b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006010 tx=0x7f8bcc011150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0c356638dd51d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.413+0000 7f8bc2000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.413+0000 7f8bc2000640 1 -- 172.21.15.27:0/3867700063 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc020050 con 0x55efb8dba1b0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.413+0000 7f8bc2000640 1 -- 172.21.15.27:0/3867700063 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f8bcc004250 con 0x55efb8dba1b0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.413+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efb908db50 con 0x55efb8dba1b0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8bc2000640 1 -- 172.21.15.27:0/3867700063 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bcc034050 con 0x55efb8dba1b0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 10 client.20406 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efb90a8910 con 0x55efb8dba1b0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8bc2000640 1 -- 172.21.15.27:0/3867700063 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55efb908db50 con 0x55efb8dba1b0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8bc2000640 1 -- 172.21.15.27:0/3867700063 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8bcc018020 con 0x55efb8dba1b0 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 10 client.20406 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 10 client.20406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 20 client.20406 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8da19f0 0x55efb91a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 4 client.20406 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:55.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.414+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efb9102ec0 con 0x55efb8da19f0 2024-09-21T07:29:55.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.415+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8da19f0 0x55efb91a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.415+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8da19f0 0x55efb91a77c0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9e44d4ecfa3ac9 server_cookie=6d15ebb2bff4313c in_seq=0 out_seq=0 2024-09-21T07:29:55.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.415+0000 7f8bc2000640 10 client.20406 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.416+0000 7f8bc2000640 1 -- 172.21.15.27:0/3867700063 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efb9102ec0 con 0x55efb8da19f0 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.416+0000 7f8bc2000640 10 client.20406 handle_command_reply: tid=0 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.416+0000 7f8be21e2740 1 client.20406 shutdown 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 msgr2=0x55efb91958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb91958b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006010 tx=0x7f8bcc011150 comp rx=0 tx=0).stop 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 shutdown_connections 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8da19f0 0x55efb91a77c0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb91a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 --2- 172.21.15.27:0/3867700063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb91958b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 >> 172.21.15.27:0/3867700063 conn(0x55efb8dd7460 msgr2=0x55efb918ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:55.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8bc2000640 0 client.20406 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:55.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 shutdown_connections 2024-09-21T07:29:55.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 1 -- 172.21.15.27:0/3867700063 wait complete. 2024-09-21T07:29:55.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.417+0000 7f8be21e2740 20 client.20406 trim_cache size 0 max 16384 2024-09-21T07:29:55.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:54 smithi153 ceph-mon[34747]: pgmap v906: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-21T07:29:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.441+0000 7f8be21e2740 1 Processor -- start 2024-09-21T07:29:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.441+0000 7f8be21e2740 1 -- start start 2024-09-21T07:29:55.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb91a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9187b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8be21e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41b5870 con 0x55efb8dc2390 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8be21e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41b3140 con 0x55efb8dba1b0 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8be21e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41b0fb0 con 0x55efb8de3070 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb91a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb91a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:35832/0 (socket says 172.21.15.27:35832) 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8bd9600640 1 -- 172.21.15.27:0/1981666727 learned_addr learned my addr 172.21.15.27:0/1981666727 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.442+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9187b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bd9600640 1 -- 172.21.15.27:0/1981666727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 msgr2=0x55efb9187b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9187b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bd9600640 1 -- 172.21.15.27:0/1981666727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 msgr2=0x55efb918fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bd9600640 1 -- 172.21.15.27:0/1981666727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb9194610 con 0x55efb8dba1b0 2024-09-21T07:29:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:29:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb91a16f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc005d70 tx=0x7f8bcc01c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc88551842f8bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bc9200640 1 -- 172.21.15.27:0/1981666727 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc020030 con 0x55efb8dba1b0 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bc9200640 1 -- 172.21.15.27:0/1981666727 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc024040 con 0x55efb8dba1b0 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.443+0000 7f8bc9200640 1 -- 172.21.15.27:0/1981666727 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc023040 con 0x55efb8dba1b0 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 -- 172.21.15.27:0/1981666727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 msgr2=0x55efb91a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb91a16f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc005d70 tx=0x7f8bcc01c040 comp rx=0 tx=0).stop 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 -- 172.21.15.27:0/1981666727 shutdown_connections 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8de3070 0x55efb9187b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dc2390 0x55efb918fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1981666727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8dba1b0 0x55efb91a16f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 -- 172.21.15.27:0/1981666727 >> 172.21.15.27:0/1981666727 conn(0x55efb8dd7460 msgr2=0x55efb9187410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 -- 172.21.15.27:0/1981666727 shutdown_connections 2024-09-21T07:29:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.444+0000 7f8be21e2740 1 -- 172.21.15.27:0/1981666727 wait complete. 2024-09-21T07:29:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.445+0000 7f8be21e2740 1 -- start start 2024-09-21T07:29:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.445+0000 7f8be21e2740 1 Processor -- start 2024-09-21T07:29:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.445+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb919dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.445+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9192640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.445+0000 7f8be21e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb9192c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.445+0000 7f8be21e2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41b1600 con 0x55efb8dba1b0 2024-09-21T07:29:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.445+0000 7f8be21e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41b2a70 con 0x55efb8de3070 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8be21e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd41b1cd0 con 0x55efb8dc2390 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8be21e2740 10 client.0 _ll_register_callbacks cb 0x7f8bcc003b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb919dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9192640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb919dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53112/0 (socket says 172.21.15.27:53112) 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 -- 172.21.15.27:0/1011885677 learned_addr learned my addr 172.21.15.27:0/1011885677 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb9192c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 -- 172.21.15.27:0/1011885677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 msgr2=0x55efb9192640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9192640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 -- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 msgr2=0x55efb9192c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb9192c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.446+0000 7f8bd9600640 1 -- 172.21.15.27:0/1011885677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb9193420 con 0x55efb8dba1b0 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.447+0000 7f8bd9600640 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb919dfd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006010 tx=0x7f8bcc02be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b07abfa1d5d9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.447+0000 7f8bc2000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:29:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.447+0000 7f8bc2000640 1 -- 172.21.15.27:0/1011885677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc020070 con 0x55efb8dba1b0 2024-09-21T07:29:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.447+0000 7f8bc2000640 1 -- 172.21.15.27:0/1011885677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f8bcc0050c0 con 0x55efb8dba1b0 2024-09-21T07:29:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.447+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc024040 con 0x55efb8dba1b0 2024-09-21T07:29:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.447+0000 7f8bc2000640 1 -- 172.21.15.27:0/1011885677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bcc04c050 con 0x55efb8dba1b0 2024-09-21T07:29:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 10 client.20412 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efb9212020 con 0x55efb8dba1b0 2024-09-21T07:29:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8bc2000640 1 -- 172.21.15.27:0/1011885677 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc024040 con 0x55efb8dba1b0 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8bc2000640 1 -- 172.21.15.27:0/1011885677 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8bcc018040 con 0x55efb8dba1b0 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 10 client.20412 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 10 client.20412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 20 client.20412 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8d76da0 0x55efb9214700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 4 client.20412 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.448+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efb9196290 con 0x55efb8d76da0 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.449+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8d76da0 0x55efb9214700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.449+0000 7f8bd3e00640 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8d76da0 0x55efb9214700 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67815c271dc3e34 server_cookie=62c261d784433c57 in_seq=0 out_seq=0 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.449+0000 7f8bc2000640 10 client.20412 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8bc2000640 1 -- 172.21.15.27:0/1011885677 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7474 (crc 0 0 0) 0x55efb9196290 con 0x55efb8d76da0 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8bc2000640 10 client.20412 handle_command_reply: tid=0 2024-09-21T07:29:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 client.20412 shutdown 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 msgr2=0x55efb919dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb919dfd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006010 tx=0x7f8bcc02be80 comp rx=0 tx=0).stop 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 shutdown_connections 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x55efb8d76da0 0x55efb9214700 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efb8dc2390 0x55efb9192640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efb8de3070 0x55efb9192c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8bc2000640 0 client.20412 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 --2- 172.21.15.27:0/1011885677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55efb8dba1b0 0x55efb919dfd0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 >> 172.21.15.27:0/1011885677 conn(0x55efb8dd7460 msgr2=0x55efb91861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 shutdown_connections 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 1 -- 172.21.15.27:0/1011885677 wait complete. 2024-09-21T07:29:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.450+0000 7f8be21e2740 20 client.20412 trim_cache size 0 max 16384 2024-09-21T07:29:55.455 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:29:55.455 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-09-21T07:29:55.455 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {}, 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_stats": { 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "7da61c44-5e2e-4487-b8b6-2df9843a330d": { 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:29:21.203018+0000", 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "23d6d7e9-06b5-4a6e-adfb-86c244736c20": { 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:49.824406+0000", 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "aeb1e447-52d6-4d04-96be-5e4dce252fa0": { 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:20.701737+0000", 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "ddc913bc-1450-443e-8516-d628dd0da731": { 2024-09-21T07:29:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:26:08.055068+0000", 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "842dd810-32aa-4200-bb13-a6fd281291ee": { 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:19.355342+0000", 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "06f8f5c7-be29-4731-b3a2-a4e12b5538d6": { 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:25:06.333683+0000", 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "594683c9-bab5-4ebb-ae50-7e80b1ad2ba8": { 2024-09-21T07:29:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:28:35.097223+0000", 2024-09-21T07:29:55.460 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "0f52bbc6-199d-4b79-9044-1ebcfed058a5": { 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:27:05.198573+0000", 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.461 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "9a0044a9-e2bc-476f-884e-4ed21d019901": { 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:20.871873+0000", 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ecab2dc3-aa2b-40f9-bc04-9dcbbf928543": { 2024-09-21T07:29:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:22:34.847650+0000", 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "f52934f5-6079-4ed1-a0f3-77cab733e4d5": { 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:23:22.832393+0000", 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "0ad2e08c-dd6f-42b7-b797-c85976aaee82": { 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:50.053824+0000", 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "b05ba1b3-e12a-4ea5-89d1-fb5c0450e63b": { 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:12:56.759713+0000", 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "2a5d2a1e-61ae-4ebf-b3ce-4eee84c50ba5": { 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:15:19.840999+0000", 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "97b1e9ea-fe6d-41a3-b306-92790bbfa6cb": { 2024-09-21T07:29:55.465 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:24:08.109208+0000", 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "e2a1fe8e-18f3-445f-b7e7-67885d99e650": { 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:18:20.755297+0000", 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "a21b9482-1919-4fdf-88af-92fc0d74616b": { 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:14:33.752195+0000", 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "93e8af8f-b26b-4c09-ad37-673f408ca863": { 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:05.111823+0000", 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.467 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "860f11a5-87ac-4626-9267-1ad21477cbcd": { 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:16:49.475670+0000", 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "88a53041-1567-48da-a7e8-c17f689afb84": { 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:07.925070+0000", 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.468 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "7aef99b8-6e45-4b02-ae58-adedbae905f2": { 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:19:52.172915+0000", 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "c896104d-ae00-459e-a046-11dd33f800e9": { 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:17:34.538499+0000", 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "d0ace04d-6f23-4748-8446-09ba2828b765": { 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:13:42.868516+0000", 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "31290691-93d9-4f7e-b443-15fac385f103": { 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:04.458593+0000", 2024-09-21T07:29:55.470 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "4426db32-e542-4900-a1fd-f4a7dd06dc29": { 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:20:50.746380+0000", 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:29:55.471 INFO:teuthology.orchestra.run.smithi027.stdout: "277625f4-5a50-4dd0-9451-0842adf228cf": { 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: "start_time": "2024-09-21T07:21:37.533153+0000", 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_passed": 0, 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_failed": 0, 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_skipped": 0 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bb80b6680 msgr2=0x7f8bb80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bb80b6680 0x7f8bb80b8b90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55efb9188970 tx=0x7f8bbc005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:55.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4103990 msgr2=0x7f8bd419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4103990 0x7f8bd419c510 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc035040 tx=0x7f8bbc032d50 comp rx=0 tx=0).stop 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 shutdown_connections 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd41092d0 0x7f8bd419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8bb80b6680 0x7f8bb80b8b90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd4109c20 0x7f8bd41a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 --2- 172.21.15.27:0/1393696843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4103990 0x7f8bd419c510 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 >> 172.21.15.27:0/1393696843 conn(0x7f8bd40ff3e0 msgr2=0x7f8bd4101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 shutdown_connections 2024-09-21T07:29:55.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:55.451+0000 7f8bda000640 1 -- 172.21.15.27:0/1393696843 wait complete. 2024-09-21T07:29:56.479 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:29:56.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:29:56.863 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:57.199 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:56 smithi027 ceph-mon[30779]: pgmap v907: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-21T07:29:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:56 smithi119 ceph-mon[34519]: pgmap v907: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-21T07:29:57.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:56 smithi153 ceph-mon[34747]: pgmap v907: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-21T07:29:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 -- 172.21.15.27:0/3974653651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260078290 msgr2=0x7fe260114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 --2- 172.21.15.27:0/3974653651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260078290 0x7fe260114200 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009900 tx=0x7fe24c02a410 comp rx=0 tx=0).stop 2024-09-21T07:29:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 -- 172.21.15.27:0/3974653651 shutdown_connections 2024-09-21T07:29:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 --2- 172.21.15.27:0/3974653651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe260114740 0x7fe260116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 --2- 172.21.15.27:0/3974653651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260078290 0x7fe260114200 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 --2- 172.21.15.27:0/3974653651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe260077940 0x7fe260077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 -- 172.21.15.27:0/3974653651 >> 172.21.15.27:0/3974653651 conn(0x7fe260101810 msgr2=0x7fe260103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 -- 172.21.15.27:0/3974653651 shutdown_connections 2024-09-21T07:29:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.523+0000 7fe266200640 1 -- 172.21.15.27:0/3974653651 wait complete. 2024-09-21T07:29:57.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.524+0000 7fe266200640 1 Processor -- start 2024-09-21T07:29:57.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.525+0000 7fe266200640 1 -- start start 2024-09-21T07:29:57.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.525+0000 7fe266200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260077940 0x7fe2601a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:57.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.525+0000 7fe266200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe260078290 0x7fe2601a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.525+0000 7fe266200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe260114740 0x7fe2601a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.525+0000 7fe266200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe26011a0d0 con 0x7fe260077940 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.525+0000 7fe266200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe260119f50 con 0x7fe260078290 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.525+0000 7fe266200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe26011a250 con 0x7fe260114740 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260077940 0x7fe2601a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260077940 0x7fe2601a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53120/0 (socket says 172.21.15.27:53120) 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 -- 172.21.15.27:0/1913691061 learned_addr learned my addr 172.21.15.27:0/1913691061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:57.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25ea00640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe260078290 0x7fe2601a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:57.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25fe00640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe260114740 0x7fe2601a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:57.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 -- 172.21.15.27:0/1913691061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe260114740 msgr2=0x7fe2601a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:57.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe260114740 0x7fe2601a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 -- 172.21.15.27:0/1913691061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe260078290 msgr2=0x7fe2601a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe260078290 0x7fe2601a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.526+0000 7fe25f400640 1 -- 172.21.15.27:0/1913691061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2601a6a30 con 0x7fe260077940 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.527+0000 7fe25f400640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260077940 0x7fe2601a0ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe25000e9f0 tx=0x7fe25000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.527+0000 7fe257e00640 1 -- 172.21.15.27:0/1913691061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25000cd80 con 0x7fe260077940 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.527+0000 7fe257e00640 1 -- 172.21.15.27:0/1913691061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2500054d0 con 0x7fe260077940 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.527+0000 7fe257e00640 1 -- 172.21.15.27:0/1913691061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250010690 con 0x7fe260077940 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.527+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2601a6d20 con 0x7fe260077940 2024-09-21T07:29:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.527+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601ad480 con 0x7fe260077940 2024-09-21T07:29:57.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.528+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe260077d50 con 0x7fe260077940 2024-09-21T07:29:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.533+0000 7fe257e00640 1 -- 172.21.15.27:0/1913691061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe250004e30 con 0x7fe260077940 2024-09-21T07:29:57.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.534+0000 7fe257e00640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe2440b6700 0x7fe2440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:57.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.534+0000 7fe257e00640 1 -- 172.21.15.27:0/1913691061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fe2500e8110 con 0x7fe260077940 2024-09-21T07:29:57.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.534+0000 7fe25ea00640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe2440b6700 0x7fe2440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:57.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.535+0000 7fe25ea00640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe2440b6700 0x7fe2440b8c10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe24c02aa00 tx=0x7fe24c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:57.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.536+0000 7fe257e00640 1 -- 172.21.15.27:0/1913691061 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe250056710 con 0x7fe260077940 2024-09-21T07:29:57.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.774+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2601a1bf0 con 0x7fe260077940 2024-09-21T07:29:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.775+0000 7fe257e00640 1 -- 172.21.15.27:0/1913691061 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fe2601a1bf0 con 0x7fe260077940 2024-09-21T07:29:57.778 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:29:57.778 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:29:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:29:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe2440b6700 msgr2=0x7fe2440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe2440b6700 0x7fe2440b8c10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe24c02aa00 tx=0x7fe24c005a20 comp rx=0 tx=0).stop 2024-09-21T07:29:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260077940 msgr2=0x7fe2601a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260077940 0x7fe2601a0ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe25000e9f0 tx=0x7fe25000edf0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 shutdown_connections 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe260114740 0x7fe2601a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe2440b6700 0x7fe2440b8c10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe260078290 0x7fe2601a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 --2- 172.21.15.27:0/1913691061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe260077940 0x7fe2601a0ac0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.778+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 >> 172.21.15.27:0/1913691061 conn(0x7fe260101810 msgr2=0x7fe2601167c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.779+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 shutdown_connections 2024-09-21T07:29:57.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:57.779+0000 7fe266200640 1 -- 172.21.15.27:0/1913691061 wait complete. 2024-09-21T07:29:58.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:57 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1913691061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:57 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1913691061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:58.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:57 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1913691061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:29:58.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:29:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:29:59.246 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:29:58 smithi027 ceph-mon[30779]: pgmap v908: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-21T07:29:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:29:58 smithi119 ceph-mon[34519]: pgmap v908: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-21T07:29:59.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:29:58 smithi153 ceph-mon[34747]: pgmap v908: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-21T07:29:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 -- 172.21.15.27:0/635185311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c077940 msgr2=0x7f9a0c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 --2- 172.21.15.27:0/635185311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c077940 0x7f9a0c077d50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009900 tx=0x7f99fc02a410 comp rx=0 tx=0).stop 2024-09-21T07:29:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 -- 172.21.15.27:0/635185311 shutdown_connections 2024-09-21T07:29:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 --2- 172.21.15.27:0/635185311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a0c114740 0x7f9a0c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 --2- 172.21.15.27:0/635185311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a0c078290 0x7f9a0c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 --2- 172.21.15.27:0/635185311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c077940 0x7f9a0c077d50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 -- 172.21.15.27:0/635185311 >> 172.21.15.27:0/635185311 conn(0x7f9a0c101810 msgr2=0x7f9a0c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 -- 172.21.15.27:0/635185311 shutdown_connections 2024-09-21T07:29:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.552+0000 7f9a12800640 1 -- 172.21.15.27:0/635185311 wait complete. 2024-09-21T07:29:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.553+0000 7f9a12800640 1 Processor -- start 2024-09-21T07:29:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.553+0000 7f9a12800640 1 -- start start 2024-09-21T07:29:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a12800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a0c077940 0x7f9a0c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a12800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a0c078290 0x7f9a0c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a12800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c114740 0x7f9a0c1a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a12800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c11a100 con 0x7f9a0c114740 2024-09-21T07:29:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a12800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c119f80 con 0x7f9a0c077940 2024-09-21T07:29:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a12800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c11a280 con 0x7f9a0c078290 2024-09-21T07:29:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c114740 0x7f9a0c1a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c114740 0x7f9a0c1a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59718/0 (socket says 172.21.15.27:59718) 2024-09-21T07:29:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a0c077940 0x7f9a0c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a0be00640 1 -- 172.21.15.27:0/2181100713 learned_addr learned my addr 172.21.15.27:0/2181100713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.554+0000 7f9a03e00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a0c078290 0x7f9a0c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a0be00640 1 -- 172.21.15.27:0/2181100713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a0c078290 msgr2=0x7f9a0c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a0be00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a0c078290 0x7f9a0c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a0be00640 1 -- 172.21.15.27:0/2181100713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a0c077940 msgr2=0x7f9a0c1a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a0be00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a0c077940 0x7f9a0c1a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a0be00640 1 -- 172.21.15.27:0/2181100713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c1a6a60 con 0x7f9a0c114740 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a03e00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a0c078290 0x7f9a0c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a0b400640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a0c077940 0x7f9a0c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a0be00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c114740 0x7f9a0c1a6330 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400eb10 tx=0x7f9a0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a08c00640 1 -- 172.21.15.27:0/2181100713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04004b20 con 0x7f9a0c114740 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a08c00640 1 -- 172.21.15.27:0/2181100713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a04004cc0 con 0x7f9a0c114740 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c1a6d50 con 0x7f9a0c114740 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a08c00640 1 -- 172.21.15.27:0/2181100713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04010670 con 0x7f9a0c114740 2024-09-21T07:29:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.555+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c1ad4b0 con 0x7f9a0c114740 2024-09-21T07:29:59.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.558+0000 7f9a1941e740 1 Processor -- start 2024-09-21T07:29:59.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.559+0000 7f9a08c00640 1 -- 172.21.15.27:0/2181100713 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9a04005020 con 0x7f9a0c114740 2024-09-21T07:29:59.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.559+0000 7f9a1941e740 1 -- start start 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edf2390 0x564c4f1ba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a1941e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564c4f130fb0 con 0x564c4ee13070 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a1941e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564c4f1203e0 con 0x564c4edea1b0 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a1941e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564c4f1241f0 con 0x564c4edf2390 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a03e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1ba2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54160/0 (socket says 172.21.15.27:54160) 2024-09-21T07:29:59.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a0b400640 1 -- 172.21.15.27:0/2753928329 learned_addr learned my addr 172.21.15.27:0/2753928329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:59.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a03e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1b9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59720/0 (socket says 172.21.15.27:59720) 2024-09-21T07:29:59.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a03e00640 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edf2390 0x564c4f1ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.560+0000 7f9a0b400640 1 -- 172.21.15.27:0/2753928329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edf2390 msgr2=0x564c4f1ba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a0b400640 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edf2390 0x564c4f1ba820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a0b400640 1 -- 172.21.15.27:0/2753928329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 msgr2=0x564c4f1b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a0b400640 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1b9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a0b400640 1 -- 172.21.15.27:0/2753928329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4f0eacd0 con 0x564c4edea1b0 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a0b400640 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1ba2e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009230 tx=0x7f99fc004470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3594b15093f3ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a08c00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f99f00b6680 0x7f99f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a00c00640 1 -- 172.21.15.27:0/2753928329 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc035040 con 0x564c4edea1b0 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.561+0000 7f9a08c00640 1 -- 172.21.15.27:0/2181100713 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f9a040e8100 con 0x7f9a0c114740 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.562+0000 7f9a03e00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f99f00b6680 0x7f99f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.562+0000 7f9a00c00640 1 -- 172.21.15.27:0/2753928329 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc004770 con 0x564c4edea1b0 2024-09-21T07:29:59.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.562+0000 7f9a00c00640 1 -- 172.21.15.27:0/2753928329 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc003970 con 0x564c4edea1b0 2024-09-21T07:29:59.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 -- 172.21.15.27:0/2753928329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 msgr2=0x564c4f1ba2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a03e00640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f99f00b6680 0x7f99f00b8b90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f99f40058c0 tx=0x7f99f4005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:59.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1ba2e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009230 tx=0x7f99fc004470 comp rx=0 tx=0).stop 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 -- 172.21.15.27:0/2753928329 shutdown_connections 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edf2390 0x564c4f1ba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1ba2e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2753928329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 -- 172.21.15.27:0/2753928329 >> 172.21.15.27:0/2753928329 conn(0x564c4ee07460 msgr2=0x564c4f1b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 -- 172.21.15.27:0/2753928329 shutdown_connections 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 -- 172.21.15.27:0/2753928329 wait complete. 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.563+0000 7f9a1941e740 1 -- start start 2024-09-21T07:29:59.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 1 Processor -- start 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edf2390 0x564c4f1c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f1d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564c4ef81700 con 0x564c4edf2390 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564c4f0bb580 con 0x564c4edea1b0 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564c4f1c8520 con 0x564c4ee13070 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a1941e740 10 client.0 _ll_register_callbacks cb 0x564c4f0cb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a03e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edf2390 0x564c4f1c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.564+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54168/0 (socket says 172.21.15.27:54168) 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 -- 172.21.15.27:0/2151767106 learned_addr learned my addr 172.21.15.27:0/2151767106 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0be00640 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f1d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 -- 172.21.15.27:0/2151767106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 msgr2=0x564c4f1d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f1d1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 -- 172.21.15.27:0/2151767106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edf2390 msgr2=0x564c4f1c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edf2390 0x564c4f1c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 -- 172.21.15.27:0/2151767106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4ee7e650 con 0x564c4edea1b0 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f9a0b400640 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1c5590 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005ab0 tx=0x7f99fc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459ece3fe3736add server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.565+0000 7f99f9600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.566+0000 7f99f9600640 1 -- 172.21.15.27:0/2151767106 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc052060 con 0x564c4edea1b0 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.566+0000 7f99f9600640 1 -- 172.21.15.27:0/2151767106 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f99fc004970 con 0x564c4edea1b0 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.566+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564c4f0bdb90 con 0x564c4edea1b0 2024-09-21T07:29:59.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.566+0000 7f99f9600640 1 -- 172.21.15.27:0/2151767106 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99fc035040 con 0x564c4edea1b0 2024-09-21T07:29:59.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.566+0000 7f9a1941e740 10 client.30443 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:59.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.566+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c4f0d8950 con 0x564c4edea1b0 2024-09-21T07:29:59.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.566+0000 7f99f9600640 1 -- 172.21.15.27:0/2151767106 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564c4f0bdb90 con 0x564c4edea1b0 2024-09-21T07:29:59.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f99f9600640 1 -- 172.21.15.27:0/2151767106 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f99fc04c070 con 0x564c4edea1b0 2024-09-21T07:29:59.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f9a1941e740 10 client.30443 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f9a1941e740 10 client.30443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f9a1941e740 20 client.30443 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4edd19f0 0x564c4f1d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f9a1941e740 4 client.30443 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c4f132f00 con 0x564c4edd19f0 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.567+0000 7f9a03e00640 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4edd19f0 0x564c4f1d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.568+0000 7f9a03e00640 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4edd19f0 0x564c4f1d74f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8700976c53aa5f3d server_cookie=95e6d920f5d90bd8 in_seq=0 out_seq=0 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.568+0000 7f99f9600640 10 client.30443 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f99f9600640 1 -- 172.21.15.27:0/2151767106 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c4f132f00 con 0x564c4edd19f0 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f99f9600640 10 client.30443 handle_command_reply: tid=0 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 client.30443 shutdown 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 msgr2=0x564c4f1c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1c5590 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005ab0 tx=0x7f99fc02a410 comp rx=0 tx=0).stop 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 shutdown_connections 2024-09-21T07:29:59.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4edd19f0 0x564c4f1d74f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f1d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edf2390 0x564c4f1c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f99f9600640 0 client.30443 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 --2- 172.21.15.27:0/2151767106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edea1b0 0x564c4f1c5590 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 >> 172.21.15.27:0/2151767106 conn(0x564c4ee07460 msgr2=0x564c4f1bcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 shutdown_connections 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 1 -- 172.21.15.27:0/2151767106 wait complete. 2024-09-21T07:29:59.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.569+0000 7f9a1941e740 20 client.30443 trim_cache size 0 max 16384 2024-09-21T07:29:59.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.583+0000 7f9a1941e740 1 Processor -- start 2024-09-21T07:29:59.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.583+0000 7f9a1941e740 1 -- start start 2024-09-21T07:29:59.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edea1b0 0x564c4f1c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f1c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a1941e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1b9d90 con 0x564c4ee13070 2024-09-21T07:29:59.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a1941e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1b7660 con 0x564c4edf2390 2024-09-21T07:29:59.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a1941e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1b54d0 con 0x564c4edea1b0 2024-09-21T07:29:59.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edea1b0 0x564c4f1c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edea1b0 0x564c4f1c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53786/0 (socket says 172.21.15.27:53786) 2024-09-21T07:29:59.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a0b400640 1 -- 172.21.15.27:0/1802446820 learned_addr learned my addr 172.21.15.27:0/1802446820 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:59.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.584+0000 7f9a03e00640 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f1c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f9a0be00640 1 -- 172.21.15.27:0/1802446820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edea1b0 msgr2=0x564c4f1c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f9a0be00640 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edea1b0 0x564c4f1c2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f9a0be00640 1 -- 172.21.15.27:0/1802446820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 msgr2=0x564c4f1c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f9a0be00640 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f1c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f9a0be00640 1 -- 172.21.15.27:0/1802446820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4f1c6850 con 0x564c4ee13070 2024-09-21T07:29:59.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f9a03e00640 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f1c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f9a0be00640 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1da520 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9a040e5410 tx=0x7f9a0400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a335fe374eb45580 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f99fbe00640 1 -- 172.21.15.27:0/1802446820 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a040ad2b0 con 0x564c4ee13070 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f99fbe00640 1 -- 172.21.15.27:0/1802446820 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a040ad650 con 0x564c4ee13070 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.585+0000 7f99fbe00640 1 -- 172.21.15.27:0/1802446820 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a040b2200 con 0x564c4ee13070 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 -- 172.21.15.27:0/1802446820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 msgr2=0x564c4f1da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1da520 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9a040e5410 tx=0x7f9a0400cee0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 -- 172.21.15.27:0/1802446820 shutdown_connections 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4ee13070 0x564c4f1da520 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f1c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1802446820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4edea1b0 0x564c4f1c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 -- 172.21.15.27:0/1802446820 >> 172.21.15.27:0/1802446820 conn(0x564c4ee07460 msgr2=0x564c4f1bb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 -- 172.21.15.27:0/1802446820 shutdown_connections 2024-09-21T07:29:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.586+0000 7f9a1941e740 1 -- 172.21.15.27:0/1802446820 wait complete. 2024-09-21T07:29:59.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 -- start start 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 Processor -- start 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edea1b0 0x564c4f235e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f2363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f23acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1b5b20 con 0x564c4edea1b0 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1b6f90 con 0x564c4edf2390 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1b61f0 con 0x564c4ee13070 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.587+0000 7f9a1941e740 10 client.0 _ll_register_callbacks cb 0x7f99fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f2363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a0b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edea1b0 0x564c4f235e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f2363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54190/0 (socket says 172.21.15.27:54190) 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 -- 172.21.15.27:0/1313515376 learned_addr learned my addr 172.21.15.27:0/1313515376 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:29:59.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a0be00640 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f23acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 -- 172.21.15.27:0/1313515376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 msgr2=0x564c4f23acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f23acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 -- 172.21.15.27:0/1313515376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edea1b0 msgr2=0x564c4f235e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edea1b0 0x564c4f235e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 -- 172.21.15.27:0/1313515376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4f23b330 con 0x564c4edf2390 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a0be00640 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f23acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f9a03e00640 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f2363e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009fd0 tx=0x7f99f4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7da8cf5d34ea92ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.588+0000 7f99f9600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.589+0000 7f99f9600640 1 -- 172.21.15.27:0/1313515376 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400fe80 con 0x564c4edf2390 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.589+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a040ad650 con 0x564c4edf2390 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.589+0000 7f99f9600640 1 -- 172.21.15.27:0/1313515376 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f99f4009040 con 0x564c4edf2390 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.589+0000 7f99f9600640 1 -- 172.21.15.27:0/1313515376 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99f403c050 con 0x564c4edf2390 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.589+0000 7f9a1941e740 10 client.30449 fetch_fsmap learned FSMap version 27 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.589+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c4f2427b0 con 0x564c4edf2390 2024-09-21T07:29:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.589+0000 7f99f9600640 1 -- 172.21.15.27:0/1313515376 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a040ad650 con 0x564c4edf2390 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f99f9600640 1 -- 172.21.15.27:0/1313515376 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f99f402dc70 con 0x564c4edf2390 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a1941e740 10 client.30449 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a1941e740 10 client.30449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a1941e740 20 client.30449 populate_metadata read hostname 'smithi027' 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4eda6da0 0x564c4f244f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a1941e740 4 client.30449 mds_command: new command op to 24740 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 --> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c4f12e9b0 con 0x564c4eda6da0 2024-09-21T07:29:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a0b400640 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4eda6da0 0x564c4f244f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:29:59.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f9a0b400640 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4eda6da0 0x564c4f244f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98e579641f75ecee server_cookie=4ee5343617fa113 in_seq=0 out_seq=0 2024-09-21T07:29:59.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.590+0000 7f99f9600640 10 client.30449 ms_handle_connect on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:59.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f99f9600640 1 -- 172.21.15.27:0/1313515376 <== mds.2 v2:172.21.15.119:6836/4207522386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c4f12e9b0 con 0x564c4eda6da0 2024-09-21T07:29:59.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f99f9600640 10 client.30449 handle_command_reply: tid=0 2024-09-21T07:29:59.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 client.30449 shutdown 2024-09-21T07:29:59.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 msgr2=0x564c4f2363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f2363e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009fd0 tx=0x7f99f4005f90 comp rx=0 tx=0).stop 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 shutdown_connections 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:6836/4207522386,v1:172.21.15.119:6837/4207522386] conn(0x564c4eda6da0 0x564c4f244f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c4ee13070 0x564c4f23acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c4edf2390 0x564c4f2363e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f99f9600640 0 client.30449 ms_handle_reset on v2:172.21.15.119:6836/4207522386 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 --2- 172.21.15.27:0/1313515376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564c4edea1b0 0x564c4f235e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 >> 172.21.15.27:0/1313515376 conn(0x564c4ee07460 msgr2=0x564c4f1bb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 shutdown_connections 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 1 -- 172.21.15.27:0/1313515376 wait complete. 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.591+0000 7f9a1941e740 20 client.30449 trim_cache size 0 max 16384 2024-09-21T07:29:59.596 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f99f00b6680 msgr2=0x7f99f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f99f00b6680 0x7f99f00b8b90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f99f40058c0 tx=0x7f99f4005850 comp rx=0 tx=0).stop 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c114740 msgr2=0x7f9a0c1a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c114740 0x7f9a0c1a6330 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400eb10 tx=0x7f9a0400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 shutdown_connections 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a0c078290 0x7f9a0c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f99f00b6680 0x7f99f00b8b90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a0c114740 0x7f9a0c1a6330 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 --2- 172.21.15.27:0/2181100713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a0c077940 0x7f9a0c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:29:59.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 >> 172.21.15.27:0/2181100713 conn(0x7f9a0c101810 msgr2=0x7f9a0c1167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:29:59.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 shutdown_connections 2024-09-21T07:29:59.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:29:59.592+0000 7f9a12800640 1 -- 172.21.15.27:0/2181100713 wait complete. 2024-09-21T07:30:00.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:30:00.796 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:01.261 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:00 smithi027 ceph-mon[30779]: pgmap v909: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T07:30:01.261 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:00 smithi027 ceph-mon[30779]: overall HEALTH_OK 2024-09-21T07:30:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:00 smithi119 ceph-mon[34519]: pgmap v909: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T07:30:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:00 smithi119 ceph-mon[34519]: overall HEALTH_OK 2024-09-21T07:30:01.357 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:00 smithi153 ceph-mon[34747]: pgmap v909: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T07:30:01.357 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:00 smithi153 ceph-mon[34747]: overall HEALTH_OK 2024-09-21T07:30:01.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 -- 172.21.15.27:0/1945209325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 msgr2=0x7f44f0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 --2- 172.21.15.27:0/1945209325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f0112840 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009ca0 tx=0x7f44e802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 -- 172.21.15.27:0/1945209325 shutdown_connections 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 --2- 172.21.15.27:0/1945209325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f010b6c0 0x7f44f010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 --2- 172.21.15.27:0/1945209325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f0112840 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 --2- 172.21.15.27:0/1945209325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0105d80 0x7f44f0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 -- 172.21.15.27:0/1945209325 >> 172.21.15.27:0/1945209325 conn(0x7f44f01017f0 msgr2=0x7f44f0103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 -- 172.21.15.27:0/1945209325 shutdown_connections 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.445+0000 7f44f5e00640 1 -- 172.21.15.27:0/1945209325 wait complete. 2024-09-21T07:30:01.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.446+0000 7f44f5e00640 1 Processor -- start 2024-09-21T07:30:01.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44f5e00640 1 -- start start 2024-09-21T07:30:01.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0105d80 0x7f44f019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f010b6c0 0x7f44f019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44f5e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0115cd0 con 0x7f44f010c010 2024-09-21T07:30:01.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44f5e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0115b50 con 0x7f44f0105d80 2024-09-21T07:30:01.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44f5e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0115e50 con 0x7f44f010b6c0 2024-09-21T07:30:01.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.447+0000 7f44eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0105d80 0x7f44f019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f01a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59762/0 (socket says 172.21.15.27:59762) 2024-09-21T07:30:01.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 -- 172.21.15.27:0/471965731 learned_addr learned my addr 172.21.15.27:0/471965731 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:01.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44e7e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f010b6c0 0x7f44f019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 -- 172.21.15.27:0/471965731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f010b6c0 msgr2=0x7f44f019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f010b6c0 0x7f44f019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 -- 172.21.15.27:0/471965731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0105d80 msgr2=0x7f44f019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0105d80 0x7f44f019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 -- 172.21.15.27:0/471965731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f01a25d0 con 0x7f44f010c010 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.448+0000 7f44ef400640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f01a1ea0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005cc0 tx=0x7f44e8030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.449+0000 7f44f4a00640 1 -- 172.21.15.27:0/471965731 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8004040 con 0x7f44f010c010 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.449+0000 7f44f4a00640 1 -- 172.21.15.27:0/471965731 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e8040070 con 0x7f44f010c010 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.449+0000 7f44f4a00640 1 -- 172.21.15.27:0/471965731 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8045b10 con 0x7f44f010c010 2024-09-21T07:30:01.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.449+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f01a2860 con 0x7f44f010c010 2024-09-21T07:30:01.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.449+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f01a9020 con 0x7f44f010c010 2024-09-21T07:30:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.452+0000 7f44fc9c6740 1 Processor -- start 2024-09-21T07:30:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44f4a00640 1 -- 172.21.15.27:0/471965731 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f44e8045cd0 con 0x7f44f010c010 2024-09-21T07:30:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44fc9c6740 1 -- start start 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f1b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ea26e91fb0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ea26e813e0 con 0x55ea26b53390 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.453+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea26e851f0 con 0x55ea26b74070 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f1b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f1b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59774/0 (socket says 172.21.15.27:59774) 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 -- 172.21.15.27:0/696589855 learned_addr learned my addr 172.21.15.27:0/696589855 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44e7e00640 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44e7e00640 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 -- 172.21.15.27:0/696589855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 msgr2=0x55ea26f1ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 -- 172.21.15.27:0/696589855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 msgr2=0x55ea26f1b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 -- 172.21.15.27:0/696589855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea26e4bcd0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44e7e00640 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:01.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.454+0000 7f44eea00640 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f1b2e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009cf0 tx=0x7f44e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0826636c00379bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:01.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.455+0000 7f44dfe00640 1 -- 172.21.15.27:0/696589855 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0004230 con 0x55ea26b4b1b0 2024-09-21T07:30:01.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.455+0000 7f44dfe00640 1 -- 172.21.15.27:0/696589855 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e00043d0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.455+0000 7f44dfe00640 1 -- 172.21.15.27:0/696589855 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0019820 con 0x55ea26b4b1b0 2024-09-21T07:30:01.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.455+0000 7f44f4a00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f44d40b6680 0x7f44d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.455+0000 7f44f4a00640 1 -- 172.21.15.27:0/471965731 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f44e8108b40 con 0x7f44f010c010 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.455+0000 7f44e7e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f44d40b6680 0x7f44d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 -- 172.21.15.27:0/696589855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 msgr2=0x55ea26f1b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f1b2e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009cf0 tx=0x7f44e0011810 comp rx=0 tx=0).stop 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 -- 172.21.15.27:0/696589855 shutdown_connections 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f1b2e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/696589855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 -- 172.21.15.27:0/696589855 >> 172.21.15.27:0/696589855 conn(0x55ea26b68460 msgr2=0x55ea26f18c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 -- 172.21.15.27:0/696589855 shutdown_connections 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44fc9c6740 1 -- 172.21.15.27:0/696589855 wait complete. 2024-09-21T07:30:01.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.456+0000 7f44e7e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f44d40b6680 0x7f44d40b8b90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f44d800bfd0 tx=0x7f44d800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:01.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.457+0000 7f44fc9c6740 1 -- start start 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.457+0000 7f44fc9c6740 1 Processor -- start 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b53390 0x55ea26f26a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ea26ce2700 con 0x55ea26b53390 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ea26e1c580 con 0x55ea26b4b1b0 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea26f29580 con 0x55ea26b74070 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44fc9c6740 10 client.0 _ll_register_callbacks cb 0x55ea26e2c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44e7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b53390 0x55ea26f26a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44ef400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54220/0 (socket says 172.21.15.27:54220) 2024-09-21T07:30:01.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 -- 172.21.15.27:0/529552222 learned_addr learned my addr 172.21.15.27:0/529552222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:01.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 -- 172.21.15.27:0/529552222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 msgr2=0x55ea26f328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 -- 172.21.15.27:0/529552222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b53390 msgr2=0x55ea26f26a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b53390 0x55ea26f26a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.458+0000 7f44eea00640 1 -- 172.21.15.27:0/529552222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea26bdf650 con 0x55ea26b4b1b0 2024-09-21T07:30:01.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.459+0000 7f44eea00640 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44e0003f50 tx=0x7f44e0005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe11c63689b8ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:01.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.459+0000 7f44dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T07:30:01.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.459+0000 7f44dcc00640 1 -- 172.21.15.27:0/529552222 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0027070 con 0x55ea26b4b1b0 2024-09-21T07:30:01.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.459+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea26e1eb90 con 0x55ea26b4b1b0 2024-09-21T07:30:01.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.459+0000 7f44dcc00640 1 -- 172.21.15.27:0/529552222 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f44e0005450 con 0x55ea26b4b1b0 2024-09-21T07:30:01.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44dcc00640 1 -- 172.21.15.27:0/529552222 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44e0033050 con 0x55ea26b4b1b0 2024-09-21T07:30:01.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 10 client.30461 fetch_fsmap learned FSMap version 27 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea26e39950 con 0x55ea26b4b1b0 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44dcc00640 1 -- 172.21.15.27:0/529552222 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ea26e1eb90 con 0x55ea26b4b1b0 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44dcc00640 1 -- 172.21.15.27:0/529552222 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f44e00048c0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 10 client.30461 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 10 client.30461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 20 client.30461 populate_metadata read hostname 'smithi027' 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b329f0 0x55ea26f385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 4 client.30461 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.460+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea26e93f00 con 0x55ea26b329f0 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.461+0000 7f44e7e00640 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b329f0 0x55ea26f385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.461+0000 7f44dcc00640 10 client.30461 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:30:01.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.461+0000 7f44e7e00640 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b329f0 0x55ea26f385e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9b20c02099fee5e server_cookie=323ac8029957cf0b in_seq=0 out_seq=0 2024-09-21T07:30:01.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.462+0000 7f44dcc00640 1 -- 172.21.15.27:0/529552222 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ea26e93f00 con 0x55ea26b329f0 2024-09-21T07:30:01.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.462+0000 7f44dcc00640 10 client.30461 handle_command_reply: tid=0 2024-09-21T07:30:01.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.462+0000 7f44fc9c6740 1 client.30461 shutdown 2024-09-21T07:30:01.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.462+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b4b1b0 msgr2=0x55ea26f26550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.462+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44e0003f50 tx=0x7f44e0005d30 comp rx=0 tx=0).stop 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 shutdown_connections 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b329f0 0x55ea26f385e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b53390 0x55ea26f26a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44dcc00640 0 client.30461 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/529552222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26550 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 >> 172.21.15.27:0/529552222 conn(0x55ea26b68460 msgr2=0x55ea26f1df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 shutdown_connections 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 1 -- 172.21.15.27:0/529552222 wait complete. 2024-09-21T07:30:01.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.463+0000 7f44fc9c6740 20 client.30461 trim_cache size 0 max 16384 2024-09-21T07:30:01.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.477+0000 7f44fc9c6740 1 Processor -- start 2024-09-21T07:30:01.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.477+0000 7f44fc9c6740 1 -- start start 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b58c0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b3190 con 0x55ea26b53390 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b1000 con 0x55ea26b74070 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59790/0 (socket says 172.21.15.27:59790) 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 -- 172.21.15.27:0/1696026398 learned_addr learned my addr 172.21.15.27:0/1696026398 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44e7e00640 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44ef400640 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 -- 172.21.15.27:0/1696026398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 msgr2=0x55ea26f1e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 -- 172.21.15.27:0/1696026398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 msgr2=0x55ea26f1dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44eea00640 1 -- 172.21.15.27:0/1696026398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea26f1d3b0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.478+0000 7f44ef400640 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:30:01.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.479+0000 7f44eea00640 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26b30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f44e0011850 tx=0x7f44e0004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a06217e68ea28e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:01.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.479+0000 7f44df400640 1 -- 172.21.15.27:0/1696026398 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0027030 con 0x55ea26b4b1b0 2024-09-21T07:30:01.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.479+0000 7f44df400640 1 -- 172.21.15.27:0/1696026398 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e003c070 con 0x55ea26b4b1b0 2024-09-21T07:30:01.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.479+0000 7f44df400640 1 -- 172.21.15.27:0/1696026398 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0019940 con 0x55ea26b4b1b0 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 -- 172.21.15.27:0/1696026398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 msgr2=0x55ea26f26b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26b30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f44e0011850 tx=0x7f44e0004350 comp rx=0 tx=0).stop 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 -- 172.21.15.27:0/1696026398 shutdown_connections 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f1e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f1dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/1696026398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f26b30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 -- 172.21.15.27:0/1696026398 >> 172.21.15.27:0/1696026398 conn(0x55ea26b68460 msgr2=0x55ea26f24dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 -- 172.21.15.27:0/1696026398 shutdown_connections 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.480+0000 7f44fc9c6740 1 -- 172.21.15.27:0/1696026398 wait complete. 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 -- start start 2024-09-21T07:30:01.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 Processor -- start 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f2ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f2a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f2a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b1650 con 0x55ea26b4b1b0 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b2ac0 con 0x55ea26b53390 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.481+0000 7f44fc9c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b1d20 con 0x55ea26b74070 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44fc9c6740 10 client.0 _ll_register_callbacks cb 0x7f44e0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f2ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f2ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59806/0 (socket says 172.21.15.27:59806) 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 -- 172.21.15.27:0/2863878068 learned_addr learned my addr 172.21.15.27:0/2863878068 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44e7e00640 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f2a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44ef400640 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f2a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 -- 172.21.15.27:0/2863878068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 msgr2=0x55ea26f2a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f2a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 -- 172.21.15.27:0/2863878068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 msgr2=0x55ea26f2a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f2a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.482+0000 7f44eea00640 1 -- 172.21.15.27:0/2863878068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea26f2ac40 con 0x55ea26b4b1b0 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44eea00640 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f2ed10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f44e0004c90 tx=0x7f44e00099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb42a00bd9368db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44dcc00640 1 -- 172.21.15.27:0/2863878068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0027040 con 0x55ea26b4b1b0 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44dcc00640 1 -- 172.21.15.27:0/2863878068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f44e0004610 con 0x55ea26b4b1b0 2024-09-21T07:30:01.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e003c070 con 0x55ea26b4b1b0 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44dcc00640 1 -- 172.21.15.27:0/2863878068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44e004e050 con 0x55ea26b4b1b0 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44fc9c6740 10 client.20454 fetch_fsmap learned FSMap version 27 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.483+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea26fa2ab0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44dcc00640 1 -- 172.21.15.27:0/2863878068 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e003c070 con 0x55ea26b4b1b0 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44dcc00640 1 -- 172.21.15.27:0/2863878068 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f44e00035f0 con 0x55ea26b4b1b0 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44fc9c6740 10 client.20454 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44fc9c6740 10 client.20454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44fc9c6740 20 client.20454 populate_metadata read hostname 'smithi027' 2024-09-21T07:30:01.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b07da0 0x55ea26fa51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44fc9c6740 4 client.20454 mds_command: new command op to 24748 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 --> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea26e8f9b0 con 0x55ea26b07da0 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.484+0000 7f44e7e00640 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b07da0 0x55ea26fa51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44e7e00640 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b07da0 0x55ea26fa51d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c254a7487e15cc48 server_cookie=2aa8ed80c6616d3 in_seq=0 out_seq=0 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44dcc00640 10 client.20454 ms_handle_connect on v2:172.21.15.27:6838/609554839 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44dcc00640 1 -- 172.21.15.27:0/2863878068 <== mds.1 v2:172.21.15.27:6838/609554839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea26e8f9b0 con 0x55ea26b07da0 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44dcc00640 10 client.20454 handle_command_reply: tid=0 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44fc9c6740 1 client.20454 shutdown 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 msgr2=0x55ea26f2ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f2ed10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f44e0004c90 tx=0x7f44e00099a0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 shutdown_connections 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:6838/609554839,v1:172.21.15.27:6839/609554839] conn(0x55ea26b07da0 0x55ea26fa51d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.485+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea26b74070 0x55ea26f2a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.486+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea26b53390 0x55ea26f2a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.486+0000 7f44fc9c6740 1 --2- 172.21.15.27:0/2863878068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ea26b4b1b0 0x55ea26f2ed10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.486+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 >> 172.21.15.27:0/2863878068 conn(0x55ea26b68460 msgr2=0x55ea26f1dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.486+0000 7f44dcc00640 0 client.20454 ms_handle_reset on v2:172.21.15.27:6838/609554839 2024-09-21T07:30:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.486+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 shutdown_connections 2024-09-21T07:30:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.486+0000 7f44fc9c6740 1 -- 172.21.15.27:0/2863878068 wait complete. 2024-09-21T07:30:01.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.486+0000 7f44fc9c6740 20 client.20454 trim_cache size 0 max 16384 2024-09-21T07:30:01.490 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f44d40b6680 msgr2=0x7f44d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f44d40b6680 0x7f44d40b8b90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f44d800bfd0 tx=0x7f44d800b390 comp rx=0 tx=0).stop 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 msgr2=0x7f44f01a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f01a1ea0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005cc0 tx=0x7f44e8030d70 comp rx=0 tx=0).stop 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 shutdown_connections 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44f010b6c0 0x7f44f019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f44d40b6680 0x7f44d40b8b90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44f010c010 0x7f44f01a1ea0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 --2- 172.21.15.27:0/471965731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0105d80 0x7f44f019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 >> 172.21.15.27:0/471965731 conn(0x7f44f01017f0 msgr2=0x7f44f010ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:01.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 shutdown_connections 2024-09-21T07:30:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:01.487+0000 7f44f5e00640 1 -- 172.21.15.27:0/471965731 wait complete. 2024-09-21T07:30:02.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:30:02.803 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:03.255 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:02 smithi027 ceph-mon[30779]: pgmap v910: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-09-21T07:30:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:02 smithi119 ceph-mon[34519]: pgmap v910: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-09-21T07:30:03.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:02 smithi153 ceph-mon[34747]: pgmap v910: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-09-21T07:30:03.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 -- 172.21.15.27:0/2728408626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410be40 msgr2=0x7f7954112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 --2- 172.21.15.27:0/2728408626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410be40 0x7f7954112670 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f794c009ca0 tx=0x7f794c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 -- 172.21.15.27:0/2728408626 shutdown_connections 2024-09-21T07:30:03.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 --2- 172.21.15.27:0/2728408626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795410b4f0 0x7f795410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 --2- 172.21.15.27:0/2728408626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410be40 0x7f7954112670 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 --2- 172.21.15.27:0/2728408626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7954105bb0 0x7f7954105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 -- 172.21.15.27:0/2728408626 >> 172.21.15.27:0/2728408626 conn(0x7f7954101680 msgr2=0x7f7954103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:03.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 -- 172.21.15.27:0/2728408626 shutdown_connections 2024-09-21T07:30:03.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.442+0000 7f7953e00640 1 -- 172.21.15.27:0/2728408626 wait complete. 2024-09-21T07:30:03.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 Processor -- start 2024-09-21T07:30:03.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 -- start start 2024-09-21T07:30:03.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7954105bb0 0x7f795419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410b4f0 0x7f795419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f795410be40 0x7f79541a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7954115b50 con 0x7f795410b4f0 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79541159d0 con 0x7f795410be40 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.444+0000 7f7953e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7954115cd0 con 0x7f7954105bb0 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7954105bb0 0x7f795419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410b4f0 0x7f795419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7954105bb0 0x7f795419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:53842/0 (socket says 172.21.15.27:53842) 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410b4f0 0x7f795419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59820/0 (socket says 172.21.15.27:59820) 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952000640 1 -- 172.21.15.27:0/895988743 learned_addr learned my addr 172.21.15.27:0/895988743 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7953400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f795410be40 0x7f79541a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952a00640 1 -- 172.21.15.27:0/895988743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f795410be40 msgr2=0x7f79541a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952a00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f795410be40 0x7f79541a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952a00640 1 -- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410b4f0 msgr2=0x7f795419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952a00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410b4f0 0x7f795419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952a00640 1 -- 172.21.15.27:0/895988743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79541a2430 con 0x7f7954105bb0 2024-09-21T07:30:03.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.445+0000 7f7952000640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410b4f0 0x7f795419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:03.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.446+0000 7f7952a00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7954105bb0 0x7f795419c4c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9f0 tx=0x7f794400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.446+0000 7f794b400640 1 -- 172.21.15.27:0/895988743 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794400ccc0 con 0x7f7954105bb0 2024-09-21T07:30:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.446+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79541a2720 con 0x7f7954105bb0 2024-09-21T07:30:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.446+0000 7f794b400640 1 -- 172.21.15.27:0/895988743 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79440108e0 con 0x7f7954105bb0 2024-09-21T07:30:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.446+0000 7f794b400640 1 -- 172.21.15.27:0/895988743 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944018880 con 0x7f7954105bb0 2024-09-21T07:30:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.446+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79541a8e80 con 0x7f7954105bb0 2024-09-21T07:30:03.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.449+0000 7f79605e6740 1 Processor -- start 2024-09-21T07:30:03.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.449+0000 7f79605e6740 1 -- start start 2024-09-21T07:30:03.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f79605e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f79605e6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f79605e6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baacdf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f79605e6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557baac55fb0 con 0x557baa917390 2024-09-21T07:30:03.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f79605e6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557baac453e0 con 0x557baa90f1b0 2024-09-21T07:30:03.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f79605e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557baac491f0 con 0x557baa938070 2024-09-21T07:30:03.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f7952000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baacdf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f7953400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f7952000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baacdf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59824/0 (socket says 172.21.15.27:59824) 2024-09-21T07:30:03.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.450+0000 7f7952000640 1 -- 172.21.15.27:0/957443657 learned_addr learned my addr 172.21.15.27:0/957443657 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:03.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f794b400640 1 -- 172.21.15.27:0/895988743 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7944010430 con 0x7f7954105bb0 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f7952000640 1 -- 172.21.15.27:0/957443657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 msgr2=0x557baacded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f7952000640 1 --2- 172.21.15.27:0/957443657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f7952000640 1 -- 172.21.15.27:0/957443657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 msgr2=0x557baacdf270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f7952000640 1 --2- 172.21.15.27:0/957443657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdf270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f7952000640 1 -- 172.21.15.27:0/957443657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557baac0fcd0 con 0x557baa917390 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f7952000640 1 --2- 172.21.15.27:0/957443657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baacdf7b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7940011540 tx=0x7f7940011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f4294661b5c59b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.451+0000 7f793fe00640 1 -- 172.21.15.27:0/957443657 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940005020 con 0x557baa917390 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f793fe00640 1 -- 172.21.15.27:0/957443657 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79400047a0 con 0x557baa917390 2024-09-21T07:30:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f793fe00640 1 -- 172.21.15.27:0/957443657 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794000f9b0 con 0x557baa917390 2024-09-21T07:30:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f79605e6740 1 -- 172.21.15.27:0/957443657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 msgr2=0x557baacdf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f79605e6740 1 --2- 172.21.15.27:0/957443657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baacdf7b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7940011540 tx=0x7f7940011a00 comp rx=0 tx=0).stop 2024-09-21T07:30:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f79605e6740 1 -- 172.21.15.27:0/957443657 shutdown_connections 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f79605e6740 1 --2- 172.21.15.27:0/957443657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baacdf7b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f79605e6740 1 --2- 172.21.15.27:0/957443657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f79605e6740 1 --2- 172.21.15.27:0/957443657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.452+0000 7f79605e6740 1 -- 172.21.15.27:0/957443657 >> 172.21.15.27:0/957443657 conn(0x557baa92c460 msgr2=0x557baacdcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.453+0000 7f794b400640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79380b6680 0x7f79380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.453+0000 7f79605e6740 1 -- 172.21.15.27:0/957443657 shutdown_connections 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.453+0000 7f79605e6740 1 -- 172.21.15.27:0/957443657 wait complete. 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.453+0000 7f794b400640 1 -- 172.21.15.27:0/895988743 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f79440e7e00 con 0x7f7954105bb0 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.453+0000 7f7952000640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79380b6680 0x7f79380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.453+0000 7f79605e6740 1 -- start start 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 1 Processor -- start 2024-09-21T07:30:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f7952000640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79380b6680 0x7f79380b8b90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7940009e90 tx=0x7f7940004ea0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baaceadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557baaaa6700 con 0x557baa90f1b0 2024-09-21T07:30:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557baabe0580 con 0x557baa917390 2024-09-21T07:30:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557baaced510 con 0x557baa938070 2024-09-21T07:30:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.454+0000 7f79605e6740 10 client.0 _ll_register_callbacks cb 0x557baabf0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacea870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59834/0 (socket says 172.21.15.27:59834) 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 -- 172.21.15.27:0/2559200135 learned_addr learned my addr 172.21.15.27:0/2559200135 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952000640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baaceadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7953400640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 -- 172.21.15.27:0/2559200135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 msgr2=0x557baacf6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 -- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 msgr2=0x557baaceadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baaceadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952a00640 1 -- 172.21.15.27:0/2559200135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557baa9a3650 con 0x557baa90f1b0 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.455+0000 7f7952000640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baaceadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f7952a00640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacea870 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f79440e57a0 tx=0x7f79440ad0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e893a89449f6377b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f793cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:30:03.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f793cc00640 1 -- 172.21.15.27:0/2559200135 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440b2380 con 0x557baa90f1b0 2024-09-21T07:30:03.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f793cc00640 1 -- 172.21.15.27:0/2559200135 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f79440aebd0 con 0x557baa90f1b0 2024-09-21T07:30:03.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557baabe2b90 con 0x557baa90f1b0 2024-09-21T07:30:03.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f793cc00640 1 -- 172.21.15.27:0/2559200135 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f79440b72c0 con 0x557baa90f1b0 2024-09-21T07:30:03.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f79605e6740 10 client.20466 fetch_fsmap learned FSMap version 27 2024-09-21T07:30:03.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557baabfd950 con 0x557baa90f1b0 2024-09-21T07:30:03.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.456+0000 7f793cc00640 1 -- 172.21.15.27:0/2559200135 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557baabe2b90 con 0x557baa90f1b0 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f793cc00640 1 -- 172.21.15.27:0/2559200135 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f79440ac790 con 0x557baa90f1b0 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f79605e6740 10 client.20466 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f79605e6740 10 client.20466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f79605e6740 20 client.20466 populate_metadata read hostname 'smithi027' 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f79605e6740 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8f69f0 0x557baacfc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f79605e6740 4 client.20466 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557baac57f00 con 0x557baa8f69f0 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.457+0000 7f7952000640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8f69f0 0x557baacfc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.458+0000 7f7952000640 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8f69f0 0x557baacfc720 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5278ed95b8c0281 server_cookie=720026ac87d7308b in_seq=0 out_seq=0 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.458+0000 7f793cc00640 10 client.20466 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:30:03.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f793cc00640 1 -- 172.21.15.27:0/2559200135 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557baac57f00 con 0x557baa8f69f0 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f793cc00640 10 client.20466 handle_command_reply: tid=0 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 client.20466 shutdown 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 msgr2=0x557baacea870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacea870 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f79440e57a0 tx=0x7f79440ad0c0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 shutdown_connections 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8f69f0 0x557baacfc720 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baaceadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 --2- 172.21.15.27:0/2559200135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacea870 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 >> 172.21.15.27:0/2559200135 conn(0x557baa92c460 msgr2=0x557baace1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.459+0000 7f793cc00640 0 client.20466 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.460+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 shutdown_connections 2024-09-21T07:30:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.460+0000 7f79605e6740 1 -- 172.21.15.27:0/2559200135 wait complete. 2024-09-21T07:30:03.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.460+0000 7f79605e6740 20 client.20466 trim_cache size 0 max 16384 2024-09-21T07:30:03.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 Processor -- start 2024-09-21T07:30:03.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 -- start start 2024-09-21T07:30:03.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baace0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b5720 con 0x557baa917390 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b2ff0 con 0x557baa90f1b0 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.474+0000 7f79605e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b0e60 con 0x557baa938070 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baace0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54276/0 (socket says 172.21.15.27:54276) 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 -- 172.21.15.27:0/666791114 learned_addr learned my addr 172.21.15.27:0/666791114 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:03.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7953400640 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 -- 172.21.15.27:0/666791114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 msgr2=0x557baacf6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 -- 172.21.15.27:0/666791114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 msgr2=0x557baace0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baace0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.475+0000 7f7952a00640 1 -- 172.21.15.27:0/666791114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557baaceb930 con 0x557baa90f1b0 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.476+0000 7f7952a00640 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdbbc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f79440ad650 tx=0x7f79440ad1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1743039a0471684 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.476+0000 7f793f400640 1 -- 172.21.15.27:0/666791114 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440aebd0 con 0x557baa90f1b0 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.476+0000 7f793f400640 1 -- 172.21.15.27:0/666791114 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79440aed70 con 0x557baa90f1b0 2024-09-21T07:30:03.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.476+0000 7f793f400640 1 -- 172.21.15.27:0/666791114 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440ac860 con 0x557baa90f1b0 2024-09-21T07:30:03.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 -- 172.21.15.27:0/666791114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 msgr2=0x557baacdbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdbbc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f79440ad650 tx=0x7f79440ad1f0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 -- 172.21.15.27:0/666791114 shutdown_connections 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baacf6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa917390 0x557baace0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 --2- 172.21.15.27:0/666791114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa90f1b0 0x557baacdbbc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 -- 172.21.15.27:0/666791114 >> 172.21.15.27:0/666791114 conn(0x557baa92c460 msgr2=0x557baacdfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 -- 172.21.15.27:0/666791114 shutdown_connections 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 -- 172.21.15.27:0/666791114 wait complete. 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.477+0000 7f79605e6740 1 -- start start 2024-09-21T07:30:03.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 1 Processor -- start 2024-09-21T07:30:03.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacf27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baacf2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baaced750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b14b0 con 0x557baa90f1b0 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b2920 con 0x557baa917390 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b1b80 con 0x557baa938070 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.478+0000 7f79605e6740 10 client.0 _ll_register_callbacks cb 0x7f79440a97f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacf27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacf27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59854/0 (socket says 172.21.15.27:59854) 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 -- 172.21.15.27:0/41971475 learned_addr learned my addr 172.21.15.27:0/41971475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952000640 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baacf2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7953400640 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baaced750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 -- 172.21.15.27:0/41971475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 msgr2=0x557baaced750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baaced750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 -- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 msgr2=0x557baacf2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baacf2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 -- 172.21.15.27:0/41971475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557baacf3490 con 0x557baa90f1b0 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f7952a00640 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacf27a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f79440057a0 tx=0x7f794400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ca683b35db0847 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.479+0000 7f793cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f793cc00640 1 -- 172.21.15.27:0/41971475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440aebd0 con 0x557baa90f1b0 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f793cc00640 1 -- 172.21.15.27:0/41971475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f79440cf050 con 0x557baa90f1b0 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79440aed70 con 0x557baa90f1b0 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f793cc00640 1 -- 172.21.15.27:0/41971475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f79440b3810 con 0x557baa90f1b0 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f79605e6740 10 client.20472 fetch_fsmap learned FSMap version 27 2024-09-21T07:30:03.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557baad67620 con 0x557baa90f1b0 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f793cc00640 1 -- 172.21.15.27:0/41971475 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79440aed70 con 0x557baa90f1b0 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f793cc00640 1 -- 172.21.15.27:0/41971475 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f79440c3a50 con 0x557baa90f1b0 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f79605e6740 10 client.20472 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f79605e6740 10 client.20472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.480+0000 7f79605e6740 20 client.20472 populate_metadata read hostname 'smithi027' 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.481+0000 7f79605e6740 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8cbda0 0x557baad69b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.481+0000 7f79605e6740 4 client.20472 mds_command: new command op to 24773 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.481+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 --> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557baac539b0 con 0x557baa8cbda0 2024-09-21T07:30:03.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.481+0000 7f7952000640 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8cbda0 0x557baad69b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.481+0000 7f7952000640 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8cbda0 0x557baad69b60 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a2069f4c39dd5f server_cookie=d95b117cee497493 in_seq=0 out_seq=0 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.481+0000 7f793cc00640 10 client.20472 ms_handle_connect on v2:172.21.15.119:6838/1565967470 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f793cc00640 1 -- 172.21.15.27:0/41971475 <== mds.0 v2:172.21.15.119:6838/1565967470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557baac539b0 con 0x557baa8cbda0 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f793cc00640 10 client.20472 handle_command_reply: tid=0 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 client.20472 shutdown 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 msgr2=0x557baacf27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacf27a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f79440057a0 tx=0x7f794400cef0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 shutdown_connections 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:6838/1565967470,v1:172.21.15.119:6839/1565967470] conn(0x557baa8cbda0 0x557baad69b60 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557baa938070 0x557baaced750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557baa917390 0x557baacf2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 --2- 172.21.15.27:0/41971475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557baa90f1b0 0x557baacf27a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f793cc00640 0 client.20472 ms_handle_reset on v2:172.21.15.119:6838/1565967470 2024-09-21T07:30:03.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 >> 172.21.15.27:0/41971475 conn(0x557baa92c460 msgr2=0x557baacdb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:03.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 shutdown_connections 2024-09-21T07:30:03.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.482+0000 7f79605e6740 1 -- 172.21.15.27:0/41971475 wait complete. 2024-09-21T07:30:03.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f79605e6740 20 client.20472 trim_cache size 0 max 16384 2024-09-21T07:30:03.487 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79380b6680 msgr2=0x7f79380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79380b6680 0x7f79380b8b90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7940009e90 tx=0x7f7940004ea0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7954105bb0 msgr2=0x7f795419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7954105bb0 0x7f795419c4c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9f0 tx=0x7f794400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 shutdown_connections 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f79380b6680 0x7f79380b8b90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f795410be40 0x7f79541a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795410b4f0 0x7f795419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.483+0000 7f7953e00640 1 --2- 172.21.15.27:0/895988743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7954105bb0 0x7f795419c4c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.484+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 >> 172.21.15.27:0/895988743 conn(0x7f7954101680 msgr2=0x7f7954109b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:03.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.484+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 shutdown_connections 2024-09-21T07:30:03.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:03.484+0000 7f7953e00640 1 -- 172.21.15.27:0/895988743 wait complete. 2024-09-21T07:30:04.458 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T07:30:04.459 INFO:tasks.fwd_scrub:done joining 2024-09-21T07:30:04.459 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T07:30:04.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:30:04.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 -- 172.21.15.27:0/2671180153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc10db70 msgr2=0x7f02fc110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:04.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 --2- 172.21.15.27:0/2671180153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc10db70 0x7f02fc110060 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009ca0 tx=0x7f02f402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 -- 172.21.15.27:0/2671180153 shutdown_connections 2024-09-21T07:30:04.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 --2- 172.21.15.27:0/2671180153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc10db70 0x7f02fc110060 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 --2- 172.21.15.27:0/2671180153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc105060 0x7f02fc10d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 --2- 172.21.15.27:0/2671180153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02fc104710 0x7f02fc104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 -- 172.21.15.27:0/2671180153 >> 172.21.15.27:0/2671180153 conn(0x7f02fc0fe1e0 msgr2=0x7f02fc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:04.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 -- 172.21.15.27:0/2671180153 shutdown_connections 2024-09-21T07:30:04.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.706+0000 7f0302000640 1 -- 172.21.15.27:0/2671180153 wait complete. 2024-09-21T07:30:04.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.707+0000 7f0302000640 1 Processor -- start 2024-09-21T07:30:04.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.707+0000 7f0302000640 1 -- start start 2024-09-21T07:30:04.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0302000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc104710 0x7f02fc102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0302000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc105060 0x7f02fc102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0302000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02fc10db70 0x7f02fc102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0302000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc113830 con 0x7f02fc105060 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0302000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1136b0 con 0x7f02fc104710 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0302000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1139b0 con 0x7f02fc10db70 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f02fbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc105060 0x7f02fc102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f02fbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc105060 0x7f02fc102980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59868/0 (socket says 172.21.15.27:59868) 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f02fbe00640 1 -- 172.21.15.27:0/3896530308 learned_addr learned my addr 172.21.15.27:0/3896530308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0300c00640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc104710 0x7f02fc102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.708+0000 7f0301600640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02fc10db70 0x7f02fc102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02fbe00640 1 -- 172.21.15.27:0/3896530308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02fc10db70 msgr2=0x7f02fc102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02fbe00640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02fc10db70 0x7f02fc102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02fbe00640 1 -- 172.21.15.27:0/3896530308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc104710 msgr2=0x7f02fc102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:04.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02fbe00640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc104710 0x7f02fc102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02fbe00640 1 -- 172.21.15.27:0/3896530308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc1a8460 con 0x7f02fc105060 2024-09-21T07:30:04.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f0300c00640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc104710 0x7f02fc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:04.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02fbe00640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc105060 0x7f02fc102980 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e9f0 tx=0x7f02e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:04.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02f9600640 1 -- 172.21.15.27:0/3896530308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e800ccc0 con 0x7f02fc105060 2024-09-21T07:30:04.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc1a8750 con 0x7f02fc105060 2024-09-21T07:30:04.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02f9600640 1 -- 172.21.15.27:0/3896530308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e80108e0 con 0x7f02fc105060 2024-09-21T07:30:04.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc1a8cf0 con 0x7f02fc105060 2024-09-21T07:30:04.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.709+0000 7f02f9600640 1 -- 172.21.15.27:0/3896530308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8018800 con 0x7f02fc105060 2024-09-21T07:30:04.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.710+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc104b20 con 0x7f02fc105060 2024-09-21T07:30:04.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.715+0000 7f02f9600640 1 -- 172.21.15.27:0/3896530308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f02e8010430 con 0x7f02fc105060 2024-09-21T07:30:04.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.716+0000 7f02f9600640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02e40b66b0 0x7f02e40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:04.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.717+0000 7f02f9600640 1 -- 172.21.15.27:0/3896530308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f02e80e82d0 con 0x7f02fc105060 2024-09-21T07:30:04.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.717+0000 7f0300c00640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02e40b66b0 0x7f02e40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:04.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.717+0000 7f0300c00640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02e40b66b0 0x7f02e40b8bc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f02fc103e90 tx=0x7f02f000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:04.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.719+0000 7f02f9600640 1 -- 172.21.15.27:0/3896530308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02e80ae410 con 0x7f02fc105060 2024-09-21T07:30:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.919+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02fc1a90b0 con 0x7f02fc105060 2024-09-21T07:30:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.920+0000 7f02f9600640 1 -- 172.21.15.27:0/3896530308 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25422 (secure 0 0 0) 0x7f02e80b27f0 con 0x7f02fc105060 2024-09-21T07:30:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:30:04.923 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":108,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:29:11.849433+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10},{"from":5,"to":1}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]},{"pool":4,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02e40b66b0 msgr2=0x7f02e40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02e40b66b0 0x7f02e40b8bc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f02fc103e90 tx=0x7f02f000b390 comp rx=0 tx=0).stop 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc105060 msgr2=0x7f02fc102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc105060 0x7f02fc102980 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e9f0 tx=0x7f02e800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 shutdown_connections 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02fc10db70 0x7f02fc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f02e40b66b0 0x7f02e40b8bc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02fc105060 0x7f02fc102980 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 --2- 172.21.15.27:0/3896530308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc104710 0x7f02fc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 >> 172.21.15.27:0/3896530308 conn(0x7f02fc0fe1e0 msgr2=0x7f02fc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 shutdown_connections 2024-09-21T07:30:04.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:04.923+0000 7f0302000640 1 -- 172.21.15.27:0/3896530308 wait complete. 2024-09-21T07:30:04.948 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:04:59.755438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T07:09:50.469679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T07:09:53.416727+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T07:10:03.398488+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T07:30:04.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:30:05.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 -- 172.21.15.27:0/3756019163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010b600 msgr2=0x7fd48010ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 --2- 172.21.15.27:0/3756019163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010b600 0x7fd48010ba10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd468009900 tx=0x7fd46802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 -- 172.21.15.27:0/3756019163 shutdown_connections 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 --2- 172.21.15.27:0/3756019163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd48010bf50 0x7fd480112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 --2- 172.21.15.27:0/3756019163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010b600 0x7fd48010ba10 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 --2- 172.21.15.27:0/3756019163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd480105cd0 0x7fd4801060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 -- 172.21.15.27:0/3756019163 >> 172.21.15.27:0/3756019163 conn(0x7fd4801016f0 msgr2=0x7fd480103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 -- 172.21.15.27:0/3756019163 shutdown_connections 2024-09-21T07:30:05.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.180+0000 7fd484a00640 1 -- 172.21.15.27:0/3756019163 wait complete. 2024-09-21T07:30:05.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.181+0000 7fd484a00640 1 Processor -- start 2024-09-21T07:30:05.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd484a00640 1 -- start start 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd484a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd480105cd0 0x7fd48019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd484a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48010b600 0x7fd48019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd484a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010bf50 0x7fd4801a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd484a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd480115c10 con 0x7fd48010bf50 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd484a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd480115a90 con 0x7fd48010b600 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd484a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd480115d90 con 0x7fd480105cd0 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd47e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010bf50 0x7fd4801a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd47e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010bf50 0x7fd4801a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59884/0 (socket says 172.21.15.27:59884) 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd47e800640 1 -- 172.21.15.27:0/3307933153 learned_addr learned my addr 172.21.15.27:0/3307933153 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd47de00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd480105cd0 0x7fd48019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.182+0000 7fd477e00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48010b600 0x7fd48019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd47e800640 1 -- 172.21.15.27:0/3307933153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd480105cd0 msgr2=0x7fd48019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd47e800640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd480105cd0 0x7fd48019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd47e800640 1 -- 172.21.15.27:0/3307933153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48010b600 msgr2=0x7fd48019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd47e800640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48010b600 0x7fd48019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd47e800640 1 -- 172.21.15.27:0/3307933153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801a2520 con 0x7fd48010bf50 2024-09-21T07:30:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd47e800640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010bf50 0x7fd4801a1e20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd47800eb10 tx=0x7fd47800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd476a00640 1 -- 172.21.15.27:0/3307933153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478004b20 con 0x7fd48010bf50 2024-09-21T07:30:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4801a2810 con 0x7fd48010bf50 2024-09-21T07:30:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.183+0000 7fd476a00640 1 -- 172.21.15.27:0/3307933153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd478004cc0 con 0x7fd48010bf50 2024-09-21T07:30:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.184+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a8fa0 con 0x7fd48010bf50 2024-09-21T07:30:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.184+0000 7fd476a00640 1 -- 172.21.15.27:0/3307933153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478021640 con 0x7fd48010bf50 2024-09-21T07:30:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.185+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480106f40 con 0x7fd48010bf50 2024-09-21T07:30:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.187+0000 7fd476a00640 1 -- 172.21.15.27:0/3307933153 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd478005000 con 0x7fd48010bf50 2024-09-21T07:30:05.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.189+0000 7fd476a00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4640b6680 0x7fd4640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.189+0000 7fd476a00640 1 -- 172.21.15.27:0/3307933153 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fd4780e8070 con 0x7fd48010bf50 2024-09-21T07:30:05.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.189+0000 7fd47de00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4640b6680 0x7fd4640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.190+0000 7fd47de00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4640b6680 0x7fd4640b8b90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd4700095f0 tx=0x7fd470005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:05.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.192+0000 7fd476a00640 1 -- 172.21.15.27:0/3307933153 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4780ae1b0 con 0x7fd48010bf50 2024-09-21T07:30:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:04 smithi119 ceph-mon[34519]: pgmap v911: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-21T07:30:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:04 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3896530308' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:30:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:04 smithi027 ceph-mon[30779]: pgmap v911: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-21T07:30:05.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:04 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3896530308' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:30:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.390+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd48002e640 con 0x7fd48010bf50 2024-09-21T07:30:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.390+0000 7fd476a00640 1 -- 172.21.15.27:0/3307933153 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7fd4780a9030 con 0x7fd48010bf50 2024-09-21T07:30:05.392 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4640b6680 msgr2=0x7fd4640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4640b6680 0x7fd4640b8b90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd4700095f0 tx=0x7fd470005f90 comp rx=0 tx=0).stop 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010bf50 msgr2=0x7fd4801a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010bf50 0x7fd4801a1e20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd47800eb10 tx=0x7fd47800c2d0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 shutdown_connections 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd4640b6680 0x7fd4640b8b90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd48010bf50 0x7fd4801a1e20 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48010b600 0x7fd48019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 --2- 172.21.15.27:0/3307933153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd480105cd0 0x7fd48019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 >> 172.21.15.27:0/3307933153 conn(0x7fd4801016f0 msgr2=0x7fd480103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 shutdown_connections 2024-09-21T07:30:05.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.393+0000 7fd484a00640 1 -- 172.21.15.27:0/3307933153 wait complete. 2024-09-21T07:30:05.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T07:30:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:04 smithi153 ceph-mon[34747]: pgmap v911: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-21T07:30:05.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:04 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3896530308' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.651+0000 7fe0fb200640 1 -- 172.21.15.27:0/1085684393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc1038f0 msgr2=0x7fe0fc103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.651+0000 7fe0fb200640 1 --2- 172.21.15.27:0/1085684393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc103d00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009900 tx=0x7fe0ec02a410 comp rx=0 tx=0).stop 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.651+0000 7fe0fb200640 1 -- 172.21.15.27:0/1085684393 shutdown_connections 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.651+0000 7fe0fb200640 1 --2- 172.21.15.27:0/1085684393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0fc109320 0x7fe0fc109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.651+0000 7fe0fb200640 1 --2- 172.21.15.27:0/1085684393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc109c70 0x7fe0fc1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.651+0000 7fe0fb200640 1 --2- 172.21.15.27:0/1085684393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc103d00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.651+0000 7fe0fb200640 1 -- 172.21.15.27:0/1085684393 >> 172.21.15.27:0/1085684393 conn(0x7fe0fc0ff330 msgr2=0x7fe0fc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.652+0000 7fe0fb200640 1 -- 172.21.15.27:0/1085684393 shutdown_connections 2024-09-21T07:30:05.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.652+0000 7fe0fb200640 1 -- 172.21.15.27:0/1085684393 wait complete. 2024-09-21T07:30:05.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 Processor -- start 2024-09-21T07:30:05.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 -- start start 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc109320 0x7fe0fc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0fc109c70 0x7fe0fc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc113930 con 0x7fe0fc109320 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1137b0 con 0x7fe0fc1038f0 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0fb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc113ab0 con 0x7fe0fc109c70 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0f9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc109320 0x7fe0fc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54338/0 (socket says 172.21.15.27:54338) 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.653+0000 7fe0f9e00640 1 -- 172.21.15.27:0/2139773263 learned_addr learned my addr 172.21.15.27:0/2139773263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.654+0000 7fe0fa800640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0fc109c70 0x7fe0fc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.654+0000 7fe0f9e00640 1 -- 172.21.15.27:0/2139773263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0fc109c70 msgr2=0x7fe0fc19fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.654+0000 7fe0f9e00640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0fc109c70 0x7fe0fc19fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.654+0000 7fe0f9e00640 1 -- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc109320 msgr2=0x7fe0fc19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.654+0000 7fe0f9e00640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc109320 0x7fe0fc19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.654+0000 7fe0f9e00640 1 -- 172.21.15.27:0/2139773263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc1a02a0 con 0x7fe0fc1038f0 2024-09-21T07:30:05.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.654+0000 7fe0f9e00640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc19a370 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec005940 tx=0x7fe0ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:05.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.655+0000 7fe0f2a00640 1 -- 172.21.15.27:0/2139773263 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec03f030 con 0x7fe0fc1038f0 2024-09-21T07:30:05.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.655+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc1a0530 con 0x7fe0fc1038f0 2024-09-21T07:30:05.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.655+0000 7fe0f2a00640 1 -- 172.21.15.27:0/2139773263 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec003aa0 con 0x7fe0fc1038f0 2024-09-21T07:30:05.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.655+0000 7fe0f2a00640 1 -- 172.21.15.27:0/2139773263 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec035070 con 0x7fe0fc1038f0 2024-09-21T07:30:05.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.655+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1a6d20 con 0x7fe0fc1038f0 2024-09-21T07:30:05.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.656+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc103d00 con 0x7fe0fc1038f0 2024-09-21T07:30:05.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.659+0000 7fe0f2a00640 1 -- 172.21.15.27:0/2139773263 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe0ec042030 con 0x7fe0fc1038f0 2024-09-21T07:30:05.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.660+0000 7fe0f2a00640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0e00b6680 0x7fe0e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:05.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.660+0000 7fe0f2a00640 1 -- 172.21.15.27:0/2139773263 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fe0ec108460 con 0x7fe0fc1038f0 2024-09-21T07:30:05.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.660+0000 7fe0f9400640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0e00b6680 0x7fe0e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:05.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.661+0000 7fe0f9400640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0e00b6680 0x7fe0e00b8b90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc108ec0 tx=0x7fe0e800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:05.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.662+0000 7fe0f2a00640 1 -- 172.21.15.27:0/2139773263 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0ec0ce5a0 con 0x7fe0fc1038f0 2024-09-21T07:30:05.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.863+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0fc10ba40 con 0x7fe0fc1038f0 2024-09-21T07:30:05.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.864+0000 7fe0f2a00640 1 -- 172.21.15.27:0/2139773263 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) ==== 104+0+11 (secure 0 0 0) 0x7fe0ec0d2980 con 0x7fe0fc1038f0 2024-09-21T07:30:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0e00b6680 msgr2=0x7fe0e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0e00b6680 0x7fe0e00b8b90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc108ec0 tx=0x7fe0e800b410 comp rx=0 tx=0).stop 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc1038f0 msgr2=0x7fe0fc19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc19a370 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec005940 tx=0x7fe0ec004270 comp rx=0 tx=0).stop 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 shutdown_connections 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0fc109c70 0x7fe0fc19fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fe0e00b6680 0x7fe0e00b8b90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0fc109320 0x7fe0fc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 --2- 172.21.15.27:0/2139773263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0fc1038f0 0x7fe0fc19a370 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 >> 172.21.15.27:0/2139773263 conn(0x7fe0fc0ff330 msgr2=0x7fe0fc101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 shutdown_connections 2024-09-21T07:30:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:05.867+0000 7fe0fb200640 1 -- 172.21.15.27:0/2139773263 wait complete. 2024-09-21T07:30:05.895 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 -- 172.21.15.27:0/1440365114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 msgr2=0x7f9e1010ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 --2- 172.21.15.27:0/1440365114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 0x7f9e1010ba00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009860 tx=0x7f9dfc02a340 comp rx=0 tx=0).stop 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 -- 172.21.15.27:0/1440365114 shutdown_connections 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 --2- 172.21.15.27:0/1440365114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e1010bf40 0x7f9e10112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 --2- 172.21.15.27:0/1440365114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 0x7f9e1010ba00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 --2- 172.21.15.27:0/1440365114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e10105cc0 0x7f9e101060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 -- 172.21.15.27:0/1440365114 >> 172.21.15.27:0/1440365114 conn(0x7f9e101016c0 msgr2=0x7f9e10103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 -- 172.21.15.27:0/1440365114 shutdown_connections 2024-09-21T07:30:06.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.128+0000 7f9e17400640 1 -- 172.21.15.27:0/1440365114 wait complete. 2024-09-21T07:30:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.129+0000 7f9e17400640 1 Processor -- start 2024-09-21T07:30:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.129+0000 7f9e17400640 1 -- start start 2024-09-21T07:30:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.129+0000 7f9e17400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e10105cc0 0x7f9e1019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.129+0000 7f9e17400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 0x7f9e1019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e17400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e1010bf40 0x7f9e101a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e17400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10115c00 con 0x7f9e10105cc0 2024-09-21T07:30:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e17400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10115a80 con 0x7f9e1010b5f0 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e17400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10115d80 con 0x7f9e1010bf40 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e10105cc0 0x7f9e1019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e10105cc0 0x7f9e1019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59916/0 (socket says 172.21.15.27:59916) 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 -- 172.21.15.27:0/2667191586 learned_addr learned my addr 172.21.15.27:0/2667191586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e0fe00640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 0x7f9e1019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e15400640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e1010bf40 0x7f9e101a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 -- 172.21.15.27:0/2667191586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e1010bf40 msgr2=0x7f9e101a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e1010bf40 0x7f9e101a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 -- 172.21.15.27:0/2667191586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 msgr2=0x7f9e1019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 0x7f9e1019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.130+0000 7f9e14a00640 1 -- 172.21.15.27:0/2667191586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e101a2610 con 0x7f9e10105cc0 2024-09-21T07:30:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.131+0000 7f9e14a00640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e10105cc0 0x7f9e1019c6e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400e9f0 tx=0x7f9e0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:06.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.131+0000 7f9e0d600640 1 -- 172.21.15.27:0/2667191586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400cd80 con 0x7f9e10105cc0 2024-09-21T07:30:06.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.131+0000 7f9e0d600640 1 -- 172.21.15.27:0/2667191586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e040054d0 con 0x7f9e10105cc0 2024-09-21T07:30:06.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.131+0000 7f9e0d600640 1 -- 172.21.15.27:0/2667191586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04010690 con 0x7f9e10105cc0 2024-09-21T07:30:06.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.131+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e101a2900 con 0x7f9e10105cc0 2024-09-21T07:30:06.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.131+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101a9090 con 0x7f9e10105cc0 2024-09-21T07:30:06.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.132+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e101060d0 con 0x7f9e10105cc0 2024-09-21T07:30:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.134+0000 7f9e0d600640 1 -- 172.21.15.27:0/2667191586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9e04004e90 con 0x7f9e10105cc0 2024-09-21T07:30:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.137+0000 7f9e0d600640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9df80b6700 0x7f9df80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.137+0000 7f9e0d600640 1 -- 172.21.15.27:0/2667191586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f9e040e7ea0 con 0x7f9e10105cc0 2024-09-21T07:30:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.137+0000 7f9e0fe00640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9df80b6700 0x7f9df80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.138+0000 7f9e0fe00640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9df80b6700 0x7f9df80b8c10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc005940 tx=0x7f9dfc0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.141+0000 7f9e0d600640 1 -- 172.21.15.27:0/2667191586 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e040adfe0 con 0x7f9e10105cc0 2024-09-21T07:30:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:05 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3307933153' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:30:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:05 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2139773263' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:30:06.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:05 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3307933153' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:30:06.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:05 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2139773263' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:30:06.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.345+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e1002e640 con 0x7f9e10105cc0 2024-09-21T07:30:06.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.345+0000 7f9e0d600640 1 -- 172.21.15.27:0/2667191586 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) ==== 100+0+11 (secure 0 0 0) 0x7f9e040b23c0 con 0x7f9e10105cc0 2024-09-21T07:30:06.347 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-09-21T07:30:06.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9df80b6700 msgr2=0x7f9df80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9df80b6700 0x7f9df80b8c10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc005940 tx=0x7f9dfc0058d0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e10105cc0 msgr2=0x7f9e1019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e10105cc0 0x7f9e1019c6e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400e9f0 tx=0x7f9e0400edf0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 shutdown_connections 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e1010bf40 0x7f9e101a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f9df80b6700 0x7f9df80b8c10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e1010b5f0 0x7f9e1019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 --2- 172.21.15.27:0/2667191586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e10105cc0 0x7f9e1019c6e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 >> 172.21.15.27:0/2667191586 conn(0x7f9e101016c0 msgr2=0x7f9e10108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 shutdown_connections 2024-09-21T07:30:06.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.348+0000 7f9e17400640 1 -- 172.21.15.27:0/2667191586 wait complete. 2024-09-21T07:30:06.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T07:30:06.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:05 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3307933153' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:30:06.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:05 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2139773263' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:30:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.603+0000 7fc221600640 1 -- 172.21.15.27:0/1019253038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c10bd80 msgr2=0x7fc21c1125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.603+0000 7fc221600640 1 --2- 172.21.15.27:0/1019253038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c10bd80 0x7fc21c1125b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc214009be0 tx=0x7fc21402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.604+0000 7fc221600640 1 -- 172.21.15.27:0/1019253038 shutdown_connections 2024-09-21T07:30:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.604+0000 7fc221600640 1 --2- 172.21.15.27:0/1019253038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c10bd80 0x7fc21c1125b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.604+0000 7fc221600640 1 --2- 172.21.15.27:0/1019253038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc21c10b430 0x7fc21c10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.604+0000 7fc221600640 1 --2- 172.21.15.27:0/1019253038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc21c105b00 0x7fc21c105f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.604+0000 7fc221600640 1 -- 172.21.15.27:0/1019253038 >> 172.21.15.27:0/1019253038 conn(0x7fc21c101520 msgr2=0x7fc21c103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.604+0000 7fc221600640 1 -- 172.21.15.27:0/1019253038 shutdown_connections 2024-09-21T07:30:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.604+0000 7fc221600640 1 -- 172.21.15.27:0/1019253038 wait complete. 2024-09-21T07:30:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 Processor -- start 2024-09-21T07:30:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 -- start start 2024-09-21T07:30:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c105b00 0x7fc21c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc21c10b430 0x7fc21c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc21c10bd80 0x7fc21c1a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c115a40 con 0x7fc21c10b430 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c1158c0 con 0x7fc21c105b00 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc221600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c115bc0 con 0x7fc21c10bd80 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc21b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc21c10b430 0x7fc21c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc21be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c105b00 0x7fc21c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.605+0000 7fc21b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc21c10b430 0x7fc21c19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59934/0 (socket says 172.21.15.27:59934) 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc21b400640 1 -- 172.21.15.27:0/1380025641 learned_addr learned my addr 172.21.15.27:0/1380025641 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc220c00640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc21c10bd80 0x7fc21c1a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc21be00640 1 -- 172.21.15.27:0/1380025641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc21c10bd80 msgr2=0x7fc21c1a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc21be00640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc21c10bd80 0x7fc21c1a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc21be00640 1 -- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc21c10b430 msgr2=0x7fc21c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc21be00640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc21c10b430 0x7fc21c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc21be00640 1 -- 172.21.15.27:0/1380025641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c1a22f0 con 0x7fc21c105b00 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc21be00640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c105b00 0x7fc21c19c3c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e9f0 tx=0x7fc20c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.606+0000 7fc218c00640 1 -- 172.21.15.27:0/1380025641 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00ccc0 con 0x7fc21c105b00 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.607+0000 7fc218c00640 1 -- 172.21.15.27:0/1380025641 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc20c0108e0 con 0x7fc21c105b00 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.607+0000 7fc218c00640 1 -- 172.21.15.27:0/1380025641 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c018880 con 0x7fc21c105b00 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.607+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c1a25e0 con 0x7fc21c105b00 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.607+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c1a8d70 con 0x7fc21c105b00 2024-09-21T07:30:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.608+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c105f10 con 0x7fc21c105b00 2024-09-21T07:30:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.610+0000 7fc218c00640 1 -- 172.21.15.27:0/1380025641 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc20c010430 con 0x7fc21c105b00 2024-09-21T07:30:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.612+0000 7fc218c00640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2000b6700 0x7fc2000b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.612+0000 7fc218c00640 1 -- 172.21.15.27:0/1380025641 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7fc20c0e8240 con 0x7fc21c105b00 2024-09-21T07:30:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.612+0000 7fc21b400640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2000b6700 0x7fc2000b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.613+0000 7fc21b400640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2000b6700 0x7fc2000b8c10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc21c104dd0 tx=0x7fc208009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.614+0000 7fc218c00640 1 -- 172.21.15.27:0/1380025641 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc20c0b2760 con 0x7fc21c105b00 2024-09-21T07:30:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.811+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc21c02e640 con 0x7fc21c105b00 2024-09-21T07:30:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.812+0000 7fc218c00640 1 -- 172.21.15.27:0/1380025641 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v108) ==== 103+0+11 (secure 0 0 0) 0x7fc20c0b76a0 con 0x7fc21c105b00 2024-09-21T07:30:06.814 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T07:30:06.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2000b6700 msgr2=0x7fc2000b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2000b6700 0x7fc2000b8c10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc21c104dd0 tx=0x7fc208009260 comp rx=0 tx=0).stop 2024-09-21T07:30:06.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c105b00 msgr2=0x7fc21c19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:06.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c105b00 0x7fc21c19c3c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e9f0 tx=0x7fc20c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 shutdown_connections 2024-09-21T07:30:06.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc21c10bd80 0x7fc21c1a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fc2000b6700 0x7fc2000b8c10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc21c10b430 0x7fc21c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 --2- 172.21.15.27:0/1380025641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21c105b00 0x7fc21c19c3c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 >> 172.21.15.27:0/1380025641 conn(0x7fc21c101520 msgr2=0x7fc21c108a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 shutdown_connections 2024-09-21T07:30:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:06.815+0000 7fc221600640 1 -- 172.21.15.27:0/1380025641 wait complete. 2024-09-21T07:30:06.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T07:30:07.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 -- 172.21.15.27:0/2901095452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5024075730 msgr2=0x7f5024075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 --2- 172.21.15.27:0/2901095452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5024075730 0x7f5024075b40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5014009860 tx=0x7f501402a340 comp rx=0 tx=0).stop 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 -- 172.21.15.27:0/2901095452 shutdown_connections 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 --2- 172.21.15.27:0/2901095452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f502410fe80 0x7f50241122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 --2- 172.21.15.27:0/2901095452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024076080 0x7f502410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 --2- 172.21.15.27:0/2901095452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5024075730 0x7f5024075b40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 -- 172.21.15.27:0/2901095452 >> 172.21.15.27:0/2901095452 conn(0x7f50240ff2d0 msgr2=0x7f5024101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 -- 172.21.15.27:0/2901095452 shutdown_connections 2024-09-21T07:30:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.073+0000 7f5029600640 1 -- 172.21.15.27:0/2901095452 wait complete. 2024-09-21T07:30:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.074+0000 7f5029600640 1 Processor -- start 2024-09-21T07:30:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5029600640 1 -- start start 2024-09-21T07:30:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5029600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024075730 0x7f502419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5029600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5024076080 0x7f502419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5029600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f502410fe80 0x7f50241a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5029600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50241159c0 con 0x7f5024076080 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5029600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5024115840 con 0x7f5024075730 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5029600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5024115b40 con 0x7f502410fe80 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5023400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5024076080 0x7f502419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5023e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024075730 0x7f502419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5023400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5024076080 0x7f502419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59952/0 (socket says 172.21.15.27:59952) 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5023e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024075730 0x7f502419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:54392/0 (socket says 172.21.15.27:54392) 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5023400640 1 -- 172.21.15.27:0/2514600984 learned_addr learned my addr 172.21.15.27:0/2514600984 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.075+0000 7f5028c00640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f502410fe80 0x7f50241a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5023e00640 1 -- 172.21.15.27:0/2514600984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f502410fe80 msgr2=0x7f50241a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5023e00640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f502410fe80 0x7f50241a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5023e00640 1 -- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5024076080 msgr2=0x7f502419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5023e00640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5024076080 0x7f502419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5023e00640 1 -- 172.21.15.27:0/2514600984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50241a23b0 con 0x7f5024075730 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5023400640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5024076080 0x7f502419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5023e00640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024075730 0x7f502419c480 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5014035040 tx=0x7f5014032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5020c00640 1 -- 172.21.15.27:0/2514600984 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014040020 con 0x7f5024075730 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50241a2640 con 0x7f5024075730 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5020c00640 1 -- 172.21.15.27:0/2514600984 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5014030d50 con 0x7f5024075730 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.076+0000 7f5020c00640 1 -- 172.21.15.27:0/2514600984 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014037040 con 0x7f5024075730 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.077+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a8e30 con 0x7f5024075730 2024-09-21T07:30:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.078+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5024075b40 con 0x7f5024075730 2024-09-21T07:30:07.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.080+0000 7f5020c00640 1 -- 172.21.15.27:0/2514600984 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5014043020 con 0x7f5024075730 2024-09-21T07:30:07.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.081+0000 7f5020c00640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f50080b6700 0x7f50080b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:07.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.082+0000 7f5020c00640 1 -- 172.21.15.27:0/2514600984 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9461+0+0 (secure 0 0 0) 0x7f5014107c40 con 0x7f5024075730 2024-09-21T07:30:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.082+0000 7f5023400640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f50080b6700 0x7f50080b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.083+0000 7f5023400640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f50080b6700 0x7f50080b8c10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5024108510 tx=0x7f5010009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.083+0000 7f5020c00640 1 -- 172.21.15.27:0/2514600984 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f501404a020 con 0x7f5024075730 2024-09-21T07:30:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:06 smithi119 ceph-mon[34519]: pgmap v912: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T07:30:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:06 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2667191586' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:30:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:06 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1380025641' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:30:07.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.329+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f50241a9060 con 0x7f5024075730 2024-09-21T07:30:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:06 smithi027 ceph-mon[30779]: pgmap v912: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T07:30:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:06 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2667191586' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:30:07.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:06 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1380025641' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:30:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.340+0000 7f5020c00640 1 -- 172.21.15.27:0/2514600984 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+834053 (secure 0 0 0) 0x7f50140cdee0 con 0x7f5024075730 2024-09-21T07:30:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:30:07.353 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:30:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 27, 2024-09-21T07:30:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 14226, 2024-09-21T07:30:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3854502753 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-21T07:30:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3854502753 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.27:6801/3854502753", 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-09-21T07:02:53.267424+0000", 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24160, 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "z", 2024-09-21T07:30:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540719139924082687, 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T07:30:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T07:30:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T07:30:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T07:30:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T07:30:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:30:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:30:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T07:30:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T07:30:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T07:30:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T07:30:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:30:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T07:30:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T07:30:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:30:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T07:30:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:30:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T07:30:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T07:30:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T07:30:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T07:30:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T07:30:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T07:30:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T07:30:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T07:30:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:30:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:30:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T07:30:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T07:30:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T07:30:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T07:30:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T07:30:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T07:30:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T07:30:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:30:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T07:30:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:30:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T07:30:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T07:30:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:30:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:30:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:30:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T07:30:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:30:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:30:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:30:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:30:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T07:30:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:30:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T07:30:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.593 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.597 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:30:07.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.600 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:07.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:07.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:30:07.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.605 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:30:07.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.609 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T07:30:07.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:30:07.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.624 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.625 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.628 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.630 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T07:30:07.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T07:30:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T07:30:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T07:30:07.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.641 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:30:07.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.643 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T07:30:07.657 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.661 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.662 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T07:30:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:30:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:30:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T07:30:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:30:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T07:30:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T07:30:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:30:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.775 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.779 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.781 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.781 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.781 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.781 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.781 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:30:07.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.784 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:07.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.790 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T07:30:07.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.792 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.792 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.792 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T07:30:07.792 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T07:30:07.792 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.792 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.792 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.793 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.793 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T07:30:07.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T07:30:07.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.795 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.796 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.799 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.799 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.799 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.800 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.800 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.800 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.801 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.802 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.802 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.802 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.802 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.802 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.803 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.803 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.804 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.804 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:30:07.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.809 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.812 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.813 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.814 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.815 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.816 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.825 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.828 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.831 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.832 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T07:30:07.834 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.834 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.840 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.841 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.842 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.848 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.849 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.849 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.850 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T07:30:07.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.854 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.856 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.860 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.862 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.867 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.871 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:30:07.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.872 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.873 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.878 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T07:30:07.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T07:30:07.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.884 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.898 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.899 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.899 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.901 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.906 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.906 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.906 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.906 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.906 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T07:30:07.909 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T07:30:07.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T07:30:07.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.924 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.927 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.927 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T07:30:07.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T07:30:07.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.927 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T07:30:07.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.932 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.935 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.938 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.938 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.938 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.942 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T07:30:07.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.948 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.950 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.953 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.961 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.962 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.962 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.962 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.968 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:30:07.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:30:07.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.970 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.970 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T07:30:07.973 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.973 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.973 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.973 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T07:30:07.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T07:30:07.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:07.973 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T07:30:07.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.978 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.981 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24191, 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540719139924082687, 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T07:30:07.989 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.992 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.993 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.994 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:07.994 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:07.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:07.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:07.998 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:07.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:07.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:07.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:07.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:07.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:30:08.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T07:30:08.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.003 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T07:30:08.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T07:30:08.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T07:30:08.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.006 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.015 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T07:30:08.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.022 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.023 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.028 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:30:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:30:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.039 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T07:30:08.040 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:30:08.041 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.043 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T07:30:08.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.046 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T07:30:08.048 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T07:30:08.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T07:30:08.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T07:30:08.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:30:08.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.061 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.062 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.064 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T07:30:08.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T07:30:08.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.070 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.070 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.070 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:30:08.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.072 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:30:08.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.072 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T07:30:08.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T07:30:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:30:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:30:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T07:30:08.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T07:30:08.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.086 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.090 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.091 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.095 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.097 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:30:08.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.102 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:30:08.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.105 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:30:08.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.111 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:30:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T07:30:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T07:30:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:30:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T07:30:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T07:30:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:30:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:30:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:30:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:30:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:30:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:30:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:30:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:30:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T07:30:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T07:30:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T07:30:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:30:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T07:30:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T07:30:08.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.229 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T07:30:08.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T07:30:08.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:30:08.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.247 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.248 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.250 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.258 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.260 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:30:08.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.265 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.266 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.267 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.270 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T07:30:08.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.275 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.285 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.291 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.293 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.298 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T07:30:08.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.302 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.303 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.303 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.303 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.303 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.303 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T07:30:08.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.313 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.316 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T07:30:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.319 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T07:30:08.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T07:30:08.325 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.327 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.327 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.327 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.327 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.327 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.327 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.329 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.330 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.335 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.337 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.339 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.340 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.353 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T07:30:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T07:30:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:30:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:30:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:30:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T07:30:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T07:30:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T07:30:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T07:30:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.471 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T07:30:08.474 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.481 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.489 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.490 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.504 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.505 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.510 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.519 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.522 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.523 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.525 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T07:30:08.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.538 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard", 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "restful", 2024-09-21T07:30:08.544 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_schedule" 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.547 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.549 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T07:30:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.557 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.558 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:30:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.563 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T07:30:08.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T07:30:08.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T07:30:08.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.572 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T07:30:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.581 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.582 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T07:30:08.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T07:30:08.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T07:30:08.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T07:30:08.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T07:30:08.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.598 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T07:30:08.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:30:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T07:30:08.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T07:30:08.606 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:30:08.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T07:30:08.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T07:30:08.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T07:30:08.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.623 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.624 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:30:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.630 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.632 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.633 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.637 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.638 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.640 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.641 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T07:30:08.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T07:30:08.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.648 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.649 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.656 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.657 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.658 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.659 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:30:08.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.672 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:30:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:30:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:30:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:30:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.691 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.692 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:30:08.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:30:08.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.696 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:30:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.705 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.710 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.717 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T07:30:08.718 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.724 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T07:30:08.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.730 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:30:08.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.733 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.734 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.745 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T07:30:08.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.751 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.753 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.754 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T07:30:08.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T07:30:08.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T07:30:08.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.759 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.760 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.762 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.765 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.768 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T07:30:08.771 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T07:30:08.772 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.774 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.776 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.779 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.780 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.781 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.783 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:30:08.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.787 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.789 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.791 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.792 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.797 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.805 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.806 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.814 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.815 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.816 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T07:30:08.816 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T07:30:08.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.817 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.819 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.828 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.830 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.832 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.834 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.842 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.844 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T07:30:08.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.849 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.854 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T07:30:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T07:30:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:30:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T07:30:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T07:30:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T07:30:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T07:30:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T07:30:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T07:30:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T07:30:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T07:30:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T07:30:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.959 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.960 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.963 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.966 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.967 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.970 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T07:30:08.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.983 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T07:30:08.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.986 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:08.992 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:08.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.004 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:09.007 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:09.008 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:09.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.010 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:09.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T07:30:09.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.015 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T07:30:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T07:30:09.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.033 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "services": { 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard": "https://172.21.15.27:8443/" 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T07:30:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T07:30:09.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T07:30:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T07:30:09.037 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T07:30:09.038 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 5, 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T07:30:09.039 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2037380229 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 476022973 2024-09-21T07:30:09.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1400087737 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2008981265 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T07:30:09.042 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3210011465 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T07:30:09.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T07:30:09.044 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T07:30:09.044 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3140690905 2024-09-21T07:30:09.044 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.044 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.044 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:09.044 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T07:30:09.044 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:30:09.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f50080b6700 msgr2=0x7f50080b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f50080b6700 0x7f50080b8c10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5024108510 tx=0x7f5010009260 comp rx=0 tx=0).stop 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024075730 msgr2=0x7f502419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024075730 0x7f502419c480 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5014035040 tx=0x7f5014032d50 comp rx=0 tx=0).stop 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 shutdown_connections 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f502410fe80 0x7f50241a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f50080b6700 0x7f50080b8c10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5024076080 0x7f502419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 --2- 172.21.15.27:0/2514600984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5024075730 0x7f502419c480 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 >> 172.21.15.27:0/2514600984 conn(0x7f50240ff2d0 msgr2=0x7f5024108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 shutdown_connections 2024-09-21T07:30:09.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:07.348+0000 7f5029600640 1 -- 172.21.15.27:0/2514600984 wait complete. 2024-09-21T07:30:09.053 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T07:30:09.053 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T07:30:09.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:07 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2514600984' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T07:30:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:07 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2514600984' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T07:30:09.054 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:06 smithi153 ceph-mon[34747]: pgmap v912: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T07:30:09.054 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:06 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2667191586' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:30:09.055 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:06 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1380025641' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:30:09.055 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:07 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2514600984' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T07:30:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:08 smithi119 ceph-mon[34519]: pgmap v913: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T07:30:09.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:08 smithi027 ceph-mon[30779]: pgmap v913: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T07:30:09.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:08 smithi153 ceph-mon[34747]: pgmap v913: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T07:30:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:09 smithi119 ceph-mon[34519]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T07:30:10.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:09 smithi027 ceph-mon[30779]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T07:30:10.309 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:30:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 334, 2024-09-21T07:30:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 327, 2024-09-21T07:30:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1179104, 2024-09-21T07:30:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 202160, 2024-09-21T07:30:10.309 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.081134647, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.040392810, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.049992308, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 326, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.025021277, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000076752 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:10.310 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1179104, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 202160 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 250, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 204928, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T07:30:10.311 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.034866963, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.028561734, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.002301461, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 248, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.018357159, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000074020 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:10.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 204928 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 4, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 4, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 3040, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 448, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.003486379, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000037754, 2024-09-21T07:30:10.313 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.003363004, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.006121761, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002040587 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:10.315 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:10.316 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:10.317 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 149735, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 30, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T07:30:10.318 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 968, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-09-21T07:30:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T07:30:10.320 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 336, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1131879, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 336, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 336, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1131879, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:10.321 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:10.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:10.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:10.322 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:30:10.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:09 smithi153 ceph-mon[34747]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T07:30:11.243 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:10 smithi027 ceph-mon[30779]: pgmap v915: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T07:30:11.281 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-21T07:30:11.281 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-21T07:30:11.281 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-21T07:30:11.281 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-21T07:30:11.281 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-21T07:30:11.282 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-21T07:30:11.282 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-21T07:30:11.282 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T07:30:11.282 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T07:30:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:10 smithi119 ceph-mon[34519]: pgmap v915: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T07:30:11.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:10 smithi153 ceph-mon[34747]: pgmap v915: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T07:30:11.670 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:12.389 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:30:12.389 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:12.389 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-09-21T07:30:12.389 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.003748151, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000813711, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:12.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 339, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 332, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1180464, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 205360, 2024-09-21T07:30:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.070271316, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.032303301, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.049999831, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 331, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.024768825, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000074830 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:12.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.393 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1180464, 2024-09-21T07:30:12.393 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 205360 2024-09-21T07:30:12.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.393 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:12.393 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 6, 2024-09-21T07:30:12.393 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 257, 2024-09-21T07:30:12.394 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 3360, 2024-09-21T07:30:12.394 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 206256, 2024-09-21T07:30:12.394 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:12.394 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.038139255, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.028563588, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.005671886, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 256, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.024781918, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000096804 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:12.395 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 206256 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:12.396 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:12.398 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:12.398 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:12.399 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:12.400 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148704, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 29, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:12.401 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 968, 2024-09-21T07:30:12.402 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 341, 2024-09-21T07:30:12.403 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1132529, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 341, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 341, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1132529, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:12.404 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:12.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:12.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:12.405 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:30:13.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:12 smithi027 ceph-mon[30779]: pgmap v916: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-09-21T07:30:13.249 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T07:30:13.249 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T07:30:13.249 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-21T07:30:13.249 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-21T07:30:13.249 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T07:30:13.249 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T07:30:13.249 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T07:30:13.249 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T07:30:13.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T07:30:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:12 smithi119 ceph-mon[34519]: pgmap v916: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-09-21T07:30:13.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:12 smithi153 ceph-mon[34747]: pgmap v916: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-09-21T07:30:13.637 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:14.405 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2126, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 1438, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 559672, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 485194, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 13, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.240213355, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.110625885, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.135343501, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.007779229, 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1437, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.106768074, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000074299 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3368, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000838801, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000249 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 434, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 450, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1250434, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 276207, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 13, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 5, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.105968410, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.051273169, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.067379165, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000496298, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 449, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.050074193, 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000111523 2024-09-21T07:30:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 49, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000031900, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000651 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1242480, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 213824 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 64, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 315, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 10700, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 1236302, 2024-09-21T07:30:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 60, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 5, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.073423521, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.030443055, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.033978483, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000377869, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 313, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.029939603, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000095653 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 58, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000009958, 2024-09-21T07:30:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000171 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 611168 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 41, 2024-09-21T07:30:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.001757691, 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000042870 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2, 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000021848, 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000010924 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-09-21T07:30:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 1, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 1, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 11, 2024-09-21T07:30:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 16, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 16, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 13, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 4, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 3, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 45, 2024-09-21T07:30:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 38, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 1, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 6, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-09-21T07:30:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 4, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 234, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 6, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:30:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-09-21T07:30:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 18, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 12, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 12, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-09-21T07:30:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-09-21T07:30:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 16, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 95, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0, 2024-09-21T07:30:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_succeeded": 0, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_write_failed": 0 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 5, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 5, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-09-21T07:30:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 2, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 2, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 4195735, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-09-21T07:30:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.008670846, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002167711 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 20, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 22, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 2, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 19, 2024-09-21T07:30:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 23, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 4, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 16, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 20, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 4, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 117792, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 325853184 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 1, 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 1, 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 18, 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-09-21T07:30:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-09-21T07:30:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 2, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 1, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 1, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 1049, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T07:30:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 246997, 2024-09-21T07:30:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 80, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9792, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 200, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 83525, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 390, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-09-21T07:30:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 40, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 11733, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 33, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 40, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 40, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.272993031, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.006824825 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-09-21T07:30:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 40, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 62 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 40, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 0, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 40, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 10, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 0, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 4, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 7, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-09-21T07:30:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 6, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 20, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-09-21T07:30:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-09-21T07:30:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 109, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 27, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 9, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 15, 2024-09-21T07:30:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 6, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 14, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 0, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 1 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-09-21T07:30:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 2620, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1561408, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 2620, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2620, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1561408, 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 40, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 11733, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 40, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 11733, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-09-21T07:30:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 40, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 40, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 40, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 40, 2024-09-21T07:30:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T07:30:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x561f3e129450": { 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 5, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1431, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 5, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T07:30:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 4, 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1431, 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "trackedop": { 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 111 2024-09-21T07:30:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:14.459 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:30:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:15 smithi119 ceph-mon[34519]: pgmap v917: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:15.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:14 smithi027 ceph-mon[30779]: pgmap v917: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:15.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:15 smithi153 ceph-mon[34747]: pgmap v917: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:15.447 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T07:30:15.448 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T07:30:15.448 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-21T07:30:15.448 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=18 2024-09-21T07:30:15.448 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-09-21T07:30:15.448 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:30:15.448 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:30:15.448 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T07:30:15.448 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T07:30:15.856 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:16 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:30:16.292 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:16 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:30:16.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:16 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:30:16.594 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.003916896, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000888562, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:16.595 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:30:16.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 256, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 205984, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.035786837, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.028926476, 2024-09-21T07:30:16.597 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.002855515, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 254, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.018173368, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000071548 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 205984 2024-09-21T07:30:16.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 345, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 336, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1184048, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 207088, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.071543045, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.031249046, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.053909333, 2024-09-21T07:30:16.599 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 336, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.030659839, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000091249 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1184048, 2024-09-21T07:30:16.600 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 207088 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:16.601 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:16.602 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:16.603 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148721, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 30, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:16.604 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:16.605 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 968, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T07:30:16.606 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 343, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1132789, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 343, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 343, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1132789, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T07:30:16.607 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T07:30:16.608 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T07:30:16.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:16.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T07:30:16.608 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T07:30:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:17 smithi119 ceph-mon[34519]: pgmap v918: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:17 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:30:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:17 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:30:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:17 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:30:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:17 smithi119 ceph-mon[34519]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T07:30:17.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:17 smithi027 ceph-mon[30779]: pgmap v918: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:17.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:17 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:30:17.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:17 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:30:17.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:17 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:30:17.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:17 smithi027 ceph-mon[30779]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T07:30:17.361 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T07:30:17.361 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T07:30:17.361 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-21T07:30:17.361 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-21T07:30:17.361 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T07:30:17.361 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T07:30:17.361 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T07:30:17.361 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T07:30:17.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T07:30:17.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:17 smithi153 ceph-mon[34747]: pgmap v918: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:17.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:17 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:30:17.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:17 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:30:17.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:17 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' 2024-09-21T07:30:17.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:17 smithi153 ceph-mon[34747]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T07:30:17.622 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 5722, 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 7103, 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1823683, 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 20448431, 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 406, 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 8, 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.889540144, 2024-09-21T07:30:18.106 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.332846078, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.360843669, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.150864795, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 7098, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.334517702, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000047128 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3977, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001473729, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000370 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.107 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 91920, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 34592 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 4564, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 4266, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 2471689, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 11120409, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 12, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.591325334, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.259891376, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.248765458, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.106019588, 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:18.108 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4269, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.293372382, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000068721 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4955, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001157566, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000233 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1828048, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 436160 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 2484, 2024-09-21T07:30:18.109 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 3414, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 723404, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 6745303, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 36, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.317503356, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.159899435, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.132969017, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.033100617, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3412, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.196543589, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000057603 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.110 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3357, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001052024, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000313 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 632288 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T07:30:18.111 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 6829, 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.953489351, 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000286057 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 16, 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000430520, 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000026907 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-09-21T07:30:18.112 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 900, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 900, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 900, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 17.170205006, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.019078005 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 489, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-09-21T07:30:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 45, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 16, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 29, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 22, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 3, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 4, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 2, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 1197, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 1185, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-09-21T07:30:18.114 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 357, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 4, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 234, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 6, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 357, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-09-21T07:30:18.115 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 230, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 26, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 204, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 127, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 722, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 691, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 599, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 830, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 727, 2024-09-21T07:30:18.116 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 3, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 630 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 11, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 24, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 2, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 3, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 1, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-09-21T07:30:18.117 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_path": 16, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_inode": 220, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 26, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-09-21T07:30:18.118 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_started": 0, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_succeeded": 0, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_write_failed": 0 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 2691, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 2663, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 2663, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 28, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 14, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 83, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 82, 2024-09-21T07:30:18.119 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 82, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 1, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 1, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 1, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 29846098, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 30056644, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2593, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 6.372862344, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002457717 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.120 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 40, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 67, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 27, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 20, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 20, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 0, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 45, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 46, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 1, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 4, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 21, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 17, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 128008, 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 340541440 2024-09-21T07:30:18.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 2188, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 1, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 900, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 41, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 2, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 19, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.011400616, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000600032 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 200, 2024-09-21T07:30:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.040620590, 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000203102 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001668845, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000556281 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T07:30:18.124 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000480695, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000240347 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-09-21T07:30:18.125 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.126 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.126 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.126 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18, 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.003566094, 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000198116 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 8, 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 13.465748093, 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 1.683218511 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-09-21T07:30:18.127 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.094393144, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005244063 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000378174, 2024-09-21T07:30:18.128 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000378174 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.004398767, 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002199383 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.002225570, 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001112785 2024-09-21T07:30:18.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T07:30:18.130 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.012036564, 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006018282 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 12, 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.060316616, 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005026384 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.131 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 601, 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 3.444440687, 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005731182 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.132 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.133 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 11, 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.026652452, 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002422950 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.134 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001878099, 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001878099 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:30:18.135 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 5, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 3, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 75, 2024-09-21T07:30:18.136 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 37, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 1041, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:18.137 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:18.138 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 320120, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 118, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:18.139 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 9792, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 200, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 190787, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 1803, 2024-09-21T07:30:18.140 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 5011, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 27638925, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 37, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 5011, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 5011, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 31.347184265, 2024-09-21T07:30:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006255674 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 5011, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 6032 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 5011, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 410, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 4601, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 469, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 30, 2024-09-21T07:30:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 27, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 2594, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 191, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 526, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 383, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 55, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-09-21T07:30:18.143 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 599, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-09-21T07:30:18.144 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 110, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 35, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 14, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 2, 2024-09-21T07:30:18.145 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 1145, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 0, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 13 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 2, 2024-09-21T07:30:18.146 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 1, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 2, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 1, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 12766, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 3814162, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 12766, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:18.147 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 12766, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 3814162, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:18.148 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 5011, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 27639082, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 5011, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 27639082, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-09-21T07:30:18.149 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 5011, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 5011, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 5011, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 5011, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 1, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 157, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 1, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 1, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 157, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.151 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x55c7d775d2b0": { 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 2691, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 25862340, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 2691, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2593, 2024-09-21T07:30:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 25862340, 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: "trackedop": { 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 271 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:18.153 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T07:30:19.097 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T07:30:19.097 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T07:30:19.097 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-09-21T07:30:19.098 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-21T07:30:19.098 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T07:30:19.098 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=18 2024-09-21T07:30:19.098 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=12 2024-09-21T07:30:19.098 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T07:30:19.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T07:30:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:19 smithi119 ceph-mon[34519]: pgmap v920: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 232 B/s wr, 6 op/s 2024-09-21T07:30:19.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:19 smithi027 ceph-mon[30779]: pgmap v920: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 232 B/s wr, 6 op/s 2024-09-21T07:30:19.344 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:30:19.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:19 smithi153 ceph-mon[34747]: pgmap v920: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 232 B/s wr, 6 op/s 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1180, 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1180, 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 313363, 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 374480, 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 7, 2024-09-21T07:30:19.859 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 2, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.141673970, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.071670300, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.028963167, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.029251034, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1179, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.051438648, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000043629 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.860 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1452, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1443, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1492462, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 554573, 2024-09-21T07:30:19.861 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.191674464, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.086832623, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.069452917, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.037440826, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1442, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.061835143, 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000042881 2024-09-21T07:30:19.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1227744, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 212400 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1169, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1423, 2024-09-21T07:30:19.863 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 392608, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 891872, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.204327348, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.092718931, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.042000974, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.052364057, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1421, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.088460310, 2024-09-21T07:30:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000062252 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 520448 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T07:30:19.865 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 5690, 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.188635116, 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000033152 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 0, 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 0, 2024-09-21T07:30:19.867 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-09-21T07:30:19.868 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2024-09-21T07:30:19.869 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 1, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-09-21T07:30:19.870 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-09-21T07:30:19.871 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2024-09-21T07:30:19.872 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-09-21T07:30:19.873 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-09-21T07:30:19.874 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_started": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_succeeded": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_write_failed": 0 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 6, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-09-21T07:30:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 0, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 4195956, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 4195815, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 6 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 3, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 3, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 0, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 1, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 1, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 0, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 0, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 0, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 0, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2024-09-21T07:30:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 116892, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 316416000 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-09-21T07:30:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-09-21T07:30:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 0, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 2, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 1, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 0, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 0, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 1695, 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-09-21T07:30:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:19.892 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:19.893 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:19.894 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 190991, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 55, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:19.895 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 9552, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 194, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 8672, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 13, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:19.896 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 3420, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 21, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 3, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 3420, 2024-09-21T07:30:19.897 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3420, 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2.007947601, 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000587119 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3420, 2024-09-21T07:30:19.898 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 3421 2024-09-21T07:30:19.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.899 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 3420, 2024-09-21T07:30:19.899 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 3420, 2024-09-21T07:30:19.899 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 0, 2024-09-21T07:30:19.899 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-09-21T07:30:19.899 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-09-21T07:30:19.899 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 2242, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 1176, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 0, 2024-09-21T07:30:19.900 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 1, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:30:19.901 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-09-21T07:30:19.902 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 110, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 28, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 5, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 6, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 1, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 0, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 2, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 0 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.903 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.904 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 3797, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1834384, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 3797, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 3797, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1834384, 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:19.905 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 3420, 2024-09-21T07:30:19.906 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 1657, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 3420, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1657, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:19.907 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 3420, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 3420, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 3420, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 3420, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.908 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.909 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x558fc9235380": { 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:19.910 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: "trackedop": { 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:19.911 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T07:30:20.545 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T07:30:20.545 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T07:30:20.545 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-21T07:30:20.545 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-21T07:30:20.545 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-21T07:30:20.545 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:30:20.545 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:30:20.545 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T07:30:20.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T07:30:20.787 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:30:21.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:21 smithi119 ceph-mon[34519]: pgmap v921: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:21.326 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T07:30:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 79, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 151, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 19152, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 655943, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 46, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.032576670, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.006956253, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.018429501, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000566871, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 150, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.029769952, 2024-09-21T07:30:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000198466 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 68, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000010028, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000147 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 8160, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 225040 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 37, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 219, 2024-09-21T07:30:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 7421, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 652219, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 28, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 8, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.025190793, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.012200733, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.009867782, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000362331, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 217, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.029051519, 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000133877 2024-09-21T07:30:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 20, 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000002420, 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000121 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 392416 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 2519, 2024-09-21T07:30:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1817, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1795885, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 535385, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 11, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 4, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.269791438, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.139624845, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.155881149, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.005489291, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1816, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.087380130, 2024-09-21T07:30:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000048116 2024-09-21T07:30:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3399, 2024-09-21T07:30:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001178731, 2024-09-21T07:30:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000346 2024-09-21T07:30:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1240096, 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 216560 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 60, 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001644665, 2024-09-21T07:30:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000027411 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000024943, 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000012471 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 6, 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 4, 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4, 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.002018286, 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000504571 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-09-21T07:30:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 2, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 11, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 16, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 0, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 16, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-09-21T07:30:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 13, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 4, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 3, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 53, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 45, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 7, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-09-21T07:30:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 4, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 234, 2024-09-21T07:30:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 6, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 3, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:30:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 3, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 1 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 16, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 11, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 12, 2024-09-21T07:30:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:30:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_path": 16, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_inode": 95, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0, 2024-09-21T07:30:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_started": 0, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_succeeded": 0, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_write_failed": 0 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 6, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 6, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-09-21T07:30:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 2, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 2, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 4195956, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 5, 2024-09-21T07:30:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.009945089, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001989017 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 20, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 22, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 2, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 19, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 22, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 3, 2024-09-21T07:30:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 16, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 20, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 4, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 4, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 4, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 119552, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 325861376 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 10, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-09-21T07:30:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 6, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 8, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 1, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001587850, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000793925 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T07:30:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000430436, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000215218 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-09-21T07:30:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-09-21T07:30:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.352 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:21 smithi027 ceph-mon[30779]: pgmap v921: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-09-21T07:30:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-09-21T07:30:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 0, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 2, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 0, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 3, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 3, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 65, 2024-09-21T07:30:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 255927, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 87, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 9792, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 200, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T07:30:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 83629, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 403, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 44, 2024-09-21T07:30:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 12269, 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 33, 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 44, 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 44, 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.271992928, 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006181657 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-09-21T07:30:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-09-21T07:30:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 44, 2024-09-21T07:30:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 68 2024-09-21T07:30:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 44, 2024-09-21T07:30:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 0, 2024-09-21T07:30:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 44, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 0, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 10, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 0, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 5, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 8, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-09-21T07:30:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 6, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 20, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-09-21T07:30:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-09-21T07:30:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-09-21T07:30:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 110, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 28, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 9, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 13, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 4, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 17, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 0, 2024-09-21T07:30:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 2 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-09-21T07:30:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T07:30:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 2631, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1561070, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 2631, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2631, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1561070, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 44, 2024-09-21T07:30:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 12269, 2024-09-21T07:30:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 44, 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 12269, 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 44, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 44, 2024-09-21T07:30:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 44, 2024-09-21T07:30:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 44, 2024-09-21T07:30:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x558baebdb380": { 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 6, 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1652, 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 6, 2024-09-21T07:30:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 5, 2024-09-21T07:30:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1652, 2024-09-21T07:30:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "trackedop": { 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 111 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:21.387 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T07:30:21.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:21 smithi153 ceph-mon[34747]: pgmap v921: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:22.171 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T07:30:22.172 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T07:30:22.172 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-21T07:30:22.172 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=16 2024-09-21T07:30:22.172 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T07:30:22.172 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:30:22.172 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:30:22.172 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T07:30:22.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T07:30:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:30:22.924 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T07:30:22.924 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:22.924 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 0, 2024-09-21T07:30:22.924 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 2, 2024-09-21T07:30:22.924 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 0, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 0, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 0, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.001087853, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.000200171, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:22.925 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.926 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 349, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 340, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1185760, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 209344, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 2, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.067704120, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.031106152, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.050027847, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:22.927 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 340, 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.025569031, 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000075203 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1185760, 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 209344 2024-09-21T07:30:22.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 2, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 258, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 320, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 206336, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 2, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.022199130, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.017937780, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.001015241, 2024-09-21T07:30:22.929 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 256, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.019697730, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000076944 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:22.930 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 206336 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 0, 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:22.931 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:22.932 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:22.933 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 148704, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 29, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:22.934 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 968, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 2, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 347, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1133946, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 347, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 347, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1133946, 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T07:30:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T07:30:22.937 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T07:30:22.937 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T07:30:22.937 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:22.937 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T07:30:22.937 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T07:30:23.236 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:23 smithi119 ceph-mon[34519]: pgmap v922: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:23.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:23 smithi027 ceph-mon[30779]: pgmap v922: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:23.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:23 smithi153 ceph-mon[34747]: pgmap v922: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T07:30:23.623 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-21T07:30:23.623 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-21T07:30:23.623 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-21T07:30:23.624 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-21T07:30:23.624 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-21T07:30:23.624 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-21T07:30:23.624 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-21T07:30:23.624 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T07:30:23.624 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T07:30:23.887 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:30:24.434 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-21T07:30:24.434 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:24.434 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1551, 2024-09-21T07:30:24.434 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1540, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1624304, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 584904, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.260208013, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.112435435, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.093503383, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.052778220, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1540, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.098722231, 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000064105 2024-09-21T07:30:24.435 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1230512, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 212208 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 4, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 257, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 876, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 611434, 2024-09-21T07:30:24.436 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.018223310, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.014320718, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.001380886, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000062153, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 255, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.016333367, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000064052 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.437 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 610736 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2238, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2240, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 571376, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 711691, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.207287259, 2024-09-21T07:30:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.090945443, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.043618790, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.060485897, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2238, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.117355261, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000052437 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:30:24.439 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 5757, 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.187342872, 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000032541 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.440 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-21T07:30:24.441 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-21T07:30:24.442 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 1, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-21T07:30:24.443 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:30:24.444 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-21T07:30:24.445 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-21T07:30:24.446 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:30:24.447 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "uninline_started": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "uninline_succeeded": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "uninline_write_failed": 0 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-21T07:30:24.448 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 5, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 2, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-21T07:30:24.449 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 4195735, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 4195594, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 5 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 3, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 3, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 0, 2024-09-21T07:30:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 1, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 1, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 0, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 0, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 0, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 0, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 116376, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 316424192 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-21T07:30:24.452 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.453 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.454 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.455 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.456 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.457 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.457 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.457 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-21T07:30:24.457 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.457 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.458 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.459 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.460 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.461 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-21T07:30:24.462 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.463 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.464 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 2, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 1, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 2, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 4280, 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-21T07:30:24.465 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:24.466 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:24.467 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 182127, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 49, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-21T07:30:24.468 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9552, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 194, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 8672, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 13, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:24.469 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 3412, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 21, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 3, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 3412, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3412, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2.086962479, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000611653 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3412, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 3413 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 3412, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 3412, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 2231, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 1179, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-09-21T07:30:24.471 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 1, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-21T07:30:24.472 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-21T07:30:24.473 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 110, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 28, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 4, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 5, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 1, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-09-21T07:30:24.474 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-21T07:30:24.475 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 3789, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1833236, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 3789, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3789, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1833236, 2024-09-21T07:30:24.476 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:24.477 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3412, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 1436, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3412, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1436, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.478 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3412, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 3412, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3412, 2024-09-21T07:30:24.479 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 3412, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:24.480 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.481 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x5566f0615450": { 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:24.482 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:24.483 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:24.483 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:24.483 INFO:teuthology.orchestra.run.smithi153.stdout: "trackedop": { 2024-09-21T07:30:24.483 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-21T07:30:24.483 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:24.483 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-21T07:30:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:25 smithi119 ceph-mon[34519]: pgmap v923: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T07:30:25.334 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:25 smithi153 ceph-mon[34747]: pgmap v923: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T07:30:25.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:25 smithi027 ceph-mon[30779]: pgmap v923: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T07:30:25.360 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T07:30:25.361 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T07:30:25.361 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-21T07:30:25.361 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-21T07:30:25.361 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-21T07:30:25.361 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:30:25.361 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:30:25.361 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T07:30:25.361 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T07:30:25.606 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 341, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 334, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1181008, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 206640, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 0, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.067971092, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.032410129, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.048363848, 2024-09-21T07:30:26.153 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 333, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.023295586, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000069956 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1181008, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 206640 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.154 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 256, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 320, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 205984, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.023102462, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.018619887, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.001148443, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 254, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.019576507, 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000077072 2024-09-21T07:30:26.155 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 205984 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 4, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 4, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 3040, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 448, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 2, 2024-09-21T07:30:26.156 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.002140353, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.000032027, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.002094482, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.003784515, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001261505 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.157 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 0, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:26.158 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:26.159 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:26.160 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 148682, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 29, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-21T07:30:26.161 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 968, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 2, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-21T07:30:26.162 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 343, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1132789, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 343, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 343, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1132789, 2024-09-21T07:30:26.163 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:26.164 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:26.164 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:26.164 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:26.164 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:26.164 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:26.164 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-21T07:30:26.873 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-09-21T07:30:26.873 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-09-21T07:30:26.873 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-09-21T07:30:26.873 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-09-21T07:30:26.873 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-09-21T07:30:26.873 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-09-21T07:30:26.873 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-09-21T07:30:26.873 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T07:30:26.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T07:30:27.119 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:30:27.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:27 smithi153 ceph-mon[34747]: pgmap v924: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T07:30:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:27 smithi119 ceph-mon[34519]: pgmap v924: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T07:30:27.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:27 smithi027 ceph-mon[30779]: pgmap v924: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T07:30:27.663 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-21T07:30:27.663 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:27.663 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2499, 2024-09-21T07:30:27.663 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2490, 2024-09-21T07:30:27.663 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 10253608, 2024-09-21T07:30:27.663 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 887038, 2024-09-21T07:30:27.663 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.360111336, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.150232753, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.123428237, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.078405033, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2489, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.116606778, 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000046848 2024-09-21T07:30:27.664 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1227632, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 211920 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1065, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1318, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 9353502, 2024-09-21T07:30:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 950484, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.141155812, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.063823270, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.035202482, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.031784857, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1316, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.061755433, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000046926 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.666 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.667 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.667 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:27.667 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 612384 2024-09-21T07:30:27.667 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.667 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:27.667 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1040, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1040, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 8651234, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 329903, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.128127076, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.053685307, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.033132526, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.031972780, 2024-09-21T07:30:27.668 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1039, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.044450869, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000042782 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-21T07:30:27.669 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 7195, 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.217476984, 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000030226 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-21T07:30:27.670 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-21T07:30:27.671 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 36, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 16, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 20, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 14, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 2, 2024-09-21T07:30:27.672 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:30:27.673 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 4, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 234, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 6, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:30:27.674 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-21T07:30:27.675 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-21T07:30:27.676 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-21T07:30:27.676 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-09-21T07:30:27.676 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-21T07:30:27.677 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-21T07:30:27.677 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-21T07:30:27.677 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:30:27.677 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:30:27.678 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "uninline_started": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "uninline_succeeded": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "uninline_write_failed": 0 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-21T07:30:27.679 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 2691, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 2, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 77, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 29846098, 2024-09-21T07:30:27.680 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 30056644, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 30056508, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 2691 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 31, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 32, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 1, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 11, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 11, 2024-09-21T07:30:27.681 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 36, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 36, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 119040, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 317468672 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-21T07:30:27.682 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.683 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-21T07:30:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.685 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.686 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-21T07:30:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.688 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.688 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.688 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-21T07:30:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-21T07:30:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-21T07:30:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.691 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.692 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.693 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.694 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.695 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.696 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.697 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-21T07:30:27.698 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 2, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 5, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 3, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 3, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 3475, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:27.699 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:27.700 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:27.701 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:27.702 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-21T07:30:27.702 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:27.702 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:27.702 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:27.702 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:27.702 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 199354, 2024-09-21T07:30:27.703 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 87, 2024-09-21T07:30:27.703 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:27.703 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:27.703 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-21T07:30:27.703 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-21T07:30:27.703 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:27.703 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:27.704 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:27.704 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:27.704 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9552, 2024-09-21T07:30:27.704 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 194, 2024-09-21T07:30:27.704 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:27.704 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 149657, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 1502, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 4222, 2024-09-21T07:30:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 26, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 4, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 4222, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4222, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2.505954955, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000593546 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4222, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 4223 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 4222, 2024-09-21T07:30:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 4222, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 2348, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 1871, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-21T07:30:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 2, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-21T07:30:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-21T07:30:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 110, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 28, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 6, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 6, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-09-21T07:30:27.710 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 4600, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 27821988, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 4600, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4600, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 27821988, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4222, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 25862350, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4222, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 25862350, 2024-09-21T07:30:27.713 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:27.714 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4222, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 4222, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4222, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 4222, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:27.715 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x5640e5abf520": { 2024-09-21T07:30:27.716 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:27.717 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:27.718 INFO:teuthology.orchestra.run.smithi153.stdout: "trackedop": { 2024-09-21T07:30:27.718 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-21T07:30:27.718 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:27.718 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-21T07:30:28.475 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T07:30:28.475 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T07:30:28.475 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-21T07:30:28.475 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-21T07:30:28.475 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T07:30:28.475 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:30:28.475 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:30:28.475 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T07:30:28.476 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T07:30:28.723 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:30:29.125 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:29 smithi153 ceph-mon[34747]: pgmap v925: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 93 B/s wr, 6 op/s 2024-09-21T07:30:29.246 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-21T07:30:29.246 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 351, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 342, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1186304, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 210624, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 0, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.067645190, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.030580246, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.047394926, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:29.247 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 342, 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.027307504, 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000079846 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1186304, 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 210624 2024-09-21T07:30:29.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2, 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 260, 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 320, 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 206688, 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 4, 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-21T07:30:29.249 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.023086481, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.018538103, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.001269211, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 258, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.019819697, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000076820 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:29.250 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 206688 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 0, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 0, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 0, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 2, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 1, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.000419219, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.000322019, 2024-09-21T07:30:29.251 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.252 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 0, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:30:29.253 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:30:29.254 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-21T07:30:29.255 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 148721, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 30, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:30:29.256 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 968, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 2, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-21T07:30:29.257 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 349, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1134206, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 349, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-21T07:30:29.258 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 349, 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1134206, 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-21T07:30:29.259 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-21T07:30:29.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:29 smithi027 ceph-mon[30779]: pgmap v925: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 93 B/s wr, 6 op/s 2024-09-21T07:30:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:29 smithi119 ceph-mon[34519]: pgmap v925: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 93 B/s wr, 6 op/s 2024-09-21T07:30:29.974 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-21T07:30:29.975 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-21T07:30:29.975 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-21T07:30:29.975 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-21T07:30:29.975 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-21T07:30:29.975 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-21T07:30:29.975 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-21T07:30:29.975 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T07:30:29.986 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T07:30:29.986 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:30:29.986 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:30:30.029 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T07:30:30.029 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:30:30.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T07:30:30.537 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 -- 172.21.15.27:0/2364797379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c105d80 msgr2=0x7f250c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 --2- 172.21.15.27:0/2364797379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c105d80 0x7f250c106190 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009900 tx=0x7f24fc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 -- 172.21.15.27:0/2364797379 shutdown_connections 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 --2- 172.21.15.27:0/2364797379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f250c10c010 0x7f250c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 --2- 172.21.15.27:0/2364797379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c10b6c0 0x7f250c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 --2- 172.21.15.27:0/2364797379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c105d80 0x7f250c106190 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 -- 172.21.15.27:0/2364797379 >> 172.21.15.27:0/2364797379 conn(0x7f250c101810 msgr2=0x7f250c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.186+0000 7f2513a00640 1 -- 172.21.15.27:0/2364797379 shutdown_connections 2024-09-21T07:30:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.187+0000 7f2513a00640 1 -- 172.21.15.27:0/2364797379 wait complete. 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 Processor -- start 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 -- start start 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c105d80 0x7f250c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c10b6c0 0x7f250c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f250c10c010 0x7f250c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f250c115cd0 con 0x7f250c10b6c0 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f250c115b50 con 0x7f250c105d80 2024-09-21T07:30:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.188+0000 7f2513a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f250c115e50 con 0x7f250c10c010 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c105d80 0x7f250c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c105d80 0x7f250c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40038/0 (socket says 172.21.15.27:40038) 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 -- 172.21.15.27:0/2556852430 learned_addr learned my addr 172.21.15.27:0/2556852430 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2503e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c10b6c0 0x7f250c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511a00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f250c10c010 0x7f250c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 -- 172.21.15.27:0/2556852430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f250c10c010 msgr2=0x7f250c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f250c10c010 0x7f250c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 -- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c10b6c0 msgr2=0x7f250c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c10b6c0 0x7f250c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 -- 172.21.15.27:0/2556852430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c1a2630 con 0x7f250c105d80 2024-09-21T07:30:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2503e00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c10b6c0 0x7f250c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.189+0000 7f2511000640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c105d80 0x7f250c19c6c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f24fc005940 tx=0x7f24fc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.190+0000 7f250a000640 1 -- 172.21.15.27:0/2556852430 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc03f030 con 0x7f250c105d80 2024-09-21T07:30:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.190+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c1a28c0 con 0x7f250c105d80 2024-09-21T07:30:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.190+0000 7f250a000640 1 -- 172.21.15.27:0/2556852430 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24fc0044c0 con 0x7f250c105d80 2024-09-21T07:30:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.190+0000 7f250a000640 1 -- 172.21.15.27:0/2556852430 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc035070 con 0x7f250c105d80 2024-09-21T07:30:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.190+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c1a9080 con 0x7f250c105d80 2024-09-21T07:30:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.191+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c106190 con 0x7f250c105d80 2024-09-21T07:30:31.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.194+0000 7f250a000640 1 -- 172.21.15.27:0/2556852430 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f24fc042030 con 0x7f250c105d80 2024-09-21T07:30:31.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.195+0000 7f250a000640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6680 0x7f24f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:31.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.195+0000 7f250a000640 1 -- 172.21.15.27:0/2556852430 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9461+0+0 (secure 0 0 0) 0x7f24fc1080f0 con 0x7f250c105d80 2024-09-21T07:30:31.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.195+0000 7f2503e00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6680 0x7f24f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.196+0000 7f2503e00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6680 0x7f24f00b8b90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f250c10b150 tx=0x7f24f4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:31.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.197+0000 7f250a000640 1 -- 172.21.15.27:0/2556852430 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24fc0d2610 con 0x7f250c105d80 2024-09-21T07:30:31.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:31 smithi027 ceph-mon[30779]: pgmap v926: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-21T07:30:31.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.409+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f250c19d7f0 con 0x7f250c105d80 2024-09-21T07:30:31.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.410+0000 7f250a000640 1 -- 172.21.15.27:0/2556852430 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v110) ==== 81+0+884 (secure 0 0 0) 0x7f24fc0d7550 con 0x7f250c105d80 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/785463945 2024-09-22T07:02:53.267293+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2240730116 2024-09-22T07:01:14.297512+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2151540061 2024-09-22T07:01:14.297512+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1830236521 2024-09-22T07:01:14.297512+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2513510737 2024-09-22T07:01:45.325213+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/727966245 2024-09-22T07:02:53.267293+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2811364463 2024-09-22T07:01:14.297512+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2513510737 2024-09-22T07:01:45.325213+0000 2024-09-21T07:30:31.412 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2151540061 2024-09-22T07:01:14.297512+0000 2024-09-21T07:30:31.413 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3503043760 2024-09-22T07:01:45.325213+0000 2024-09-21T07:30:31.413 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2989055970 2024-09-22T07:01:45.325213+0000 2024-09-21T07:30:31.413 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1823929888 2024-09-22T07:02:53.267293+0000 2024-09-21T07:30:31.413 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/727966245 2024-09-22T07:02:53.267293+0000 2024-09-21T07:30:31.413 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4193484002 2024-09-22T07:01:45.325213+0000 2024-09-21T07:30:31.413 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542737158 2024-09-22T07:02:53.267293+0000 2024-09-21T07:30:31.413 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T07:30:31.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6680 msgr2=0x7f24f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:31.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6680 0x7f24f00b8b90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f250c10b150 tx=0x7f24f4005f90 comp rx=0 tx=0).stop 2024-09-21T07:30:31.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c105d80 msgr2=0x7f250c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c105d80 0x7f250c19c6c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f24fc005940 tx=0x7f24fc0040c0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 shutdown_connections 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f250c10c010 0x7f250c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f24f00b6680 0x7f24f00b8b90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f250c10b6c0 0x7f250c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 --2- 172.21.15.27:0/2556852430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f250c105d80 0x7f250c19c6c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 >> 172.21.15.27:0/2556852430 conn(0x7f250c101810 msgr2=0x7f250c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 shutdown_connections 2024-09-21T07:30:31.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:31.412+0000 7f2513a00640 1 -- 172.21.15.27:0/2556852430 wait complete. 2024-09-21T07:30:31.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:31 smithi153 ceph-mon[34747]: pgmap v926: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-21T07:30:31.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:31 smithi119 ceph-mon[34519]: pgmap v926: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-21T07:30:32.245 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:30:32.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:30:32.246 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:30:32.296 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:32 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2556852430' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:30:32.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:30:32.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:30:32.329 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:30:32.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:30:32.385 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T07:30:32.423 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:30:32.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:30:32.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:30:32.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:32 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2556852430' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:30:32.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:30:32.523 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-21T07:30:32.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:32 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2556852430' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:30:32.867 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T07:30:32.878 INFO:tasks.cephadm:Teardown begin 2024-09-21T07:30:32.879 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:30:32.879 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:30:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:33.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:33 smithi027 ceph-mon[30779]: pgmap v927: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:30:33.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:33 smithi153 ceph-mon[34747]: pgmap v927: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:30:33.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:33 smithi119 ceph-mon[34519]: pgmap v927: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T07:30:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.877+0000 7f3add000640 1 -- 172.21.15.27:0/2565675482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8114740 msgr2=0x7f3ad8116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.877+0000 7f3add000640 1 --2- 172.21.15.27:0/2565675482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8114740 0x7f3ad8116b60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009ca0 tx=0x7f3ad002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.878+0000 7f3add000640 1 -- 172.21.15.27:0/2565675482 shutdown_connections 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.878+0000 7f3add000640 1 --2- 172.21.15.27:0/2565675482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8114740 0x7f3ad8116b60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.878+0000 7f3add000640 1 --2- 172.21.15.27:0/2565675482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 0x7f3ad8114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.878+0000 7f3add000640 1 --2- 172.21.15.27:0/2565675482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad8077940 0x7f3ad8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.878+0000 7f3add000640 1 -- 172.21.15.27:0/2565675482 >> 172.21.15.27:0/2565675482 conn(0x7f3ad81017d0 msgr2=0x7f3ad8103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.878+0000 7f3add000640 1 -- 172.21.15.27:0/2565675482 shutdown_connections 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.878+0000 7f3add000640 1 -- 172.21.15.27:0/2565675482 wait complete. 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 Processor -- start 2024-09-21T07:30:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 -- start start 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8077940 0x7f3ad81a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 0x7f3ad81a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad8114740 0x7f3ad81a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad811a0b0 con 0x7f3ad8077940 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8119f30 con 0x7f3ad8078290 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.879+0000 7f3add000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad811a230 con 0x7f3ad8114740 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8077940 0x7f3ad81a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 0x7f3ad81a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 0x7f3ad81a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:40068/0 (socket says 172.21.15.27:40068) 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 -- 172.21.15.27:0/4059661815 learned_addr learned my addr 172.21.15.27:0/4059661815 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8077940 0x7f3ad81a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37648/0 (socket says 172.21.15.27:37648) 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad6800640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad8114740 0x7f3ad81a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:33.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 -- 172.21.15.27:0/4059661815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad8114740 msgr2=0x7f3ad81a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad8114740 0x7f3ad81a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 -- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8077940 msgr2=0x7f3ad81a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8077940 0x7f3ad81a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 -- 172.21.15.27:0/4059661815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad81a69d0 con 0x7f3ad8078290 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5e00640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8077940 0x7f3ad81a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.880+0000 7f3ad5400640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 0x7f3ad81a1000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e9f0 tx=0x7f3ac400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.881+0000 7f3acea00640 1 -- 172.21.15.27:0/4059661815 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400ccc0 con 0x7f3ad8078290 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.881+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad81a6cc0 con 0x7f3ad8078290 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.881+0000 7f3acea00640 1 -- 172.21.15.27:0/4059661815 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac40108e0 con 0x7f3ad8078290 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.881+0000 7f3acea00640 1 -- 172.21.15.27:0/4059661815 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4018860 con 0x7f3ad8078290 2024-09-21T07:30:33.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.881+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad81ad420 con 0x7f3ad8078290 2024-09-21T07:30:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.882+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad8077d50 con 0x7f3ad8078290 2024-09-21T07:30:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.885+0000 7f3acea00640 1 -- 172.21.15.27:0/4059661815 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3ac4010430 con 0x7f3ad8078290 2024-09-21T07:30:33.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.886+0000 7f3acea00640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3abc0b6680 0x7f3abc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:33.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.887+0000 7f3acea00640 1 -- 172.21.15.27:0/4059661815 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9461+0+0 (secure 0 0 0) 0x7f3ac40e8200 con 0x7f3ad8078290 2024-09-21T07:30:33.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.887+0000 7f3ad5e00640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3abc0b6680 0x7f3abc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:33.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.887+0000 7f3ad5e00640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3abc0b6680 0x7f3abc0b8b90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005d00 tx=0x7f3ac800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:33.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:33.888+0000 7f3acea00640 1 -- 172.21.15.27:0/4059661815 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ac40b2720 con 0x7f3ad8078290 2024-09-21T07:30:34.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.126+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ad81a1b90 con 0x7f3ad8078290 2024-09-21T07:30:34.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.127+0000 7f3acea00640 1 -- 172.21.15.27:0/4059661815 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f3ac40b7660 con 0x7f3ad8078290 2024-09-21T07:30:34.129 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:30:34.129 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:30:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:30:34.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3abc0b6680 msgr2=0x7f3abc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:34.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3abc0b6680 0x7f3abc0b8b90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005d00 tx=0x7f3ac800b410 comp rx=0 tx=0).stop 2024-09-21T07:30:34.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 msgr2=0x7f3ad81a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 0x7f3ad81a1000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e9f0 tx=0x7f3ac400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 shutdown_connections 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad8114740 0x7f3ad81a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f3abc0b6680 0x7f3abc0b8b90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad8078290 0x7f3ad81a1000 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 --2- 172.21.15.27:0/4059661815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ad8077940 0x7f3ad81a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.129+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 >> 172.21.15.27:0/4059661815 conn(0x7f3ad81017d0 msgr2=0x7f3ad8116790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.130+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 shutdown_connections 2024-09-21T07:30:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:34.130+0000 7f3add000640 1 -- 172.21.15.27:0/4059661815 wait complete. 2024-09-21T07:30:34.773 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:30:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:35 smithi027 ceph-mon[30779]: pgmap v928: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:30:35.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:35 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/4059661815' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:30:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:35 smithi153 ceph-mon[34747]: pgmap v928: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:30:35.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:35 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/4059661815' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:30:35.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:35 smithi119 ceph-mon[34519]: pgmap v928: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T07:30:35.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:35 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/4059661815' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:30:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 -- 172.21.15.27:0/2133894700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 msgr2=0x7f0508104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 --2- 172.21.15.27:0/2133894700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f0508104dc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009900 tx=0x7f04f802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 -- 172.21.15.27:0/2133894700 shutdown_connections 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 --2- 172.21.15.27:0/2133894700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f050810de10 0x7f0508110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 --2- 172.21.15.27:0/2133894700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0508105300 0x7f050810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 --2- 172.21.15.27:0/2133894700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f0508104dc0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 -- 172.21.15.27:0/2133894700 >> 172.21.15.27:0/2133894700 conn(0x7f0508079760 msgr2=0x7f0508079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 -- 172.21.15.27:0/2133894700 shutdown_connections 2024-09-21T07:30:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.835+0000 7f050e800640 1 -- 172.21.15.27:0/2133894700 wait complete. 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.836+0000 7f050e800640 1 Processor -- start 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f050e800640 1 -- start start 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f050e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f050819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f050e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0508105300 0x7f050819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f050e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f050810de10 0x7f050819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f050e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0508113a90 con 0x7f05081049b0 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f050e800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0508113910 con 0x7f050810de10 2024-09-21T07:30:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f050e800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0508113c10 con 0x7f0508105300 2024-09-21T07:30:35.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f0507400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f050819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f0507400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f050819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37670/0 (socket says 172.21.15.27:37670) 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f0507400640 1 -- 172.21.15.27:0/3748001621 learned_addr learned my addr 172.21.15.27:0/3748001621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f0506a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0508105300 0x7f050819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.837+0000 7f0507e00640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f050810de10 0x7f050819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0507400640 1 -- 172.21.15.27:0/3748001621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0508105300 msgr2=0x7f050819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0507400640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0508105300 0x7f050819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0507400640 1 -- 172.21.15.27:0/3748001621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f050810de10 msgr2=0x7f050819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0507400640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f050810de10 0x7f050819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0507400640 1 -- 172.21.15.27:0/3748001621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05081a03d0 con 0x7f05081049b0 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0506a00640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0508105300 0x7f050819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0507e00640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f050810de10 0x7f050819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.838+0000 7f0507400640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f050819a460 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f04f8005940 tx=0x7f04f80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.839+0000 7f04ffe00640 1 -- 172.21.15.27:0/3748001621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f803f030 con 0x7f05081049b0 2024-09-21T07:30:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.839+0000 7f04ffe00640 1 -- 172.21.15.27:0/3748001621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f80044c0 con 0x7f05081049b0 2024-09-21T07:30:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.839+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05081a0660 con 0x7f05081049b0 2024-09-21T07:30:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.839+0000 7f04ffe00640 1 -- 172.21.15.27:0/3748001621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8035070 con 0x7f05081049b0 2024-09-21T07:30:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.839+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a6e20 con 0x7f05081049b0 2024-09-21T07:30:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.840+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050806c2c0 con 0x7f05081049b0 2024-09-21T07:30:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.844+0000 7f04ffe00640 1 -- 172.21.15.27:0/3748001621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f04f8042030 con 0x7f05081049b0 2024-09-21T07:30:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.845+0000 7f04ffe00640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04ec0b6700 0x7f04ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.846+0000 7f04ffe00640 1 -- 172.21.15.27:0/3748001621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9461+0+0 (secure 0 0 0) 0x7f04f81080b0 con 0x7f05081049b0 2024-09-21T07:30:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.846+0000 7f0506a00640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04ec0b6700 0x7f04ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.847+0000 7f0506a00640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04ec0b6700 0x7f04ec0b8c10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0508102430 tx=0x7f04f4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:35.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:35.848+0000 7f04ffe00640 1 -- 172.21.15.27:0/3748001621 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f04f80d25d0 con 0x7f05081049b0 2024-09-21T07:30:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.087+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0508108f70 con 0x7f05081049b0 2024-09-21T07:30:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.087+0000 7f04ffe00640 1 -- 172.21.15.27:0/3748001621 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f04f80d7510 con 0x7f05081049b0 2024-09-21T07:30:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:30:36.090 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"btime":"2024-09-21T07:12:44:936199+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2637528669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2637528669},{"type":"v1","addr":"172.21.15.27:6841","nonce":2637528669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/4121519709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":4121519709},{"type":"v1","addr":"172.21.15.153:6833","nonce":4121519709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24677,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1660738486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1660738486},{"type":"v1","addr":"172.21.15.119:6833","nonce":1660738486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24689,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2994040514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2994040514},{"type":"v1","addr":"172.21.15.27:6835","nonce":2994040514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2900670620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2900670620},{"type":"v1","addr":"172.21.15.27:6837","nonce":2900670620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24709,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/717203459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":717203459},{"type":"v1","addr":"172.21.15.153:6835","nonce":717203459}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T07:09:56.473755+0000","modified":"2024-09-21T07:12:44.098389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24773,"mds_1":24748,"mds_2":24740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24713":{"gid":24713,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3894045960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3894045960},{"type":"v1","addr":"172.21.15.119:6835","nonce":3894045960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24721":{"gid":24721,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/1555665457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1555665457},{"type":"v1","addr":"172.21.15.153:6837","nonce":1555665457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24736":{"gid":24736,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6839/2081315981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2081315981},{"type":"v1","addr":"172.21.15.153:6839","nonce":2081315981}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24740":{"gid":24740,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.119:6837/4207522386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":4207522386},{"type":"v1","addr":"172.21.15.119:6837","nonce":4207522386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24748":{"gid":24748,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6839/609554839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":609554839},{"type":"v1","addr":"172.21.15.27:6839","nonce":609554839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24773":{"gid":24773,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1565967470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1565967470},{"type":"v1","addr":"172.21.15.119:6839","nonce":1565967470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24773,"qdb_cluster":[24740,24748,24773]},"id":1}]} 2024-09-21T07:30:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-09-21T07:30:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04ec0b6700 msgr2=0x7f04ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04ec0b6700 0x7f04ec0b8c10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0508102430 tx=0x7f04f4005f90 comp rx=0 tx=0).stop 2024-09-21T07:30:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 msgr2=0x7f050819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f050819a460 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f04f8005940 tx=0x7f04f80040c0 comp rx=0 tx=0).stop 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 shutdown_connections 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0508105300 0x7f050819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f04ec0b6700 0x7f04ec0b8c10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f050810de10 0x7f050819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 --2- 172.21.15.27:0/3748001621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05081049b0 0x7f050819a460 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 >> 172.21.15.27:0/3748001621 conn(0x7f0508079760 msgr2=0x7f05081007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.090+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 shutdown_connections 2024-09-21T07:30:36.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:36.091+0000 7f050e800640 1 -- 172.21.15.27:0/3748001621 wait complete. 2024-09-21T07:30:36.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:30:37.305 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:37 smithi027 ceph-mon[30779]: pgmap v929: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T07:30:37.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:37 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3748001621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:30:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:37 smithi153 ceph-mon[34747]: pgmap v929: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T07:30:37.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:37 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3748001621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:30:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:37 smithi119 ceph-mon[34519]: pgmap v929: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T07:30:37.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:37 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3748001621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:30:37.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 -- 172.21.15.27:0/2769216742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 msgr2=0x7fbd18110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:37.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/2769216742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 0x7fbd18110620 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009ca0 tx=0x7fbd1002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 -- 172.21.15.27:0/2769216742 shutdown_connections 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/2769216742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd181094a0 0x7fbd181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/2769216742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 0x7fbd18110620 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/2769216742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd18103b60 0x7fbd18103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 -- 172.21.15.27:0/2769216742 >> 172.21.15.27:0/2769216742 conn(0x7fbd1807b850 msgr2=0x7fbd1807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 -- 172.21.15.27:0/2769216742 shutdown_connections 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.870+0000 7fbd1fc00640 1 -- 172.21.15.27:0/2769216742 wait complete. 2024-09-21T07:30:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.871+0000 7fbd1fc00640 1 Processor -- start 2024-09-21T07:30:37.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1fc00640 1 -- start start 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1fc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18103b60 0x7fbd18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd181094a0 0x7fbd18075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1fc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 0x7fbd18076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1fc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18113ab0 con 0x7fbd18109df0 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1fc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18113930 con 0x7fbd181094a0 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1fc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18113c30 con 0x7fbd18103b60 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1d200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18103b60 0x7fbd18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1d200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18103b60 0x7fbd18075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46188/0 (socket says 172.21.15.27:46188) 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.872+0000 7fbd1d200640 1 -- 172.21.15.27:0/3316506990 learned_addr learned my addr 172.21.15.27:0/3316506990 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd1dc00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 0x7fbd18076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd0fe00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd181094a0 0x7fbd18075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd1d200640 1 -- 172.21.15.27:0/3316506990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd181094a0 msgr2=0x7fbd18075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:37.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd1d200640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd181094a0 0x7fbd18075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:37.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd1d200640 1 -- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 msgr2=0x7fbd18076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:37.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd1d200640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 0x7fbd18076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:37.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd1d200640 1 -- 172.21.15.27:0/3316506990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd181a8720 con 0x7fbd18103b60 2024-09-21T07:30:37.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.873+0000 7fbd1d200640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18103b60 0x7fbd18075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800e9f0 tx=0x7fbd0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:37.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.874+0000 7fbd16000640 1 -- 172.21.15.27:0/3316506990 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0800cd80 con 0x7fbd18103b60 2024-09-21T07:30:37.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.874+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd181a8a10 con 0x7fbd18103b60 2024-09-21T07:30:37.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.874+0000 7fbd16000640 1 -- 172.21.15.27:0/3316506990 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd080108e0 con 0x7fbd18103b60 2024-09-21T07:30:37.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.874+0000 7fbd16000640 1 -- 172.21.15.27:0/3316506990 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08018980 con 0x7fbd18103b60 2024-09-21T07:30:37.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.874+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a8fa0 con 0x7fbd18103b60 2024-09-21T07:30:37.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.875+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd18103f70 con 0x7fbd18103b60 2024-09-21T07:30:37.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.878+0000 7fbd16000640 1 -- 172.21.15.27:0/3316506990 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbd08010430 con 0x7fbd18103b60 2024-09-21T07:30:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.879+0000 7fbd16000640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbcfc0b6680 0x7fbcfc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.879+0000 7fbd16000640 1 -- 172.21.15.27:0/3316506990 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9461+0+0 (secure 0 0 0) 0x7fbd080e8390 con 0x7fbd18103b60 2024-09-21T07:30:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.880+0000 7fbd0fe00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbcfc0b6680 0x7fbcfc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:37.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.880+0000 7fbd0fe00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbcfc0b6680 0x7fbcfc0b8b90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbd18109040 tx=0x7fbd00005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:37.881+0000 7fbd16000640 1 -- 172.21.15.27:0/3316506990 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd080ae4d0 con 0x7fbd18103b60 2024-09-21T07:30:38.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.084+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd1802e640 con 0x7fbd18103b60 2024-09-21T07:30:38.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.085+0000 7fbd16000640 1 -- 172.21.15.27:0/3316506990 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+25422 (secure 0 0 0) 0x7fbd080b28b0 con 0x7fbd18103b60 2024-09-21T07:30:38.088 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T07:30:38.088 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":110,"fsid":"0ff1e276-77e7-11ef-baf5-efdc52797490","created":"2024-09-21T07:00:45.001187+0000","modified":"2024-09-21T07:30:16.866447+0000","last_up_change":"2024-09-21T07:09:02.012608+0000","last_in_change":"2024-09-21T07:08:39.578485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:04:59.755438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:09:50.469679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:09:53.416727+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:10:03.398488+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d999c06a-d17f-4840-9bec-67d0325b7546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6803","nonce":1967162431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6805","nonce":1967162431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6809","nonce":1967162431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1967162431},{"type":"v1","addr":"172.21.15.27:6807","nonce":1967162431}]},"public_addr":"172.21.15.27:6803/1967162431","cluster_addr":"172.21.15.27:6805/1967162431","heartbeat_back_addr":"172.21.15.27:6809/1967162431","heartbeat_front_addr":"172.21.15.27:6807/1967162431","state":["exists","up"]},{"osd":1,"uuid":"4063fc7c-3ba2-4b37-b188-45faf89d4127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6801","nonce":3407674054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6803","nonce":3407674054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6807","nonce":3407674054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3407674054},{"type":"v1","addr":"172.21.15.119:6805","nonce":3407674054}]},"public_addr":"172.21.15.119:6801/3407674054","cluster_addr":"172.21.15.119:6803/3407674054","heartbeat_back_addr":"172.21.15.119:6807/3407674054","heartbeat_front_addr":"172.21.15.119:6805/3407674054","state":["exists","up"]},{"osd":2,"uuid":"f871d34f-c35e-4ce0-9c20-09911568fc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6801","nonce":206718354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6803","nonce":206718354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6807","nonce":206718354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":206718354},{"type":"v1","addr":"172.21.15.153:6805","nonce":206718354}]},"public_addr":"172.21.15.153:6801/206718354","cluster_addr":"172.21.15.153:6803/206718354","heartbeat_back_addr":"172.21.15.153:6807/206718354","heartbeat_front_addr":"172.21.15.153:6805/206718354","state":["exists","up"]},{"osd":3,"uuid":"db7d7254-c55f-4b12-8c53-9292a254ffcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6811","nonce":984637291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6813","nonce":984637291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6817","nonce":984637291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":984637291},{"type":"v1","addr":"172.21.15.27:6815","nonce":984637291}]},"public_addr":"172.21.15.27:6811/984637291","cluster_addr":"172.21.15.27:6813/984637291","heartbeat_back_addr":"172.21.15.27:6817/984637291","heartbeat_front_addr":"172.21.15.27:6815/984637291","state":["exists","up"]},{"osd":4,"uuid":"fa23aaaf-55c1-4c48-bf7b-c2c2039624ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6809","nonce":2014804508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6811","nonce":2014804508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6815","nonce":2014804508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2014804508},{"type":"v1","addr":"172.21.15.119:6813","nonce":2014804508}]},"public_addr":"172.21.15.119:6809/2014804508","cluster_addr":"172.21.15.119:6811/2014804508","heartbeat_back_addr":"172.21.15.119:6815/2014804508","heartbeat_front_addr":"172.21.15.119:6813/2014804508","state":["exists","up"]},{"osd":5,"uuid":"2ab0114f-b98e-4b90-b112-aa1dec91830c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6809","nonce":2063779127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6811","nonce":2063779127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6815","nonce":2063779127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2063779127},{"type":"v1","addr":"172.21.15.153:6813","nonce":2063779127}]},"public_addr":"172.21.15.153:6809/2063779127","cluster_addr":"172.21.15.153:6811/2063779127","heartbeat_back_addr":"172.21.15.153:6815/2063779127","heartbeat_front_addr":"172.21.15.153:6813/2063779127","state":["exists","up"]},{"osd":6,"uuid":"d7bb5b9d-337b-41e4-ae7a-93db10aa151e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6819","nonce":3534557792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6821","nonce":3534557792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534557792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3534557792},{"type":"v1","addr":"172.21.15.27:6823","nonce":3534557792}]},"public_addr":"172.21.15.27:6819/3534557792","cluster_addr":"172.21.15.27:6821/3534557792","heartbeat_back_addr":"172.21.15.27:6825/3534557792","heartbeat_front_addr":"172.21.15.27:6823/3534557792","state":["exists","up"]},{"osd":7,"uuid":"2cb69c8e-353c-49e9-87f9-802de8728313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6817","nonce":1162667989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162667989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162667989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162667989},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162667989}]},"public_addr":"172.21.15.119:6817/1162667989","cluster_addr":"172.21.15.119:6819/1162667989","heartbeat_back_addr":"172.21.15.119:6823/1162667989","heartbeat_front_addr":"172.21.15.119:6821/1162667989","state":["exists","up"]},{"osd":8,"uuid":"13844110-df7e-451b-b574-38419af2df5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6817","nonce":2174139924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6819","nonce":2174139924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6823","nonce":2174139924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2174139924},{"type":"v1","addr":"172.21.15.153:6821","nonce":2174139924}]},"public_addr":"172.21.15.153:6817/2174139924","cluster_addr":"172.21.15.153:6819/2174139924","heartbeat_back_addr":"172.21.15.153:6823/2174139924","heartbeat_front_addr":"172.21.15.153:6821/2174139924","state":["exists","up"]},{"osd":9,"uuid":"5c254d35-aa04-43e4-844e-5d6d2adb11ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6827","nonce":611699615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6829","nonce":611699615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6833","nonce":611699615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":611699615},{"type":"v1","addr":"172.21.15.27:6831","nonce":611699615}]},"public_addr":"172.21.15.27:6827/611699615","cluster_addr":"172.21.15.27:6829/611699615","heartbeat_back_addr":"172.21.15.27:6833/611699615","heartbeat_front_addr":"172.21.15.27:6831/611699615","state":["exists","up"]},{"osd":10,"uuid":"87219b10-05f8-49ae-8be3-3e7d79454a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6825","nonce":3847228474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6827","nonce":3847228474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6831","nonce":3847228474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3847228474},{"type":"v1","addr":"172.21.15.119:6829","nonce":3847228474}]},"public_addr":"172.21.15.119:6825/3847228474","cluster_addr":"172.21.15.119:6827/3847228474","heartbeat_back_addr":"172.21.15.119:6831/3847228474","heartbeat_front_addr":"172.21.15.119:6829/3847228474","state":["exists","up"]},{"osd":11,"uuid":"8ee8dc56-e77b-4969-8e32-4380b8b12b21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6825","nonce":1280738577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6827","nonce":1280738577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6831","nonce":1280738577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1280738577},{"type":"v1","addr":"172.21.15.153:6829","nonce":1280738577}]},"public_addr":"172.21.15.153:6825/1280738577","cluster_addr":"172.21.15.153:6827/1280738577","heartbeat_back_addr":"172.21.15.153:6831/1280738577","heartbeat_front_addr":"172.21.15.153:6829/1280738577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:05.611150+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:31.600988+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:04:56.366742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:25.976641+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:05:52.010699+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:17.945757+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:06:48.790591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:14.074475+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:07:38.954501+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:10.531872+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:08:36.534665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:09:00.876858+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10},{"from":5,"to":1}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/785463945":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2240730116":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/1830236521":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6801/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/2811364463":"2024-09-22T07:01:14.297512+0000","172.21.15.27:6800/2513510737":"2024-09-22T07:01:45.325213+0000","172.21.15.27:6801/2151540061":"2024-09-22T07:01:14.297512+0000","172.21.15.27:0/3503043760":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/2989055970":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/1823929888":"2024-09-22T07:02:53.267293+0000","172.21.15.27:6800/727966245":"2024-09-22T07:02:53.267293+0000","172.21.15.27:0/4193484002":"2024-09-22T07:01:45.325213+0000","172.21.15.27:0/542737158":"2024-09-22T07:02:53.267293+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]},{"pool":4,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:30:38.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.088+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbcfc0b6680 msgr2=0x7fbcfc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.088+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbcfc0b6680 0x7fbcfc0b8b90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbd18109040 tx=0x7fbd00005f90 comp rx=0 tx=0).stop 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.088+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18103b60 msgr2=0x7fbd18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.088+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18103b60 0x7fbd18075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800e9f0 tx=0x7fbd0800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 shutdown_connections 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fbcfc0b6680 0x7fbcfc0b8b90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18109df0 0x7fbd18076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd181094a0 0x7fbd18075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 --2- 172.21.15.27:0/3316506990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18103b60 0x7fbd18075710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 >> 172.21.15.27:0/3316506990 conn(0x7fbd1807b850 msgr2=0x7fbd181074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 shutdown_connections 2024-09-21T07:30:38.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:38.089+0000 7fbd1fc00640 1 -- 172.21.15.27:0/3316506990 wait complete. 2024-09-21T07:30:38.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T07:30:39.305 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:39 smithi027 ceph-mon[30779]: pgmap v930: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:30:39.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:39 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3316506990' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:30:39.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:39 smithi153 ceph-mon[34747]: pgmap v930: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:30:39.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:39 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3316506990' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:30:39.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:39 smithi119 ceph-mon[34519]: pgmap v930: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:30:39.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:39 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3316506990' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:30:39.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 -- 172.21.15.27:0/384774988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 msgr2=0x7ff618105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 --2- 172.21.15.27:0/384774988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff618105fc0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff608009900 tx=0x7ff60802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 -- 172.21.15.27:0/384774988 shutdown_connections 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 --2- 172.21.15.27:0/384774988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff61810be40 0x7ff618112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 --2- 172.21.15.27:0/384774988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61810b4f0 0x7ff61810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 --2- 172.21.15.27:0/384774988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff618105fc0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 -- 172.21.15.27:0/384774988 >> 172.21.15.27:0/384774988 conn(0x7ff618101680 msgr2=0x7ff618103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 -- 172.21.15.27:0/384774988 shutdown_connections 2024-09-21T07:30:39.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.965+0000 7ff61cc00640 1 -- 172.21.15.27:0/384774988 wait complete. 2024-09-21T07:30:39.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.966+0000 7ff61cc00640 1 Processor -- start 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.966+0000 7ff61cc00640 1 -- start start 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.966+0000 7ff61cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff61819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.966+0000 7ff61cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff61810b4f0 0x7ff61819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff61cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61810be40 0x7ff6181a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff61cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff618115b00 con 0x7ff618105bb0 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff61cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff618115980 con 0x7ff61810be40 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff61cc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff618115c80 con 0x7ff61810b4f0 2024-09-21T07:30:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff61819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff61819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44876/0 (socket says 172.21.15.27:44876) 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 -- 172.21.15.27:0/2827218247 learned_addr learned my addr 172.21.15.27:0/2827218247 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff60ea00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff61810b4f0 0x7ff61819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617e00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61810be40 0x7ff6181a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 -- 172.21.15.27:0/2827218247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff61810b4f0 msgr2=0x7ff61819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff61810b4f0 0x7ff61819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 -- 172.21.15.27:0/2827218247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61810be40 msgr2=0x7ff6181a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61810be40 0x7ff6181a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.967+0000 7ff617400640 1 -- 172.21.15.27:0/2827218247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6181a2480 con 0x7ff618105bb0 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.968+0000 7ff617400640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff61819c510 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff608005940 tx=0x7ff6080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.968+0000 7ff614c00640 1 -- 172.21.15.27:0/2827218247 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60803f030 con 0x7ff618105bb0 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.968+0000 7ff614c00640 1 -- 172.21.15.27:0/2827218247 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff608003cb0 con 0x7ff618105bb0 2024-09-21T07:30:39.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.968+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6181a2710 con 0x7ff618105bb0 2024-09-21T07:30:39.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.968+0000 7ff614c00640 1 -- 172.21.15.27:0/2827218247 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608035070 con 0x7ff618105bb0 2024-09-21T07:30:39.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.968+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181a8ed0 con 0x7ff618105bb0 2024-09-21T07:30:39.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.969+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618106e30 con 0x7ff618105bb0 2024-09-21T07:30:39.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.972+0000 7ff614c00640 1 -- 172.21.15.27:0/2827218247 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff608042030 con 0x7ff618105bb0 2024-09-21T07:30:39.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.974+0000 7ff614c00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5fc0b6700 0x7ff5fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:39.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.974+0000 7ff614c00640 1 -- 172.21.15.27:0/2827218247 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9461+0+0 (secure 0 0 0) 0x7ff608108510 con 0x7ff618105bb0 2024-09-21T07:30:39.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.974+0000 7ff60ea00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5fc0b6700 0x7ff5fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:39.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.975+0000 7ff60ea00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5fc0b6700 0x7ff5fc0b8c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff61810b0c0 tx=0x7ff60000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:39.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:39.976+0000 7ff614c00640 1 -- 172.21.15.27:0/2827218247 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff6080d2a30 con 0x7ff618105bb0 2024-09-21T07:30:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.218+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff61810dc10 con 0x7ff618105bb0 2024-09-21T07:30:40.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.220+0000 7ff614c00640 1 -- 172.21.15.27:0/2827218247 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7ff6080d7970 con 0x7ff618105bb0 2024-09-21T07:30:40.222 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:30:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.224+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5fc0b6700 msgr2=0x7ff5fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.224+0000 7ff61cc00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5fc0b6700 0x7ff5fc0b8c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff61810b0c0 tx=0x7ff60000b390 comp rx=0 tx=0).stop 2024-09-21T07:30:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.224+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 msgr2=0x7ff61819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.224+0000 7ff61cc00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff61819c510 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff608005940 tx=0x7ff6080042f0 comp rx=0 tx=0).stop 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 shutdown_connections 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff61810b4f0 0x7ff61819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7ff5fc0b6700 0x7ff5fc0b8c10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61810be40 0x7ff6181a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 --2- 172.21.15.27:0/2827218247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff618105bb0 0x7ff61819c510 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 >> 172.21.15.27:0/2827218247 conn(0x7ff618101680 msgr2=0x7ff618103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 shutdown_connections 2024-09-21T07:30:40.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:40.225+0000 7ff61cc00640 1 -- 172.21.15.27:0/2827218247 wait complete. 2024-09-21T07:30:40.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a[30775]: 2024-09-21T07:30:40.218+0000 7fe9c2600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:30:40.594 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:30:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g[73427]: ignoring --setuser ceph since I am not root 2024-09-21T07:30:40.594 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:30:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g[73427]: ignoring --setgroup ceph since I am not root 2024-09-21T07:30:40.594 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:30:40 smithi027 ceph-mds[73431]: -- 172.21.15.27:0/810806763 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56455bb41860 con 0x56455fce9180 2024-09-21T07:30:40.594 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:30:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g[73427]: starting mds.g at 2024-09-21T07:30:40.677 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:30:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i[65220]: ignoring --setuser ceph since I am not root 2024-09-21T07:30:40.677 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:30:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i[65220]: ignoring --setgroup ceph since I am not root 2024-09-21T07:30:40.678 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:30:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i[65220]: starting mds.i at 2024-09-21T07:30:40.678 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:30:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c[64942]: ignoring --setuser ceph since I am not root 2024-09-21T07:30:40.678 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:30:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c[64942]: ignoring --setgroup ceph since I am not root 2024-09-21T07:30:40.678 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:30:40 smithi153 ceph-mds[64946]: -- 172.21.15.153:0/2515817291 <== mon.1 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558265b69c20 con 0x558269d11a80 2024-09-21T07:30:40.678 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:30:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c[64942]: starting mds.c at 2024-09-21T07:30:40.784 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: ignoring --setuser ceph since I am not root 2024-09-21T07:30:40.784 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: ignoring --setgroup ceph since I am not root 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-mds[63808]: -- 172.21.15.119:0/3213159025 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ed2ff5860 con 0x559ed719d180 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: starting mds.b at 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e[63249]: ignoring --setuser ceph since I am not root 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e[63249]: ignoring --setgroup ceph since I am not root 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-mds[63253]: -- 172.21.15.119:0/1406695605 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5d5233c20 con 0x55a5d93db180 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e[63249]: starting mds.e at 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h[63518]: ignoring --setuser ceph since I am not root 2024-09-21T07:30:40.785 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h[63518]: ignoring --setgroup ceph since I am not root 2024-09-21T07:30:40.786 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-mds[63522]: -- 172.21.15.119:0/2713090934 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561796b8b860 con 0x56179ad33180 2024-09-21T07:30:40.786 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:30:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h[63518]: starting mds.h at 2024-09-21T07:30:40.952 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:30:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:41 smithi027 ceph-mon[30779]: pgmap v931: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:30:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2827218247' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:30:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:41 smithi027 ceph-mon[30779]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T07:30:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:41 smithi027 ceph-mon[30779]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:30:41.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:41 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/2827218247' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T07:30:41.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:41 smithi027 ceph-mon[30779]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T07:30:41.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:41 smithi027 ceph-mon[30779]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T07:30:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:41 smithi153 ceph-mon[34747]: pgmap v931: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:30:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2827218247' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:30:41.427 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:41 smithi153 ceph-mon[34747]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T07:30:41.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:41 smithi153 ceph-mon[34747]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:30:41.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:41 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/2827218247' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T07:30:41.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:41 smithi153 ceph-mon[34747]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T07:30:41.428 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:41 smithi153 ceph-mon[34747]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T07:30:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:41 smithi119 ceph-mon[34519]: pgmap v931: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T07:30:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2827218247' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:30:41.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:41 smithi119 ceph-mon[34519]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T07:30:41.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:41 smithi119 ceph-mon[34519]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:30:41.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:41 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/2827218247' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T07:30:41.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:41 smithi119 ceph-mon[34519]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T07:30:41.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:41 smithi119 ceph-mon[34519]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 -- 172.21.15.27:0/142071959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 msgr2=0x7fd9e0112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/142071959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 0x7fd9e0112680 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009ca0 tx=0x7fd9d802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 -- 172.21.15.27:0/142071959 shutdown_connections 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/142071959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 0x7fd9e0112680 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/142071959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e010b500 0x7fd9e010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/142071959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e0105bc0 0x7fd9e0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 -- 172.21.15.27:0/142071959 >> 172.21.15.27:0/142071959 conn(0x7fd9e0101670 msgr2=0x7fd9e0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 -- 172.21.15.27:0/142071959 shutdown_connections 2024-09-21T07:30:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.033+0000 7fd9e4e00640 1 -- 172.21.15.27:0/142071959 wait complete. 2024-09-21T07:30:42.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.035+0000 7fd9e4e00640 1 Processor -- start 2024-09-21T07:30:42.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.035+0000 7fd9e4e00640 1 -- start start 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.035+0000 7fd9e4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e0105bc0 0x7fd9e019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.035+0000 7fd9e4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e010b500 0x7fd9e019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9e4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 0x7fd9e01a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9e4e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0115b60 con 0x7fd9e010be50 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9e4e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01159e0 con 0x7fd9e0105bc0 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9e4e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0115ce0 con 0x7fd9e010b500 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e0105bc0 0x7fd9e019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e0105bc0 0x7fd9e019c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:36458/0 (socket says 172.21.15.27:36458) 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 -- 172.21.15.27:0/3863368894 learned_addr learned my addr 172.21.15.27:0/3863368894 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9dfe00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 0x7fd9e01a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9d7e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e010b500 0x7fd9e019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 -- 172.21.15.27:0/3863368894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e010b500 msgr2=0x7fd9e019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e010b500 0x7fd9e019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 -- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 msgr2=0x7fd9e01a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 0x7fd9e01a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.036+0000 7fd9df400640 1 -- 172.21.15.27:0/3863368894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e01a2520 con 0x7fd9e0105bc0 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.037+0000 7fd9df400640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e0105bc0 0x7fd9e019c5b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000e9f0 tx=0x7fd9d000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.037+0000 7fd9dcc00640 1 -- 172.21.15.27:0/3863368894 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000cd80 con 0x7fd9e0105bc0 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.037+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e01a2810 con 0x7fd9e0105bc0 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.037+0000 7fd9dcc00640 1 -- 172.21.15.27:0/3863368894 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d00054d0 con 0x7fd9e0105bc0 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.037+0000 7fd9dcc00640 1 -- 172.21.15.27:0/3863368894 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0010690 con 0x7fd9e0105bc0 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.037+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e01a8f70 con 0x7fd9e0105bc0 2024-09-21T07:30:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.038+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e0105fd0 con 0x7fd9e0105bc0 2024-09-21T07:30:42.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.041+0000 7fd9dcc00640 1 -- 172.21.15.27:0/3863368894 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd9d0004e30 con 0x7fd9e0105bc0 2024-09-21T07:30:42.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.043+0000 7fd9dcc00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9c40b6680 0x7fd9c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:42.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.043+0000 7fd9dcc00640 1 -- 172.21.15.27:0/3863368894 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9631+0+0 (secure 0 0 0) 0x7fd9d00e8180 con 0x7fd9e0105bc0 2024-09-21T07:30:42.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.043+0000 7fd9d7e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9c40b6680 0x7fd9c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:42.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.044+0000 7fd9d7e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9c40b6680 0x7fd9c40b8b90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0104d70 tx=0x7fd9c8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:42.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.044+0000 7fd9dcc00640 1 -- 172.21.15.27:0/3863368894 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9d00ae220 con 0x7fd9e0105bc0 2024-09-21T07:30:42.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:42.283+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd9e002e640 con 0x7fd9e0105bc0 2024-09-21T07:30:42.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.153:6836/3128136414,v1:172.21.15.153:6838/3128136414] up:boot 2024-09-21T07:30:42.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.153:6837/1357564099,v1:172.21.15.153:6839/1357564099] up:boot 2024-09-21T07:30:42.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6834/2800664957,v1:172.21.15.119:6835/2800664957] up:boot 2024-09-21T07:30:42.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6836/623104286,v1:172.21.15.119:6838/623104286] up:boot 2024-09-21T07:30:42.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.119:6837/86449908,v1:172.21.15.119:6839/86449908] up:boot 2024-09-21T07:30:42.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: mds.? [v2:172.21.15.27:6838/3996241612,v1:172.21.15.27:6839/3996241612] up:boot 2024-09-21T07:30:42.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T07:30:42.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:30:42.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:30:42.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:30:42.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:30:42.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:30:42.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:42 smithi119 ceph-mon[34519]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:30:42.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.153:6836/3128136414,v1:172.21.15.153:6838/3128136414] up:boot 2024-09-21T07:30:42.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.153:6837/1357564099,v1:172.21.15.153:6839/1357564099] up:boot 2024-09-21T07:30:42.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6834/2800664957,v1:172.21.15.119:6835/2800664957] up:boot 2024-09-21T07:30:42.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6836/623104286,v1:172.21.15.119:6838/623104286] up:boot 2024-09-21T07:30:42.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.119:6837/86449908,v1:172.21.15.119:6839/86449908] up:boot 2024-09-21T07:30:42.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: mds.? [v2:172.21.15.27:6838/3996241612,v1:172.21.15.27:6839/3996241612] up:boot 2024-09-21T07:30:42.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T07:30:42.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:30:42.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:30:42.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:30:42.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:30:42.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:30:42.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:42 smithi027 ceph-mon[30779]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:30:42.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.153:6836/3128136414,v1:172.21.15.153:6838/3128136414] up:boot 2024-09-21T07:30:42.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.153:6837/1357564099,v1:172.21.15.153:6839/1357564099] up:boot 2024-09-21T07:30:42.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6834/2800664957,v1:172.21.15.119:6835/2800664957] up:boot 2024-09-21T07:30:42.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6836/623104286,v1:172.21.15.119:6838/623104286] up:boot 2024-09-21T07:30:42.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.119:6837/86449908,v1:172.21.15.119:6839/86449908] up:boot 2024-09-21T07:30:42.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: mds.? [v2:172.21.15.27:6838/3996241612,v1:172.21.15.27:6839/3996241612] up:boot 2024-09-21T07:30:42.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T07:30:42.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:30:42.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:30:42.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:30:42.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:30:42.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:30:42.679 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:42 smithi153 ceph-mon[34747]: from='mgr.14226 172.21.15.27:0/293903556' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:30:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.236+0000 7fd9dcc00640 1 -- 172.21.15.27:0/3863368894 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fd9d00b2600 con 0x7fd9e0105bc0 2024-09-21T07:30:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9c40b6680 msgr2=0x7fd9c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9c40b6680 0x7fd9c40b8b90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0104d70 tx=0x7fd9c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:30:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e0105bc0 msgr2=0x7fd9e019c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e0105bc0 0x7fd9e019c5b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000e9f0 tx=0x7fd9d000edf0 comp rx=0 tx=0).stop 2024-09-21T07:30:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 shutdown_connections 2024-09-21T07:30:43.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9e010b500 0x7fd9e019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:43.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fd9c40b6680 0x7fd9c40b8b90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:43.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9e010be50 0x7fd9e01a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:43.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 --2- 172.21.15.27:0/3863368894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9e0105bc0 0x7fd9e019c5b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:43.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 >> 172.21.15.27:0/3863368894 conn(0x7fd9e0101670 msgr2=0x7fd9e0108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:43.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 shutdown_connections 2024-09-21T07:30:43.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:43.240+0000 7fd9e4e00640 1 -- 172.21.15.27:0/3863368894 wait complete. 2024-09-21T07:30:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:43 smithi119 ceph-mon[34519]: pgmap v933: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:30:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:43 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3863368894' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:30:43.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:43 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:30:43.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:43 smithi027 ceph-mon[30779]: pgmap v933: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:30:43.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:43 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3863368894' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:30:43.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:43 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:30:43.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:43 smithi153 ceph-mon[34747]: pgmap v933: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-21T07:30:43.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:43 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3863368894' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:30:43.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:43 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:30:44.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:30:44.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:44 smithi027 ceph-mon[30779]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T07:30:44.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:44 smithi027 ceph-mon[30779]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T07:30:44.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:44 smithi027 ceph-mon[30779]: Cluster is now healthy 2024-09-21T07:30:44.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:44 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:30:44.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:44 smithi027 ceph-mon[30779]: fsmap 12 up:standby 2024-09-21T07:30:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:44.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:44 smithi119 ceph-mon[34519]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T07:30:44.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:44 smithi119 ceph-mon[34519]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T07:30:44.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:44 smithi119 ceph-mon[34519]: Cluster is now healthy 2024-09-21T07:30:44.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:44 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:30:44.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:44 smithi119 ceph-mon[34519]: fsmap 12 up:standby 2024-09-21T07:30:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:44 smithi153 ceph-mon[34747]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T07:30:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:44 smithi153 ceph-mon[34747]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T07:30:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:44 smithi153 ceph-mon[34747]: Cluster is now healthy 2024-09-21T07:30:44.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:44 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:30:44.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:44 smithi153 ceph-mon[34747]: fsmap 12 up:standby 2024-09-21T07:30:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 -- 172.21.15.27:0/1136393490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c077940 msgr2=0x7f8d4c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 --2- 172.21.15.27:0/1136393490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c077940 0x7f8d4c077d50 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009900 tx=0x7f8d3c02a410 comp rx=0 tx=0).stop 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 -- 172.21.15.27:0/1136393490 shutdown_connections 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 --2- 172.21.15.27:0/1136393490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d4c078290 0x7f8d4c114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 --2- 172.21.15.27:0/1136393490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d4c114570 0x7f8d4c116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 --2- 172.21.15.27:0/1136393490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c077940 0x7f8d4c077d50 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 -- 172.21.15.27:0/1136393490 >> 172.21.15.27:0/1136393490 conn(0x7f8d4c101680 msgr2=0x7f8d4c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 -- 172.21.15.27:0/1136393490 shutdown_connections 2024-09-21T07:30:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.137+0000 7f8d52000640 1 -- 172.21.15.27:0/1136393490 wait complete. 2024-09-21T07:30:45.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.138+0000 7f8d52000640 1 Processor -- start 2024-09-21T07:30:45.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.138+0000 7f8d52000640 1 -- start start 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d52000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d4c077940 0x7f8d4c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d52000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d4c078290 0x7f8d4c1a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d52000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c114570 0x7f8d4c1a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d52000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c119f40 con 0x7f8d4c114570 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d52000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c119dc0 con 0x7f8d4c078290 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d52000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c11a0c0 con 0x7f8d4c077940 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d50c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d4c077940 0x7f8d4c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d50c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d4c077940 0x7f8d4c1a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:36090/0 (socket says 172.21.15.27:36090) 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d51600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c114570 0x7f8d4c1a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d50c00640 1 -- 172.21.15.27:0/1764656519 learned_addr learned my addr 172.21.15.27:0/1764656519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.139+0000 7f8d43e00640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d4c078290 0x7f8d4c1a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d43e00640 1 -- 172.21.15.27:0/1764656519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d4c077940 msgr2=0x7f8d4c1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d43e00640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d4c077940 0x7f8d4c1a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d43e00640 1 -- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c114570 msgr2=0x7f8d4c1a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d43e00640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c114570 0x7f8d4c1a6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d43e00640 1 -- 172.21.15.27:0/1764656519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c1a6880 con 0x7f8d4c078290 2024-09-21T07:30:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d51600640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c114570 0x7f8d4c1a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d43e00640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d4c078290 0x7f8d4c1a0eb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c06c8c0 tx=0x7f8d3400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d4a000640 1 -- 172.21.15.27:0/1764656519 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400cc40 con 0x7f8d4c078290 2024-09-21T07:30:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.140+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c1a6b70 con 0x7f8d4c078290 2024-09-21T07:30:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.141+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c1ad2d0 con 0x7f8d4c078290 2024-09-21T07:30:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.141+0000 7f8d4a000640 1 -- 172.21.15.27:0/1764656519 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d340044d0 con 0x7f8d4c078290 2024-09-21T07:30:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.141+0000 7f8d4a000640 1 -- 172.21.15.27:0/1764656519 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3401c070 con 0x7f8d4c078290 2024-09-21T07:30:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.142+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d280052c0 con 0x7f8d4c078290 2024-09-21T07:30:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.144+0000 7f8d4a000640 1 -- 172.21.15.27:0/1764656519 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8d34005020 con 0x7f8d4c078290 2024-09-21T07:30:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.146+0000 7f8d4a000640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d300b6700 0x7f8d300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.146+0000 7f8d4a000640 1 -- 172.21.15.27:0/1764656519 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9631+0+0 (secure 0 0 0) 0x7f8d340e79b0 con 0x7f8d4c078290 2024-09-21T07:30:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.146+0000 7f8d50c00640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d300b6700 0x7f8d300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.147+0000 7f8d50c00640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d300b6700 0x7f8d300b8c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0096c0 tx=0x7f8d3c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:45.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.147+0000 7f8d4a000640 1 -- 172.21.15.27:0/1764656519 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d340adae0 con 0x7f8d4c078290 2024-09-21T07:30:45.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:45.347+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d280055b0 con 0x7f8d4c078290 2024-09-21T07:30:45.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:45 smithi119 ceph-mon[34519]: pgmap v934: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-21T07:30:45.578 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:45 smithi153 ceph-mon[34747]: pgmap v934: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-21T07:30:45.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:45 smithi027 ceph-mon[30779]: pgmap v934: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-21T07:30:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.260+0000 7f8d4a000640 1 -- 172.21.15.27:0/1764656519 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7f8d340b6030 con 0x7f8d4c078290 2024-09-21T07:30:46.263 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:30:46.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.265+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d300b6700 msgr2=0x7f8d300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:46.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.265+0000 7f8d52000640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d300b6700 0x7f8d300b8c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0096c0 tx=0x7f8d3c005a20 comp rx=0 tx=0).stop 2024-09-21T07:30:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.265+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d4c078290 msgr2=0x7f8d4c1a0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.265+0000 7f8d52000640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d4c078290 0x7f8d4c1a0eb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c06c8c0 tx=0x7f8d3400ee50 comp rx=0 tx=0).stop 2024-09-21T07:30:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.266+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 shutdown_connections 2024-09-21T07:30:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.266+0000 7f8d52000640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7f8d300b6700 0x7f8d300b8c10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.266+0000 7f8d52000640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d4c114570 0x7f8d4c1a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.266+0000 7f8d52000640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d4c078290 0x7f8d4c1a0eb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.266+0000 7f8d52000640 1 --2- 172.21.15.27:0/1764656519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d4c077940 0x7f8d4c1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.266+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 >> 172.21.15.27:0/1764656519 conn(0x7f8d4c101680 msgr2=0x7f8d4c1165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.267+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 shutdown_connections 2024-09-21T07:30:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:46.267+0000 7f8d52000640 1 -- 172.21.15.27:0/1764656519 wait complete. 2024-09-21T07:30:46.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:46 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/1764656519' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:46.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:46 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:46.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:46 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/1764656519' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:46.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:46 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:46.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:46 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/1764656519' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:46.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:46 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:46.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:30:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:47.525 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:47 smithi027 ceph-mon[30779]: pgmap v935: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 204 B/s wr, 4 op/s 2024-09-21T07:30:47.525 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:47 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:47.525 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:47 smithi027 ceph-mon[30779]: osdmap e112: 12 total, 12 up, 12 in 2024-09-21T07:30:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:47 smithi119 ceph-mon[34519]: pgmap v935: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 204 B/s wr, 4 op/s 2024-09-21T07:30:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:47 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:47 smithi119 ceph-mon[34519]: osdmap e112: 12 total, 12 up, 12 in 2024-09-21T07:30:47.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:47 smithi153 ceph-mon[34747]: pgmap v935: 129 pgs: 129 active+clean; 2.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 204 B/s wr, 4 op/s 2024-09-21T07:30:47.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:47 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:47.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:47 smithi153 ceph-mon[34747]: osdmap e112: 12 total, 12 up, 12 in 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.062+0000 7fafa9200640 1 -- 172.21.15.27:0/132545790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 msgr2=0x7fafa410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.062+0000 7fafa9200640 1 --2- 172.21.15.27:0/132545790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 0x7fafa410d860 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009900 tx=0x7faf8c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.063+0000 7fafa9200640 1 -- 172.21.15.27:0/132545790 shutdown_connections 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.063+0000 7fafa9200640 1 --2- 172.21.15.27:0/132545790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa410de20 0x7fafa4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.063+0000 7fafa9200640 1 --2- 172.21.15.27:0/132545790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 0x7fafa410d860 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.063+0000 7fafa9200640 1 --2- 172.21.15.27:0/132545790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 0x7fafa4104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.063+0000 7fafa9200640 1 -- 172.21.15.27:0/132545790 >> 172.21.15.27:0/132545790 conn(0x7fafa4079730 msgr2=0x7fafa4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.063+0000 7fafa9200640 1 -- 172.21.15.27:0/132545790 shutdown_connections 2024-09-21T07:30:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.063+0000 7fafa9200640 1 -- 172.21.15.27:0/132545790 wait complete. 2024-09-21T07:30:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 Processor -- start 2024-09-21T07:30:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 -- start start 2024-09-21T07:30:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 0x7fafa419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 0x7fafa419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa410de20 0x7fafa419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:48.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4113a80 con 0x7fafa4105310 2024-09-21T07:30:48.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4113900 con 0x7fafa41049c0 2024-09-21T07:30:48.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.064+0000 7fafa9200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4113c00 con 0x7fafa410de20 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 0x7fafa419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 0x7fafa419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.27:41734/0 (socket says 172.21.15.27:41734) 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 -- 172.21.15.27:0/951205210 learned_addr learned my addr 172.21.15.27:0/951205210 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa2800640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa410de20 0x7fafa419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7faf9be00640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 0x7fafa419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 -- 172.21.15.27:0/951205210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa410de20 msgr2=0x7fafa419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa410de20 0x7fafa419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 -- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 msgr2=0x7fafa419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 0x7fafa419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 -- 172.21.15.27:0/951205210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa41a0460 con 0x7fafa41049c0 2024-09-21T07:30:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.065+0000 7fafa1e00640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 0x7fafa419a4f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e9c0 tx=0x7faf9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.066+0000 7faf9aa00640 1 -- 172.21.15.27:0/951205210 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9400cde0 con 0x7fafa41049c0 2024-09-21T07:30:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.066+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa41a0750 con 0x7fafa41049c0 2024-09-21T07:30:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.066+0000 7faf9aa00640 1 -- 172.21.15.27:0/951205210 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf940108e0 con 0x7fafa41049c0 2024-09-21T07:30:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.066+0000 7faf9aa00640 1 -- 172.21.15.27:0/951205210 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf940188d0 con 0x7fafa41049c0 2024-09-21T07:30:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.066+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa41a6eb0 con 0x7fafa41049c0 2024-09-21T07:30:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.067+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa406c1e0 con 0x7fafa41049c0 2024-09-21T07:30:48.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.069+0000 7faf9aa00640 1 -- 172.21.15.27:0/951205210 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7faf9401f020 con 0x7fafa41049c0 2024-09-21T07:30:48.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.071+0000 7faf9aa00640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf880b6680 0x7faf880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:48.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.071+0000 7faf9aa00640 1 -- 172.21.15.27:0/951205210 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9135+0+0 (secure 0 0 0) 0x7faf940e7f30 con 0x7fafa41049c0 2024-09-21T07:30:48.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.071+0000 7faf9be00640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf880b6680 0x7faf880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.072+0000 7faf9be00640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf880b6680 0x7faf880b8b90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009230 tx=0x7faf8c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.072+0000 7faf9aa00640 1 -- 172.21.15.27:0/951205210 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf94119710 con 0x7fafa41049c0 2024-09-21T07:30:48.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:48.272+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fafa402e640 con 0x7fafa41049c0 2024-09-21T07:30:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.265+0000 7faf9aa00640 1 -- 172.21.15.27:0/951205210 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7faf940ae1c0 con 0x7fafa41049c0 2024-09-21T07:30:49.270 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-21T07:30:49.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf880b6680 msgr2=0x7faf880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:49.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf880b6680 0x7faf880b8b90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009230 tx=0x7faf8c031400 comp rx=0 tx=0).stop 2024-09-21T07:30:49.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 msgr2=0x7fafa419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:49.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 0x7fafa419a4f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e9c0 tx=0x7faf9400ee80 comp rx=0 tx=0).stop 2024-09-21T07:30:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 shutdown_connections 2024-09-21T07:30:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa410de20 0x7fafa419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7faf880b6680 0x7faf880b8b90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa4105310 0x7fafa419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 --2- 172.21.15.27:0/951205210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafa41049c0 0x7fafa419a4f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.271+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 >> 172.21.15.27:0/951205210 conn(0x7fafa4079730 msgr2=0x7fafa4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.272+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 shutdown_connections 2024-09-21T07:30:49.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:49.272+0000 7fafa9200640 1 -- 172.21.15.27:0/951205210 wait complete. 2024-09-21T07:30:49.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:49 smithi119 ceph-mon[34519]: pgmap v937: 97 pgs: 97 active+clean; 577 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:49.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:49 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/951205210' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:49.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:49 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:49.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:49 smithi027 ceph-mon[30779]: pgmap v937: 97 pgs: 97 active+clean; 577 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:49.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:49 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/951205210' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:49.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:49 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:49.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:49 smithi153 ceph-mon[34747]: pgmap v937: 97 pgs: 97 active+clean; 577 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:49.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:49 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/951205210' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:49.678 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:49 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:50.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T07:30:50.423 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:30:50.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:50 smithi119 ceph-mon[34519]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:50.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:50 smithi119 ceph-mon[34519]: osdmap e113: 12 total, 12 up, 12 in 2024-09-21T07:30:50.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:50 smithi027 ceph-mon[30779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:50.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:50 smithi027 ceph-mon[30779]: osdmap e113: 12 total, 12 up, 12 in 2024-09-21T07:30:50.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:50 smithi153 ceph-mon[34747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:50.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:50 smithi153 ceph-mon[34747]: osdmap e113: 12 total, 12 up, 12 in 2024-09-21T07:30:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.027+0000 7fb31ce00640 1 -- 172.21.15.27:0/3435473570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 msgr2=0x7fb318075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.027+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3435473570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb318075b40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb308009900 tx=0x7fb30802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.027+0000 7fb31ce00640 1 -- 172.21.15.27:0/3435473570 shutdown_connections 2024-09-21T07:30:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.027+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3435473570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31810ff90 0x7fb3181123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.027+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3435473570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb318076080 0x7fb31810fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.027+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3435473570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb318075b40 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.027+0000 7fb31ce00640 1 -- 172.21.15.27:0/3435473570 >> 172.21.15.27:0/3435473570 conn(0x7fb3180ff400 msgr2=0x7fb318101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.028+0000 7fb31ce00640 1 -- 172.21.15.27:0/3435473570 shutdown_connections 2024-09-21T07:30:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.028+0000 7fb31ce00640 1 -- 172.21.15.27:0/3435473570 wait complete. 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 Processor -- start 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 -- start start 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb31819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb318076080 0x7fb31819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31810ff90 0x7fb3181a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb318115ad0 con 0x7fb318075730 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb318115950 con 0x7fb31810ff90 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb31ce00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb318115c50 con 0x7fb318076080 2024-09-21T07:30:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb317400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb31819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb317400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb31819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53406/0 (socket says 172.21.15.27:53406) 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.029+0000 7fb317400640 1 -- 172.21.15.27:0/3618740096 learned_addr learned my addr 172.21.15.27:0/3618740096 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb316a00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb318076080 0x7fb31819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb317e00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31810ff90 0x7fb3181a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb317400640 1 -- 172.21.15.27:0/3618740096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb318076080 msgr2=0x7fb31819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb317400640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb318076080 0x7fb31819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb317400640 1 -- 172.21.15.27:0/3618740096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31810ff90 msgr2=0x7fb3181a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb317400640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31810ff90 0x7fb3181a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb317400640 1 -- 172.21.15.27:0/3618740096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3181a2410 con 0x7fb318075730 2024-09-21T07:30:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb316a00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb318076080 0x7fb31819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:30:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.030+0000 7fb317400640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb31819c4a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb308005940 tx=0x7fb308004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.031+0000 7fb30fe00640 1 -- 172.21.15.27:0/3618740096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30803f030 con 0x7fb318075730 2024-09-21T07:30:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.031+0000 7fb30fe00640 1 -- 172.21.15.27:0/3618740096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb308003aa0 con 0x7fb318075730 2024-09-21T07:30:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.031+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3181a26a0 con 0x7fb318075730 2024-09-21T07:30:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.031+0000 7fb30fe00640 1 -- 172.21.15.27:0/3618740096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308035070 con 0x7fb318075730 2024-09-21T07:30:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.031+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3181a8e60 con 0x7fb318075730 2024-09-21T07:30:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.032+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb318076c40 con 0x7fb318075730 2024-09-21T07:30:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.035+0000 7fb30fe00640 1 -- 172.21.15.27:0/3618740096 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb308042030 con 0x7fb318075730 2024-09-21T07:30:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.037+0000 7fb30fe00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2fc0b6700 0x7fb2fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:30:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.037+0000 7fb30fe00640 1 -- 172.21.15.27:0/3618740096 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8518+0+0 (secure 0 0 0) 0x7fb308107f50 con 0x7fb318075730 2024-09-21T07:30:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.038+0000 7fb316a00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2fc0b6700 0x7fb2fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:30:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.038+0000 7fb316a00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2fc0b6700 0x7fb2fc0b8c10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb3181124d0 tx=0x7fb30400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:30:51.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.040+0000 7fb30fe00640 1 -- 172.21.15.27:0/3618740096 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb3080ce440 con 0x7fb318075730 2024-09-21T07:30:51.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.240+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb318075b40 con 0x7fb318075730 2024-09-21T07:30:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.285+0000 7fb30fe00640 1 -- 172.21.15.27:0/3618740096 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v114) ==== 178+0+0 (secure 0 0 0) 0x7fb3080d2820 con 0x7fb318075730 2024-09-21T07:30:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' removed 2024-09-21T07:30:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.288+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2fc0b6700 msgr2=0x7fb2fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.288+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2fc0b6700 0x7fb2fc0b8c10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb3181124d0 tx=0x7fb30400b410 comp rx=0 tx=0).stop 2024-09-21T07:30:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.288+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 msgr2=0x7fb31819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:30:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.288+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb31819c4a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb308005940 tx=0x7fb308004270 comp rx=0 tx=0).stop 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.289+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 shutdown_connections 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.289+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb318076080 0x7fb31819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.289+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:6800/3854502753,v1:172.21.15.27:6801/3854502753] conn(0x7fb2fc0b6700 0x7fb2fc0b8c10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.289+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31810ff90 0x7fb3181a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.289+0000 7fb31ce00640 1 --2- 172.21.15.27:0/3618740096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb318075730 0x7fb31819c4a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.289+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 >> 172.21.15.27:0/3618740096 conn(0x7fb3180ff400 msgr2=0x7fb318101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.289+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 shutdown_connections 2024-09-21T07:30:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T07:30:51.290+0000 7fb31ce00640 1 -- 172.21.15.27:0/3618740096 wait complete. 2024-09-21T07:30:51.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:51 smithi119 ceph-mon[34519]: pgmap v939: 33 pgs: 33 active+clean; 577 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:51.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:51 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3618740096' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:51.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:51 smithi027 ceph-mon[30779]: pgmap v939: 33 pgs: 33 active+clean; 577 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:51.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:51 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3618740096' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:51.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:51 smithi153 ceph-mon[34747]: pgmap v939: 33 pgs: 33 active+clean; 577 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:51.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:51 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3618740096' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:30:52.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:30:52.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:30:52.167 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:30:52.200 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T07:30:52.201 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T07:30:52.237 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T07:30:52.255 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T07:30:52.274 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T07:30:52.274 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T07:30:52.274 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a 2024-09-21T07:30:52.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:52 smithi119 ceph-mon[34519]: from='client.? 172.21.15.27:0/3618740096' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:52.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:52 smithi119 ceph-mon[34519]: osdmap e114: 12 total, 12 up, 12 in 2024-09-21T07:30:52.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:52 smithi119 ceph-mon[34519]: pgmap v941: 1 pgs: 1 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:52.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:52 smithi027 ceph-mon[30779]: from='client.? 172.21.15.27:0/3618740096' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:52.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:52 smithi027 ceph-mon[30779]: osdmap e114: 12 total, 12 up, 12 in 2024-09-21T07:30:52.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:52 smithi027 ceph-mon[30779]: pgmap v941: 1 pgs: 1 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:52.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:52 smithi027 systemd[1]: Stopping Ceph mon.a for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:30:52.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:52 smithi153 ceph-mon[34747]: from='client.? 172.21.15.27:0/3618740096' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:30:52.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:52 smithi153 ceph-mon[34747]: osdmap e114: 12 total, 12 up, 12 in 2024-09-21T07:30:52.677 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:52 smithi153 ceph-mon[34747]: pgmap v941: 1 pgs: 1 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:30:52.975 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:52 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a[30775]: 2024-09-21T07:30:52.696+0000 7fe9c9400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:30:52.976 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:52 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a[30775]: 2024-09-21T07:30:52.696+0000 7fe9c9400640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T07:30:53.271 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:52 smithi027 podman[128319]: 2024-09-21 07:30:52.973913443 +0000 UTC m=+0.400079561 container died cd5196f7b24b359047ef0e17b8abab361d2b49b8047e84599ebeeb4a6df11e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:30:53.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:53 smithi027 podman[128319]: 2024-09-21 07:30:53.401578665 +0000 UTC m=+0.827744782 container cleanup cd5196f7b24b359047ef0e17b8abab361d2b49b8047e84599ebeeb4a6df11e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:30:53.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:53 smithi027 bash[128319]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a 2024-09-21T07:30:53.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 07:30:53 smithi027 podman[128331]: 2024-09-21 07:30:53.653658518 +0000 UTC m=+0.669256166 container remove cd5196f7b24b359047ef0e17b8abab361d2b49b8047e84599ebeeb4a6df11e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T07:30:53.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T07:30:54.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:30:54.051 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T07:30:54.051 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T07:30:54.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.b 2024-09-21T07:30:54.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:54 smithi119 systemd[1]: Stopping Ceph mon.b for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:30:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b[34515]: 2024-09-21T07:30:54.388+0000 7f61d0860640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:30:54.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b[34515]: 2024-09-21T07:30:54.388+0000 7f61d0860640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T07:30:54.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:54 smithi119 podman[67743]: 2024-09-21 07:30:54.593501231 +0000 UTC m=+0.339503154 container died fcfb2e88df905fc54d4ccd1d691c80571186dae25445f550413b74536d67e6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T07:30:55.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:55 smithi119 podman[67743]: 2024-09-21 07:30:55.006204567 +0000 UTC m=+0.752206490 container cleanup fcfb2e88df905fc54d4ccd1d691c80571186dae25445f550413b74536d67e6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T07:30:55.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:55 smithi119 bash[67743]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b 2024-09-21T07:30:55.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T07:30:55.504 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:55 smithi119 podman[67755]: 2024-09-21 07:30:55.25000768 +0000 UTC m=+0.655716560 container remove fcfb2e88df905fc54d4ccd1d691c80571186dae25445f550413b74536d67e6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T07:30:55.504 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:55 smithi119 systemd[1]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.b.service: Deactivated successfully. 2024-09-21T07:30:55.504 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:55 smithi119 systemd[1]: Stopped Ceph mon.b for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:30:55.504 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 07:30:55 smithi119 systemd[1]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.b.service: Consumed 20.121s CPU time. 2024-09-21T07:30:55.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:30:55.535 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T07:30:55.538 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T07:30:55.538 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.c 2024-09-21T07:30:55.866 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:55 smithi153 systemd[1]: Stopping Ceph mon.c for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:30:56.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c[34743]: 2024-09-21T07:30:55.864+0000 7fa8915a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:30:56.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c[34743]: 2024-09-21T07:30:55.864+0000 7fa8915a4640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T07:30:56.177 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:56 smithi153 podman[69814]: 2024-09-21 07:30:56.048852364 +0000 UTC m=+0.311722590 container died 51e0fe7fc2e44923133069353694036923720e3ab18c792dd6d5c2c4b7932b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T07:30:56.802 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:56 smithi153 podman[69814]: 2024-09-21 07:30:56.56612005 +0000 UTC m=+0.828990318 container cleanup 51e0fe7fc2e44923133069353694036923720e3ab18c792dd6d5c2c4b7932b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T07:30:56.802 INFO:journalctl@ceph.mon.c.smithi153.stdout:Sep 21 07:30:56 smithi153 bash[69814]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mon-c 2024-09-21T07:30:56.934 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T07:30:56.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:30:56.981 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T07:30:56.981 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T07:30:56.981 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.x 2024-09-21T07:30:57.341 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:30:57 smithi027 systemd[1]: Stopping Ceph mgr.x for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:30:57.810 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:30:57 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:30:57.393+0000 7f76c4200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:30:57.810 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:30:57 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x[31004]: 2024-09-21T07:30:57.393+0000 7f76c4200640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T07:30:57.810 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:30:57 smithi027 podman[128441]: 2024-09-21 07:30:57.485754842 +0000 UTC m=+0.217112293 container died 56905a59dc48dac07207282cc6b01789f213fe62765f915a7b19112ed0181706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T07:30:58.064 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:30:58 smithi027 podman[128441]: 2024-09-21 07:30:58.057922638 +0000 UTC m=+0.789280088 container cleanup 56905a59dc48dac07207282cc6b01789f213fe62765f915a7b19112ed0181706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T07:30:58.064 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:30:58 smithi027 bash[128441]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x 2024-09-21T07:30:58.518 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 07:30:58 smithi027 podman[128452]: 2024-09-21 07:30:58.260037605 +0000 UTC m=+0.760657154 container remove 56905a59dc48dac07207282cc6b01789f213fe62765f915a7b19112ed0181706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T07:30:58.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T07:30:58.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:30:58.601 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T07:30:58.601 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T07:30:58.601 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.y 2024-09-21T07:30:58.933 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:30:58 smithi119 systemd[1]: Stopping Ceph mgr.y for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:30:59.214 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:30:58 smithi119 podman[67866]: 2024-09-21 07:30:58.932057648 +0000 UTC m=+0.127767706 container died b4e81ec188d43802e3b7d660dcd7d9ad0c5985e5c3fca023144cd59685444d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-y, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T07:30:59.534 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:30:59 smithi119 podman[67866]: 2024-09-21 07:30:59.320023178 +0000 UTC m=+0.515733235 container cleanup b4e81ec188d43802e3b7d660dcd7d9ad0c5985e5c3fca023144cd59685444d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:30:59.534 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 07:30:59 smithi119 bash[67866]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-y 2024-09-21T07:30:59.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T07:30:59.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:30:59.779 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T07:30:59.779 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T07:30:59.779 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.z 2024-09-21T07:31:00.109 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 21 07:30:59 smithi153 systemd[1]: Stopping Ceph mgr.z for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:00.406 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 21 07:31:00 smithi153 podman[69937]: 2024-09-21 07:31:00.108437167 +0000 UTC m=+0.136707700 container died f6eae6f2140abdcd328e561ae0cda4ca3bfc87b9fcf75c40a87c8fa53f7d63f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-z, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:00.677 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 21 07:31:00 smithi153 podman[69937]: 2024-09-21 07:31:00.521595484 +0000 UTC m=+0.549866014 container cleanup f6eae6f2140abdcd328e561ae0cda4ca3bfc87b9fcf75c40a87c8fa53f7d63f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T07:31:00.677 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 21 07:31:00 smithi153 bash[69937]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-z 2024-09-21T07:31:01.084 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 21 07:31:00 smithi153 podman[69948]: 2024-09-21 07:31:00.806972247 +0000 UTC m=+0.694023596 container remove f6eae6f2140abdcd328e561ae0cda4ca3bfc87b9fcf75c40a87c8fa53f7d63f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:01.084 INFO:journalctl@ceph.mgr.z.smithi153.stdout:Sep 21 07:31:00 smithi153 systemd[1]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-21T07:31:01.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T07:31:01.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:01.147 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T07:31:01.147 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T07:31:01.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.0 2024-09-21T07:31:01.559 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:01 smithi027 systemd[1]: Stopping Ceph osd.0 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:01.841 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0[48167]: 2024-09-21T07:31:01.557+0000 7f3c2ec00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:01.841 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0[48167]: 2024-09-21T07:31:01.557+0000 7f3c2ec00640 -1 osd.0 114 *** Got signal Terminated *** 2024-09-21T07:31:01.841 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0[48167]: 2024-09-21T07:31:01.557+0000 7f3c2ec00640 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:07.266 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:06 smithi027 podman[128563]: 2024-09-21 07:31:06.953484318 +0000 UTC m=+5.511229771 container stop bd1cad9b3836c472619538ce98f7b9a75d6d5ab99b7aaa54b05535d6a0a1ce3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:31:07.266 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:06 smithi027 podman[128563]: 2024-09-21 07:31:06.97947081 +0000 UTC m=+5.537216252 container died bd1cad9b3836c472619538ce98f7b9a75d6d5ab99b7aaa54b05535d6a0a1ce3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:07.516 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:07 smithi027 podman[128563]: 2024-09-21 07:31:07.479518723 +0000 UTC m=+6.037264164 container cleanup bd1cad9b3836c472619538ce98f7b9a75d6d5ab99b7aaa54b05535d6a0a1ce3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T07:31:07.516 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:07 smithi027 bash[128563]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0 2024-09-21T07:31:07.841 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:07 smithi027 podman[128582]: 2024-09-21 07:31:07.748332941 +0000 UTC m=+0.774862337 container remove bd1cad9b3836c472619538ce98f7b9a75d6d5ab99b7aaa54b05535d6a0a1ce3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T07:31:08.591 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:08 smithi027 podman[128673]: 2024-09-21 07:31:08.217260652 +0000 UTC m=+0.027899309 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:31:08.591 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:08 smithi027 podman[128673]: 2024-09-21 07:31:08.324236218 +0000 UTC m=+0.134874864 container create 94c7b1c4c2a80e0456454d0de147a831c1dd2e7793113171a4667baac85cd8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:08.919 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:08 smithi027 podman[128673]: 2024-09-21 07:31:08.644560326 +0000 UTC m=+0.455198998 container init 94c7b1c4c2a80e0456454d0de147a831c1dd2e7793113171a4667baac85cd8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:08.919 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:08 smithi027 podman[128673]: 2024-09-21 07:31:08.651548936 +0000 UTC m=+0.462187591 container start 94c7b1c4c2a80e0456454d0de147a831c1dd2e7793113171a4667baac85cd8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:31:08.920 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:08 smithi027 podman[128673]: 2024-09-21 07:31:08.74404383 +0000 UTC m=+0.554682492 container attach 94c7b1c4c2a80e0456454d0de147a831c1dd2e7793113171a4667baac85cd8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:09.273 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 07:31:08 smithi027 podman[128673]: 2024-09-21 07:31:08.918600175 +0000 UTC m=+0.729238844 container died 94c7b1c4c2a80e0456454d0de147a831c1dd2e7793113171a4667baac85cd8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T07:31:09.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T07:31:09.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:09.810 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T07:31:09.810 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T07:31:09.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.1 2024-09-21T07:31:10.101 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:09 smithi119 systemd[1]: Stopping Ceph osd.1 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:10.534 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1[39229]: 2024-09-21T07:31:10.099+0000 7f94d9d5b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:10.534 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1[39229]: 2024-09-21T07:31:10.099+0000 7f94d9d5b640 -1 osd.1 114 *** Got signal Terminated *** 2024-09-21T07:31:10.534 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1[39229]: 2024-09-21T07:31:10.099+0000 7f94d9d5b640 -1 osd.1 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:15.952 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:15 smithi119 podman[67990]: 2024-09-21 07:31:15.64660551 +0000 UTC m=+5.647968353 container died f43cc6c1e219edf8231ba598f700c08bb86c90497e48cb5c78400c1621e14b17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T07:31:16.284 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 podman[67990]: 2024-09-21 07:31:16.03301078 +0000 UTC m=+6.034373622 container cleanup f43cc6c1e219edf8231ba598f700c08bb86c90497e48cb5c78400c1621e14b17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T07:31:16.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 bash[67990]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1 2024-09-21T07:31:16.608 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 podman[68007]: 2024-09-21 07:31:16.301348434 +0000 UTC m=+0.655790197 container remove f43cc6c1e219edf8231ba598f700c08bb86c90497e48cb5c78400c1621e14b17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:16.881 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 podman[68096]: 2024-09-21 07:31:16.511150136 +0000 UTC m=+0.016082743 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:31:16.881 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 podman[68096]: 2024-09-21 07:31:16.635636709 +0000 UTC m=+0.140569330 container create d1bc4e71bf1c33f76f191b73b6ba0fa204926499339dc909f576c1c4cd021495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:16.881 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 podman[68096]: 2024-09-21 07:31:16.880320241 +0000 UTC m=+0.385252848 container init d1bc4e71bf1c33f76f191b73b6ba0fa204926499339dc909f576c1c4cd021495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T07:31:17.284 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 podman[68096]: 2024-09-21 07:31:16.88395188 +0000 UTC m=+0.388884487 container start d1bc4e71bf1c33f76f191b73b6ba0fa204926499339dc909f576c1c4cd021495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:31:17.284 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:16 smithi119 podman[68096]: 2024-09-21 07:31:16.972067075 +0000 UTC m=+0.476999688 container attach d1bc4e71bf1c33f76f191b73b6ba0fa204926499339dc909f576c1c4cd021495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T07:31:17.285 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 07:31:17 smithi119 podman[68096]: 2024-09-21 07:31:17.077112364 +0000 UTC m=+0.582044977 container died d1bc4e71bf1c33f76f191b73b6ba0fa204926499339dc909f576c1c4cd021495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:17.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T07:31:17.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:17.775 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T07:31:17.775 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T07:31:17.775 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.2 2024-09-21T07:31:18.091 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:17 smithi153 systemd[1]: Stopping Ceph osd.2 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:18.354 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2[40027]: 2024-09-21T07:31:18.091+0000 7fee10ec4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:18.354 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2[40027]: 2024-09-21T07:31:18.092+0000 7fee10ec4640 -1 osd.2 114 *** Got signal Terminated *** 2024-09-21T07:31:18.354 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2[40027]: 2024-09-21T07:31:18.092+0000 7fee10ec4640 -1 osd.2 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:23.829 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:23 smithi153 podman[70062]: 2024-09-21 07:31:23.534789627 +0000 UTC m=+5.565132938 container died ef9399ef0a239ba007100c292458fbf128ab84bbb19b906fbe8d18cc2214ac8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:24.177 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:23 smithi153 podman[70062]: 2024-09-21 07:31:23.909106993 +0000 UTC m=+5.939450292 container cleanup ef9399ef0a239ba007100c292458fbf128ab84bbb19b906fbe8d18cc2214ac8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:31:24.177 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:23 smithi153 bash[70062]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2 2024-09-21T07:31:24.529 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:24 smithi153 podman[70089]: 2024-09-21 07:31:24.185846123 +0000 UTC m=+0.648663033 container remove ef9399ef0a239ba007100c292458fbf128ab84bbb19b906fbe8d18cc2214ac8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:24.790 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:24 smithi153 podman[70177]: 2024-09-21 07:31:24.430254248 +0000 UTC m=+0.016194236 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:31:24.790 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:24 smithi153 podman[70177]: 2024-09-21 07:31:24.545300323 +0000 UTC m=+0.131240304 container create 201cd9582f219fda98e45594125c0f15a7a5239c493602847dc49c482db9ff95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:31:24.791 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:24 smithi153 podman[70177]: 2024-09-21 07:31:24.789619496 +0000 UTC m=+0.375559477 container init 201cd9582f219fda98e45594125c0f15a7a5239c493602847dc49c482db9ff95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T07:31:25.177 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:24 smithi153 podman[70177]: 2024-09-21 07:31:24.793078879 +0000 UTC m=+0.379018862 container start 201cd9582f219fda98e45594125c0f15a7a5239c493602847dc49c482db9ff95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:31:25.177 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:24 smithi153 podman[70177]: 2024-09-21 07:31:24.881505837 +0000 UTC m=+0.467445833 container attach 201cd9582f219fda98e45594125c0f15a7a5239c493602847dc49c482db9ff95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:25.178 INFO:journalctl@ceph.osd.2.smithi153.stdout:Sep 21 07:31:24 smithi153 podman[70177]: 2024-09-21 07:31:24.994463115 +0000 UTC m=+0.580403097 container died 201cd9582f219fda98e45594125c0f15a7a5239c493602847dc49c482db9ff95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:25.604 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T07:31:25.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:25.646 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T07:31:25.646 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T07:31:25.646 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.3 2024-09-21T07:31:26.091 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:25 smithi027 systemd[1]: Stopping Ceph osd.3 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:26.591 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:26 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:31:26.120+0000 7f7c11c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:26.591 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:26 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:31:26.120+0000 7f7c11c00640 -1 osd.3 114 *** Got signal Terminated *** 2024-09-21T07:31:26.591 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:26 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:31:26.120+0000 7f7c11c00640 -1 osd.3 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:28.557 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:28 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:28.145+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:28.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:28.282+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:28.841 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:28 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:31:28.555+0000 7f7c08600640 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:04.978702+0000 front 2024-09-21T07:31:04.978836+0000 (oldest deadline 2024-09-21T07:31:27.878863+0000) 2024-09-21T07:31:29.516 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:29 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:29.152+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:29.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:29.293+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:29.841 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:29 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:31:29.515+0000 7f7c08600640 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:04.978702+0000 front 2024-09-21T07:31:04.978836+0000 (oldest deadline 2024-09-21T07:31:27.878863+0000) 2024-09-21T07:31:30.544 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:30 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:30.131+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:30.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:30.255+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:30.841 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:30 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3[54762]: 2024-09-21T07:31:30.541+0000 7f7c08600640 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:04.978702+0000 front 2024-09-21T07:31:04.978836+0000 (oldest deadline 2024-09-21T07:31:27.878863+0000) 2024-09-21T07:31:31.502 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:31 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:31.100+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:31.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:31.213+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:31.753 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:31 smithi027 podman[128773]: 2024-09-21 07:31:31.50120767 +0000 UTC m=+5.550353223 container died 0cda1597c0d77ff4d548337f61402d40aa375932e90597638220c4d33ee931a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T07:31:32.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:31.793+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:32.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:32 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:32.135+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:32.341 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:32 smithi027 podman[128773]: 2024-09-21 07:31:32.009603347 +0000 UTC m=+6.058748895 container cleanup 0cda1597c0d77ff4d548337f61402d40aa375932e90597638220c4d33ee931a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T07:31:32.341 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:32 smithi027 bash[128773]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3 2024-09-21T07:31:32.341 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:32 smithi027 podman[128791]: 2024-09-21 07:31:32.194461279 +0000 UTC m=+0.687512157 container remove 0cda1597c0d77ff4d548337f61402d40aa375932e90597638220c4d33ee931a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:31:32.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:32.225+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:33.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:32 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:32.641+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:33.065 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:32 smithi027 podman[128880]: 2024-09-21 07:31:32.644977794 +0000 UTC m=+0.028057690 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:31:33.065 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:32 smithi027 podman[128880]: 2024-09-21 07:31:32.811828246 +0000 UTC m=+0.194908117 container create 01921cfcc8d06bd255b839a2b941b2d9ce9c78320e2bcd0530ed0a7309993b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True) 2024-09-21T07:31:33.065 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:33 smithi027 podman[128880]: 2024-09-21 07:31:33.064354236 +0000 UTC m=+0.447434111 container init 01921cfcc8d06bd255b839a2b941b2d9ce9c78320e2bcd0530ed0a7309993b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:33.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:32.799+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:33.177 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:33.012+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:33.339 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:33 smithi027 podman[128880]: 2024-09-21 07:31:33.071576029 +0000 UTC m=+0.454655910 container start 01921cfcc8d06bd255b839a2b941b2d9ce9c78320e2bcd0530ed0a7309993b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:33.339 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:33 smithi027 podman[128880]: 2024-09-21 07:31:33.16468315 +0000 UTC m=+0.547763024 container attach 01921cfcc8d06bd255b839a2b941b2d9ce9c78320e2bcd0530ed0a7309993b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:33.339 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:33 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:33.147+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:33.591 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 07:31:33 smithi027 podman[128880]: 2024-09-21 07:31:33.33805408 +0000 UTC m=+0.721133960 container died 01921cfcc8d06bd255b839a2b941b2d9ce9c78320e2bcd0530ed0a7309993b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T07:31:33.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:33.181+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:33.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T07:31:34.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:33 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:33.683+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:34.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:34.074 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T07:31:34.074 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T07:31:34.074 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.4 2024-09-21T07:31:34.135 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:33.820+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:34.135 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:34.050+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:34.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:34 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:34.111+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:34.381 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:34 smithi119 systemd[1]: Stopping Ceph osd.4 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:34.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:34.132+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:34.656 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:34 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:31:34.379+0000 7f2b15f76640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:34.656 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:34 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:31:34.379+0000 7f2b15f76640 -1 osd.4 114 *** Got signal Terminated *** 2024-09-21T07:31:34.656 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:34 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:31:34.379+0000 7f2b15f76640 -1 osd.4 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:35.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:34 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:34.654+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:35.089 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:34.809+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:35.089 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:35 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:35.018+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:35.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:35 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:35.134+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:35.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:35 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:35.157+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:36.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:35 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:35.696+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:36.089 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:35 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:35.827+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:36.090 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:36 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:36.045+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:36.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:36 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:36.088+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:36.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:36 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:36.112+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:37.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:36 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:36.728+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:37.128 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:37 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:37.001+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:37.128 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:36 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:36.859+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:37.129 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:36 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:36.859+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:37.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:37 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:37.069+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:37.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:37 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:37.078+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:37.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:37 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:37.126+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:37.534 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:37 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:31:37.130+0000 7f2b0e559640 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:12.855148+0000 front 2024-09-21T07:31:12.855251+0000 (oldest deadline 2024-09-21T07:31:36.955325+0000) 2024-09-21T07:31:38.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:37 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:37.750+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:38.157 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:37 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:37.966+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:38.158 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:37 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:37.852+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:38.158 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:37 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:37.852+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:38.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:38 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:38.043+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:38.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:38 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:38.107+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:38.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:38 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:38.155+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:38.479 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:38 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:31:38.167+0000 7f2b0e559640 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:12.855148+0000 front 2024-09-21T07:31:12.855251+0000 (oldest deadline 2024-09-21T07:31:36.955325+0000) 2024-09-21T07:31:38.784 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:38 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:38.717+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:38.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:38 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:38.477+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:39.177 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:39 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:39.011+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:39.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:38 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:38.861+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:39.178 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:38 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:38.861+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:39.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:39 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:39.013+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:39.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:39 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:39.132+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:39.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:39 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:39.132+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:39.480 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:39 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4[45812]: 2024-09-21T07:31:39.119+0000 7f2b0e559640 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:12.855148+0000 front 2024-09-21T07:31:12.855251+0000 (oldest deadline 2024-09-21T07:31:36.955325+0000) 2024-09-21T07:31:39.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:39 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:39.180+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:39.677 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:39 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:39.180+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:39.784 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:39 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:39.702+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:39.785 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:39 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:39.478+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:40.147 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:40.036+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:40.147 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:39 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:39.847+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:40.147 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:39 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:39.847+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:40.155 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:39 smithi119 podman[68211]: 2024-09-21 07:31:39.811642915 +0000 UTC m=+5.540660068 container died f65d1eb1132607a6a55e0b7a974fef0bd227ac4b4d1d0df7f62b43ef51b8d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:31:40.155 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:40 smithi119 podman[68211]: 2024-09-21 07:31:40.154225594 +0000 UTC m=+5.883242747 container cleanup f65d1eb1132607a6a55e0b7a974fef0bd227ac4b4d1d0df7f62b43ef51b8d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T07:31:40.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:40.061+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:40.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:40.179+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:40.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:40 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:40.179+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:40.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:40.145+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:40.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:40.145+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:40.467 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:40 smithi119 bash[68211]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4 2024-09-21T07:31:40.467 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:40 smithi119 podman[68228]: 2024-09-21 07:31:40.355861639 +0000 UTC m=+0.543371146 container remove f65d1eb1132607a6a55e0b7a974fef0bd227ac4b4d1d0df7f62b43ef51b8d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T07:31:40.767 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:40.465+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:40.767 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:40.658+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:40.768 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:40 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:40.658+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:41.026 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:40 smithi119 podman[68318]: 2024-09-21 07:31:40.669004182 +0000 UTC m=+0.015957202 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:31:41.027 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:40 smithi119 podman[68318]: 2024-09-21 07:31:40.78194317 +0000 UTC m=+0.128896230 container create 1e5ae8f1bb39f23116efbb98212c474db5b9d9302a66958cca329618a72b0d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:41.027 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:41 smithi119 podman[68318]: 2024-09-21 07:31:41.025837667 +0000 UTC m=+0.372790680 container init 1e5ae8f1bb39f23116efbb98212c474db5b9d9302a66958cca329618a72b0d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T07:31:41.086 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:40.809+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:41.087 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:40 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:40.809+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:41.284 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:41 smithi119 podman[68318]: 2024-09-21 07:31:41.02921579 +0000 UTC m=+0.376168814 container start 1e5ae8f1bb39f23116efbb98212c474db5b9d9302a66958cca329618a72b0d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:41.285 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:41 smithi119 podman[68318]: 2024-09-21 07:31:41.126428835 +0000 UTC m=+0.473381862 container attach 1e5ae8f1bb39f23116efbb98212c474db5b9d9302a66958cca329618a72b0d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:41.285 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 07:31:41 smithi119 podman[68318]: 2024-09-21 07:31:41.221628543 +0000 UTC m=+0.568581564 container died 1e5ae8f1bb39f23116efbb98212c474db5b9d9302a66958cca329618a72b0d6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T07:31:41.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:41 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:41.084+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:41.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:41 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:41.129+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:41.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:41 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:41.129+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:41.591 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:41 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:41.100+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:41.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:41 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:41.141+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:41.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:41 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:41.141+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:41.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:41 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:41.460+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:41.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:41 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:41.642+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:41.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:41 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:41.642+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:41.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T07:31:41.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:41.908 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T07:31:41.908 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T07:31:41.908 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.5 2024-09-21T07:31:42.063 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:41 smithi153 systemd[1]: Stopping Ceph osd.5 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:42.063 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:41 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:41.775+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:42.063 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:41 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:41.775+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:42.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:42 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:42.057+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:42.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:42 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:42.140+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:42.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:42 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:42.140+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:42.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:42.061+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:42.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:42.098+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:42.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:42.098+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:42.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:42.221+0000 7f2f65287640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:42.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:42.221+0000 7f2f65287640 -1 osd.5 114 *** Got signal Terminated *** 2024-09-21T07:31:42.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:42.221+0000 7f2f65287640 -1 osd.5 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:42.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:42 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:42.481+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:42.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:42 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:42.665+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:42.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:42 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:42.665+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:43.032 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:42.730+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:43.032 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:42 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:42.730+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:43.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:43 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:43.062+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:43.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:43 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:43.100+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:43.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:43 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:43.100+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:43.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:43 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:43.030+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:43.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:43 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:43.135+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:43.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:43 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:43.135+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:43.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:43 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:43.499+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:43.784 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:43 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:43.712+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:43.784 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:43 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:43.712+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:44.029 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:43 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:43.718+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:44.029 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:43 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:43.718+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:44.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:44 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:44.081+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:44.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:44 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:44.085+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:44.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:44 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:44.085+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:44.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:44 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:44.177+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:44.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:44 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:44.177+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:44.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:44 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:44.027+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:44.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:44 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:44.524+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:44.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:44 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:44.524+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:44.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:44 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:44.732+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:44.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:44 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:44.732+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:45.029 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:44 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:44.683+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:45.029 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:44 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:44.683+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:45.029 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:44 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:44.683+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:45.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:45.060+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:45.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:45.115+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:45.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:45.115+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:45.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:45 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:45.115+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:45.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:45.131+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:45.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:45.131+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:45.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:45.131+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:18.511088+0000 front 2024-09-21T07:31:18.511176+0000 (oldest deadline 2024-09-21T07:31:44.411185+0000) 2024-09-21T07:31:45.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:45.027+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:45.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:45.027+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:45.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:45.645+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:45.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:45.645+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:45.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:45.645+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:46.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:45 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:45.566+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:46.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:45 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:45.566+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:46.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:45 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:45.737+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:46.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:45 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:45.737+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:46.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:46.081+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:46.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:46.081+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:46.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:46.067+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:46.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:46.067+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:46.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:46.067+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:46.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:46.126+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:46.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:46.126+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:46.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:46.126+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:18.511088+0000 front 2024-09-21T07:31:18.511176+0000 (oldest deadline 2024-09-21T07:31:44.411185+0000) 2024-09-21T07:31:46.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:45.991+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:46.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:45 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:45.991+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:46.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:46 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:46.530+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:46.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:46 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:46.530+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:46.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:46 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:46.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:46.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:46 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:46.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:46.785 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:46 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:46.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:46.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:46.631+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:46.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:46.631+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:46.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:46.631+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:47.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:47.067+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:47.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:47.067+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:47.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:47.112+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:47.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:47.112+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:47.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:47 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:47.112+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:47.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:47 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:47.081+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.908183+0000 front 2024-09-21T07:31:05.907848+0000 (oldest deadline 2024-09-21T07:31:27.607997+0000) 2024-09-21T07:31:47.427 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:47 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:47.081+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:13.811057+0000 front 2024-09-21T07:31:13.811093+0000 (oldest deadline 2024-09-21T07:31:38.510760+0000) 2024-09-21T07:31:47.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:47 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5[46585]: 2024-09-21T07:31:47.081+0000 7f2f5d86a640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:18.511088+0000 front 2024-09-21T07:31:18.511176+0000 (oldest deadline 2024-09-21T07:31:44.411185+0000) 2024-09-21T07:31:47.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:46.982+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:47.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:46 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:46.982+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:47.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:47 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:47.521+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:47.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:47 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:47.521+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:47.927 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:47 smithi153 podman[70280]: 2024-09-21 07:31:47.684988399 +0000 UTC m=+5.582828214 container died c88407a2a5aacdf9e7a0d37e3f4c4c49544045a1ae8fb9a3c46d7eed763eefcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T07:31:47.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:47 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:47.623+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:47.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:47 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:47.623+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:47.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:47 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:47.623+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:48.157 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:47 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:47.788+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:48.157 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:47 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:47.788+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:48.157 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:47 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:47.788+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:48.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:48 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:48.054+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:48.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:48 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:48.054+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:48.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:48 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:48.144+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:48.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:48 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:48.144+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:48.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:48 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:48.144+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:48.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:48 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:48.023+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:48.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:48 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:48.023+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:48.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:48 smithi153 podman[70280]: 2024-09-21 07:31:48.088208487 +0000 UTC m=+5.986048344 container cleanup c88407a2a5aacdf9e7a0d37e3f4c4c49544045a1ae8fb9a3c46d7eed763eefcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T07:31:48.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:48 smithi153 bash[70280]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5 2024-09-21T07:31:48.762 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:48 smithi153 podman[70295]: 2024-09-21 07:31:48.464527034 +0000 UTC m=+0.775793321 container remove c88407a2a5aacdf9e7a0d37e3f4c4c49544045a1ae8fb9a3c46d7eed763eefcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T07:31:48.762 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:48 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:48.641+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:48.763 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:48 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:48.641+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:48.763 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:48 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:48.641+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:48.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:48 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:48.517+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:48.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:48 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:48.517+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:49.029 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:48 smithi153 podman[70384]: 2024-09-21 07:31:48.663977029 +0000 UTC m=+0.015981712 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:31:49.029 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:48 smithi153 podman[70384]: 2024-09-21 07:31:48.792698379 +0000 UTC m=+0.144703054 container create 51fa2627555c6946cf029a8e713a248beca6db8c631421c0ca4fbde6a089ca5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T07:31:49.030 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:49 smithi153 podman[70384]: 2024-09-21 07:31:49.028507416 +0000 UTC m=+0.380512097 container init 51fa2627555c6946cf029a8e713a248beca6db8c631421c0ca4fbde6a089ca5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:49.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:48 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:48.834+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:49.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:48 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:48.834+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:49.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:48 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:48.834+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:49.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:49.032+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:49.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:49.032+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:49.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:49.173+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:49.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:49.173+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:49.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:49.173+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:49.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:49 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:49.042+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:49.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:49 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:49.042+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:49.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:49 smithi153 podman[70384]: 2024-09-21 07:31:49.032106937 +0000 UTC m=+0.384111612 container start 51fa2627555c6946cf029a8e713a248beca6db8c631421c0ca4fbde6a089ca5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:49.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:49 smithi153 podman[70384]: 2024-09-21 07:31:49.136852694 +0000 UTC m=+0.488857383 container attach 51fa2627555c6946cf029a8e713a248beca6db8c631421c0ca4fbde6a089ca5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:49.428 INFO:journalctl@ceph.osd.5.smithi153.stdout:Sep 21 07:31:49 smithi153 podman[70384]: 2024-09-21 07:31:49.232903661 +0000 UTC m=+0.584908334 container died 51fa2627555c6946cf029a8e713a248beca6db8c631421c0ca4fbde6a089ca5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T07:31:49.817 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:49 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:49.560+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:49.817 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:49 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:49.560+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:49.908 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:49 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:49.661+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:49.908 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:49 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:49.661+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:49.908 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:49 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:49.661+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:49.922 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T07:31:49.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:49.961 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T07:31:49.961 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T07:31:49.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.6 2024-09-21T07:31:50.177 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:50 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:50.044+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:50.177 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:50 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:50.044+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:50.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:49 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:49.816+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:50.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:49 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:49.816+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:50.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:49 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:49.816+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:50.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:50.159+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:50.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:50.159+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:50.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:50.159+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:50.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:49.995+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:50.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:49.995+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:50.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:50 smithi027 systemd[1]: Stopping Ceph osd.6 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:50.841 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:50.389+0000 7f4fbe000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:50.841 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:50.389+0000 7f4fbe000640 -1 osd.6 114 *** Got signal Terminated *** 2024-09-21T07:31:50.842 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:50.389+0000 7f4fbe000640 -1 osd.6 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:50.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:50 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:50.649+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:50.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:50 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:50.649+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:50.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:50 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:50.649+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:51.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:50 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:50.605+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:51.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:50 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:50.605+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:51.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:50 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:50.851+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:51.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:50 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:50.851+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:51.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:50 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:50.851+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:51.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:50.987+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:51.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:50 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:50.987+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:51.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:51 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:51.161+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:51.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:51 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:51.161+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:51.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:51 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:51.161+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:51.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:51 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:51.073+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:51.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:51 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:51.073+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:52.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:51 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:51.612+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:52.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:51 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:51.612+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:52.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:51 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:51.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:52.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:51 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:51.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:52.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:51 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:51.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:52.076 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:51 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:51.687+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:52.076 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:51 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:51.687+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:52.076 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:51 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:51.687+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:52.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:51 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:51.973+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:52.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:51 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:51.973+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:52.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:52 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:52.200+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:52.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:52 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:52.200+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:52.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:52 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:52.200+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:52.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:52 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:52.200+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:31:52.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:52 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:52.074+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:52.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:52 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:52.074+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:53.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:52 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:52.622+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:53.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:52 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:52.622+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:53.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:52 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:52.817+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:53.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:52 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:52.817+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:53.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:52 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:52.817+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:53.119 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:52 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:52.710+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:53.119 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:52 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:52.710+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:53.119 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:52 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:52.710+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:53.119 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:52 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:52.710+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:31:53.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:53.022+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:53.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:53.022+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:53.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:53.175+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:53.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:53.175+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:53.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:53.175+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:53.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:53.175+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:31:53.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:53 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:53.117+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:53.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:53 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:53.117+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:54.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:53.625+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:54.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:53.625+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:54.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:53.625+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:31:54.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:53.773+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:54.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:53.773+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:54.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:53 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:53.773+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:54.081 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:53 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:53.744+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:54.081 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:53 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:53.744+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:54.081 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:53 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:53.744+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:54.081 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:53 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:53.744+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:31:54.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:53.979+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:54.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:53.979+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:54.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:53 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:53.979+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.497643+0000 front 2024-09-21T07:31:28.497867+0000 (oldest deadline 2024-09-21T07:31:53.797675+0000) 2024-09-21T07:31:54.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:54 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:54.149+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:54.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:54 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:54.149+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:54.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:54 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:54.149+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:54.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:54 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:54.149+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:31:54.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:54 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:54.079+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:54.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:54 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:54.079+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:54.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:54 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:54.079+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:31:55.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:54.600+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:55.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:54.600+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:55.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:54.600+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:31:55.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:54.822+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:55.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:54.822+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:55.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:54.822+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:55.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:54 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:54.822+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:31:55.123 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:54 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:54.746+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:55.123 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:54 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:54.746+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:55.123 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:54 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:54.746+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:55.123 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:54 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:54.746+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:31:55.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:54 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:54.946+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.594142+0000 front 2024-09-21T07:31:14.594296+0000 (oldest deadline 2024-09-21T07:31:36.894393+0000) 2024-09-21T07:31:55.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:54 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:54.946+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:22.696369+0000 front 2024-09-21T07:31:22.696745+0000 (oldest deadline 2024-09-21T07:31:45.596322+0000) 2024-09-21T07:31:55.341 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:54 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6[61099]: 2024-09-21T07:31:54.946+0000 7f4fb9a00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.497643+0000 front 2024-09-21T07:31:28.497867+0000 (oldest deadline 2024-09-21T07:31:53.797675+0000) 2024-09-21T07:31:55.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:55.117+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:55.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:55.117+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:55.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:55.117+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:55.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:55 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:55.117+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:31:55.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:55.120+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:55.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:55.120+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:55.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:55.120+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:31:56.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:55.598+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:56.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:55.598+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:56.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:55.598+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:31:56.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:55.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:56.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:55.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:56.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:55.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:56.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:55.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:31:56.089 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:55 smithi027 podman[128982]: 2024-09-21 07:31:55.812806997 +0000 UTC m=+5.553574430 container died 8d4ae64416e3368394df77fd9aeae5fe5cfb4c897386f48dad9998af2020f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:56.168 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:55.755+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:56.168 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:55.755+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:56.168 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:55.755+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:56.168 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:55 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:55.755+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:31:56.340 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:56 smithi027 podman[128982]: 2024-09-21 07:31:56.320702062 +0000 UTC m=+6.061469515 container cleanup 8d4ae64416e3368394df77fd9aeae5fe5cfb4c897386f48dad9998af2020f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T07:31:56.340 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:56 smithi027 bash[128982]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6 2024-09-21T07:31:56.340 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:56 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:56.154+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:56.340 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:56 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:56.154+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:56.340 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:56 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:56.154+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:56.340 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:56 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:56.154+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:31:56.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:56 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:56.166+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:56.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:56 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:56.166+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:56.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:56 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:56.166+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:31:56.825 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:56 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:56.559+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:56.826 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:56 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:56.559+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:56.826 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:56 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:56.559+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:31:56.841 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:56 smithi027 podman[129000]: 2024-09-21 07:31:56.580246299 +0000 UTC m=+0.756453178 container remove 8d4ae64416e3368394df77fd9aeae5fe5cfb4c897386f48dad9998af2020f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T07:31:57.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:56 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:56.766+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:57.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:56 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:56.766+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:57.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:56 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:56.766+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:57.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:56 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:56.766+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:31:57.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:56 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:56.823+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:57.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:56 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:56.823+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:57.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:56 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:56.823+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:57.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:56 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:56.823+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:31:57.336 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:57 smithi027 podman[129087]: 2024-09-21 07:31:56.948403063 +0000 UTC m=+0.028586693 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:31:57.336 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:57 smithi027 podman[129087]: 2024-09-21 07:31:57.064581413 +0000 UTC m=+0.144765021 container create f308ff5b4aa39c2c39133d9604dc82af6d8616cd4da199a4f961259bee6453dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T07:31:57.336 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:57 smithi027 podman[129087]: 2024-09-21 07:31:57.334461293 +0000 UTC m=+0.414644913 container init f308ff5b4aa39c2c39133d9604dc82af6d8616cd4da199a4f961259bee6453dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:31:57.336 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:57 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:57.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:57.336 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:57 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:57.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:57.336 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:57 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:57.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:57.336 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:57 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:57.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:31:57.591 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:57 smithi027 podman[129087]: 2024-09-21 07:31:57.34266723 +0000 UTC m=+0.422850848 container start f308ff5b4aa39c2c39133d9604dc82af6d8616cd4da199a4f961259bee6453dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:57.591 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:57 smithi027 podman[129087]: 2024-09-21 07:31:57.433679753 +0000 UTC m=+0.513863361 container attach f308ff5b4aa39c2c39133d9604dc82af6d8616cd4da199a4f961259bee6453dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T07:31:57.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:57.204+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:57.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:57.204+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:57.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:57.204+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:31:57.870 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:57.569+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:57.870 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:57.569+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:57.870 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:57.569+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:31:57.937 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 07:31:57 smithi027 podman[129087]: 2024-09-21 07:31:57.610943301 +0000 UTC m=+0.691126910 container died f308ff5b4aa39c2c39133d9604dc82af6d8616cd4da199a4f961259bee6453dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:31:58.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:57.769+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:58.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:57.769+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:58.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:57.769+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:58.178 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:57 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:57.769+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:31:58.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:57.867+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:58.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:57.867+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:58.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:57.867+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:58.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:57.867+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:31:58.314 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:58 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:58.162+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:58.314 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:58 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:58.162+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:58.314 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:58 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:58.162+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:58.314 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:58 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:58.162+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:31:58.352 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T07:31:58.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:31:58.430 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T07:31:58.430 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T07:31:58.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.7 2024-09-21T07:31:58.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:58 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:58.200+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:58.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:58 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:58.200+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:58.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:58 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:58.200+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:31:58.742 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:58 smithi119 systemd[1]: Stopping Ceph osd.7 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:31:58.743 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:58.592+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:31:58.743 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:58.592+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:31:58.743 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:58.592+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:31:58.743 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:58.739+0000 7efc6e4e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:31:58.743 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:58.739+0000 7efc6e4e2640 -1 osd.7 114 *** Got signal Terminated *** 2024-09-21T07:31:58.743 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:58.739+0000 7efc6e4e2640 -1 osd.7 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:31:59.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:58.845+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:31:59.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:58.845+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:31:59.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:58.845+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:31:59.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:58 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:58.845+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:31:59.172 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:58 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:58.762+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:31:59.172 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:58 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:58.762+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:31:59.172 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:58 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:58.762+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:31:59.172 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:58 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:58.762+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:31:59.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:59.170+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:31:59.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:59.170+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:31:59.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:31:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:31:59.170+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:31:59.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:59 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:59.123+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:31:59.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:59 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:59.123+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:31:59.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:59 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:59.123+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:31:59.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:31:59 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:31:59.123+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:00.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:59 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:59.643+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:32:00.034 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:59 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:59.643+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:32:00.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:31:59 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:31:59.643+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:32:00.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:59 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:59.809+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:00.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:59 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:59.809+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:00.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:59 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:59.809+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:00.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:31:59 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:31:59.809+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:00.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:59.750+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:00.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:59.750+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:00.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:59.750+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:00.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:31:59 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:31:59.750+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:00.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:00.141+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:00.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:00.141+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:00.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:00.141+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:00.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:00.141+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:00.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:00 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:00.141+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:00.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:00 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:00.188+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:00.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:00 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:00.189+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:00.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:00 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:00.189+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:01.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:00.620+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:32:01.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:00.620+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:32:01.035 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:00.620+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:32:01.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:00.853+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:01.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:00.853+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:01.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:00.853+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:01.036 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:00.853+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:01.036 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:00 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:00.853+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:01.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:00 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:00.767+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:01.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:00 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:00.767+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:01.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:00 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:00.767+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:01.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:00 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:00.767+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:01.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:01.153+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:01.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:01.153+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:01.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:01.153+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:01.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:01.153+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:01.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:01 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:01.153+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:01.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:01.236+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:01.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:01.236+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:01.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:01.236+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:01.899 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:01.583+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:32:01.899 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:01.583+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:32:01.899 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:01.583+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:32:01.899 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:01.583+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.120553+0000 front 2024-09-21T07:31:39.120613+0000 (oldest deadline 2024-09-21T07:32:01.420642+0000) 2024-09-21T07:32:02.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:01.732+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:02.178 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:01.732+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:02.178 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:01.732+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:02.178 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:01 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:01.732+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:02.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:01.897+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:02.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:01.897+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:02.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:01.897+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:02.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:01.897+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:02.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:01.897+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:02.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:02 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:02.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:02.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:02 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:02.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:02.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:02 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:02.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:02.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:02 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:02.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:02.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:02 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:02.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:02.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:02 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:02.235+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:02.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:02 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:02.235+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:02.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:02 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:02.235+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:02.907 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:02.565+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:32:02.907 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:02.565+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:32:02.907 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:02.565+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:32:02.907 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:02.565+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.120553+0000 front 2024-09-21T07:31:39.120613+0000 (oldest deadline 2024-09-21T07:32:01.420642+0000) 2024-09-21T07:32:03.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:02 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:02.773+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:03.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:02 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:02.773+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:03.178 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:02 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:02.773+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:03.178 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:02 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:02.773+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:03.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:02.905+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:03.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:02.905+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:03.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:02.905+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:03.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:02.905+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:03.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:02 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:02.905+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:03.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:03 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:03.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:03.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:03 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:03.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:03.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:03 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:03.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:03.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:03 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:03.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:03.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:03 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:03.191+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:03.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:03.210+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:03.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:03.210+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:03.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:03.210+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:03.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:03.210+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:03.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:03.526+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:09.415921+0000 front 2024-09-21T07:31:09.415745+0000 (oldest deadline 2024-09-21T07:31:38.216704+0000) 2024-09-21T07:32:03.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:03.526+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.117831+0000 front 2024-09-21T07:31:21.117616+0000 (oldest deadline 2024-09-21T07:31:44.017597+0000) 2024-09-21T07:32:03.785 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:03.526+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.718804+0000 front 2024-09-21T07:31:28.718782+0000 (oldest deadline 2024-09-21T07:31:52.818721+0000) 2024-09-21T07:32:03.785 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7[52269]: 2024-09-21T07:32:03.526+0000 7efc6aacd640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.120553+0000 front 2024-09-21T07:31:39.120613+0000 (oldest deadline 2024-09-21T07:32:01.420642+0000) 2024-09-21T07:32:04.169 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:03.795+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:04.169 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:03.795+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:04.170 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:03.795+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:04.170 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:03 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:03.795+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:04.284 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:04 smithi119 podman[68432]: 2024-09-21 07:32:04.148586048 +0000 UTC m=+5.523379365 container died 75a442b25dad861f7a84e712520e59f886f683ba4df0690d1a312c214bcd8de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T07:32:04.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:03.907+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:04.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:03.907+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:04.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:03.907+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:04.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:03.907+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:04.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:03 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:03.907+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:04.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:04.168+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:04.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:04.168+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:04.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:04.168+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:04.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:04.168+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:04.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:04 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:04.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:04.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:04 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:04.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:04.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:04 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:04.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:04.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:04 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:04.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:04.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:04 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:04.166+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:04.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:04 smithi119 podman[68432]: 2024-09-21 07:32:04.532906272 +0000 UTC m=+5.907699587 container cleanup 75a442b25dad861f7a84e712520e59f886f683ba4df0690d1a312c214bcd8de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T07:32:04.784 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:04 smithi119 bash[68432]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7 2024-09-21T07:32:05.113 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:04 smithi119 podman[68448]: 2024-09-21 07:32:04.803366608 +0000 UTC m=+0.647702625 container remove 75a442b25dad861f7a84e712520e59f886f683ba4df0690d1a312c214bcd8de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T07:32:05.114 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:04 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:04.931+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:05.114 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:04 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:04.931+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:05.114 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:04 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:04.931+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:05.114 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:04 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:04.931+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:05.114 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:04 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:04.931+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:05.171 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:04.753+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:05.171 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:04.753+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:05.171 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:04.753+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:05.171 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:04 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:04.753+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:05.388 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:05 smithi119 podman[68538]: 2024-09-21 07:32:05.015330655 +0000 UTC m=+0.015470023 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:32:05.388 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:05 smithi119 podman[68538]: 2024-09-21 07:32:05.135122913 +0000 UTC m=+0.135262283 container create 1fd4852319e9107668427076e91947781a001946ef1ea77da05d53722f9cc47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T07:32:05.389 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:05 smithi119 podman[68538]: 2024-09-21 07:32:05.387744698 +0000 UTC m=+0.387884069 container init 1fd4852319e9107668427076e91947781a001946ef1ea77da05d53722f9cc47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T07:32:05.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:05.170+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:05.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:05.170+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:05.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:05.170+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:05.427 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:05.170+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:05.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:05 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:05.133+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:05.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:05 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:05.133+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:05.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:05 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:05.133+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:05.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:05 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:05.133+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:05.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:05 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:05.133+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:05.785 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:05 smithi119 podman[68538]: 2024-09-21 07:32:05.391171678 +0000 UTC m=+0.391311050 container start 1fd4852319e9107668427076e91947781a001946ef1ea77da05d53722f9cc47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T07:32:05.785 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:05 smithi119 podman[68538]: 2024-09-21 07:32:05.48786934 +0000 UTC m=+0.488008719 container attach 1fd4852319e9107668427076e91947781a001946ef1ea77da05d53722f9cc47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T07:32:05.785 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 07:32:05 smithi119 podman[68538]: 2024-09-21 07:32:05.581770059 +0000 UTC m=+0.581909432 container died 1fd4852319e9107668427076e91947781a001946ef1ea77da05d53722f9cc47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T07:32:06.154 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:05.715+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:06.154 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:05.715+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:06.154 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:05.715+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:06.154 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:05 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:05.715+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:06.192 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:05 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:05.908+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:06.193 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:05 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:05.908+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:06.193 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:05 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:05.908+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:06.193 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:05 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:05.908+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:06.193 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:05 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:05.908+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:06.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T07:32:06.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:06.246 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T07:32:06.246 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T07:32:06.246 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.8 2024-09-21T07:32:06.427 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 systemd[1]: Stopping Ceph osd.8 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:06.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:06.152+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:06.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:06.152+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:06.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:06.152+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:06.428 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:06.152+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:06.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:06 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:06.142+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:06.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:06 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:06.142+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:06.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:06 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:06.142+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:06.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:06 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:06.142+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:06.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:06 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:06.142+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:06.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:06.550+0000 7fa0392e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:06.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:06.550+0000 7fa0392e2640 -1 osd.8 114 *** Got signal Terminated *** 2024-09-21T07:32:06.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:06.550+0000 7fa0392e2640 -1 osd.8 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:32:06.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:06.712+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:06.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:06.712+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:06.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:06.712+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:06.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:06 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:06.712+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:07.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:06 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:06.923+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:07.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:06 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:06.923+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:07.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:06 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:06.923+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:07.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:06 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:06.923+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:07.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:06 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:06.923+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:07.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:07 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:07.171+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:07.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:07 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:07.171+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:07.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:07 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:07.171+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:07.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:07 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:07.171+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:07.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:07 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:07.171+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:07.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:07.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:07.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:07.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:07.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:07.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:07.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:07.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:08.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:07.714+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:08.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:07.714+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:08.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:07.714+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:08.177 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:07 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:07.714+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:08.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:07 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:07.881+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:08.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:07 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:07.881+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:08.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:07 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:07.881+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:08.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:07 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:07.881+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:08.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:07 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:07.881+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:08.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:08 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:08.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:08.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:08 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:08.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:08.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:08 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:08.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:08.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:08 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:08.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:08.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:08 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:08.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:08.675 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:08.193+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:08.675 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:08.193+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:08.675 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:08.193+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:08.675 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:08.193+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:08.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:08.673+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:08.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:08.673+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:08.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:08.673+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:08.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:08.673+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:08.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:08 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:08.673+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:44.852338+0000 front 2024-09-21T07:31:44.852362+0000 (oldest deadline 2024-09-21T07:32:07.752470+0000) 2024-09-21T07:32:09.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:08 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:08.865+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:09.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:08 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:08.865+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:09.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:08 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:08.865+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:09.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:08 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:08.865+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:09.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:08 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:08.865+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:09.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:08 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:08.865+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:09.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:09 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:09.239+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:09.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:09 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:09.239+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:09.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:09 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:09.239+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:09.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:09 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:09.239+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:09.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:09 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:09.239+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:09.667 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:09.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:09.667 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:09.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:09.667 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:09.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:09.667 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:09.195+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:09.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:09.665+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:09.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:09.665+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:09.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:09.665+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:09.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:09.665+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:09.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:09 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:09.665+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:44.852338+0000 front 2024-09-21T07:31:44.852362+0000 (oldest deadline 2024-09-21T07:32:07.752470+0000) 2024-09-21T07:32:10.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:09 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:09.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:10.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:09 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:09.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:10.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:09 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:09.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:10.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:09 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:09.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:10.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:09 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:09.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:10.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:09 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:09.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:10.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:10 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:10.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:10.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:10 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:10.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:10.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:10 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:10.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:10.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:10 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:10.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:10.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:10 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:10.213+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:10.672 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:10.241+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:10.673 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:10.241+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:10.673 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:10.241+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:10.673 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:10.241+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:10.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:10.671+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.346298+0000 front 2024-09-21T07:31:05.346454+0000 (oldest deadline 2024-09-21T07:31:31.246215+0000) 2024-09-21T07:32:10.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:10.671+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:11.246796+0000 front 2024-09-21T07:31:11.246942+0000 (oldest deadline 2024-09-21T07:31:35.946764+0000) 2024-09-21T07:32:10.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:10.671+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:21.048591+0000 front 2024-09-21T07:31:21.048625+0000 (oldest deadline 2024-09-21T07:31:44.548645+0000) 2024-09-21T07:32:10.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:10.671+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:28.649535+0000 front 2024-09-21T07:31:28.649505+0000 (oldest deadline 2024-09-21T07:31:52.149426+0000) 2024-09-21T07:32:10.928 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8[53190]: 2024-09-21T07:32:10.671+0000 7fa0318c5640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:44.852338+0000 front 2024-09-21T07:31:44.852362+0000 (oldest deadline 2024-09-21T07:32:07.752470+0000) 2024-09-21T07:32:11.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:10.801+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:11.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:10.801+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:11.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:10.801+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:11.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:10.801+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:11.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:10.801+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:11.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:10 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:10.801+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:11.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:11.217+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:11.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:11.217+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:11.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:11.217+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:11.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:11.217+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:11.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:11.217+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:11.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:11 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:11.217+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:11.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:11 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:11.223+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:11.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:11 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:11.223+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:11.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:11 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:11.223+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:11.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:11 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:11.223+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:12.227 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:11 smithi153 podman[70488]: 2024-09-21 07:32:11.955220959 +0000 UTC m=+5.515454598 container died 61710cf454f4de227866f0ea9e616b506a0486dae32ac39254622a5ee191df2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:32:12.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:11 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:11.805+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:12.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:11 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:11.805+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:12.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:11 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:11.805+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:12.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:11 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:11.805+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:12.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:11 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:11.805+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:12.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:11 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:11.805+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:12.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:12 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:12.258+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:12.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:12 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:12.258+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:12.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:12 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:12.258+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:12.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:12 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:12.258+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:12.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:12 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:12.258+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:12.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:12 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:12.258+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:12.603 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:12 smithi153 podman[70488]: 2024-09-21 07:32:12.325798102 +0000 UTC m=+5.886031776 container cleanup 61710cf454f4de227866f0ea9e616b506a0486dae32ac39254622a5ee191df2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T07:32:12.603 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:12 smithi153 bash[70488]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8 2024-09-21T07:32:12.603 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:12 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:12.249+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:12.603 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:12 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:12.249+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:12.603 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:12 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:12.249+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:12.603 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:12 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:12.249+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:12.603 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:12 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:12.249+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:12.927 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:12 smithi153 podman[70502]: 2024-09-21 07:32:12.602006531 +0000 UTC m=+0.644265139 container remove 61710cf454f4de227866f0ea9e616b506a0486dae32ac39254622a5ee191df2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T07:32:13.222 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:12 smithi153 podman[70591]: 2024-09-21 07:32:12.842199303 +0000 UTC m=+0.016241313 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:32:13.223 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:12 smithi153 podman[70591]: 2024-09-21 07:32:12.961218125 +0000 UTC m=+0.135260129 container create 514598945a3250cb637a9d75f566b740889bfb5fc87578761c07406de9e9f3d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:32:13.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:12 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:12.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:13.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:12 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:12.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:13.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:12 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:12.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:13.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:12 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:12.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:13.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:12 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:12.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:13.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:12 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:12.815+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:13.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:13 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:13.301+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:13.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:13 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:13.301+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:13.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:13 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:13.301+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:13.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:13 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:13.301+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:13.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:13 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:13.301+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:13.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:13 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:13.301+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:13.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:13 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:13.255+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:13 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:13.255+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:13 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:13.255+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:13 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:13.255+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:13 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:13.255+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:13 smithi153 podman[70591]: 2024-09-21 07:32:13.222016428 +0000 UTC m=+0.396058433 container init 514598945a3250cb637a9d75f566b740889bfb5fc87578761c07406de9e9f3d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:13 smithi153 podman[70591]: 2024-09-21 07:32:13.225640474 +0000 UTC m=+0.399682482 container start 514598945a3250cb637a9d75f566b740889bfb5fc87578761c07406de9e9f3d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:13 smithi153 podman[70591]: 2024-09-21 07:32:13.305610077 +0000 UTC m=+0.479652094 container attach 514598945a3250cb637a9d75f566b740889bfb5fc87578761c07406de9e9f3d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T07:32:13.678 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:13 smithi153 podman[70591]: 2024-09-21 07:32:13.416239997 +0000 UTC m=+0.590282005 container died 514598945a3250cb637a9d75f566b740889bfb5fc87578761c07406de9e9f3d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T07:32:14.037 INFO:journalctl@ceph.osd.8.smithi153.stdout:Sep 21 07:32:14 smithi153 podman[70615]: 2024-09-21 07:32:14.017497969 +0000 UTC m=+0.595498067 container remove 514598945a3250cb637a9d75f566b740889bfb5fc87578761c07406de9e9f3d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T07:32:14.042 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T07:32:14.080 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:14.081 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T07:32:14.081 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T07:32:14.081 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.9 2024-09-21T07:32:14.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:13.794+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:14.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:13.794+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:14.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:13.794+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:14.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:13.794+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:14.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:13.794+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:14.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:13 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:13.794+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:14.291 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:14 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:14.269+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:14.291 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:14 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:14.269+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:14.291 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:14 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:14.269+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:14.291 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:14 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:14.269+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:14.291 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:14 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:14.269+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:14.495 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 systemd[1]: Stopping Ceph osd.9 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:14.495 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.312+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.312+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.312+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.312+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.312+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.312+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.493+0000 7f5f85000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.493+0000 7f5f85000640 -1 osd.9 114 *** Got signal Terminated *** 2024-09-21T07:32:14.496 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:14 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:14.493+0000 7f5f85000640 -1 osd.9 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:32:15.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:14.813+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:15.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:14.813+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:15.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:14.813+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:15.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:14.813+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:15.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:14.813+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:15.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:14 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:14.813+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:15.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:15 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:15.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:15.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:15 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:15.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:15.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:15 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:15.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:15.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:15 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:15.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:15.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:15 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:15.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:15.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:15 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:15.347+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:15.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:15 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:15.347+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:15.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:15 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:15.347+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:15.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:15 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:15.347+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:15.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:15 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:15.347+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:15.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:15 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:15.347+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:16.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:15 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:15.842+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:16.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:15 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:15.842+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:16.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:15 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:15.842+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:16.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:15 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:15.842+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:16.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:15 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:15.842+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:16.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:15 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:15.842+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:16.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:16 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:16.306+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:16.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:16 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:16.306+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:16.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:16 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:16.306+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:16.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:16 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:16.306+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:16.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:16 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:16.306+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:16.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:16 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:16.306+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:16.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:16 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:16.331+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:16.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:16 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:16.331+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:16.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:16 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:16.331+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:16.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:16 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:16.331+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:16.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:16 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:16.331+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:17.186 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:16 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:16.858+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:17.186 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:16 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:16.858+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:17.186 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:16 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:16.858+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:17.186 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:16 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:16.858+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:17.186 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:16 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:16.858+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:17.186 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:16 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:16.858+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:17.186 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:16 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:16.858+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:17.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:17.295+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:17.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:17.295+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:17.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:17.295+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:17.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:17.295+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:17.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:17.295+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:17.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:17 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:17.295+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:17.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:17 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:17.333+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:17.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:17 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:17.333+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:17.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:17 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:17.333+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:17.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:17 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:17.333+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:17.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:17 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:17.333+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:18.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:17 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:17.864+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:18.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:17 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:17.864+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:18.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:17 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:17.864+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:18.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:17 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:17.864+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:18.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:17 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:17.864+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:18.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:17 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:17.864+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:18.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:17 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:17.864+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:18.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:18 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:18.279+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:18.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:18 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:18.279+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:18.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:18 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:18.279+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:18.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:18 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:18.279+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:18.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:18 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:18.279+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:18.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:18 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:18.279+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:18.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:18.332+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:18.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:18.332+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:18.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:18.332+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:18.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:18.332+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:18.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:18.332+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:19.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:18 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:18.870+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:19.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:18 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:18.870+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:19.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:18 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:18.870+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:19.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:18 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:18.870+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:19.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:18 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:18.870+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:19.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:18 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:18.870+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:19.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:18 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:18.870+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:19.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:19 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:19.235+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:05.698125+0000 front 2024-09-21T07:31:05.698265+0000 (oldest deadline 2024-09-21T07:31:27.998149+0000) 2024-09-21T07:32:19.591 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:19 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:19.235+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.400207+0000 front 2024-09-21T07:31:14.400179+0000 (oldest deadline 2024-09-21T07:31:39.100107+0000) 2024-09-21T07:32:19.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:19 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:19.235+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:23.003081+0000 front 2024-09-21T07:31:23.002983+0000 (oldest deadline 2024-09-21T07:31:44.702991+0000) 2024-09-21T07:32:19.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:19 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:19.235+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:25.803987+0000 front 2024-09-21T07:31:25.803901+0000 (oldest deadline 2024-09-21T07:31:51.703855+0000) 2024-09-21T07:32:19.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:19 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:19.235+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.305882+0000 front 2024-09-21T07:31:39.305582+0000 (oldest deadline 2024-09-21T07:31:59.805290+0000) 2024-09-21T07:32:19.592 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:19 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9[67506]: 2024-09-21T07:32:19.235+0000 7f5f80a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:45.006762+0000 front 2024-09-21T07:31:45.006895+0000 (oldest deadline 2024-09-21T07:32:10.306597+0000) 2024-09-21T07:32:19.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:19.320+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:19.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:19.320+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:19.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:19.320+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:19.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:19.320+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:19.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:19.320+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:19.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:19 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:19.320+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:20.159 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:19 smithi027 podman[129189]: 2024-09-21 07:32:19.868595192 +0000 UTC m=+5.487556560 container died 517a63f36d0b0c6ce6361b0b7bbfeed757e2bce9cca245e9503bf3a9ce970749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T07:32:20.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:19 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:19.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:20.286 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:19 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:19.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:20.286 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:19 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:19.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:20.286 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:19 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:19.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:20.286 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:19 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:19.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:20.286 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:19 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:19.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:20.286 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:19 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:19.857+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:20.528 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:20 smithi027 podman[129189]: 2024-09-21 07:32:20.339212587 +0000 UTC m=+5.958173945 container cleanup 517a63f36d0b0c6ce6361b0b7bbfeed757e2bce9cca245e9503bf3a9ce970749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T07:32:20.528 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:20 smithi027 bash[129189]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9 2024-09-21T07:32:20.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:20 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:20.277+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:20.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:20 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:20.277+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:20.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:20 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:20.277+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:20.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:20 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:20.277+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:20.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:20 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:20.277+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:20.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:20 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:20.277+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:20.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:20 smithi027 podman[129209]: 2024-09-21 07:32:20.523821215 +0000 UTC m=+0.643238060 container remove 517a63f36d0b0c6ce6361b0b7bbfeed757e2bce9cca245e9503bf3a9ce970749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:32:21.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:20 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:20.906+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:21.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:20 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:20.906+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:21.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:20 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:20.906+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:21.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:20 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:20.906+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:21.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:20 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:20.906+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:21.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:20 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:20.906+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:21.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:20 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:20.906+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:21.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:21 smithi027 podman[129295]: 2024-09-21 07:32:20.98670996 +0000 UTC m=+0.027815080 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:32:21.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:21 smithi027 podman[129295]: 2024-09-21 07:32:21.099570892 +0000 UTC m=+0.140676009 container create ef79317a7c715ce7f8a8ede31882e214825946ebd4d6bef54d5a26d8a40919e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:32:21.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:21 smithi027 podman[129295]: 2024-09-21 07:32:21.310436908 +0000 UTC m=+0.351542027 container init ef79317a7c715ce7f8a8ede31882e214825946ebd4d6bef54d5a26d8a40919e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T07:32:21.342 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:21 smithi027 podman[129295]: 2024-09-21 07:32:21.317521214 +0000 UTC m=+0.358626333 container start ef79317a7c715ce7f8a8ede31882e214825946ebd4d6bef54d5a26d8a40919e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T07:32:21.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:21.252+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:21.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:21.252+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:21.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:21.252+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:21.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:21.252+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:21.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:21.252+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:21.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:21.252+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:21.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:21 smithi027 podman[129295]: 2024-09-21 07:32:21.410445826 +0000 UTC m=+0.451550963 container attach ef79317a7c715ce7f8a8ede31882e214825946ebd4d6bef54d5a26d8a40919e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:32:21.841 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 07:32:21 smithi027 podman[129295]: 2024-09-21 07:32:21.57975473 +0000 UTC m=+0.620859849 container died ef79317a7c715ce7f8a8ede31882e214825946ebd4d6bef54d5a26d8a40919e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:32:22.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T07:32:22.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:21 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:21.876+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:22.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:21 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:21.876+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:22.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:21 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:21.876+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:22.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:21 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:21.876+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:22.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:21 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:21.876+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:22.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:21 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:21.876+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:22.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:21 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:21.876+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:22.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:22.309 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T07:32:22.309 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T07:32:22.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.10 2024-09-21T07:32:22.620 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 systemd[1]: Stopping Ceph osd.10 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:22.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:22 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:22.233+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:22.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:22 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:22.233+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:22.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:22 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:22.233+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:22.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:22 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:22.233+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:22.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:22 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:22.233+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:22.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:22 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:22.233+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:23.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.618+0000 7fdb4a211640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:23.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.618+0000 7fdb4a211640 -1 osd.10 114 *** Got signal Terminated *** 2024-09-21T07:32:23.034 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.618+0000 7fdb4a211640 -1 osd.10 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:32:23.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:23.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:23.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:23.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:23.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:23.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:23.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:22 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:22.836+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:23.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:23 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:23.247+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:23.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:23 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:23.247+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:23.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:23 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:23.247+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:23.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:23 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:23.247+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:23.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:23 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:23.247+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:23.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:23 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:23.247+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:24.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:23.840+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:24.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:23.840+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:24.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:23.840+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:24.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:23.840+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:24.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:23.840+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:24.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:23.840+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:24.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:23 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:23.840+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:24.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:24 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:24.267+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:24.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:24 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:24.267+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:24.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:24 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:24.267+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:24.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:24 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:24.267+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:24.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:24 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:24.267+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:24.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:24 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:24.267+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:25.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:24 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:24.806+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:25.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:24 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:24.806+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:25.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:24 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:24.806+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:25.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:24 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:24.806+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:25.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:24 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:24.806+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:25.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:24 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:24.806+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:25.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:24 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:24.806+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:25.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:25 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:25.248+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:25.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:25 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:25.248+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:25.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:25 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:25.248+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:25.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:25 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:25.248+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:25.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:25 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:25.248+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:25.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:25 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:25.248+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:26.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:25 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:25.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:26.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:25 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:25.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:26.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:25 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:25.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:26.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:25 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:25.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:26.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:25 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:25.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:26.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:25 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:25.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:26.035 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:25 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:25.779+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:26.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:26 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:26.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:26.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:26 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:26.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:26.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:26 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:26.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:26.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:26 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:26.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:26.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:26 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:26.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:26.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:26 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:26.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:27.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:26 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:26.796+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.550919+0000 front 2024-09-21T07:31:06.550691+0000 (oldest deadline 2024-09-21T07:31:31.850635+0000) 2024-09-21T07:32:27.284 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:26 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:26.796+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T07:31:14.751467+0000 front 2024-09-21T07:31:14.751562+0000 (oldest deadline 2024-09-21T07:31:40.651598+0000) 2024-09-21T07:32:27.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:26 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:26.796+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:20.652253+0000 front 2024-09-21T07:31:20.652105+0000 (oldest deadline 2024-09-21T07:31:46.552149+0000) 2024-09-21T07:32:27.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:26 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:26.796+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:26.552960+0000 front 2024-09-21T07:31:26.552920+0000 (oldest deadline 2024-09-21T07:31:54.153119+0000) 2024-09-21T07:32:27.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:26 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:26.796+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:39.254954+0000 front 2024-09-21T07:31:39.254984+0000 (oldest deadline 2024-09-21T07:32:00.355073+0000) 2024-09-21T07:32:27.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:26 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:26.796+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:43.756911+0000 front 2024-09-21T07:31:43.756973+0000 (oldest deadline 2024-09-21T07:32:08.456601+0000) 2024-09-21T07:32:27.285 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:26 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10[58489]: 2024-09-21T07:32:26.796+0000 7fdb427f4640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:54.858041+0000 front 2024-09-21T07:31:54.858057+0000 (oldest deadline 2024-09-21T07:32:15.957742+0000) 2024-09-21T07:32:27.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:27 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:27.266+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:27.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:27 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:27.266+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:27.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:27 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:27.266+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:27.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:27 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:27.266+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:27.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:27 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:27.266+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:27.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:27 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:27.266+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:28.370 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:28 smithi119 podman[68654]: 2024-09-21 07:32:28.097895792 +0000 UTC m=+5.594491241 container died 5e6172f192afe85d35d3a3e03f571b44cbfcde401249d14d80c5026c3845cc0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T07:32:28.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:28.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:28.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:28.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:28.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:28.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:28.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:28.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:28.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:28.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:28.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:28.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:28.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:28 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:28.245+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T07:31:59.281480+0000 front 2024-09-21T07:31:59.281350+0000 (oldest deadline 2024-09-21T07:32:28.081663+0000) 2024-09-21T07:32:28.784 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:28 smithi119 podman[68654]: 2024-09-21 07:32:28.443144431 +0000 UTC m=+5.939739862 container cleanup 5e6172f192afe85d35d3a3e03f571b44cbfcde401249d14d80c5026c3845cc0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T07:32:28.784 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:28 smithi119 bash[68654]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10 2024-09-21T07:32:28.784 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:28 smithi119 podman[68669]: 2024-09-21 07:32:28.604931471 +0000 UTC m=+0.501451137 container remove 5e6172f192afe85d35d3a3e03f571b44cbfcde401249d14d80c5026c3845cc0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:32:29.258 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 podman[68761]: 2024-09-21 07:32:28.908888498 +0000 UTC m=+0.015754787 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:32:29.258 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 podman[68761]: 2024-09-21 07:32:29.02209418 +0000 UTC m=+0.128960453 container create e336e549719f1abf634e698abada0629e5733fb1aedd849302070171b88ad4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True) 2024-09-21T07:32:29.258 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 podman[68761]: 2024-09-21 07:32:29.257523956 +0000 UTC m=+0.364390230 container init e336e549719f1abf634e698abada0629e5733fb1aedd849302070171b88ad4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:32:29.534 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 podman[68761]: 2024-09-21 07:32:29.260769087 +0000 UTC m=+0.367635362 container start e336e549719f1abf634e698abada0629e5733fb1aedd849302070171b88ad4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T07:32:29.534 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 podman[68761]: 2024-09-21 07:32:29.340892717 +0000 UTC m=+0.447759000 container attach e336e549719f1abf634e698abada0629e5733fb1aedd849302070171b88ad4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T07:32:29.535 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 podman[68761]: 2024-09-21 07:32:29.446750621 +0000 UTC m=+0.553616907 container died e336e549719f1abf634e698abada0629e5733fb1aedd849302070171b88ad4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:32:29.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:29.283+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:29.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:29.283+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:29.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:29.283+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:29.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:29.283+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:29.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:29.283+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:29.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:29.283+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:29.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:29 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:29.283+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T07:31:59.281480+0000 front 2024-09-21T07:31:59.281350+0000 (oldest deadline 2024-09-21T07:32:28.081663+0000) 2024-09-21T07:32:29.964 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T07:32:29.987 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 podman[68783]: 2024-09-21 07:32:29.943801493 +0000 UTC m=+0.491713111 container remove e336e549719f1abf634e698abada0629e5733fb1aedd849302070171b88ad4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:32:29.987 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 systemd[1]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.10.service: Deactivated successfully. 2024-09-21T07:32:29.987 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 systemd[1]: Stopped Ceph osd.10 for 0ff1e276-77e7-11ef-baf5-efdc52797490. 2024-09-21T07:32:29.987 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 07:32:29 smithi119 systemd[1]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.10.service: Consumed 19.469s CPU time. 2024-09-21T07:32:30.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:30.002 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T07:32:30.002 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T07:32:30.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.11 2024-09-21T07:32:30.321 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 systemd[1]: Stopping Ceph osd.11 for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:30.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.318+0000 7f058ae75640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:30.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.318+0000 7f058ae75640 -1 osd.11 114 *** Got signal Terminated *** 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.318+0000 7f058ae75640 -1 osd.11 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.324+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.324+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.324+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.324+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.324+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.324+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:30.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:30 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:30.324+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T07:31:59.281480+0000 front 2024-09-21T07:31:59.281350+0000 (oldest deadline 2024-09-21T07:32:28.081663+0000) 2024-09-21T07:32:31.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:31.302+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:31.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:31.302+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:31.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:31.302+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:31.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:31.302+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:31.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:31.302+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:31.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:31.302+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:31.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:31 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:31.302+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T07:31:59.281480+0000 front 2024-09-21T07:31:59.281350+0000 (oldest deadline 2024-09-21T07:32:28.081663+0000) 2024-09-21T07:32:32.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:32.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:32.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:32.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:32.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:32.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:32.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:32.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:32.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:32.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:32.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:32.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:32.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:32 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:32.318+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T07:31:59.281480+0000 front 2024-09-21T07:31:59.281350+0000 (oldest deadline 2024-09-21T07:32:28.081663+0000) 2024-09-21T07:32:33.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:33.336+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:33.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:33.336+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:33.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:33.336+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:33.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:33.336+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:33.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:33.336+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:33.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:33.336+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:33.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:33 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:33.336+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T07:31:59.281480+0000 front 2024-09-21T07:31:59.281350+0000 (oldest deadline 2024-09-21T07:32:28.081663+0000) 2024-09-21T07:32:34.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:34.378+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T07:31:06.371940+0000 front 2024-09-21T07:31:06.371915+0000 (oldest deadline 2024-09-21T07:31:32.172491+0000) 2024-09-21T07:32:34.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:34.378+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6804 osd.2 since back 2024-09-21T07:31:19.976867+0000 front 2024-09-21T07:31:19.976674+0000 (oldest deadline 2024-09-21T07:31:44.676788+0000) 2024-09-21T07:32:34.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:34.378+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T07:31:29.977936+0000 front 2024-09-21T07:31:29.977910+0000 (oldest deadline 2024-09-21T07:31:53.477898+0000) 2024-09-21T07:32:34.677 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:34.378+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T07:31:37.578898+0000 front 2024-09-21T07:31:37.578978+0000 (oldest deadline 2024-09-21T07:32:02.278914+0000) 2024-09-21T07:32:34.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:34.378+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.153:6812 osd.5 since back 2024-09-21T07:31:42.279245+0000 front 2024-09-21T07:31:42.279220+0000 (oldest deadline 2024-09-21T07:32:11.679867+0000) 2024-09-21T07:32:34.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:34.378+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T07:31:55.180989+0000 front 2024-09-21T07:31:55.181037+0000 (oldest deadline 2024-09-21T07:32:19.280758+0000) 2024-09-21T07:32:34.678 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:34 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11[59826]: 2024-09-21T07:32:34.378+0000 7f0587460640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T07:31:59.281480+0000 front 2024-09-21T07:31:59.281350+0000 (oldest deadline 2024-09-21T07:32:28.081663+0000) 2024-09-21T07:32:35.904 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:35 smithi153 podman[70705]: 2024-09-21 07:32:35.631084301 +0000 UTC m=+5.434884458 container died 7b389e6c17dd8f3295632a38ff3cc3922f135820942abecbb3285e44f0b5cb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T07:32:36.178 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70705]: 2024-09-21 07:32:36.027236561 +0000 UTC m=+5.831036704 container cleanup 7b389e6c17dd8f3295632a38ff3cc3922f135820942abecbb3285e44f0b5cb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:32:36.178 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 bash[70705]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11 2024-09-21T07:32:36.596 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70721]: 2024-09-21 07:32:36.278499316 +0000 UTC m=+0.641030725 container remove 7b389e6c17dd8f3295632a38ff3cc3922f135820942abecbb3285e44f0b5cb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T07:32:36.882 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70808]: 2024-09-21 07:32:36.595944861 +0000 UTC m=+0.108448799 container create 42dc412251a9a875784a1ab32cc9a48b41e79f232b437fb24c3f4351e04e8bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T07:32:36.882 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70808]: 2024-09-21 07:32:36.503707173 +0000 UTC m=+0.016211125 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:32:36.882 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70808]: 2024-09-21 07:32:36.798098863 +0000 UTC m=+0.310602803 container init 42dc412251a9a875784a1ab32cc9a48b41e79f232b437fb24c3f4351e04e8bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T07:32:36.882 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70808]: 2024-09-21 07:32:36.80162225 +0000 UTC m=+0.314126194 container start 42dc412251a9a875784a1ab32cc9a48b41e79f232b437fb24c3f4351e04e8bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T07:32:36.882 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70808]: 2024-09-21 07:32:36.881309413 +0000 UTC m=+0.393813356 container attach 42dc412251a9a875784a1ab32cc9a48b41e79f232b437fb24c3f4351e04e8bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T07:32:37.177 INFO:journalctl@ceph.osd.11.smithi153.stdout:Sep 21 07:32:36 smithi153 podman[70808]: 2024-09-21 07:32:36.991234959 +0000 UTC m=+0.503738922 container died 42dc412251a9a875784a1ab32cc9a48b41e79f232b437fb24c3f4351e04e8bdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T07:32:37.544 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T07:32:37.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:37.581 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T07:32:37.582 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T07:32:37.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.a 2024-09-21T07:32:38.003 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:32:37 smithi027 systemd[1]: Stopping Ceph mds.a for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:38.341 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:32:38 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-a[73730]: 2024-09-21T07:32:38.001+0000 7f6546800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:38.341 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:32:38 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-a[73730]: 2024-09-21T07:32:38.001+0000 7f6546800640 -1 mds.a *** got signal Terminated *** 2024-09-21T07:32:41.091 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:32:40 smithi027 podman[129397]: 2024-09-21 07:32:40.836684618 +0000 UTC m=+2.951012424 container died 51d98d36c1fddddf095483408ed1970bf01c56f6779611c69c208f24fafa23e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T07:32:41.841 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:32:41 smithi027 podman[129397]: 2024-09-21 07:32:41.447948279 +0000 UTC m=+3.562276084 container cleanup 51d98d36c1fddddf095483408ed1970bf01c56f6779611c69c208f24fafa23e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-a, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T07:32:41.841 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:32:41 smithi027 bash[129397]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-a 2024-09-21T07:32:41.841 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 07:32:41 smithi027 podman[129408]: 2024-09-21 07:32:41.683119115 +0000 UTC m=+0.836007285 container remove 51d98d36c1fddddf095483408ed1970bf01c56f6779611c69c208f24fafa23e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:32:41.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T07:32:42.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:42.032 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T07:32:42.032 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T07:32:42.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.d 2024-09-21T07:32:42.341 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:32:42 smithi027 systemd[1]: Stopping Ceph mds.d for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:42.841 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:32:42 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d[72706]: 2024-09-21T07:32:42.413+0000 7fe052400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:42.841 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:32:42 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d[72706]: 2024-09-21T07:32:42.413+0000 7fe052400640 -1 mds.d *** got signal Terminated *** 2024-09-21T07:32:45.639 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:32:45 smithi027 podman[129518]: 2024-09-21 07:32:45.373254546 +0000 UTC m=+3.052964075 container died ac72738df547f0d54a7716374ca0a12aab6ca44f2470c9ded6fcc17f1a49d83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:32:46.030 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:32:45 smithi027 podman[129518]: 2024-09-21 07:32:45.818548623 +0000 UTC m=+3.498258150 container cleanup ac72738df547f0d54a7716374ca0a12aab6ca44f2470c9ded6fcc17f1a49d83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T07:32:46.030 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:32:45 smithi027 bash[129518]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d 2024-09-21T07:32:46.295 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 07:32:46 smithi027 podman[129530]: 2024-09-21 07:32:46.028215714 +0000 UTC m=+0.644320758 container remove ac72738df547f0d54a7716374ca0a12aab6ca44f2470c9ded6fcc17f1a49d83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:32:46.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T07:32:46.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:46.396 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T07:32:46.396 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T07:32:46.396 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.g 2024-09-21T07:32:46.591 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:32:46 smithi027 systemd[1]: Stopping Ceph mds.g for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:47.091 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:32:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g[73427]: 2024-09-21T07:32:46.817+0000 7f3ee1000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:47.091 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:32:46 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g[73427]: 2024-09-21T07:32:46.817+0000 7f3ee1000640 -1 mds.g *** got signal Terminated *** 2024-09-21T07:32:48.783 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:32:48 smithi027 podman[129636]: 2024-09-21 07:32:48.412640706 +0000 UTC m=+1.738343830 container died a58697b2e4a96a3e42a43aac837ae5a6f604412dfe1136495994dc1736b79d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:32:49.091 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:32:48 smithi027 podman[129636]: 2024-09-21 07:32:48.963831433 +0000 UTC m=+2.289534554 container cleanup a58697b2e4a96a3e42a43aac837ae5a6f604412dfe1136495994dc1736b79d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T07:32:49.091 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:32:48 smithi027 bash[129636]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g 2024-09-21T07:32:49.362 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 07:32:49 smithi027 podman[129647]: 2024-09-21 07:32:49.110442208 +0000 UTC m=+0.686880904 container remove a58697b2e4a96a3e42a43aac837ae5a6f604412dfe1136495994dc1736b79d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:32:49.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T07:32:49.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:49.449 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T07:32:49.449 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T07:32:49.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.j 2024-09-21T07:32:49.823 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:32:49 smithi027 systemd[1]: Stopping Ceph mds.j for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:50.091 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:32:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j[72329]: 2024-09-21T07:32:49.820+0000 7fb931000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:50.091 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:32:49 smithi027 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j[72329]: 2024-09-21T07:32:49.820+0000 7fb931000640 -1 mds.j *** got signal Terminated *** 2024-09-21T07:32:53.895 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:32:53 smithi027 podman[129755]: 2024-09-21 07:32:53.618741362 +0000 UTC m=+3.887490827 container died 71965bfaefa825ebf7ac6be4c29a4472403ed77d9a8b86a69e55880215a643f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T07:32:54.295 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:32:54 smithi027 podman[129755]: 2024-09-21 07:32:54.07616027 +0000 UTC m=+4.344909731 container cleanup 71965bfaefa825ebf7ac6be4c29a4472403ed77d9a8b86a69e55880215a643f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T07:32:54.295 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:32:54 smithi027 bash[129755]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j 2024-09-21T07:32:54.570 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 07:32:54 smithi027 podman[129781]: 2024-09-21 07:32:54.293864801 +0000 UTC m=+0.659195916 container remove 71965bfaefa825ebf7ac6be4c29a4472403ed77d9a8b86a69e55880215a643f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T07:32:54.593 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T07:32:54.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:54.666 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T07:32:54.666 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T07:32:54.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.b 2024-09-21T07:32:55.028 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:32:54 smithi119 systemd[1]: Stopping Ceph mds.b for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:55.284 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:32:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: 2024-09-21T07:32:55.026+0000 7fd16c342640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:55.284 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:32:55 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b[63804]: 2024-09-21T07:32:55.026+0000 7fd16c342640 -1 mds.b *** got signal Terminated *** 2024-09-21T07:32:56.595 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:32:56 smithi119 podman[68864]: 2024-09-21 07:32:56.316518109 +0000 UTC m=+1.461767090 container died 88eb88a42e9ac48cef5f9a930f391ff4ac159917daba5afdf704332f162d7398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T07:32:56.946 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:32:56 smithi119 podman[68864]: 2024-09-21 07:32:56.677011875 +0000 UTC m=+1.822260838 container cleanup 88eb88a42e9ac48cef5f9a930f391ff4ac159917daba5afdf704332f162d7398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T07:32:56.946 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 07:32:56 smithi119 bash[68864]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-b 2024-09-21T07:32:57.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T07:32:57.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:32:57.122 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T07:32:57.122 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T07:32:57.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.e 2024-09-21T07:32:57.458 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:32:57 smithi119 systemd[1]: Stopping Ceph mds.e for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:32:57.784 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:32:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e[63249]: 2024-09-21T07:32:57.456+0000 7f265fd0c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:32:57.784 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:32:57 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e[63249]: 2024-09-21T07:32:57.456+0000 7f265fd0c640 -1 mds.e *** got signal Terminated *** 2024-09-21T07:33:00.598 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:33:00 smithi119 podman[68986]: 2024-09-21 07:33:00.324780021 +0000 UTC m=+2.988656388 container died 0e099bd5a490a6be184f7d99fdce7b746aace8bfc3bda32c15fee84f193c110f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T07:33:01.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:33:00 smithi119 podman[68986]: 2024-09-21 07:33:00.672094499 +0000 UTC m=+3.335970864 container cleanup 0e099bd5a490a6be184f7d99fdce7b746aace8bfc3bda32c15fee84f193c110f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T07:33:01.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:33:00 smithi119 bash[68986]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e 2024-09-21T07:33:01.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 07:33:00 smithi119 podman[69001]: 2024-09-21 07:33:00.83148175 +0000 UTC m=+0.497917714 container remove 0e099bd5a490a6be184f7d99fdce7b746aace8bfc3bda32c15fee84f193c110f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T07:33:01.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T07:33:01.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:33:01.073 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T07:33:01.073 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T07:33:01.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.h 2024-09-21T07:33:01.401 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:33:01 smithi119 systemd[1]: Stopping Ceph mds.h for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:33:01.784 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:33:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h[63518]: 2024-09-21T07:33:01.400+0000 7f7cc9b8f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:33:01.784 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:33:01 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h[63518]: 2024-09-21T07:33:01.400+0000 7f7cc9b8f640 -1 mds.h *** got signal Terminated *** 2024-09-21T07:33:04.610 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:33:04 smithi119 podman[69107]: 2024-09-21 07:33:04.322454111 +0000 UTC m=+3.036115767 container died 36c9de9089c1f8453aa839303f9afd59dba028cdf5f14c9e53cdf3e83df04e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T07:33:04.944 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:33:04 smithi119 podman[69107]: 2024-09-21 07:33:04.683630688 +0000 UTC m=+3.397292346 container cleanup 36c9de9089c1f8453aa839303f9afd59dba028cdf5f14c9e53cdf3e83df04e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T07:33:04.944 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 07:33:04 smithi119 bash[69107]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-h 2024-09-21T07:33:05.105 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T07:33:05.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:33:05.137 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T07:33:05.137 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T07:33:05.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.k 2024-09-21T07:33:05.480 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:33:05 smithi119 systemd[1]: Stopping Ceph mds.k for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:33:05.784 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:33:05 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k[62968]: 2024-09-21T07:33:05.479+0000 7fd9e04b6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:33:05.784 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:33:05 smithi119 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k[62968]: 2024-09-21T07:33:05.479+0000 7fd9e04b6640 -1 mds.k *** got signal Terminated *** 2024-09-21T07:33:09.780 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:33:09 smithi119 podman[69229]: 2024-09-21 07:33:09.486473755 +0000 UTC m=+4.135660151 container died ad65fc685cca749149609856a282fe8232868f99bed884c62dd61c7964e52f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T07:33:10.034 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:33:09 smithi119 podman[69229]: 2024-09-21 07:33:09.862217351 +0000 UTC m=+4.511403702 container cleanup ad65fc685cca749149609856a282fe8232868f99bed884c62dd61c7964e52f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T07:33:10.034 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:33:09 smithi119 bash[69229]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k 2024-09-21T07:33:10.034 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 07:33:10 smithi119 podman[69244]: 2024-09-21 07:33:10.013087289 +0000 UTC m=+0.516390084 container remove ad65fc685cca749149609856a282fe8232868f99bed884c62dd61c7964e52f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T07:33:10.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T07:33:10.267 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:33:10.267 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T07:33:10.267 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T07:33:10.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.c 2024-09-21T07:33:10.560 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:33:10 smithi153 systemd[1]: Stopping Ceph mds.c for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:33:10.927 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:33:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c[64942]: 2024-09-21T07:33:10.558+0000 7fef6604c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:33:10.927 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:33:10 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c[64942]: 2024-09-21T07:33:10.558+0000 7fef6604c640 -1 mds.c *** got signal Terminated *** 2024-09-21T07:33:12.594 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:33:12 smithi153 podman[70909]: 2024-09-21 07:33:12.311442234 +0000 UTC m=+1.851012463 container died bc8dcfc1d30f1fd7bd9c54985150d98926353f015b642f7c92e495d7fee8644f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T07:33:12.927 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:33:12 smithi153 podman[70909]: 2024-09-21 07:33:12.666577183 +0000 UTC m=+2.206147399 container cleanup bc8dcfc1d30f1fd7bd9c54985150d98926353f015b642f7c92e495d7fee8644f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T07:33:12.927 INFO:journalctl@ceph.mds.c.smithi153.stdout:Sep 21 07:33:12 smithi153 bash[70909]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-c 2024-09-21T07:33:13.085 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T07:33:13.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:33:13.121 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T07:33:13.121 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T07:33:13.121 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.f 2024-09-21T07:33:13.427 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:33:13 smithi153 systemd[1]: Stopping Ceph mds.f for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:33:13.927 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:33:13 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f[64672]: 2024-09-21T07:33:13.453+0000 7f6c6d8ff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:33:13.927 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:33:13 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f[64672]: 2024-09-21T07:33:13.453+0000 7f6c6d8ff640 -1 mds.f *** got signal Terminated *** 2024-09-21T07:33:17.739 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:33:17 smithi153 podman[71042]: 2024-09-21 07:33:17.470389471 +0000 UTC m=+4.134826909 container died 69145071ef92c505c2fceb3df6b9ec17083472bc208d92afe9b2cc0b106747a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:33:18.097 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:33:17 smithi153 podman[71042]: 2024-09-21 07:33:17.819558785 +0000 UTC m=+4.483996217 container cleanup 69145071ef92c505c2fceb3df6b9ec17083472bc208d92afe9b2cc0b106747a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:33:18.097 INFO:journalctl@ceph.mds.f.smithi153.stdout:Sep 21 07:33:17 smithi153 bash[71042]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-f 2024-09-21T07:33:18.253 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T07:33:18.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:33:18.291 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T07:33:18.291 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T07:33:18.291 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.i 2024-09-21T07:33:18.633 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:33:18 smithi153 systemd[1]: Stopping Ceph mds.i for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:33:18.927 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:33:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i[65220]: 2024-09-21T07:33:18.631+0000 7f4bd3e99640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:33:18.927 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:33:18 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i[65220]: 2024-09-21T07:33:18.631+0000 7f4bd3e99640 -1 mds.i *** got signal Terminated *** 2024-09-21T07:33:20.624 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:33:20 smithi153 podman[71162]: 2024-09-21 07:33:20.320880539 +0000 UTC m=+1.802269980 container died 407106754a06791a9010037c7d4528ca97e7b009e66eae7d366be8617edd9711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T07:33:20.624 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:33:20 smithi153 podman[71162]: 2024-09-21 07:33:20.623396997 +0000 UTC m=+2.104786439 container cleanup 407106754a06791a9010037c7d4528ca97e7b009e66eae7d366be8617edd9711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T07:33:20.927 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:33:20 smithi153 bash[71162]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i 2024-09-21T07:33:20.927 INFO:journalctl@ceph.mds.i.smithi153.stdout:Sep 21 07:33:20 smithi153 podman[71174]: 2024-09-21 07:33:20.866302923 +0000 UTC m=+0.546690799 container remove 407106754a06791a9010037c7d4528ca97e7b009e66eae7d366be8617edd9711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-i, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:33:21.002 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T07:33:21.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:33:21.034 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T07:33:21.034 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T07:33:21.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.l 2024-09-21T07:33:21.353 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:33:21 smithi153 systemd[1]: Stopping Ceph mds.l for 0ff1e276-77e7-11ef-baf5-efdc52797490... 2024-09-21T07:33:21.677 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:33:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l[64402]: 2024-09-21T07:33:21.351+0000 7ff6f377e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:33:21.677 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:33:21 smithi153 ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l[64402]: 2024-09-21T07:33:21.351+0000 7ff6f377e640 -1 mds.l *** got signal Terminated *** 2024-09-21T07:33:23.329 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:33:23 smithi153 podman[71284]: 2024-09-21 07:33:23.069679685 +0000 UTC m=+1.821208622 container died 226bdb383465a6ee7d1a1a786dfe5f76fe84343bf7be95f00c11badf76f90732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:33:23.677 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:33:23 smithi153 podman[71284]: 2024-09-21 07:33:23.385349937 +0000 UTC m=+2.136878862 container cleanup 226bdb383465a6ee7d1a1a786dfe5f76fe84343bf7be95f00c11badf76f90732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T07:33:23.677 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:33:23 smithi153 bash[71284]: ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l 2024-09-21T07:33:23.677 INFO:journalctl@ceph.mds.l.smithi153.stdout:Sep 21 07:33:23 smithi153 podman[71296]: 2024-09-21 07:33:23.569930851 +0000 UTC m=+0.499266259 container remove 226bdb383465a6ee7d1a1a786dfe5f76fe84343bf7be95f00c11badf76f90732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-0ff1e276-77e7-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T07:33:23.780 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ff1e276-77e7-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T07:33:23.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:33:23.813 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T07:33:23.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T07:33:24.102 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:33:28.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T07:33:28.492 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:33:30.976 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T07:33:31.177 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:33:33.542 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:33:33.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:33:33.642 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:33:33.668 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T07:33:33.670 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi027/crash 2024-09-21T07:33:33.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash -- . 2024-09-21T07:33:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T07:33:33.743 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-09-21T07:33:33.745 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi119/crash 2024-09-21T07:33:33.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash -- . 2024-09-21T07:33:33.882 INFO:teuthology.orchestra.run.smithi119.stderr:tar: /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T07:33:33.882 INFO:teuthology.orchestra.run.smithi119.stderr:tar: Error is not recoverable: exiting now 2024-09-21T07:33:33.884 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi153/crash 2024-09-21T07:33:33.885 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash -- . 2024-09-21T07:33:33.918 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T07:33:33.918 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-09-21T07:33:33.919 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T07:33:33.919 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T07:33:34.009 INFO:tasks.cephadm:Compressing logs... 2024-09-21T07:33:34.009 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:33:34.014 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:33:34.016 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:33:34.040 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T07:33:34.041 INFO:teuthology.orchestra.run.smithi153.stderr:: No such file or directory 2024-09-21T07:33:34.041 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T07:33:34.042 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T07:33:34.042 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T07:33:34.042 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T07:33:34.042 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T07:33:34.045 INFO:teuthology.orchestra.run.smithi119.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T07:33:34.045 INFO:teuthology.orchestra.run.smithi119.stderr:: No such file or directory 2024-09-21T07:33:34.046 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T07:33:34.046 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T07:33:34.046 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T07:33:34.046 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T07:33:34.046 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T07:33:34.046 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T07:33:34.046 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.k.log: -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T07:33:34.047 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T07:33:34.047 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.11.log: 82.1% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T07:33:34.047 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T07:33:34.047 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log 2024-09-21T07:33:34.048 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T07:33:34.048 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log: 90.3% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T07:33:34.048 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T07:33:34.048 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T07:33:34.049 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.l.log: 87.4% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T07:33:34.049 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T07:33:34.049 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log: 90.2% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T07:33:34.050 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T07:33:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log: 82.2% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T07:33:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log 2024-09-21T07:33:34.055 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.8.log: 94.9% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T07:33:34.055 INFO:teuthology.orchestra.run.smithi153.stderr: 90.9% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T07:33:34.055 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T07:33:34.056 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.y.log: 94.9% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T07:33:34.057 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose --/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.z.log: /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T07:33:34.058 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T07:33:34.059 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log: 90.8% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T07:33:34.060 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T07:33:34.060 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.h.log: 87.5% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T07:33:34.063 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T07:33:34.063 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.b.log: 93.0% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T07:33:34.063 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T07:33:34.064 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.c.log: 93.1% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T07:33:34.065 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log: No such file or directory 2024-09-21T07:33:34.065 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-21T07:33:34.065 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T07:33:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T07:33:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T07:33:34.068 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T07:33:34.069 INFO:teuthology.orchestra.run.smithi027.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T07:33:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T07:33:34.069 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T07:33:34.070 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T07:33:34.071 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T07:33:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T07:33:34.074 INFO:teuthology.orchestra.run.smithi027.stderr: 84.5% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T07:33:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T07:33:34.074 INFO:teuthology.orchestra.run.smithi027.stderr: 90.1% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T07:33:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log 2024-09-21T07:33:34.077 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.f.log: 94.9% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T07:33:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.6.log: /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T07:33:34.082 INFO:teuthology.orchestra.run.smithi027.stderr: 95.0% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T07:33:34.083 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T07:33:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T07:33:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T07:33:34.086 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T07:33:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.x.log: 87.4%/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T07:33:34.088 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T07:33:34.091 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.3.log: -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T07:33:34.092 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T07:33:34.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T07:33:34.108 INFO:teuthology.orchestra.run.smithi027.stderr: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T07:33:34.112 INFO:teuthology.orchestra.run.smithi027.stderr: 94.8% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T07:33:34.176 INFO:teuthology.orchestra.run.smithi153.stderr: 94.7% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T07:33:34.181 INFO:teuthology.orchestra.run.smithi119.stderr: 94.4% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T07:33:34.210 INFO:teuthology.orchestra.run.smithi119.stderr: 92.9% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T07:33:34.239 INFO:teuthology.orchestra.run.smithi027.stderr: 92.8% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T07:33:34.415 INFO:teuthology.orchestra.run.smithi153.stderr: 93.4% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T07:33:34.566 INFO:teuthology.orchestra.run.smithi153.stderr: 91.6% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T07:33:34.610 INFO:teuthology.orchestra.run.smithi027.stderr: 90.4% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T07:33:34.820 INFO:teuthology.orchestra.run.smithi119.stderr: 91.6% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T07:33:35.227 INFO:teuthology.orchestra.run.smithi153.stderr: 92.1% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T07:33:35.564 INFO:teuthology.orchestra.run.smithi027.stderr: 92.3% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T07:33:35.711 INFO:teuthology.orchestra.run.smithi119.stderr: 93.2% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T07:33:35.775 INFO:teuthology.orchestra.run.smithi027.stderr: 92.5% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T07:33:35.780 INFO:teuthology.orchestra.run.smithi119.stderr: 93.0% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T07:33:35.833 INFO:teuthology.orchestra.run.smithi027.stderr: 90.4% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T07:33:35.845 INFO:teuthology.orchestra.run.smithi027.stderr: 92.1% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T07:33:35.965 INFO:teuthology.orchestra.run.smithi119.stderr: 92.0% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T07:33:36.260 INFO:teuthology.orchestra.run.smithi153.stderr: 92.7% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T07:33:36.319 INFO:teuthology.orchestra.run.smithi027.stderr: 92.9% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T07:33:36.322 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-21T07:33:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m2.279s 2024-09-21T07:33:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m9.025s 2024-09-21T07:33:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.764s 2024-09-21T07:33:36.756 INFO:teuthology.orchestra.run.smithi119.stderr: 92.8% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T07:33:36.810 INFO:teuthology.orchestra.run.smithi153.stderr: 92.5% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T07:33:36.913 INFO:teuthology.orchestra.run.smithi153.stderr: 92.3% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T07:33:36.915 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-09-21T07:33:36.915 INFO:teuthology.orchestra.run.smithi153.stderr:real 0m2.884s 2024-09-21T07:33:36.915 INFO:teuthology.orchestra.run.smithi153.stderr:user 0m9.746s 2024-09-21T07:33:36.915 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m0.363s 2024-09-21T07:33:37.017 INFO:teuthology.orchestra.run.smithi119.stderr: 92.5% -- replaced with /var/log/ceph/0ff1e276-77e7-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T07:33:37.019 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-21T07:33:37.019 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m2.988s 2024-09-21T07:33:37.019 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m11.702s 2024-09-21T07:33:37.019 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.387s 2024-09-21T07:33:37.020 INFO:tasks.cephadm:Archiving logs... 2024-09-21T07:33:37.020 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi027/log 2024-09-21T07:33:37.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:33:38.265 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi119/log 2024-09-21T07:33:38.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:33:39.858 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi153/log 2024-09-21T07:33:39.858 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:33:41.302 INFO:tasks.cephadm:Removing cluster... 2024-09-21T07:33:41.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 --force 2024-09-21T07:33:41.591 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:33:42.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 --force 2024-09-21T07:33:42.260 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:33:42.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ff1e276-77e7-11ef-baf5-efdc52797490 --force 2024-09-21T07:33:42.747 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 0ff1e276-77e7-11ef-baf5-efdc52797490 2024-09-21T07:33:43.016 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T07:33:43.016 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T07:33:43.059 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T07:33:43.076 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T07:33:43.091 INFO:tasks.cephadm:Teardown complete 2024-09-21T07:33:43.092 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T07:33:43.108 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T07:33:43.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:33:43.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:33:43.118 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:33:43.236 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T07:33:43.250 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T07:33:43.250 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:33:43.255 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:33:43.257 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:33:43.270 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:33:43.271 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:33:43.274 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:33:43.274 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-09-21T07:33:43.274 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +52us[ +38us] +/- 58ms 2024-09-21T07:33:43.274 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -174us[ -187us] +/- 68ms 2024-09-21T07:33:43.274 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +131us[ +118us] +/- 73ms 2024-09-21T07:33:43.274 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:33:43.276 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:33:43.276 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-21T07:33:43.276 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +47us[ +42us] +/- 58ms 2024-09-21T07:33:43.276 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -185us[ -185us] +/- 69ms 2024-09-21T07:33:43.276 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +131us[ +126us] +/- 73ms 2024-09-21T07:33:43.276 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:33:43.286 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:33:43.293 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:33:43.293 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-21T07:33:43.293 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +38us[ +38us] +/- 58ms 2024-09-21T07:33:43.293 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -180us[ -180us] +/- 69ms 2024-09-21T07:33:43.293 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +142us[ +142us] +/- 73ms 2024-09-21T07:33:43.293 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:33:43.294 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T07:33:43.304 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T07:33:43.305 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T07:33:43.315 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:33:43.479 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:33:43.548 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:33:43.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T07:33:43.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:43.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T07:33:43.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:43.707 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T07:33:43.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:43.735 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T07:33:43.747 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T07:33:43.784 INFO:teuthology.task.internal:Duration was 2460.648577 seconds 2024-09-21T07:33:43.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T07:33:43.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T07:33:43.795 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:33:43.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:33:43.802 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:33:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:33:43.841 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:33:43.884 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:33:44.323 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T07:33:44.323 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T07:33:44.323 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:33:44.375 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-09-21T07:33:44.376 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:33:44.403 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-09-21T07:33:44.403 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:33:44.429 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T07:33:44.430 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:33:44.434 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:33:44.446 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:33:44.497 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T07:33:44.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:33:44.502 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:33:44.516 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:33:46.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T07:33:46.384 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T07:33:46.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:33:46.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:33:46.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:33:46.501 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T07:33:46.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:33:46.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:33:46.520 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:33:46.544 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-09-21T07:33:46.569 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-09-21T07:33:46.585 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-09-21T07:33:46.617 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:33:46.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:46.651 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:33:46.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:46.667 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:33:46.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:46.685 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T07:33:46.696 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T07:33:46.697 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi027 2024-09-21T07:33:46.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:33:46.800 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi119 2024-09-21T07:33:46.800 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:33:46.838 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913574/remote/smithi153 2024-09-21T07:33:46.838 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:33:46.881 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T07:33:46.881 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:33:46.885 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:33:46.888 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:33:46.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T07:33:46.953 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T07:33:46.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T07:33:46.974 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T07:33:46.975 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:33:46.979 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:33:46.981 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:33:46.996 INFO:teuthology.orchestra.run.smithi119.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:33 /home/ubuntu/cephtest 2024-09-21T07:33:46.999 INFO:teuthology.orchestra.run.smithi153.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:33 /home/ubuntu/cephtest 2024-09-21T07:33:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: 393956 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:33 /home/ubuntu/cephtest 2024-09-21T07:33:47.013 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T07:33:47.025 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T07:33:47.098 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2460.648576974869 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T07:33:47.098 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:33:47.182 INFO:teuthology.run:pass